Valve Index Camera Passthrough #

index_camera_passthrough GitHub repository

Warning

index_camera_passthrough is still to be considered alpha-quality and highly experimental.

index_camera_passthrough is a dual API (OpenXR and OpenVR) view corrected passthrough overlay for lighthouse tracked devices with not less or more than 2 external cameras.

Despite the name this application supports all lighthouse style devices as long as two HMD cameras are present and the device has a valid config.json stored on the firmware, factory calibrated with said camera intrinsics.

Currently the overlay is set to toggle off/on with the press of both controller B buttons, at the same time.

Here is an example configuration for monado which should be written to the path ~/.config/index_camera_passthrough/index_camera_passthrough.toml

For SteamVR usage, change the backend to openvr.

## This is the configuration file for index_camera_passthrough.
## This file should live at ~/.config/index_camera_passthrough/index_camera_passthrough.toml

## This is your selected backend
## possible values: "openxr" | "openvr"
backend="openxr"

## camera device to use. auto detect if not set
camera_device = "/dev/video0"

## which button should toggle the overlay visibility. press things
## button on both controllers to toggle the overlay.
## possible values: "Menu" | "Grip" | "Trigger" | "A" | "B"
toggle_button = "B"

## how long does the button need to be held before the overlay open,
## closing the overlay is always instantaneous
open_delay = "0s"

[overlay.position]
## how will the overlay be positioned.
## possible values:
##   - "Hmd":      stay in front of your Hmd
##   - "Sticky":   will remain in place upon the location of activation
##   - "Absolute": fixed place in VR space
mode = "Hmd"

## how far away should the overlay be placed
## only meaningful if mode is "Hmd"
distance = 0.7

## transformation matrix for absolute position, column-major
## only meaningful if mode is "Absolute"
# transform = [ [1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1] ]

[display_mode]
## the display mode.
## possible values:
##   - "Stereo": show a 3D image, how much you can see is limited by how
##               big the overlay is in your field of view.
##   - "Flat":   show a flat image
mode = "Stereo"

## which camera's image to display in Flat mode
## only meaningful if mode is "Flat"
# eye = "Left"

## Because your eyes and the cameras are at different physical locations, it is impossible
## to project camera view into VR space perfectly. There are trade offs approximating
## this projection.
##
## possible values:
## (a smaller viewing range here means things too close to you will give you double vision).
##
##   - "FromCamera": in this mode, we assume your eyes are at the cameras' physical location. this mode
##                   has larger viewing range, but everything will look smaller to you.
##   - "FromEye":    in this mode, we assume your cameras are at your eyes' physical location. everything will
##                   have the right scale in this mode, but the viewing range is smaller.
##
## only available if mode is "Stereo"
projection_mode = "FromEye"