Page Comparison

...

The purpose of LPVR-AIR is to wirelessly stream image data from a SteamVR application such as Autodesk VRED to a wireless HMD like the Meta Quest 2 or ProVIVE Focus. LP-Research's FusionHub software in combination with the open-source application ALVR fulfill this purpose well.

...

On the headset:

...

FusionHub authenticates itself via the GUI client application. In order to run the client application make sure to insert the LPVR USB dongle into the host computer. After the GUI client is connected to FusionHub on the HMD, FusionHub will start streaming pose data to ALVR. Alternatively we offer the possibility to use use an online licensing system that unlocks HMD nodes via the internet. By default we deliver the dongled version.

Running LPVR-AIR

Installation

Install the FusionHub APK and ALVR client APK on the headset using a side-loading tool like Sidequest. In case of a Meta Quest HMD this will require you to put the HMD into developer mode. See here the steps for putting the HMD into developer mode: https://developer.oculus.com/documentation/native/android/mobile-device-setup/

The FusionHub GUI client and ALVR server can be started on the host PC without further installation, they can be run right out of the deployment folder.

Preparation of Headset

There are a few steps that we need to consider when starting the system. Some of these are due to limitations of our overall system implementation, some are necessary because of issues introduced by the Quest’s firmware.

• Make sure you turn the guardian (ie. the automatic tracking boundary detection for the Quest’s internal tracking) is turned off in the developer settings of the HMD.

• If inside-out tracking is not being used, we recommend to switch headset tracking off ie. switch to 3DOF tracking in the Quest system settings. This will free additional CPU performance for the decoding of the video stream and therefore improve image quality.

• When powering-up or re-starting the HMD, it will automatically turn headset tracking to ON. If not needed, turn headset tracking to OFF to save computing resources.

• When powering-up or re-starting the HMD, it will not automatically connect to a previously saved network, if the network is not connected to the internet. In this case manually connect to the designated access point after start-up. Internet is not needed for operation, this is a bug in Meta’s firmware. This is an issue that might be solved by a software update from Meta.

• To reduce impact of latency and jitter from optical tracking, turn on some smoothing and prediction in the ART DTrack or Optitrack Motive settings.

• If needed, cover the proximity sensor in the center between the lenses of the HMD with a piece of dark tape to prevent the HMD from going to sleep. Generally LPVR-AIR is able to deal with the HMD going to sleep and waking up, but in case you discover any issues the tape will help.

Start-up

FusionHub

...

Start FusionHub on the HMD. A window showing the FusionHub console output should open.

...

Start the FusionHub GUI client on the host computer

...

In case you’re using a VIVE Focus 3 headset you need to do something similar as described here: https://developer.vive.com/resources/hardware-guides/vive-focus-specs-user-guide/how-do-i-put-focus-developer-mode/

The FusionHub GUI client and ALVR server can be started on the host PC without further installation, they can be run right out of the deployment folder.

ALVR requires SteamVR to be set up on the host computer. If you haven’t installed it on your computer yet, please refer to the instructions here: https://store.steampowered.com/app/250820/SteamVR/

Preparation of Headset

There are a few steps that we need to consider when starting the system. Some of these are due to limitations of our overall system implementation, some are necessary because of issues introduced by the Quest’s firmware.

• Make sure you turn the guardian (ie. the automatic tracking boundary detection for the Quest’s internal tracking) is turned off in the developer settings of the HMD.

• If inside-out tracking is not being used, we recommend to switch headset tracking off ie. switch to 3DOF tracking in the Quest system settings. This will free additional CPU performance for the decoding of the video stream and therefore improve image quality.

• When powering-up or re-starting the HMD, it will automatically turn headset tracking to ON. If not needed, turn headset tracking to OFF to save computing resources.

• For some older Meta Quest firmware versions, when powering-up or re-starting the HMD, it will not automatically connect to a previously saved network, if the network is not connected to the internet. In this case manually connect to the designated access point after start-up. Internet is not needed for operation, this is a bug in Meta’s firmware. This is an issue that might be solved by a software update from Meta.

• To reduce impact of latency and jitter from optical tracking, turn on some smoothing and prediction in the ART DTrack or Optitrack Motive settings. Try different amounts of prediction and smoothing until you feel comfortable with the result.

• If needed, cover the proximity sensor in the center between the lenses of the HMD with a piece of dark tape to prevent the HMD from going to sleep. Generally LPVR-AIR is able to deal with the HMD going to sleep and waking up, but in case you discover any issues the tape will help.

Start-up

FusionHub

• Start FusionHub on the HMD. A window showing the FusionHub console output should open.

• Start the FusionHub GUI client on the host computer

• Connect the GUI client to FusionHub on the HMD. Make sure HMD and host are in the same subnet. Enter the the correct IP of the HMD in the client before pressing connect.

• Adjust parameter blocks as needed. Refer to the description of FusionHub BASE for configuration options. Note the following input and output ports that are hard-coded in the ALVR FusionHub API layer. These are already correctly set in the default configuration file installed with the FusionHub APK, so usually there is no need to change them.

...

• Start the ALVR server on the host. While the ALVR server starts up, it will automatically run SteamVR.

• Start the ALVR client on the HMD. The HMD should be shown in the list of ALVR clients in the ALVR server application. In some cases you need to click the Trust button in the application to start streaming.

• Once streaming starts, you should see the SteamVR default envinronment through the headset. Check if the nIMU counter in the FusionHub GUI is increasing. If both nOptical and nIMU are increasing then the communication between ALVR, optical tracking and FusionHub is working.

• You can now use and calibrate the system as described in the FusioHub BASE section.

Optical tracking systems

Configuration options

Advanced Realtime Tracking (ART):

FusionHub works with all ART tracking systems, based on their DTrack tracking software.

"type": "DTrack",
"settings": {
    "port": 5005,
    "bodyID": 3,
    "endpoint": "inproc://optical_data_source_1"
}

Optitrack:

FusionHub works with all Optitrack tracking systems based on their Motive tracking software.

"type": "Optitrack",
"settings": {
    "localAddress": "127.0.0.1",
    "remoteAddress": "192.168.137.1",
    "connectionType": "Multicast",
    "bodyID": 444
}

VICON:

...

• FusioHub BASE section.

Optical tracking systems

Configuration options

Advanced Realtime Tracking (ART):

FusionHub works with all ART tracking systems, based on their DTrack tracking software.

"type": "ViconDTrack",
"settings": {
    "settingsport": {5005,
    "hostbodyID": "localhost"3,
    "subjectendpoint": "VCaminproc://optical_data_source_1"
}

AntilatencyOptitrack:

FusionHub connects directly to Antilatency’s USB or wireless trackersworks with all Optitrack tracking systems based on their Motive tracking software.

"type": "AntilatencyOptitrack",
"settings": {
    "endpointlocalAddress": "127.0.0.1",
    "remoteAddress"inproc://optical_data_source_: "192.168.137.1",
    "environmentLinkconnectionType": "AntilatencyAltEnvironmentHorizontalGrid~AgAEBLhTiT_cRqA-r45jvZqZmT4AAAAAAAAAAACamRk_AQQCAwICAgICAQICAAIMulticast",
    "placementLinkbodyID": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"444
}

Marker adjustment

The optical system is the tracking reference, its pose is what is received by the visualization backend. The orientation of the IMU sensor is calibrated relative to the optical markers on the HMD. Therefore it is important to set up the tracking body or rigid body in the optical tracking software (DTrack, Motive etc.) in a way that its axes align with the optical axes of the head mounted display.

...

We will add Refer to this section soon. In the meantime refer to this page for more information on ART setups and this page for OptiTrack setups from the LPVR documentation.

...

In order to avoid excessive buffering and data loss, make sure to reduce the amount of data being streamed from Motive. We recommend the streaming settings below. Make sure to set the local interface IP to the IP of the network connection that is being used for communicating with the HMD. It is a common mistake to not set this IP correctly. If the correct IP doesn’t show up, restart Motive.

...

Intercalibration (Only on Meta Quest)

...

1. In the DTrack software on PC, click “stop”.

2. Go to Tracking → Body Calibration, select your headset body in the dropdown menu, and click “Load file(s)” to load the calibrated body file xxx_HORAUD.txt under the same directory as your exported body file in section 1.

Advanced Troubleshooting

The best way to diagnose what is going wrong with the system if something doesn’t work as expected is to look at the log output of FusionHubLauncher. The log can be recorded by connecting your HMD to a PC using a USB (most likely USB-C) cable.

...

External pose input / internal late latching conflict

By default Meta Quest HMDs use their built-in inside-out tracking for measuring the user's head pose. For this project we needed to exchange the built-in tracking with our customized pose estimation. Generally the Quest HMD's software is not made to allow this application. Correct VR rendering is optimized to work with the built-in tracking and alternative ways of pose input are not supported.

...