Introduction

The purpose of LPVR-AIR is to wirelessly stream image data from a SteamVR application such as Autodesk VRED to a wireless HMD like the Meta Quest or VIVE Focus. LP-Research's FusionHub software in combination with the open-source application ALVR fulfill this purpose well.

ALVR by default uses the internal inside-out tracking of Meta Quest for pose calculation. LPVR-AIR exchanges the Quest’s native inside-out tracking with combined IMU and ART / Optitrack outside-in tracking to allow simultaneous, spatially synchronized operation of several HMDs in large tracking volumes.

To make the tracking functionality of FusionHub available to standalone augmented and virtual reality headsets, it can be integrated with Android-compatible OpenXR HMDs. This works via a customized version of the ALVR open source project. ALVR allows streaming image data wirelessly from a host computer and interfaces to 3D content engines through SteamVR. While the original ALVR client was built to work on Meta Quest HMDs, ALVR works in principle on any OpenXR compatible headset.

We use a thin client library to receive IMU data from the HMD API, pass it to FusionHub, process it there and then re-inject the information into the video pipeline of the headset. Depending on the type of HMD this happens within the ALVR client’s standard interface or in a separate hardware-specific API layer.

System components

Applications

The following applications need to be started on the head mounted display and the host computer. They should all the included in the installation package that you received from us. We will discuss the order of starting these applications and what their status output should be below.

On the headset:

Application	Purpose	Name (can vary by release)

Application	Purpose	Name (can vary by release)
ALVR client	Receives image data from ALVR server Sends pose information to ALVR server	`alvr_client_openxr.apk`

On the host computer:

Application	Purpose	Name

Application	Purpose	Name
FusionHub GUI	Connects to FusionHub server Configures FusionHub	`FusionHubUI.exe`
ALVR server	Communicates with FusionHub server Receives pose information from client Sends pose information to SteamVR Sends image data to ALVR client Receives image information from SteamVR	`ALVR Dashboard.exe`
FusionHub	Calculates sensor fusion between IMU and optical data	`FusionHub.exe`
Starter script	Starts all server-side applications above	`Start-LPVR-AIR.bat`

LPVR-AIR is copy protected by a USB dongle that needs to inserted when the application is started.

Starting LPVR-AIR

Installation

Install the ALVR client APK on the headset using a side-loading tool like Sidequest. In case of a Meta Quest HMD this will require you to put the HMD into developer mode. See here the steps for putting the HMD into developer mode: https://developer.oculus.com/documentation/native/android/mobile-device-setup/

In case you’re using a VIVE Focus 3 headset you need to do something similar as described here: https://developer.vive.com/resources/hardware-guides/vive-focus-specs-user-guide/how-do-i-put-focus-developer-mode/

ALVR requires SteamVR to be set up on the host computer. If you haven’t installed it on your computer yet, please refer to the instructions here: https://store.steampowered.com/app/250820/SteamVR/

By default SteamVR is started via Steam, therefore Steam has to run in order to start SteamVR.

It is however possible to start SteamVR directly from it’s binary directory by using the STEAMVR_BIN_DIR environment variable. LPVR-AIR detects automatically if this variable is set and will attempt to start SteamVR. A typical path for STEAMVR_BIN_DIR would be C:\Program Files (x86)\Steam\steamapps\common\SteamVR\bin\win64.

Make sure you turn the guardian (i.e. the automatic tracking boundary detection for the Quest’s internal tracking) is turned off in the developer settings of the HMD. For VIVE Focus use the equivalent setting in the Focus' configuration.

Start-up

Run alvr_client_android/alvr_client_android.apk on your headset
Start your optical tracking system (ART DTrack or Optitrack Motive)
Run Start-LPVR-AIR.bat

Notes:

Make sure the copy protection dongle is inserted into your computer
To upload an APK to your HMD you might need to activate its developer mode
In the FusionHub configuration script make sure to correctly configure your optical tracking system. The script can be accessed through the FusionHub GUI.
Make sure to use a pre-configured HMD optical target provided by ART or Optitrack

Once streaming starts, you should see the SteamVR default environment through the headset. Check if the nIMU counter in the FusionHub GUI is increasing. If both nOptical and nIMU are increasing then the communication between ALVR, optical tracking and FusionHub is working.

Using the FusionHub GUI

Connect the GUI to FusionHub from the start screen of the FusionHub GUI. Note that running the GUI is optional, FusionHub works normally without the GUI running.

Once connected the FusionHub on the headset select base configuration to see the current configuration of FusionHub:

Adjust parameter blocks as needed. Refer to the description of FusionHub BASE for configuration options:

Note the following input and output ports that are hard-coded in the ALVR FusionHub API layer. These are already correctly set in the default configuration file installed with FusionHub, so usually there is no need to change them.

Endpoint	Direction	Purpose

Endpoint	Direction	Purpose
`tcp://*:8799`	Output	Fused pose data
`tcp://localhost:8898`	Input	IMU data

If it’s not running yet make sure to start and configure your optical tracking system. Once optical data is streamed to FusionHub, the nOptical counter in the GUI should be increasing.
Default configuration script with optical input defined for ART DTrack:

{
  "LicenseInfo": {
    "LicenseKey": "",
    "ResponseKey": ""
  },
  "settings": {
    "websocketDataOutputRate": 20
  },
  "sinks": {
    "VRPN": {
      "settings": {
        "inputEndpoints": [
          "inproc://optical_data_source_1"
        ],
        "settings": {
          "deviceName": "FusionHub",
          "port": 3883,
          "tracker0": "HMD"
        }
      }
    },
    "fusion": {
      "dataEndpoint": "tcp://*:8799",
      "inputEndpoints": [
        "inproc://optical_data_source_1",
        "tcp://localhost:8898"
      ],
      "settings": {
        "Autocalibration": {
          "minAgeS": 60,
          "nSamplesForAutocalibration": 1500,
          "nSamplesForSteady": 256,
          "noiseRmsLimit": 0.02,
          "steadyThresholdAverage": 0.2,
          "steadyThresholdRms": 1
        },
        "Intercalibration": {},
        "MotionDetection": {
          "omegaLimit": 3,
          "positionSampleInterval": 1000,
          "rotationFilterAlpha": 0.9,
          "timeToUnknown": 500
        },
        "SensorFusion": {
          "alignment": {
            "w": 0.990892966476337,
            "x": 0.13458639604387848,
            "y": 0.0005637732357904688,
            "z": 0.004160907038605602
          },
          "orientationWeight": 0.005,
          "predictionIntervalMs": 10,
          "sggPointsEachSide": 5,
          "sggPolynomialOrder": 5,
          "tiltCorrection": null,
          "yawWeight": 0.01
        },
        "runIntercalibration": false
      },
      "type": "ImuOpticalFusion"
    }
  },
  "sources": {
    "optical": {
      "settings": {
        "bodyIDs": [
          1
        ],
        "endpoints": [
          "inproc://optical_data_source_1"
        ],
        "port": 5000
      },
      "type": "DTrack"
    }
  }
}

Press the buttons ‘Set’ and ‘Save’ after changing the configuration script to make your changes active. It might take 1-2 seconds for FusionHub to reset.

In case you happen to enter an invalid configuration, FusionHub might not restart correctly. If you would like to reset your settings, just re-install the FusionHub APK.

Once the configuration is correct, you’ll most likely not have to touch the script again in the foreseeable future.

Optical tracking systems

Optical marker target setup

LPVR-AIR uses the OpenVR coordinate frame convention as shown below:

Make sure that your optical target is configured with the correct coordinate frame. Make sure to check even if you have a pre-defined optical tracking body.

ART DTrack

See this guide for further information about how to configure DTrack. Also, refer to this page for more information on ART.

Optitrack Motive

Refer to this page for Optitrack setup from the LPVR documentation.

Configuration options

Advanced Realtime Tracking (ART) DTrack

FusionHub works with all ART tracking systems, based on their DTrack tracking software.

"optical": {
    "settings": {
        "port": 5000,        
        
        "bodyIDs": [
            1
        ],
        "endpoints": [
            "inproc://optical_data_source_1"
        ],
        "objectNameMapping": {
            "1": "QuestPro"
        }
    },
    "type": "DTrack"
}

Adjust the body ID of the HMD as configured in DTrack.

Optitrack

FusionHub works with all Optitrack tracking systems based on their Motive tracking software.

"optical": {
    "type": "Optitrack",
    
    "settings": {
        "connectionType": "Multicast",
        "localAddress": "192.168.0.99", 
        "remoteAddress": "192.168.0.100",
            
        "bodyIDs": [
            1
        ],
        "endpoints": [
            "inproc://optical_data_source_1"
        ],
        "objectNameMapping": {
            "1": "QuestPro"
        }
    }
}

Adjust the body ID of the HMD as configured in Motive.

Network setup

Router

In order to establish high bandwidth communication between the host and Quest 2 / Pro we recommend setting up a 5GHz or for optimum performance a 6GHz (WIFI 6e) WIFI router with 2 channels per HMD. We recommend the Eero WIFI 6E router.

Network topography

We recommend to set up a simple network structure, to minimize potential error sources in the installation process as shown in the image below.

Network performance

A WIFI 6E connection is recommended to achieve optimum performance:

Transmission speeds are expected to be around 2 Gbps for a stable 6 GHz connection.

A typical ALVR performance graph is shown below. Overall latencies in good environments should be between 70 and 90ms.

In SteamVR check Advanced Frame Timing for performance problems:

In case of a sufficient rendering performance the advanced frame timing window should look like the output below:

Challenges and limitations of using Android-based wireless HMDs

WIFI environment quality

LPVR-AIR transmits images from the server to the HMD through a regular WIFI connection. Usually a 5G band is used, in the optimum case we switch to WIFI 6E. In environments without much WIFI interference, i.e. other devices using the same WIFI bands, this works very well. Crowded WIFI environments limit the bandwidth of the used WIFI transmission. This can lead to unpredictable loss of image and tracking quality. Examples of crowded spaces are public locations such as exhibition. Beware!

The Meta Quest firmware doesn't allow using a wired ethernet connection to mitigate the issue to provide a quick fix to this problem in urgent situations. Again, a clean execution of this project is prevented by inflexibility of Meta's software. Unfortunately there are no alternative wireless HMDs on the market that allow the modifications we need for an optimum implementation.

In an ideal setup, with several HMDs being used, each HMD uses a separate WIFI 6e channel, with the selected channels as far apart as possible.

Optical tracking parsing latency

Due to limited WIFI bandwidth and computing power limitations on the HMD pose information streamed from the the optical tracking system is parsed on the HMD with a significant delay. So far we have not found a way to reduce this delay. As described in the sensor fusion section, we added input from native inside-out tracking to the fusion in order to compensate for this latency.

References

ALVR open source project: https://github.com/alvr-org/ALVR
FusionHub documentation: FusionHub Manual

LP-RESEARCH Knowledge Base

LPVR-AIR Manual (Server-Side Version)

Table of contents