...
| Table of Contents |
|---|
Introduction
| Info |
|---|
LPVR-AIR is based on FusionHub. This is a condensed version of the FusionHub manual that focuses on the VR use-case. Find the full FusionHub documentation here. |
The purpose of LPVR-AIR is to wirelessly stream image data from a SteamVR application such as Autodesk VRED to a wireless HMD like the Meta Quest or VIVE Focus. LP-Research's FusionHub software in combination with the open-source application ALVR fulfill this purpose well.
...
Application | Purpose | Name |
|---|---|---|
FusionHub GUI |
|
|
ALVR server |
|
|
FusionHub |
|
|
Starter script |
|
|
LPVR-AIR is copy protected by a USB dongle that needs to inserted when the application is started.
...
ALVR requires SteamVR to be set up on the host computer. If you haven’t installed it on your computer yet, please refer to the instructions here: https://store.steampowered.com/app/250820/SteamVR/
| Info |
|---|
Make sure you turn the guardian (i.e. the automatic tracking boundary detection for the Quest’s internal tracking) is turned off in the developer settings of the HMD. For VIVE Focus use the equivalent setting in the Focus' configuration. |
Start-up
Run
alvr_client_android/alvr_client_android.apkon your headsetStart your optical tracking system (ART DTrack or Optitrack Motive)
Run
Start-LPVR-AIR.bat
Notes:
Make sure the copy protection dongle is inserted into your computer
To upload an APK to your HMD you might need to activate its developer mode
In the FusionHub configuration script make sure to correctly configure your optical tracking system. The script can be accessed through the FusionHub GUI.
Make sure to use a pre-configured HMD optical target provided by ART or Optitrack
...
By default SteamVR is started via Steam, therefore Steam has to run in order to start SteamVR.
It is however possible to start SteamVR directly from it’s binary directory by using the STEAMVR_BIN_DIR environment variable. LPVR-AIR detects automatically if this variable is set and will attempt to start SteamVR. A typical path for STEAMVR_BIN_DIR would be C:\Program Files (x86)\Steam\steamapps\common\SteamVR\bin\win64.
| Info |
|---|
Make sure you turn the guardian (i.e. the automatic tracking boundary detection for the Quest’s internal tracking) is turned off in the developer settings of the HMD. For VIVE Focus use the equivalent setting in the Focus' configuration. |
Start-up
Run
alvr_client_android/alvr_client_android.apkon your headsetStart your optical tracking system (ART DTrack or Optitrack Motive)
Run
Start-LPVR-AIR.bat
Notes:
Make sure the copy protection dongle is inserted into your computer
To upload an APK to your HMD you might need to activate its developer mode
In the FusionHub configuration script make sure to correctly configure your optical tracking system. The script can be accessed through the FusionHub GUI.
Make sure to use a pre-configured HMD optical target provided by ART or Optitrack
| Info |
|---|
Once streaming starts, you should see the SteamVR default environment through the headset. Check if the nIMU counter in the FusionHub GUI is increasing. If both nOptical and nIMU are increasing then the communication between ALVR, optical tracking and FusionHub is working. |
...
A typical ALVR performance graph is shown below. Overall latencies in good environments should be between 70 and 90ms.
...
In SteamVR check Advanced Frame Timing for performance problems:
...
In case of a sufficient rendering performance the advanced frame timing window should look like the output below:
...
Challenges and limitations of using Android-based wireless HMDs
External pose input / internal late latching conflict
| Note |
|---|
This issue has been mostly resolved. Please report to us if you have issues with bad reprojection. |
By default Meta Quest HMDs use their built-in inside-out tracking for measuring the user's head pose. For this project we needed to exchange the built-in tracking with our customized pose estimation. Generally the Quest HMD's software is not made to allow this application. Correct VR rendering is optimized to work with the built-in tracking and alternative ways of pose input are not supported.
We were able to create a workaround for this problem to some degree, by exchanging the pose calculated by the internal system with the pose calculated by FusionHub, before it is sent to the server for rendering. On the rendering side this works well, but unfortunately creates a conflict between the pose reported by the HMD firmware and the pose used for rendering that affects the quality of the HMD's pose prediction, late latching / asynchronous time/space warp. This conflict is perceived as uncomfortable, slight jittering of the image and overcompensation of head motions.
A way to reduce this jittering effect is to reduce the prediction time of the internal pose calculation. This doesn't fully solve the problem though, as decreased prediction time also reduces the responsiveness of the internal late latching. This in turn leads to blurring at edges of the image when the user turns his head rapidly.
Therefore we tried to find a compromise between image stability and late latching capability. If this compromise works well, strongly depends on user requirements and quality of the WIFI connection established to the HMD. A perfect solution is unfortunately not possible without support from the HMD manufacturer Meta.
WIFI environment quality
LPVR-AIR transmits images from the server to the HMD through a regular WIFI connection. Usually a 5G band is used, in the optimum case we switch to WIFI 6E. In environments without much WIFI interference, i.e. other devices using the same WIFI bands, this works very well. Crowded WIFI environments limit the bandwidth of the used WIFI transmission. This can lead to unpredictable loss of image and tracking quality. Examples of crowded spaces are public locations such as exhibition. Beware!
The Meta Quest firmware doesn't allow using a wired ethernet connection to mitigate the issue to provide a quick fix to this problem in urgent situations. Again, a clean execution of this project is prevented by inflexibility of Meta's software. Unfortunately there are no alternative wireless HMDs on the market that allow the modifications we need for an optimum implementation.
In an ideal setup, with several HMDs being used, each HMD uses a separate WIFI 6e channel, with the selected channels as far apart as possible.
Optical tracking parsing latency
Due to limited WIFI bandwidth and computing power limitations on the HMD pose information streamed from the the optical tracking system is parsed on the HMD with a significant delay. So far we have not found a way to reduce this delay. As described in the sensor fusion section, we added input from native inside-out tracking to the fusion in order to compensate for this latency.
HMD built-in CPU performance limitation
...
...
In SteamVR check Advanced Frame Timing for performance problems:
...
In case of a sufficient rendering performance the advanced frame timing window should look like the output below:
...
Challenges and limitations of using Android-based wireless HMDs
WIFI environment quality
LPVR-AIR transmits images from the server to the HMD through a regular WIFI connection. Usually a 5G band is used, in the optimum case we switch to WIFI 6E. In environments without much WIFI interference, i.e. other devices using the same WIFI bands, this works very well. Crowded WIFI environments limit the bandwidth of the used WIFI transmission. This can lead to unpredictable loss of image and tracking quality. Examples of crowded spaces are public locations such as exhibition. Beware!
The Meta Quest firmware doesn't allow using a wired ethernet connection to mitigate the issue to provide a quick fix to this problem in urgent situations. Again, a clean execution of this project is prevented by inflexibility of Meta's software. Unfortunately there are no alternative wireless HMDs on the market that allow the modifications we need for an optimum implementation.
In an ideal setup, with several HMDs being used, each HMD uses a separate WIFI 6e channel, with the selected channels as far apart as possible.
Optical tracking parsing latency
Due to limited WIFI bandwidth and computing power limitations on the HMD pose information streamed from the the optical tracking system is parsed on the HMD with a significant delay. So far we have not found a way to reduce this delay. As described in the sensor fusion section, we added input from native inside-out tracking to the fusion in order to compensate for this latency.
References
ALVR open source project: https://github.com/alvr-org/ALVR
FusionHub documentation: FusionHub Manual
...