...
| Table of Contents |
|---|
Introduction
| Info |
|---|
LPVR-AIR is based on FusionHub. This is a condensed version of the FusionHub manual that focuses on the VR use-case. Find the full FusionHub documentation here. |
The purpose of LPVR-AIR is to wirelessly stream image data from a SteamVR application such as Autodesk VRED to a wireless HMD like the Meta Quest or VIVE Focus. LP-Research's FusionHub software in combination with the open-source application ALVR fulfill this purpose well.
...
Application | Purpose | Name |
|---|---|---|
FusionHub GUI |
|
|
ALVR server |
|
|
FusionHub |
|
|
Starter script |
|
|
LPVR-AIR is copy protected by a USB dongle that needs to inserted when the application is started.
...
Challenges and limitations of using Android-based wireless HMDs
External pose input / internal late latching conflict
| Note |
|---|
This issue has been mostly resolved. Please report to us if you have issues with bad reprojection. |
By default Meta Quest HMDs use their built-in inside-out tracking for measuring the user's head pose. For this project we needed to exchange the built-in tracking with our customized pose estimation. Generally the Quest HMD's software is not made to allow this application. Correct VR rendering is optimized to work with the built-in tracking and alternative ways of pose input are not supported.
We were able to create a workaround for this problem to some degree, by exchanging the pose calculated by the internal system with the pose calculated by FusionHub, before it is sent to the server for rendering. On the rendering side this works well, but unfortunately creates a conflict between the pose reported by the HMD firmware and the pose used for rendering that affects the quality of the HMD's pose prediction, late latching / asynchronous time/space warp. This conflict is perceived as uncomfortable, slight jittering of the image and overcompensation of head motions.
A way to reduce this jittering effect is to reduce the prediction time of the internal pose calculation. This doesn't fully solve the problem though, as decreased prediction time also reduces the responsiveness of the internal late latching. This in turn leads to blurring at edges of the image when the user turns his head rapidly.
Therefore we tried to find a compromise between image stability and late latching capability. If this compromise works well, strongly depends on user requirements and quality of the WIFI connection established to the HMD. A perfect solution is unfortunately not possible without support from the HMD manufacturer Meta.
WIFI environment quality
LPVR-AIR transmits images from the server to the HMD through a regular WIFI connection. Usually a 5G band is used, in the optimum case we switch to WIFI 6E. In environments without much WIFI interference, i.e. other devices using the same WIFI bands, this works very well. Crowded WIFI environments limit the bandwidth of the used WIFI transmission. This can lead to unpredictable loss of image and tracking quality. Examples of crowded spaces are public locations such as exhibition. Beware!
The Meta Quest firmware doesn't allow using a wired ethernet connection to mitigate the issue to provide a quick fix to this problem in urgent situations. Again, a clean execution of this project is prevented by inflexibility of Meta's software. Unfortunately there are no alternative wireless HMDs on the market that allow the modifications we need for an optimum implementation.
In an ideal setup, with several HMDs being used, each HMD uses a separate WIFI 6e channel, with the selected channels as far apart as possible.
Optical tracking parsing latency
Due to limited WIFI bandwidth and computing power limitations on the HMD pose information streamed from the the optical tracking system is parsed on the HMD with a significant delay. So far we have not found a way to reduce this delay. As described in the sensor fusion section, we added input from native inside-out tracking to the fusion in order to compensate for this latency.
HMD built-in CPU performance limitation
...
WIFI environment quality
LPVR-AIR transmits images from the server to the HMD through a regular WIFI connection. Usually a 5G band is used, in the optimum case we switch to WIFI 6E. In environments without much WIFI interference, i.e. other devices using the same WIFI bands, this works very well. Crowded WIFI environments limit the bandwidth of the used WIFI transmission. This can lead to unpredictable loss of image and tracking quality. Examples of crowded spaces are public locations such as exhibition. Beware!
The Meta Quest firmware doesn't allow using a wired ethernet connection to mitigate the issue to provide a quick fix to this problem in urgent situations. Again, a clean execution of this project is prevented by inflexibility of Meta's software. Unfortunately there are no alternative wireless HMDs on the market that allow the modifications we need for an optimum implementation.
In an ideal setup, with several HMDs being used, each HMD uses a separate WIFI 6e channel, with the selected channels as far apart as possible.
Optical tracking parsing latency
Due to limited WIFI bandwidth and computing power limitations on the HMD pose information streamed from the the optical tracking system is parsed on the HMD with a significant delay. So far we have not found a way to reduce this delay. As described in the sensor fusion section, we added input from native inside-out tracking to the fusion in order to compensate for this latency.
References
ALVR open source project: https://github.com/alvr-org/ALVR
FusionHub documentation: FusionHub Manual
...