Hardware Installation

The system consists of the following hardware components:

• A Varjo Headset

• ART optical tracking system in an in-vehicle configuration (e.g. Smarttrack 3)

• An external LPMS-IG1 IMU that is fixed to the vehicle frame (“platform IMU”)

• A PC running Varjo Base that powers the VR/AR experience

The optical tracking system needs to be installed as defined by the optical tracking system manufacturer. Please take care that the floor of the tracking space is calibrated vertically. A water balance built-into the calibration equipment should assist with that.

The following image (from the ART website) illustrates a possible placement of the optical tracking system.

Open

For differential IMU tracking, we need to install a further IMU inside the vehicle, in a location visible to the optical tracking system. Alternatively, this IMU can also be installed in a location not visible to the optical tracking, but with exactly known orientation within the optical reference system.

We will provide further instructions on how to calibrate these components to work in the same coordinate system in the following chapters.

Headset Tracking Body Setup

Please follow the instructions on this page to setup ART’s tracking of the headset Automatic HMD / ART-Body Intercalibration - LP-RESEARCH Knowledge Base - Confluence (atlassian.net). We recommend to do this before installing the driver, but the procedure can followed at any time.

Driver Setup

System Requirements

• LPVR in its current version runs on Windows 10 and above.

• Varjo Base must be installed and operational.

• Applications that use the Varjo headset don’t have to be modified for use with the LPVR-DUO-Varjo driver

Driver Installation

HMD

In the following section we will describe how to setup LPVR-CAD.

• Make sure your optical tracking system software (e.g. DTrack) is running and can see the optical markers attached your HMD and hand controllers. For more information on how to configure your rigid bodies refer to section System Calibration.

• First, we need to download the installer for the LPVR-CAD middleware. Log in to your LP-Research user account to access your dedicated customer area here:

https://lp-research.com/customer-area

If you haven’t received login credentials from us yet, please contact us at support@lp-research.com. The screenshots below show how to download the installer files from the customer area.

Open

• Insert the LPVR dongle into a free USB port of your computer.

• Quit Varjo Base in case it is running.

• Unzip the archive you downloaded and run the LPVR-CAD installer. Figure 6 shows the output of the installer after successful completion.

• Once the driver has finished installing start Varjo Base.

• In Varjo Base go to the system tab and select LPVR-DUO-Varjo as headset tracking method from the dropdown menu.

Driver Configuration

Point your web browser to http://localhost:7119. A configuration webpage (depicted below) should pop up.

The configuration interface includes several pages, accessible through the links at the bottom. A typical setup would work through them starting with the “Configuration” page (depicted), then move on to “Calibration”. If need be, detailed configuration parameters can be entered in “Expert Mode”. Finally, this documentation can be reached via the “Documentation” link.

Tracking System Setup

Please return to the “Configuration” page in order to set up the optical tracking of the headset.

Set the “Port Number” to match the output in DTrack and the “Body ID” in the headset section to match the body configured for the headset in DTrack. The “Status” field next to the Tracking System Settings should turn green and say “Receiving Data”. The left bubble in the “Headset” section should say “Tracked”. Please be sure to click the “Apply settings” button after any changes.

You should now see the VR content move in your headset according to the headset’s motion. If the motion appears incorrect, refer to the “Headset Tracking Body Setup” section above.

Configuration of the Platform IMU

Click on “Calibration” near the bottom of the screen to proceed to the setup of the Platform IMU. You will be offered a selection of options based on your specific setup.

Each of these options is appropriate in different cases. In the simplest case, the Platform IMU (also called “Reference IMU”) is attached to the camera system and can be moved along with it. This corresponds to the first case. If the IMU is mounted within the field of view of the camera system and a marker target can be attached to it for the purposes of the system calibration, the second option is appropriate. In cases where a precise relationship has to be established otherwise (say, from technical drawings), the third option is appropriate. Let’s discuss each option briefly.

First alternative: Calibration with the Platform IMU attached to the camera system

This calibration can be performed once after the IMU has been tightly attached to the camera system. Additionally, it requires moving the camera system. Therefore, we recommend doing it before installing the system in the more constrained space of a car or simulator cabin. Assembling this setup is separately documented here https://lp-research.atlassian.net/wiki/spaces/LKB/pages/2029617191

The procedure is explained quickly: the camera system and the IMU are moved together while pointing at a stationary body which is tracked by the camera system, typically the headset. By comparing the tracking data for the body and the IMU data, the relationship between the two can be established. LPVR-DUO performs this comparison and derives the necessary calibration parameters.

After selecting the calibration on the page, the screen will look something like the above. Make sure that the right-hand bubble reads “IMU active” if not, please check the wiring and restart the VR software and/or the LPVR driver. Now select the correct id for the reference object, which can be any body tracked by your tracking system. It should be stationary during the course of the calibration. If you are using the headset, please make sure it is steadily supported, and doesn’t move during the calibration. Once you are ready and the reference object is in the field of view of the system, you can click the “Start Calibration” button and proceed to the calibration.

A progress bar will appear on the screen. Please pick up the camera system, and move it gently turning it around its different axes while keeping the reference object in the field of view. Make sure to avoid sudden movements and pause in a number of different orientations until the system has recorded sufficient data. This will typically take 15-30 seconds.

Once the calibration completes, a button will appear which allows you to apply the result to the configuration. Clicking it should complete the calibration, and your LPVR-DUO system should be operational.

Second Alternative: Calibration with Platform IMU installed in the field of view of the tracking system

This calibration is divided into two steps: first the relation between the IMU and the body used to track it will be established. During this calibration, the IMU will have to be moved within the field of view of the camera. By comparing the tracking data for the body and the IMU data, the relationship between the two can be established. LPVR-DUO performs this comparison and derives the necessary calibration parameters. In a second step, the IMU will be mounted into the final position and the orientation will be read off by the optical tracking system.

After selecting the calibration on the page, the screen will look something like the below. Make sure that the right-hand bubble reads “IMU active” if not, please check the wiring and restart the VR software and/or the LPVR driver. Now select the correct id for the tracking body of the IMU. Once you are ready and the IMU is in the field of view of the system, you can click the “Start Calibration” button and proceed to the calibration.

A progress bar will appear on the screen. Please pick up the IMU, and move it gently turning it around its different axes while keeping inside the field of view. Make sure to avoid sudden movements and pause in a number of different orientations until the system has recorded sufficient data. This will typically take 15-30 seconds.

Once the calibration completes, a button will appear which allows you to apply the result to the configuration. Clicking it should complete the calibration, and your LPVR-DUO system should be operational.

Now, please put the IMU into the final position inside the car, simulator platform or whatever movable platform you are using. It should be in the field of view of the tracking system.

You should now be able to click the “Use current orientation” button to complete the calibration. The tracking body can now be removed from the IMU and you LPVR-DUO system should be fully functional.

Third alternative: Manual entry of the orientation of the Platform IMU

This is only recommended to experts. You can enter a quaternion manually which converts from the IMU’s coordinate system (as printed on the IMU) to the Optical Tracking coordinate frame. Entering the members of the unnormalized quaternion followed by “Apply settings” will update the system to use these settings.

Validation Steps

After this basic configuration all necessary data should be available to Varjo Base. We suggest to take a few minutes to make sure this is the case. Namely, at this point the headset motion should be picked up by DTrack, IMU data should be flowing and the driver should provide an (uncalibrated) synthesis to Varjo Base. In other words, the VR scene should respond to motions of the headset and the IG1 imu, but not necessarily in the correct way.

In particular the following items can be verified:

• DTrack tracking works - Make sure that the headset is tracked by DTrack. DTrack should be running and the DTrack software should show it’s body number on a green background

• All required data reaches the driver - After clicking “Show details” near driver status on the configuration page, all sources (DTrack, HMD Imu, Platform Imu) should be displayed in green. The below screenshot shows a sitatuation where the connection of the Platform Imu should be verified.

Troubleshooting Suggestions

• If the DTrack indicator is not green please make sure that DTrack is running and that the Output settings in DTrack and the port number configured for the LPVR driver are correct and agree. If you are working in a restricted IT environment, you may have to select “act as router for tracking output” in DTrack’s output settings.

• If the indicator for the platform IMU is red, make sure its USB cable is connected and restart Varjo Base. We discuss its setup in detail further down.

• If the headset IMU indicator is red, please restart Varjo Base. If it remains red, please make sure Varjo Base doesn’t display any errors.

Detailed Validation Steps

If the system misbehaves it may be useful to look in more detail to isolate which part is failing. In particular the motion can be subdivided into the following parts which can be validated separately.

• The headset reacts to rotations: Stop the DTrack measurement. If you now rotate the headset, the VR scene should rotate accordingly.

• Positional data is transferred correctly: Start the DTrack measurement again. Moving the headset without rotation yields the expected motion. E.g. moving the headset vertically leads to a corresponding shift of the VR scene.

• The tracking body for the headset is calibrated correctly, in particular, the rotation of IMU and tracking body are matched. To verify this, make sure that the DTrack measurement is running and the data is flowing as in the two previous steps. The rotations should appear smooth. Miscalibrations can lead to rotational jumps or other artefacts.

• Platform IMU data is received and incorporated: rotating the Platform IMU leads to a response in the VR scene. Before calibration, rotation axes may not match.

If alignment problems persist, please take the following steps to isolate the source of the problem:

Track headset with IMU-only (no optical)

In expert Mode:

• Stop ART tracking in DTrack

• Set "ignoreGravity": false to turn on the vertical gravity reference

• Set "combinerType": "Default" to turn off relative orientation tracking with the reference IMU

• Push to driver

With these settings the headset will be tracked by the HMD IMU only. When wearing the headset, if the IMU alignment settings are correct, orientation tracking should feel natural. Please note that without optical system there is no position tracking. If orientation tracking is not working well, the alignment between IMU and the HMD optics (display, lenses) is not good.

Track headset with optical system-only (no IMU)

• Make sure ART tracking is running

• If it doesn’t exist yet, in imuSources create

  { "id": "no_imu", "type": "None” }

• Set "imuSource": "no_imu" in trackedObjects

• Push to driver

Within the range of the optical system, the HMD should now be orientation and position tracked. The tracking will feel a lot less smooth when only using the optical system. However, this mode is a good way to assess the quality of the rigid body adjustment. If orientation tracking completely off, it is better to readjust the rigid body.