Hardware Installation

The system consists of the following hardware components:

• A Varjo Headset

• ART optical tracking system in an in-vehicle configuration (e.g. Smarttrack 3)

• An external LPMS-IG1 IMU that is fixed to the vehicle frame (“platform IMU”)

• A PC running Varjo Base that powers the VR/AR experience

The optical tracking system needs to be installed as defined by the optical tracking system manufacturer. Please take care that the floor of the tracking space is calibrated vertically. A water balance built-into the calibration equipment should assist with that.

The following image (from the ART website) illustrates a possible placement of the optical tracking system.

...

For differential IMU tracking, we need to install a further IMU inside the vehicle, in a location visible to the optical tracking system. Alternatively, this IMU can also be installed in a location not visible to the optical tracking, but with exactly known orientation within the optical reference system.

We will provide further instructions on how to calibrate these components to work in the same coordinate system in the following chapters.

Headset Tracking Body Setup

Please follow the instructions on this page to setup ART’s tracking of the headset Automatic HMD / ART-Body Intercalibration - LP-RESEARCH Knowledge Base - Confluence (atlassian.net). We recommend to do this before installing the driver, but the procedure can followed at any time.

Driver Setup

System Requirements

• LPVR in its current version runs exclusively on Windows 10.

• Varjo Base must be installed and operational.

• Applications that use the Varjo headset don’t have to be modified for use with the LPVR-DUO-Varjo driver

Driver Installation

HMD

In the following section we will describe how to setup LPVR-CAD.

• Make sure your optical tracking system software (e.g. DTrack) is running and can see the optical markers attached your HMD and hand controllers. For more information on how to configure your rigid bodies refer to section System Calibration.

• First, we need to download the installer for the LPVR-CAD middleware. Log in to your LP-Research user account to access your dedicated customer area here:

https://lp-research.com/customer-area

If you haven’t received login credentials from us yet, please contact us at support@lp-research.com. The screenshots below show how to download the installer files from the customer area.

...

• Insert the LPVR dongle into a free USB port of your computer.

• Quit Varjo Base in case it is running.

• Unzip the archive you downloaded and run the LPVR-CAD installer. Figure 6 shows the output of the installer after successful completion.

...

• Once the driver has finished installing start Varjo Base.

...

• In Varjo Base go to the system tab and select LPVR-DUO-Varjo as headset tracking method from the dropdown menu.

...

Driver Configuration

Point your web browser to http://localhost:7119. A configuration webpage (depicted below) should pop up.

...

• For reference see the contents of the default configuration which can be expanded below:

{
      "absoluteSources": {
        "DTrack": {
          "settings": {
            "oneBasedCounting": true,
            "port": 5000
          },
          "type": "DTrack"
        },
        "my_dummy": {
          "type": "Dummy"
        }
      },
      "emitters": {
        "HMD": {
          "type": "Keeper"
        },
        "console": {
          "type": "Console"
        }
      },
      "imuSources": {
        "HMD IMU": {
          "type": "Varjo"
        },
        "Platform IMU": {
          "settings": {
            "autodetectType": "ig1"
          },
          "type": "OpenZen"
        }
      },
      "mapper": {
        "dataProxy": {
          "port": 14355
        }
      },
      "trackedObjects": [
        {
          "absoluteSource": {
            "name": "DTrack",
            "trackingId": 1
          },
          "combinerType": "DifferentialImu",
          "emitterName": "HMD",
          "imuSource": "HMD IMU",
          "settings": {
            "ignoreGravity": true,
            "opticalWeight": 0.005,
            "referenceImu": "Platform IMU",
            "referenceOrientationQuat": {
              "w": 1,
              "x": 0,
              "y": 0,
              "z": 0
            },
            "referenceToOpticalQuat": {
              "w": 1,
              "x": 0,
              "y": 0,
              "z": 0
            }
          }
        },
        {
          "absoluteSource": {
            "_the platform IMU body": "goes here",
            "name": "DTrack",
            "trackingId": 2
          },
          "combinerType": "Default",
          "emitterName": "console",
          "settings": {}
        }
      ]
  }

• In order to use LPVR-DUO we need to connect the various parts and we need to calibrate the orientation of the platform IMU that is mounted to the vehicle. For the following settings we give the line numbers in the default configuration.

• In order to receive data from DTrack the port number configured on line 7 needs to match the one configured in DTrack. The default configuration uses port number 5000 which is also the default for DTrack.

...

• In DTrack’s Output Settings you need to set one channel to “active” and configure it to send to “this computer” with at least “ts” and “6d” output enabled. Depending on your firewall settings you may have to select “act as router for tracking output”.

...

• You also need to configure the correct body ID for the tracked body associated with the headset. This number goes on line 42 of the configuration.

...

• Lastly, for the purpose of calibration, we also need a tracking body attached to the platform IMU. Its tracking body ID goes on line 69.

...

• Once these settings are complete, click the button labelled “Push To Driver” above the input field. The driver should now have adopted the settings.

If you have already calibrated the headset tracking body (see above), you should now be able to now use the headset with DTrack tracking. It should also react to motion of the platform IMU. Though as long as the platform IMU is not calibrated, this motion should not make much sense.

Validation Steps

After this basic configuration all necessary data should be available to Varjo Base. We suggest to take a few minutes to make sure this is the case. Namely, at this point the headset motion should be picked up by DTrack, IMU data should be flowing and the driver should provide an (uncalibrated) synthesis to Varjo Base. In other words, the VR scene should respond to motions of the headset and the IG1 imu, but not necessarily in the correct way.

In particular the following items can be verified:

• DTrack tracking works - Make sure that the headset is tracked by DTrack. DTrack should be running and the DTrack software should show it’s body number on a green background

...

• All required data reaches the driver - After clicking “Show details” near driver status on the configuration page, all sources (DTrack, HMD Imu, Platform Imu) should be displayed in green. The below screenshot shows a sitatuation where the connection of the Platform Imu should be verified.

...

Troubleshooting Suggestions

• If the DTrack indicator is not green please make sure that DTrack is running and that the Output settings in DTrack and the port number configured for the LPVR driver are correct and agree. If you are working in a restricted IT environment, you may have to select “act as router for tracking output” in DTrack’s output settings.

• If the indicator for the platform IMU is red, make sure its USB cable is connected and restart Varjo Base. We discuss its setup in detail further down.

• If the headset IMU indicator is red, please restart Varjo Base. If it remains red, please make sure Varjo Base doesn’t display any errors.

Detailed Validation Steps

If the system misbehaves it may be useful to look in more detail to isolate which part is failing. In particular the motion can be subdivided into the following parts which can be validated separately.

• The headset reacts to rotations: Stop the DTrack measurement. If you now rotate the headset, the VR scene should rotate accordingly.

• Positional data is transferred correctly: Start the DTrack measurement again. Moving the headset without rotation yields the expected motion. E.g. moving the headset vertically leads to a corresponding shift of the VR scene.

• The tracking body for the headset is calibrated correctly, in particular, the rotation of IMU and tracking body are matched. To verify this, make sure that the DTrack measurement is running and the data is flowing as in the two previous steps. The rotations should appear smooth. Miscalibrations can lead to rotational jumps or other artefacts.

• Platform IMU data is received and incorporated: rotating the Platform IMU leads to a response in the VR scene. Before calibration, rotation axes may not match.

If alignment problems persist, please take the following steps to isolate the source of the problem:

Track headset with IMU-only (no optical)

...

Stop ART tracking in DTrack

...

Push to driver

With these settings the headset will be tracked by the HMD IMU only. When wearing the headset, if the IMU alignment settings are correct, orientation tracking should feel natural. Please note that without optical system there is no position tracking. If orientation tracking is not working well, the alignment between IMU and the HMD optics (display, lenses) is not good.

Track headset with optical system-only (no IMU)

...

Make sure ART tracking is running

...

{ 
  "id": "no_imu", 
  "type": "None” 
}

...

Push to driver

Within the range of the optical system, the HMD should now be orientation and position tracked. The tracking will feel a lot less smooth when only using the optical system. However, this mode is a good way to assess the quality of the rigid body adjustment. If orientation tracking completely off, it is better to readjust the rigid body.

Adjustment of Platform IMU

General

• Please follow our tutorial for initial installation of the vehicle-fixed LPMS-IG1 IMU (the “Platform IMU”).

• If you are using LPVR-DUO for Varjo with the ART SmartTrack and our provided IMU holder, you should be done at this step. If you are using another setup, a detailed calibration guide can be found in the fold-out below.

Image Removed

If LPMS-IG1 is fixed on Smarttrack 3, use the default settings in LPVR-DUO, no further calibration needed.

In this case the referenceOrientationQuat needs to be set to:

Code Block
"referenceOrientationQuat": { "w": 1, "x": -1, "y": 1, "z": 1 }

,

Image Removed

If your LPMS-IG1 is installed on a different location inside the optical tracking volume, attach the marker target above and perform the manual calibration described below.

Manual Calibration Procedure

• In a first step the orientation of the internal coordinate system of the vehicle fixed IMU and the optical tracking system needs to be defined. For this purpose, attach an optical marker to the enclosure of the IMU and create a new rigid body in the optical tracking software.

• To activate the calibration features of LPVR, click on Show Additional Features on the LPVR configuration page.

...

During the calibration process, slowly rotate the IMU with attached optical marker within the view of the optical tracking system. Pause the rotation from time-to-time to allow for some internal processing. After the calibration process is finished the result will be output below the calibration feature dialog. The output quaternion needs to be manually inserted into referenceToOpticalQuat.

...

• Fix the platform IMU to the vehicle in the intended location with the optical marker still attached. Make sure that it is visible to DTrack. We now need to read the orientation quaternion from Varjo Bases log files and enter them as referenceOrientationQuat around line 59 of the configuration. Accessing the Varjo Base log files takes a few steps which we detail below.

• Open Diagnostics from the Support menu in Varjo Base

...

• On the next screen, click Start

...

• Click “Skip” in order to not record eye tracking data

...

• You are now presented with the following screen which provides you with a link to the location where the log files are stored. Click on “Show folder”

...

• Enter the log folder, out of the files named tracking_<number>.log select the latest one and open it. Near its end search for lines similar to this
  20210120 15:16:31.156 --- [Varjo::Tracking::PluginUtilityAPI::WriteLogMessage] PLUGIN: [INFO] new pose: q = {"w":0.9913584114738726,"x":0.1302836073262127,"y":0.006246731684853627,"z":-0.013979051208529776} pos = {"x":-1.1893610000000001,"y":0.831757,"z":-0.936706}

• Replace the contents of referenceOrientationQuat in the configuration web page with the values of q in the log file, i.e. {"w":0.9913584114738726,"x":0.1302836073262127,"y":0.006246731684853627,"z":-0.013979051208529776} in this example.

• After the above procedure the optical marker can be detached from the platform IMU.

...

The platform IMU should now not be moved anymore. The system should be sufficiently calibrated for operation.

...

The configuration interface includes several pages, accessible through the links at the bottom. A typical setup would work through them starting with the “Configuration” page (depicted), then move on to “Calibration”. If need be, detailed configuration parameters can be entered in “Expert Mode”. Finally, this documentation can be reached via the “Documentation” link.

Tracking System Setup

Please return to the “Configuration” page in order to set up the optical tracking of the headset.

Set the “Port Number” to match the output in DTrack and the “Body ID” in the headset section to match the body configured for the headset in DTrack. The “Status” field next to the Tracking System Settings should turn green and say “Receiving Data”. The left bubble in the “Headset” section should say “Tracked”. Please be sure to click the “Apply settings” button after any changes.

You should now see the VR content move in your headset according to the headset’s motion. If the motion appears incorrect, refer to the “Headset Tracking Body Setup” section above.

Configuration of the Platform IMU

Click on “Calibration” near the bottom of the screen to proceed to the setup of the Platform IMU. You will be offered a selection of options based on your specific setup.

...

Each of these options is appropriate in different cases. In the simplest case, the Platform IMU (also called “Reference IMU”) is attached to the camera system and can be moved along with it. This corresponds to the first case. If the IMU is mounted within the field of view of the camera system and a marker target can be attached to it for the purposes of the system calibration, the second option is appropriate. In cases where a precise relationship has to be established otherwise (say, from technical drawings), the third option is appropriate. Let’s discuss each option briefly.

First alternative: Calibration with the Platform IMU attached to the camera system

This calibration can be performed once after the IMU has been tightly attached to the camera system. Additionally, it requires moving the camera system. Therefore, we recommend doing it before installing the system in the more constrained space of a car or simulator cabin.

...

The procedure is explained quickly: the camera system and the IMU are moved together while pointing at a stationary body which is tracked by the camera system, typically the headset. By comparing the tracking data for the body and the IMU data, the relationship between the two can be established. LPVR-DUO performs this comparison and derives the necessary calibration parameters.

...

After selecting the calibration on the page, the screen will look something like the above. Make sure that the right-hand bubble reads “IMU active” if not, please check the wiring and restart the VR software and/or the LPVR driver. Now select the correct id for the reference object, which can be any body tracked by your tracking system. It should be stationary during the course of the calibration. If you are using the headset, please make sure it is steadily supported, and doesn’t move during the calibration. Once you are ready and the reference object is in the field of view of the system, you can click the “Start Calibration” button and proceed to the calibration.

A progress bar will appear on the screen. Please pick up the camera system, and move it gently turning it around its different axes while keeping the reference object in the field of view. Make sure to avoid sudden movements and pause in a number of different orientations until the system has recorded sufficient data. This will typically take 15-30 seconds.

...

Once the calibration completes, a button will appear which allows you to apply the result to the configuration. Clicking it should complete the calibration, and your LPVR-DUO system should be operational.

...

Second Alternative: Calibration with Platform IMU installed in the field of view of the tracking system

This calibration is divided into two steps: first the relation between the IMU and the body used to track it will be established. During this calibration, the IMU will have to be moved within the field of view of the camera. By comparing the tracking data for the body and the IMU data, the relationship between the two can be established. LPVR-DUO performs this comparison and derives the necessary calibration parameters. In a second step, the IMU will be mounted into the final position and the orientation will be read off by the optical tracking system.

...

After selecting the calibration on the page, the screen will look something like the below. Make sure that the right-hand bubble reads “IMU active” if not, please check the wiring and restart the VR software and/or the LPVR driver. Now select the correct id for the tracking body of the IMU. Once you are ready and the IMU is in the field of view of the system, you can click the “Start Calibration” button and proceed to the calibration.

...

A progress bar will appear on the screen. Please pick up the IMU, and move it gently turning it around its different axes while keeping inside the field of view. Make sure to avoid sudden movements and pause in a number of different orientations until the system has recorded sufficient data. This will typically take 15-30 seconds.

...

Once the calibration completes, a button will appear which allows you to apply the result to the configuration. Clicking it should complete the calibration, and your LPVR-DUO system should be operational.

...

Now, please put the IMU into the final position inside the car, simulator platform or whatever movable platform you are using. It should be in the field of view of the tracking system.

...

You should now be able to click the “Use current orientation” button to complete the calibration. The tracking body can now be removed from the IMU and you LPVR-DUO system should be fully functional.

Third alternative: Manual entry of the orientation of the Platform IMU

...

This is only recommended to experts. You can enter a quaternion manually which converts from the IMU’s coordinate system (as printed on the IMU) to the Optical Tracking coordinate frame. Entering the members of the unnormalized quaternion followed by “Apply settings” will update the system to use these settings.

Validation Steps

After this basic configuration all necessary data should be available to Varjo Base. We suggest to take a few minutes to make sure this is the case. Namely, at this point the headset motion should be picked up by DTrack, IMU data should be flowing and the driver should provide an (uncalibrated) synthesis to Varjo Base. In other words, the VR scene should respond to motions of the headset and the IG1 imu, but not necessarily in the correct way.

In particular the following items can be verified:

• DTrack tracking works - Make sure that the headset is tracked by DTrack. DTrack should be running and the DTrack software should show it’s body number on a green background

...

• All required data reaches the driver - After clicking “Show details” near driver status on the configuration page, all sources (DTrack, HMD Imu, Platform Imu) should be displayed in green. The below screenshot shows a sitatuation where the connection of the Platform Imu should be verified.

...

Troubleshooting Suggestions

• If the DTrack indicator is not green please make sure that DTrack is running and that the Output settings in DTrack and the port number configured for the LPVR driver are correct and agree. If you are working in a restricted IT environment, you may have to select “act as router for tracking output” in DTrack’s output settings.

• If the indicator for the platform IMU is red, make sure its USB cable is connected and restart Varjo Base. We discuss its setup in detail further down.

• If the headset IMU indicator is red, please restart Varjo Base. If it remains red, please make sure Varjo Base doesn’t display any errors.

Detailed Validation Steps

If the system misbehaves it may be useful to look in more detail to isolate which part is failing. In particular the motion can be subdivided into the following parts which can be validated separately.

• The headset reacts to rotations: Stop the DTrack measurement. If you now rotate the headset, the VR scene should rotate accordingly.

• Positional data is transferred correctly: Start the DTrack measurement again. Moving the headset without rotation yields the expected motion. E.g. moving the headset vertically leads to a corresponding shift of the VR scene.

• The tracking body for the headset is calibrated correctly, in particular, the rotation of IMU and tracking body are matched. To verify this, make sure that the DTrack measurement is running and the data is flowing as in the two previous steps. The rotations should appear smooth. Miscalibrations can lead to rotational jumps or other artefacts.

• Platform IMU data is received and incorporated: rotating the Platform IMU leads to a response in the VR scene. Before calibration, rotation axes may not match.

If alignment problems persist, please take the following steps to isolate the source of the problem:

Track headset with IMU-only (no optical)

In expert Mode:

• Stop ART tracking in DTrack

• Set"ignoreGravity": falseto turn on the vertical gravity reference

• Set "combinerType": "Default"to turn off relative orientation tracking with the reference IMU

• Push to driver

With these settings the headset will be tracked by the HMD IMU only. When wearing the headset, if the IMU alignment settings are correct, orientation tracking should feel natural. Please note that without optical system there is no position tracking. If orientation tracking is not working well, the alignment between IMU and the HMD optics (display, lenses) is not good.

Track headset with optical system-only (no IMU)

• Make sure ART tracking is running

• If it doesn’t exist yet, in imuSourcescreate

  Code Block
  { "id": "no_imu", "type": "None” }

• Set "imuSource": "no_imu"intrackedObjects

• Push to driver

Within the range of the optical system, the HMD should now be orientation and position tracked. The tracking will feel a lot less smooth when only using the optical system. However, this mode is a good way to assess the quality of the rigid body adjustment. If orientation tracking completely off, it is better to readjust the rigid body.