Hardware Installation

The system consists of the following hardware components:

• A Varjo Headset

• ART optical tracking system in an in-vehicle configuration (e.g. Smarttrack 3)

• An external LPMS-IG1 IMU that is fixed to the vehicle frame (“platform IMU”)

• A PC running Varjo Base that powers the VR/AR experience

The optical tracking system needs to be installed as defined by the optical tracking system manufacturer. Please take care that the floor of the tracking space is calibrated vertically. A water balance built-into the calibration equipment should assist with that.

The following image (from the ART website) illustrates a possible placement of the optical tracking system.

For differential IMU tracking, we need to install a further IMU inside the vehicle, in a location visible to the optical tracking system. Alternatively, this IMU can also be installed in a location not visible to the optical tracking, but with exactly known orientation within the optical reference system.

We will provide further instructions on how to calibrate these components to work in the same coordinate system in the following chapters.

Headset Tracking Body Setup

Please follow the instructions on this page to setup ART’s tracking of the headset Automatic HMD / ART-Body Intercalibration - LP-RESEARCH Knowledge Base - Confluence (atlassian.net). We recommend to do this before installing the driver, but the procedure can followed at any time.

Driver Setup

System Requirements

• LPVR in its current version runs exclusively on Windows 10.

• Varjo Base must be installed and operational.

• Applications that use the Varjo headset don’t have to be modified for use with the LPVR-DUO-Varjo driver

Driver Installation

HMD

In the following section we will describe how to setup LPVR-CAD.

• Make sure your optical tracking system software (e.g. DTrack) is running and can see the optical markers attached your HMD and hand controllers. For more information on how to configure your rigid bodies refer to section System Calibration.

• First, we need to download the installer for the LPVR-CAD middleware. Log in to your LP-Research user account to access your dedicated customer area here:

https://lp-research.com/customer-area

If you haven’t received login credentials from us yet, please contact us at support@lp-research.com. The screenshots below show how to download the installer files from the customer area.

• Insert the LPVR dongle into a free USB port of your computer.

• Quit Varjo Base in case it is running.

• Unzip the archive you downloaded and run the LPVR-CAD installer. Figure 6 shows the output of the installer after successful completion.

• Once the driver has finished installing start Varjo Base.

• In Varjo Base go to the system tab and select LPVR-DUO-Varjo as headset tracking method from the dropdown menu.

Driver Configuration

• Point your web browser to http://localhost:7119. A configuration webpage (depicted below) should pop up.

• For reference see the contents of the default configuration which can be expanded below:

{
      "absoluteSources": {
        "DTrack": {
          "settings": {
            "oneBasedCounting": true,
            "port": 5000
          },
          "type": "DTrack"
        },
        "my_dummy": {
          "type": "Dummy"
        }
      },
      "emitters": {
        "HMD": {
          "type": "Keeper"
        },
        "console": {
          "type": "Console"
        }
      },
      "imuSources": {
        "HMD IMU": {
          "type": "Varjo"
        },
        "Platform IMU": {
          "settings": {
            "autodetectType": "ig1"
          },
          "type": "OpenZen"
        }
      },
      "mapper": {
        "dataProxy": {
          "port": 14355
        }
      },
      "trackedObjects": [
        {
          "absoluteSource": {
            "name": "DTrack",
            "trackingId": 1
          },
          "combinerType": "DifferentialImu",
          "emitterName": "HMD",
          "imuSource": "HMD IMU",
          "settings": {
            "ignoreGravity": true,
            "opticalWeight": 0.005,
            "referenceImu": "Platform IMU",
            "referenceOrientationQuat": {
              "w": 1,
              "x": 0,
              "y": 0,
              "z": 0
            },
            "referenceToOpticalQuat": {
              "w": 1,
              "x": 0,
              "y": 0,
              "z": 0
            }
          }
        },
        {
          "absoluteSource": {
            "_the platform IMU body": "goes here",
            "name": "DTrack",
            "trackingId": 2
          },
          "combinerType": "Default",
          "emitterName": "console",
          "settings": {}
        }
      ]
  }

• In order to use LPVR-DUO we need to connect the various parts and we need to calibrate the orientation of the platform IMU that is mounted to the vehicle. For the following settings we give the line numbers in the default configuration.

• In order to receive data from DTrack the port number configured on line 7 needs to match the one configured in DTrack. The default configuration uses port number 5000 which is also the default for DTrack.

• In DTrack’s Output Settings you need to set one channel to “active” and configure it to send to “this computer” with at least “ts” and “6d” output enabled. Depending on your firewall settings you may have to select “act as router for tracking output”.

• You also need to configure the correct body ID for the tracked body associated with the headset. This number goes on line 42 of the configuration.

• Lastly, for the purpose of calibration, we also need a tracking body attached to the platform IMU. Its tracking body ID goes on line 69.

• Once these settings are complete, click the button labelled “Push To Driver” above the input field. The driver should now have adopted the settings.

If you have already calibrated the headset tracking body (see above), you should now be able to now use the headset with DTrack tracking. It should also react to motion of the platform IMU. Though as long as the platform IMU is not calibrated, this motion should not make much sense.

Validation Steps

After this basic configuration all necessary data should be available to Varjo Base. We suggest to take a few minutes to make sure this is the case. Namely, at this point the headset motion should be picked up by DTrack, IMU data should be flowing and the driver should provide an (uncalibrated) synthesis to Varjo Base. In other words, the VR scene should respond to motions of the headset and the IG1 imu, but not necessarily in the correct way.

In particular the following items can be verified:

• DTrack tracking works - Make sure that the headset is tracked by DTrack. DTrack should be running and the DTrack software should show it’s body number on a green background

• All required data reaches the driver - After clicking “Show details” near driver status on the configuration page, all sources (DTrack, HMD Imu, Platform Imu) should be displayed in green. The below screenshot shows a sitatuation where the connection of the Platform Imu should be verified.

Troubleshooting Suggestions

• If the DTrack indicator is not green please make sure that DTrack is running and that the Output settings in DTrack and the port number configured for the LPVR driver are correct and agree. If you are working in a restricted IT environment, you may have to select “act as router for tracking output” in DTrack’s output settings.

• If the indicator for the platform IMU is red, make sure its USB cable is connected and restart Varjo Base. We discuss its setup in detail further down.

• If the headset IMU indicator is red, please restart Varjo Base. If it remains red, please make sure Varjo Base doesn’t display any errors.

Detailed Validation Steps

If the system misbehaves it may be useful to look in more detail to isolate which part is failing. In particular the motion can be subdivided into the following parts which can be validated separately.

• The headset reacts to rotations: Stop the DTrack measurement. If you now rotate the headset, the VR scene should rotate accordingly.

• Positional data is transferred correctly: Start the DTrack measurement again. Moving the headset without rotation yields the expected motion. E.g. moving the headset vertically leads to a corresponding shift of the VR scene.

• The tracking body for the headset is calibrated correctly, in particular, the rotation of IMU and tracking body are matched. To verify this, make sure that the DTrack measurement is running and the data is flowing as in the two previous steps. The rotations should appear smooth. Miscalibrations can lead to rotational jumps or other artefacts.

• Platform IMU data is received and incorporated: rotating the Platform IMU leads to a response in the VR scene. Before calibration, rotation axes may not match.

If alignment problems persist, please take the following steps to isolate the source of the problem:

Track headset with IMU-only (no optical)

• Stop ART tracking in DTrack

• Set "ignoreGravity": false to turn on the vertical gravity reference

• Set "combinerType": "Default" to turn off relative orientation tracking with the reference IMU

• Push to driver

With these settings the headset will be tracked by the HMD IMU only. When wearing the headset, if the IMU alignment settings are correct, orientation tracking should feel natural. Please note that without optical system there is no position tracking. If orientation tracking is not working well, the alignment between IMU and the HMD optics (display, lenses) is not good.

Track headset with optical system-only (no IMU)

• Make sure ART tracking is running

• If it doesn’t exist yet, in imuSources create

  { 
    "id": "no_imu", 
    "type": "None” 
  }

• Set "imuSource": "no_imu" in trackedObjects

• Push to driver

Within the range of the optical system, the HMD should now be orientation and position tracked. The tracking will feel a lot less smooth when only using the optical system. However, this mode is a good way to assess the quality of the rigid body adjustment. If orientation tracking completely off, it is better to readjust the rigid body.

Adjustment of Platform IMU

General

• Please follow our tutorial for initial installation of the vehicle-fixed LPMS-IG1 IMU (the “Platform IMU”).

• If you are using LPVR-DUO for Varjo with the ART SmartTrack and our provided IMU holder, you should be done at this step. If you are using another setup, a detailed calibration guide can be found in the fold-out below.

Manual Calibration Procedure

• In a first step the orientation of the internal coordinate system of the vehicle fixed IMU and the optical tracking system needs to be defined. For this purpose, attach an optical marker to the enclosure of the IMU and create a new rigid body in the optical tracking software.

• To activate the calibration features of LPVR, click on Show Additional Features on the LPVR configuration page.

• Activate the automatic pivot point calibration by selecting the optical tracking system (“DTrack” or “my_art”) in the absolute source field with rigid body id tracking id and the vehicle fixed IMU (“Platform IMU” or “reference_imu”) in the imu source field. Clicking Find Rotation will start the calibration process.

• During the calibration process, slowly rotate the IMU with attached optical marker within the view of the optical tracking system. Pause the rotation from time-to-time to allow for some internal processing. After the calibration process is finished the result will be output below the calibration feature dialog. The output quaternion needs to be manually inserted into referenceToOpticalQuat.

• Fix the platform IMU to the vehicle in the intended location with the optical marker still attached. Make sure that it is visible to DTrack. We now need to read the orientation quaternion from Varjo Bases log files and enter them as referenceOrientationQuat around line 59 of the configuration. Accessing the Varjo Base log files takes a few steps which we detail below.

• Open Diagnostics from the Support menu in Varjo Base

• On the next screen, click Start

• Click “Skip” in order to not record eye tracking data

• You are now presented with the following screen which provides you with a link to the location where the log files are stored. Click on “Show folder”

• Enter the log folder, out of the files named tracking_<number>.log select the latest one and open it. Near its end search for lines similar to this
  20210120 15:16:31.156 --- [Varjo::Tracking::PluginUtilityAPI::WriteLogMessage] PLUGIN: [INFO] new pose: q = {"w":0.9913584114738726,"x":0.1302836073262127,"y":0.006246731684853627,"z":-0.013979051208529776} pos = {"x":-1.1893610000000001,"y":0.831757,"z":-0.936706}

• Replace the contents of referenceOrientationQuat in the configuration web page with the values of q in the log file, i.e. {"w":0.9913584114738726,"x":0.1302836073262127,"y":0.006246731684853627,"z":-0.013979051208529776} in this example.

• After the above procedure the optical marker can be detached from the platform IMU.