Hardware Installation

The system consists of the following hardware components:

A Varjo Headset
ART optical tracking system in an in-vehicle configuration (e.g. Smarttrack 3)
An external LPMS-IG1 IMU that is fixed to the vehicle frame (“reference IMU”)
A PC running Varjo Base that powers the VR/AR experience

The optical tracking system needs to be installed as defined by the optical tracking system manufacturer. Please take care that the floor of the tracking space is calibrated vertically. A water balance built-into the calibration equipment should assist with that.

The following image (from the ART website) illustrates a possible placement of the optical tracking system.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20201211-053228.png

For differential IMU tracking, we need to install a further IMU inside the vehicle, in a location visible to the optical tracking system. Alternatively, this IMU can also be installed in a location not visible to the optical tracking, but with exactly known orientation within the optical reference system.

We will provide further instructions on how to calibrate these components to work in the same coordinate system in the following chapters.

Headset Tracking Body Setup

Please follow the instructions on this page to setup ART’s tracking of the headset Automatic HMD / ART-Body Intercalibration - LP-RESEARCH Knowledge Base - Confluence (atlassian.net). We recommend to do this before installing the driver, but the procedure can followed at any time.

Driver Setup

System Requirements

LPVR in its current version runs exclusively on Windows 10.
Varjo Base must be installed and operational.
Applications that use the Varjo headset don’t have to be modified for use with the LPVR-DUO-Varjo driver

Driver Installation

HMD

In the following section we will describe how to setup LPVR-CAD.

Make sure your optical tracking system software (e.g. DTrack) is running and can see the optical markers attached your HMD and hand controllers. For more information on how to configure your rigid bodies refer to section System Calibration.
First, we need to download the installer for the LPVR-CAD middleware. Log in to your LP-Research user account to access your dedicated customer area here:

https://lp-research.com/customer-area

If you haven’t received login credentials from us yet, please contact us at support@lp-research.com. The screenshots below show how to download the installer files from the customer area.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20201105-081909.png

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20201105-081931.png

Insert the LPVR dongle into a free USB port of your computer.
Quit Varjo Base in case it is running.
Unzip the archive you downloaded and run the LPVR-CAD installer. Figure 6 shows the output of the installer after successful completion.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20201105-065530.png

Once the driver has finished installing start Varjo Base.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20201105-080143.png

In Varjo Base go to the system tab and select LPVR-DUO-Varjo as headset tracking method from the dropdown menu.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-062609.png

Driver Configuration

Point your web browser to http://localhost:7119. A configuration webpage (depicted below) should pop up.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-063307.png

For reference see the contents of the default configuration below:

{
      "absoluteSources": {
        "DTrack": {
          "settings": {
            "oneBasedCounting": true,
            "port": 5000
          },
          "type": "DTrack"
        },
        "my_dummy": {
          "type": "Dummy"
        }
      },
      "emitters": {
        "HMD": {
          "type": "Keeper"
        },
        "console": {
          "type": "Console"
        }
      },
      "imuSources": {
        "HMD IMU": {
          "type": "Varjo"
        },
        "Platform IMU": {
          "settings": {
            "autodetectType": "ig1"
          },
          "type": "OpenZen"
        }
      },
      "mapper": {
        "dataProxy": {
          "port": 14355
        }
      },
      "trackedObjects": [
        {
          "absoluteSource": {
            "name": "DTrack",
            "trackingId": 1
          },
          "combinerType": "DifferentialImu",
          "emitterName": "HMD",
          "imuSource": "HMD IMU",
          "settings": {
            "ignoreGravity": true,
            "opticalWeight": 0.005,
            "referenceImu": "Platform IMU",
            "referenceOrientationQuat": {
              "w": 1,
              "x": 0,
              "y": 0,
              "z": 0
            },
            "referenceToOpticalQuat": {
              "w": 1,
              "x": 0,
              "y": 0,
              "z": 0
            }
          }
        },
        {
          "absoluteSource": {
            "_the platform IMU body": "goes here",
            "name": "DTrack",
            "trackingId": 2
          },
          "combinerType": "Default",
          "emitterName": "console",
          "settings": {}
        }
      ]
  }

In order to use LPVR-DUO we need to connect the various parts and we need to calibrate the orientation of the reference IMU that is mounted to the vehicle. For the following settings we give the line numbers in the default configuration.
In order to receive data from DTrack the port number configured on line 7 needs to match the one configured in DTrack. The default configuration uses port number 5000 which is also the default for DTrack.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-063604.png

In DTrack’s Output Settings you need to set one channel to “active” and configure it to send to “this computer” with at least “ts” and “6d” output enabled. Depending on your firewall settings you may have to select “act as router for tracking output”.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-063813.png

You also need to configure the correct body ID for the tracked body associated with the headset. This number goes on line 42 of the configuration.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-064104.png

Lastly, for the purpose of calibration, we also need a tracking body attached to the reference IMU. Its tracking body ID goes on line 69.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-064205.png

Once these settings are complete, click the button labelled “Push To Driver” above the input field. The driver should now have adopted the settings.

If you have already calibrated the headset tracking body (see above), you should now be able to now use the headset with DTrack tracking. It should also react to motion of the reference IMU. Though as long as the reference IMU is not calibrated, this motion should not make much sense.

Validation Steps

After this basic configuration all necessary data should be available to Varjo Base. We suggest to take a few minutes to make sure this is the case. Namely, at this point the headset motion should be picked up by DTrack, IMU data should be flowing and the driver should provide an (uncalibrated) synthesis to Varjo Base.

In particular the following items can be verified:

DTrack tracking works - Make sure that the headset is tracked by DTrack. DTrack should be running and the DTrack software should show it’s body number on a green background

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20220609-064638.png

All required data reaches the driver - After clicking “Show details” near driver status on the configuration page, all sources (DTrack, HMD Imu, Platform Imu) should be displayed in green. The below screenshot shows a sitatuation where the connection of the Platform Imu should be varified.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20220609-070217.png

Some troubleshooting suggestions:

If the DTrack indicator is not green make sure that DTrack is running and that the Output settings in DTrack and the port number configured for the LPVR driver are correct and agree.
If the indicator for the platform IMU is red, make sure its USB cable is connected and restart Varjo Base. We discuss its setup in detail further down.
If the headset IMU indicator is red, please restart Varjo Base. If it remains red, please make sure Varjo Base doesn’t display any errors.

Adjustment of Vehicle Fixed IMU

Follow this tutorial to install and run the vehicle-fixed LPMS-IG1 IMU. If you are using LPVR-DUO for Varjo with the ART SmartTrack and our provided IMU holder, you should be done at this step. If you are using another setup, please follow the steps below.

Calibration Procedure

In a first step the orientation of the internal coordinate system of the vehicle fixed IMU and the optical tracking system needs to be defined. For this purpose, attach an optical marker to the enclosure of the IMU and create a new rigid body in the optical tracking software.
To activate the calibration features of LPVR, click on Show Additional Features on the LPVR configuration page.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > bM1EpzJ76It_ORor87axRU_mCkVUewRyJ56CHKBJDao-ewq0PVHfd0NwBiIwZtnqGg9kEtkbEeAACuvKjcHUHcDzjFfIc2pI46IktEWGfghxOsgf_x3krNcxAJ6ME8LvGuAakrsM

Activate the automatic pivot point calibration by selecting the optical tracking system (“DTrack” or “my_art”) in the absolute source field with rigid body id tracking id and the vehicle fixed IMU (“Platform IMU” or “reference_imu”) in the imu source field. Clicking Find Rotation will start the calibration process.
During the calibration process, slowly rotate the IMU with attached optical marker within the view of the optical tracking system. Pause the rotation from time-to-time to allow for some internal processing. After the calibration process is finished the result will be output below the calibration feature dialog. The output quaternion needs to be manually inserted into referenceToOpticalQuat.

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > tfb8Hs4bqKWG5SrwklGKtLUSNt2rYpRE79pR99_MGx99zXc-4Lm_-307FxvWGHdJeSSzzyHgDXv5XDx5jKgTc68Rxxd-3ghlGpfDMBIgqKaP2NOKRpOMiXH9fUODzk9n3JaxGKyE

Fix the reference IMU to the vehicle in the intended location with the optical marker still attached. Make sure that it is visible to DTrack. We now need to read the orientation quaternion from Varjo Bases log files and enter them as referenceOrientationQuat around line 59 of the configuration. Accessing the Varjo Base log files takes a few steps which we detail below.
Open Diagnostics from the Support menu in Varjo Base

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-061613.png

On the next screen, click Start

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-061911.png

Click “Skip” in order to not record eye tracking data

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-061929.png

You are now presented with the following screen which provides you with a link to the location where the log files are stored. Click on “Show folder”

LP-RESEARCH Knowledge Base > LPVR-DUO-Varjo Setup, Configuration and Calibration > image-20210120-061840.png

Enter the log folder, out of the files named tracking_<number>.log select the latest one and open it. Near its end search for lines similar to this
20210120 15:16:31.156 --- [Varjo::Tracking::PluginUtilityAPI::WriteLogMessage] PLUGIN: [INFO] new pose: q = {"w":0.9913584114738726,"x":0.1302836073262127,"y":0.006246731684853627,"z":-0.013979051208529776} pos = {"x":-1.1893610000000001,"y":0.831757,"z":-0.936706}
Replace the contents of referenceOrientationQuat in the configuration web page with the values of q in the log file, i.e. {"w":0.9913584114738726,"x":0.1302836073262127,"y":0.006246731684853627,"z":-0.013979051208529776} in this example.
After the above procedure the optical marker can be detached from the reference IMU.

NOTE: The reference IMU should now not be moved anymore. The system should now be sufficiently calibrated for operation.

You should now be able to display content fixed to the vehicle coordinate system. E.g. when you drive a curve in a vehicle and keep looking straight ahead, objects in VR/AR should not change their position and orientation.