Page Comparison

...

FusionHub is a software application that has the purpose of combining a number of sensor data inputs to create a higher level information output. LPVR-POS is based on FusionHub and combines odometry, GPS and IMU data from a vehicle to calculate high-accuracy and low-latency global localization information. Typical applications: Automobile localization, There is a range of typical applications such as automotive and robot localization.

The diagram below shows the general structure of FusionHub. Sources and sinks are connected by a filter unit. The sensor fusion functionality is contained in this filter unit. The filter parameters as well as the parameters of input and output blocks can be configured via a configuration script or the a graphical user interface.

...

• The main application

• A graphical user interface application

Insert the security dongle into a USB port of your computer.

The main FusionHub application is started by running FusionHub.exe. No specific installation is needed, the application can be run directly out of its deployment directory. It is a command line application that uses the file config.json for its configuration. We will explan explain the contents and options of the configuration file further below.

...

FusionHub has two options for license protection:. The application will only run correctly of one of the two license options is satisfied.

Hardware dongle

License authentication using a hardware dongle; This is especially interesting for air-gapped installations that are not connected to the internet. As long as the dongle is inserted into a USB slot of the host system, FusionHub will run. Please note that for the Android (eg. on Quest 2 HMD) version of FusionHub, the GUI running on the streaming host is dongle protected, see more detailed information in the specific manual chapter.

...

1. Enter license key in configuration file. You receive your personal license key from us.

2. Send FusionHub application sends license key and machine code to server.

3. Server checks if license is valid and returns response code, if it is valid.

4. Copy the response code from the log and enter it in the config file to the ResponseKey parameter. Save the config file.

5. This allows FusionHub to run on this specific machine without reconnecting to the internet. One license unit will be subtracted from your license account. Please ask us for assitance if you’d like to move your license.

...

LPVR-POS combines odometry, GPS and IMU data from a vehicle to calculate high-accuracy and low-latency global localization information. While GPS or RTK-GPS measurements alone provide similar positioning accuracy, the output frequency of these systems is relatively low, making them unsuitable for applications where localization information at higher framerates is required, such as positioning objects in an augmented reality environment.

...

• Low-dynamics filter (LD)

• High-dynamics filter (HD)

The diagram below shows an a general overview of a simple the LPVR-POS filter setup.

...

Installation of Hardware Components

...

LPMS-IG1P needs to be installed in the vehicle in a known orientation ideally with the coordinate axes of the IMU arranged in parallel to the vehicle coordinate system. As vehicle reference frame we are using the VW coordinate system shown in the image below.

Connect the USB connector of LPMS-IG1P to the host computer. If needed an active or passive USB extension can be used. Make sure to check data integrity with the LpmsControl 2 data acquisition tool, we have noticed communication issues with some passive USB extensions.

...

Global Positioning System (GPS)

We provide two options of for using a global position reference:

• The GPS receiver is integrated with the LPMS-IG1P sensor. ; Connect the antenna cable and place the GPS antenna on top of the vehicle

• Standalone multi-channel RTK GPS receiver module. ; This requires setting up an RTK-GPS basestationbase station. Details are provided in this chapter.

Notes Note on the antenna placement: We recommend placing the GPS antenna on top, in the center of the car.

...

• PCAN USB interface

• Vector CAN USB interface

Both LPVR-POS filters filter types require reading out wheel velocities from the vehicle’s CAN bus. This means we need to connect the CAN interface that’s being used with FusionHub to the vehicle’s CAN lines. This can happen for example through the car’s OBD2 connector, or by directly tapping into the car’s its wire harness.

Once access to the CAN bus is available the CAN messages from the car need to be decoded to calculate the correct wheel velocity values. This decoding will in the future be configurable. At the moment we offer decoder configuration presets for a few vehicle types. Please contact us for details.

...

Both the LD and the HD filter need the following sources as input. One option is to operate the filter with our LPMS-IG1P sensor that contains IMU and a GPS receiver. This allows for standard GPS absolute position accuracy. Relative accuracy and update rate are higher, based on odometry and IMU data. The other option is to use a separate RTK-GPS unit to get high accuracy RTK-GPS readings. We will look at how to set up an RTK-GPS system for LPVR-POS in a following this chapter.

IMU & GPS Option 1 - LPMS-IG1P

...

Data Source for IMU and GPS

...

Data

LPMS-IG1P Source

...

LPMS-IG1P combines IMU sensor and GPS data source.

"imuP": {
    "type": "DualRtk",
    "settings": {
        "sensor1": {
            // If specification needed, insert first IG1 sensor name here
            //"name": "ig1p232800650050",
            "autodetectType": "ig1p"
        },
        "rtcm": false,
        "imuEndpoint": "tcp://*:8802"
    }
}

...

...

IMU & GPS Option 2 - Separate LPMS-IG1 IMU and RTK GPS

...

Sources (with NTRIP

...

Caster for RTK

...

Correction)

LPMS-IG1 Source

This node connects to an IMU sensor. For LPVR-POS use LPMS-IG1P. Note that this node doesn’t read out GPS data from LPMS-IG1P.

"imu": {
  "type": "OpenZen",
  "settings": {
    "autodetectType": "ig1"
  }
}

RTCM Source

Node to connect to an NTRIP caster and read in RTCM information from there.

"RTCM": {
    "type": "NTRIP",
    "settings": {
        "host": "some-host-name",
        "port": "2101",
        "mountpoint": "some-mount-point",
        "user": "some-user",
        "password": "some-password",
        "userAgent": "LPVR",
        "initialLatitude": 35.65736,
        "initialLongitude": 139.73239,
        "forwardGnss":  true
    }
}

GNSS Source

This is the connection to the actual GNSS module such as the ZED-F9P module. The nodes receives data in NMEA format and sends RTK correction information to the node using RTCM.

"gnss": {
    "type": "NMEA",
    "settings": {
        "port": "/dev/ttyUSB0",
        "baudrate": 115200,
        "rtcm": true
    }
}

Odometry

CAN

...

Bus and Vehicle Decoder Source

This node receives and decodes data from the vehcile’s CAN bus. The output from this node is forwarded to the filter node.

"vehicle": {
    "type": "Automotive",
    "vehicleStateEndpoint": "tcp://*:8999",
    "settings": {
        "canInterface": "PeakCAN",
        "vehicleType": "R56"
    }
}

Fusion Filter Option 1 - Low-dynamics Filter

General

The low dynamics (LD) filter combines odometry, GPS and IMU data to calculate the global position of a vehicle. This filter was initially concipated to work only with odometry and GPS data to calculate the 2D position and yaw angle of a vehicle. It is therefore suitable for simple tracking scenarios. For more complicated, for example augmented reality, applications we recommed using the HD filter as it outputs globally referenced 3D orientation additionally to globally referenced position. This filter relies heavily on the wheel velocity data of the car published on the car’s CAN bus. Therefore the quality of the output of this filter depends on the accuracy of this information.

...

The LD filter outputs fusedVehiclePose. The FusedVehiclePose fusedVehiclePose contains a 3D acceleration vector. The acceleration is defined in the following manner: There's a configuration flag imuToCarRotation which takes a quaternion used to rotate vectors in the IMU frame to the car frame. By default it is the identity quaternion. For the LD model, the measured IMU acceleration is simply rotated by the imuToCarRotation and written to the output.

...

Insert the following configuration block into your config.json file in the sinks section to activate the LD filter. The filter's node name is vehicularFusion.

..
  "sinks": {
    ..,
    "vehicularFusion": {
        "echoFusedPose": false,
        "endpoint": "tcp://*:8801",
        "fuser": {
            "fitModel": "SimpleCarModel",
            "driveModel": "Differential",
            "velError": 0.277777778,
            "omegaError": 0.5,
            "measurementError": 0.1,
            "smoothFit": true,
            "useImuTurnRate": true,
            "imuTurnRateAxis": {
                "x": 1,
                "y": 0,
                "z": 0
            },
            "imuToCarRotation": {
                "w": 1,
                "x": 0,
                "y": 0,
                "z": 0,
            "z": 0}
        }
    }
}..

See below a description of the parameter options of the LD filter.

...

Fusion Filter Option 2 - High-Dynamics Filter

General

The high dynamics filter combines IMU and GPS data to calculate the global position of a vehicle. Instead of using the odometry it uses IMU data to determine the orientation changes of a car on the X, Y and Z axis. The direction of the vehicle is globally referenced from the GPS system. For increasing the direction reference quality a dual-antenna GPS system can be used.

The high dynamics filter works well for scenarios with agressive driving maneuvers such as drifting and cornering. During such maneuvers the turning motion of the wheels generally doesn’t directly correspond with the direction of the vehicle. Therefore for this filter don’t rely on wheel velocity measurements. This filter uses information from the wheels only to determine if the car has come to a full stop.

As the filter relies heavily on GPS measurements it doesn’t deliver good results indoors. The better the GPS reception, the better the resulting output of the filter. In it’s current state it therefore only works outdoors. In a future version that combines LD and HD filters, this issue will be resolved.

...

Insert the following configuration block into your config.json file in the sinks section to activate the HD filter. The filter's node name is gnssImuFusion.

..
  "sinks": {
    ..,
    "gnssImuFusion": {
        "echoFusedPose": false,
        "endpoint": "tcp://*:8803",
        "fuser": {
            "fitModel": "ModelGnssImu",
            "accelError": 0.01,
            "omegaError": 0.02,
            "measurementError": 0.05,
            "imuToCarRotation": {
                "w": 1,
                "x": 0,
                "y": 0,
                "z": 0
            }
        }
    }
..

See below a description of the parameter options for the HD filter.

...

The WebSocket server can be accessed via 19358 port on the machine hosting the FusionHub service. To accelerate development download the Simple WebSocket Client Chrome plugin. This allows you to manually enter API commands and check the replies from the server.

{
  "command": "getConfig"
}

{
  "command": "getSavedConfig"
}

{
  "command": "saveConfig"
}

{
  "command": "setConfig",
  "data": {
    "sources": {
      "optical": {
        "settings": {
          "port": 5005
        }
      }
    }
  }
}

{
  "command": "overwriteConfig",
  "data": {
    "settings": { ... },
    "sources": { ... },
    "sinks": { ... }
  }
}

{
  "command": "getIntercalibrationStatus"
}

{
  "command": "applyIntercalibrationResults"
}

{
  "command": "restartBackend"
}

{
    "command": "startRecording",
    "data": {
        "endpoints": ["inproc://imu_data_source_1"],
        "format": "json",
        "filename": "subaruDrive"
    }
}

{
    "command": "stopRecording",
}

{
    "command": "listRecording",
}

{
  "command": "getVersion"
}

{
  "command": "getVersion",
  "status": "ok",
  "data": {
    "version": "1.0.0"
  }
}

Sending FusionHub Data to External Applications via the ZeroMQ Interface

...

As low level protocol to emit the output data we use ZeroMQ (publisher / subscriber). The data itself is in JSON format and is encoded as Protocol Buffers. Protocol Buffers are documented here. Message Messages are defined in the Protobuf (.protoc) format as defined in the file stream_data.proto. This file is contained in the installation folder of FusionHub.

...

The correction data is sent from the base station to the rover as so-called RTCM data packets, that are sent using the NTRIP protocol. Base and rover are connected via a caster service, that relays the data from the base to the rover.

...

Note: These instructions work for LPMS-IG1 (IMU only) and LPMS-IG1P (IMU + GPS).

• First, download LpmsControl 2 from here and install it.

• Connect LPMS-IG1(P) to your computer and start LpmsControl 2.

• In LpmsControl 2 select one of the LPMS-IG1(P) sensors and connect to it.

...

• In case the sensor is in VCP (virtual COM port) mode as shown below, click on Convert to switch the sensor to USBxpress mode. This is required for communication with FusionHub.

...

• After converting the sensor to USBxpress mode it should be displayed as such.

...

• The image below shows typical output from LPMS-IG1(P) after connecting.

...

• Close LpmsControl 2 to disconnect from the sensor. You are now ready to use LPMS-IG1(P) in FusionHub.