/
LPPOS Manual

LPPOS Manual

Releases

See a list of LPPOS releases here: https://lp-research.atlassian.net/wiki/spaces/LKB/pages/2735407118

This documentation needs to be updated to the latest version of LPPOS. We’ve added a lot of features and some aren’t reflected in the manual yet.

Introduction

FusionHub is a software application that has the purpose of combining a number of sensor data inputs to create a higher level information output. LPVR-POS is based on FusionHub and combines odometry, GPS and IMU data from a vehicle to calculate high-accuracy and low-latency global localization information. There is a range of typical applications such as automotive and robot localization.

The diagram below shows the general structure of FusionHub. Sources and sinks are connected by a filter unit. The sensor fusion functionality is contained in this filter unit. The filter parameters as well as the parameters of input and output blocks can be configured via a configuration script or a graphical user interface. Note that this image is a general block diagram of FusionHub showing more items than the functionality needed for LPVR-POS.

The graphical user interface is detached from the main FusionHub application and both applications can therefore run on separate computers. This provides flexibility for running FusionHub on devices with limited monitoring capabilities like a head mounted display.

General

Starting FusionHub

FusionHub consists of two components:

  • The main application

  • A graphical user interface application

The main FusionHub application is started by running FusionHub.exe. No specific installation is needed, the application can be run directly out of its deployment directory. It is a command line application that uses the file config.json for its configuration. We will explain the contents and options of the configuration file further below.

Please install the graphical user interface by running lp-fusionhub-dashboard_0.1.0_x64_en-US.msi. It installs lp-fusionhub-dashboard in your start menu, launch the application from there. Press the Connect button after starting FusionHub.exe to connect client and server. In case you are running FusionHub on a separate machine make sure to enter the correct IP address.

The screenshot below shows the connection elements of the GUI dashboard.

Licensing

FusionHub has two options for license protection. The application will only run correctly if one of the two license options is satisfied.

Hardware dongle

License authentication using a hardware dongle; This is especially interesting for air-gapped installations that are not connected to the internet. As long as the dongle is inserted into a USB slot of the host system, FusionHub will run. Please note that for the Android (eg. on Quest 2 HMD) version of FusionHub, the GUI running on the streaming host is dongle protected, see more detailed information in the specific manual chapter.

Online license

License authentication using a software, online license; This makes sense for systems that are connected to the internet at least during the initial installation of FusionHub. The software checks its license status with our license server with following sequence:

  1. Enter license key in configuration file. You receive your personal license key from us.

  2. FusionHub application sends license key and machine code to server.

  3. Server checks if license is valid and returns response code, if it is valid.

  4. Copy the response code from the log and enter it in the config file to the ResponseKey parameter. Save the config file.

  5. This allows FusionHub to run on this specific machine without reconnecting to the internet. One license unit will be subtracted from your license account. Please ask us for assitance if you’d like to move your license.

If your default configuration file config.json doesn't contain it already, add the LicenseInfo block as shown below. Enter your personal key you received from us as LicenseKey.

{ ... "LicenseInfo": { "LicenseKey": "EKKCO-GZYLT-NJKET-SASDC", "ResponseKey": "" } ... }

LPVR-POS Filter Configuration

LPVR-POS combines odometry, GPS and IMU data from a vehicle to calculate high-accuracy and low-latency global localization information. While GPS or RTK-GPS measurements alone provide similar positioning accuracy, the output frequency of these systems is relatively low, making them unsuitable for applications where localization information at higher framerates is required, such as positioning objects in an augmented reality environment.

By additionally using odometry (wheel speeds, steering angle etc.) information, the localization data from the GPS measurements is interpolated to achieve framerates limited only by IMU and odometry sampling speeds.

The LPVR-POS filter has two operation modes with different configuration blocks in config.json and different output formats. The two modes are:

  • Low-dynamics filter (LD)

  • High-dynamics filter (HD)

The diagram below shows a general overview of the LPVR-POS filter setup.

Installation of Hardware Components

Inertial Measurement Unit (IMU)

LPMS-IG1P needs to be installed in the vehicle in a known orientation ideally with the coordinate axes of the IMU arranged in parallel to the vehicle coordinate system. As vehicle reference frame we are using the VW coordinate system shown in the image below.

Connect the USB connector of LPMS-IG1P to the host computer. If needed an active or passive USB extension can be used. Make sure to check data integrity with the LpmsControl 2 data acquisition tool, we have noticed communication issues with some passive USB extensions.

See the appendix for further information how to set up LPMS-IG1.

VW frame x: back y: right z: up
VW coordinate frame

Global Positioning System (GPS)

We provide two options for using a global position reference:

  • The GPS receiver integrated with the LPMS-IG1P sensor; Connect the antenna cable and place the GPS antenna on top of the vehicle

  • Standalone multi-channel RTK GPS receiver module; This requires setting up an RTK-GPS base station. Details are provided in this chapter.

Note on the antenna placement: We recommend placing the GPS antenna on top, in the center of the car.

CAN Bus Connection

FusionHub can be connected to the vehicle CAN bus by using one of the following CAN bus interfaces:

Both LPVR-POS filter types require reading out wheel velocities from the vehicle’s CAN bus. This means we need to connect the CAN interface that’s being used with FusionHub to the vehicle’s CAN lines. This can happen for example through the car’s OBD2 connector, or by directly tapping into its wire harness.

Once access to the CAN bus is available the CAN messages from the car need to be decoded to calculate the correct wheel velocity values. This decoding will in the future be configurable. At the moment we offer decoder configuration presets for a few vehicle types. Please contact us for details.

LPVR-POS Sensor Fusion Filter

Filter Inputs

Both the LD and the HD filter need the following sources as input. One option is to operate the filter with our LPMS-IG1P sensor that contains IMU and a GPS receiver. This allows for standard GPS absolute position accuracy. Relative accuracy and update rate are higher, based on odometry and IMU data. The other option is to use a separate RTK-GPS unit to get high accuracy RTK-GPS readings. We will look at how to set up an RTK-GPS system for LPVR-POS in this chapter.

IMU & GPS Option 1 - LPMS-IG1P Data Source for IMU and GPS Data

LPMS-IG1P Source

LPMS-IG1P combines IMU sensor and GPS data source.

"imuP": { "type": "DualRtk", "settings": { "sensor1": { // If specification needed, insert first IG1 sensor name here //"name": "ig1p232800650050", "autodetectType": "ig1p" }, "rtcm": false, "imuEndpoint": "tcp://*:8802" } }

Parameter name

Description

Default

Parameter name

Description

Default

type

Type of GPS receiver. Currently only DualRtk is allowed.

DualRTK

name

The name of the LPMS-IG1P sensor used in this setup. This parameter is optional. If FusionHub is operated at the same time with LPVR-DUO, we recommend specifying the sensor name. Look up the sensor name in LpmsControl 2.

n/a

autodetectType

Type of sensor to be autodetcted

ig1p

rtcm

Set to true if RTCM input is to be received eg. from an NTRIP source.

false

imuEndpoint

Output endpoint of IMU data. This parameter is optional.

tcp://*:8802

We currently don’t have an elegant way to copy & paste the name of the IG1 sensor to be used by FusionHub to the configuration script from LPMS-Control. Please see the image below where to (manually) copy the name from.

IMU & GPS Option 2 - Separate LPMS-IG1 IMU and RTK GPS Sources (with NTRIP Caster for RTK Correction)

LPMS-IG1 IMU Source

This node connects to an IMU sensor. For LPVR-POS use LPMS-IG1P. Note that this node doesn’t read out GPS data from LPMS-IG1P.

"imu": { "type": "OpenZen", "settings": { "autodetectType": "ig1" } }

Parameter name

Description

Default

Parameter name

Description

Default

type

Type of IMU. At the moment only OpenZen IMUs are supported.

OpenZen

name

The name of the LPMS-IG1 sensor used in this setup. This parameter is optional. If FusionHub is operated at the same time with LPVR-DUO, we recommend specifying the sensor name. Look up the sensor name in LpmsControl 2.

n/a

autodetectType

Type of sensor to be autodetcted

ig1

imuEndpoint

Output endpoint of IMU data. This parameter is optional.

tcp://*:8802

RTCM Source

Node to connect to an NTRIP caster and read in RTCM information from there.

"RTCM": { "type": "NTRIP", "settings": { "host": "some-host-name", "port": "2101", "mountpoint": "some-mount-point", "user": "some-user", "password": "some-password", "userAgent": "LPVR", "initialLatitude": 35.65736, "initialLongitude": 139.73239, "forwardGnss": true } }

Parameter name

Description

Default

Parameter name

Description

Default

type

Type of RTCM correction data source. Currently only NTRIP is allowed.

NTRIP

host

NTrip caster host.

192.168.1.1

port

NTrip caster port.

2101

mountpoint

NTrip mountpoint or stream to receive rtcm correction data.

 

user

NTrip caster username.

 

password

NTrip caster password.

 

userAgent

Name of user agent when connecting to NTrip caster.

LPVR-POS

initialLatitude

Latitude to forward to Ntrip caster on first connect.

0.0

initialLongitude

Longitude to forward to Ntrip caster on first connect.

0.0

forwardGnss

Set true if gnss data from gnss source is to be forwarded to NTRIP caster. This is useful if Ntrip caster offers dynamic switching of RTCM correction data based on forwarded location.

false

Single Antenna GNSS Source

This is the connection to the actual GNSS module such as the ZED-F9P module. The nodes receives data in NMEA format and sends RTK correction information to the node using RTCM.

"gnss": { "type": "NMEA", "settings": { "port": "/dev/ttyUSB0", "baudrate": 115200, "rtcm": true } }

Parameter name

Description

Default

Parameter name

Description

Default

type

Data output format for gnss data source. Currently only NMEA is allowed.

NMEA

port

Serial port number for gnss source.

  • For Linux this parameter needs to be in format /dev/ttyXXX

  • For Windows it is COMxxx

 

baudrate

Serial port baudrate to connect to GNSS source.

 

rtcm

Set true to enable RTCM correction data forwarding from RTCM source to gnss module.

false

Dual Antenna GNSS Source

Some GNSS systems such as the Unicore board in the LPPOS hardware box support dual-antenna setups and therefore deliver orientation information additionally to position data even when standing still. A typical configuration of a the Unicore GNSS module looks like the following.

"gnss": { "dataEndpoint": "inproc://gnss_data", "inputEndpoints": [ "inproc://rtcm_data" ], "settings": { "baudrate": 460800, "port": "COM9", "rtcm": true, "dualGPS": true }, "type": "NMEA" },

Parameter name

Description

Default

Parameter name

Description

Default

type

Data output format for gnss data source. Currently only NMEA is allowed.

NMEA

port

Serial port number for gnss source.

  • For Linux this parameter needs to be in format /dev/ttyXXX

  • For Windows it is COMxxx

 

baudrate

Serial port baudrate to connect to GNSS source.

115200

rtcm

Set true to enable RTCM correction data forwarding from RTCM source to gnss module.

false

dualGPS

If the connected GNSS system is a dual antenna system

true

For details about the overall configuration of the RTK-GPS system refer to this section.

GNSS Output Format

JSON

"gnssData": { "altitude": 0.0, "hdop": 0.0, "heading": 0, "latitude": 0.0, "longitude": 0.0, "height": 0.0, "nSat": 0, "orientation": { "w": 1.0, "x": 0.0, "y": 0.0, "z": 0.0 }, "quality": 0, "sensorName": "", "timestamp": 0, "tmg": 0.0, "undulation": 0.0 }

Protobuf

message GnssData { int64 timestamp = 1; double latitude = 3; double longitude = 4; double period = 5; int32 frame_count = 6; double latency = 8; string sensor_name = 11; Quaternion orientation = 12; double height = 13; double vertical_accuracy = 14; double horizontal_accuracy = 15; int32 quality = 16; int32 n_sat = 17; double hdop = 18; double tmg = 19; double heading = 20; double altitude = 21; double undulation = 22; }

Important Output Data

Parameter name

Description

Unit

Parameter name

Description

Unit

timestamp

Timestamp of current GNSS data

nanoseconds (in case GPS time is used as clock this is the duration since start of the day in UTC time)

longitude

Longitude part of GPS position

degree

latitude

Latitude part of GPS position

degree

height

The height of the receiver above the earth surface reference ellipsoid

m

quality

Quality of the GPS signal:

0: Fix not valid
1: GPS fix
2: Differential GPS fix
3: Not applicable
4: RTK fixed
5: RTK float
6: INS Dead reckoning

n/a (comment: for RTK enabled systems RTK fix provides the highest cm accuracy. If this quality can’t be reached, the number of visible satellites or connection to base station might be an issue.)

Odometry

The vehicle source is the data source that reads raw CAN data messages from a given CAN bus interface and parses the vehicle data using to the provided CAN protocol.

This section explains how to set up the configuration for a custom CAN parsing node. Depending on the available information about the vehicle CAN protocol, there are three types of CAN parsing available.

  • Minimal parsing where we only have information about the vehicle speed CAN message.

  • External separate four wheel speed parsing, velocity sign information may be added if the gear stick reading is available.

  • ExternalWithSteering parsing with steering model information, there are two ways to specify the steering tables depending on the information available: steering wheel angle vs (right or left) wheel angles or steering wheel angle vs average wheel angles. Using this data we can perform linear interpolation to compute the wheel angles for any given steering angle and use it for Ackermann steering.

In most cases, having the overall vehicle speed message or the separate wheel speed messages is more than enough and no steering information is needed. If available, the gear stick reading is useful to help identify the reverse driving.

Minimal Vehicle Parsing

This parser only requires information about the overall vehicle speed message. So it only provide this data as input to the fusion algorithm. Even though this can be enough for some use cases, it may be better to provide separate wheel speeds to extract and use the vehicle angular velocity information as well.

Parameter name

Description

Default

Parameter name

Description

Default

canInterface

CAN bus interface, supported values are "PeakCAN" and "Vector"

PeakCAN

vehicleType

"Minimal"

n/a

velocityCanId

CAN id for the velocities message

n/a

velocityScale

Value to multiply the parsed raw velocity by in order to get velocities in m/s.

1.0

velocityStartBit

Start bit of the vehicle wheel velocity.

0

velocityBitsLength

Number of bits used for wheel velocity encoding.

16

Configuration example
"vehicle": { "type": "Automotive", "vehicleStateEndpoint": "tcp://*:8999", "settings": { "canInterface": "Internal", "canEndpoint": "tcp://localhost:9921", "vehicleType": "Minimal", "canProtocol": { "velocityCanId": 209, "velocityScale": 0.015625012, "endianness": "little", "velocityBitsLength": 16, "velocityStartBit": 0 } } }

Vehicle Parsing with Separate Wheel Velocities

Parameter name

Description

Default

Parameter name

Description

Default

canInterface

CAN bus interface, supported values are "PeakCAN" and "Vector"

PeakCAN

vehicleType

Used to specify the parser type, available values are:

"Minimal", "External" and "ExternalWithSteering"

n/a

wheelBase

Distance between front and back wheels.

n/a

trackWidth

Distance between right and left back wheels.

n/a

velocityCanId

CAN id for the velocities message

n/a

velocityScale

Value to multiply the parsed raw velocities by in order to get velocities in m/s.

1.0

velocityStartBits

Within this parameter, one need to specify the starting bits of each wheel velocity "FrontLeftWheel", "FrontRightWheel", "RearLeftWheel"and "RearRightWheel" .

0, 16, 32, 48

respectively.

velocityBitsLength

Number of bits used for each wheel velocity encoding.

16

endianness

Specify if the CAN data endianness is "little" or "big" .

little

driveModel

When no steering information is available, only Differential model can be used.

Differential

canEndpoint

Output endpoint of Vehicle data. This parameter is optional.

tcp://*:8802

Configuration example
"vehicle": { "type": "Automotive", "vehicleStateEndpoint": "tcp://*:8999", "settings": { "canInterface": "PeakCAN", "canEndpoint": "tcp://localhost:9921", "vehicleType": "External", "wheelBase": 2.750, "trackWidth": 1.595, "canProtocol": { "velocityCanId": 212, "velocityScale": 0.015625012, "endianness": "little", "velocityStartBits": { "FrontLeftWheel": 0, "FrontRightWheel": 16, "RearLeftWheel": 32, "RearRightWheel": 48 } } } }

Additional Parameters

Gear Stick Position for Velocity Sign

If available, the gear stick reading data can be parsed to determine the velocity sign.

Parameter name

Description

Default

Parameter name

Description

Default

gearCanId

Gear stick message CAN id

n/a

gearStartBit

Gear stick message start bit

n/a

gearBitsLength

Number of bits used for gear stick message encoding.

n/a

gearReverseValue

Reverse driving gear stick value.

n/a

Configuration template
"vehicle": { ... "settings": { ... "vehicleType": "External", "canProtocol": { "gearCanId": 1, "gearStartBit": 2, "gearBitslength": 3, "gearReverseValue": 4 } } }

Steering information for Ackermann model

If available, the steering wheel message can also be parsed but using it in the drive model requires input about the vehicle steering model as detailed below. Although we provide this option, using simpler drive models proved to work just fine.

Parameter name

Description

Default

Parameter name

Description

Default

steeringCanId

CAN id for the steering wheel angle message

n/a