...

Code Block

"imuP": {
    "type": "DualRtk",
    "settings": {
        "sensor1": {
            // If specification needed, insert first IG1 sensor name here
            //"name": "ig1p232800650050",
            "autodetectType": "ig1p"
        },
        "rtcm": false,
        "imuEndpoint": "tcp://*:8802"
    }
}

Parameter name	Description	Default
type	Type of GPS receiver. Currently only `DualRtk` is allowed.	DualRTK
name	The name of the LPMS-IG1P sensor used in this setup. This parameter is optional. If FusionHub is operated at the same time with LPVR-DUO, we recommend specifying the sensor name. Look up the sensor name in LpmsControl 2.	n/a
autodetectType	Type of sensor to be autodetcted	ig1p
rtcm	Set to true if RTCM input is to be received eg. from an NTRIP source.	false
imuEndpoint	Output endpoint of IMU data. This parameter is optional.	tcp://*:8802

Info
We currently don’t have an elegant way to copy & paste the name of the IG1 sensor to be used by FusionHub to the configuration script from LPMS-Control. Please see the image below where to (manually) copy the name from.

...

Code Block
"imu": { "type": "OpenZen", "settings": { "autodetectType": "ig1" } }

Parameter name	Description	Default
type	Type of IMU. At the moment only OpenZen IMUs are supported.	OpenZen
name	The name of the LPMS-IG1 sensor used in this setup. This parameter is optional. If FusionHub is operated at the same time with LPVR-DUO, we recommend specifying the sensor name. Look up the sensor name in LpmsControl 2.	n/a
autodetectType	Type of sensor to be autodetcted	ig1
imuEndpoint	Output endpoint of IMU data. This parameter is optional.	tcp://*:8802

RTCM Source

Node to connect to an NTRIP caster and read in RTCM information from there.

Code Block

"RTCM": {
    "type": "NTRIP",
    "settings": {
        "host": "some-host-name",
        "port": "2101",
        "mountpoint": "some-mount-point",
        "user": "some-user",
        "password": "some-password",
        "userAgent": "LPVR",
        "initialLatitude": 35.65736,
        "initialLongitude": 139.73239,
        "forwardGnss":  true
    }
}

Parameter name	Description	Default
type	Type of RTCM correction data source. Currently only `NTRIP` is allowed.	NTRIP
host	NTrip caster host.	192.168.1.1
port	NTrip caster port.	2101
mountpoint	NTrip mountpoint or stream to receive rtcm correction data.
user	NTrip caster username.
password	NTrip caster password.
userAgent	Name of user agent when connecting to NTrip caster.	LPVR-POS
initialLatitude	Latitude to forward to Ntrip caster on first connect.	0.0
initialLongitude	Longitude to forward to Ntrip caster on first connect.	0.0
forwardGnss	Set true if gnss data from gnss source is to be forwarded to NTRIP caster. This is useful if Ntrip caster offers dynamic switching of RTCM correction data based on forwarded location.	false

GNSS Source

This is the connection to the actual GNSS module such as the ZED-F9P module. The nodes receives data in NMEA format and sends RTK correction information to the node using RTCM.

Code Block
"gnss": { "type": "NMEA", "settings": { "port": "/dev/ttyUSB0", "baudrate": 115200, "rtcm": true } }

Parameter name	Description	Default
type	Data output format for gnss data source. Currently only `NMEA` is allowed.	NMEA
port	Serial port number for gnss source.
baudrate	Serial port baudrate to connect to GNSS source. For Linux this parameter needs to be in format `/dev/ttyXXX` For Windows it is `COMxxx`	/dev/ttyUSB0
rtcm	Set true to enable RTCM correction data forwarding from RTCM source to gnss module.	false

Odometry

CAN Bus and Vehicle Decoder Source

This node receives and decodes data from the vehcile’s CAN bus. The output from this node is forwarded to the filter nodeThe vehicle source is the data source that reads raw CAN data messages from a given CAN bus interface and parses the vehicle data using to the provided CAN protocol.

This section explains how to set up the configuration for a custom CAN parsing node. Depending on the available information about the vehicle CAN protocol, there are three types of CAN parsing available.

Minimal parsing where we only have information about the vehicle speed CAN message.
Separate four wheel speed parsing, velocity sign information may be added if the gear stick reading is available.
Parsing with steering model information, there are two ways to specify the steering tables depending on the information available: steering wheel angle vs (right or left) wheel angles or steering wheel angle vs average wheel angles. Using this data we can perform linear interpolation to compute the wheel angles for any given steering angle and use it for Ackermann steering.

Minimal vehicle parsing

This parser only requires information about the overall vehicle speed message. So it only provide this data as input to the fusion algorithm. Even though this can be enough for some use cases, it may be better to provide separate wheel speeds to extract and use the vehicle angular velocity information as well.

Parameter name	Description	Default
canInterface	CAN bus interface, supported values are `"PeakCAN"` and `"Vector"`	PeakCAN
vehicleType	`"Minimal"`	n/a
velocityCanId	CAN id for the velocities message	n/a
velocityScale	Value to multiply the parsed raw velocity by in order to get velocities in m/s.	1.0
velocityStartBit	Start bit of the vehicle wheel velocity.	0
velocityBitsLength	Number of bits used for wheel velocity encoding.	16

Configuration example

Code Block

    "vehicle": {
      "type": "Automotive",
      "vehicleStateEndpoint": "tcp://*:8999",
      "settings": {
        "canInterface": "Internal",
        "canEndpoint": "tcp://localhost:9921",        
        "vehicleType": "Minimal",
        "canProtocol": {
          "velocityCanId": 209,
          "velocityScale": 0.015625012,
          "endianness": "little",
          "velocityBitsLength": 16,
          "velocityStartBit": 0
        }
      }
    }

Vehicle parsing with separate wheel velocities

Parameter name	Description	Default
canInterface	CAN bus interface, supported values are `"PeakCAN"` and `"Vector"`	PeakCAN
vehicleType	Used to specify the parser type, available values are: `"Minimal", "External"` and `"ExternalWithSteering"`	n/a
wheelBase	Distance between front and back wheels.	n/a
trackWidth	Distance between right and left back wheels.	n/a
velocityCanId	CAN id for the velocities message	n/a
velocityScale	Value to multiply the parsed raw velocities by in order to get velocities in m/s.	1.0
velocityStartBits	Within this parameter, one need to specify the starting bits of each wheel velocity `"FrontLeftWheel", "FrontRightWheel", "RearLeftWheel"and "RearRightWheel"` .	0, 16, 32, 48 respectively.
velocityBitsLength	Number of bits used for each wheel velocity encoding.	16
endianness	Specify if the CAN data endianness is `"little"` or `"big"` .	little
driveModel	When no steering information is available, only `Differential` model can be used.	Differential
canEndpoint	Output endpoint of Vehicle data. This parameter is optional.	tcp://*:8802

Configuration example

Code Block

    "vehicle": {
      "type": "Automotive",
      "vehicleStateEndpoint": "tcp://*:8999",
      "settings": {
        "canInterface": "Internal",
        "canEndpoint": "tcp://localhost:9921",
        "vehicleType": "External",
        "wheelBase": 2.750,
        "trackWidth": 1.595,
        "canProtocol": {
          "velocityCanId": 212,
          "velocityScale": 0.015625012,
          "endianness": "little",
          "velocityStartBits": {
            "FrontLeftWheel": 0,
            "FrontRightWheel": 16,
            "RearLeftWheel": 32,
            "RearRightWheel": 48
          }  
        }
      }
    }

Additional parameters

Gear stick position for velocity sign

If available, the gear stick reading data can be parsed to determine the velocity sign.

Parameter name	Description	Default
gearCanId	gear stick message CAN id	n/a
gearStartBit	gear stick message start bit	n/a
gearBitsLength	Number of bits used for gear stick message encoding.	n/a
gearReverseValue	Reverse driving gear stick value.	n/a

Configuration template

Code Block
"vehicle": { ... "settings": { ... "vehicleType": "External", ... "canProtocol": { ... }, "gearCanId": 1, "gearStartBit": 2, "gearBitslength": 3, "gearReverseValue": 4 } } }

Steering information for Ackermann model

If available, the steering wheel message can also be parsed but using it in the drive model requires input about the vehicle steering model as detailed below.

Parameter name	Description	Default
steeringCanId	CAN id for the steering wheel angle message	n/a
steeringScale	Value to multiply the parsed raw steering wheel angle by in order to get angle in degrees.	1.0
steeringWheelAngles	The steering table column containing sample steering wheel angles.	n/a
wheelAngleType	Specification of provided `wheelAngles` type, valid options are "Right", "Left" or "Average"	n/a
wheelAngles	Steering table column containing corresponding wheel angle values.	n/a
driveModel	In addition to `Differential` model, `Ackermann` or `SimpleAckermann` can be used. the latter is a version of Ackermann steering that uses the average wheel angles.	Differential

Configuration template

If the input steering table is like follows (note that the input table should contain values for both right and left turns aka: positive and negative steering wheel angles),

steering wheel angles	right wheel angles
-360	-30.0
-340	-25.5
…	…
340	35.5
360	40.0

The steeringTable configuration block would look something like below.

Code Block

    "vehicle": {
    ...
      "settings": {
      ...       
        "vehicleType": "ExternalWithSteering",
        ...
        "canProtocol": {
        ...
          },
        "steeringTable": {
            "steeringWheelAngles": [
              -360, -340, -320, -300,
              ...
              300, 320, 340, 360],
          "wheelAngleType": "Right",
          "wheelAngles": [
            -30.0, -25.5, -20.0, -15.0,
            ...
            19.0, 26.0, 35.5, 40.0]
        }
        }
      }
    }

Predefined Vehicle Parsing

For a small number of test vehicles we created a “hardcoded” parser to translate the CAN bus data of these vehicles to input values for our sensor fusion. This predefined parsing is now mostly obsolete because of our new flexible parser. We’ll put the definition of this parser here for completeness.

Code Block
"vehicle": { "type": "Automotive", "vehicleStateEndpoint": "tcp://*:8999", "settings": { "canInterface": "PeakCAN", "vehicleType": "R56" } }

Parameter name	Description	Default
type	Type of vehicle. Currently only `Automotive` allowed.	Automotive
vehicleStateEndpoint	Endpoint for vehicle state output	tcp://*:8999
canInterface	CAN interface used for readin odometry data. Allowed options: PeakCAN Vector	PeakCAN
vehicleType	Type of vehicle. Currently supported vehicles have to be manually added. Contact us for details.	R56 (BMW Mini)

Vehicle source output data types

The vehicle source published the raw CANData it reads from the CAN bus in addition to VehicleState and VehicleSpeed message. The latter is used as input to the sensor fusion algorithms.

Fusion Filter Option 1 - Low-dynamics Filter

...

See below a description of the parameter options of the LD filter.

Parameter name	Description	Default
echoFusedPose	fusedVehiclePose output is printed to command line	false
endpoint	Output port for the fusion result	8801
fitModel	Model to use for fusion. At the moment only `SimpleCarModel` is supported.	SimpleCarModel
driveModel	Model used to calculate the car trajectory from CAN bus data. If the steering wheel data and steering model are provided, `Ackermann` model can be used.	Differential
velError	Velocity error for Kalman filter. Keep default value.	0.277777778
omegaError	Omega error for Kalman filter. Keep default value.	0.5
measurementError	Measurement error for Kalman filter. Keep default value.	0.1
smoothFit	Enable this option to prevent filter output from jumping between odometry data and GPS measurement. Keep enabled.	true
useImuTurnRate	If enabled the IMU turn rate is used instead of the wheel velocity based turn rate. Recommended.	false
imuTurnRateAxis	The IMU axis to use for the Turn rate if `useImuTurnRate` is enabled.	1, 0, 0
imuToCarRotation	Rotation that is applied to accelerometer data from IMU before output	1, 0, 0, 0

Output Format

See a technical description of FusionHub’s communication interface in one of the following chapters.

...

Code Block

syntax = "proto3";

package Fusion.proto;

message Vector2 {
    double x = 2;
    double y = 3;
}

message Vector {
    double x = 2;
    double y = 3;
    double z = 4;
}

message FusedVehiclePose {
    int64 timestamp = 1;
    Vector2 position = 2;
    Vector2 global_position = 3;
    double yaw = 4;
    string utm_zone = 5;
    int64 timecode = 6; // Optional: if 0 not set.    
    Vector acceleration = 7;
}

message StreamData {
    int32 sequence_number = 1;
    FusedVehiclePose fused_vehicle_pose = 9;
}

Parameter name	Description	Unit
acceleration	3D acceleration vector as measured by IMU. Describes the orientation of the vehicle in the vehicle coordinate system.	m/s^2
globalPosition	Longitude and latitude in degrees	degrees
lastDataTime	Unused	s
position	Position relative to starting point with X pointing North and Y pointing East in the current UTM frame	m
timestamp	Timestamp of data acquisition	ns
utmZone	UTM zone	UTM string
yaw	Globally referenced yaw angle	rad

Info
After starting FusionHub, while the car is static, the filter will not deliver a correct yaw angle. The angle will be adjusted to the correct direction after a few seconds of driving the vehicle. The exact output data format is described below.

...

The used car frame is the Volkswagen (VW) coordinate frame convention:

Code Block
VW frame x: back y: right z: up

Image Modified

The IMU sensor can be mounted in any way but the ImuToCarRotation quaternion needs to be provided to transform the IMU data into VW frame. For example, if the IMU is mounted like follows:

...

See below a description of the parameter options for the HD filter.

Parameter name	Description	Default
echoFusedPose	`fusedVehiclePose` output is printed to command line	false
endpoint	Output port for the fusion result	8801
fitModel	Model to use for fusion.	ModelGnssImu
accelError	Acceleration error for Kalman filter. Keep default value.	0.01
omegaError	Omega error for Kalman filter. Keep default value.	0.02
measurementError	Measurement error for Kalman filter. Keep default value.	0.05
imuToCarRotation	Orientation quaternion of IMU relative to car frame	1, 0, 0, 0
smoothFit	Enable this option to prevent filter output from jumping between IMU data and GPS measurement. Keep enabled.	true

Output Data Format

FusedVehiclePose

...

Code Block

syntax = "proto3";

package Fusion.proto;

message Vector2 {
  double x = 2;
  double y = 3;
}

message Vector {
  double x = 2;
  double y = 3;
  double z = 4;
}

message FusedVehiclePose {
  int64 timestamp = 1;
  Vector2 position = 2;
  Vector2 global_position = 3;
  double yaw = 4;
  string utm_zone = 5;
  int64 timecode = 6; // Optional: if 0 not set.
  Vector acceleration = 7;
}

message StreamData {
  int32 sequence_number = 1;
  FusedVehiclePose fused_vehicle_pose = 9;
}

Parameter name	Description	Unit
acceleration	3D acceleration vector as measured by IMU. Describes the orientation of the vehicle.	m/s^2
globalPosition	Longitude and latitude in degrees	degrees
lastDataTime	Unused	s
position	Position within UTM zone	m
timestamp	Timestamp of data acquisition	ns
utmZone	UTM zone	UTM string
yaw	Globally referenced yaw angle	rad

FusedPose

JSON

Code Block
{ "fusedPose": { "lastDataTime": { "timestamp": 0 }, "orientation": { "w": 1.0, "x": 0.0, "y": 0.0, "z": 0.0 }, "position": { "x": 0.0, "y": 0.0, "z": 0.0 }, "timestamp": { "timestamp": 0 } } }

...

Code Block

syntax = "proto3";

package Fusion.proto;

message Quaternion {
  double w = 1;
  double x = 2;
  double y = 3;
  double z = 4;
}

message Vector {
  double x = 2;
  double y = 3;
  double z = 4;
}

message FusedPose {
  int64 timestamp = 1;
  Vector position = 2;
  Quaternion orientation = 3;
  Vector angular_velocity = 4;
  int64 timecode = 5; // Optional: if 0 not set.  
}

message StreamData {
  int32 sequence_number = 1;
  FusedPose fused_pose = 4;
}

Parameter name	Description	Unit
lastDataTime	Unused	s
orientation	Orientation quaternion in ENU coordinate frame	without unit
position	X, y position + height	m
timestamp	Time of data acqusition	ns

GlobalFusedPose

JSON

Code Block
{ "globalFusedPose" { "orientation": { "w": 1.0, "x": 0.0, "y": 0.0, "z": 0.0 }, "timestamp": { "timestamp": 0 }, "position": { "longitude": 0.0, "latitude": 0.0, "height": 0.0 } } }

...

Code Block

syntax = "proto3";

package Fusion.proto;

message Quaternion {
  double w = 1;
  double x = 2;
  double y = 3;
  double z = 4;
}

message GpsPoint {
  double longitude = 2;
  double latitude = 3;
  double height = 4;
}

message GlobalFusedPose {
  int64 timecode = 5; // Optional: if 0 not set.
  int64 timestamp = 1;
  GpsPoint position = 2;
  Quaternion orientation = 3;
}

message StreamData {
  int32 sequence_number = 1;
  GlobalFusedPose global_fused_pose = 10;
}

Parameter name	Description	Unit
orientation	Orientation quaternion	without unit
position	Longitude, latitude, height	deg, deg, m
timestamp	Time of data acqusition	ns

Recording and Replaying Data

...

Code Block
"record": { "filename": "driveData.json", "format": "json" }

Parameter name	Description	Default
filename	Filename of the file to be recorded.	driveData.json
format	The file format. At the moment only JSON is possible.	json

Data Replay

In order to replay data from a JSON file a separate application ReplayExecutable.exe that we deliver with LPVR-POS is used. The replay executable reads the JSON data from a defined file, pushes the data to a replay queue and sends them to the network (tcp://localhost:9921 by default). To run the ReplayExecutable use the following command:

Code Block
ReplayExecutable.exe -r <path/to/file.json> [--replay-speed 1] [--queue-size 100] [--echo-data] [--verbose]

Key	Description	Type	Example value
-r	Path to read in file	String	“log.json”
--replay-speed	Speed to the actual recording	Double	1
--queue-size	The size of queue that file reader would stop pushing new data to the replay queue. Increase this value when you see lots of data is published at the same time when running with `--verbose`	Integer	100
--echo-data	Listen to the publishing endpoint and display the replayed data	N/A	N/A
--verbose	Print the debugging information, i.e., the timestamp a packet is added to the replay queue, replayed from the replay queue, and discarded from the replay queue.	N/A	N/A

FusionHub can receive data from the replay application by adding the replay application’s endpoint source to the configuration file:

...

The WebSocket server can be accessed via 19358 port on the machine hosting the FusionHub service. To accelerate development download the Simple WebSocket Client Chrome plugin. This allows you to manually enter API commands and check the replies from the server.

Endpoint

Sample Requests

Sample Response / Description

getConfig

Code Block

language	json

{
  "command": "getConfig"
}

Get in memory configurations.

getSavedConfig

Code Block

language	json

{
  "command": "getSavedConfig"
}

Get on disk configurations.

saveConfig

Code Block

language	json

{
  "command": "saveConfig"
}

Save the in-memory configurations to the disk.

setConfig

Code Block

language	json

{
  "command": "setConfig",
  "data": {
    "sources": {
      "optical": {
        "settings": {
          "port": 5005
        }
      }
    }
  }
}

Update in-memory configurations. This api create new key-value pairs, or update the existing values.

It does not save configurations to the disk.

Note that in "data" you just need to specify the path to the json key to update: the exmaple on the right would change the port to 5005 while everything else is left unchanged.

overwriteConfig

Code Block

language	json

{
  "command": "overwriteConfig",
  "data": {
    "settings": { ... },
    "sources": { ... },
    "sinks": { ... }
  }
}

Overwrite the in-memory configurations. This is suitable when user want to remove a key from the configuration.

getIntercalibrationStatus

Code Block

language	json

{
  "command": "getIntercalibrationStatus"
}

Get the current intercalibration status. Useful for refetching current status when the frontnend accidentally disconnects.

applyIntercalibrationResults

Code Block

language	json

{
  "command": "applyIntercalibrationResults"
}

Apply the current intercalibration quaternion to the in-memory copy of config. This does NOT save to disk.

restartBackend

Code Block
{ "command": "restartBackend" }

Restart the backend. Internally the while loop reset the DataBlock, causing all sources and sinks to be freed from memory, and instantiate them again.

startRecording

Code Block
{ "command": "startRecording", "data": { "endpoints": ["inproc://imu_data_source_1"], "format": "json", "filename": "subaruDrive" } }

Listen to data published to endpoints, and write to a file of YYYYMMDD-HHMMSS-{comment}.{format}

stopRecording

Code Block
{ "command": "stopRecording", }

Stop the current recording.

listRecording

Code Block
{ "command": "listRecording", }

List the recorded filenames since the FusionHub booted up.

getVersion

Code Block

language	json

{
  "command": "getVersion"
}

Code Block

language	json

{
  "command": "getVersion",
  "status": "ok",
  "data": {
    "version": "1.0.0"
  }
}

Sending FusionHub Data to External Applications via the ZeroMQ Interface

...

Info
Note that both, the base station as well as the vehicle need to be connected to the internet in order to make communication between the two work. You can directly insert a SIM card into the Emlid RS2+ base station that we optionally provide with LPVR-POS.

Components

Image Modified

...

In order to set up an RTK GPS system to work with LPVR-POS follow the below steps. You need to do these steps only once, after that the system should be ready to work when you turn it on:

...

Version	Old Version 16	New Version 17
Changes made by	Klaus Petersen	Klaus Petersen
Saved on	Sept 17, 2023	Sept 20, 2023

Page Comparison

Versions Compared

Key

RTCM Source

GNSS Source

Odometry

CAN Bus and Vehicle Decoder Source

Minimal vehicle parsing

Configuration example

Vehicle parsing with separate wheel velocities

Configuration example

Additional parameters

Gear stick position for velocity sign

Configuration template

Steering information for Ackermann model

Configuration template

Predefined Vehicle Parsing

Vehicle source output data types

Fusion Filter Option 1 - Low-dynamics Filter

Output Format

Output Data Format

FusedVehiclePose

FusedPose

JSON

GlobalFusedPose

JSON

Recording and Replaying Data

Data Replay

Sending FusionHub Data to External Applications via the ZeroMQ Interface

Components