Femto Mega IMU convention and sensor synchronization

Hello,

I recently got my Femto Mega and started playing with it. So far it’s going great since the documentation and all the wrappers around it are working seamlessly.

However, I have a few questions and also doubts regarding the calibration and sensor synchronization.

First of all, I need more information about IMU calibration. What’s the axis convention in XYZ? Is it left-front-up, right-back-down…? I couldn’t find any useful documentation explaining this in detail. This is crucial since I would like to use Femto Mega mainly for SLAM applications and I need that information in order to configure my setup.

The desired IMU convention is [0, 0, +9.81] when the sensor is at rest. But I guess the z-axis is pointing downward since the value is -9.81 when at rest. Anyway, I need more clarification on that and how to properly set the extrinsic T_lidar_imu: Transformation from the IMU frame to the depth frame, please.

Also I have two different extrinsics,

  1. when I “ros2 launch orbbec_camera femto_mega.launch.py”
[component_container-1] [INFO] [1753050050.125701645] [camera.camera]: Publishing static transform from accel to depth
[component_container-1] [INFO] [1753050050.125721213] [camera.camera]: Translation -40.9004, -7.90367, 75.1984
[component_container-1] [INFO] [1753050050.125752514] [camera.camera]: Rotation 0.525483, 0.473147, -0.473147, 0.525483
[component_container-1] [INFO] [1753050050.125774296] [camera.camera]: Publishing static transform from gyro to depth
[component_container-1] [INFO] [1753050050.125794080] [camera.camera]: Translation -40.9004, -7.90367, 75.1984
[component_container-1] [INFO] [1753050050.125813019] [camera.camera]: Rotation 0.525483, 0.473147, -0.473147, 0.525483
  1. when I “ros2 topic echo /camera/depth_to_accel”
ros2 topic echo /camera/depth_to_accel
header:
  stamp:
    sec: 0
    nanosec: 0
  frame_id: depth_to_accel_extrinsics
rotation:
- 2.6794900520599185e-08
- 0.10452800244092941
- -0.994521975517273
- -1.0
- 2.8008300123616436e-09
- -2.664810061503431e-08
- -5.551119755337213e-17
- 0.994521975517273
- 0.10452800244092941
translation:
- 0.07561260223388672
- -0.040900394439697264
- 3.528594970703125e-08
---
  • Which one is the correct extrinsics?

The second topic is about sensor synchronization; I get different sensor offsets and I am not completely sure how to use HW sync to maintain streaming consistency. I’ve added my launch file - can you please check that and tell me if I am doing something wrong or if there are any other parameters that could be beneficial?

from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import PushRosNamespace
from launch.actions import GroupAction
from launch_ros.actions import ComposableNodeContainer
from launch_ros.descriptions import ComposableNode
from launch_ros.actions import Node
import os


def generate_launch_description():
    # Declare arguments
    args = [
        DeclareLaunchArgument('camera_name', default_value='camera'),
        DeclareLaunchArgument('depth_registration', default_value='true'),
        DeclareLaunchArgument('serial_number', default_value=''),
        DeclareLaunchArgument('usb_port', default_value=''),
        DeclareLaunchArgument('net_device_ip', default_value=''),
        DeclareLaunchArgument('net_device_port', default_value='0'),
        DeclareLaunchArgument('device_num', default_value='1'),
        DeclareLaunchArgument('vendor_id', default_value='0x2bc5'),
        DeclareLaunchArgument('product_id', default_value=''),
        DeclareLaunchArgument('enable_point_cloud', default_value='true'),
        DeclareLaunchArgument('enable_colored_point_cloud', default_value='true'),
        DeclareLaunchArgument('cloud_frame_id', default_value=''),
        DeclareLaunchArgument('point_cloud_qos', default_value='default'),
        DeclareLaunchArgument('connection_delay', default_value='100'),
        DeclareLaunchArgument('color_width', default_value='1280'),
        DeclareLaunchArgument('color_height', default_value='720'),
        DeclareLaunchArgument('color_fps', default_value='30'),
        DeclareLaunchArgument('color_format', default_value='MJPG'),
        DeclareLaunchArgument('enable_color', default_value='true'),
        DeclareLaunchArgument('flip_color', default_value='false'),
        DeclareLaunchArgument('color_qos', default_value='default'),
        DeclareLaunchArgument('color_camera_info_qos', default_value='default'),
        DeclareLaunchArgument('enable_color_auto_exposure', default_value='true'),
        DeclareLaunchArgument('color_exposure', default_value='-1'),
        DeclareLaunchArgument('color_gain', default_value='-1'),
        DeclareLaunchArgument('enable_color_auto_white_balance', default_value='true'),
        DeclareLaunchArgument('color_white_balance', default_value='-1'),
        DeclareLaunchArgument('depth_width', default_value='640'),
        DeclareLaunchArgument('depth_height', default_value='576'),
        DeclareLaunchArgument('depth_fps', default_value='30'),
        DeclareLaunchArgument('depth_format', default_value='Y16'),
        DeclareLaunchArgument('enable_depth', default_value='true'),
        DeclareLaunchArgument('flip_depth', default_value='false'),
        DeclareLaunchArgument('depth_qos', default_value='default'),
        DeclareLaunchArgument('depth_camera_info_qos', default_value='default'),
        DeclareLaunchArgument('ir_width', default_value='640'),
        DeclareLaunchArgument('ir_height', default_value='576'),
        DeclareLaunchArgument('ir_fps', default_value='30'),
        DeclareLaunchArgument('ir_format', default_value='Y16'),
        DeclareLaunchArgument('enable_ir', default_value='true'),
        DeclareLaunchArgument('flip_ir', default_value='false'),
        DeclareLaunchArgument('ir_qos', default_value='default'),
        DeclareLaunchArgument('ir_camera_info_qos', default_value='default'),
        DeclareLaunchArgument('enable_ir_auto_exposure', default_value='true'),
        DeclareLaunchArgument('ir_exposure', default_value='-1'),
        DeclareLaunchArgument('ir_gain', default_value='-1'),
        DeclareLaunchArgument('enable_sync_output_accel_gyro', default_value='true'),
        DeclareLaunchArgument('enable_accel', default_value='true'),
        DeclareLaunchArgument('accel_rate', default_value='500hz'),
        DeclareLaunchArgument('accel_range', default_value='4g'),
        DeclareLaunchArgument('enable_gyro', default_value='true'),
        DeclareLaunchArgument('gyro_rate', default_value='500hz'),
        DeclareLaunchArgument('gyro_range', default_value='500dps'),
        DeclareLaunchArgument('liner_accel_cov', default_value='0.01'),
        DeclareLaunchArgument('angular_vel_cov', default_value='0.01'),
        DeclareLaunchArgument('publish_tf', default_value='true'),
        DeclareLaunchArgument('tf_publish_rate', default_value='0.0'),
        DeclareLaunchArgument('ir_info_url', default_value=''),
        DeclareLaunchArgument('color_info_url', default_value=''),
        DeclareLaunchArgument('enable_noise_removal_filter', default_value='true'),
        # Network device settings: default net_device_ip is 192.168.1.10 and net_device_port is 8090
        # If you don't want to use ip addr and port, set enumerate_net_device to true
        # and leave net_device_ip blank , it can enumerate automate
        DeclareLaunchArgument('enumerate_net_device', default_value='true'),
        DeclareLaunchArgument('log_level', default_value='none'),
        DeclareLaunchArgument('enable_publish_extrinsic', default_value='true'),
        DeclareLaunchArgument('enable_d2c_viewer', default_value='false'),
        DeclareLaunchArgument('enable_ldp', default_value='true'),
        DeclareLaunchArgument('enable_soft_filter', default_value='true'),
        DeclareLaunchArgument('soft_filter_max_diff', default_value='-1'),
        DeclareLaunchArgument('soft_filter_speckle_size', default_value='-1'),
        DeclareLaunchArgument('sync_mode', default_value='standalone'),
        DeclareLaunchArgument('depth_delay_us', default_value='0'),
        DeclareLaunchArgument('color_delay_us', default_value='0'),
        DeclareLaunchArgument('trigger2image_delay_us', default_value='0'),
        DeclareLaunchArgument('trigger_out_delay_us', default_value='0'),
        DeclareLaunchArgument('trigger_out_enabled', default_value='false'),
        DeclareLaunchArgument('enable_frame_sync', default_value='true'),
        DeclareLaunchArgument('ordered_pc', default_value='false'),
        DeclareLaunchArgument('use_hardware_time', default_value='true'),
        DeclareLaunchArgument('enable_depth_scale', default_value='true'),
        DeclareLaunchArgument('enable_temporal_filter', default_value='true'),
        DeclareLaunchArgument('time_domain', default_value='device'),
        DeclareLaunchArgument('align_mode', default_value='HW'),
        DeclareLaunchArgument('laser_energy_level', default_value='-1'),
        DeclareLaunchArgument('enable_heartbeat', default_value='false'),
    ]

    # Node configuration
    parameters = [{arg.name: LaunchConfiguration(arg.name)} for arg in args]
    # get  ROS_DISTRO
    ros_distro = os.environ["ROS_DISTRO"]
    if ros_distro == "foxy":
        return LaunchDescription(
            args
            + [
                Node(
                    package="orbbec_camera",
                    executable="orbbec_camera_node",
                    name="ob_camera_node",
                    namespace=LaunchConfiguration("camera_name"),
                    parameters=parameters,
                    output="screen",
                )
            ]
        )
    # Define the ComposableNode
    else:
        # Define the ComposableNode
        compose_node = ComposableNode(
            package="orbbec_camera",
            plugin="orbbec_camera::OBCameraNodeDriver",
            name=LaunchConfiguration("camera_name"),
            namespace="",
            parameters=parameters,
        )
        # Define the ComposableNodeContainer
        container = ComposableNodeContainer(
            name="camera_container",
            namespace="",
            package="rclcpp_components",
            executable="component_container",
            composable_node_descriptions=[
                compose_node,
            ],
            output="screen",
        )
        # Launch description
        ld = LaunchDescription(
            args
            + [
                GroupAction(
                    [PushRosNamespace(LaunchConfiguration("camera_name")), container]
                )
            ]
        )
        return ld

Thanks in advance for your support!