ROSOrin Pro Explained: The Full ROS 2 AI-Agent Pipeline — OpenClaw, SLAM, YOLO, Nav2, and MoveIt2

Tell the robot "Bring me the red block on the table," and it drives to the table, finds the block, picks it up with the 6-DOF arm, and sets it down where you asked. That loop is not a single algorithm. Voice I/O, slam_toolbox laser SLAM, Nav2, OpenCV color detection, and arm kinematics each run as their own ROS 2 services. This article walks the official ROSOrin Pro (Jetson Orin NX) source tree and docs, one module at a time.

1. System function overview

Hiwonder ROSOrin Pro is a ROS 2 robot car built around NVIDIA Jetson Orin NX. The kit includes a depth camera, LiDAR, an AI voice box, and a 6-DOF robotic arm. A spoken command drives the full loop: understand -> navigate to the target -> recognize the block -> grasp -> place.

The main computer is a configuration option. Kits can ship with Jetson Orin Nano, Orin NX, or Orin AGX. This teardown uses the Jetson Orin NX 8 GB variant.

2. Hardware architecture

ROSOrin Pro uses a dual-controller design: Jetson Orin NX + STM32. Orin NX runs the high-level stack (SLAM, YOLO, navigation, language). STM32 runs the real-time loop (motors, bus servos, raw sensors). Perception devices attach over USB to the Jetson. Actuators are driven by the STM32.

Jetson Orin NX takes the depth camera, LiDAR, and microphone array over USB, and talks to the STM32 through USB-UART (/dev/nrc, 1 Mbps). The STM32 drives chassis motors with PWM, arm servos on the dedicated servo bus, the OLED and IMU over I2C/SPI, and battery voltage on the onboard ADC. Wi-Fi comes from the Jetson radio. Splitting host + MCU keeps YOLO inference and SLAM optimization from blocking the real-time loop (motor PWM, servo bus updates).

3. Software framework

3.1 Layered architecture

Software is five layers: Interaction, Decision, Perception, Control, and Hardware Abstraction. Data flows sensors -> Perception -> Decision -> Control -> HAL -> actuators. A layer talks only to its neighbors. No skip-level calls.

3.2 How modules talk

4.1 Voice I/O and the OpenClaw agent

There are two stages. The front end is the WonderEcho Pro voice box (capture + ASR). The back end is the OpenClaw agent: language understanding and task decisions.

Voice front end (WonderEcho Pro): Implemented by the wonder_echo_pro_node in the xf_mic_asr_offline package. It talks to the microphone over serial and publishes /vocal_detect/wakeup (Bool) and /vocal_detect/asr_result (String).

Agent brain (OpenClaw): Implemented by the claw_voice node in the openclaw_controller package. It subscribes to WonderEcho Pro's ASR result, then sends an HTTP POST to the OpenClaw gateway at 127.0.0.1:18789 for language understanding. The API matches the OpenAI Chat Completions format.

# Source: claw_voice.py
payload = {
    "model": "openclaw/default",
    "messages": [{"role": "user", "content": text}],
    "user": SESSION_KEY
}
resp = requests.post(
    f"{GATEWAY_URL}/v1/chat/completions",
    headers=headers,
    json=payload
)
reply = result["choices"][0]["message"]["content"]

OpenClaw is more than speech understanding. Package openclaw_controller registers eight nodes: speech understanding, speech synthesis, target tracking, track-and-grasp, arm group control, chassis motion, navigation manager, and AprilTag localization. Together they are the robot "brain": they split a sentence into navigate / detect / grasp subtasks and dispatch them.

Cloud brain vs local brain. By default the gateway forwards to a remote API. If onboard compute is high enough (the article uses "100 TOPS+" as a rule of thumb), point the model field in ~/.openclaw/openclaw.json at a local checkpoint. Local inference cuts latency and works offline, but RAM is tight: SLAM + YOLO + an LLM on 8 GB is a squeeze.

4.2 SLAM mapping

ROSOrin Pro maps with slam_toolbox's sync_slam_toolbox_node (synchronous SLAM). The front end is scan matching. The back end is pose-graph optimization with Ceres Solver. Each laser scan is aligned to the map while the robot pose and the occupancy grid are updated together.

Core parameters in slam/config/slam.yaml:

Official mapping backends. Launch argument slam_method selects the stack. Pick the one that fits the room:

Switch at launch:

ros2 launch slam slam.launch.py slam_method:=cartographer

Configs live at:

• slam/config/slam.yaml (slam_toolbox)
• slam/config/cartographer_2d.lua (Cartographer)
• slam/config/gmapping.yaml (Gmapping)

Mapping procedure:

# 1. Bring up chassis and sensors
ros2 launch bringup bringup.launch.py

# 2. Start SLAM
ros2 launch slam slam.launch.py

# 3. Drive slowly with a joystick or keyboard while the map fills in

# 4. Save the map
ros2 run nav2_map_server map_saver_cli -f ~/my_map

Map quality depends on three things: speed (keep under 0.3 m/s), scene texture (empty corridors need extra passes), and a completed loop (close the path at least once).

4.3 Nav2 navigation

ROSOrin Pro uses Nav2 as a two-layer stack: global plan + local control.

Nav2 is not only "go to one pose." Built-in patterns:

Local controller knobs in nav2_params.yaml:

controller_server:
  ros__parameters:
    controller_frequency: 20.0   # 20 Hz control loop
    FollowPath:
      plugin: "teb_local_planner::TEBController"
      max_vel_x: 0.5             # max linear speed 0.5 m/s
      max_vel_theta: 1.0         # max yaw rate 1.0 rad/s
      xy_goal_tolerance: 0.1     # 10 cm XY tolerance
      yaw_goal_tolerance: 0.1    # ~5.7 deg yaw tolerance

4.4 YOLO detection and OpenCV color tracking

Two vision stacks, two jobs:

• YOLO - traffic-sign detection for the autonomous-driving demos (straight, right turn, stop, traffic light, crosswalk). Ultralytics stack. Default engine YOLOv26n; optional YOLOv11s. Exported to TensorRT .engine files and run on the Jetson.
• OpenCV color detection - block finding for the arm. LAB threshold + contours, implemented by class ColorTracker in node claw_track_and_grab.

YOLO node (yolo_node) - core loop

# Source: yolo_node.py - detection loop
results = self.yolo_detect(image, conf=self.conf, task=self.task)
for result in results:
    for i in range(len(items)):
        confidence = items.conf[i].item()
        class_name = self.classes[class_id]
        object_info = ObjectInfo()
        object_info.class_name = class_name   # class label
        object_info.score = float(confidence) # 0-1 confidence
        object_info.box = box_coords          # [x1, y1, x2, y2]
        objects_info.append(object_info)
object_msg = ObjectsInfo()
object_msg.objects = objects_info
self.object_pub.publish(object_msg)          # ~/object_detect

YOLO interfaces:

ObjectsInfo fields (interfaces/msg/ObjectInfo.msg):

OpenCV color tracker used by the gripper:

# Source: claw_track_and_grab.py - ColorTracker
img_lab = cv2.cvtColor(img_blur, cv2.COLOR_BGR2LAB)  # BGR -> LAB
mask = cv2.inRange(img_lab, tuple(color['min']), tuple(color['max']))
contours = cv2.findContours(
    dilated, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE
)[-2]
# Keep the largest contour -> min enclosing circle -> block center
(center_x, center_y), radius = cv2.minEnclosingCircle(best_c[0])

Model choice: YOLO + OpenCV. YOLO is a TensorRT engine on Orin NX. With spare TOPS you can swap in Faster R-CNN or SSD by changing the engine:= parameter only.

Launch examples:

# Start YOLO (default YOLOv26n traffic weights)
ros2 run example yolo_node --ros-args -p engine:=best_traffic_26

# Switch to YOLOv11s
ros2 run example yolo_node --ros-args -p engine:=best_traffic_11

4.5 MoveIt2 and arm control

ROSOrin Pro ships a 6-DOF arm on a bus-servo link. The current official path is direct bus-servo control, in three layers.

Kinematic model: modified Denavit-Hartenberg. Five revolute joints plus a gripper = six DOF. Inverse kinematics takes target [x, y, z] and pitch, then solves the five arm joints. Gripper open/close is independent. If the requested pitch has no solution, the solver searches pitch_range (default ±180°) at resolution (default 1°).

When to add MoveIt2. Direct servo packets are enough for fixed pick-and-place. For live obstacle avoidance, trajectory smoothing, or coordinated multi-axis motion, put MoveIt2 in front of the same servo layer:

Integration steps:

# 1. Install MoveIt2 (ROS 2 Humble)
sudo apt install ros-humble-moveit

# 2. Load the arm URDF in MoveIt Setup Assistant -> generate moveit_config

# 3. Map FollowJointTrajectory to servo_controller

# 4. Launch
ros2 launch moveit_config move_group.launch.py

MoveIt2 Python API sketch:

from moveit.planning import MoveItPy
from moveit.core.robot_state import RobotState

# Init
robot = MoveItPy(node_name="arm_planner")
arm = robot.get_planning_component("arm")

# Goal + plan
arm.set_start_state_to_current_state()
arm.set_goal_state(configuration_name="ready")
plan_result = arm.plan()
if plan_result:
    robot.execute(plan_result.trajectory, controllers=[])

MoveItPy is the Humble-era Python binding. configuration_name="ready" is a named state from the SRDF produced by Setup Assistant. After plan() succeeds, execute() sends FollowJointTrajectory to the controller mapped onto the Hiwonder bus servos. Keep the existing kinematics service for simple scripted picks; use MoveIt when the path must weave around the camera mast or the chassis.

5. Summary

ROSOrin Pro covers the main teaching blocks of a mobile manipulator:

• WonderEcho Pro is the ASR front end. OpenClaw is the brain that parses language and schedules modules. Default brain is cloud; a local model is optional if memory allows.
• SLAM backends: slam_toolbox (default), Cartographer (large, high quality), Gmapping (small, cheap). Switch with slam_method.
• Nav2 is more than point-to-point: waypoints, transport missions, custom behavior trees, multi-robot.
• YOLO detects traffic signs. OpenCV color tracking finds blocks. Engines can be swapped (Faster R-CNN, SSD, etc.).
• MoveIt2 upgrades the arm from bus-servo setpoints to planned, collision-checked trajectories.

The design idea is layered decoupling: five software layers, each talking only next door; Jetson + STM32 on the metal; Topic / Service / Action covering stream vs one-shot vs long job.

Appendix A - Topic / service cheat sheet

Appendix B - Launch commands

ros2 launch bringup bringup.launch.py
ros2 launch slam slam.launch.py
ros2 launch slam slam.launch.py slam_method:=cartographer
ros2 run nav2_map_server map_saver_cli -f ~/my_map
ros2 run example yolo_node --ros-args -p engine:=best_traffic_26
ros2 run example yolo_node --ros-args -p engine:=best_traffic_11
sudo apt install ros-humble-moveit
ros2 launch moveit_config move_group.launch.py