Skip to content

One Agent for Any Robot.

Tell Kinex Kinex about your robot and the job to do. It adapts to your robot, carries out tasks autonomously, and learns from every run, both in simulation or on real hardware, including ROS 2 devices.

Install on macOS and Ubuntu.

$curl -fsSL https://raw.githubusercontent.com/stacx-lab/kinex/main/install.sh | bash

Prefer to use Pi, Codex, or Claude Code? Explore kinex-plugins →

Get started → Watch the demos ↓

Kinex is built on Pi.

Kinex
RoboEnv ~/lab
roboenv launch abc/hang-mug-on-mug-rack --detach
Ready: abc/hang-mug-on-mug-rack Endpoint: http://127.0.0.1:8123
mkdir -p workspace
(cd workspace && roboenv setup abc/abc-yam)
Robot: abc/abc-yam Destination: workspace/.kinex
cd workspace
kinex
Kinex· robotics coding agent
what's in the scene?
device read camera_top✓
RGB-D · 640 × 480
read camera_top✓
Overhead observation: three mugs, a wooden rack, and two robot arms.
camera_top · overhead
Three mugs and a wooden rack. The mint mug is nearest the right arm.
hang the mint mug on the rack
device read right_arm✓
joints: [0.202, 0.390, 1.118, -0.873, 0.133, 0.426] faults: []
write mug_position.py✓
# Backproject the selected mug pixel from RGB-D.
u, v = 449, 296
z = depth[v * width + u] * scale
xyz = [r * z for r in ray(intr, u, v)]
distance = math.sqrt(sum(v * v for v in xyz))
bash python mug_position.py✓
camera xyz ≈ [0.278, 0.121, 0.929] m distance ≈ 0.977 m
relative_pose world → right_arm✓
world → right_arm · approach flange [0.1775, 0.0280, 0.2747, 0.7071, 0.7071, 0, 0]
servo right_arm · approach + descend✓
TCP [0.430, -0.282, 0.805] m · error 0.44 mm
device write right_gripper✓
close · aperture 0.0688 m
servo right_arm · lift + carry✓
TCP [0.567, -0.130, 0.984] m · grip maintained
read camera_top + camera_right✓
Overhead observation of the right arm carrying the mint mug toward the rack.
camera_top · overhead
Right wrist camera: close-up of the mint mug held between the fingers.
camera_right · right wrist
The mug blocks the right wrist. I’ll use the left wrist to align the handle.
relative_pose camera_left → world✓
handle hole [0.5029, -0.0188, 0.9543] m plane RMS 0.50 mm
servo right_arm · align + insert✓
speedFraction 0.25 · check contact from both views
read camera_left + camera_top✓
Left wrist view of the peg through the mint mug handle before release.
camera_left · left wrist
Overhead view of the mug at the rack before release.
camera_top · overhead
device write right_gripper✓
open · release the mug
servo right_arm · withdraw✓
TCP [0.500, -0.150, 1.030] m · gripper clear
devices read camera_left + camera_top✓
Settled · new wrist + overhead observations
read camera_left + camera_top✓
Left wrist verification: the mint mug hangs by its handle on the wooden peg, clear of the gripper.
camera_left · left wrist
Overhead verification: the mint mug remains on the rack after the right gripper withdraws.
camera_top · overhead
The mug is hanging. Both grippers are clear; it stayed supported after release.
~/lab/workspaceKinex · ask
Session completeSession demo

02 · Architecture

Three layers of abstraction for acting with arbitrary robots.

An agent needs a way to turn a goal into physical action. Kinex separates that work into three layers: skills describe how to approach a task, tools carry out perception, computation, and control, and devices expose the robot’s capabilities through typed interfaces. Results flow back to the agent so it can decide what to do next.

Kinex architecture The agent loads skills, calls tools and calls devices. Tools operate devices. Device drivers talk to hardware, simulators and model services. Kinex Agent Plan · reason Pick tool/device Read results Skills Instructions, procedures setup-device · calibration · verification Tools Executable perception, computation, control locate_object_3d · servo · calib_sample Devices Typed interfaces, profiles, drivers arm · camera · detector · world Environment Hardware · simulators · model endpoints Load Call Result Call Result Ops Obs/state Driver req Feedback

A system of devices rather than a monolithic robot

Kinex exposes a robot as independent devices: a mobile base, torso, arms, grippers, and cameras. The agent writes and runs tools and code to coordinate these devices for each task. This makes workflows more flexible, lets each task compose the robot it needs, and helps them generalize across robot configurations. If a camera is missing, for example, the agent can inspect the remaining devices, use another viewpoint where available, and revise its workflow to fit the capabilities it actually has.

One coordinated grasp
Control
Camera observations become coordinated robot actions Cameras send observations to the agent, which computes a target position and uses Kinex tools and code to send an arm motion command. The arm reaches the object and the gripper closes. Images, state, and execution results from the devices return to the agent. The animation illustrates one continuous control loop. Images + depth Motion command Grasp command Images · measured state · execution results Agent Reads camera observations Kinex Writes & runs tools / code Base Torso

Camera observations → target position → arm motion → grasp → feedback

03 · Benchmark results

Zero-Shot Robotic Manipulation Evaluation

We evaluate 132 tasks from LIBERO, RoboTwin, and RoboDojo under a zero-shot protocol with native success criteria. Control-step budgets are unchanged except for RoboDojo, increased to 1,875 steps per episode. Following BenchMend, we rewrite task instructions to make hidden criteria explicit, such as parking the arm at its home position.

For GPT-6-Astra (high effort), we compare Codex with the benchmark-native action API, with the device abstraction, and Kinex. Kinex raises success from 72.7% to 78.0% at 42.9% lower cost than Codex + Bare action API. See the Hugging Face release for evaluation details, demos, and agent traces.

Choose benchmark results

All benchmarks · 132 tasks per setup

GPT-6-Astra

Codex · high effort

Task success rate

Higher is better
72.7% (96 of 132 tasks)
Codex + Bare action API
72.7% (96 of 132 tasks)
Codex + Device API
78.0% (103 of 132 tasks)
Kinex

Average cost per task

USD · lower is better
$3.31
Codex + Bare action API
$2.16
Codex + Device API
$1.89
Kinex

Average weights all 132 tasks equally. Success rate is successful tasks / evaluated tasks; cost is the mean API-equivalent estimate in USD per task, including successful and failed episodes.

Evaluation details, demos, and agent traces ↗

04 · Where to start

Already using ROS 2? Skill: onboard-ros2 Reuse existing ROS packages with adapters for common interfaces. Supports ROS 2 Jazzy on Ubuntu 24.04. → Play with simulators $ roboenv setup RoboEnv exposes simulated robots as Kinex devices, with the same interfaces as real hardware. → Work with real robots $ /onboarding Describe the hardware and the task. Kinex connects the devices and calibrates what the task needs. →

05 · Current contributors

Georgia Tech

Harvard

Contributions welcome. New skills, device drivers, robot support and fixes all help Kinex work on more robots.

Search