Skip to content

Concepts

Architecture

Devices, tools and skills — and how the agent, runtime and host fit together.

Architecture overview

Kinex connects a coding agent to robots and simulators through three layers.

Kinex architecture The agent loads skills, calls tools and calls devices. Tools operate devices. Device drivers talk to hardware, simulators and model services. Kinex Agent Plan · reason Pick tool/device Read results Skills Instructions, procedures setup-device · calibration · verification Tools Executable perception, computation, control locate_object_3d · servo · calib_sample Devices Typed interfaces, profiles, drivers arm · camera · detector · world Environment Hardware · simulators · model endpoints Load Call Result Call Result Ops Obs/state Driver req Feedback

Devices: typed physical or virtual resources

A device is anything the agent can read from or command: an arm, a camera, a gripper, but also a detection model or the simulator’s clock. Each device has a type, which fixes its operations and data conventions; a profile, which records how to reach it and its specifications; and a driver, which talks to the real SDK, simulator or service. Because real hardware and simulators share the same types, the agent works with both the same way. → Devices

Tools: executable operations over devices and computation

Tools bundle computation or a sequence of device calls into one named operation, so the agent doesn’t have to orchestrate every step itself. For example:

  • locate_object_3d combines a camera image, a detection or segmentation model and depth to estimate where an object is in 3D.
  • servo plans a joint trajectory from measured state, sends it and waits its planned duration by default. Arrival still needs fresh feedback.
  • exec runs Python source or script files to combine device calls, bounded feedback loops and computation, returning structured results and selected camera images for the agent’s next decision.

You can add your own as a tool.yaml plus tool.py. Python tools never load or unload devices; only the device tool and your own commands can. → Tools & skills

Skills: task procedures the agent loads on demand

A skill is a written procedure, a SKILL.md with optional references and scripts, that tells the agent how to approach a kind of task, such as setting up a device or calibrating a camera. The agent loads one when it needs it. Loading a skill only adds instructions to the conversation; it runs nothing and grants no extra permissions. → Tools & skills

The coding-agent analogy

LayerIn a coding agentIn Kinex
DevicesDisks, GPUs behind OS/driver interfacesCameras, arms, model services behind Kinex drivers
Toolsbash, grep turn resources into operationslocate_object_3d, servo, exec
Skills“Reproduce → inspect → fix → test”“Sample → compute → verify” for calibration

Packages

Kinex is split into three packages that ship together at the same version:

  • @kinex/core is the robotics layer: devices and drivers, built-in tools and the runtime Python. It doesn’t depend on any agent or UI, so other harnesses can use it too.
  • @kinex/setup adds onboarding, calibration and ROS 2 setup on top of core.
  • kinex is the command-line app you run. It combines the two with the Pi agent, which provides the model connection, the conversation loop, sessions and the terminal UI. It also decides what needs your approval.

For the full contracts, see the design docs.

Search