README.md

metadata

title: Reachy Eleven Agent
emoji: 🤖
colorFrom: purple
colorTo: gray
sdk: static
pinned: false
tags:
  - reachy_mini
  - reachy_mini_python_app

Reachy Eleven Agent

Reachy Mini voice interaction app powered by ElevenLabs ElevenAgents.

The app listens through Reachy Mini's media audio stream, sends the conversation to an ElevenLabs agent, plays the agent response through Reachy's speaker, and taps that audio stream so Reachy moves while speaking. The app also exposes client tools that let the ElevenLabs agent trigger simple Reachy gestures.

Requirements

• A Reachy Mini with reachy-mini-daemon running.
• Python 3.12.
• uv for dependency management.
• An ElevenLabs Conversational AI agent ID.
• Reachy Mini media audio available through the daemon.

Quick Start

From the Reachy Mini dashboard

1. Open the Reachy Mini dashboard.
2. Install this Space from the app catalog.
3. Click Play.
4. Open the app settings page.
5. Enter your ElevenLabs agent ID and optional API key.
6. Click Save settings, then Start.

The settings are saved locally for the app instance, so future launches can use the same ElevenLabs configuration without SSH.

Local development

From the repository root, which is the directory containing pyproject.toml:

export UV_LINK_MODE=copy  # optional on mounted volumes where hardlinks are not supported
uv sync --dev
cp .env.example .env

Edit .env and set:

ELEVENLABS_AGENT_ID=agent_your_public_agent_id

Start reachy-mini-daemon, then launch the app. Direct CLI launches start the ElevenLabs conversation immediately from environment variables:

uv run reachy-eleven-agent

If your daemon uses a robot name:

uv run reachy-eleven-agent --robot-name <robot-name>

Install uv if it is not already available:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv --version

Setup

Clone this repository, then work from the directory that contains pyproject.toml:

git clone <repository-url>
cd <repository-directory>

Create the synchronized environment:

# Optional on mounted volumes where hardlinks are not supported.
export UV_LINK_MODE=copy

uv sync --dev

This environment pins reachy-mini[wireless-version]==1.7.1 so the SDK and reachy-mini-daemon stay aligned. Launch both the daemon and the app with uv run from this repository so they use the same locked package version.

Create your local environment file:

cp .env.example .env

Edit .env and set at least:

REACHY_AGENT_PROVIDER=elevenlabs
ELEVENLABS_AGENT_ID=agent_your_public_agent_id
REACHY_ELEVEN_KEEPALIVE_INTERVAL_S=20
REACHY_ELEVEN_RECONNECT_DELAY_S=2
REACHY_ELEVEN_ROBOT_RECONNECT_DELAY_S=2
REACHY_ELEVEN_ROBOT_RECONNECT_MAX_DELAY_S=30
REACHY_ELEVEN_MEDIA_BACKEND=no_media

For a private ElevenLabs agent, also set:

ELEVENLABS_API_KEY=your_elevenlabs_api_key

Do not commit .env.

ElevenLabs Agent Setup

In the ElevenLabs dashboard, create or open a Conversational AI agent. Add the Reachy controls as client tools. ElevenLabs' client tool documentation is available at https://elevenlabs.io/docs/conversational-ai/customization/tools/client-tools.

1. Open your agent in ElevenLabs.
2. Go to Tools.
3. Add a Client tool.
4. Click Edit as JSON.
5. Replace the generated JSON with one of the blocks below.
6. Save the tool, then repeat for the remaining tools.
7. Make sure each saved tool is enabled for the agent.

Tool names are case-sensitive. They must match the names registered by this app: reachy_move_head, reachy_set_antennas, reachy_set_body_yaw, reachy_expression, reachy_sleep, and reachy_wake.

These tools use expects_response: false because they are non-blocking robot gestures. The app still returns short status strings internally, but the agent does not need to pause the conversation waiting for them.

reachy_move_head

{
  "type": "client",
  "name": "reachy_move_head",
  "description": "Move Reachy's head. Use small, expressive movements to look left, right, up, down, forward, or to set roll/pitch/yaw angles directly.",
  "expects_response": false,
  "response_timeout_secs": 1,
  "parameters": [
    {
      "id": "direction",
      "type": "string",
      "value_type": "llm_prompt",
      "description": "Optional preset direction. Use one of: left, right, up, down, front.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": ["left", "right", "up", "down", "front"],
      "required": false
    },
    {
      "id": "roll_deg",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Optional roll angle in degrees from -40 to 40.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": false
    },
    {
      "id": "pitch_deg",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Optional pitch angle in degrees from -40 to 40.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": false
    },
    {
      "id": "yaw_deg",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Optional yaw angle in degrees from -65 to 65.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": false
    },
    {
      "id": "duration_s",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Optional movement duration in seconds from 0.3 to 4.0. Use 0.8 for a normal gesture.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": false
    }
  ],
  "dynamic_variables": {
    "dynamic_variable_placeholders": {}
  },
  "assignments": [],
  "disable_interruptions": false,
  "pre_tool_speech": "auto",
  "tool_call_sound": null,
  "tool_call_sound_behavior": "auto",
  "execution_mode": "immediate",
  "response_mocks": []
}

reachy_set_antennas

{
  "type": "client",
  "name": "reachy_set_antennas",
  "description": "Move Reachy's right and left antennas. Use this for small expressive ear-like gestures.",
  "expects_response": false,
  "response_timeout_secs": 1,
  "parameters": [
    {
      "id": "right_deg",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Right antenna angle in degrees from -175 to 175.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": true
    },
    {
      "id": "left_deg",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Left antenna angle in degrees from -175 to 175.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": true
    },
    {
      "id": "duration_s",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Optional movement duration in seconds from 0.3 to 4.0. Use 0.7 for a normal gesture.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": false
    }
  ],
  "dynamic_variables": {
    "dynamic_variable_placeholders": {}
  },
  "assignments": [],
  "disable_interruptions": false,
  "pre_tool_speech": "auto",
  "tool_call_sound": null,
  "tool_call_sound_behavior": "auto",
  "execution_mode": "immediate",
  "response_mocks": []
}

reachy_set_body_yaw

{
  "type": "client",
  "name": "reachy_set_body_yaw",
  "description": "Rotate Reachy's body around its vertical axis. Use sparingly for larger orientation changes.",
  "expects_response": false,
  "response_timeout_secs": 1,
  "parameters": [
    {
      "id": "yaw_deg",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Body yaw angle in degrees from -160 to 160.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": true
    },
    {
      "id": "duration_s",
      "type": "number",
      "value_type": "llm_prompt",
      "description": "Optional movement duration in seconds from 0.3 to 5.0. Use 1.0 for a normal turn.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": null,
      "required": false
    }
  ],
  "dynamic_variables": {
    "dynamic_variable_placeholders": {}
  },
  "assignments": [],
  "disable_interruptions": false,
  "pre_tool_speech": "auto",
  "tool_call_sound": null,
  "tool_call_sound_behavior": "auto",
  "execution_mode": "immediate",
  "response_mocks": []
}

reachy_expression

{
  "type": "client",
  "name": "reachy_expression",
  "description": "Play a short built-in Reachy expression. Prefer this tool for common conversational gestures.",
  "expects_response": false,
  "response_timeout_secs": 1,
  "parameters": [
    {
      "id": "name",
      "type": "string",
      "value_type": "llm_prompt",
      "description": "Expression name. Use one of: nod, shake_no, happy, curious, reset.",
      "dynamic_variable": "",
      "constant_value": "",
      "enum": ["nod", "shake_no", "happy", "curious", "reset"],
      "required": true
    }
  ],
  "dynamic_variables": {
    "dynamic_variable_placeholders": {}
  },
  "assignments": [],
  "disable_interruptions": false,
  "pre_tool_speech": "auto",
  "tool_call_sound": null,
  "tool_call_sound_behavior": "auto",
  "execution_mode": "immediate",
  "response_mocks": []
}

reachy_sleep

{
  "type": "client",
  "name": "reachy_sleep",
  "description": "Put Reachy into its sleep pose and pause app-level motion. Use only when the user asks Reachy to sleep, rest, or shut down.",
  "expects_response": false,
  "response_timeout_secs": 1,
  "parameters": [],
  "dynamic_variables": {
    "dynamic_variable_placeholders": {}
  },
  "assignments": [],
  "disable_interruptions": false,
  "pre_tool_speech": "auto",
  "tool_call_sound": null,
  "tool_call_sound_behavior": "auto",
  "execution_mode": "immediate",
  "response_mocks": []
}

reachy_wake

{
  "type": "client",
  "name": "reachy_wake",
  "description": "Wake Reachy and resume app-level motion. Use when the user asks Reachy to wake up or continue.",
  "expects_response": false,
  "response_timeout_secs": 1,
  "parameters": [],
  "dynamic_variables": {
    "dynamic_variable_placeholders": {}
  },
  "assignments": [],
  "disable_interruptions": false,
  "pre_tool_speech": "auto",
  "tool_call_sound": null,
  "tool_call_sound_behavior": "auto",
  "execution_mode": "immediate",
  "response_mocks": []
}

Prompt guidance for the agent: speak as Reachy, keep spoken answers concise, and call movement tools sparingly to punctuate the conversation. Use reachy_expression for common gestures before composing lower-level head or antenna motions.

Suggested system prompt addition:

You are embodied as Reachy Mini. Keep replies concise and conversational.
Use Reachy client tools only when motion adds meaning to the interaction.
Prefer reachy_expression for simple gestures such as nodding, curiosity, or happiness.
Do not call motion tools repeatedly or continuously. Use small movements and let speech remain the main interaction.
Use reachy_sleep only when the user asks you to sleep, rest, or shut down.
Use reachy_wake only when the user asks you to wake up or resume.

Development And Robot Deployment

Most developers should edit and test packaging from a workstation, then synchronize the repository to the Reachy Mini and run the live app on the robot. This matters because the ElevenLabs provider uses the local machine's audio pipeline. If you run reachy-eleven-agent on your workstation, it will use the workstation microphone and speaker, not Reachy's onboard audio.

A typical workflow is:

1. Develop on your workstation.
2. Run local checks:

uv sync --dev
uv run pytest tests/test_elevenlabs_agent.py tests/test_moves.py -q
uv run reachy-mini-app-assistant check .

3. Synchronize the repository to the Reachy Mini with your normal deployment method, such as git pull on the robot or rsync from your workstation.
4. SSH into the robot and run the daemon/app commands there.

The Reachy Mini daemon also runs on the robot. Restart it on the robot, not on your development workstation:

ssh pollen@<reachy-mini-host-or-ip>
pkill -f reachy-mini-daemon
uv run reachy-mini-daemon --wireless-version --no-media

Confirm the app environment is using the pinned SDK/daemon package:

uv run python -c "import reachy_mini; print(reachy_mini.__version__)"

If your robot uses a custom robot name, pass the same name to both commands:

uv run reachy-mini-daemon --wireless-version --no-media --robot-name <robot-name>
uv run reachy-eleven-agent --robot-name <robot-name>

Run On Reachy Mini

When installed from the app catalog, start the app from the dashboard. The dashboard settings page lets you save the ElevenLabs agent ID, optional private agent API key, optional user ID, and start/stop the conversation.

For SSH-based development on the robot, start the Reachy Mini daemon first, then run:

uv run reachy-eleven-agent

If your daemon was started with a robot name:

uv run reachy-eleven-agent --robot-name <robot-name>

The default provider is ElevenLabs. The original OpenAI realtime conversation template remains available for development:

uv run reachy-eleven-agent --provider openai --gradio

Validate

Run the focused tests and Reachy app packaging check:

uv run pytest tests/test_elevenlabs_dashboard.py -q
uv run pytest tests/test_elevenlabs_agent.py -q
uv run reachy-mini-app-assistant check .

Publish

Authenticate with Hugging Face using a token that can create Spaces:

uv run hf auth login
uv run hf auth whoami

Publish the app:

uv run reachy-mini-app-assistant publish . "Publish Reachy Eleven Agent"

Use --private or --public if you need to control Space visibility:

uv run reachy-mini-app-assistant publish --private . "Publish private Reachy Eleven Agent"

Updating

After pulling changes:

git pull
uv sync --dev
uv run reachy-mini-app-assistant check .

Troubleshooting

• ModuleNotFoundError: No module named 'reachy_mini': run commands with uv run from the directory containing pyproject.toml, or rerun uv sync --dev.
• No microphone or speaker audio: confirm reachy-mini-daemon is running, the app is connected to the expected robot, and the daemon's media backend can record and play audio.
• v4l2h264dec0: Too old frames: the app is opening a video decoder. For the ElevenLabs provider, keep REACHY_ELEVEN_MEDIA_BACKEND=no_media so the app opens only the local audio pipeline.
• OSError: [Errno -9985] Device unavailable: update to the current code. Older revisions used the ElevenLabs SDK PyAudio adapter, which opens host audio devices instead of Reachy's media stream.
• Failed to set robot target: Lost connection with the server: the app lost its control connection to reachy-mini-daemon. In ElevenLabs mode the voice app stays alive and reconnects the Reachy Mini SDK websocket in the background; check logs for Attempting Reachy Mini daemon reconnect and Reconnected to Reachy Mini daemon.
• Reachy Mini SDK and daemon versions do not match: restart the daemon with uv run reachy-mini-daemon --wireless-version --no-media from this repository, then restart uv run reachy-eleven-agent. Both commands should use reachy-mini==1.7.1.
• Robot keeps moving but no longer responds to voice: the motion loop may still be running after the ElevenLabs websocket ended. The app sends keepalives and automatically reconnects; check logs for ElevenLabs conversation ended and Restarting ElevenLabs conversation. Lower REACHY_ELEVEN_KEEPALIVE_INTERVAL_S if your agent times out quickly.
• Hugging Face 403 Forbidden while publishing: the token is valid but does not have permission to create Spaces in the target namespace.
• Reachy connection timeout: verify reachy-mini-daemon is running and pass --robot-name if the daemon uses one.