Reachy Mini — Gemini Live Edition

Fork additions

What this adds over the base app

Everything in pollen-robotics/reachy_mini_conversation_app still works. These are the new layers on top.

🧠

Persistent memory per user

Vertex AI Agent Engine stores facts about each person across sessions. Reachy greets you by name, picks up mid-topic, and keeps each household member's context private. Recency-weighted retrieval surfaces what matters now; Reachy can call recall mid-conversation, or remember(fact, lifetime_days=…) for ephemeral facts that auto-expire.

↗ Vertex AI Agent Engine ↗ Memory bank pattern

🧹

Memory hygiene

One-click Compact memories in the Personality Studio runs an LLM-driven dedup pass over the active user's facts: deletes duplicates, merges overlaps, drops expired ephemerals. Preview (red = delete, teal = merge) before anything is touched, then click Apply to execute exactly what you reviewed.

🔍

Google Search grounding

Per-profile toggle. When on, Gemini retrieves live search results before answering time-sensitive questions — weather, news, sports scores, flight status.

↗ Gemini grounding docs

🔌

MCP servers — extended tool support

Added MCP server support, extending the tool ecosystem beyond built-in tools. Each profile defines its own servers with secure credential injection. Home Assistant, weather, calendar — swap tools per persona without touching code.

↗ Model Context Protocol

🏠

Home Assistant control

Lights, climate, media, locks, shades, sprinklers, Google Calendar — all via natural voice through the HA MCP server. No entity IDs exposed to the user.

↗ HA MCP Server

😴

Wake word + sleep modes

Say "Hey Reachy" to wake from sleep — a rooster crow chirps when detected. Tell Reachy "good night" or "you can rest now" and it ends the session itself. Auto-sleeps after a configurable idle window (1–60 min, default 10) — Gemini billing stops, Edge Impulse model detects the wake word offline on-device.

↗ luisomoreau/hey_reachy_wake_word_detection

📺

Live browser monitor

1fps camera feed, real-time transcript stream, activity log, and wake/sleep controls — all in the browser at http://ROBOT_IP:7860. Not in the base app.

↗ Gemini Live API

🎭

Personality Studio

Headless web profile editor at http://ROBOT_IP:7860 — edit system prompt, voice (30 Gemini voices with audio preview), tools, MCP servers, and per-profile feature flags from your phone. No SSH, no file editing. Save startup profile, apply live without restart.

👤

Multi-user awareness

The identify_user tool uses the camera to recognize who's speaking and switches the active memory scope mid-conversation. Each household member gets their own private context — preferences, ongoing topics, calendar, all kept separate. One robot, every person remembered individually.

Multi-user family assistant

One robot, every household member remembered

📸 Camera identifies who's there at session start.
🧠 Loads that person's memories — preferences, projects, ongoing topics.
🔄 Switches context mid-conversation when someone else walks up.
🔒 Each person's memories are scoped privately by user + profile.

Built on Pollen Robotics' foundation

All the original features, plus a Google Cloud layer

The base app's voice loop, dance library, emotion system, camera tools, and profile UI are all here. This fork adds Gemini Live as the sole backend, Vertex AI for memory, and MCP for extensible home control.

Gemini Live API Agent Engine memory MCP servers Edge Impulse wake word HA integration Live monitor

Reachy Mini that remembers you.