I’m building Hoot, a small robot companion for learning. A little owl that can look at what I’m working on, talk things through with me, and help me practice. It’s my noncompetitive project for the Nebius × NVIDIA Global AI Hackathon.
Hoot’s current collection of parts, ready for storytime.
Learning together
Chinese storytime is my starting point. I’ve been making tools to help me study Chinese for a while, and Hoot builds on that: capture book pages, explore unfamiliar characters, and save words for later practice. I’d like to see where else a little learning companion could be useful.
Tap a character for pinyin, audio, and a way to save it or ask Hoot for help.
What’s inside
- Conversation and vision: NVIDIA Nemotron 3.5 Super VL through Nebius Token Factory, with Hermes managing context and tools.
- The physical owl: a Jetson Orin Nano, camera, USB microphone and speaker, and Sense HAT sensors and LED expressions. Visual indicators show when the microphone is recording, when the camera is on, and when Hoot is speaking.
- English and Mandarin voice: local Faster-Whisper recognition and Kokoro speech, or optional Gemini transcription and TTS. In cloud mode, Nemotron still writes the answers.
- Reading and review: captured pages, editable text, pinyin, saved vocabulary, and practice cards, backed by local SQLite.
- Web dashboard: Vite and Three.js power the animated owl, camera preview, chat, and device controls.
The Sense HAT LEDs show expressions and microphone, camera, and speaking status.
Hoot on screen
The dashboard runs in the browser. The Jetson serves it through a local web server, which connects to Hermes running on the device.
The dashboard is the interface; Hermes manages the agent and tools on the Jetson.
It’s a working prototype. In cloud mode, Hoot needs an internet connection to Nebius for the assistant’s reasoning. Speech can already run locally, and a local mode could use a model running directly on the Jetson Orin Nano for the reasoning too.
A shoulder owl?
I already have a battery for Hoot, and I want to work out a way to mount it on my shoulder. I’d also like to be able to position it in different places around a room, wherever it has a useful view of what I’m working on.
A handheld tilt test while I work on the physical setup.
The shoulder mount is still a plan. But having a little owl hanging out on my shoulder while I learn something sounds pretty fun.
On the software side, I want faster responses, better reading alignment, and pronunciation help that notices when I hesitate or get stuck. Still plenty to work on, but I’m glad to have the pieces working together.