# Hoot: A Small Robot Companion For Learning

> Building a small robot companion for learning with a Jetson Orin Nano, Nemotron, and Nebius Token Factory.

URL: [Original article](https://mattcool.tech/posts/hoot-a-small-robot-companion-for-learning)

Published: 2026-10-19T19:00:00.000Z

Tags: notes, ai

I'm building [Hoot](https://github.com/matthewbcool/Hoot), a small robot companion for learning. A little owl that can look at what I'm working on, talk things through with me, and help me practice. It's my noncompetitive project for the [Nebius × NVIDIA Global AI Hackathon](https://nebiusglobalaihackathon.devpost.com/).

[Image: Hoot]

Caption:
Hoot's current collection of parts, ready for storytime.

## Learning together

Chinese storytime is my starting point. I've been [making tools to help me study Chinese](/posts/hands-free-chinese-flashcard-maker/) for a while, and Hoot builds on that: capture book pages, explore unfamiliar characters, and save words for later practice. I'd like to see where else a little learning companion could be useful.

[Image: The character 在 selected in Hoot]

Caption:
Tap a character for [pinyin](https://github.com/zh-lx/pinyin-pro), audio,
and a way to save it or ask Hoot for help.

## What's inside

- **Conversation and vision:** NVIDIA Nemotron 3.5 Super VL through [Nebius Token Factory](https://docs.tokenfactory.nebius.com/quickstart), with [Hermes](https://hermes-agent.nousresearch.com/docs/) managing context and tools.
- **The physical owl:** a [Jetson Orin Nano](https://developer.nvidia.com/embedded/learn/getting-started-jetson), camera, USB microphone and speaker, and [Sense HAT](https://www.raspberrypi.com/documentation/accessories/sense-hat.html) sensors and LED expressions. Visual indicators show when the microphone is recording, when the camera is on, and when Hoot is speaking.
- **English and Mandarin voice:** local [Faster-Whisper](https://github.com/SYSTRAN/faster-whisper) recognition and [Kokoro](https://github.com/hexgrad/kokoro) speech, or optional [Gemini transcription](https://ai.google.dev/gemini-api/docs/transcribe) and [TTS](https://ai.google.dev/gemini-api/docs/speech-generation). In cloud mode, Nemotron still writes the answers.
- **Reading and review:** captured pages, editable text, [pinyin](https://github.com/zh-lx/pinyin-pro), saved vocabulary, and practice cards, backed by local [SQLite](https://nodejs.org/api/sqlite.html).
- **Web dashboard:** [Vite](https://vite.dev/guide/) and [Three.js](https://threejs.org/docs/) power the animated owl, camera preview, chat, and device controls.

[Image: A close view of the Sense HAT connected to the Jetson, with a pattern lit on its LED matrix.]

Caption:
The Sense HAT LEDs show expressions and microphone, camera, and speaking
status.

## Hoot on screen

[Image: Hoot]

Caption: Hoot on screen, with a much tidier home than my desk.

The dashboard runs in the browser. The Jetson serves it through a local web server, which connects to Hermes running on the device.

[Image: Architecture diagram: the browser dashboard connects to a local web server, which connects to Hermes. The server and agent run on the Jetson Orin Nano.]

Caption:
The dashboard is the interface; Hermes manages the agent and tools on the
Jetson.

It's a working prototype. In cloud mode, Hoot needs an internet connection to Nebius for the assistant's reasoning. Speech can already run locally, and a local mode could use a model running directly on the Jetson Orin Nano for the reasoning too.

## A shoulder owl?

I already have a battery for Hoot, and I want to work out a way to mount it on my shoulder. I'd also like to be able to position it in different places around a room, wherever it has a useful view of what I'm working on.

[Image: Holding the Jetson prototype at an angle during a handheld tilt test.]

Caption:
A handheld tilt test while I work on the physical setup.

The shoulder mount is still a plan. But having a little owl hanging out on my shoulder while I learn something sounds pretty fun.

[Image: Hoot expectations versus reality: an elaborate owl robot beside the blue owl face, green case, camera, and speaker of the current prototype.]

Caption: The plan vs. the prototype.

On the software side, I want faster responses, better reading alignment, and pronunciation help that notices when I hesitate or get stuck. Still plenty to work on, but I'm glad to have the pieces working together.

## Demo