Inspiration
Caring for a newborn means your hands are constantly occupied. I wanted a way to keep learning Chinese during those long hours of rocking or soothing a baby. Most flashcard apps require taps, swipes, or clicks. That doesn’t work when you’re holding a baby. So I built a tool that doesn’t require hands at all. It also serves people with limited hand mobility or anyone who prefers a low-effort way to study.
What it does
Hands Free Chinese is a web app that lets you study Chinese flashcards using just head tilts and sound.
- Tilt your head left or right to move through flashcards
- Say the word you think the card shows to trigger the reveal
There are no buttons, taps, or swipes. Just your face and your voice. You can use it while multitasking or relaxing.
How I built it
- Bolt quickly scaffold and built the entire app. It handled much of the heavy lifting so I could focus on interaction design.
- Mediapipe: I used Google’s Mediapipe to track head tilts with a webcam and translate those into navigation inputs.
- Volume Detection: I implemented basic microphone input to detect when a user says the answer out loud. This keeps the experience light and responsive without needing full speech recognition.
- React: I used React to build the frontend and manage state transitions cleanly.
Challenges I ran into
- Calibrating the head tilt thresholds so they don’t trigger accidentally but still feel responsive
- Filtering out ambient noise so only intentional sounds trigger the reveal
- Handling browser microphone permissions smoothly across platforms
Accomplishments that I’m proud of
- Created a fully hands-free flashcard app that works in real conditions
- Made learning accessible to parents and users with limited mobility
- Used modern tools to go from concept to working app in a short amount of time
What I learned
- Bolt is surprisingly fast for getting real-world ideas off the ground
- Head and sound-based interactions feel natural once dialed in, but require careful tuning
- You don’t need full speech recognition to make voice interaction useful
What’s next for Hands Free Chinese
- Add support for custom vocab decks
- Improve volume detection with better noise handling and sensitivity controls
- Explore mobile browser optimization
- Add optional features like spaced repetition and learning stats
- Support deck creation by uploading images (menus, signs, textbooks) and extracting Chinese characters you want to learn from them
- Potentially expand to other subjects like hands-free geography or math practice