This past weekend we attended an Elsa themed birthday party celebrating the 3rd birthday of our dear friends daughter. They gave all the kids magic wands so my son and I worked on creating this little web game when we got home. He loved waving his wand and making it snow on the screen! Of course, we had equal fun just using our imaginations and melting the snow with our fire breath. ROOOARRR.
How It Works
This game combines several web technologies:
- MediaPipe Hands: Google’s machine learning model that detects and tracks 21 hand landmarks in real-time
- Three.js + React Three Fiber: For rendering the 3D snowflakes with realistic physics
- Media Stream API: Standard browser API to access your device’s camera locally
Features
- Real-time hand tracking using the index finger tip (landmark #8)
- Magic Wand Visualization: A glowing 3D wand that tracks your finger orientation and position
- Melodic Feedback: Snow generation triggers notes from a certain song using the Web Audio API
- Dynamic Physics: Snowflake velocity and direction influenced by wand movement
- Motion Streaks: Visual trail effects triggered by high-velocity gestures
- Particle system with gravity, wind-sway, and variable drift
- Mirrored video feed for natural interaction
Privacy Note
All hand tracking happens entirely in your browser - no video or data is sent to any server. The camera feed is processed locally using MediaPipe’s client-side machine learning models.
Future Enhancements (Wishlist)
- Depth perception: Use body segmentation to make snow appear behind you
- Snow accumulation: Track surfaces where snow can “stick”
- Two-hand gestures: Use both hands for more complex effects
- Color customization: Change snow to different colored particles
Try waving your hand around and experiment with different speeds and movements to see what effects you can create!