01 / Why
A small instrument with an immediate invitation
Most music software begins with a panel of controls. Gesture Synth begins with a physical question: what would a sound do if you lifted one hand, crossed the frame, or pinched the air?
The project is designed for quick exploration, not for replacing a studio instrument. You can open it, choose Camera or Mouse Mode, make a note, and record a short performance without creating an account or installing software.
02 / How
Movement becomes pitch, tone, and expression
In Camera Mode, hand landmarks are detected in your browser. The instrument maps position and pinch gestures to musical controls, while the Web Audio engine produces the sound in real time. Mouse Mode offers the same core idea without camera access.
A built-in performance demonstrates the musical range before you grant any permission. The current arrangement is an original Gesture Synth recording of the public-domain melody “Twinkle, Twinkle, Little Star.”
03 / Privacy
Camera input stays on the device
Live camera analysis, the camera stream and hand landmarks stay on your device. An optional microphone is requested separately and mixed only into the recording, without speaker monitoring. Only after you confirm the purchase upload does the selected recording go to private storage before payment, including your voice if you enabled the microphone.
Playing, recording and full local preview are free without signing in. Single-recording video export costs US$9.99. Optional download packs show their price and included recordings before payment. No subscription. Sign in with Google or email and password to export. A recording can be downloaded for 30 days from single payment or pack unlock. Beat Jam MIDI note export remains free; it contains neither video nor the same audio. The browser attempts to keep a local preview for recovery for up to 24 hours; this is a recovery window, not guaranteed physical deletion or a backup. Browser storage may be cleared. Uploaded unpaid recordings are retained for up to 30 days. Recordings are available for 30 days from single payment or pack unlock, then removed through scheduled cleanup. Downloading again does not extend that period.
The full data and advertising disclosure is available in the Privacy Policy.
04 / Technology
Built on open web capabilities
Gesture Synth uses MediaPipe Tasks Vision for hand tracking, Tone.js and the Web Audio API for synthesis, and browser media APIs for local recording. The project is not sponsored by or affiliated with Google, browser vendors, or social platforms.
Questions, accessibility feedback, and bug reports are welcome at support@gesture-synth.com.