← All projects

KioKI

An AI-powered drum transcription tool that turns audio into an editable, playable groove.

KioKI

Problem

Automated drum transcription has been a research topic for years, but the outputs are usually read-only transcriptions you can't modify or build on. Producers and drummers want the reverse: capture a groove you heard or played, and get something you can actually edit, loop, and ship.

Approach

A React + TypeScript + Vite web app wrapped in Capacitor 6 for iOS and Android. The AI pipeline runs server-side on FastAPI + Python + Railway: librosa extracts onsets, a custom convolutional classifier ('KioDrum', six drum classes — kick, snare, hi-hat, tom, crash, ride) tags each onset, and the hits are aligned into a quantised pattern. The output opens directly in the editor as a starting point, not an endpoint — the user tweaks hits, adds measures, swings, adjusts velocity, and plays back with real samples. Exports include PDF (via jsPDF + VexFlow), General MIDI drum map, and MusicXML 4.0.

Stack

ReactTypeScriptCapacitorFastAPIlibrosa

Synopsis

KioKI covers the full transcription loop — audio in, onset detection, drum-voice classification, quantisation, and a grid editor. The key product decision is that the model output is editable, not authoritative: producers and drummers get something they can tweak, loop, and ship, rather than a read-only transcription they can't build on.

Gallery

The editable drum grid

A step grid with per-instrument rows, swing, time signature, and velocity controls. AI transcriptions land here as a starting point — the user adds measures, tweaks hits, and plays back with real samples.
A step grid with per-instrument rows, swing, time signature, and velocity controls. AI transcriptions land here as a starting point — the user adds measures, tweaks hits, and plays back with real samples.

Audio in, chart out

Upload an audio file and the server-side pipeline runs onset detection, drum-voice classification, and quantisation. The output opens directly in the editor, ready to edit.
Upload an audio file and the server-side pipeline runs onset detection, drum-voice classification, and quantisation. The output opens directly in the editor, ready to edit.

Notation and export

Rendered drum notation with export to PDF (via jsPDF + VexFlow), General MIDI drum map, and MusicXML 4.0. The editor is for composing; the score view is for handing a chart to anyone else in the chain — producer, drummer, teacher.
Rendered drum notation with export to PDF (via jsPDF + VexFlow), General MIDI drum map, and MusicXML 4.0. The editor is for composing; the score view is for handing a chart to anyone else in the chain — producer, drummer, teacher.

Chart library

Every chart the user has transcribed or composed, tagged with BPM, bar count, and hit count. Import and export via .kioki JSON keep charts portable between projects.
Every chart the user has transcribed or composed, tagged with BPM, bar count, and hit count. Import and export via .kioki JSON keep charts portable between projects.