AMTFlow
AMTFlow is a modular framework for multi-instrument music transcription: it turns a song into MIDI in which every note carries its instrument. It first works out which instruments are playing, then tells a pretrained transcription model (by default MuScriptor) which ones to write. Without that information, MuScriptor often hears a whole band as a piano, or writes a piano as a distorted guitar.
uv run amtflow transcribe song.wav --out out/
Results
On 392 real pop recordings (MulTTiPop test set), AMTFlow assigns more notes to the right instrument than the model it drives: +4.3 family F1 and +1.7 exact-instrument F1 over MuScriptor without tags, while finding the same notes.
| system | instrument family | exact | agnostic | |
|---|---|---|---|---|
| MT3 | 15.5 | 10.7 | 28.1 | |
| YourMT3+ | 19.6 | 11.5 | 29.3 | |
| MuScriptor medium, no instrument tags | 29.4 | 17.2 | 43.0 | |
| AMTFlow (same MuScriptor medium) | 33.7 | 18.9 | 43.1 | |
Note-onset F1 ×100 (50 ms), mean over segments. 95% CIs (paired bootstrap): family [+3.4, +5.1], exact [+1.1, +2.4]. MT3, YourMT3+ and MuScriptor scores use the transcriptions published with MulTTiPop. Full results
Listen: where MuScriptor needs instrument tags
Each example below is a recording on which MuScriptor without tags picks the wrong instruments, and AMTFlow does not. Switch between the three transcriptions while the recording plays.