Hacker News new | past | comments | ask | show | jobs | submit
cvwright | on: Ask HN: What are you working on? (November 2026)
I’ve been building a little Python tool to extract a MIDI drum track from an mp3 or other song file. Eventually I want to buy an e-drum kit and learn to play along with my favorite songs.

From what I can tell, we’re about even with the current state of the art in academic publications right now, at least on MDB. Runs faster than real time in pytorch on a few years old MacBook.

It even does a decent job creating easy/medium/hard versions from the original “expert” track. I compared ours to some charts from Rhythmverse and while we’re not as good as the official ones, we’re pretty darn close.

Spent some time trying to extract an instrument model for the drum sounds from the recording too, but I think that simply requires too much data for lots of the songs that I like where the mix is really dense with a ton of sound from distorted guitars and cymbals etc. Works a treat on super clean mixes like Back In Black though.

Next step is probably to create my own rhythm game, since my initial experience with Clone Hero and YARG left a lot to be desired.

XCSme | parent
That sounds cool. For my uni project 10 years ago I chose to use neural networks to detect guitar notes, but not by pitch, but to detect exactly which string/fret was played (because you can play same note on different strings). It worked okish with a very basic network, quite suprised. I made a small game you could play with your guitar.

Since then I wanted to develop further such a game, and now I'm almost ready to release https://ultimidi.com , check it out, it's not really a rhythm game, but my goal is to add more mini-games to it (including rhythm games) and more instruments (now has piano, but planning to add guitar, ukulele too, not sure if drums would work too).

cvwright | root | parent
Very nice! Extracting the guitar track would be awesome but it’s so much hardier than drums. Even cymbals are hard. And guitars have chords and distortion and…
XCSme | root | parent
Yeah, that's why my idea was to use neural nets trained on manually labeled data, instead of relying solely on pitch detection.

Also, probably AIs nowadays can help a lot with generating training data, and even improving detection.

That being said, even SOTA voice recognition has only like 95% accuracy, so it gets wrong 1 in 20 words. Imagine if transcribing a melody and 1 in 20 notes are wrong, for a fast melody that would mean a wrong note every few bars.