A theory for music to funscript

what is MIDI? midi is a standard old as hell.
It has 2 parts.
On the one hand, there are the midi events that are a port used by dj consoles and instruments. this is used when you connect a piano or dj console to the computer.
in the other part are the mid files that are used to express music in the form of musical notes and instruments. no audio just notes and instruments, midi files are very very light, a few Kb.
Mid File Example: METALLICA.One.mid — BitMidi
Theoretically, reading the mid files and their different channels (instruments) could retrieve the notes of each channel and its time in the song to generate a movement in the funscript at a certain speed according to the pitch or some other midi property.
since ancient times there are also tools that allow you to convert audio music files to midi files. both in the form of applications and open source libraries.
this technique is used long ago by rhythm games such as audiosurft, ddr and others or Shazam

I personally don’t have the desire or time to do this. but I leave this theory since it seems to me a viable idea if someone wants to try to implement it

reference links:

6 Likes

There’s already MIDI plugins for Unity and Unreal Engine, so adding MIDI to software should be a fairly straightforward thing.

If this turns into a real project, I’d suggest using MIDI 2.0 as a the standard, since it has a lot more resolution and good support for advanced features.

1 Like