Reading MIDI Files

MIDI is a specification for music, describing how electronic musical instruments and computers can communicate with each other.

Unlike digital audio formats such as MP3, the Standard MIDI File does not contain sounds, but rather a stream of instructions for playing notes, volume, tempo, and sound effects, as well as track names and other descriptive information. Because of this, MIDI files tend to be much smaller and typically allow the music to be easily rearranged or edited.

Using Factor, we will be creating a parser for reading MIDI files in under 180 lines of code.

Variable-Length Quantity

Some integers will be encoded as variable length, using 7 bits per byte with one bit reserved for the stop bit (indicating you have finished reading the number). This means the numbers 0 through 127 can be encoded in a single byte, but larger numbers will require additional bytes.

: read-number ( -- number )
    0 [ 7 bit? ] [
        7 shift read1 [ 0x7f bitand + ] keep
    ] do while ;

MIDI Events

There are three types of events: MIDI events, system-exclusive events, and meta events. The majority of events will usually be MIDI events, so we will parse those first.

Some MIDI events will include the channel in 4 bits of the status byte, so we handle those separately from the system common and realtime messages.

TUPLE: midi-event delta name value ;

C: <midi-event> midi-event

: read-message ( delta status -- message )
    dup 0xf0 < [
        [
            ! channel messages
            [ 0x0f bitand "channel" ,, ] [ 0xf0 bitand ] bi {
                { 0x80 [ "note-off"
                    read1 "note" ,, read1 "velocity" ,, ] }
                { 0x90 [ "note-on"
                    read1 "note" ,, read1 "velocity" ,, ] }
                { 0xa0 [ "polytouch"
                    read1 "note" ,, read1 "value" ,, ] }
                { 0xb0 [ "control-change"
                    read1 "control" ,, read1 "value" ,, ] }
                { 0xc0 [ "program-change"
                    read1 "program" ,, ] }
                { 0xd0 [ "aftertouch"
                    read1 "value" ,, ] }
                { 0xe0 [ "pitchwheel"
                    read1 read1 7 shift + "pitch" ,, ] }
            } case
        ] H{ } make
    ] [
        {
            ! system common messages
            { 0xf0 [ "sysex" { 0xf7 } read-until drop ] }
            { 0xf1 [ "quarter-made" [
                    read1
                    [ -4 shift "frame-type" ,, ]
                    [ 0x0f bitand "frame-value" ,, ] bi
                ] H{ } make ] }
            { 0xf2 [ "songpos" read1 read1 7 shift + ] }
            { 0xf3 [ "song-select" read1 ] }
            { 0xf6 [ "tune-request" f ] }

            ! real-time messages
            { 0xf8 [ "clock" f ] }
            { 0xfa [ "start" f ] }
            { 0xfb [ "continue" f ] }
            { 0xfc [ "stop" f ] }
            { 0xfe [ "active-sensing" f ] }
            { 0xff [ "reset" f ] }
        } case
    ] if <midi-event> ;

Meta Events

Meta events contain descriptive information such as track name, tempo and time signatures. They are also used to indicate the end of the track has been reached.

TUPLE: meta-event delta name value ;

C: <meta-event> meta-event

: parse-meta ( status bytes -- name value )
    swap {
        { 0x00 [ 2 head be> "sequence-number" ] }
        { 0x01 [ utf8 decode "text" ] }
        { 0x02 [ utf8 decode "copyright" ] }
        { 0x03 [ utf8 decode "track-name" ] }
        { 0x04 [ utf8 decode "instrument-name" ] }
        { 0x05 [ utf8 decode "lyrics" ] }
        { 0x06 [ utf8 decode "marker" ] }
        { 0x07 [ utf8 decode "cue-point" ] }
        { 0x09 [ utf8 decode "device-name" ] }
        { 0x20 [ first "channel-prefix" ] }
        { 0x21 [ first "midi-port" ] }
        { 0x2f [ drop t "end-of-track" ] }
        { 0x51 [ 3 head be> "set-tempo" ] }
        { 0x54 [
            [
                5 firstn {
                    [
                        [ -6 shift "frame-rate" ,, ]
                        [ 0x3f bitand "hours" ,, ] bi
                    ]
                    [ "minutes" ,, ]
                    [ "seconds" ,, ]
                    [ "frames" ,, ]
                    [ "subframes" ,, ]
                } spread
            ] H{ } make "smpte-offset" ] }
        { 0x58 [
            [
                first4 {
                    [ "numerator" ,, ]
                    [ 2 * "denominator" ,, ]
                    [ "clocks-per-tick" ,, ]
                    [ "notated-32nd-notes-per-beat" ,, ]
                } spread
            ] H{ } make "time-signature" ] }
        { 0x59 [ "key-signature" ] }
        { 0x7f [ "sequencer-specific" ] }
    } case swap ;

: read-meta ( delta -- event )
    read1 read-number read parse-meta <meta-event> ;

Sysex Events

For system-exclusive events, which are typically a sequence of bytes that are proprietary to particularly MIDI devices, we just preserve the type (0xf0 or 0xf7) and raw bytes.

TUPLE: sysex-event delta status bytes ;

C: <sysex-event> sysex-event

: read-sysex ( delta status -- event )
    read-number read <sysex-event> ;

Reading Events

We can now read all types of events, dispatching on the status byte.

: read-event ( delta status -- event )
    {
        { 0xf0 [ 0xf0 read-sysex ] }
        { 0xf7 [ 0xf7 read-sysex ] }
        { 0xff [ read-meta ] }
        [ read-message ]
    } case ;

Status bytes can be "running", which means that for channel events they can be dropped from the stream if they are identical to the previous MIDI channel event. Meta events (0xff) do not set the running status.

: read-status ( prev-status -- prev-status' status )
    peek1 dup 0x80 < [
        drop dup
    ] [
        drop read1 dup 0xff = [
            nip dup
        ] unless
    ] if ;

Each event has a header that is the delta-time (encoded as a variable length integer) and the status (which may not be present if it is "running").

: read-event-header ( prev-status -- prev-status' delta status )
    [ read-number ] dip read-status swapd ;

There are a few ways to parse all events from a byte-array, but I thought it was a good opportunity to try out peekable streams, checking if the next event is present.

: parse-events ( data -- events )
    binary <byte-reader> <peek-stream> [
        f [
            peek1 [ read-event-header ] [ f f ] if dup
        ] [ read-event ] produce 2nip nip
    ] with-input-stream ;

Reading MIDI

MIDI files are grouped into a series of chunks. The first chunk is a MIDI header indicating the format (single or multiple simultaneous tracks), number of tracks in the file, and division (indicating how to interpret the delta-times in the file).

TUPLE: midi-header format #chunks division ;

: <midi-header> ( bytes -- header )
    2 cut 2 cut [ be> ] tri@ midi-header boa ;

Typically, that is followed by MIDI tracks, each containing a series of events.

TUPLE: midi-track events ;

: <midi-track> ( bytes -- track )
    parse-events midi-track boa ;

Reading the chunks in the file dispatch off the "chunk type":

: read-chunk ( -- chunk )
    4 read 4 read be> read swap {
        { $[ "MThd" >byte-array ] [ <midi-header> ] }
        { $[ "MTrk" >byte-array ] [ <midi-track> ] }
    } case ;

To read a MIDI stream, we read the header and then all the chunks in the file, storing them in a midi tuple.

TUPLE: midi header chunks ;

C: <midi> midi

: read-header ( -- header )
    read-chunk dup midi-header? t assert= ;

: read-chunks ( header -- chunks )
    #chunks>> [ read-chunk ] replicate ;

: read-midi ( -- midi )
    read-header dup read-chunks <midi> ;

Parsing a MIDI from raw bytes or a file:

: >midi ( byte-array -- midi )
    binary [ read-midi ] with-byte-reader ;

: file>midi ( path -- midi )
    binary [ read-midi ] with-file-reader ;

This is available now in the midi vocabulary.