Re: [linux-audio-dev] midi events in jack callback / ALSA Sequencer

Re: [linux-audio-dev] midi events in jack callback / ALSA Sequencer

[Frank van de Pol]

Martijn, 

the ALSA sequencer is exactly designed to serve a mechanism to route (midi)
_events_ from a source to a sink (either driver, application or whatever). 

On Mon, Aug 19, 2002 at 07:12:47PM +0100, Martijn Sipkema wrote:
> > >MIDI through and any other 'immediate' type MIDI messages do
> > >not need to be scheduled, they can be written to the interface
> immediately.
> >
> > Yes, they could. It would however necessitate different input routes
> > for 'immediate' and 'queued' events to the MIDI output handler.
> 
> The MIDI I/O API I am working on has 'scheduled' and 'immediate' queues. I
> don't think there is a way around this unless 'immediate' messages are not
> used
> at all and that is clearly not an option.

Within ALSA we have two priority queues, one for tick (bar,beat) scheduled
events, and one for clock (ns) scheduled events. In case of immediate
scheduling the priority queue is bypassed and the event submitted in the
receiver's fifo (which would be your immediate queue).

Due to potential blocking at the receivers you'll need a fifo for every
destination. 

Reason for having 2 priority queues with different reference is to cope with
tempo/signature changes while remaining in sync. The clock and tick priority
queues are in fact parallel in ALSA. 

Since especially for soft synths (but also for some -undocumented!- USB midi
interfaces, like Emagic AMT) the events need to be scheduled ahead (sort of
a pre-delay, say 10ms or more) to let the device/softsynth handle the micro
scheduling, it would seem a good idea to handle this at the clock based
queue. Since variable predelay in ms would be not quite friendly to the tick
based queue (different units), it might make sense to have the tick based
queue send events into the clock based queue instead of immediate delivery).

Your links about UST sound very good in this perspective, since the
_monotonic_ clock ensures compatibility between the event (sequencer)
subsystem, and other audio/video subsystems. 

The current clock based ALSA prioq could then simply be a reference against
the UST instead of keeping track of time itself. High level applications can
have the freedom to take whatever route they want, use the ALSA sequencer
API or go directly to the UST based scheduler. All these applications can
still cooperate in this framework :-) (try that in MS Windows.....)

A good reason for applications to use (UST/ALSA) scheduling instead of
taking care of micro scheduling itself and using rawmidi interfaces is
better support for softsynths to trigger at the right spot in the buffer,
and for the upcoming smart (eg. USB) midi devices.

> 
> > This
> > would not help make things simpler. It would also mean that a Sysex
> > or pitch/CC burst routed through can delay MIDI clocks because of the
> > limited bandwidth on the MIDI wire.
> 
> Sysex can hurt timing for other events, but that's MIDI. MIDI clock (any
> MIDI realtime message) can interleave other messages. And yes, merging
> MIDI streams is not easy.

You can't overcome the limits of the MIDI physical line if that's your
target transport. However when sending events to soft- or onboard synths
these limits are different (typically less of an issue). 

When using events instead of midi bytes the merging is a no brainer (and
leaves room for events not defined in midi spec).

> 
> > Thinking about it -- it's hypothetical because we don't have them in
> > Linux yet -- I believe a decent MIDI out handler using a firm timer
> > would be an order of magnitude more complicated than one based on the
> > RTC. Have you coded one yet?
> 
> Yes, and it is not that complex I think. Note that this would only have to
> be done
> in a driver process or a user-space sequencer application and not for every
> client
> application.
> 
> I'll try to get a version of my MIDI I/O API/framework ready, but it will
> probably still
> take me some time to get finished.

Perhaps you might want to look at the ALSA sequencer, and take it to a
higher level? I'd love to see UST time references...

> 
> --martijn
> 

-- 
+---- --- -- -  -   -    - 
| Frank van de Pol                  -o)    A-L-S-A
| FvdPol@home.nl                    /\\  Sounds good!
| http://www.alsa-project.org      _\_v
| Linux - Why use Windows if we have doors available?


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: [linux-audio-dev] midi events in jack callback / ALSA Sequencer

[Martijn Sipkema]

[...]
> Within ALSA we have two priority queues, one for tick (bar,beat) scheduled
> events, and one for clock (ns) scheduled events.

As MIDI uses MIDI tick messages for time based sync and MIDI clock messages
for tempo based sync I kind of feel the ALSA sequencer naming is a little
confusing :)

> In case of immediate
> scheduling the priority queue is bypassed and the event submitted in the
> receiver's fifo (which would be your immediate queue).
>
> Due to potential blocking at the receivers you'll need a fifo for every
> destination.

Correct, that's what I've been calling queues.

> Reason for having 2 priority queues with different reference is to cope
with
> tempo/signature changes while remaining in sync. The clock and tick
priority
> queues are in fact parallel in ALSA.

I don't want to support tempo (MIDI clock) scheduling in my MIDI API. This
could be better handled in the application itself. Also, when slaved to MIDI
clock
it is no longer possible to send messages ahead of time, and not supporting
this
in the API makes that clear to the application programmer.

> Since especially for soft synths (but also for some -undocumented!- USB
midi
> interfaces, like Emagic AMT)

Yes, I've repeatedly asked Emagic for documentation on their AMT protocol
without success. :(

> the events need to be scheduled ahead (sort of
> a pre-delay, say 10ms or more) to let the device/softsynth handle the
micro
> scheduling, it would seem a good idea to handle this at the clock based
> queue. Since variable predelay in ms would be not quite friendly to the
tick
> based queue (different units), it might make sense to have the tick based
> queue send events into the clock based queue instead of immediate
delivery).

I'd rather see MIDI clock sync handled in the application. This also keeps
the
API cleaner.

[...]
> A good reason for applications to use (UST/ALSA) scheduling instead of
> taking care of micro scheduling itself and using rawmidi interfaces is
> better support for softsynths to trigger at the right spot in the buffer,
> and for the upcoming smart (eg. USB) midi devices.

Or even MWPP.

[...]
> You can't overcome the limits of the MIDI physical line if that's your
> target transport. However when sending events to soft- or onboard synths
> these limits are different (typically less of an issue).
>
> When using events instead of midi bytes the merging is a no brainer

I was planning on doing that, but even then there are issues with for
example
(N)RPNs.

> leaves room for events not defined in midi spec).

...I'm not sure that is a good idea. What kind of events?

--martijn





-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Re: [linux-audio-dev] midi events in jack callback / ALSA Sequencer

[Frank van de Pol]

On Wed, Aug 21, 2002 at 02:10:35AM +0100, Martijn Sipkema wrote:
> [...]
> > Within ALSA we have two priority queues, one for tick (bar,beat) scheduled
> > events, and one for clock (ns) scheduled events.
> 
> As MIDI uses MIDI tick messages for time based sync and MIDI clock messages
> for tempo based sync I kind of feel the ALSA sequencer naming is a little
> confusing :)

I understand. OTOH the problem might also be introduced back in the 80's
when MIDI 1.0 spec was introduced. The "MIDI Clock" (0xf8) message was
introduced with only beat sync in mind. I believe the phrase 'tick' to
specify song position came from the standard midi file (SMF 1.0)
specification.

In fact the definitions in seq_event.h are less confusing:

/** Real-time data record */
typedef struct snd_seq_real_time {
        unsigned int tv_sec;            /**< seconds */
        unsigned int tv_nsec;           /**< nanoseconds */
} snd_seq_real_time_t;

/** (MIDI) Tick-time data record */
typedef unsigned int snd_seq_tick_time_t;

/** unioned time stamp */
typedef union snd_seq_timestamp {
        snd_seq_tick_time_t tick;       /**< tick-time */
        struct snd_seq_real_time time;  /**< real-time */
} snd_seq_timestamp_t;


> 
> I don't want to support tempo (MIDI clock) scheduling in my MIDI API. This
> could be better handled in the application itself. Also, when slaved to MIDI
> clock
> it is no longer possible to send messages ahead of time, and not supporting
> this
> in the API makes that clear to the application programmer.

The concept of the public available alsa timing queues allow outboard
applications to schedule events in a tempo locked fashion. Think of
applications like drum machines, appegiators, tempo delay etc. Of course you
might be using one big monolithic (cubase?) sequencer application which does
do everything you need....

Compare it to an external (outboard) drum machine you're using since
programming a pattern in it so user friendly (hence you end up being more
creative), and sync that from your favorite sequencer which has the tempo
map for your song.

Of course all of this could be done without a queue which supports tempo
scheduling, but then you'll need to emit MIDI Clock events and rely on
immediate processing. In case a soft synth (or other potentially high
latency device) is triggered from the MIDI Clock you loose the ability to
correct for this.

[...] 
> > Since especially for soft synths (but also for some -undocumented!- USB
> midi
> > interfaces, like Emagic AMT)
> 
> Yes, I've repeatedly asked Emagic for documentation on their AMT protocol
> without success. :(

let them burn in peace.... 

[...]
> > When using events instead of midi bytes the merging is a no brainer
> 
> I was planning on doing that, but even then there are issues with for
> example
> (N)RPNs.
> 
> > leaves room for events not defined in midi spec).
> 
> ...I'm not sure that is a good idea. What kind of events?

eg.

- system events like announcements of topology changes
- (N)RPNs as a 14 bit value instead of 2x 7bit
- SMF like meta data
- controls for current sequencer queue: tempo, position, etc.

to name a few

Cheers,
Frank.

-- 
+---- --- -- -  -   -    - 
| Frank van de Pol                  -o)    A-L-S-A
| FvdPol@home.nl                    /\\  Sounds good!
| http://www.alsa-project.org      _\_v
| Linux - Why use Windows if we have doors available?


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Re: [linux-audio-dev] midi events in jack callback / ALSA Sequencer

[Martijn Sipkema]

> > I don't want to support tempo (MIDI clock) scheduling in my MIDI API.
This
> > could be better handled in the application itself. Also, when slaved to
MIDI
> > clock
> > it is no longer possible to send messages ahead of time, and not
supporting
> > this
> > in the API makes that clear to the application programmer.
>
> The concept of the public available alsa timing queues allow outboard
> applications to schedule events in a tempo locked fashion. Think of
> applications like drum machines, appegiators, tempo delay etc. Of course
you
> might be using one big monolithic (cubase?) sequencer application which
does
> do everything you need....

No, I like the concept of a lot of small applications. And I want those to
be able
to sync (slave) to MIDI clock.

> Compare it to an external (outboard) drum machine you're using since
> programming a pattern in it so user friendly (hence you end up being more
> creative), and sync that from your favorite sequencer which has the tempo
> map for your song.

For this, the sequencer is master and just sends the MIDI messages including
MIDI clock to a scheduled UST queue. Timing will be as good as is possible
with a MIDI clock slaved external instrument.

> Of course all of this could be done without a queue which supports tempo
> scheduling, but then you'll need to emit MIDI Clock events and rely on
> immediate processing.

Isn't that what will eventually happen using a tempo queue also?

> In case a soft synth (or other potentially high
> latency device) is triggered from the MIDI Clock you loose the ability to
> correct for this.

I don't see how having a MIDI clock scheduled tree will help. When slaving
to MIDI clock you cannot know when a future beat will occur, so a software
synth salved to MIDI clock will behave just like a software synth in real
time,
i.e. it cannot compensate for its own latency. However, if the master sends
the MIDI clock messages ahead of time, and for a sequencer application this
is exactly what it would do, then the software synth can compensate for its
latency and might even interpolate between clocks. This would not be
possible
having a tempo based queue since the UST of the messages in that queue are
not known in advance, right?

> > > leaves room for events not defined in midi spec).
> >
> > ...I'm not sure that is a good idea. What kind of events?
>
> eg.
>
> - system events like announcements of topology changes

I'm thinking of handling these out-of-band.

> - (N)RPNs as a 14 bit value instead of 2x 7bit

This I would like to solve by having the posibility of sending a short
sequence
of MIDI messages that are guaranteed to not be interleaved, e.g. 4 messages,
two for setting the current (N)RPN and 2 for setting its value.

> - SMF like meta data

Is it really necessary to support karaoke :)

> - controls for current sequencer queue: tempo, position, etc.

These aren't needed when a tempo queue isn't used.

I think it is important to keep the API as simple as possible.

--martijn





-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

RE: Re: [linux-audio-dev] midi events in jack callback / ALSA Sequencer

[mikko.a.helin]

Why is it important to keep the API simple, shouldn't it be functional in first place and make the API usage simply?

Anyway (IMHO), there should really be an API which combines audio and MIDI playback, recording and timing of events and makes it possible to keep MIDI in sync with audio (ALSA sequencer API seems to support only tick and time synchronization modes, no audio clock sync mode). Professional API's like ASIO and VST (and DirectMusic) synchronizes everything to audio clock (sample position). Even when MTC is used the audio clock is the main clock source, digital I/O or world clock is used to sync the audio clock to the audio clock of the external device. There is no need for other sync modes than audio (_maybe the tick mode could be useful if ALSA takes care of sending MIDI clock, SPP, MTC etc. data to external devices). Sequencer application can easily convert sample position from the beginning of the song to measures / beats, time etc. whatever is needed when the time signature, tempo and sample rate is known - reading the sample position from the sound card is no problem at all, PCI chipset's DMA registers have usually such current play position register, or if not, the sample position can be calculated by measuring the time between IRQ's and calculating the time between current time and the time when last interrupt was received.

-Mikko

> etc. Of course
> you
> > might be using one big monolithic (cubase?) sequencer 
> application which
> does
> > do everything you need....
> 
> No, I like the concept of a lot of small applications. And I 
> want those to
> be able
> to sync (slave) to MIDI clock.
> 
> 
> I think it is important to keep the API as simple as possible.
> 
> --martijn


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Re: [linux-audio-dev] midi events in jack callback / ALSA Sequencer

[Martijn Sipkema]

> Why is it important to keep the API simple, shouldn't it be functional in
first place and make the API usage simply?

Who says a simple API can't be functional?

> Anyway (IMHO), there should really be an API which combines audio and MIDI
playback, recording and timing of events and makes it possible to keep MIDI
in sync with audio (ALSA sequencer API seems to support only tick and time
synchronization modes, no audio clock sync mode). Professional API's like
ASIO and VST (and DirectMusic) synchronizes everything to audio clock
(sample position). Even when MTC is used the audio clock is the main clock
source, digital I/O or world clock is used to sync the audio clock to the
audio clock of the external device. There is no need for other sync modes
than audio (_maybe the tick mode could be useful if ALSA takes care of
sending MIDI clock, SPP, MTC etc. data to external devices). Sequencer
application can easily convert sample position from the beginning of the
song to measures / beats, time etc. whatever is needed when the time
signature, tempo and sample rate is known - reading the sample position from
the sound card is no problem at all!

Well, I disagree. I think using UST is better. You might want to use MIDI
without audio. Also
MIDI hardware that supports scheduling will not support scheduling to the
audio position. Using
UST just means mapping different clocks to one common clock.

> , PCI chipset's DMA registers have usually such current play position
register, or if not, the sample position can be calculated by measuring the
time between IRQ's and calculating the time between current time and the
time when last interrupt was received.

The current sample position does not tell me anything about when some
earlier sample occured or
a later sample will occur. In your example you already use two UST/MSC
values to estimate
the current MSC. Allthough it is possible on some hardware to read the
current sample position
(MSC) it is not possible to the OS (or other hardware) to schedule on it and
so you'll have to
convert to another clock at some point.

--martijn







-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390