Forcing an absolute timestamp for every midi event.

Forcing an absolute timestamp for every midi event.

[Juan Linietsky]

Hi! I wanted to ask, how about forcing
an absolute timestamp for _every_ midi event?
I think this would be great for softsynths,
so they dont need to work with root/schedfifo/lowlatency
to have a decent timing. Not allways you are willing
to process midi at the lowest latency possible.
I say because you dont really need all that if you
sequence in the computer and control softsynths and maybe
some external device. 
This way, the softsynth gets the event with the timestamp,
gets the current time, substracts the audio delay (latency) to that
and just mixes internally in smaller blocks processing each 
event in the right time.


Juan Linietsky








-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Patrick Shirkey]

Juan Linietsky wrote:
> Hi! I wanted to ask, how about forcing
> an absolute timestamp for _every_ midi event?
> I think this would be great for softsynths,
> so they dont need to work with root/schedfifo/lowlatency
> to have a decent timing. Not allways you are willing
> to process midi at the lowest latency possible.
> I say because you dont really need all that if you
> sequence in the computer and control softsynths and maybe
> some external device. 
> This way, the softsynth gets the event with the timestamp,
> gets the current time, substracts the audio delay (latency) to that
> and just mixes internally in smaller blocks processing each 
> event in the right time.
> 

Yesterday Juan gave me an example of the difference between having this 
and not and it is absolutely clear that the sound quality is better. I 
think many users would appreciate the difference even if they wouldn't 
know why.

Currently if users want to get this kind of quality they have to have 
root access so it will be less hassle to have it available from user space.

It will also make ALSA one step more advanced :)


-- 
Patrick Shirkey - Boost Hardware Ltd.
For the discerning hardware connoisseur
Http://www.boosthardware.com
Http://www.boosthardware.com/LAU/guide/
========================================



-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Frank van de Pol]

On Sun, Aug 18, 2002 at 05:57:40AM -0300, Juan Linietsky wrote:
> Hi! I wanted to ask, how about forcing
> an absolute timestamp for _every_ midi event?

It's not 100% clear to me wat you mean. When using the sequencer api you
already have a timestamp (either tick/clock) for every event scheduled. Midi
data captured from eg. the midi input port is timestamped upon receive IRQ.

what is missing?


> I think this would be great for softsynths,
> so they dont need to work with root/schedfifo/lowlatency
> to have a decent timing. Not allways you are willing
> to process midi at the lowest latency possible.
> I say because you dont really need all that if you
> sequence in the computer and control softsynths and maybe
> some external device. 
> This way, the softsynth gets the event with the timestamp,
> gets the current time, substracts the audio delay (latency) to that
> and just mixes internally in smaller blocks processing each 
> event in the right time.

agreed; this is the whole concept of providing scheduling information
(ie timestamp) with the events. One hairy bit is the handling of time based
events versus tick based events in case your sequencer needs to cope with
a variable tempo or sync to midi clock sources.

Frank

-- 
+---- --- -- -  -   -    - 
| Frank van de Pol                  -o)    A-L-S-A
| FvdPol@home.nl                    /\\  Sounds good!
| http://www.alsa-project.org      _\_v
| Linux - Why use Windows if we have doors available?


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Juan Linietsky]

On Sun, 18 Aug 2002 14:48:49 +0200
Frank van de Pol <fvdpol@home.nl> wrote:

> 
> 
> 
> On Sun, Aug 18, 2002 at 05:57:40AM -0300, Juan Linietsky wrote:
> > Hi! I wanted to ask, how about forcing
> > an absolute timestamp for _every_ midi event?
> 
> It's not 100% clear to me wat you mean. When using the sequencer api
> you already have a timestamp (either tick/clock) for every event
> scheduled. Midi data captured from eg. the midi input port is
> timestamped upon receive IRQ.

Well, thats what i thought too by watching the code, but I check any
received
event, from any source and all variables are all zero. Instead, alsa
promises to deliver events at the right time, which is fine.. but not
enough!

include/alsa/seq_event.h:row 421
This is the struct of any event I receive when polling the device,
on my sequencer client:

typedef struct snd_seq_event {
[..] which includes...
        snd_seq_timestamp_t time;
[..]
} snd_seq_event_t;

this struct has...

typedef union snd_seq_timestamp {
        snd_seq_tick_time_t tick;       /**< tick-time */
        struct snd_seq_real_time time;  /**< real-time */
} snd_seq_timestamp_t;

To which i wonder.. first, why is this an union? why cant both be
there?

well, anyway.. snd_seq_tick_time_t is an int,
and snd_seq_real_time is:

typedef struct snd_seq_real_time {
        unsigned int tv_sec;            /**< seconds */
        unsigned int tv_nsec;           /**< nanoseconds */
} snd_seq_real_time_t;

This should technically give me info about the absolute time in which
the event has to be played,
basically what i need in order to do a certain amount of offline
processing.
This way, my softsynth wont need extremely low audio latency to play
accurate timing.

But! All this struct is zeroed, _allways_ zeroed, no info is ever
proovided to my sequencer client.
I think this is either a design problem, or a serious bug in the api. 


Juan Linietsky


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Takashi Iwai]

At Sun, 18 Aug 2002 19:17:01 -0300,
Juan Linietsky wrote:
> 
> On Sun, 18 Aug 2002 14:48:49 +0200
> Frank van de Pol <fvdpol@home.nl> wrote:
> 
> > 
> > 
> > 
> > On Sun, Aug 18, 2002 at 05:57:40AM -0300, Juan Linietsky wrote:
> > > Hi! I wanted to ask, how about forcing
> > > an absolute timestamp for _every_ midi event?
> > 
> > It's not 100% clear to me wat you mean. When using the sequencer api
> > you already have a timestamp (either tick/clock) for every event
> > scheduled. Midi data captured from eg. the midi input port is
> > timestamped upon receive IRQ.
> 
> Well, thats what i thought too by watching the code, but I check any
> received
> event, from any source and all variables are all zero. Instead, alsa
> promises to deliver events at the right time, which is fine.. but not
> enough!
> 
> include/alsa/seq_event.h:row 421
> This is the struct of any event I receive when polling the device,
> on my sequencer client:
> 
> typedef struct snd_seq_event {
> [..] which includes...
>         snd_seq_timestamp_t time;
> [..]
> } snd_seq_event_t;
> 
> this struct has...
> 
> typedef union snd_seq_timestamp {
>         snd_seq_tick_time_t tick;       /**< tick-time */
>         struct snd_seq_real_time time;  /**< real-time */
> } snd_seq_timestamp_t;
> 
> To which i wonder.. first, why is this an union? why cant both be
> there?
 
because a tick time cannot be always mapped to the wallclock time.
the tempo can be changed at any time.

for incoming events, yes, both of them can be, though.
in such a case, you can connect twice the same route with different
flags (see below).


> well, anyway.. snd_seq_tick_time_t is an int,
> and snd_seq_real_time is:
> 
> typedef struct snd_seq_real_time {
>         unsigned int tv_sec;            /**< seconds */
>         unsigned int tv_nsec;           /**< nanoseconds */
> } snd_seq_real_time_t;
> 
> This should technically give me info about the absolute time in which
> the event has to be played,

yes.
but please note that the time is not system time but is a time of the
specified queue.

> basically what i need in order to do a certain amount of offline
> processing.
> This way, my softsynth wont need extremely low audio latency to play
> accurate timing.
> 
> But! All this struct is zeroed, _allways_ zeroed, no info is ever
> proovided to my sequencer client.
> I think this is either a design problem, or a serious bug in the api. 

the time-stamping on incoming events is enabled only "update_time"
flag is set at the subscription (connection), corresponding

void snd_seq_port_subscribe_set_time_update(snd_seq_port_subscribe_t *info, int val);
void snd_seq_port_subscribe_set_time_real(snd_seq_port_subscribe_t *info, int val);
void snd_seq_port_subscribe_set_queue(snd_seq_port_subscribe_t *info, int q);

the first two set "yes i'll update the timestamps of incoming events"
and "yes, i'll use the real (wallclock) time as timestamp values".
and the last one specifies the queue to be referred to.
if the queue is not running, the time is always same (if not started
at all, then is zero).


ciao,

Takashi


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Martijn Sipkema]

> Hi! I wanted to ask, how about forcing
> an absolute timestamp for _every_ midi event?
> I think this would be great for softsynths,
> so they dont need to work with root/schedfifo/lowlatency
> to have a decent timing. Not allways you are willing
> to process midi at the lowest latency possible.
> I say because you dont really need all that if you
> sequence in the computer and control softsynths and maybe
> some external device. 
> This way, the softsynth gets the event with the timestamp,
> gets the current time, substracts the audio delay (latency) to that
> and just mixes internally in smaller blocks processing each 
> event in the right time.

I'm not sure what you mean, but below is what I think would be the
best approach for audio/MIDI I/O.

----

UST = unadjusted system time
MSC = media stream count

see also:

http://www.lurkertech.com/lg/time/intro.html

(synchronous) Audio I/O API
- Callback based.
- All buffers in a callback are of the same size.
- All frames with a particular MSC occur at the same time. (I don't think
 OpenML requires this, EASI does have this with its 'position')
- The audio callback provides an MSC value for every buffer corresponding
 to the first frame in the buffer.
- The (constant) latency (in frames) between an input and an output can be
 seen from the difference between their MSC values.
- The audio callback provides an UST value for every input buffer
 corresponding to the end of that buffer/start of the next buffer.
- The UST value for the start/end of an output buffer can be estimated.

MIDI I/O API
- MIDI messages are received with a UST stamp.
- Timestamps are measured at the start of the message on the wire.
- MIDI messages are either sent immediately or scheduled to UST.
- MIDI messages must have monotonically increasing timestamps if
 scheduled.

For a software synthesizer the MIDI messages received at some UST
can be mapped to a corresponding MSC value and then rendered at
a constant offset from this MSC, most likely the audio I/O latency
(the audio I/O latency may not be the same for all inputs/outputs,
also since MIDI messages always arrive late a small extra latency
should be introduced in this case by increasing the MIDI message's
UST by a constant value in order to compensate for the MIDI interface
and scheduling latency so they arrive a little early instead of late
in order to reduce jitter with MIDI messages arriving near buffer
bounderies).
MIDI -> audio output latency would then be slightly higher (say 2ms,
noting that transmitting a note-on message already takes 3*320usec)
then audio I/O latency.



--martijn





-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: [linux-audio-dev] Re: Forcing an absolute timestamp for every midi event.

[Frank van de Pol]

Hi Martijn,

I read the pages explaining the position of UST for synchronising audio/midi
etc. It really looks good! 

It this moment the ALSA sequencer API supports already accurate timestamping
of incoming midi data, but since the it is not coupled to the audio streams
synchronisation is still tricky unless you use the audio frame interrupt as
a clock source. 

Over time I'd like to migrate the more complex portions of the ALSA
sequencer from kernel to user space; leaving only the minimal required
functions in kernel (eg. timestamping/micro scheduling). All of this has yet
to be determined of course, it are just thoughts for the future direction. 

Is there already a commonly available UST on linux? To my knowledge the only
thing that comes close is the (cpu specific) cycle counter.

Frank.


On Sun, Aug 18, 2002 at 09:12:52PM +0100, Martijn Sipkema wrote:
> I'm not sure what you mean, but below is what I think would be the
> best approach for audio/MIDI I/O.
> 
> ----
> 
> UST = unadjusted system time
> MSC = media stream count
> 
> see also:
> 
> http://www.lurkertech.com/lg/time/intro.html
> 
> (synchronous) Audio I/O API
> - Callback based.
> - All buffers in a callback are of the same size.
> - All frames with a particular MSC occur at the same time. (I don't think
>  OpenML requires this, EASI does have this with its 'position')
> - The audio callback provides an MSC value for every buffer corresponding
>  to the first frame in the buffer.
> - The (constant) latency (in frames) between an input and an output can be
>  seen from the difference between their MSC values.
> - The audio callback provides an UST value for every input buffer
>  corresponding to the end of that buffer/start of the next buffer.
> - The UST value for the start/end of an output buffer can be estimated.
> 
> MIDI I/O API
> - MIDI messages are received with a UST stamp.
> - Timestamps are measured at the start of the message on the wire.
> - MIDI messages are either sent immediately or scheduled to UST.
> - MIDI messages must have monotonically increasing timestamps if
>  scheduled.
> 
> For a software synthesizer the MIDI messages received at some UST
> can be mapped to a corresponding MSC value and then rendered at
> a constant offset from this MSC, most likely the audio I/O latency
> (the audio I/O latency may not be the same for all inputs/outputs,
> also since MIDI messages always arrive late a small extra latency
> should be introduced in this case by increasing the MIDI message's
> UST by a constant value in order to compensate for the MIDI interface
> and scheduling latency so they arrive a little early instead of late
> in order to reduce jitter with MIDI messages arriving near buffer
> bounderies).
> MIDI -> audio output latency would then be slightly higher (say 2ms,
> noting that transmitting a note-on message already takes 3*320usec)
> then audio I/O latency.
> 
> 
> 
> --martijn
> 

-- 
+---- --- -- -  -   -    - 
| Frank van de Pol                  -o)    A-L-S-A
| FvdPol@home.nl                    /\\  Sounds good!
| http://www.alsa-project.org      _\_v
| Linux - Why use Windows if we have doors available?


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: [linux-audio-dev] Re: Forcing an absolute timestamp for every midi event.

[Martijn Sipkema]

[...]
> Is there already a commonly available UST on linux? To my knowledge the
only
> thing that comes close is the (cpu specific) cycle counter.

No, not yet. I think we should try to get hard- or firm-timers and POSIX
CLOCK_MONOTONIC into the Linux kernel.

--martijn






-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: [linux-audio-dev] Re: Forcing an absolute timestamp for every midi event.

[Jaroslav Kysela]

On Mon, 19 Aug 2002, Martijn Sipkema wrote:

> [...]
> > Is there already a commonly available UST on linux? To my knowledge the
> only
> > thing that comes close is the (cpu specific) cycle counter.
> 
> No, not yet. I think we should try to get hard- or firm-timers and POSIX
> CLOCK_MONOTONIC into the Linux kernel.

I agree. I'm willing to help with implementation when time permits.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project  http://www.alsa-project.org
SuSE Linux    http://www.suse.com



-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Juan Linietsky]

On Sun, 18 Aug 2002 21:12:52 +0100
"Martijn Sipkema" <msipkema@sipkema-digital.com> wrote:

> > Hi! I wanted to ask, how about forcing
> > an absolute timestamp for _every_ midi event?
> > I think this would be great for softsynths,
> > so they dont need to work with root/schedfifo/lowlatency
> > to have a decent timing. Not allways you are willing
> > to process midi at the lowest latency possible.
> > I say because you dont really need all that if you
> > sequence in the computer and control softsynths and maybe
> > some external device. 
> > This way, the softsynth gets the event with the timestamp,
> > gets the current time, substracts the audio delay (latency) to
> > that and just mixes internally in smaller blocks processing each 
> > event in the right time.
> 
> I'm not sure what you mean, but below is what I think would be the
> best approach for audio/MIDI I/O.
> 
> ----
> 
> UST = unadjusted system time
> MSC = media stream count
> 

I completely agree on using some sort of unadjusted system time, but.


> - Callback based.

Callbacks on midi are retarded, and VERY annoying. It's simply
throwing more work to the programmer which the lib can and should do.
Seriosly, think about it, what is better?

Callback system:

The programmer has to setup a waiting callback, so when the event
arrives, it will
get the current time, write it to a special event struct he did with
the event data,
and push it to an (argably blocking) fifo. Then the audio thread will
get events from it while 
it mixes.

Event polling:

The user polls for an event, which comes nicely formatted into a
struct, with timestamp and everything.

Conclusion:

The callback system is unnecesary, and even makes things worse because
if you run the program with low priority, the app can even get
callbacks wrong. Sure, you could send the event with timestamp
to a callback already, but then what do you want the callback for?
It's much better if the
api, which runs at kernel level, timestamps the event itself. So
callbacks are straight unnecesary.
We talked about this already on irc, if you still have more reasons
for using a callback, let me know :)


> - All buffers in a callback are of the same size.

This is also unnecesary, since the api takes care of this. I know you
are probably
thinking about mlocking the buffers, because they are constant size so
you dont
pagefault and stuff. This is outright paranoia :). Hey, I worked with
midi on linux
for years, the events allways get there in time! And even with that,
why should you care?
they are timestamped anyway.


YES! i know you also told me you hate that alsa takes care of so much,
and many people
has said that alsa seq api is overcomplicated. But personally, having
programmed midi on almost every OS, and having written a midi
sequencer, i can easily say that
alsa seq is the most comfortable api I have worked with. Linux does
not need really another
midi api, only needs GOOD DOCUMENTATION OF THE API, because i had to
resort to diving into
the alsadriver and lib source code to figure out stuff.


> - All frames with a particular MSC occur at the same time. (I don't
> think
>  OpenML requires this, EASI does have this with its 'position')
> - The audio callback provides an MSC value for every buffer
> corresponding
>  to the first frame in the buffer.

fine, what happens if your app xruns the audio buffer for X time? all
the count gets screwed?
you will shutdown the app like jack? will you ask devices for sync?
will you drop events? 

> - The (constant) latency (in frames) between an input and an output
> can be
>  seen from the difference between their MSC values.

This sounds nice, and comes by default, but what is this good for? I'd
like a good example.

> - The audio callback provides an UST value for every input buffer
>  corresponding to the end of that buffer/start of the next buffer.

This is actually a great idea, Very good for audio/video
syncronization,
or for JACK. Which may have to process serial chained clients in the
graph,
Even if not 100% necesary for midi, it  could also make good use of
it. Doesnt alsa already support
something like that? Still tho, it would need to have to handle
xruns.


> - The UST value for the start/end of an output buffer can be
> estimated.
> 
> MIDI I/O API
> - MIDI messages are received with a UST stamp.

Ths is what i've been complaining in the past two mails ;)

> - Timestamps are measured at the start of the message on the wire.

yes.

> - MIDI messages are either sent immediately or scheduled to UST.
> - MIDI messages must have monotonically increasing timestamps if
>  scheduled.
> 

These two can be done too, and it's good, but some people will argue
you
that you should be able to queue events with relative timestaps in
midi clocks,
so you can pre-send the events and then play with the tempo (BPM) and
events
will respond to that. This is quite a problem, since on one side you
have
the ones like us, who want absolute timestamps because
they work much better on a computer than midi clocks (which are easy
to lose sync to).
But on the other hand you have people who works all with hardware
stuff
and wants midi clocks because their equipment will process stuff in
realtime.
All in all, i'm all for timestamps because this is a software api, and
we work
with software in a multitasking OS. Thats why my original mail is
"forcing absolute timestamps". It means, whathever the input is, my
softsynth
will only work with timestamps, so timestamps need to be forced.




> For a software synthesizer the MIDI messages received at some UST
> can be mapped to a corresponding MSC value and then rendered at
> a constant offset from this MSC, most likely the audio I/O latency
> (the audio I/O latency may not be the same for all inputs/outputs,
> also since MIDI messages always arrive late a small extra latency
> should be introduced in this case by increasing the MIDI message's
> UST by a constant value in order to compensate for the MIDI
> interface and scheduling latency so they arrive a little early
> instead of late in order to reduce jitter with MIDI messages
> arriving near buffer bounderies).

Bad idea, as long as things are not audible, dont adding extra crap
is the best. Midi hardware works like this and the delay is even
higher,
yet you cant hear it. So I think this is beyond pointless. 



Juan Linietsky


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Martijn Sipkema]

[...]
> > - Callback based.
> 
> Callbacks on midi are retarded, and VERY annoying. It's simply
> throwing more work to the programmer which the lib can and should do.
> Seriosly, think about it, what is better?

Actually I was talking about the audio API in this case.

> > - All buffers in a callback are of the same size.
> 
> This is also unnecesary, since the api takes care of this.

No, this is absolutely necessary. Note that ASIO/EASI/JACK all
have this.

> I know you
> are probably
> thinking about mlocking the buffers, because they are constant size so
> you dont
> pagefault and stuff. This is outright paranoia :). Hey, I worked with
> midi on linux
> for years, the events allways get there in time! And even with that,
> why should you care?
> they are timestamped anyway.

Again, this was about audio. And the frames per callback do not have to
be constant.
And yes, I think the realtime part of an application should mlock().

> > - All frames with a particular MSC occur at the same time. (I don't
> > think
> >  OpenML requires this, EASI does have this with its 'position')
> > - The audio callback provides an MSC value for every buffer
> > corresponding
> >  to the first frame in the buffer.
> 
> fine, what happens if your app xruns the audio buffer for X time? all
> the count gets screwed?
> you will shutdown the app like jack? will you ask devices for sync?
> will you drop events? 

The application can detect xruns by looking at the MSCs.

> > - The audio callback provides an UST value for every input buffer
> >  corresponding to the end of that buffer/start of the next buffer.
> 
> This is actually a great idea, Very good for audio/video
> syncronization,
> or for JACK. Which may have to process serial chained clients in the
> graph,
> Even if not 100% necesary for midi, it  could also make good use of
> it. Doesnt alsa already support
> something like that? Still tho, it would need to have to handle
> xruns.

This is absolutely necessary for MIDI, or else scheduling MIDI to UST
doesn't make much sense.

> > - MIDI messages are either sent immediately or scheduled to UST.
> > - MIDI messages must have monotonically increasing timestamps if
> >  scheduled.
> > 
> 
> These two can be done too, and it's good, but some people will argue
> you
> that you should be able to queue events with relative timestaps in
> midi clocks,
> so you can pre-send the events and then play with the tempo (BPM) and
> events
> will respond to that. This is quite a problem, since on one side you
> have
> the ones like us, who want absolute timestamps because
> they work much better on a computer than midi clocks (which are easy
> to lose sync to).
> But on the other hand you have people who works all with hardware
> stuff
> and wants midi clocks because their equipment will process stuff in
> realtime.
> All in all, i'm all for timestamps because this is a software api, and
> we work
> with software in a multitasking OS. Thats why my original mail is
> "forcing absolute timestamps". It means, whathever the input is, my
> softsynth
> will only work with timestamps, so timestamps need to be forced.

MIDI clock sync (slave) should be handled by the application and by its
very nature will not allow scheduling events ahead of time.

> > For a software synthesizer the MIDI messages received at some UST
> > can be mapped to a corresponding MSC value and then rendered at
> > a constant offset from this MSC, most likely the audio I/O latency
> > (the audio I/O latency may not be the same for all inputs/outputs,
> > also since MIDI messages always arrive late a small extra latency
> > should be introduced in this case by increasing the MIDI message's
> > UST by a constant value in order to compensate for the MIDI
> > interface and scheduling latency so they arrive a little early
> > instead of late in order to reduce jitter with MIDI messages
> > arriving near buffer bounderies).
> 
> Bad idea, as long as things are not audible, dont adding extra crap
> is the best. Midi hardware works like this and the delay is even
> higher,
> yet you cant hear it. So I think this is beyond pointless. 

I disagree. I'd rather have a small extra latency than jitter.

-martijn





-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Juan Linietsky]

On Mon, 19 Aug 2002 07:41:30 +0100
"Martijn Sipkema" <msipkema@sipkema-digital.com> wrote:

> [...]
> > > - Callback based.
> > 
> > Callbacks on midi are retarded, and VERY annoying. It's simply
> > throwing more work to the programmer which the lib can and should
> > do. Seriosly, think about it, what is better?
> 
> Actually I was talking about the audio API in this case.

oh, nevermind then! :)
I thought all that fuzz was for midi. sorry!

> 
> > > - All frames with a particular MSC occur at the same time. (I
> > > don't think
> > >  OpenML requires this, EASI does have this with its 'position')
> > > - The audio callback provides an MSC value for every buffer
> > > corresponding
> > >  to the first frame in the buffer.
> > 
> > fine, what happens if your app xruns the audio buffer for X time?
> > all the count gets screwed?
> > you will shutdown the app like jack? will you ask devices for
> > sync? will you drop events? 
> 
> The application can detect xruns by looking at the MSCs.

Well, i guess i didnt make myself clear enough.
Technically, when you use some count/clock/tick based method
for doing things, there is a certain hardware that generates such
things.
How are you sure that you will be able to generate those reliably
within
the OS? The only way is generating them in realtime, and prequeuing
would be impossible. It happens the same when the OS generates
midi clocks, if the load affects wathever is timing internally,
all your queue gets screwed. This has its uses i guess anyway.

> 
> > > - The audio callback provides an UST value for every input
> > > buffer
> > >  corresponding to the end of that buffer/start of the next
> > >  buffer.
> > 
> > This is actually a great idea, Very good for audio/video
> > syncronization,
> > or for JACK. Which may have to process serial chained clients in
> > the graph,
> > Even if not 100% necesary for midi, it  could also make good use
> > of it. Doesnt alsa already support
> > something like that? Still tho, it would need to have to handle
> > xruns.
> 
> This is absolutely necessary for MIDI, or else scheduling MIDI to
> UST doesn't make much sense.

Of course it is, otherwise discussing it would be out of the question
;)
What i mean is the audio callback prooviding the UST time. I mean it's
not necesary
since you can compute the delay in time from ust and the audio buffer
size, but
I am _definitively_ in favor of this. Alsa should proovide this, and
JACK _must_ do it,
so clients can syncronize properly. 

> MIDI clock sync (slave) should be handled by the application and by
> its very nature will not allow scheduling events ahead of time.

not clock sync, i mean the midi api. Thats why i think it's pointless.
ant hear it. So I think this is beyond pointless. 

> 
> I disagree. I'd rather have a small extra latency than jitter.
>

We're taling about something WAY beyond human-hearable latency.
Say you send the event, it will probably reach in nanoseconds if
things
go right, or useconds if things go not so right. The event would
need to delay like, 5 milliseconds for you to notice any jitter.
And this will _never_ happen. Absolutely *never*. 


Well, all in all we're both talking about the same. ALSA and JACK
should proovide
access to some timer similar to UST. Now the real problem is the
implementation.
How to do this? Not all architectures proovide a secondary clock for
such things,
and on X86 I cant think of something other than the pentium cycle
counter.

Juan Linietsky



-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Martijn Sipkema]

[...]
> Well, i guess i didnt make myself clear enough.
> Technically, when you use some count/clock/tick based method
> for doing things, there is a certain hardware that generates such
> things.
> How are you sure that you will be able to generate those reliably
> within
> the OS? The only way is generating them in realtime, and prequeuing
> would be impossible.

The MIDI API just allows scheduling MIDI for transmission at a certain
future UST.
Either scheduling is done (partly) in hardware or in a process that
schedules
the messages (using POSIX clock_nanosleep() or a similar function).

I'm thinking instead of the current sequencer API there is a user space
'sequencer' that handles merging and accepts clients to send it either
scheduled or immediate messages (queued) and have a 'driver' be a
combination of a kernel driver (ALSA rawmidi or device specific) and
a 'driver' process that registers queues with the user-space sequencer.

I'm working on this, but it is not that easy since merging MIDI streams
is non-trivial and I want to be able to have hardware do the scheduling if
it supports it and this means events can only be queued in timed order.

> It happens the same when the OS generates
> midi clocks, if the load affects wathever is timing internally,
> all your queue gets screwed. This has its uses i guess anyway.


[...]
> What i mean is the audio callback prooviding the UST time. I mean it's
> not necesary
> since you can compute the delay in time from ust and the audio buffer
> size, but

OK, I'm not sure what you mean here, but as I said earlier, I think the
audio
API should provide a MSC for every buffer in a callback and a UST for every
input buffer corresponding to the end of that buffer. Perhaps better: a
callback could
provide a single UST/MSC pair on every callback where the MSC is higher than
the highest MSC in any input buffer.

> I am _definitively_ in favor of this. Alsa should proovide this, and
> JACK _must_ do it,
> so clients can syncronize properly.

I wonder if Paul can be convinced to add UST timestamping to JACK.
He seems to be agreeing with me lately so things are looking good :)

> > I disagree. I'd rather have a small extra latency than jitter.
>
> We're taling about something WAY beyond human-hearable latency.
> Say you send the event, it will probably reach in nanoseconds if
> things
> go right, or useconds if things go not so right. The event would
> need to delay like, 5 milliseconds for you to notice any jitter.
> And this will _never_ happen. Absolutely *never*.

Since the MIDI API I was talking about timestamps the messages at the
start of the message on the wire it will always arrive at the application
between
1-2 msec late. To avoid possible jitter at audio buffer bounderies messages
should arrive early instead of late and thus some extra latency should be
added
to the MIDI messages' UST. A 2 msec jitter is irritating, an extra 1-2 msec
delay isn't.

Note that at the moment there aren't any softsynths on any platform that I
know
of that can do this, and many have audio buffer size MIDI message jitter on
output. ( http://www.rme-audio.de/english/techinfo/lola/latec.htm ).

> Well, all in all we're both talking about the same. ALSA and JACK
> should proovide
> access to some timer similar to UST. Now the real problem is the
> implementation.
> How to do this? Not all architectures proovide a secondary clock for
> such things,
> and on X86 I cant think of something other than the pentium cycle
> counter.

As long as POSIX clocks are available from both user- and kernel-space and
we
all agree that CLOCK_MONOTONIC is UST for Linux, that'll do. ALSA drivers
could then use CLOCK_MONOTONIC to timestamp buffers. How
CLOCK_MONOTONIC is implemented is not that important, It'll probably have
to be the firm- or hard- timers patch for Linux.

--martijn







-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

aside - alsa-docs

[Patrick Shirkey]

I just noticed that the tarball has a full set of drivers. Now I can get 
the docs working fully.

Handy:)

-- 
Patrick Shirkey - Boost Hardware Ltd.
For the discerning hardware connoisseur
Http://www.boosthardware.com
Http://www.boosthardware.com/LAU/guide/
========================================



-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Frank van de Pol]

On Mon, Aug 19, 2002 at 01:39:02PM +0100, Martijn Sipkema wrote:
> 
> The MIDI API just allows scheduling MIDI for transmission at a certain
> future UST.
> Either scheduling is done (partly) in hardware or in a process that
> schedules
> the messages (using POSIX clock_nanosleep() or a similar function).

agreed; I'd like to see the 'firm timer stuff' here :-)

> 
> I'm thinking instead of the current sequencer API there is a user space
> 'sequencer' that handles merging and accepts clients to send it either
> scheduled or immediate messages (queued) and have a 'driver' be a
> combination of a kernel driver (ALSA rawmidi or device specific) and
> a 'driver' process that registers queues with the user-space sequencer.

In the future I'd like to move as much of ALSA sequencer from kernel space
to user space. If a (new) underlaying UST queue is present I see no
objection to move things into userland/alsalib. 

> 
> I'm working on this, but it is not that easy since merging MIDI streams
> is non-trivial and I want to be able to have hardware do the scheduling if
> it supports it and this means events can only be queued in timed order.

Not necessarely; If you have the ability to adjust when your timer is to be
fired/generate irq, you can readjust the schedule time when an event is
received (after all, that's the whole fun with an event based interface).
Merging streams is not a problem; You'll need a priority queue anyway for
merging multiple streams.

> the highest MSC in any input buffer.
> 
> > I am _definitively_ in favor of this. Alsa should proovide this, and
> > JACK _must_ do it,
> > so clients can syncronize properly.
> 
> I wonder if Paul can be convinced to add UST timestamping to JACK.
> He seems to be agreeing with me lately so things are looking good :)

:-)

> 
> > > I disagree. I'd rather have a small extra latency than jitter.

Yes, jitter is far worse than latency. 

> >
> > We're taling about something WAY beyond human-hearable latency.
> > Say you send the event, it will probably reach in nanoseconds if
> > things
> > go right, or useconds if things go not so right. The event would
> > need to delay like, 5 milliseconds for you to notice any jitter.
> > And this will _never_ happen. Absolutely *never*.

> 
> Since the MIDI API I was talking about timestamps the messages at the
> start of the message on the wire it will always arrive at the application
> between
> 1-2 msec late. To avoid possible jitter at audio buffer bounderies messages
> should arrive early instead of late and thus some extra latency should be
> added
> to the MIDI messages' UST. A 2 msec jitter is irritating, an extra 1-2 msec
> delay isn't.
> 
> Note that at the moment there aren't any softsynths on any platform that I
> know
> of that can do this, and many have audio buffer size MIDI message jitter on
> output. ( http://www.rme-audio.de/english/techinfo/lola/latec.htm ).

hmmm, quite a surprise nobody seems to do this, because it seems a logical
step to me to do microscheduling inside the buffer. Please nobody is playing
tricks with these !@#$%^&* software patents here???

> 
> > Well, all in all we're both talking about the same. ALSA and JACK
> > should proovide
> > access to some timer similar to UST. Now the real problem is the
> > implementation.
> > How to do this? Not all architectures proovide a secondary clock for
> > such things,
> > and on X86 I cant think of something other than the pentium cycle
> > counter.
> 
> As long as POSIX clocks are available from both user- and kernel-space and
> we
> all agree that CLOCK_MONOTONIC is UST for Linux, that'll do. ALSA drivers
> could then use CLOCK_MONOTONIC to timestamp buffers. How
> CLOCK_MONOTONIC is implemented is not that important, It'll probably have
> to be the firm- or hard- timers patch for Linux.
> 

looking good!

Frank.

-- 
+---- --- -- -  -   -    - 
| Frank van de Pol                  -o)    A-L-S-A
| FvdPol@home.nl                    /\\  Sounds good!
| http://www.alsa-project.org      _\_v
| Linux - Why use Windows if we have doors available?


-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390

Re: Forcing an absolute timestamp for every midi event.

[Martijn Sipkema]

[...]
> > I'm working on this, but it is not that easy since merging MIDI streams
> > is non-trivial and I want to be able to have hardware do the scheduling
if
> > it supports it and this means events can only be queued in timed order.
>
> Not necessarely; If you have the ability to adjust when your timer is to
be
> fired/generate irq, you can readjust the schedule time when an event is
> received (after all, that's the whole fun with an event based interface).
> Merging streams is not a problem; You'll need a priority queue anyway for
> merging multiple streams.

Perhaps it is not necessary to have the messages queued in timed order, but
on the other hand and application should be able to know the order of the
messages and thus has no good reason not queue them time ordered. I think
a requirement for time-ordered queueing in combination with a global
pre-schedule
value (i.e. minimum time scheduled events are to be queued early) could
simplify
implementation.

--martijn





-------------------------------------------------------
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone?  Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390