smix plugin available?

smix plugin available?

[Mark Swanson]

Hello,

A gentleman by the name of Abramo Bagnara recently stated he may have some 
code that would kick-start the development of the smix plugin. I think this 
is a very useful and increasingly important component. Abramo stated he had 
no problem releasing it, but I could not find where or if he did so.

If Abramo - or anyone - could point me in the right direction I would 
appreciate it.

The reasons I'm looking for this is to solve the following problems with 
soundcards that do not have a hardware mixer:

When using Gnome/KDE (esd/arts) no programs can access the sound device unless 
they are written to use arts/esd/jack/etc...

It is not logical for every program to write support for esd, artsd, jack, 
alsa, etc. Programs should write to ALSA and let ALSA do software mixing if 
required. Windows provided this since DirectX (3?). Solaris provides this too 
(esd apparently doesn't block on Solaris).

I believe that the majority of sound cards in use (on Linux) do not have 
hardware mixing, and that this is already a large problem that will continue 
to get larger.

One of the big reasons this is affecting me is that Java sound will not work 
unless you have a hardware mixer. My understanding is that the Sun folks seem 
to think that it is wrong to have to implement many different ways to create 
sound when the sound library (ALSA) should do it for them - the way it works 
in Windows/Solaris. I completely agree with them.

Cheers.

-- 
Schedule your world with ScheduleWorld.com
http://www.ScheduleWorld.com/



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

2 clarifications:

>It is not logical for every program to write support for esd, artsd, jack, 
>alsa, etc. Programs should write to ALSA and let ALSA do software mixing if 
>required. Windows provided this since DirectX (3?). Solaris provides this too 
>(esd apparently doesn't block on Solaris).

Windows provides this, true. But most "prosumer" and professional
applications do *not* use it. The Windows kernel mixer was the curse
of pro-ish apps for a long time, hence driver models like ASIO and now
WDM. AFAIK, the new WDM drivers are generally used by apps in ways
that preclude "sharing" with other apps, and certainly ASIO drivers
cannot be shared in this way.

>One of the big reasons this is affecting me is that Java sound will not work 
>unless you have a hardware mixer. My understanding is that the Sun folks seem 
>to think that it is wrong to have to implement many different ways to create 
>sound when the sound library (ALSA) should do it for them - the way it works 
>in Windows/Solaris. I completely agree with them.

ALSA is *a* sound library. There are lots of things that it doesn't
contain, and its written around a fairly specific programming
paradigm. There are those of us (many people on LAD) who believe that
its too hard to fit a callback-driven model into the existing ALSA
design, and that its therefore better to implement such a model
outside of ALSA.

You see, if all apps are written to use the ALSA API, that's going to
be great for the purposes you have in mind, but totally awful for
those of us who want our audio apps to work in a sample synchronous
way and ignorant of the ultimate routing of their data. Many of us
don't think that an API based on the open/close/read/write paradigm is
appropriate for real time streaming media.

All that being said, I'd love to see the smix plugin implemented and
available, if only because it would allow ALSA native apps to
participate in a JACK system, albeit without sample synchronous
behaviour.

--p




-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Mark Swanson]

On November 25, 2002 10:19 pm, Paul Davis wrote:
> 2 clarifications:
> >It is not logical for every program to write support for esd, artsd, jack,
> >alsa, etc. Programs should write to ALSA and let ALSA do software mixing
> > if required. Windows provided this since DirectX (3?). Solaris provides
> > this too (esd apparently doesn't block on Solaris).
>
> Windows provides this, true. But most "prosumer" and professional
> applications do *not* use it. The Windows kernel mixer was the curse
> of pro-ish apps for a long time, hence driver models like ASIO and now
> WDM. AFAIK, the new WDM drivers are generally used by apps in ways
> that preclude "sharing" with other apps, and certainly ASIO drivers
> cannot be shared in this way.

For "prosumer" and professional applications, lock it and block it. I have no 
idea how bad it is for miscellaneous simultaneous use of the sound device wrt 
interfering with these types of apps. But, if it is a problem - and I believe 
you that it is - then lock away. I don't think anyone would mind that. I 
don't think it interferes with what people would use smix for.

> >One of the big reasons this is affecting me is that Java sound will not
> > work unless you have a hardware mixer. My understanding is that the Sun
> > folks seem to think that it is wrong to have to implement many different
> > ways to create sound when the sound library (ALSA) should do it for them
> > - the way it works in Windows/Solaris. I completely agree with them.
>
> ALSA is *a* sound library. There are lots of things that it doesn't
> contain, and its written around a fairly specific programming
> paradigm. There are those of us (many people on LAD) who believe that
> its too hard to fit a callback-driven model into the existing ALSA
> design, and that its therefore better to implement such a model
> outside of ALSA.
>
> You see, if all apps are written to use the ALSA API, that's going to
> be great for the purposes you have in mind, but totally awful for
> those of us who want our audio apps to work in a sample synchronous
> way and ignorant of the ultimate routing of their data. Many of us
> don't think that an API based on the open/close/read/write paradigm is
> appropriate for real time streaming media.

If there was a way to temporarily disable the smix plugin, or temporarily gain 
exclusive ownership of the sound device for your purposes would that meet 
100% of your requirements?

> All that being said, I'd love to see the smix plugin implemented and
> available, if only because it would allow ALSA native apps to
> participate in a JACK system, albeit without sample synchronous
> behaviour.

Great.
Know where to find it? :-)
Even totally broken code that does not compile or do anything useful would be 
a wonderful head-start relative to starting from scratch.

Cheers.

-- 
Schedule your world with ScheduleWorld.com
http://www.ScheduleWorld.com/



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

>> You see, if all apps are written to use the ALSA API, that's going to
>> be great for the purposes you have in mind, but totally awful for
>> those of us who want our audio apps to work in a sample synchronous
>> way and ignorant of the ultimate routing of their data. Many of us
>> don't think that an API based on the open/close/read/write paradigm is
>> appropriate for real time streaming media.
>
>If there was a way to temporarily disable the smix plugin, or temporarily gain
>exclusive ownership of the sound device for your purposes would that meet 
>100% of your requirements?

no, it wouldn't meet any of them. the problem is not exclusive
access. its the fundamental API model. ALSA (like OSS before it, as
well as SGI's DMedia APIs) has promoted the open/close/read/write
model. this is the central problem. ALSA certainly *allows* for a
callback model (its what allows JACK to work), but there are almost no
applications that use ALSA in this way. using the o/c/r/w paradigm
makes sample synchronous execution of multiple applications basically
impossible, and more importantly it encourages application designers
to construct programs based on the idea that the program controls when
to read/write audio data. this doesn't work properly except for
heavily buffered, single applications. the APIs that are used to write
almost all audio software code in production these days all use a
callback model. porting from the o/c/r/w model to the callback one is
hard. do you want another generation of apps stuck with this problem?

if you want a genuinely portable solution, use PortAudio. it works
with (but hides) OSS, Windows MME, ASIO, CoreAudio, and several
others. JACK and ALSA support is present in CVS. it encourages a
callback model.

>> All that being said, I'd love to see the smix plugin implemented and
>> available, if only because it would allow ALSA native apps to
>> participate in a JACK system, albeit without sample synchronous
>> behaviour.
>
>Great.
>Know where to find it? :-)

i think abramo posted a copy of what he had to this list within the
last 5 weeks. unfortunately, sf.net's archives don't allow searching,
so finding it might be tedious, to put it mildly.

--p


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Frans Ketelaars]

On Mon, 25 Nov 2002 23:13:35 -0500
Paul Davis <paul@linuxaudiosystems.com> wrote:

<snip>

> >> All that being said, I'd love to see the smix plugin implemented and
> >> available, if only because it would allow ALSA native apps to
> >> participate in a JACK system, albeit without sample synchronous
> >> behaviour.
> >
> >Great.
> >Know where to find it? :-)
> 
> i think abramo posted a copy of what he had to this list within the
> last 5 weeks. unfortunately, sf.net's archives don't allow searching,
> so finding it might be tedious, to put it mildly.
> 
> --p

http://www.mail-archive.com/alsa-devel@lists.sourceforge.net/msg04592.html

HTH,

    -Frans


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Mark Swanson]

On November 26, 2002 03:13 am, Frans Ketelaars wrote:
>
> http://www.mail-archive.com/alsa-devel@lists.sourceforge.net/msg04592.html

Thanks!
I had seen this posting but the particular web interface I was using didn't 
show the attachment. Got it.

-- 
Schedule your world with ScheduleWorld.com
http://www.ScheduleWorld.com/



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Florian Bomers]

Hi Paul,

very interesting discussion.

Paul Davis wrote:
> 
> (...)
> 
> >One of the big reasons this is affecting me is that Java sound will not work
> >unless you have a hardware mixer. My understanding is that the Sun folks seem
> >to think that it is wrong to have to implement many different ways to create
> >sound when the sound library (ALSA) should do it for them - the way it works
> >in Windows/Solaris. I completely agree with them.
> 
> ALSA is *a* sound library. There are lots of things that it doesn't

I would really say: ALSA is *the* sound library (at least on Linux). Isn't it in kernel
2.5+ ?

> contain, and its written around a fairly specific programming
> paradigm. There are those of us (many people on LAD) who believe that
> its too hard to fit a callback-driven model into the existing ALSA
> design, and that its therefore better to implement such a model
> outside of ALSA.
> 
> You see, if all apps are written to use the ALSA API, that's going to
> be great for the purposes you have in mind, but totally awful for
> those of us who want our audio apps to work in a sample synchronous
> way and ignorant of the ultimate routing of their data. Many of us
> don't think that an API based on the open/close/read/write paradigm is
> appropriate for real time streaming media.

I don't know if I get the point right here, but it reads: don't use ALSA. But it's the
only sound library that will be delivered with all distributions in future, so what choice
do mainstream application writers have ? OSS ?

> (...)
> 
> >If there was a way to temporarily disable the smix plugin, or temporarily gain
> >exclusive ownership of the sound device for your purposes would that meet
> >100% of your requirements?
> 
> no, it wouldn't meet any of them. the problem is not exclusive
> access. its the fundamental API model. ALSA (like OSS before it, as
> well as SGI's DMedia APIs) has promoted the open/close/read/write
> model. this is the central problem. ALSA certainly *allows* for a
> callback model (its what allows JACK to work), but there are almost no
> applications that use ALSA in this way. using the o/c/r/w paradigm
> makes sample synchronous execution of multiple applications basically
> impossible, and more importantly it encourages application designers
> to construct programs based on the idea that the program controls when
> to read/write audio data. this doesn't work properly except for
> heavily buffered, single applications. 

So you discourage use of ALSA because it suggests/allows a non-professional programming
paradigm? 

> the APIs that are used to write almost all audio software code in 
> production these days all use a callback model. 

Sorry for questioning this statement. Of course we all don't have any statistical data but
you miss what I see as the majority of applications that use audio devices:

1) games
2) media players
3) GUI sounds (i.e. accessibility)

What point is in comparing the amount of "audio software code" ? Compare the number of
people using the above types of software with the number of people using semi-pro level
audio software.

> porting from the o/c/r/w model to the callback one is
> hard. do you want another generation of apps stuck with this problem?

So you think the solution is to lead developers to the right programming paradigm by not
making o/c/r/w type of APIs available anymore.

Again, I don't see the point. The 99% of Linux users using the above types of programs do
not care about the programming paradigm. They want to hear their apps. Since Linux
distributions enable artsd/esd sound daemons by default, people don't hear applications
that don't support the specific sound daemon. On Windows, we do have the choice and it all
happily coexists. 

My perfect world would look like this:
- ALSA (becoming the default audio HAL on Linux) has the future smix plugin enabled by
default, but only if the soundcard does not provide hardware mixing
- sound daemons can all run at the same time, and they can continue to block the device if
they really think that's a good idea
- apps with higher requirements (low latency, sample synchronization, etc.) will need the
user to stop the daemons and will use ALSA hardware devices hw: directly.

I also see the problem that audio daemons block the soundcard. But that's another story.

> if you want a genuinely portable solution, use PortAudio. it works
> with (but hides) OSS, Windows MME, ASIO, CoreAudio, and several
> others. JACK and ALSA support is present in CVS. it encourages a
> callback model.

The "genuinely" portable solution is Java :) 

Well, my point is that I want mainstream apps work out-of-the box, no matter if they talk
to a daemon, to ALSA or use OSS. For semi-pro audio apps, a little effort to make them use
the full capabilities is necessary anyway, and those people probably won't mind stopping a
sound daemon.

Florian



-- 
Florian Bomers
Java Sound
Java Software/Sun Microsystems, Inc.
http://java.sun.com/products/java-media/sound/


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

>> ALSA is *a* sound library. There are lots of things that it doesn't
>
>I would really say: ALSA is *the* sound library (at least on Linux). Isn't it 
>in kernel
>2.5+ ?

alsa-lib isn't in kernel 2.5, because its not part of the kernel.

alsa-lib doesn't contain any code to read or write audio files, for
example. it contains no code to do dithering. it contains no code to
do audio rate resampling, at least no code that is accessible by a
regular program.

>I don't know if I get the point right here, but it reads: don't use
>ALSA. But it's the only sound library that will be delivered with all
>distributions in future, so what choice do mainstream application
>writers have ? OSS ?

PortAudio would be a much better choice.

>So you discourage use of ALSA because it suggests/allows a
>non-professional programming paradigm?

not just non-professional. the read/write paradigm isn't just
non-professional, its wrong. its masking a fundamental aspect of the
hardware. it works for video (which is also, essentially a real time
streaming media device) because nobody notices anything odd about a
video screen that doesn't change for a little while (unless they are
trying to watch a real-time video stream). but any audio device that
outputs the same data twice around its hardware buffer is immediately
noticeable by any (non-hearing impaired) user. you can't fake
this. apps that use large buffers are just getting away with
pretending; it may work for them, but it doesn't work for most apps. i
find the lag on the xmms eq controls, for example, quite irritating.

>> the APIs that are used to write almost all audio software code in 
>> production these days all use a callback model. 
>
>Sorry for questioning this statement. Of course we all don't have any statisti
>cal data but
>you miss what I see as the majority of applications that use audio devices:
>
>1) games
>2) media players
>3) GUI sounds (i.e. accessibility)

this is a fair point. i apologize. i have my head so far inside the
"audio software for musicians" box that i tend to fail to see such
applications :)

however, the very fact that the applications developers of such
programs really don't know much (if anything) about audio programming,
and probably don't want to know much about it either, suggests that an
API like ALSA which exposes the full HAL is probably a mistake. again,
i consider PortAudio a vastly preferable solution.

i would note in passing that your second class of apps might very well
be something that needs to be integrated with prosumer/pro apps. the
division between, say, alsaplayer and gdam (to give two examples from
the linux world) is hard to find when thinking up reasons why one
should never be useful as a participant in something like JACK, and
another one should. 

>> porting from the o/c/r/w model to the callback one is
>> hard. do you want another generation of apps stuck with this problem?
>
>So you think the solution is to lead developers to the right programming parad
>igm by not making o/c/r/w type of APIs available anymore.

pretty much, yes.

>Again, I don't see the point. The 99% of Linux users using the above
>types of programs do not care about the programming paradigm. They
>want to hear their apps. Since L inux distributions enable artsd/esd
>sound daemons by default, people don't hear app lications that don't
>support the specific sound daemon. On Windows, we do have the choic e
>and it all happily coexists.

you're talking about a world in which there is a sharp divide between
the classes of apps you've listed above and the prosumer/pro apps i
tend to focus on. i'd rather see linux support audio APIs that
provide, if not an integrated user environment for all apps (this may
not be possible), then an integrated and consistent audio API, one
that recognizes the inherently real-time nature of audio software and
uses a callback-driven model. can you say "CoreAudio"? :))

>Well, my point is that I want mainstream apps work out-of-the box, no
>matter i f they talk to a daemon, to ALSA or use OSS. For semi-pro
>audio apps, a little effort to m ake them use the full capabilities
>is necessary anyway, and those people probably won't mind stopping a
>sound daemon.

Apple have done the right thing, IMHO. OS X forces developers to
either (1) deal with a full featured and complex HAL, but in so doing
lose the ability to interact with other software, which users won't
like or (2) use the AudioUnits API (callback based) which allows full
integration. The problem is, as I noted several times before, there is
nobody in the linux world who can force this on developers.

--p


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Florian Bomers]

That makes sense. I also fully agree that callback-driven APIs are better suited for
audio. On the other hand, nobody would doubt that o/c/r/w apps would not allow low enough
latency for GUI-type apps like EQs (e.g. with buffer sizes of 10-20ms the total latency
isn't so inacceptable).

But that's all getting a bit too far off of the original problem: sound on Linux is not
playing, because another app or daemon blocks the sound device. For that particular
problem, PortAudio is merely a work-around than a solution.

I think such an smix plugin would work like this (please correct if wrong):

1) standard "mainstream" users will use smix. Their software will need to use the
"plughw:" devices in ALSA; smix is enabled by default (OSS programs, ALSA programs, sound
daemons, ...).

2) more professional audio apps use "hw:" devices and fail if the smix plugin is being
used. The user needs to make sure that all software that uses the "plughw:" devices are
terminated and then start the audio app again. 

3) since smix shouldn't be activated for devices with hardware mixing, those users get the
best from both worlds. Needless to say that people who use semi-pro audio software are not
so likely to have only one non-mixing soundcard.

JACK is really in between, but maybe some users would rather use a "crippled" JACK working
on "plughw:" than having no OSS/artsd/etc. sound simultaneously.

Florian

PS: sorry for my ignorance, but I haven't said "CoreAudio" yet, I guess I should learn to
do so...


Paul Davis wrote:
> 
> >> ALSA is *a* sound library. There are lots of things that it doesn't
> >
> >I would really say: ALSA is *the* sound library (at least on Linux). Isn't it
> >in kernel
> >2.5+ ?
> 
> alsa-lib isn't in kernel 2.5, because its not part of the kernel.
> 
> alsa-lib doesn't contain any code to read or write audio files, for
> example. it contains no code to do dithering. it contains no code to
> do audio rate resampling, at least no code that is accessible by a
> regular program.
> 
> >I don't know if I get the point right here, but it reads: don't use
> >ALSA. But it's the only sound library that will be delivered with all
> >distributions in future, so what choice do mainstream application
> >writers have ? OSS ?
> 
> PortAudio would be a much better choice.
> 
> >So you discourage use of ALSA because it suggests/allows a
> >non-professional programming paradigm?
> 
> not just non-professional. the read/write paradigm isn't just
> non-professional, its wrong. its masking a fundamental aspect of the
> hardware. it works for video (which is also, essentially a real time
> streaming media device) because nobody notices anything odd about a
> video screen that doesn't change for a little while (unless they are
> trying to watch a real-time video stream). but any audio device that
> outputs the same data twice around its hardware buffer is immediately
> noticeable by any (non-hearing impaired) user. you can't fake
> this. apps that use large buffers are just getting away with
> pretending; it may work for them, but it doesn't work for most apps. i
> find the lag on the xmms eq controls, for example, quite irritating.
> 
> >> the APIs that are used to write almost all audio software code in
> >> production these days all use a callback model.
> >
> >Sorry for questioning this statement. Of course we all don't have any statisti
> >cal data but
> >you miss what I see as the majority of applications that use audio devices:
> >
> >1) games
> >2) media players
> >3) GUI sounds (i.e. accessibility)
> 
> this is a fair point. i apologize. i have my head so far inside the
> "audio software for musicians" box that i tend to fail to see such
> applications :)
> 
> however, the very fact that the applications developers of such
> programs really don't know much (if anything) about audio programming,
> and probably don't want to know much about it either, suggests that an
> API like ALSA which exposes the full HAL is probably a mistake. again,
> i consider PortAudio a vastly preferable solution.
> 
> i would note in passing that your second class of apps might very well
> be something that needs to be integrated with prosumer/pro apps. the
> division between, say, alsaplayer and gdam (to give two examples from
> the linux world) is hard to find when thinking up reasons why one
> should never be useful as a participant in something like JACK, and
> another one should.
> 
> >> porting from the o/c/r/w model to the callback one is
> >> hard. do you want another generation of apps stuck with this problem?
> >
> >So you think the solution is to lead developers to the right programming parad
> >igm by not making o/c/r/w type of APIs available anymore.
> 
> pretty much, yes.
> 
> >Again, I don't see the point. The 99% of Linux users using the above
> >types of programs do not care about the programming paradigm. They
> >want to hear their apps. Since L inux distributions enable artsd/esd
> >sound daemons by default, people don't hear app lications that don't
> >support the specific sound daemon. On Windows, we do have the choic e
> >and it all happily coexists.
> 
> you're talking about a world in which there is a sharp divide between
> the classes of apps you've listed above and the prosumer/pro apps i
> tend to focus on. i'd rather see linux support audio APIs that
> provide, if not an integrated user environment for all apps (this may
> not be possible), then an integrated and consistent audio API, one
> that recognizes the inherently real-time nature of audio software and
> uses a callback-driven model. can you say "CoreAudio"? :))
> 
> >Well, my point is that I want mainstream apps work out-of-the box, no
> >matter i f they talk to a daemon, to ALSA or use OSS. For semi-pro
> >audio apps, a little effort to m ake them use the full capabilities
> >is necessary anyway, and those people probably won't mind stopping a
> >sound daemon.
> 
> Apple have done the right thing, IMHO. OS X forces developers to
> either (1) deal with a full featured and complex HAL, but in so doing
> lose the ability to interact with other software, which users won't
> like or (2) use the AudioUnits API (callback based) which allows full
> integration. The problem is, as I noted several times before, there is
> nobody in the linux world who can force this on developers.
> 
> --p
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Get the new Palm Tungsten T
> handheld. Power & Color in a compact size!
> http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/alsa-devel

-- 
Florian Bomers
Java Sound
Java Software/Sun Microsystems, Inc.
http://java.sun.com/products/java-media/sound/


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[James Courtier-Dutton]

Paul Davis wrote:

>  
>
>>>the APIs that are used to write almost all audio software code in 
>>>production these days all use a callback model. 
>>>      
>>>
>>Sorry for questioning this statement. Of course we all don't have any statisti
>>cal data but
>>you miss what I see as the majority of applications that use audio devices:
>>
>>1) games
>>2) media players
>>3) GUI sounds (i.e. accessibility)
>>    
>>
>
>this is a fair point. i apologize. i have my head so far inside the
>"audio software for musicians" box that i tend to fail to see such
>applications :)
>
>however, the very fact that the applications developers of such
>programs really don't know much (if anything) about audio programming,
>and probably don't want to know much about it either, suggests that an
>API like ALSA which exposes the full HAL is probably a mistake. again,
>i consider PortAudio a vastly preferable solution.
>
>  
>
I would like to point out that a "callback" api would work just as well 
as an open/write/close api for

1) games
2) media players
3) GUI sounds (i.e. accessibility)

I have to agree with Paul on the fact that a "callback" approach is really the ONLY real option.
Here is my reasoning: -
1) My perspective is from "(2) Media players" and not "Pro-Audio"
2) Sound Hardware tends to have very small buffers.
3) For nice sounding audio, these buffers should never run dry. I.E. No XRUNs.
4) A open/write/close api will never ever be able to guarantee no XRUNs, as it has no control on when it will get scheduling time to do the next write.
5) With a "callback" approach, the kernel would be notified by the sound hardware that it was ready for new samples, the kernel could then adjust the scheduler, so that the "callback" function was called ASAP.
The "callback" function then only has to decide which samples to give. If the "callback" function could receive a "delay" value from the sound hardware at each callback, a media player would then have all the information it needed to do full audio/video sync.
6) I don't need "sample sync", but I do NEED "callback" based api to provide me with "no XRUNs".

Summary: - 
The only way to cure XRUN problems is with a "callback" based api.
All application that currently use open/write/close apis, can just as easily use a "callback" api.

Cheers
James





-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Jaroslav Kysela]

On Wed, 27 Nov 2002, James Courtier-Dutton wrote:

> Paul Davis wrote:
> 
> >  
> >
> >>>the APIs that are used to write almost all audio software code in 
> >>>production these days all use a callback model. 
> >>>      
> >>>
> >>Sorry for questioning this statement. Of course we all don't have any statisti
> >>cal data but
> >>you miss what I see as the majority of applications that use audio devices:
> >>
> >>1) games
> >>2) media players
> >>3) GUI sounds (i.e. accessibility)
> >>    
> >>
> >
> >this is a fair point. i apologize. i have my head so far inside the
> >"audio software for musicians" box that i tend to fail to see such
> >applications :)
> >
> >however, the very fact that the applications developers of such
> >programs really don't know much (if anything) about audio programming,
> >and probably don't want to know much about it either, suggests that an
> >API like ALSA which exposes the full HAL is probably a mistake. again,
> >i consider PortAudio a vastly preferable solution.
> >
> >  
> >
> I would like to point out that a "callback" api would work just as well 
> as an open/write/close api for
> 
> 1) games
> 2) media players
> 3) GUI sounds (i.e. accessibility)
> 
> I have to agree with Paul on the fact that a "callback" approach is really the ONLY real option.
> Here is my reasoning: -
> 1) My perspective is from "(2) Media players" and not "Pro-Audio"
> 2) Sound Hardware tends to have very small buffers.
> 3) For nice sounding audio, these buffers should never run dry. I.E. No XRUNs.
> 4) A open/write/close api will never ever be able to guarantee no XRUNs, as it has no control on when it will get scheduling time to do the next write.
> 5) With a "callback" approach, the kernel would be notified by the sound hardware that it was ready for new samples, the kernel could then adjust the scheduler, so that the "callback" function was called ASAP.
> The "callback" function then only has to decide which samples to give. If the "callback" function could receive a "delay" value from the sound hardware at each callback, a media player would then have all the information it needed to do full audio/video sync.

Sorry, it's not as easy as you've described. It's not possible to invoke
any user code from the kernel code directly. There is a scheduler which is
informed that a task has been woken up. It depends on scheduler when the
task is really invoked. It's quite same as for the r/w model where the
application is notified over poll that something occured.

> 6) I don't need "sample sync", but I do NEED "callback" based api to provide me with "no XRUNs".

I don't think that there is some difference. If the scheduler don't give 
you enough time, the audio stream is somehow broken on all architectures.

> Summary: - 
> The only way to cure XRUN problems is with a "callback" based api.
> All application that currently use open/write/close apis, can just as easily use a "callback" api.

Let's go and see the implementation:

The callback model is good for perfect sync between applications. It can 
do (and does) chaining of more sources, arbitrating (removing invalid 
sources) and so on. It is simply something "over" the audio HAL. If it 
really helps, it's a different point.

The discussed difference (a few months ago) was in a count of the task 
context switches.

With jack, you have these context switches (daemon and two applications 
mixed together) for one period of samples (* means a context switch):

jackd -*> app1 -*> app2 -*> jackd -> soundcard

With r/w model and a sample mixing server implemented in the user space, 
you can get this for one period of samples:

mserver -> soundcard
        -*(ring buffer does not contain enough samples)> app1
        -*(ring buffer does not contain enough samples)> app2

In real real-time setup, there will be two periods (thus app1 and app2) 
will be woken up all times. So, in real world the context switches are for 
one period of samples:

mserver -*> app1 -*> app2   or
mserver -*> app2 -*> app1

Note: mserver implementation uses same assumptions as jack (or any other
callback model). The ring buffers are shared between app and mserver and
the period of samples is a constant. The playback / capture pointers are
incrementing by the period size steps. Thus there is no need to commit 
result directly back to the mserver. mserver will be woken up by the 
kernel scheduler when next poll() event occurs (on next period boundary).

Ok, it's only simple example, that there are more solutions than Paul 
suggests. I fully agree, that the callback model is suitable for the 
perfect synchronization among more applications. Also, imagine that
mserver is not using a soundcard as output device, but jackd. So, 
applications using r/w can use benefits of jackd (of course, there will be 
one more period of samples buffered, but who will care when the start is 
synchronized?).

In my brain, there is also totaly different solution with the zero context 
switching overhead - sharing the soundcard DMA buffer among more 
applications. There is only one problem: snd_pcm_rewind() implementation 
cannot be perfect, because of wrapping added sample values (we lose 
information which cannot be recovered). The question is, if it's a fatal 
problem.

Let's discuss these things. Hopefully, I'll get some time to implement at 
least the mserver functionality.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[tomasz motylewski]

On Wed, 27 Nov 2002, Jaroslav Kysela wrote:

> In my brain, there is also totaly different solution with the zero context 
> switching overhead - sharing the soundcard DMA buffer among more 
> applications. There is only one problem: snd_pcm_rewind() implementation 

This is my personal preference. In this model the only service ALSA has to
supply are:

1. initial configuration/start/stop.
2. mmapable DMA buffer
3. fact and precise ioctl telling the current HW pointer in the buffer. If the
card is not queried each time, then the "last period interrupt" timestamp
should be included.

Please stop the complication of "available/delay" etc. Just the raw pointer.
Each application knows where its application pointer is, so it can easily
calculate delay/available and decide for itself whether there was an overrun or
not.

The driver does not care at all whether applications are writing to this buffer
or not. There should be one thread (ISR, kernel or separate application) which
zeroes part of the buffer just after the card has played it.

Each application/thread should not just copy the data to DMA buffer, but _add_
to it - in this way we get very efficient mixing (zero copy). Of course here I
assume that "add" operations on volatile buffers are atomic vs. context
switches - probably not on all architectures - we may need mutexes.

ALSA API should also tell the applications how big the "hot zone" is (the area
where the data is first played and then set to 0).

I was able to achieve about 20 ms delay (analog-to-analog) when transmitting 10
ms RTP packets of data over the LAN with no overruns! OK, I have used HZ=2000
and low latency patches+soft RT.

I am mixing several RTP  streams in this way, when 1 packet does not
arrive, the data from the other streams are there and I resynchronize just this
single stream. But the only buffering I do is in the DMA buffer of the card!
Each RTP source has its own "application ptr" and the data are added to the
buffer as they come.

Callbacks are not the most efficient there. If the data arrives 1 ms too late I
would loose the whole 10 ms period. If I still add it to the buffer, I will
loose only 1 ms and the card will play the remaining 9 ms of sound.

You will probably consider my method "ugly", "full of race conditions" etc. But
it gives mimimum sound latency.

Best regards,
--
Tomasz Motylewski



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[James Courtier-Dutton]

tomasz motylewski wrote:

>Please stop the complication of "available/delay" etc. Just the raw pointer.
>Each application knows where its application pointer is, so it can easily
>calculate delay/available and decide for itself whether there was an overrun or
>not.
>
>  
>
I use the delay() function.
I help write a multi media application that needs to sync audio to video.
So the question I ask alsa-lib is "If I write() now, how long will it be 
before those samples come out of the speakers?"
So, I very much need delay(), but I also use available so that I can 
program alsa when to exit the snd_pcm_wait() poll. I.E. Only exit the 
snd_pcm_wait when there are enough free space in the buffer for me to 
bother with a new write() call.

Cheers
James




-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

>This is my personal preference. In this model the only service ALSA has to
>supply are:
>
>1. initial configuration/start/stop.
>2. mmapable DMA buffer
>3. fact and precise ioctl telling the current HW pointer in the buffer. If the
>card is not queried each time, then the "last period interrupt" timestamp
>should be included.

ALSA supplies all 3 of these. You can use them by themselves without
the rest of the API if you want. The only problem arises with hardware
that cannot give you accurate h/w pointer positions.

>Please stop the complication of "available/delay" etc. Just the raw
>pointer.  Each application knows where its application pointer is, so
>it can easily calculate delay/available and decide for itself whether
>there was an overrun or not.

actually, it can't. if the user space application is delayed for
precisely 1 buffer's worth of data, it will see the pointer at what
appears to the the right place and believe that no xrun has
occured. the only way around this is to provide either:

	* h/w pointer position as a steadily incrementing value
	* h/w pointer position *plus* interrupt count

i favor the latter since it provides for a longer time before wrapping
becomes an issue (ULONG_MAX interrupts).

>Each application/thread should not just copy the data to DMA buffer, but _add_
>to it - in this way we get very efficient mixing (zero copy). Of course here I
>assume that "add" operations on volatile buffers are atomic vs. context
>switches - probably not on all architectures - we may need mutexes.

they are not atomic on any architecture that i know of.

>I was able to achieve about 20 ms delay (analog-to-analog) when
>transmitting 10 ms RTP packets of data over the LAN with no
>overruns! OK, I have used HZ=2000 and low latency patches+soft RT.

for the record, i've used this approach (inspired by someone from
poland whose name i forget) with an ISA card (no PCI burst size
issues) to achieve <10 sample processing latency. i scribbled a
special bit pattern into the hardware buffer, then looped over it
watching to see where the audio interface was and writing/reading data
directly ahead of it or behind it. it works, but of course it burns
CPU cycles like crazy (there is no wait for the device or anything
else, its just a continuous while loop).

>Callbacks are not the most efficient there. If the data arrives 1 ms
>too late I would loose the whole 10 ms period. If I still add it to
>the buffer, I will loose only 1 ms and the card will play the
>remaining 9 ms of sound.

this isn't a function of callbacks. its a function of the response of
the callback system to the delay. if the system notices that its
running late but goes ahead and executes anyway, the results are just
the same as you suggest for your approach. however, since there is
still a glitch in the audio stream (1ms of data is repeated/lost), a
system like JACK considers this an error.

--p



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Jaroslav Kysela]

On Wed, 27 Nov 2002, tomasz motylewski wrote:

> On Wed, 27 Nov 2002, Jaroslav Kysela wrote:
> 
> > In my brain, there is also totaly different solution with the zero context 
> > switching overhead - sharing the soundcard DMA buffer among more 
> > applications. There is only one problem: snd_pcm_rewind() implementation 
> 
> This is my personal preference. In this model the only service ALSA has to
> supply are:
> 
> 1. initial configuration/start/stop.
> 2. mmapable DMA buffer
> 3. fact and precise ioctl telling the current HW pointer in the buffer. If the
> card is not queried each time, then the "last period interrupt" timestamp
> should be included.

We have already this sort of timestamp - SNDRV_PCM_TSTAMP_MMAP, but it's 
quite useless when we haven't a continuous timer source. Hopefully, things 
will change after 2.5 when high-res timers are included.

> Please stop the complication of "available/delay" etc. Just the raw pointer.
> Each application knows where its application pointer is, so it can easily
> calculate delay/available and decide for itself whether there was an overrun or
> not.

I got your point. It would be good to let read hw_ptr and appl_ptr by 
application when it operates in "don't report xruns" mode (stop_threshold 
== ULONG_MAX). We can do it quite easy extending the current API.

> The driver does not care at all whether applications are writing to this buffer
> or not. There should be one thread (ISR, kernel or separate application) which
> zeroes part of the buffer just after the card has played it.
> 
> Each application/thread should not just copy the data to DMA buffer, but _add_
> to it - in this way we get very efficient mixing (zero copy). Of course here I
> assume that "add" operations on volatile buffers are atomic vs. context
> switches - probably not on all architectures - we may need mutexes.

Exactly, that's my idea. The question is the locking method. Mutexes are 
slow. Perhaps, we can try to write a king of "user land" spinlocks which 
contents is shared as the DMA ring buffer.

> ALSA API should also tell the applications how big the "hot zone" is (the area
> where the data is first played and then set to 0).

I think that hw_ptr is quite sufficient (application will start to write 
samples from hw_ptr + period_size).

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Avoiding xruns... was Re: smix plugin available?

[James Courtier-Dutton]

I have read your comments below, and I would like to try to explain the 
problems I am coming up against when writing a multi-media app.

I am not going to say that I know everything about kernel scheduling, 
but for multi media applications, avoiding xruns is a major concern. 
This becomes particularly important in SPDIF passthru modes, because if 
one is outputting AC3 or DTS non-audio packs to an external decoder, an 
xrun will corrupt an entire AC3 or DTS pack that is equivalent to 
anything between 512 - 4096 PCM Sample Frames. So, loosing a single 
sample will be very noticable to the user.
I am currently taking the following approach: -
Always prepare 2 audio hardware periods of sample frames in advance 
inside the user app.

1) snd_pcm_wait()
2) write()
3) prepare new sample frames, then go back to (1).

Is this approach the best approach to use in order to avoid xruns ?
Using the "plug" interface does make the user app easier to write, but 
is using the "plug" interface adding too much overhead so as to increase 
the risk of xruns too much ?

Cheers
James


Jaroslav Kysela wrote:

>On Wed, 27 Nov 2002, James Courtier-Dutton wrote:
>
>  
>
>>Paul Davis wrote:
>>
>>    
>>
>>> 
>>>
>>>      
>>>
>>>>>the APIs that are used to write almost all audio software code in 
>>>>>production these days all use a callback model. 
>>>>>     
>>>>>
>>>>>          
>>>>>
>>>>Sorry for questioning this statement. Of course we all don't have any statisti
>>>>cal data but
>>>>you miss what I see as the majority of applications that use audio devices:
>>>>
>>>>1) games
>>>>2) media players
>>>>3) GUI sounds (i.e. accessibility)
>>>>   
>>>>
>>>>        
>>>>
>>>this is a fair point. i apologize. i have my head so far inside the
>>>"audio software for musicians" box that i tend to fail to see such
>>>applications :)
>>>
>>>however, the very fact that the applications developers of such
>>>programs really don't know much (if anything) about audio programming,
>>>and probably don't want to know much about it either, suggests that an
>>>API like ALSA which exposes the full HAL is probably a mistake. again,
>>>i consider PortAudio a vastly preferable solution.
>>>
>>> 
>>>
>>>      
>>>
>>I would like to point out that a "callback" api would work just as well 
>>as an open/write/close api for
>>
>>1) games
>>2) media players
>>3) GUI sounds (i.e. accessibility)
>>
>>I have to agree with Paul on the fact that a "callback" approach is really the ONLY real option.
>>Here is my reasoning: -
>>1) My perspective is from "(2) Media players" and not "Pro-Audio"
>>2) Sound Hardware tends to have very small buffers.
>>3) For nice sounding audio, these buffers should never run dry. I.E. No XRUNs.
>>4) A open/write/close api will never ever be able to guarantee no XRUNs, as it has no control on when it will get scheduling time to do the next write.
>>5) With a "callback" approach, the kernel would be notified by the sound hardware that it was ready for new samples, the kernel could then adjust the scheduler, so that the "callback" function was called ASAP.
>>The "callback" function then only has to decide which samples to give. If the "callback" function could receive a "delay" value from the sound hardware at each callback, a media player would then have all the information it needed to do full audio/video sync.
>>    
>>
>
>Sorry, it's not as easy as you've described. It's not possible to invoke
>any user code from the kernel code directly. There is a scheduler which is
>informed that a task has been woken up. It depends on scheduler when the
>task is really invoked. It's quite same as for the r/w model where the
>application is notified over poll that something occured.
>
>  
>
>>6) I don't need "sample sync", but I do NEED "callback" based api to provide me with "no XRUNs".
>>    
>>
>
>I don't think that there is some difference. If the scheduler don't give 
>you enough time, the audio stream is somehow broken on all architectures.
>
>  
>
>>Summary: - 
>>The only way to cure XRUN problems is with a "callback" based api.
>>All application that currently use open/write/close apis, can just as easily use a "callback" api.
>>    
>>
>
>Let's go and see the implementation:
>
>The callback model is good for perfect sync between applications. It can 
>do (and does) chaining of more sources, arbitrating (removing invalid 
>sources) and so on. It is simply something "over" the audio HAL. If it 
>really helps, it's a different point.
>
>The discussed difference (a few months ago) was in a count of the task 
>context switches.
>
>With jack, you have these context switches (daemon and two applications 
>mixed together) for one period of samples (* means a context switch):
>
>jackd -*> app1 -*> app2 -*> jackd -> soundcard
>
>With r/w model and a sample mixing server implemented in the user space, 
>you can get this for one period of samples:
>
>mserver -> soundcard
>        -*(ring buffer does not contain enough samples)> app1
>        -*(ring buffer does not contain enough samples)> app2
>
>In real real-time setup, there will be two periods (thus app1 and app2) 
>will be woken up all times. So, in real world the context switches are for 
>one period of samples:
>
>mserver -*> app1 -*> app2   or
>mserver -*> app2 -*> app1
>
>Note: mserver implementation uses same assumptions as jack (or any other
>callback model). The ring buffers are shared between app and mserver and
>the period of samples is a constant. The playback / capture pointers are
>incrementing by the period size steps. Thus there is no need to commit 
>result directly back to the mserver. mserver will be woken up by the 
>kernel scheduler when next poll() event occurs (on next period boundary).
>
>Ok, it's only simple example, that there are more solutions than Paul 
>suggests. I fully agree, that the callback model is suitable for the 
>perfect synchronization among more applications. Also, imagine that
>mserver is not using a soundcard as output device, but jackd. So, 
>applications using r/w can use benefits of jackd (of course, there will be 
>one more period of samples buffered, but who will care when the start is 
>synchronized?).
>
>In my brain, there is also totaly different solution with the zero context 
>switching overhead - sharing the soundcard DMA buffer among more 
>applications. There is only one problem: snd_pcm_rewind() implementation 
>cannot be perfect, because of wrapping added sample values (we lose 
>information which cannot be recovered). The question is, if it's a fatal 
>problem.
>
>Let's discuss these things. Hopefully, I'll get some time to implement at 
>least the mserver functionality.
>
>						Jaroslav
>
>-----
>Jaroslav Kysela <perex@suse.cz>
>Linux Kernel Sound Maintainer
>ALSA Project, SuSE Labs
>
>
>
>  
>




-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: Avoiding xruns... was Re: smix plugin available?

[Paul Davis]

>I am currently taking the following approach: -
>Always prepare 2 audio hardware periods of sample frames in advance 
>inside the user app.
>
>1) snd_pcm_wait()
>2) write()
>3) prepare new sample frames, then go back to (1).

for lower latency, you'd do:

 1) snd_pcm_wait()
 2) prepare new sample frames
 3) write(), then go back to (1).

but for the kinds of things you are describing, your original order
seems OK.

--p


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

>Ok, it's only simple example, that there are more solutions than Paul 
>suggests. I fully agree, that the callback model is suitable for the 
>perfect synchronization among more applications. 

Let's be totally clear about this. its not just that the callback
model is suitable - the mserver model will actually not work for
sample sync between applications. I have always been sure that the
mserver model will work well as a replacement for things like esd and
artsd. if that's all that we needed, i would never have started work
on jack but would have put my work into the mserver model.

However, for the class of users i am interested in, the mserver model
isn't adequate, and thats why jack exists.

>						  Also, imagine that
>mserver is not using a soundcard as output device, but jackd. So, 
>applications using r/w can use benefits of jackd

precisely. this is one the main reasons i'd like to see the mserver
stuff working.

>In my brain, there is also totaly different solution with the zero context 
>switching overhead - sharing the soundcard DMA buffer among more 
>applications. 

i have been thinking about this from a different perspective
recently. i recently modified ardour's internals so that the total
data flow for an audio interface looks like:

    hardware buffer 
        -> JACK ALSA driver output port buffer 
               -> ardour output port buffer
                    -> hardware buffer

it keeps occuring to me that there is no technical reason why we have
the two intermediate copies. it would be amazing to find a way to
export the mmap'ed hardware buffer up into user space as shared
memory, tell JACK to use this for the port buffers of the ALSA jack
client, and then we can skip the copy. JACK will need some minor
modifications to its internals to permit just a single step, but i can
see how to do that. without that fix, it would still be better:

    hardware buffer == JACK ALSA driver output port buffer
       -> ardour output port buffer
            -> JACK ALSA driver input port buffer == hardware buffer

i see this as more promising than the approach i think you are
thinking of. you can't avoid the context switching - they *have* to
happen so that the apps can run!! the question is *when* does it
happen ... in JACK, they are initiated in a chain when the interface
interrupts us. in the mserver model and/or the shared mmap'ed buffer
approach, they just have to happen sometime between interrupts
(otherwise the buffers are not handled in time). so there is no
avoiding them, its just a matter of when they happen. the point of
JACK's design is to force sample sync and to minimize latency - always
generating and processing audio as close to when it is handled by the
hardware as possible (hence the default 2 period setting). a model
that allows the context switching to occur "sometime" between
interrupts is more relaxed, but loses sample sync and slightly
increases (some kinds of) latency. that doesn't mean i think its a
stupid system, just one that lacks certain properties.

however, i do think that finding a kernel mechanism that would allow
the mmap'ed buffer to be used the way that shared memory can be used
is potentially extremely useful. even though data copying on modern
machines doesn't consume much of the cycles burnt by almost any audio
software, its still a cost that it would nice to reduce.

--p


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Kai Vehmanen]

On Wed, 27 Nov 2002, Paul Davis wrote:

> Let's be totally clear about this. its not just that the callback
> model is suitable - the mserver model will actually not work for
> sample sync between applications. I have always been sure that the

I think this is the critical point. ALSA's smix/mserver doesn't actually
have to provide this. As Jaroslav has described, this simplifies the
implementation of the server code (as it has to provide the full ALSA PCM
API to clients), and as mentioned by Florian, this kind of server
functionality would be generally useful for a wide selection of
applications.

> However, for the class of users i am interested in, the mserver model
> isn't adequate, and thats why jack exists.

Yup, in addition to synchronous execution, JACK also provides interfaces 
for connection and transport management (both of which are important 
but not necessarily interesting to ALSA).

> i see this as more promising than the approach i think you are
> thinking of. you can't avoid the context switching - they *have* to
> happen so that the apps can run!! the question is *when* does it
> happen ... in JACK, they are initiated in a chain when the interface
> interrupts us. in the mserver model and/or the shared mmap'ed buffer
> approach, they just have to happen sometime between interrupts
> (otherwise the buffers are not handled in time). so there is no
> avoiding them, its just a matter of when they happen. the point of
> JACK's design is to force sample sync and to minimize latency - always
> generating and processing audio as close to when it is handled by the
> hardware as possible (hence the default 2 period setting). a model
> that allows the context switching to occur "sometime" between
> interrupts is more relaxed, but loses sample sync and slightly
> increases (some kinds of) latency. that doesn't mean i think its a
> stupid system, just one that lacks certain properties.

That nicely summarizes the whole thing. I think there's room 
for both servers. It should be possible to implement plugins
for both mserver-using-jack and jack-using-mserver scenarios.

> however, i do think that finding a kernel mechanism that would allow
> the mmap'ed buffer to be used the way that shared memory can be used
> is potentially extremely useful. even though data copying on modern
> machines doesn't consume much of the cycles burnt by almost any audio
> software, its still a cost that it would nice to reduce.

Btw; I tend to think the memory bandwidth usage is a secondary issue.
Transferring data over PCI is somewhat problematic, but that's anyway 
done only once per direction. But as for memory bandwidth, today's 
computers supply a _lot_ of it. For instance PC133-SDRAM:1.06GB/s
and DDR333:2.7GB/s. 48 channels of 32bit data at 96kHz is only 17.5MB/s!
Of course, as we only want to spend small fraction of the callback to 
memory transfers it doesn't make sense to directly compare these 
figures, but they do give some perspective to the issue.

The primary issue is fast and reliable user-mode process wake-up.
Once you have that, and have deterministic audio code-paths, 
the rest is just a question of how complex processing tasks you can do 
on a given hardware (... and for once, just buying more hardware 
can solve problems).

-- 
 http://www.eca.cx
 Audio software for Linux!



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Jaroslav Kysela]

On Wed, 27 Nov 2002, Paul Davis wrote:

> i see this as more promising than the approach i think you are
> thinking of. you can't avoid the context switching - they *have* to
> happen so that the apps can run!! the question is *when* does it
> happen ... in JACK, they are initiated in a chain when the interface
> interrupts us. in the mserver model and/or the shared mmap'ed buffer
> approach, they just have to happen sometime between interrupts
> (otherwise the buffers are not handled in time). so there is no
> avoiding them, its just a matter of when they happen. the point of
> JACK's design is to force sample sync and to minimize latency - always
> generating and processing audio as close to when it is handled by the
> hardware as possible (hence the default 2 period setting). a model
> that allows the context switching to occur "sometime" between
> interrupts is more relaxed, but loses sample sync and slightly

Note that "sometime" means in same time as jack invokes the clients. The 
r/w applications blocks at the r/w or poll point, so they will be 
activated when the condition for the ring buffer is matched. Thus for 2 
periods - conditions are same as for jack.

Anyway, as Kai stated, we are reaching a consensus that both ways are 
useful for some cases.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs




-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

>Sorry, it's not as easy as you've described. It's not possible to invoke
>any user code from the kernel code directly. There is a scheduler which is
>informed that a task has been woken up. It depends on scheduler when the
>task is really invoked. It's quite same as for the r/w model where the
>application is notified over poll that something occured.

i think we can consider the behaviour of the kernel scheduler when it
schedules a SCHED_FIFO task after an audio interface interrupt has
woken it to be very, very close to the kind of callback system james
is describing. ditto for any kind of wakeup of a SCHED_FIFO task
(e.g. when its woken by a write to pipe, or a signal).

its not perfect, as i am very disappointed to discover, but its also
very close. 

--p



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Jaroslav Kysela]

On Wed, 27 Nov 2002, Paul Davis wrote:

> >Sorry, it's not as easy as you've described. It's not possible to invoke
> >any user code from the kernel code directly. There is a scheduler which is
> >informed that a task has been woken up. It depends on scheduler when the
> >task is really invoked. It's quite same as for the r/w model where the
> >application is notified over poll that something occured.
> 
> i think we can consider the behaviour of the kernel scheduler when it
> schedules a SCHED_FIFO task after an audio interface interrupt has
> woken it to be very, very close to the kind of callback system james
> is describing. ditto for any kind of wakeup of a SCHED_FIFO task
> (e.g. when its woken by a write to pipe, or a signal).
> 
> its not perfect, as i am very disappointed to discover, but its also
> very close. 

If you have more processes with same priority, the time gaps might be 
noticable as well.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Jaroslav Kysela]

On Wed, 27 Nov 2002, Paul Davis wrote:

> >This is my personal preference. In this model the only service ALSA has to
> >supply are:
> >
> >1. initial configuration/start/stop.
> >2. mmapable DMA buffer
> >3. fact and precise ioctl telling the current HW pointer in the buffer. If the
> >card is not queried each time, then the "last period interrupt" timestamp
> >should be included.
> 
> ALSA supplies all 3 of these. You can use them by themselves without
> the rest of the API if you want. The only problem arises with hardware
> that cannot give you accurate h/w pointer positions.
> 
> >Please stop the complication of "available/delay" etc. Just the raw
> >pointer.  Each application knows where its application pointer is, so
> >it can easily calculate delay/available and decide for itself whether
> >there was an overrun or not.
> 
> actually, it can't. if the user space application is delayed for
> precisely 1 buffer's worth of data, it will see the pointer at what
> appears to the the right place and believe that no xrun has
> occured. the only way around this is to provide either:
> 
> 	* h/w pointer position as a steadily incrementing value
> 	* h/w pointer position *plus* interrupt count
> 
> i favor the latter since it provides for a longer time before wrapping
> becomes an issue (ULONG_MAX interrupts).

The ALSA internal code already uses hw_ptr and appl_ptr within range 0 to
boundary, where boundary is close to ULONG_MAX and expression boundary / 
period_size == an integer value. So, the hardware pointer also contains 
count of hardware interrupts as well.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

>> actually, it can't. if the user space application is delayed for
>> precisely 1 buffer's worth of data, it will see the pointer at what
>> appears to the the right place and believe that no xrun has
>> occured. the only way around this is to provide either:
>> 
>> 	* h/w pointer position as a steadily incrementing value
>> 	* h/w pointer position *plus* interrupt count
>> 
>> i favor the latter since it provides for a longer time before wrapping
>> becomes an issue (ULONG_MAX interrupts).
>
>The ALSA internal code already uses hw_ptr and appl_ptr within range 0 to
>boundary, where boundary is close to ULONG_MAX and expression boundary / 
>period_size == an integer value. So, the hardware pointer also contains 
>count of hardware interrupts as well.

i know that. my point was that wraparound happens much earlier than if
we used a two part system (pointer-in-buffer + interrupt count). this
would be a similar system to the one used by the RTC, btw. we wouldn't
wrap with a two part system for around 66 days with 64 frames/period
at 48kHz. in the current system we wrap in about 1 day or so.

it doesn't make much difference either way, i was just pointing out a
different way to do it, and more importantly, correcting the claim
that by knowing where the h/w pointer is *in the buffer*, the
application can know if there has been an xrun. the ALSA API h/w
pointer, as you point out, implicitly contains the interrupt count,
but its not as raw as was being requested.

its also worth noting that it too is not immune from missing xruns,
but there isn't anything we can do about kernel/driver code that
blocks interrupts for an entire buffer :(

--p


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[tomasz motylewski]

On Wed, 27 Nov 2002, Paul Davis wrote:

> >Please stop the complication of "available/delay" etc. Just the raw
> >pointer.  Each application knows where its application pointer is, so
> >it can easily calculate delay/available and decide for itself whether
> >there was an overrun or not.
> 
> actually, it can't. if the user space application is delayed for
> precisely 1 buffer's worth of data, it will see the pointer at what
> appears to the the right place and believe that no xrun has
> occured. the only way around this is to provide either:

Well, but if you combine it with the current time information, then you will
know whether the buffer has wrapped around or not - if you have 1000 ms of
buffer, but always run 20 ms from the front, then if you notice 1010 ms has
passed since the last action - then you have an overrun, even if the HW pointer
seems to be in the right place. So, configuration + mmaped buffer + gettimeoday
+ HW pointer and I am happy. And may be system call "mix data into buffer
(start, n_frames,data)" for atomic add. This provides minimum interference
between programs (if they will agree on frame rate, format, etc.).

> for the record, i've used this approach (inspired by someone from
> poland whose name i forget) with an ISA card (no PCI burst size
> issues) to achieve <10 sample processing latency. i scribbled a
> special bit pattern into the hardware buffer, then looped over it
> watching to see where the audio interface was and writing/reading data
> directly ahead of it or behind it. it works, but of course it burns
> CPU cycles like crazy (there is no wait for the device or anything
> else, its just a continuous while loop).

Well, I use that method for reading data - I know how much samples I need
to gather before I send them, so I may calculate how many ms to sleep. If I
read few samples too much, I will just sleep less next time. Jitter about 300
microseconds in RTP data stream which I send, and quite CPU efficient.

But how did you do for writing data to the card? Are your read and write buffer
HW synchronized?


> this isn't a function of callbacks. its a function of the response of
> the callback system to the delay. if the system notices that its
> running late but goes ahead and executes anyway, the results are just
> the same as you suggest for your approach. however, since there is

well, no. Lets say I have got "write callback" in time and I should give some
data to the card. But I do not have it - it may arrive in 1 ms, but then I can
only through it away, because I will get next "write callback" in 9 ms from
then.

I like callbacks for "read" direction.

Best regards,
--
Tomasz Motylewski



-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en

Re: smix plugin available?

[Paul Davis]

>> actually, it can't. if the user space application is delayed for
>> precisely 1 buffer's worth of data, it will see the pointer at what
>> appears to the the right place and believe that no xrun has
>> occured. the only way around this is to provide either:
>
>Well, but if you combine it with the current time information, then
>you will know whether the buffer has wrapped around or not - if you
>have 1000 ms of buffer, but always run 20 ms from the front, then if
>you notice 1010 ms has passed since the last action - then you have
>an overrun, even if the HW pointe r seems to be in the right
>place. So, configuration + mmaped buffer + gettimeoda y + HW pointer
>and I am happy. 

you should trying playing with a system driven by varispeed word
clock. you'll soon find that gettimeofday(2) is completely useless to
you. the sample clock is the only clock that counts. say it twice
every morning when you get up :)

>		 And may be system call "mix data into buffer (start,
>n_frames,data)" for atomic add. This provides minimum interference
>between programs (if they will agree on frame rate, format, etc.).

that is quite a big "if". still, the plughw layer can help with that i
suppose. 

>Well, I use that method for reading data - I know how much samples I need
>to gather before I send them, so I may calculate how many ms to sleep. If I

if you have a kernel that will reliably sleep for 1ms, i envy you :)

>But how did you do for writing data to the card? Are your read and
>write buffer HW synchronized?

i used the same technique as for reading. the buffer gets scribbled
with a bit pattern and we look for the "leading edge" location with
two bytes containing the special pattern. this minimizes the problems
caused by actual sample values that match that pattern. the chances of
two in a row doing so are so small as to be mostly ignorable.

but note: this technique doesn't work so well for PCI cards because of
the bus transfer block size.

--p


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en