Software mixer at kernel level

Software mixer at kernel level

[Mauricio Nuñez]

Hi, I was wondering if it could be possible to implement a low-level software mixer (like OSS4's vmix) since almost everybody needs it and PulseAudio causes too much overhead on not-so-old systems.

_________________________________________________________________
Explore the seven wonders of the world
http://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=QBRE

Re: Software mixer at kernel level

[Sean McNamara]

Hi,

On Mon, Feb 9, 2009 at 9:02 PM, Mauricio Nuñez
<mauricio_n2000@hotmail.com> wrote:
>
> Hi, I was wondering if it could be possible to implement a low-level software mixer (like OSS4's vmix) since almost everybody needs it and PulseAudio causes too much overhead on not-so-old systems.

A few points that have been raised in past discussions of kernel-mode
software mixing:

1. You're stuck with integer precision. Floating point arithmetic in
the kernel is a BIG no-no.
2. Kernel core developers (including Linus) have made it clear that
this kind of math operation should be done in user mode. The reasoning
is that any bug in the kernel can bring down the whole system, or at
least, an entire subsystem (audio, graphics, etc). The point here is
that software mixing is aptly named, in that the mixing is done in
/software/, while the kernel's primary function is to interface with
/hardware/. The kernel needs special privileged memory access,
firmware access, etc. to carry out these operations, but you don't
need any special access beyond what the ALSA pcm interface already
provides, to effectively do software mixing in userspace. From a
perspective of security alone, we would want to follow the policy of
minimal privilege level wherever possible: software should not run
with more privileges than it needs to carry out its function. You
can't effectively limit permissions in kernel mode due to the hardware
design.

I would also add to the discussion that, unlike you insinuated, doing
software mixing in the kernel would not reduce its overhead
significantly. The CPU resources needed by PulseAudio are mainly due
to functions such as resampling, which, if implemented, the ALSA
kernel modules would have to do under software mixing. These
algorithms are quite optimal already given current computer science
theory, and the code is operating without much runtime overhead (it's
written in C).

However, the point about "everybody needs it" is a valid one. If I had
my druthers, it would be impossible for anyone in usermode to
initialize an ALSA pcm in a mode which does not provide software
mixing. We can still provide the software mixing in usermode, but my
reasoning is that

(a) Not using software mixing (i.e., hogging the sound card) makes
your app a poor desktop citizen, because it effectively causes other
programs to appear to malfunction from the user's point of view.
(b) There is very little justification for using the functions that
are part of the "unsafe" ALSA subset. By contrast, if everyone only
used the "safe" ALSA subset, then requiring software mixing (such as
dmix) would be fine. It's only when people want features such as
mmap() that we run into problems with software mixing.
(c) By requiring software mixing at the ALSA-lib user-mode layer, we
can prevent those poor citizens from affecting other desktop apps; we
can guarantee user experience; and we can help establish a broader
consensus as to the sound APIs that should be used on the free
desktop.

Of course, PulseAudio can be said to require software mixing: you
can't be a PulseAudio client without your data stream being subject to
its mixing routines. So they've already done what I just proposed. The
only difference is that, since it's built on top of ALSA, the same
issues are still present within ALSA as have always been there. It's
just that, for backwards compatibility's sake, we can't rightly change
our mind on these things now, unless we can live without things such
as JACK, Ardour, Audacity, Teamspeak, and a host of others.

At some basic level, software mixing requires proper tuning of the
kernel; a fairly recent system; and a "sane" compromise between
desired computational complexity and actually available computing
resources. For instance, trying to use the highest quality floating
point resampling in the libspeex library on a first-generation Pentium
4 might cause some sound quality problems when the system is under
load. That's because you're asking a lot of the CPU to make all those
floating point calculations. The mixing can be done more cheaply, but
you have to pay, to some extent, with quality: either integer samples,
or faster and less accurate algorithms.

If the existing software mixing solutions in usermode are not
performing to your liking, then you may consider upgrading your system
so that these solutions perform adequately for you; or, live with less
expensive resampling algorithms (if you're using PulseAudio). OTOH, if
you are simply dreaming of a day when software mixing is taken for
granted... I think the only way to get there is to keep educating
developers that they need to write their software with a "soft" PCM in
mind, rather than assuming they are talking directly to hardware.

But I can not see how kernel mode mixing will get us there any faster
or easier; on the contrary, it is likely to introduce instability and
become a maintenance nightmare.

HTH,

Sean

P.S. - Not a knock on OSS4 at all, but I have had my system kernel
panic with OSS4 under many different kernels and hardwares over the
years. I sort of liked it for a while when it worked well, but ALSA is
leaps and bounds more stable. This could be a sort of case study, if
you're interested, as evidence supporting the theory that kernel mode
mixing is a bad idea.

>
> _________________________________________________________________
> Explore the seven wonders of the world
> http://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=QBRE
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>

Re: Software mixer at kernel level

[Lee Revell]

On Mon, Feb 9, 2009 at 9:02 PM, Mauricio Nuñez
<mauricio_n2000@hotmail.com> wrote:
>
> Hi, I was wondering if it could be possible to implement a low-level software mixer (like OSS4's vmix) since almost everybody needs it and PulseAudio causes too much overhead on not-so-old systems.
>

What kind of overhead?  Which systems?  Details please.

It works perfectly for me with Ubuntu 8.10 on my work PC, with a
relatively old and buggy version of Pulse.

Lee

Re: Software mixer at kernel level

[stan]

Sean McNamara wrote:
> Hi,
> 

> 
> However, the point about "everybody needs it" is a valid one. If I had
> my druthers, it would be impossible for anyone in usermode to
> initialize an ALSA pcm in a mode which does not provide software
> mixing. We can still provide the software mixing in usermode, but my
> reasoning is that
> 

<rant>
I disagree.  I have one of those applications that doesn't do softwae mixing, deliberately because I want it to have 
good sound (it's a brainwave entrainment app).  It calls functions in alsa that *can't* be accessed through pulse, and 
allows users to set the frame rate to any hardware rate their card supports (in pulse, one size fits all).

Pulse is great for those who want its functionality, and I would love to see it replace dmix in alsa so that when it 
received a request for a function it doesn't support it would have the ability to step aside and allow the application 
to take over the sound device.

Linux is about freedom, and I think you are wanting to restrict that freedom for me to use my hardware the way I want. 
If you want to put a switch in place that does that, but allows it to be turned off, great, but not hard wired to only 
behave a single way.  Wait!  We already have such a switch, it's called /home/.asoundrc.

You are welcome to do whatever you want on your system, but don't assume that everyone wants what you want.
</rant>

Re: Software mixer at kernel level

[Sean McNamara]

On Mon, Feb 9, 2009 at 10:57 PM, stan <ghjeold_i_mwee@cox.net> wrote:
> Sean McNamara wrote:
>>
>> Hi,
>>
>
>>
>> However, the point about "everybody needs it" is a valid one. If I had
>> my druthers, it would be impossible for anyone in usermode to
>> initialize an ALSA pcm in a mode which does not provide software
>> mixing. We can still provide the software mixing in usermode, but my
>> reasoning is that
>>
>
> <rant>
> I disagree.  I have one of those applications that doesn't do softwae
> mixing, deliberately because I want it to have good sound (it's a brainwave
> entrainment app).  It calls functions in alsa that *can't* be accessed
> through pulse, and allows users to set the frame rate to any hardware rate
> their card supports (in pulse, one size fits all).
>
> Pulse is great for those who want its functionality, and I would love to see
> it replace dmix in alsa so that when it received a request for a function it
> doesn't support it would have the ability to step aside and allow the
> application to take over the sound device.
>
> Linux is about freedom, and I think you are wanting to restrict that freedom
> for me to use my hardware the way I want. If you want to put a switch in
> place that does that, but allows it to be turned off, great, but not hard
> wired to only behave a single way.  Wait!  We already have such a switch,
> it's called /home/.asoundrc.

Sorry, but I think you misinterpreted my suggestion.

Let me sort of imagineer the kind of functionality I was discussing in
a bit more detail, and then you can re-evaluate whether I'm trying to
restrict things in a way that you need to rant about, ok?

First, let's review how it is today.

Let's say I'm using dmix, and it's in my /etc/asound.conf as the
"default" PCM. I am happily going along, starting applications that
obey the tacit convention of opening the device named "default", which
by convention is supposed to contain whatever PCM pluggage that the
user wants.

Now let's say I open some application that, for some reason, wants to
access hw:0. As soon as I try to do this while dmix-using apps aren't
playing sound, hw:0 is now taken, and the dmix apps can no longer play
sound.

This, to me, is a very bad idea for the use case of the desktop. This
should be worked from two sides. Most importantly, we need to make
sure that desktop apps (i.e., the vast majority of apps) should be
able to use dmix or maybe even PulseAudio through the ALSA plug layer.
But an adjunct possibility would be to have a setting somewhere in
ALSA which _prevents_ applications from directly accessing hw:0 in a
way that does not use software mixing. We can then define a set of
plugins (jack, pulse, dmix, dsnoop... any others?) which safely enable
software mixing, and maybe we can have policy exceptions for
individual binaries, to support daemons.

With the changes I was referring to when I wrote my original post, the
new functionality would be thus:

1. [Prerequisite] Start PulseAudio or JACK if desired; if not, set up
dmix as the pcm.!default.
2. Set an option somewhere in the ALSA configs that says "Apps [which
are not in the exception list] may no longer hog the soundcard."
3. Apps that want to play nice with the desktop will access
dmix/pulseaudio/JACK using the "safe" ALSA subset and will open the
"default" device, and not hw:0.
4. An app that wants to open a direct PCM (ignoring the "default"
device) tries to access hw:0 while the other apps are temporarily
quiet. alsa-lib immediately returns an error, whether or not the
device is in use.
5. The apps designed to use software mixing continue on their way uninterrupted.

We could also have a distro-supplied hook which displays a GUI when an
app tries to directly access hw:0. It will be disallowed regardless of
whether the device is busy or not, if the configuration bit is set.

This behavior has the benefit of increasing the reliability of apps
which _do_ use software mixing, because it ensures they will never be
blocked by an app that directly accesses "hw:x". For desktop software
that _should_ be compatible with software mixing but is not, it has
the side-effect of bringing them into the limelight as ones that the
community should try to fix, if possible, to live with the safe subset
and open "default" instead of "hw:x". But as you can see, it in no way
entails that you will no longer be able to open a direct hw PCM.

I guess I can see how you /could/ interpret my original post as
implying some sort of draconian prevention of direct PCM access (with
no way around it), but please, let's be charitable: this is the FOSS
world, and wherever there is need for a bit-flip on points of
disagreement, there usually is one.

The change I've recommended here is merely air code, but its intention
is to suggest one way of making the desktop sound experience more
reliable, _not_ to close off paths for very special use cases like
brainwave entrainment. I'm sure you already realize the consequences
of hogging the "hw:x" devices, so it goes without saying that your app
can't work well in a desktop environment using sounds from many
programs at once. But that's a design decision you have planned for
and accepted, and as such, your program is not designed to be desktop
software on the order of Firefox, Rhythmbox, Pidgin, Evolution,
Phonon, and so on. Since your requirements are so very different, we
can not ignore either the desktop use case or your own special use
case. But that doesn't mean our hands are tied in terms of trying to
make the desktop experience better.

How many times have users wondered why their
Pidgin/Skype/Rhythmbox/Flash stuff won't play sound after they fire up
Audacity, or Second Life? This _is_ a problem with the applications
and not ALSA, but unless we try to encourage use of software mixing,
the issues don't seem to be getting addressed.

Anyway, back on topic to the OP, this matter of "firmer" software
mixing is the only valid point I could salvage from their idea, so
hopefully you'll take me with a grain of salt for talking through his
issue to its logical conclusion :)

Thanks,

Sean

>
> You are welcome to do whatever you want on your system, but don't assume
> that everyone wants what you want.
> </rant>
>

Re: Software mixer at kernel level

[Brendan Pike]

Sean McNamara wrote:
> 
> You are welcome to do whatever you want on your system, but don't assume that everyone wants what you want.
> </rant>
> _______________________________________________


Or you could just avoid all this mess with Linux audio for the time
being and use a hardware mixing device instead. I realize laptop users
are going to bitchslap me for saying that but its true.

The whole buggy software mixing situation with regards to applications
not working correctly or as intended is frustrating. OSS4 does it at
the kernel level because it tries to be transparent to all the
applications which think the device has hardware mixing capabilities. I
realize OSS4 can be buggy for some but its implementation is far more
compatible instead of the cocktail of sound servers and software
mixing routines we have in ALSA right now.

Some apps out there require direct access to /dev/dsp or whatever, what
are we going to do? Hope somebody updates the program? (usually not
going to happen with older closed source games or whatnot). Take UT99
for example, rarely does this ever work with any software mixing period.

Re: Software mixer at kernel level

[Takashi Iwai]

At Tue, 10 Feb 2009 01:50:54 -0500,
Brendan Pike wrote:
> 
> Sean McNamara wrote:
> > 
> > You are welcome to do whatever you want on your system, but don't assume that everyone wants what you want.
> > </rant>
> > _______________________________________________
> 
> 
> Or you could just avoid all this mess with Linux audio for the time
> being and use a hardware mixing device instead. I realize laptop users
> are going to bitchslap me for saying that but its true.
> 
> The whole buggy software mixing situation with regards to applications
> not working correctly or as intended is frustrating. OSS4 does it at
> the kernel level because it tries to be transparent to all the
> applications which think the device has hardware mixing capabilities. I
> realize OSS4 can be buggy for some but its implementation is far more
> compatible instead of the cocktail of sound servers and software
> mixing routines we have in ALSA right now.
> 
> Some apps out there require direct access to /dev/dsp or whatever, what
> are we going to do? Hope somebody updates the program? (usually not
> going to happen with older closed source games or whatnot). Take UT99
> for example, rarely does this ever work with any software mixing period.

Well, we can categorize the demands:

A. OSS-style access needed just because of fixed, binary-only things
B. OSS-style access needed because of old written codes
C. OSS-style access needed because of resource reduction (without
   server)

The original question was about C. 
However, in this case, it's not guaranteed whether a kernel-side
mixing requires less resources.  A server solution could be in less 
amount, depending on the implementation.  Nevertheless, one certain
thing is that this question is not for generic desktop uses, but for a
pretty limited use-case.

Honestly, I don't mind if anyone introduces a kernel-side thing for
that limited purpose like C for his own project.  But, this won't hit
the mainline, anyway...

For other cases, the resource usage isn't *THAT* big problem at all.
The only question is whether it works reliably or not.

It seems that hooking via LD_PRELOAD doesn't work always.  Another
interesting approach for the OSS fixed devices is to route via a new
FUSE char-driver extension by Tejun.  This is of course more layer
onto the existing setup, but it just works for cases A & B.


Takashi

Re: Software mixer at kernel level

[stan]

Sean McNamara wrote:
> On Mon, Feb 9, 2009 at 10:57 PM, stan <ghjeold_i_mwee@cox.net> wrote:
>> Sean McNamara wrote:
>>> Hi,
>>>

Big snip

> 
> Thanks,
> 
> Sean
> 

I think we are in agreement after I read your followup.  Thanks.

Re: Software mixer at kernel level

[Colin Guthrie]

'Twas brillig, and stan at 10/02/09 03:57 did gyre and gimble:
> (in pulse, one size fits all).

While I don't want to comment on the finer points of this thread, I will 
say that pulse 0.9.15 will be able to switch "profiles" on the fly.

I'm not 100% sure if it currently supports switching sample rates, but 
this should be trivial to implement now if it's not.

This should allow for an app which requires high quality output to 
request a specific hardware sample rate. Not sure how this would work in 
practice in terms of what apps are allowed to do this etc. but it should 
prevent software resampling kicking in in some circumstances.

Apologies if I've misinterpreted this new functionality (and it is a 
little theoretical in terms of putting it into practice), but I think 
i'm right :)

Col


-- 

Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
   Tribalogic Limited [http://www.tribalogic.net/]
Open Source:
   Mandriva Linux Contributor [http://www.mandriva.com/]
   PulseAudio Hacker [http://www.pulseaudio.org/]
   Trac Hacker [http://trac.edgewall.org/]