desktop and multimedia as an afterthought?

desktop and multimedia as an afterthought?

[Florin Andrei]

I'm not a kernel developer. I don't watch this mailing list too often.
I use Linux for pretty much everything, and especially for multimedia
(capture, process and edit video, music and sound processing, MIDI,
etc.).

I'm interested in anything that could make my system work better for my
multimedia apps. I used the Con Kolivas patches very early on, and i'm
also watching with interest the recent preemption patch posted by Ingo
Molnar.

I could be wrong, but it seems to me that at least a part of the kernel
development team has the desktop and multimedia issues very low on the
priority list.
The CK patches floated around as separate patches for a long time, even
though they brought significant improvements to the kernel w.r.t.
desktop and media.
Now the stock 2.6 kernel has a pretty bad problem with the latency.
Also, there seems to be a reluctance in accepting the autoregulated
swappiness patch, even though it markedly improves the overall behaviour
of the system as a desktop.

I am not familiar with the intricate technicalities of the Linux kernel,
so there might be purely technical reasons for keeping multimedia
improvements at an arm's length.
But on the other hand, there are many Linux users, myself included, that
repeatedly noticed how bad the stock Linux kernel is from a multimedia
perspective and how big are the improvements brought by these
seldom-if-at-all-included patches.

The CK patches have devouted "cult followers".
Projects such as PlanetCCRMA, which attempt to build a multimedia-ready
Linux distribution, are running kernels patched with the CK patches by
default.
On the Linux multimedia mailing lists and forums, one can often hear
advices in the vein of "use such-and-such patch if you want your system
to do any serious multimedia work, the vanilla kernel sucks."
And rightly so. If i reboot my computer into Windows and perform the
same multimedia tasks, there are fewer chances of it skipping frames or
otherwise behaving improperly for multimedia work than on Linux running
the vanilla kernel.

I mean, go read the forums. No one in the multimedia community uses the
vanilla kernel. They could be all wrong, sure, but there's lots of them.
:-)

There's a feature of the human brain to see patterns everywhere - i
think i see a pattern here. :-) It's like the desktop in general and
multimedia in particular matter a whole lot less than anything else on
Linux.

It seems like there's a split-brain situation within the Linux
community, with the core kernel developers sitting on one side of the
rift, and the multimedia people on the other side.

Now, i remember someone saying, a while ago, that the server stuff is
pretty much done, and the interesting things are going to happen on the
desktop. That sounds plausible, but if the kernel has poor support for
typical desktop scenarios, how are those big desktop improvements going
to take place?

I'm not saying that the whole thing happens on purpose. I'm just saying
that perhaps the latency and swappines issues and all might deserve a
bit more attention.

Thank you, and i'm looking forward to constructive criticism.

-- 
Florin Andrei

http://florin.myip.org/

Re: desktop and multimedia as an afterthought?

[Mark Hahn]

> I could be wrong, but it seems to me that at least a part of the kernel
> development team has the desktop and multimedia issues very low on the
> priority list.

no, that implies that desktop/media is somehow opposed to other development.
that's the real issue: is there some reason to believe that this opposition
exists, that tuning for desktop/media is tuning away from other serverish
uses?  people like Linus want to push the agenda that the same kernel can 
do both, until there's some definitive proof otherwise.

> The CK patches floated around as separate patches for a long time, even
> though they brought significant improvements to the kernel w.r.t.
> desktop and media.

how do you show this?  measured how, under what load, with what benefits?

> And rightly so. If i reboot my computer into Windows and perform the
> same multimedia tasks, there are fewer chances of it skipping frames or

this normally shows only that windows drivers are better.

Re: desktop and multimedia as an afterthought?

[Albert Cahalan]

Florin Andrei writes:

> I could be wrong, but it seems to me that at least a part of the kernel
> development team has the desktop and multimedia issues very low on the 
> priority list.

You're right, but you imply that this is bad. It would only be
bad if _all_ of the kernel development team had the desktop and
multimedia issues very low on the priority list. It would be
far worse if everyone worked on multimedia issues.

> The CK patches floated around as separate patches for a long time, even
> though they brought significant improvements to the kernel w.r.t.
> desktop and media.
> Now the stock 2.6 kernel has a pretty bad problem with the latency.
> Also, there seems to be a reluctance in accepting the autoregulated
> swappiness patch, even though it markedly improves the overall behaviour
> of the system as a desktop.
  
Reluctance is good. Reluctance keeps out bloat. Reluctance supplies
time to investigate alternate ideas that might be better.

> The CK patches have devouted "cult followers".
> Projects such as PlanetCCRMA, which attempt to build a multimedia-ready
> Linux distribution, are running kernels patched with the CK patches by
> default.

This is expected. It's how proposed features get widespread testing.

> On the Linux multimedia mailing lists and forums, one can often hear
> advices in the vein of "use such-and-such patch if you want your system
> to do any serious multimedia work, the vanilla kernel sucks."
...
> I mean, go read the forums. No one in the multimedia community uses the
> vanilla kernel. They could be all wrong, sure, but there's lots of them.
     
It's too bad that the multimedia community didn't participate
much during the 2.5.xx development leading up to 2.6.0. If they
had done so, the situation might be different today. Fortunately,
fixing up the multimedia problems isn't too risky to do during
the stable 2.6.xx series.
  
> It seems like there's a split-brain situation within the Linux
> community, with the core kernel developers sitting on one side of the
> rift, and the multimedia people on the other side.
  
Nah. People do forget about problems if nobody is reporting them.
Since the problem reports came in late, you get to be last.
 
> Now, i remember someone saying, a while ago, that the server stuff is  
> pretty much done, and the interesting things are going to happen on the
> desktop. That sounds plausible, but if the kernel has poor support for 
> typical desktop scenarios, how are those big desktop improvements going
> to take place?

Much of that is about hot-plug, laptops going to sleep,
busy disks being ripped from the machine, support for
recent video card features, and so on. Multimedia is only
a small part of the big picture, even for desktop usage.

Re: desktop and multimedia as an afterthought?

[Paul Davis]

>> I could be wrong, but it seems to me that at least a part of the kernel
>> development team has the desktop and multimedia issues very low on the
>> priority list.
>
>no, that implies that desktop/media is somehow opposed to other development.
>that's the real issue: is there some reason to believe that this opposition
>exists, that tuning for desktop/media is tuning away from other serverish
>uses?  people like Linus want to push the agenda that the same kernel can 
>do both, until there's some definitive proof otherwise.

there's no definitive proof, but one can say up front that the demands
have long been recognized as fairly different, and that tends to leads
to different design strategies which in turn can compromise the
response to the demands.

the key characteristic of multimedia work, audio and video in
particular (realtime 3d postscript-driven sculpture devices are still
rare at this point) is that it needs to satisfy pseudo-hard realtime
deadlines. i say pseudo only because nobody dies if the deadlines are
missed. you miss the deadline for an audio device just once, and the
entire group of people listening to the output can hear it loud and
clear. 

the serverish space never has this requirement. the main requirement
there is stability and throughput, where throughput generally
considers of delivery varying-sized chunks of data to a network
interface as a result of various types of network-delivered
requests. a server has only one catastrophic condition: failure to
serve requests. degradation in performance is expected as load rises,
and although tweaks may change the curve, its considered OK for a
machine to perform variably as a server depending on the load. 

the focus on the second of these two quite different sets of
performance requirements tends to push the kernel in one direction,
with occasional major concessions to the other. more abstractly, the
tension tends to be characterized as between throughput and response
("latency"), but this is excessively simplistic. audio work (and to
some extent video as well) requires deadline satisfaction, serverish
work requires predictable degradation in response to workload. these
are not inherently opposed to each other, but measuring the behaviour
of one rarely touches the behaviour of the other.

>> The CK patches floated around as separate patches for a long time, even
>> though they brought significant improvements to the kernel w.r.t.
>> desktop and media.
>
>how do you show this?  measured how, under what load, with what
>> benefits?

the classic example is benno sennoner's latencytest, which has been
cited in all the reports here on latency issues, along with andrew
mortons schedlat utilities like realfeel.

at the RMLL/LSM in bordeaux last week, takashi iwai of the alsa
project demonstrated very convincingly that the isochronous scheduler
(for example) is of major benefit to non-SCHED_{FIFO,RR} media
applications. this is just for stuff like people running xmms and its
cousins while doing other work.

there really is no room for disagreement here: every linux developer
and user who does serious audio work runs a patched kernel and sees
clearly measurable (and in some cases, absolutely required) benefits.

>> And rightly so. If i reboot my computer into Windows and perform the
>> same multimedia tasks, there are fewer chances of it skipping frames or
>
>this normally shows only that windows drivers are better.

this is an absurd claim. i suppose that beos' outstanding performance
with media, or the excellent performance of osx simply shows that
their drivers are better? when JACK was ported to OSX, it somewhat
embarassingly performed better on OSX than it does on Linux. thats not
because of device drivers, its because the Mach part of Darwin has
superb facilities to support the kind of stuff that JACK needs.

--p

Re: desktop and multimedia as an afterthought?

[Florin Andrei]

On Mon, 2004-07-12 at 12:12, Mark Hahn wrote:

> > The CK patches floated around as separate patches for a long time, even
> > though they brought significant improvements to the kernel w.r.t.
> > desktop and media.
> 
> how do you show this?  measured how, under what load, with what benefits?

Run the vanilla kernel. Run Ardour to do the digital audio work, plus a
sequencer (Rosegarden, Muse, Seq24) to drive the MIDI gear. Do a
multitrack record.
If you fiddle too much with Mozilla and/or Evolution, GIMP, gThumb,
OpenOffice while doing the record session, Ardour will skip parts of the
tracks.

Repeat everything, using a CK kernel instead (or the kernel provided by
PlanetCCRMA). No amounts of Mozilla/Evolution/etc normal usage will make
Ardour skip anything.

Want numbers?
0 skips if running the CK/PlanetCCRMA kernel.
>0 skips if running vanilla kernel.

Anything more than 0 is not acceptable for this type of usage.

System is:
AthlonXP/1800, 512MB RAM, NForce1 mobo, Audigy2, ALSA, Fedora 2 and
various flavours of Linux-2.6 (similar results with Red Hat 9 and
Linux-2.4)
Similar results reported by almost anyone doing sound studio type of
work with Linux, on a large variety of hardware.

> > And rightly so. If i reboot my computer into Windows and perform the
> > same multimedia tasks, there are fewer chances of it skipping frames or
> 
> this normally shows only that windows drivers are better.

Which drivers? Sound card?

The same issues manifest when doing an all-internal record.

E.g.: run a softsynth (a software that generates sound, such as
Specimen, AlsaModularSynth or ZynAddSubFX) and route the output to JACK.
Run some LADSPA effects (reverb, chorus, flanger) on top of the JACK
pipe for good measure. At the other end of the JACK pipe run Ardour,
Audacity or any other DAW (digital audio workstation) software. Add a
few more softsynth instances, and drive them via a sequencer.
All sound is routed inside the computer. It never touches the sound
card. Any driver that's involved is the IDE or SCSI driver (because the
DAW dumps the audio tracks onto the hard-drive). Even if the driver was
defective, it wouldn't probably matter, because of the multiple levels
of caching that are involved.

Yet the exact same problems happen (skipping parts of the tracks if
system is only marginally used by other apps), with the exact same
solution (apply the CK patches to the kernel).

-- 
Florin Andrei

http://florin.myip.org/

Re: desktop and multimedia as an afterthought?

[Paul Davis]

>It's too bad that the multimedia community didn't participate
>much during the 2.5.xx development leading up to 2.6.0. If they
>had done so, the situation might be different today. Fortunately,
>fixing up the multimedia problems isn't too risky to do during
>the stable 2.6.xx series.

I regret that this description is persisting here. "We" (the audio
developer community) did not participate because it was made clear
that our needs were not going to be considered. We were told that the
preemption patch was sufficient to provide "low latency", and that
rescheduling points dotted all over the place was bad engineering
(probably true). With this as the pre-rendered verdict, there's not a
lot of point in dedicating time to tracking a situation that clearly
is not going to work.

The kernel is not going to provide adequate latency for multimedia
needs without either (1) latency issues being front and center in
every kernel developer's mind, which seems unlikely and/or (2)
conditional rescheduling points added to the kernel, which appears to
require non-mainstreamed patches.

--p

  

Re: desktop and multimedia as an afterthought?

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1837 bytes --]

Paul Davis writes:

>>It's too bad that the multimedia community didn't participate
>>much during the 2.5.xx development leading up to 2.6.0. If they
>>had done so, the situation might be different today. Fortunately,
>>fixing up the multimedia problems isn't too risky to do during
>>the stable 2.6.xx series.
> 
> I regret that this description is persisting here. "We" (the audio
> developer community) did not participate because it was made clear
> that our needs were not going to be considered. We were told that the
> preemption patch was sufficient to provide "low latency", and that
> rescheduling points dotted all over the place was bad engineering
> (probably true). With this as the pre-rendered verdict, there's not a
> lot of point in dedicating time to tracking a situation that clearly
> is not going to work.
> 
> The kernel is not going to provide adequate latency for multimedia
> needs without either (1) latency issues being front and center in
> every kernel developer's mind, which seems unlikely and/or (2)
> conditional rescheduling points added to the kernel, which appears to
> require non-mainstreamed patches.

Please dont start a low level flamewar over this. Latency is firmly on the 
agenda and under consideration for the mainline kernel now. 

There is nothing wrong with using a dedicated alternative patchset for 
specific tasks, as long as any lessons learnt from it are also taken into 
consideration for mainline. Mainline kernels must have (high gain)/(low 
risk) ratio changes only. 

Rather than just saying that the desktop and multimedia is not considered it 
would be more helpful to say what helps where and why in the public forum of 
the main kernel mailing list. Off list discussion can go completely 
unnoticed (even I wasn't aware of my patchset being used and quoted!)

Cheers,
Con


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: desktop and multimedia as an afterthought?

[Andrew Morton]

Paul Davis <paul@linuxaudiosystems.com> wrote:
>
> >It's too bad that the multimedia community didn't participate
> >much during the 2.5.xx development leading up to 2.6.0. If they
> >had done so, the situation might be different today. Fortunately,
> >fixing up the multimedia problems isn't too risky to do during
> >the stable 2.6.xx series.
> 
> I regret that this description is persisting here. "We" (the audio
> developer community) did not participate because it was made clear
> that our needs were not going to be considered. We were told that the
> preemption patch was sufficient to provide "low latency", and that
> rescheduling points dotted all over the place was bad engineering
> (probably true). With this as the pre-rendered verdict, there's not a
> lot of point in dedicating time to tracking a situation that clearly
> is not going to work.

No, this is wrong.  2.6+preempt can satisfy your latency requirements
without any scheduling points.  All it requires is that the long-held locks
be addressed.  I've already done a metric ton of work in that area (notably
removal of the buffer_head LRUs and rewriting the truncate code) but more
apparently remains to be done.  We know that reiserfs has problems.

But what can I do?  I set up a preempt-on-ext3 test box, thrash the crap
out of it and see 300 usecs worst-case latency.  So I am left empty-handed,
wondering what on earth is happening out there.

I am deeply skeptical about claims that autoregulated swappiness can make
any difference.

I am deeply skeptical about claims that CPU scheduler changes make any
difference.  A scheduler change shouldn't improve responsiveness of
!SCHED_OTHER tasks at all, so perhaps there are application priority
inversion problems, or applications aren't setting SCHED_FIFO/RR correctly.
I do not know.

I am also fairly skeptical about claims that voluntary-preempt helps,
because it only pops a couple of locks, and I doubt that testers are
hitting the code paths which those changes address anyway.

So Something Is Up, and I don't know what it is.

Please double-check that there are no priority inversion problems and that
the application is correctly setting the scheduling policy and that it is
mlocking everything appropriately.

And please ensure that people are setting xrun_debug, and are sending
reports.

> The kernel is not going to provide adequate latency for multimedia
> needs without either (1) latency issues being front and center in
> every kernel developer's mind, which seems unlikely and/or (2)
> conditional rescheduling points added to the kernel, which appears to
> require non-mainstreamed patches.
> 

Nope, the conditional rescheduling points provide zero benefit on a
preemptible kernel.

Something weird is happening, I don't know what it is, I cannot reproduce
it and I need help understanding what it is, OK?  The sooner we can do
that, the sooner it gets fixed up.

Re: desktop and multimedia as an afterthought?

[Paul Davis]

>Please dont start a low level flamewar over this. Latency is firmly on the 
>agenda and under consideration for the mainline kernel now. 

Yes, and I apologize for that intemperate post from me. Quite
unnecessary and utterly unhelpful. It just irks me to be told "you're
late to the party" when we were told "the party can happen without
you". Nothing more than that. Its great to see that people *do* care
about this stuff. Sorry for the noise.

--p

Re: desktop and multimedia as an afterthought?

[Thomas Charbonnel]

> And please ensure that people are setting xrun_debug, and are sending
> reports.
> 

Hi,

On my system xruns seem related to the keyboard. I get xruns on ~8.079
seconds boundaries when the keyboard is in use, regardless of the load.
My usual test is running jack with 2 periods of 64 samples and no
client, and keep a key pressed. Those latencytest graphs give an idea of
the problem : http://www.undata.org/~thomas/latencytest/index.html

Here are the xrun_debug reports :

For the intel8x0 :
XRUN: pcmC1D0p
Stack pointer is garbage, not printing trace
Unexpected hw_pointer value [1] (stream = 1, delta: -16, max jitter =
64): wrong interrupt acknowledge?
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420
 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107824>] do_IRQ+0x194/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107754>] do_IRQ+0xc4/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0108126>] do_softirq+0x46/0x60
 [<c01077d9>] do_IRQ+0x149/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053c809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f
 =======================
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420
 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107824>] do_IRQ+0x194/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107754>] do_IRQ+0xc4/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0108126>] do_softirq+0x46/0x60
 [<c01077d9>] do_IRQ+0x149/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053c809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f
 =======================
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420
 [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107824>] do_IRQ+0x194/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107754>] do_IRQ+0xc4/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0108126>] do_softirq+0x46/0x60
 [<c01077d9>] do_IRQ+0x149/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053c809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f


For the hdsp:
XRUN: pcmC2D0c
Stack pointer is garbage, not printing trace
XRUN: pcmC2D0c
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332521>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c0368e94>] snd_hdsp_interrupt+0x174/0x180
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107824>] do_IRQ+0x194/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107754>] do_IRQ+0xc4/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0108126>] do_softirq+0x46/0x60
 [<c01077d9>] do_IRQ+0x149/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053c809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f
 =======================
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332521>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c0368e94>] snd_hdsp_interrupt+0x174/0x180
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107824>] do_IRQ+0x194/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107754>] do_IRQ+0xc4/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0108126>] do_softirq+0x46/0x60
 [<c01077d9>] do_IRQ+0x149/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053c809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f
 =======================
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332521>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c0368e94>] snd_hdsp_interrupt+0x174/0x180
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107824>] do_IRQ+0x194/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107754>] do_IRQ+0xc4/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0108126>] do_softirq+0x46/0x60
 [<c01077d9>] do_IRQ+0x149/0x1b0
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053c809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f

Thomas

Re: desktop and multimedia as an afterthought?

[Florin Andrei]

On Mon, 2004-07-12 at 17:24, Andrew Morton wrote:

> Something weird is happening, I don't know what it is, I cannot reproduce
> it and I need help understanding what it is, OK?  The sooner we can do
> that, the sooner it gets fixed up.

Sounds fair to me.

I have no time whatsoever for such things, but if i do get time, i'll do
some tests and post results. I'll lurk in the background meanwhile, to
figure out more precisely just what kind of tests you guys want, what
kind of debugging to turn on, etc.

Thank you,

-- 
Florin Andrei

http://florin.myip.org/

Re: desktop and multimedia as an afterthought?

[Florin Andrei]

On Mon, 2004-07-12 at 17:18, Con Kolivas wrote:

> (even I wasn't aware of my patchset being used and quoted!)

Well, no one's using your patch except the entire Linux sound/music
community :-) the people around the linux-audio-user mailing list, the
users of the PlanetCCRMA distro, etc.
I suspect also that the video guys are also quite familiar with your
work, at least those who do a lot of analog video capture (and maybe
digital as well).

-- 
Florin Andrei

http://florin.myip.org/

Re: desktop and multimedia as an afterthought?

[Takashi Iwai]

At Mon, 12 Jul 2004 17:24:58 -0700,
Andrew Morton wrote:
> 
> I am deeply skeptical about claims that CPU scheduler changes make any
> difference.  A scheduler change shouldn't improve responsiveness of
> !SCHED_OTHER tasks at all, so perhaps there are application priority
> inversion problems, or applications aren't setting SCHED_FIFO/RR correctly.
> I do not know.

Regarding the JACK problem, it seems that the incompatibility with
NPTL (SCHED_INHERIT is default) did wrong.


Taking a look through the thread, I feel that very different topics
are argued in the single "desktop" problem, namely, the interactivity
and the latency.  The latter, the problem of real-time audio
(e.g. JACK), must be irrelevant with the CPU scheduler.  It can 
be fixed by detecting the too long critical sections, but the fix
won't always improve the interactivity.

OTOH, the interactivity can be, and should be improved somehow with
tuning of CPU scheduler.  However, even about this word, we discuss
totally different meanings.  For example, the GUI response and the
fluent audio/video playback.  The improvement of the former doesn't
imply the improvement of the latter (often contradictorily)...


--
Takashi Iwai <tiwai@suse.de>		ALSA Developer - www.alsa-project.org

Re: desktop and multimedia as an afterthought?

[Andrew Morton]

Thomas Charbonnel <thomas@undata.org> wrote:
>
> On my system xruns seem related to the keyboard. I get xruns on ~8.079
>  seconds boundaries when the keyboard is in use, regardless of the load.
>  My usual test is running jack with 2 periods of 64 samples and no
>  client, and keep a key pressed. Those latencytest graphs give an idea of
>  the problem : http://www.undata.org/~thomas/latencytest/index.html
> 
>  Here are the xrun_debug reports :

OK, thanks.

Stack tracing seems a bit broken with 4k stacks.  Can you disable
CONFIG_4KSTACKS for future testing?

>  For the intel8x0 :
>  XRUN: pcmC1D0p
>  Stack pointer is garbage, not printing trace
>  Unexpected hw_pointer value [1] (stream = 1, delta: -16, max jitter =
>  64): wrong interrupt acknowledge?
>   [<c0105f3e>] dump_stack+0x1e/0x30
>   [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420
>   [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260
>   [<c010739b>] handle_IRQ_event+0x3b/0x70
>   [<c0107824>] do_IRQ+0x194/0x1b0
>   [<c0105ac4>] common_interrupt+0x18/0x20
>   [<c0107754>] do_IRQ+0xc4/0x1b0
>   [<c0105ac4>] common_interrupt+0x18/0x20
>   [<c0108126>] do_softirq+0x46/0x60
>   [<c01077d9>] do_IRQ+0x149/0x1b0
>   [<c0105ac4>] common_interrupt+0x18/0x20
>   [<c01030f4>] cpu_idle+0x34/0x40
>   [<c053c809>] start_kernel+0x169/0x190
>   [<c010019f>] 0xc010019f
>   =======================
>   [<c0105f3e>] dump_stack+0x1e/0x30
>   [<c033240d>] snd_pcm_period_elapsed+0x1cd/0x420
>   [<c03578cc>] snd_intel8x0_interrupt+0x1fc/0x260
>   [<c010739b>] handle_IRQ_event+0x3b/0x70
>   [<c0107824>] do_IRQ+0x194/0x1b0
>   [<c0105ac4>] common_interrupt+0x18/0x20
>   [<c0107754>] do_IRQ+0xc4/0x1b0
>   [<c0105ac4>] common_interrupt+0x18/0x20
>   [<c0108126>] do_softirq+0x46/0x60
>   [<c01077d9>] do_IRQ+0x149/0x1b0
>   [<c0105ac4>] common_interrupt+0x18/0x20
>   [<c01030f4>] cpu_idle+0x34/0x40
>   [<c053c809>] start_kernel+0x169/0x190

Re: desktop and multimedia as an afterthought?

[Thomas Charbonnel]

Andrew Morton wrote :

> Stack tracing seems a bit broken with 4k stacks.  Can you disable
> CONFIG_4KSTACKS for future testing?
> 

Now I get :

for the intel8x0 :

XRUN: pcmC1D0p
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0146728>] do_anonymous_page+0x118/0x190
 [<c01467ff>] do_no_page+0x5f/0x300
 [<c0146c92>] handle_mm_fault+0xe2/0x190
 [<c01456a0>] get_user_pages+0x150/0x380
 [<c0146e07>] make_pages_present+0x87/0xb0
 [<c01472a9>] mlock_fixup+0xa9/0xc0
 [<c01475ef>] do_mlockall+0x7f/0xa0
 [<c01476cb>] sys_mlockall+0xbb/0xc0
 [<c0105157>] syscall_call+0x7/0xb
Unexpected hw_pointer value [1] (stream = 1, delta: -50, max jitter =
64): wrong interrupt acknowledge?
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c033244d>] snd_pcm_period_elapsed+0x1cd/0x420
 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01201ab>] do_softirq+0x2b/0x30
 [<c0107798>] do_IRQ+0x108/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
Unexpected hw_pointer value [1] (stream = 0, delta: -4, max jitter =
64): wrong interrupt acknowledge?
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c033244d>] snd_pcm_period_elapsed+0x1cd/0x420
 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01201ab>] do_softirq+0x2b/0x30
 [<c0107798>] do_IRQ+0x108/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
XRUN: pcmC1D0p
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
Unexpected hw_pointer value [1] (stream = 1, delta: -22, max jitter =
64): wrong interrupt acknowledge?
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c033244d>] snd_pcm_period_elapsed+0x1cd/0x420
 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01201ab>] do_softirq+0x2b/0x30
 [<c0107798>] do_IRQ+0x108/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053e809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f
XRUN: pcmC1D0p
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
XRUN: pcmC1D0p
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c035790c>] snd_intel8x0_interrupt+0x1fc/0x260
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20

for the hdsp:

XRUN: pcmC2D0c
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01467ff>] do_no_page+0x5f/0x300
 [<c0146c92>] handle_mm_fault+0xe2/0x190
 [<c01456a0>] get_user_pages+0x150/0x380
 [<c0146e07>] make_pages_present+0x87/0xb0
 [<c01487f7>] do_mmap_pgoff+0x467/0x6c0
 [<c010ba10>] sys_mmap2+0xa0/0xe0
 [<c0105157>] syscall_call+0x7/0xb
XRUN: pcmC2D0c
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0145633>] get_user_pages+0xe3/0x380
 [<c0146e07>] make_pages_present+0x87/0xb0
 [<c01487f7>] do_mmap_pgoff+0x467/0x6c0
 [<c010ba10>] sys_mmap2+0xa0/0xe0
 [<c0105157>] syscall_call+0x7/0xb
XRUN: pcmC2D0c
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01201ab>] do_softirq+0x2b/0x30
 [<c0107798>] do_IRQ+0x108/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c01030f4>] cpu_idle+0x34/0x40
 [<c053e809>] start_kernel+0x169/0x190
 [<c010019f>] 0xc010019f
XRUN: pcmC2D0c
 [<c0105f3e>] dump_stack+0x1e/0x30
 [<c0332561>] snd_pcm_period_elapsed+0x2e1/0x420
 [<c0368ed4>] snd_hdsp_interrupt+0x174/0x180
 [<c010739b>] handle_IRQ_event+0x3b/0x70
 [<c0107726>] do_IRQ+0x96/0x150
 [<c0105ac4>] common_interrupt+0x18/0x20
 [<c0144fc5>] zap_pte_range+0x145/0x240
 [<c0145125>] zap_pmd_range+0x65/0x90
 [<c01451a5>] unmap_page_range+0x55/0x80
 [<c01452e4>] unmap_vmas+0x114/0x1e0
 [<c0149845>] exit_mmap+0x85/0x170
 [<c0119ebf>] mmput+0x6f/0xb0
 [<c011e6c4>] do_exit+0x114/0x470
 [<c011eaca>] do_group_exit+0x3a/0xb0
 [<c012700e>] get_signal_to_deliver+0x25e/0x370
 [<c0104eea>] do_signal+0x6a/0x100
 [<c0104fb9>] do_notify_resume+0x39/0x3c
 [<c01051a2>] work_notifysig+0x13/0x15

Thomas

Re: desktop and multimedia as an afterthought?

[Kasper Sandberg]

On Mon, 2004-07-12 at 17:24 -0700, Andrew Morton wrote:
> Paul Davis <paul@linuxaudiosystems.com> wrote:
> >
> > >It's too bad that the multimedia community didn't participate
> > >much during the 2.5.xx development leading up to 2.6.0. If they
> > >had done so, the situation might be different today. Fortunately,
> > >fixing up the multimedia problems isn't too risky to do during
> > >the stable 2.6.xx series.
> > 
> > I regret that this description is persisting here. "We" (the audio
> > developer community) did not participate because it was made clear
> > that our needs were not going to be considered. We were told that the
> > preemption patch was sufficient to provide "low latency", and that
> > rescheduling points dotted all over the place was bad engineering
> > (probably true). With this as the pre-rendered verdict, there's not a
> > lot of point in dedicating time to tracking a situation that clearly
> > is not going to work.
> 
> No, this is wrong.  2.6+preempt can satisfy your latency requirements
> without any scheduling points.  All it requires is that the long-held locks
> be addressed.  I've already done a metric ton of work in that area (notably
> removal of the buffer_head LRUs and rewriting the truncate code) but more
> apparently remains to be done.  We know that reiserfs has problems.
> 
> But what can I do?  I set up a preempt-on-ext3 test box, thrash the crap
> out of it and see 300 usecs worst-case latency.  So I am left empty-handed,
> wondering what on earth is happening out there.
> 
> I am deeply skeptical about claims that autoregulated swappiness can make
> any difference.
> 
> I am deeply skeptical about claims that CPU scheduler changes make any
> difference.  A scheduler change shouldn't improve responsiveness of
> !SCHED_OTHER tasks at all, so perhaps there are application priority
> inversion problems, or applications aren't setting SCHED_FIFO/RR correctly.
> I do not know.
sorry to interrupt, and i dont know if i get this right, but i have been
using nick piggins patch for quite some time, and it really does
magic :)
> 
> I am also fairly skeptical about claims that voluntary-preempt helps,
> because it only pops a couple of locks, and I doubt that testers are
> hitting the code paths which those changes address anyway.
> 
> So Something Is Up, and I don't know what it is.
> 
> Please double-check that there are no priority inversion problems and that
> the application is correctly setting the scheduling policy and that it is
> mlocking everything appropriately.
> 
> And please ensure that people are setting xrun_debug, and are sending
> reports.
> 
> > The kernel is not going to provide adequate latency for multimedia
> > needs without either (1) latency issues being front and center in
> > every kernel developer's mind, which seems unlikely and/or (2)
> > conditional rescheduling points added to the kernel, which appears to
> > require non-mainstreamed patches.
> > 
> 
> Nope, the conditional rescheduling points provide zero benefit on a
> preemptible kernel.
> 
> Something weird is happening, I don't know what it is, I cannot reproduce
> it and I need help understanding what it is, OK?  The sooner we can do
> that, the sooner it gets fixed up.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Martijn Sipkema]

[...]
> Please double-check that there are no priority inversion problems and that
> the application is correctly setting the scheduling policy and that it is
> mlocking everything appropriately.

I don't think it is currently possible to have cooperating threads with
different priorities without priority inversion when using a mutex to
serialize access to shared data; and using a mutex is in fact the only portable
way to do that...

Thus, the fact that Linux does not support protocols to prevent priority
inversion (please correct me if I am wrong) kind of suggests that supporting
realtime applications is not considered very important.

It is often heard in the Linux audio community that mutexes are not realtime
safe and a lock-free ringbuffer should be used instead. Using such a lock-free
ringbuffer requires non-standard atomic integer operations and does not
guarantee memory synchronization (and should probably not perform
significantly better than a decent mutex implementation) and is thus not
portable.

--ms

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Paul Davis]

>Thus, the fact that Linux does not support protocols to prevent priority
>inversion (please correct me if I am wrong) kind of suggests that supporting
>realtime applications is not considered very important.

we went through this (you and i in particular) right here on LAD a
year or so ago. while i might agree with you about the priority given
to RT-ish apps, my recollection of the end of that discussion is that
priority inheritance is neither necessary nor sufficient to allow
adequate RT performance. priority inversion generally can be factored
out through application redesign, and the protocols i've seen to
address it are not useful for RT purposes - they just help deadlock.

--p

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Bill Huey]

On Tue, Jul 13, 2004 at 01:09:28PM +0100, Martijn Sipkema wrote:
> [...]
> > Please double-check that there are no priority inversion problems and that
> > the application is correctly setting the scheduling policy and that it is
> > mlocking everything appropriately.
> 
> I don't think it is currently possible to have cooperating threads with
> different priorities without priority inversion when using a mutex to
> serialize access to shared data; and using a mutex is in fact the only portable
> way to do that...
> 
> Thus, the fact that Linux does not support protocols to prevent priority
> inversion (please correct me if I am wrong) kind of suggests that supporting
> realtime applications is not considered very important.

Any use of an explicit or implied blocking mutex across threads with differing
priorities can results in priority inversion problems. The real problem, however,
is contention. If you get rid of the contention in a certain critical section,
you then also get rid of latency in the system. They are one and the same problem.

> It is often heard in the Linux audio community that mutexes are not realtime
> safe and a lock-free ringbuffer should be used instead. Using such a lock-free
> ringbuffer requires non-standard atomic integer operations and does not
> guarantee memory synchronization (and should probably not perform
> significantly better than a decent mutex implementation) and is thus not
> portable.

It's to decouple the system from various time related problems with jitter.
It's critical to use this since the nature of Linus is so temporally coarse
that these techniques must be used to "smooth" over latency problems in the
Linux kernel.

I personally would love to see these audio applications run on a first-class
basis under Linux. Unfortunately, that won't happen until it gets near real
time support prevasively through the kernel just like in SGI's IRIX. Multimedia
applications really need to be under a hard real time system with special
scheduler support so that CPU resources, IO channels can be throttled.

The techniques Linux media folks are using now are basically a coarse hack
to get things basically working. This won't change unless some fundamental
concurrency issues (moving to a preemptive kernel with interrupt threads, etc..)
change in Linux. Scattering preemption points manually over 2.6 is starting to
look unmanable from all of the stack traces I've been reading in these latency
related threads.

That's all. :)

bill

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Lee Revell]

On Tue, 2004-07-13 at 15:12, Bill Huey wrote:
> On Tue, Jul 13, 2004 at 01:09:28PM +0100, Martijn Sipkema wrote:
> > [...]
> > > Please double-check that there are no priority inversion problems and that
> > > the application is correctly setting the scheduling policy and that it is
> > > mlocking everything appropriately.
> > 
> > I don't think it is currently possible to have cooperating threads with
> > different priorities without priority inversion when using a mutex to
> > serialize access to shared data; and using a mutex is in fact the only portable
> > way to do that...
> > 
> > Thus, the fact that Linux does not support protocols to prevent priority
> > inversion (please correct me if I am wrong) kind of suggests that supporting
> > realtime applications is not considered very important.
> 
> Any use of an explicit or implied blocking mutex across threads with differing
> priorities can results in priority inversion problems. The real problem, however,
> is contention. If you get rid of the contention in a certain critical section,
> you then also get rid of latency in the system. They are one and the same problem.
> 
> > It is often heard in the Linux audio community that mutexes are not realtime
> > safe and a lock-free ringbuffer should be used instead. Using such a lock-free
> > ringbuffer requires non-standard atomic integer operations and does not
> > guarantee memory synchronization (and should probably not perform
> > significantly better than a decent mutex implementation) and is thus not
> > portable.
> 
> It's to decouple the system from various time related problems with jitter.
> It's critical to use this since the nature of Linus is so temporally coarse
> that these techniques must be used to "smooth" over latency problems in the
> Linux kernel.
> 
> I personally would love to see these audio applications run on a first-class
> basis under Linux. Unfortunately, that won't happen until it gets near real
> time support prevasively through the kernel just like in SGI's IRIX. Multimedia
> applications really need to be under a hard real time system with special
> scheduler support so that CPU resources, IO channels can be throttled.
> 

I don't think invoking IRIX is going to get us a lot of sympathy on
LKML.  It is widely reviled.  BEOS is probably a better example.

Just my $0.02.

Lee

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Bill Huey]

On Tue, Jul 13, 2004 at 11:44:59PM +0100, Martijn Sipkema wrote:
[...]
> The worst case latency is the one that counts and that is the contended case. If
> you could guarantee no contention then the worst case latency would be the
> very fast uncontended case, but I doubt there are many (any?) examples of this in
> practice. There are valid uses of mutexes with priority inheritance/ceiling protocol;
> the poeple making the POSIX standard aren't stupid...

There are cases where you have to use priority inheritance, but the schemes that are
typically used either have a kind of exhaustive analysis backing it or uses a simple
late detection scheme. In a general purpose OS, the latter is useful for various kind
of overload cases. But if your system is constantly using that specific case, then it's
a sign the contention in the kernel must *also* be a problem under SMP conditions. The
constant use of priority inheritance overloads the scheduler, puts pressure on the
cache and other negative things that hurt CPU local performance of the system.

The reason why I mention this is because of Linux's hand-crafted nature of dealing
with this. These are basically contention problems expressed in a different manner.
The traditional Linux method is the correct method of deal with this in a general
purpose OS. This also applies to application structure as well. The use of these
mechanisms need to be thought out before application.

> > > It is often heard in the Linux audio community that mutexes are not realtime
> > > safe and a lock-free ringbuffer should be used instead. Using such a lock-free
> > > ringbuffer requires non-standard atomic integer operations and does not
> > > guarantee memory synchronization (and should probably not perform
> > > significantly better than a decent mutex implementation) and is thus not
> > > portable.
> > 
> > It's to decouple the system from various time related problems with jitter.
> > It's critical to use this since the nature of Linus is so temporally coarse
> > that these techniques must be used to "smooth" over latency problems in the
> > Linux kernel.

> Either use mutexes or POSIX message queues... the latter also are not
> intended for realtime use under Linux (though they are meant for it in
> POSIX), since they don't allocate memory on creation.
 
The nature these kind of applications push into a very demanding space where
typical methodologies surrounding the use of threads goes out the window. Pushing
both the IO and CPU resources of a kernel is the common case and often you have to
roll your own APIs, synchronization mechanisms to deal with these problem. Simple
Posix API and traditional mutexes are a bit too narrow in scope to solve these
cross system concurrency problems. It's not trivial stuff at all and can span
from loosely to tightly coupled systems, yes, all for pro-audio/video.

Posix and friends in these cases simply aren't good enough to cut it.

> > I personally would love to see these audio applications run on a first-class
> > basis under Linux. Unfortunately, that won't happen until it gets near real
> > time support prevasively through the kernel just like in SGI's IRIX. Multimedia
> > applications really need to be under a hard real time system with special
> > scheduler support so that CPU resources, IO channels can be throttled.
> > 
> > The techniques Linux media folks are using now are basically a coarse hack
> > to get things basically working. This won't change unless some fundamental
> > concurrency issues (moving to a preemptive kernel with interrupt threads, etc..)
> > change in Linux. Scattering preemption points manually over 2.6 is starting to
> > look unmanable from all of the stack traces I've been reading in these latency
> > related threads.
> 
> Improving the mutex and mqueue implementations to better support realtime
> use would be a significant improvement I think, making Linux quite suitable
> for realtime audio use.

Yes and no. This really all really needs to be hard real time in the long run.
With things like Tivo and other embedded application putting high demand on kernel
resources this kind of stuff needs to go into the Linux kernel sooner rather than
later so that Linux can keep up with current industry tracks.

That's my two cents. :)

bill

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Fons Adriaensen]

On Tue, Jul 13, 2004 at 11:37:12PM +0100, Martijn Sipkema wrote:

> IMHO it is the lack of a mutex implementation with priority ceiling
> or inheritance and the stories about relying on either being a design
> problem that have caused the Linux audio community to not use
> mutexes and declare them non-RT safe while in fact they are
> required according to POSIX to synchronize memory between
> cooperating threads.

Does someone have an authoritive answer to the following question:

   Will using try_lock() on a mutex ever block the caller ?

AFAIK it will not. If this is true I don't see the point of a
lock-free ring buffer at all. You will need some way to wake up the
non-RT thread anyway, in case it went to sleep when finding and
empty ring buffer. This same synchronisation method can be used
to share the state of the buffer between two threads.

For example if you use a counting semaphore (built using a condition
variable and a mutex), the RT thread would increment the sema for
every N samples it adds to a circular buffer, and the consumer will
decrement it for every N samples it reads. Then the state of the sema
at all times reflects the number of samples in the buffer.

If ever the V operation in the RT thread fails (unlikely, but possible
since it has to use try_lock()), it will remember this and increment
by one more the next time.  

The point is that there is no need to use lock-free techniques to
maintain a shared sample count - it's already provided by the sync
mechanism which you need anayway. I've been using this method for
years in critical apps, and never seen it fail.

-- 
FA

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Martijn Sipkema]

From: "Paul Davis" <paul@linuxaudiosystems.com>
> >Thus, the fact that Linux does not support protocols to prevent priority
> >inversion (please correct me if I am wrong) kind of suggests that supporting
> >realtime applications is not considered very important.
> 
> we went through this (you and i in particular) right here on LAD a
> year or so ago. while i might agree with you about the priority given
> to RT-ish apps, my recollection of the end of that discussion is that
> priority inheritance is neither necessary nor sufficient to allow
> adequate RT performance.

I don't recall that that is what was concluded.

Priority inheritance or some other protocol for priority inversion _is_
needed for realtime applications that have threads with different priorities
accessing common data. One could increase the priority of the low priority
thread before taking the mutex and release it afterwards (as in priority
ceiling), but I doubt that's optimal.

> priority inversion generally can be factored
> out through application redesign, and the protocols i've seen to
> address it are not useful for RT purposes - they just help deadlock.

For cases where some form of message passing does not work you
will need shared data with a mutex to serialize access and you _will_
need to prevent priority inversion. Mutexes are part of the POSIX
realtime threading extensions; you can use a semaphore when
priority inversion is not an issue.

IMHO it is the lack of a mutex implementation with priority ceiling
or inheritance and the stories about relying on either being a design
problem that have caused the Linux audio community to not use
mutexes and declare them non-RT safe while in fact they are
required according to POSIX to synchronize memory between
cooperating threads.

--ms

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Martijn Sipkema]

From: "Bill Huey (hui)" <bhuey@lnxw.com>
> On Tue, Jul 13, 2004 at 01:09:28PM +0100, Martijn Sipkema wrote:
> > [...]
> > > Please double-check that there are no priority inversion problems and that
> > > the application is correctly setting the scheduling policy and that it is
> > > mlocking everything appropriately.
> > 
> > I don't think it is currently possible to have cooperating threads with
> > different priorities without priority inversion when using a mutex to
> > serialize access to shared data; and using a mutex is in fact the only portable
> > way to do that...
> > 
> > Thus, the fact that Linux does not support protocols to prevent priority
> > inversion (please correct me if I am wrong) kind of suggests that supporting
> > realtime applications is not considered very important.
> 
> Any use of an explicit or implied blocking mutex across threads with differing
> priorities can results in priority inversion problems. The real problem, however,
> is contention. If you get rid of the contention in a certain critical section,
> you then also get rid of latency in the system. They are one and the same problem.

The worst case latency is the one that counts and that is the contended case. If
you could guarantee no contention then the worst case latency would be the
very fast uncontended case, but I doubt there are many (any?) examples of this in
practice. There are valid uses of mutexes with priority inheritance/ceiling protocol;
the poeple making the POSIX standard aren't stupid...

> > It is often heard in the Linux audio community that mutexes are not realtime
> > safe and a lock-free ringbuffer should be used instead. Using such a lock-free
> > ringbuffer requires non-standard atomic integer operations and does not
> > guarantee memory synchronization (and should probably not perform
> > significantly better than a decent mutex implementation) and is thus not
> > portable.
> 
> It's to decouple the system from various time related problems with jitter.
> It's critical to use this since the nature of Linus is so temporally coarse
> that these techniques must be used to "smooth" over latency problems in the
> Linux kernel.

Either use mutexes or POSIX message queues... the latter also are not
intended for realtime use under Linux (though they are meant for it in
POSIX), since they don't allocate memory on creation.

> I personally would love to see these audio applications run on a first-class
> basis under Linux. Unfortunately, that won't happen until it gets near real
> time support prevasively through the kernel just like in SGI's IRIX. Multimedia
> applications really need to be under a hard real time system with special
> scheduler support so that CPU resources, IO channels can be throttled.
> 
> The techniques Linux media folks are using now are basically a coarse hack
> to get things basically working. This won't change unless some fundamental
> concurrency issues (moving to a preemptive kernel with interrupt threads, etc..)
> change in Linux. Scattering preemption points manually over 2.6 is starting to
> look unmanable from all of the stack traces I've been reading in these latency
> related threads.

Improving the mutex and mqueue implementations to better support realtime
use would be a significant improvement I think, making Linux quite suitable
for realtime audio use.

--ms

Re: [linux-audio-dev] Re: desktop and multimedia as an afterthought?

[Martijn Sipkema]

From: "Bill Huey (hui)" <bhuey@lnxw.com>
> On Tue, Jul 13, 2004 at 11:44:59PM +0100, Martijn Sipkema wrote:
> [...]
> > The worst case latency is the one that counts and that is the contended case. If
> > you could guarantee no contention then the worst case latency would be the
> > very fast uncontended case, but I doubt there are many (any?) examples of this in
> > practice. There are valid uses of mutexes with priority inheritance/ceiling protocol;
> > the poeple making the POSIX standard aren't stupid...
> 
> There are cases where you have to use priority inheritance, but the schemes that are
> typically used either have a kind of exhaustive analysis backing it or uses a simple
> late detection scheme. In a general purpose OS, the latter is useful for various kind
> of overload cases. But if your system is constantly using that specific case, then it's
> a sign the contention in the kernel must *also* be a problem under SMP conditions. The
> constant use of priority inheritance overloads the scheduler, puts pressure on the
> cache and other negative things that hurt CPU local performance of the system.
> 
> The reason why I mention this is because of Linux's hand-crafted nature of dealing
> with this. These are basically contention problems expressed in a different manner.
> The traditional Linux method is the correct method of deal with this in a general
> purpose OS. This also applies to application structure as well. The use of these
> mechanisms need to be thought out before application.

To be honest, I don't understand a word of what you are saying here. Could you
give an example of a ``contention problem'' and how it should be solved?

> > > > It is often heard in the Linux audio community that mutexes are not realtime
> > > > safe and a lock-free ringbuffer should be used instead. Using such a lock-free
> > > > ringbuffer requires non-standard atomic integer operations and does not
> > > > guarantee memory synchronization (and should probably not perform
> > > > significantly better than a decent mutex implementation) and is thus not
> > > > portable.
> > > 
> > > It's to decouple the system from various time related problems with jitter.
> > > It's critical to use this since the nature of Linus is so temporally coarse
> > > that these techniques must be used to "smooth" over latency problems in the
> > > Linux kernel.
> 
> > Either use mutexes or POSIX message queues... the latter also are not
> > intended for realtime use under Linux (though they are meant for it in
> > POSIX), since they don't allocate memory on creation.
>  
> The nature these kind of applications push into a very demanding space where
> typical methodologies surrounding the use of threads goes out the window. Pushing
> both the IO and CPU resources of a kernel is the common case and often you have to
> roll your own APIs, synchronization mechanisms to deal with these problem. Simple
> Posix API and traditional mutexes are a bit too narrow in scope to solve these
> cross system concurrency problems. It's not trivial stuff at all and can span
> from loosely to tightly coupled systems, yes, all for pro-audio/video.
> 
> Posix and friends in these cases simply aren't good enough to cut it.

I find this a little abstract. Sure, there might be areas where POSIX doesn't supply
all the needed tools, e.g. one might want some scheduling policy especially for
audio, but to say that POSIX isn't good enough without providing much
explanation...

--ms