ALSA sample rate conversion and general performance improvements.

ALSA sample rate conversion and general performance improvements.

[James Courtier-Dutton]

Hi,

I have had some conversations with various people regarding some general
problems some audio developers have with the current ALSA sound model.
One of the biggest problems with the current ALSA model is the sample
rate converters. When discussing them here some time ago, the problem
centered around the need for a timer interrupt being needed for the
application period boundaries, where the hardware might be running at
48000Hz, and the application is trying to play a 44100Hz PCM stream.
This is essentially needed so that the poll() calls work in the
application correctly.

Here is an idea on how to fix it and also slightly improve performance.
Some general aims:
1) any ioport access to the sound card is slow and should be avoided
whenever possible.
2) where to get the 44100Hz timer from if the hardware is doing 48000Hz.

What about doing the following?:
We implement a general timer in the alsa core based on something like
gettimeofday(). I.e A high resolution timer.
Each time a period interrupt happens, the current hw pointer is read,
and recorded along with the current value of the timer.
This could be used a bit like NTP to correct the generally timer rate.
This gives us a general high resolution timer corrected to mirror the
PCM clock on the sound card.
Then, from this high resolution timer, we could derive any rate
interrupt we need for really excellent sample rate conversion, and sub
sample accurate DAC/ADC positioning.
One could then read the DAC/ADC positioning really quickly by just
reading the high resolution timer, instead of accessing IO Ports on the
sound card hardware.
This would also be a fairly minor change, as none of the current kernel
sound card hw drivers would need changing. We would just have to
slightly modify the period_elapsed callback to do a gettimeofday() call
as well as it's current hw_pointer read.

If we then made the correction factor for the high resolution timer
available to user space, user space would then not even need to read the
current hw position, as it could perfectly predict it itself using the
gettimeofday() and correction factor.

This would reduce the amount of calls to the read_hw_pointer function in
the low level sound card hardware driver, and therefore improve performance.

Any comments?

James


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: ALSA sample rate conversion and general performance improvements.

[Takashi Iwai]

At Wed, 29 Mar 2006 00:24:20 +0100,
James Courtier-Dutton wrote:
> 
> Hi,
> 
> I have had some conversations with various people regarding some general
> problems some audio developers have with the current ALSA sound model.
> One of the biggest problems with the current ALSA model is the sample
> rate converters. When discussing them here some time ago, the problem
> centered around the need for a timer interrupt being needed for the
> application period boundaries, where the hardware might be running at
> 48000Hz, and the application is trying to play a 44100Hz PCM stream.
> This is essentially needed so that the poll() calls work in the
> application correctly.
> 
> Here is an idea on how to fix it and also slightly improve performance.
> Some general aims:
> 1) any ioport access to the sound card is slow and should be avoided
> whenever possible.
> 2) where to get the 44100Hz timer from if the hardware is doing 48000Hz.
> 
> What about doing the following?:
> We implement a general timer in the alsa core based on something like
> gettimeofday(). I.e A high resolution timer.
> Each time a period interrupt happens, the current hw pointer is read,
> and recorded along with the current value of the timer.
> This could be used a bit like NTP to correct the generally timer rate.
> This gives us a general high resolution timer corrected to mirror the
> PCM clock on the sound card.
> Then, from this high resolution timer, we could derive any rate
> interrupt we need for really excellent sample rate conversion, and sub
> sample accurate DAC/ADC positioning.
> One could then read the DAC/ADC positioning really quickly by just
> reading the high resolution timer, instead of accessing IO Ports on the
> sound card hardware.
> This would also be a fairly minor change, as none of the current kernel
> sound card hw drivers would need changing. We would just have to
> slightly modify the period_elapsed callback to do a gettimeofday() call
> as well as it's current hw_pointer read.
> 
> If we then made the correction factor for the high resolution timer
> available to user space, user space would then not even need to read the
> current hw position, as it could perfectly predict it itself using the
> gettimeofday() and correction factor.
> 
> This would reduce the amount of calls to the read_hw_pointer function in
> the low level sound card hardware driver, and therefore improve performance.
> 
> Any comments?

I've had the very same idea about using the timer for the update
source, especially for sample rate conversion although my point isn't
the performance gain by avoiding IO read.

My idea is to make the system choose arbitrary irq sources, e.g. the
period irq, timer irq, etc.  The default is the period irq, or
whatever the driver provides.  But, the app or alsa-lib can switch to
a suitable one.

In the case of timer irq, it's not necessarily a constant period.  You
can tweak the period appropriately by adjusting the pace using a
software PLL.  This would require read of hwptr as well as the normal
update, but it's necessary to get the stream sync with the soundcard
time.  If you don't refer to the hw ptrs, you'll get a significant
drift in a long run.


Takashi


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: ALSA sample rate conversion and general performance improvements.

[James Courtier-Dutton]

Takashi Iwai wrote:
> At Wed, 29 Mar 2006 00:24:20 +0100,
> James Courtier-Dutton wrote:
>   
>> Hi,
>>
>> I have had some conversations with various people regarding some general
>> problems some audio developers have with the current ALSA sound model.
>> One of the biggest problems with the current ALSA model is the sample
>> rate converters. When discussing them here some time ago, the problem
>> centered around the need for a timer interrupt being needed for the
>> application period boundaries, where the hardware might be running at
>> 48000Hz, and the application is trying to play a 44100Hz PCM stream.
>> This is essentially needed so that the poll() calls work in the
>> application correctly.
>>
>> Here is an idea on how to fix it and also slightly improve performance.
>> Some general aims:
>> 1) any ioport access to the sound card is slow and should be avoided
>> whenever possible.
>> 2) where to get the 44100Hz timer from if the hardware is doing 48000Hz.
>>
>> What about doing the following?:
>> We implement a general timer in the alsa core based on something like
>> gettimeofday(). I.e A high resolution timer.
>> Each time a period interrupt happens, the current hw pointer is read,
>> and recorded along with the current value of the timer.
>> This could be used a bit like NTP to correct the generally timer rate.
>> This gives us a general high resolution timer corrected to mirror the
>> PCM clock on the sound card.
>> Then, from this high resolution timer, we could derive any rate
>> interrupt we need for really excellent sample rate conversion, and sub
>> sample accurate DAC/ADC positioning.
>> One could then read the DAC/ADC positioning really quickly by just
>> reading the high resolution timer, instead of accessing IO Ports on the
>> sound card hardware.
>> This would also be a fairly minor change, as none of the current kernel
>> sound card hw drivers would need changing. We would just have to
>> slightly modify the period_elapsed callback to do a gettimeofday() call
>> as well as it's current hw_pointer read.
>>
>> If we then made the correction factor for the high resolution timer
>> available to user space, user space would then not even need to read the
>> current hw position, as it could perfectly predict it itself using the
>> gettimeofday() and correction factor.
>>
>> This would reduce the amount of calls to the read_hw_pointer function in
>> the low level sound card hardware driver, and therefore improve performance.
>>
>> Any comments?
>>     
>
> I've had the very same idea about using the timer for the update
> source, especially for sample rate conversion although my point isn't
> the performance gain by avoiding IO read.
>
> My idea is to make the system choose arbitrary irq sources, e.g. the
> period irq, timer irq, etc.  The default is the period irq, or
> whatever the driver provides.  But, the app or alsa-lib can switch to
> a suitable one.
>
> In the case of timer irq, it's not necessarily a constant period.  You
> can tweak the period appropriately by adjusting the pace using a
> software PLL.  This would require read of hwptr as well as the normal
> update, but it's necessary to get the stream sync with the soundcard
> time.  If you don't refer to the hw ptrs, you'll get a significant
> drift in a long run.
>
>
> Takashi
>
>   

I agree, the irq source itself does not actually matter. just access the 
hw_ptr on each irq is all we need.
I think that accessing the hw_ptr once per period is perfectly Ok from a 
performance perspective, but any more than that is unnecessary. One 
could even do it once per buffer period without any serious loss of 
accuracy.
This approach would also reduce the amount of task switching because we 
could then have a userspace timer triggering events by simply using 
nanosleep().
The timer irqs would simply be for calculating the correction factor to 
gettimeofday() (get it in sync with the audio hardware clock), and could 
work asynchronously from the userspace timer triggering events.

James

 


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: ALSA sample rate conversion and general performance improvements.

[Paul Davis]

On Wed, 2006-03-29 at 11:03 +0100, James Courtier-Dutton wrote:
> Takashi Iwai wrote:
> > At Wed, 29 Mar 2006 00:24:20 +0100,
> > James Courtier-Dutton wrote:
> >   
> >> Hi,
> >>
> >> I have had some conversations with various people regarding some general
> >> problems some audio developers have with the current ALSA sound model.

Hi James. Not me by any chance? :)

> >> What about doing the following?:
> >> We implement a general timer in the alsa core based on something like
> >> gettimeofday(). I.e A high resolution timer.
> >> Each time a period interrupt happens, the current hw pointer is read,
> >> and recorded along with the current value of the timer.
> >> This could be used a bit like NTP to correct the generally timer rate.
> >> This gives us a general high resolution timer corrected to mirror the
> >> PCM clock on the sound card.

to correct something said later, this should be done using a DLL, not a
PLL. subtle but significant difference.

> >> Then, from this high resolution timer, we could derive any rate
> >> interrupt we need for really excellent sample rate conversion, and sub
> >> sample accurate DAC/ADC positioning.
> >> One could then read the DAC/ADC positioning really quickly by just
> >> reading the high resolution timer, instead of accessing IO Ports on the
> >> sound card hardware.
> >> This would also be a fairly minor change, as none of the current kernel
> >> sound card hw drivers would need changing. We would just have to
> >> slightly modify the period_elapsed callback to do a gettimeofday() call
> >> as well as it's current hw_pointer read.

i think its more than a minor change. the real benefit of this model
(which in my eyes correctly emulates the CoreAudio HAL) is that *if* you
have a schedulable high res timer (which linux, appallingly, still does
not have; we'll accept HZ=1000 for now, i guess), you don't need the
period interrupt to be isochronous/periodic any more - you drive user
space from the high res timer, knowing that you can have a lot of
confidence in your knowledge of where the h/w ptr(s) are. thus, USB and
FW devices, which do not have periodic interrupts driven by a sample
clock, can be used as if they do.

> >> If we then made the correction factor for the high resolution timer
> >> available to user space, user space would then not even need to read the
> >> current hw position, as it could perfectly predict it itself using the
> >> gettimeofday() and correction factor.

precisely.

> >> This would reduce the amount of calls to the read_hw_pointer function in
> >> the low level sound card hardware driver, and therefore improve performance.

the performance gain is small; the functionality gain for USB + FW is
much larger.

> > My idea is to make the system choose arbitrary irq sources, e.g. the
> > period irq, timer irq, etc.  The default is the period irq, or
> > whatever the driver provides.  But, the app or alsa-lib can switch to
> > a suitable one.

as i said above, this is basically what CoreAudio does and it seems to
be a much more flexible model than the current ALSA internals. it also
allows them to do resampling in a more flexible way than ALSA, although
admittedly, they do this in the kernel which ALSA would not.

--p




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: ALSA sample rate conversion and general performance improvements.

[James Courtier-Dutton]

Paul Davis wrote:
> On Wed, 2006-03-29 at 11:03 +0100, James Courtier-Dutton wrote:
>   
>> Takashi Iwai wrote:
>>     
>>> At Wed, 29 Mar 2006 00:24:20 +0100,
>>> James Courtier-Dutton wrote:
>>>   
>>>       
>>>> Hi,
>>>>
>>>> I have had some conversations with various people regarding some general
>>>> problems some audio developers have with the current ALSA sound model.
>>>>         
>
> Hi James. Not me by any chance? :)
>   
You were one of them yes. You seemed to explain it a bit better than 
other people have. I mulled over the ideas this weekend and decided that 
it would be a move in the right direction for ALSA. It would help us out 
with the current resampling problems. It might complicate error 
recovery, but I am sure we can overcome those in time. So long as we can 
still reliably detect xruns, I am all for the changes. Now, what to do 
after an xrun happens is quite another issue, but detection of it is 
certainly required.

James




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: ALSA sample rate conversion and general performance improvements.

[Christian Henz]

On Wed, Mar 29, 2006 at 09:52:39AM -0500, Paul Davis wrote:
> On Wed, 2006-03-29 at 11:03 +0100, James Courtier-Dutton wrote:
> > Takashi Iwai wrote:
> > > At Wed, 29 Mar 2006 00:24:20 +0100,
> > > James Courtier-Dutton wrote:
> > >   
> > >> Hi,
> > >>
> > >> I have had some conversations with various people regarding some general
> > >> problems some audio developers have with the current ALSA sound model.
> 
> Hi James. Not me by any chance? :)
> 
> > >> What about doing the following?:
> > >> We implement a general timer in the alsa core based on something like
> > >> gettimeofday(). I.e A high resolution timer.
> > >> Each time a period interrupt happens, the current hw pointer is read,
> > >> and recorded along with the current value of the timer.
> > >> This could be used a bit like NTP to correct the generally timer rate.
> > >> This gives us a general high resolution timer corrected to mirror the
> > >> PCM clock on the sound card.
> 
> to correct something said later, this should be done using a DLL, not a
> PLL. subtle but significant difference.
> 
> > >> Then, from this high resolution timer, we could derive any rate
> > >> interrupt we need for really excellent sample rate conversion, and sub
> > >> sample accurate DAC/ADC positioning.
> > >> One could then read the DAC/ADC positioning really quickly by just
> > >> reading the high resolution timer, instead of accessing IO Ports on the
> > >> sound card hardware.
> > >> This would also be a fairly minor change, as none of the current kernel
> > >> sound card hw drivers would need changing. We would just have to
> > >> slightly modify the period_elapsed callback to do a gettimeofday() call
> > >> as well as it's current hw_pointer read.
> 
> i think its more than a minor change. the real benefit of this model
> (which in my eyes correctly emulates the CoreAudio HAL) is that *if* you
> have a schedulable high res timer (which linux, appallingly, still does
> not have; we'll accept HZ=1000 for now, i guess), you don't need the
>

Hmm, I though that we DO have schedulable high res timers
now with the recent hrtimers merge? At least thats what I 
got from watching the video of the talk on RT patches at 
FOSDEM (http://free-electrons.com/community/videos/conferences).
I might have to watch it again to see if that only applied to
the -rt patch...


cheers,
Christian


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: ALSA sample rate conversion and general performance improvements.

[Lee Revell]

On Wed, 2006-03-29 at 19:31 +0200, Christian Henz wrote:
> Hmm, I though that we DO have schedulable high res timers
> now with the recent hrtimers merge? At least thats what I 
> got from watching the video of the talk on RT patches at 
> FOSDEM (http://free-electrons.com/community/videos/conferences).
> I might have to watch it again to see if that only applied to
> the -rt patch... 

Only in the -rt patch for now.  The hrtimers infrastructure is in
mainline now, and actual hrtimers should be available soon.

Lee



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642