Latency of mixer reconfiguration

Latency of mixer reconfiguration

[Lennart Poettering]

Heya!

So, as some of you might know in PA we extend the abilities of the
hardware volume control in software. Meaning that if a hw mixer
control only allows a limited range or granularity of a volume scale
or does not distuingish between channels then PA will configure the
hardware to the next higher available setting of the real
volume it wants to achieve, and then attenuates the PCM stream to
compensate for the rest.

This actually works quite well but there is a certain probelm with
latency: i.e. the mixer setting will be applied at a slightly
different time than the PCM attenuation. 

Now the question I have is, how should I best deal with this? I
currently assume that mixer volume changes are basically instantaneous
when I call the respective function of ALSA. But are they really? How
big is the latency at max? Do we need an API to query it?

Lennart

-- 
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4

Re: Latency of mixer reconfiguration

[James Courtier-Dutton]

On 17 February 2010 18:15, Lennart Poettering <mznyfn@0pointer.de> wrote:
> Heya!
>
> So, as some of you might know in PA we extend the abilities of the
> hardware volume control in software. Meaning that if a hw mixer
> control only allows a limited range or granularity of a volume scale
> or does not distuingish between channels then PA will configure the
> hardware to the next higher available setting of the real
> volume it wants to achieve, and then attenuates the PCM stream to
> compensate for the rest.
>
> This actually works quite well but there is a certain probelm with
> latency: i.e. the mixer setting will be applied at a slightly
> different time than the PCM attenuation.
>
> Now the question I have is, how should I best deal with this? I
> currently assume that mixer volume changes are basically instantaneous
> when I call the respective function of ALSA. But are they really? How
> big is the latency at max? Do we need an API to query it?
>
> Lennart
>

The latency is non-deterministic.
latency of the PCM stream is more predictable.
I am not sure how you might get around this problem.

Re: Latency of mixer reconfiguration

[Mark Brown]

On Wed, Feb 17, 2010 at 07:15:30PM +0100, Lennart Poettering wrote:

> Now the question I have is, how should I best deal with this? I
> currently assume that mixer volume changes are basically instantaneous
> when I call the respective function of ALSA. But are they really? How
> big is the latency at max? Do we need an API to query it?

For the embedded case the hardware update should happen synchronously
with the ALSA API call to adjust the setting and begin affecting the
played audio instantaneously.  The time taken to do the hardware update
will be dominated by I/O costs, which will in turn depend on the bus
used to access the codec - it'd be good if the buses could provide some
information to ASoC to allow it to do an estimate, but at the minute
we've got nothing really to go on.

What's more interesting from this point of view is that the update will
affect the audio going through the chip at the current moment so there
will be latency from the DMA chain and possibly also from the DSP chain
within the CODEC (which may vary depending on which volume you're
updating).

I suspect that trying to offer additional resolution in this way is more
trouble than it's worth if you're concerned about the artifacts that are
introduced during updates.  Providing per-channel differentiation if the
hardware has only mono control has much fewer problems, though.

Re: Latency of mixer reconfiguration

[Lennart Poettering]

On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:

> > Now the question I have is, how should I best deal with this? I
> > currently assume that mixer volume changes are basically instantaneous
> > when I call the respective function of ALSA. But are they really? How
> > big is the latency at max? Do we need an API to query it?
> 
> For the embedded case the hardware update should happen synchronously
> with the ALSA API call to adjust the setting and begin affecting the
> played audio instantaneously.  The time taken to do the hardware update
> will be dominated by I/O costs, which will in turn depend on the bus
> used to access the codec - it'd be good if the buses could provide some
> information to ASoC to allow it to do an estimate, but at the minute
> we've got nothing really to go on.

But what would you guess? In which range will this most likely be? < 1ms? 
1ms? 10ms? 100ms? 1s? 1h? 10h? 10d? 10y?

tbh I feared less the actually IO latency but more that some PCM data
fifos might need flushing before the volume is actually updated. And
the latency of those fifos I feared might be more than a handful of
samples?

> What's more interesting from this point of view is that the update will
> affect the audio going through the chip at the current moment so there
> will be latency from the DMA chain and possibly also from the DSP chain
> within the CODEC (which may vary depending on which volume you're
> updating).
> 
> I suspect that trying to offer additional resolution in this way is more
> trouble than it's worth if you're concerned about the artifacts that are
> introduced during updates.  Providing per-channel differentiation if the
> hardware has only mono control has much fewer problems, though.

The current logic is to not do any software adjustment if the hardware
adjustment is "close enough" to the total adjustment we want to do,
tested against a threshold. Which I think is quite a reasonable
approach because it enables/disables this feature not globally, but
looks at each case and enables this logic only if it really has a
benefit.

Lennart

-- 
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4

Re: Latency of mixer reconfiguration

[Jaroslav Kysela]

On Thu, 18 Feb 2010, Lennart Poettering wrote:

> On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
>
>>> Now the question I have is, how should I best deal with this? I
>>> currently assume that mixer volume changes are basically instantaneous
>>> when I call the respective function of ALSA. But are they really? How
>>> big is the latency at max? Do we need an API to query it?
>>
>> For the embedded case the hardware update should happen synchronously
>> with the ALSA API call to adjust the setting and begin affecting the
>> played audio instantaneously.  The time taken to do the hardware update
>> will be dominated by I/O costs, which will in turn depend on the bus
>> used to access the codec - it'd be good if the buses could provide some
>> information to ASoC to allow it to do an estimate, but at the minute
>> we've got nothing really to go on.
>
> But what would you guess? In which range will this most likely be? < 1ms?
> 1ms? 10ms? 100ms? 1s? 1h? 10h? 10d? 10y?

Some drivers uses a queue (like HDA) for serial bus to update the audio 
mixer registers. But anyway, the real write should take several ms for 
maximum (maybe more on USB bus).

> tbh I feared less the actually IO latency but more that some PCM data
> fifos might need flushing before the volume is actually updated. And
> the latency of those fifos I feared might be more than a handful of
> samples?

It's quite complex question and probably only hardware vendors knows 
these internals for all devices. I guess also, that some devices also uses 
volume ramping to avoid clicks.

 					Jaroslav

-----
Jaroslav Kysela <perex@perex.cz>
Linux Kernel Sound Maintainer
ALSA Project, Red Hat, Inc.

Re: Latency of mixer reconfiguration

[Mark Brown]

On Fri, Feb 19, 2010 at 10:01:26AM +0100, Jaroslav Kysela wrote:

> It's quite complex question and probably only hardware vendors knows 
> these internals for all devices. I guess also, that some devices also uses 
> volume ramping to avoid clicks.

The most common technique there is actually to do volume updates on zero
cross - ie, only update the gain when the signal level is zero.

Re: Latency of mixer reconfiguration

[Mark Brown]

On Thu, Feb 18, 2010 at 07:04:22PM +0100, Lennart Poettering wrote:
> On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:

> > will be dominated by I/O costs, which will in turn depend on the bus
> > used to access the codec - it'd be good if the buses could provide some
> > information to ASoC to allow it to do an estimate, but at the minute
> > we've got nothing really to go on.

> But what would you guess? In which range will this most likely be? < 1ms? 
> 1ms? 10ms? 100ms? 1s? 1h? 10h? 10d? 10y?

1ms or less normally - worst case will be a couple of I2C writes, though
potentially over a congested bus.

> tbh I feared less the actually IO latency but more that some PCM data
> fifos might need flushing before the volume is actually updated. And
> the latency of those fifos I feared might be more than a handful of
> samples?

Yes, mostly the buffers in the CPU.  These vary from very small to very
large - some systems allow relatively large audio buffers (hundreds of
kilobytes for example) in order to allow the CPU and RAM to be powered
down for extended periods of time during playback.  It's the same
problem as working out the latency for video sync.

> > I suspect that trying to offer additional resolution in this way is more
> > trouble than it's worth if you're concerned about the artifacts that are
> > introduced during updates.  Providing per-channel differentiation if the
> > hardware has only mono control has much fewer problems, though.

> The current logic is to not do any software adjustment if the hardware
> adjustment is "close enough" to the total adjustment we want to do,
> tested against a threshold. Which I think is quite a reasonable
> approach because it enables/disables this feature not globally, but
> looks at each case and enables this logic only if it really has a
> benefit.

That sounds reasonable, though it's kind of surprising to me that there
is hardware out there which benefits from it - I'd have expected either
adequate resolution or nothing at all there.

Re: Latency of mixer reconfiguration

[Lennart Poettering]

On Fri, 19.02.10 09:54, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:

> > > I suspect that trying to offer additional resolution in this way is more
> > > trouble than it's worth if you're concerned about the artifacts that are
> > > introduced during updates.  Providing per-channel differentiation if the
> > > hardware has only mono control has much fewer problems, though.
> 
> > The current logic is to not do any software adjustment if the hardware
> > adjustment is "close enough" to the total adjustment we want to do,
> > tested against a threshold. Which I think is quite a reasonable
> > approach because it enables/disables this feature not globally, but
> > looks at each case and enables this logic only if it really has a
> > benefit.
> 
> That sounds reasonable, though it's kind of surprising to me that there
> is hardware out there which benefits from it - I'd have expected either
> adequate resolution or nothing at all there.

On my own hardware this mechanism proved useful in two cases: one card
had a single volume slider for both channels and with PA you can now
set their volume independantly. And one set of external USB speakers
has a volume slider that starts at a very high level, so that the
minimum volume setting is everything but silent. PA extends the scale
downwards so that this limitation of the hw does not become apparent.

Lennart

-- 
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4

Re: Latency of mixer reconfiguration

[Raymond Yau]

2010/2/19 Lennart Poettering <mznyfn@0pointer.de>

>
> On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com)
> wrote:
>
>
> The current logic is to not do any software adjustment if the hardware
> adjustment is "close enough" to the total adjustment we want to do,
> tested against a threshold. Which I think is quite a reasonable
> approach because it enables/disables this feature not globally, but
> looks at each case and enables this logic only if it really has a
> benefit.
>
> Lennart
>
>
PA 's watermark is already 20 ms if you want to do any software adjustment
by PA server

Re: Latency of mixer reconfiguration

[Raymond Yau]

2010/2/22 Lennart Poettering <mznyfn@0pointer.de>

> On Sat, 20.02.10 11:59, Raymond Yau (superquad.vortex2@gmail.com) wrote:
>
> >
> > 2010/2/19 Lennart Poettering <mznyfn@0pointer.de>
> >
> > >
> > > On Thu, 18.02.10 10:01, Mark Brown (
> broonie@opensource.wolfsonmicro.com)
> > > wrote:
> > >
> > >
> > > The current logic is to not do any software adjustment if the hardware
> > > adjustment is "close enough" to the total adjustment we want to do,
> > > tested against a threshold. Which I think is quite a reasonable
> > > approach because it enables/disables this feature not globally, but
> > > looks at each case and enables this logic only if it really has a
> > > benefit.
>
> > PA 's watermark is already 20 ms if you want to do any software
> adjustment
> > by PA server
>
> It's not necessarily 20ms. That's just the default (which we
> unfortunately had to pick very large, since ALSA right now does not
> inform us about transfer block granularity).
>

I think ALSA did inform the appliication the the granularit is period size,
it is the responsibilty of the application which use timer scheduling to
find out the optimum granularity which may varies with different sound card,
different CPU speed



>
> But anyway, whatever the watermark is: this is a latency we know
> about, which means that we can delay the hw mixer updating
> accordingly. But that works correctly only if the hw mixer updating is
> immediate. WHich it most likely is not.
>

do you mean PA server will perform snd_pcm_rewind when the user change the
mixer volume ?


>
> So we now the software volume adjustment latency. We don't know the
> hardware volume adjustment latency, and assume it is 0. If we knew
> both we could sync up both changes perfectly, but this way our model
> has a weakness.
>

I have doubt there is any benefit if you even know those hardware volume
adjust ment latency which varies even when using the same chipset by
different OEM vendor sound card


>
> Lennart
> lsa-devel <http://mailman.alsa-project.org/mailman/listinfo/alsa-devel>
>