proposal: snd_pcm_start_at()

proposal: snd_pcm_start_at()

[Tim Cussins]

Hi all,

I'm Tim: I work at Linn Products Ltd - we make Network Music Players,
amongst other things.

As you might imagine, synchronised-start is important when multiple
devices on the network are rendering the same audio. We'd be interested
in contributing a small expansion of the alsa-lib API to support
synchronised start.

Assuming we can synchronise the audio clocks (I'm aware this is not
trivial - It's not the topic of this post), we'd propose something like:

    int snd_pcm_start_at(snd_pcm_t* pcm, snd_htimestamp_t* tstamp);

and playback would begin as close to tstamp as possible. If tstamp is in
the past, it would should return an error.

Recent work by Takashi Iwai enables client code to set the clock type of
timestamps using snd_pcm_sw_params_set_tstamp_type(). This context could
quite naturally extend to tstamp argument of snd_pcm_start_at().

Before I get stuck into working up the details under the hood, it'd be
good to get some feedback/objections regarding this approach.

Thanks in advance!
Tim

Re: proposal: snd_pcm_start_at()

[Pierre-Louis Bossart]

On 10/2/14, 9:34 AM, Tim Cussins wrote:
> Hi all,
>
> I'm Tim: I work at Linn Products Ltd - we make Network Music Players,
> amongst other things.
>
> As you might imagine, synchronised-start is important when multiple
> devices on the network are rendering the same audio. We'd be interested
> in contributing a small expansion of the alsa-lib API to support
> synchronised start.
>
> Assuming we can synchronise the audio clocks (I'm aware this is not
> trivial - It's not the topic of this post), we'd propose something like:
>
>      int snd_pcm_start_at(snd_pcm_t* pcm, snd_htimestamp_t* tstamp);
>
> and playback would begin as close to tstamp as possible. If tstamp is in
> the past, it would should return an error.
>
> Recent work by Takashi Iwai enables client code to set the clock type of
> timestamps using snd_pcm_sw_params_set_tstamp_type(). This context could
> quite naturally extend to tstamp argument of snd_pcm_start_at().
>
> Before I get stuck into working up the details under the hood, it'd be
> good to get some feedback/objections regarding this approach.

It's probably better idea to start PCM playback with a bunch of zeroes 
and then rely on existing timestamping to insert samples at the right 
location in the ring buffer - which you have to do anyway to compensate 
for drifts between your network clock and audio clock. This is a more 
predictable solution that abstracts away all the time needed to arm DMA, 
FIFOs, etc. The only hardware-dependent variable that would remain is 
the precision/granularity of the timestamping.
-Pierre

Re: proposal: snd_pcm_start_at()

[Tim Cussins]

Hi Peirre,

On 02/10/14 18:41, Pierre-Louis Bossart wrote:
> On 10/2/14, 9:34 AM, Tim Cussins wrote:
>> Hi all,
>>
>> I'm Tim: I work at Linn Products Ltd - we make Network Music Players,
>> amongst other things.
>>
>> As you might imagine, synchronised-start is important when multiple
>> devices on the network are rendering the same audio. We'd be interested
>> in contributing a small expansion of the alsa-lib API to support
>> synchronised start.
>>
>> Assuming we can synchronise the audio clocks (I'm aware this is not
>> trivial - It's not the topic of this post), we'd propose something like:
>>
>>      int snd_pcm_start_at(snd_pcm_t* pcm, snd_htimestamp_t* tstamp);
>>
>> and playback would begin as close to tstamp as possible. If tstamp is in
>> the past, it would should return an error.
>>
>> Recent work by Takashi Iwai enables client code to set the clock type of
>> timestamps using snd_pcm_sw_params_set_tstamp_type(). This context could
>> quite naturally extend to tstamp argument of snd_pcm_start_at().
>>
>> Before I get stuck into working up the details under the hood, it'd be
>> good to get some feedback/objections regarding this approach.
>
> It's probably better idea to start PCM playback with a bunch of zeroes
> and then rely on existing timestamping to insert samples at the right
> location in the ring buffer - which you have to do anyway to compensate
> for drifts between your network clock and audio clock. This is a more
> predictable solution that abstracts away all the time needed to arm DMA,
> FIFOs, etc. The only hardware-dependent variable that would remain is
> the precision/granularity of the timestamping.
> -Pierre

Thanks heaps for the feedback :)

Agreed - streaming zeros 'as required' is pretty much the obvious 
solution when confined to the existing API.

So I guess I'll have to show a few more cards from my hand :D

The next iteration of our hardware platform will be accompanied by a 
move to linux, which explains our interest in ALSA for sound delivery.

The pcm hardware for the new platform can start rendering when a compare 
register matches a hw counter (driven by the audio clock). This allows 
for starting with frame-accurate timing.

By adding another tstamp_type - let's call it 
SND_PCM_TSTAMP_TYPE_HARDWARE for now - snd_pcm_start_at() could 
internally delegate to the driver (if it supports it, otherwise error)

I take your point about drift, but we're approaching that as a related, 
but orthogonal, problem. As a business we've decided that dropouts are 
unacceptable. Our products have several audio clock sources available, 
one of which is a high-quality VXCO. With an appropriate servo, it will 
track a clock recovered from the incoming stream.

I'm still working through a couple of options for exposing the VXCO 
audio clock to a userspace servo [The linux kernel has the PCH 
framework, for example, which is almost enough, but doesn't dovetail 
with ALSA in an obvious way].

In summary, we'd like to leverage our pcm hardware's ability to start on 
time (hence the chat about snd_pcm_start_at), and we'd also like to 
expose control of the pcm's audio clock in some way (WIP, suggestions 
welcome!).

The clock control stuff is harder, but I wanted to get on the ml with 
something simple, say hi, propose a thing

Thanks for reading - I'll have questions about clock control soon 
enough, see ya then.

Cheers,
Tim

Re: proposal: snd_pcm_start_at()

[Pierre-Louis Bossart]

On 10/3/14, 7:00 AM, Tim Cussins wrote:
> Hi Peirre,
>
> On 02/10/14 18:41, Pierre-Louis Bossart wrote:
>> On 10/2/14, 9:34 AM, Tim Cussins wrote:
>>> Hi all,
>>>
>>> I'm Tim: I work at Linn Products Ltd - we make Network Music Players,
>>> amongst other things.
>>>
>>> As you might imagine, synchronised-start is important when multiple
>>> devices on the network are rendering the same audio. We'd be interested
>>> in contributing a small expansion of the alsa-lib API to support
>>> synchronised start.
>>>
>>> Assuming we can synchronise the audio clocks (I'm aware this is not
>>> trivial - It's not the topic of this post), we'd propose something like:
>>>
>>>      int snd_pcm_start_at(snd_pcm_t* pcm, snd_htimestamp_t* tstamp);
>>>
>>> and playback would begin as close to tstamp as possible. If tstamp is in
>>> the past, it would should return an error.
>>>
>>> Recent work by Takashi Iwai enables client code to set the clock type of
>>> timestamps using snd_pcm_sw_params_set_tstamp_type(). This context could
>>> quite naturally extend to tstamp argument of snd_pcm_start_at().
>>>
>>> Before I get stuck into working up the details under the hood, it'd be
>>> good to get some feedback/objections regarding this approach.
>>
>> It's probably better idea to start PCM playback with a bunch of zeroes
>> and then rely on existing timestamping to insert samples at the right
>> location in the ring buffer - which you have to do anyway to compensate
>> for drifts between your network clock and audio clock. This is a more
>> predictable solution that abstracts away all the time needed to arm DMA,
>> FIFOs, etc. The only hardware-dependent variable that would remain is
>> the precision/granularity of the timestamping.
>> -Pierre
>
> Thanks heaps for the feedback :)
>
> Agreed - streaming zeros 'as required' is pretty much the obvious
> solution when confined to the existing API.
>
> So I guess I'll have to show a few more cards from my hand :D
>
> The next iteration of our hardware platform will be accompanied by a
> move to linux, which explains our interest in ALSA for sound delivery.
>
> The pcm hardware for the new platform can start rendering when a compare
> register matches a hw counter (driven by the audio clock). This allows
> for starting with frame-accurate timing.

Interesting. I wonder if you actually need a new extension for this, you 
could write the timestamp in an ALSA control and implement your .trigger 
function by using the contents of the control, i.e. delay the actual 
start. it wouldn't be generic but your hardware isn't either.

> By adding another tstamp_type - let's call it
> SND_PCM_TSTAMP_TYPE_HARDWARE for now - snd_pcm_start_at() could
> internally delegate to the driver (if it supports it, otherwise error)
>
> I take your point about drift, but we're approaching that as a related,
> but orthogonal, problem. As a business we've decided that dropouts are
> unacceptable. Our products have several audio clock sources available,
> one of which is a high-quality VXCO. With an appropriate servo, it will
> track a clock recovered from the incoming stream.

The drift control can be done with ASRC or tweaking the clock, either 
way you need a servo loop based on timestamping info reported by the 
hardware.

> I'm still working through a couple of options for exposing the VXCO
> audio clock to a userspace servo [The linux kernel has the PCH
> framework, for example, which is almost enough, but doesn't dovetail
> with ALSA in an obvious way].

That part would benefit other implementations - whether they have a VCO 
or can do ASRC with a fixed clock. Looking forward to your ideas.

>
> In summary, we'd like to leverage our pcm hardware's ability to start on
> time (hence the chat about snd_pcm_start_at), and we'd also like to
> expose control of the pcm's audio clock in some way (WIP, suggestions
> welcome!).
>
> The clock control stuff is harder, but I wanted to get on the ml with
> something simple, say hi, propose a thing
>
> Thanks for reading - I'll have questions about clock control soon
> enough, see ya then.
>
> Cheers,
> Tim
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel

Re: proposal: snd_pcm_start_at()

[Takashi Iwai]

At Fri, 03 Oct 2014 17:24:22 -0500,
Pierre-Louis Bossart wrote:
> 
> On 10/3/14, 7:00 AM, Tim Cussins wrote:
> > Hi Peirre,
> >
> > On 02/10/14 18:41, Pierre-Louis Bossart wrote:
> >> On 10/2/14, 9:34 AM, Tim Cussins wrote:
> >>> Hi all,
> >>>
> >>> I'm Tim: I work at Linn Products Ltd - we make Network Music Players,
> >>> amongst other things.
> >>>
> >>> As you might imagine, synchronised-start is important when multiple
> >>> devices on the network are rendering the same audio. We'd be interested
> >>> in contributing a small expansion of the alsa-lib API to support
> >>> synchronised start.
> >>>
> >>> Assuming we can synchronise the audio clocks (I'm aware this is not
> >>> trivial - It's not the topic of this post), we'd propose something like:
> >>>
> >>>      int snd_pcm_start_at(snd_pcm_t* pcm, snd_htimestamp_t* tstamp);
> >>>
> >>> and playback would begin as close to tstamp as possible. If tstamp is in
> >>> the past, it would should return an error.
> >>>
> >>> Recent work by Takashi Iwai enables client code to set the clock type of
> >>> timestamps using snd_pcm_sw_params_set_tstamp_type(). This context could
> >>> quite naturally extend to tstamp argument of snd_pcm_start_at().
> >>>
> >>> Before I get stuck into working up the details under the hood, it'd be
> >>> good to get some feedback/objections regarding this approach.
> >>
> >> It's probably better idea to start PCM playback with a bunch of zeroes
> >> and then rely on existing timestamping to insert samples at the right
> >> location in the ring buffer - which you have to do anyway to compensate
> >> for drifts between your network clock and audio clock. This is a more
> >> predictable solution that abstracts away all the time needed to arm DMA,
> >> FIFOs, etc. The only hardware-dependent variable that would remain is
> >> the precision/granularity of the timestamping.
> >> -Pierre
> >
> > Thanks heaps for the feedback :)
> >
> > Agreed - streaming zeros 'as required' is pretty much the obvious
> > solution when confined to the existing API.
> >
> > So I guess I'll have to show a few more cards from my hand :D
> >
> > The next iteration of our hardware platform will be accompanied by a
> > move to linux, which explains our interest in ALSA for sound delivery.
> >
> > The pcm hardware for the new platform can start rendering when a compare
> > register matches a hw counter (driven by the audio clock). This allows
> > for starting with frame-accurate timing.
> 
> Interesting. I wonder if you actually need a new extension for this, you 
> could write the timestamp in an ALSA control and implement your .trigger 
> function by using the contents of the control, i.e. delay the actual 
> start. it wouldn't be generic but your hardware isn't either.

Well, your suggestion sounds really tricky.  The trigger is supposed
to trigger the stream immediately, and the delay isn't considered
there in principle.  The system can work with delays, but it's not in
a form of the initial design.

I think some synchronized triggering mechanism is missing in API, too.
There has been a similar request from others in the past (Digigram
wanted to have such a feature), so maybe it's not so uncommon
scenario.

This would be a good topic to be discussed in the upcoming audio
mini-summit, but both of you won't be there, right?


Takashi

Re: proposal: snd_pcm_start_at()

[Tim Cussins]

Hi Takashi,

On 06/10/14 10:45, Takashi Iwai wrote:
> At Fri, 03 Oct 2014 17:24:22 -0500,
> Pierre-Louis Bossart wrote:
>>
>> On 10/3/14, 7:00 AM, Tim Cussins wrote:
>>> Hi Peirre,
>>>

>>> The pcm hardware for the new platform can start rendering when a compare
>>> register matches a hw counter (driven by the audio clock). This allows
>>> for starting with frame-accurate timing.
>>
>> Interesting. I wonder if you actually need a new extension for this, you
>> could write the timestamp in an ALSA control and implement your .trigger
>> function by using the contents of the control, i.e. delay the actual
>> start. it wouldn't be generic but your hardware isn't either.
>
> Well, your suggestion sounds really tricky.  The trigger is supposed
> to trigger the stream immediately, and the delay isn't considered
> there in principle.  The system can work with delays, but it's not in
> a form of the initial design.
>
> I think some synchronized triggering mechanism is missing in API, too.
> There has been a similar request from others in the past (Digigram
> wanted to have such a feature), so maybe it's not so uncommon
> scenario.
>
> This would be a good topic to be discussed in the upcoming audio
> mini-summit, but both of you won't be there, right?
>

I didn't have any plans to be at the mini-summit, but Glasgow isn't too 
far away...

>
> Takashi
>

Re: proposal: snd_pcm_start_at()

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 432 bytes --]

On Mon, Oct 06, 2014 at 11:45:57AM +0200, Takashi Iwai wrote:

> I think some synchronized triggering mechanism is missing in API, too.
> There has been a similar request from others in the past (Digigram
> wanted to have such a feature), so maybe it's not so uncommon
> scenario.

Yes, I've had people ask me about this too, mainly around getting
multiple DSP streams synchronized.  It was a while ago and a bit edge
casey though.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: proposal: snd_pcm_start_at()

[Takashi Iwai]

At Wed, 8 Oct 2014 15:18:32 +0100,
Mark Brown wrote:
> 
> On Mon, Oct 06, 2014 at 11:45:57AM +0200, Takashi Iwai wrote:
> 
> > I think some synchronized triggering mechanism is missing in API, too.
> > There has been a similar request from others in the past (Digigram
> > wanted to have such a feature), so maybe it's not so uncommon
> > scenario.
> 
> Yes, I've had people ask me about this too, mainly around getting
> multiple DSP streams synchronized.  It was a while ago and a bit edge
> casey though.

OK, I added it to the topic list of audio mini-summit.
Let's see whether we can talk something about this.


Takashi

Re: proposal: snd_pcm_start_at()

[Tim Cussins]

On 08/10/14 16:29, Takashi Iwai wrote:
> At Wed, 8 Oct 2014 15:18:32 +0100,
> Mark Brown wrote:
>>
>> On Mon, Oct 06, 2014 at 11:45:57AM +0200, Takashi Iwai wrote:
>>
>>> I think some synchronized triggering mechanism is missing in API, too.
>>> There has been a similar request from others in the past (Digigram
>>> wanted to have such a feature), so maybe it's not so uncommon
>>> scenario.
>>
>> Yes, I've had people ask me about this too, mainly around getting
>> multiple DSP streams synchronized.  It was a while ago and a bit edge
>> casey though.
>
> OK, I added it to the topic list of audio mini-summit.
> Let's see whether we can talk something about this.
>

Awesome. There's a couple of things I can do here:

1) Post my ideas for selecting/controlling VCO associated with PCM _soon_
2) Come over on Tuesday and join the mini summit
3) Both!

I'm pretty new to ALSA, so I can only realistically contribute to the 
scheduled stream trigger discussion, although the Configuration/ASoC 
stuff will be relevant to my crowd too.

If no-one has a major objection to a rep from the ALSA-newbie community, 
I'll see if I can come over :)

Either way, I'll post my VCO ideas to the list ASAP.

Cheers,
Tim

Re: proposal: snd_pcm_start_at()

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 447 bytes --]

On Wed, Oct 08, 2014 at 05:09:04PM +0100, Tim Cussins wrote:
> On 08/10/14 16:29, Takashi Iwai wrote:

> >OK, I added it to the topic list of audio mini-summit.
> >Let's see whether we can talk something about this.

> 2) Come over on Tuesday and join the mini summit

This would be great, but please note that since the Linux Foundation are
doing the logistics (room and so on) for us you'd need to register for
one of the conferences that week.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: proposal: snd_pcm_start_at()

[Tim Cussins]

On 08/10/14 21:16, Mark Brown wrote:
> On Wed, Oct 08, 2014 at 05:09:04PM +0100, Tim Cussins wrote:
>> On 08/10/14 16:29, Takashi Iwai wrote:
>
>>> OK, I added it to the topic list of audio mini-summit.
>>> Let's see whether we can talk something about this.
>
>> 2) Come over on Tuesday and join the mini summit
>
> This would be great, but please note that since the Linux Foundation are
> doing the logistics (room and so on) for us you'd need to register for
> one of the conferences that week.
>

Righto. Registration has closed (all full I guess!) - I'll see if I can 
find a (legitimate) way in :)

Re: proposal: snd_pcm_start_at()

[Nick Stoughton]

Interestingly, I implemented this exact function (snd_pcm_start_at) about a
week ago before I saw this thread! We are still experimenting with it, but
I'd be happy to discuss it next week at the summit.

*______________________________*
*Nick Stoughton*
 *Aether Things Inc *
 *San Francisco*
  +1 (510) 388 1413


On Thu, Oct 9, 2014 at 2:20 AM, Tim Cussins <timcussins@eml.cc> wrote:

> On 08/10/14 21:16, Mark Brown wrote:
>
>> On Wed, Oct 08, 2014 at 05:09:04PM +0100, Tim Cussins wrote:
>>
>>> On 08/10/14 16:29, Takashi Iwai wrote:
>>>
>>
>>  OK, I added it to the topic list of audio mini-summit.
>>>> Let's see whether we can talk something about this.
>>>>
>>>
>>  2) Come over on Tuesday and join the mini summit
>>>
>>
>> This would be great, but please note that since the Linux Foundation are
>> doing the logistics (room and so on) for us you'd need to register for
>> one of the conferences that week.
>>
>>
> Righto. Registration has closed (all full I guess!) - I'll see if I can
> find a (legitimate) way in :)
>
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>