Streaming wake-up buffer from DSP to SoC over SPI

Streaming wake-up buffer from DSP to SoC over SPI

[Dylan Reid]

What is the right way to model a bursty, SPI based audio transfer?
There could be hours between samples, then several seconds of audio
will be ready.

The codec, an rt5677, is attached to the SoC with both an IIS bus and
a SPI bus.  The IIS bus is used for normal audio playback and record.
The SPI bus is used for transferring audio that caused a voice wake,
"OK Google" in this case.

When the codec detects the wake up phrase, the SoC will read out two
plus seconds of buffered data over the SPI bus and continue to stream
audio data over SPI for the rest of the query.  Nexus 9 has a custom
ioctl to access rt5677-spi and some user-space smarts to poll for more
data.

I was thinking of making a separate PCM device.  The odd things are
that it would have to originate from soc/codecs, the codec would own
the codec and cpu end of the link, and the hw_ptr position would need
to be queried over SPI.  But it would make userspace access to it more
standard looking.

Thanks for any opinions or thoughts.

Dylan

Re: Streaming wake-up buffer from DSP to SoC over SPI

[Vinod Koul]

On Mon, Dec 15, 2014 at 06:45:37PM -0800, Dylan Reid wrote:
> What is the right way to model a bursty, SPI based audio transfer?
> There could be hours between samples, then several seconds of audio
> will be ready.
> 
> The codec, an rt5677, is attached to the SoC with both an IIS bus and
> a SPI bus.  The IIS bus is used for normal audio playback and record.
> The SPI bus is used for transferring audio that caused a voice wake,
> "OK Google" in this case.
> 
> When the codec detects the wake up phrase, the SoC will read out two
> plus seconds of buffered data over the SPI bus and continue to stream
> audio data over SPI for the rest of the query.  Nexus 9 has a custom
> ioctl to access rt5677-spi and some user-space smarts to poll for more
> data.
why do we need custom ioctls :( current framework can address all your
needs..

> I was thinking of making a separate PCM device.  The odd things are
> that it would have to originate from soc/codecs, the codec would own
> the codec and cpu end of the link, and the hw_ptr position would need
> to be queried over SPI.  But it would make userspace access to it more
> standard looking.
okay here we have two things and they need to be decoupled. First is
detection by the codec for "OK Google". And second is getting the data (some
buffered while things are setup) to usermode.

For the former, the codec should implement a 'sink' for voice wake, so
essentially connecting the voice wake widget to MIC pins will power up codec
and codec can do the detection while rest of the system is sleeping

Once the event is detected, codec should send an interrupt to host, thereby
waking up the system. The driver will figure out that it is voice wake event
and should set the alsa kcontrol status, on which usermode needs to listen.

Now usermode knows the event occurred so lets read the data. During all the
activation, wakeup etc some data would be buffered, in you case 2 secs. IMO
PCM devices aren't suited for this, so you need to use a compressed device.
The data can come from SPI port (we can get all the buffered data in one big
burst". I know one vendor has implemented such a mechanism.

The device here wont be any different than regular PCM/Compressed devices.
Assume the data path is SPI rather than I2S so things would be modelled that
way. You still need to setup SPI port, DMAs like you would do on I2S.

HTH
-- 
~Vinod

Re: Streaming wake-up buffer from DSP to SoC over SPI

[Dylan Reid]

On Mon, Dec 15, 2014 at 8:41 PM, Vinod Koul <vinod.koul@intel.com> wrote:
> On Mon, Dec 15, 2014 at 06:45:37PM -0800, Dylan Reid wrote:
>> What is the right way to model a bursty, SPI based audio transfer?
>> There could be hours between samples, then several seconds of audio
>> will be ready.
>>
>> The codec, an rt5677, is attached to the SoC with both an IIS bus and
>> a SPI bus.  The IIS bus is used for normal audio playback and record.
>> The SPI bus is used for transferring audio that caused a voice wake,
>> "OK Google" in this case.
>>
>> When the codec detects the wake up phrase, the SoC will read out two
>> plus seconds of buffered data over the SPI bus and continue to stream
>> audio data over SPI for the rest of the query.  Nexus 9 has a custom
>> ioctl to access rt5677-spi and some user-space smarts to poll for more
>> data.
> why do we need custom ioctls :( current framework can address all your
> needs..

This is what I'm trying to avoid =)

>
>> I was thinking of making a separate PCM device.  The odd things are
>> that it would have to originate from soc/codecs, the codec would own
>> the codec and cpu end of the link, and the hw_ptr position would need
>> to be queried over SPI.  But it would make userspace access to it more
>> standard looking.
> okay here we have two things and they need to be decoupled. First is
> detection by the codec for "OK Google". And second is getting the data (some
> buffered while things are setup) to usermode.
>
> For the former, the codec should implement a 'sink' for voice wake, so
> essentially connecting the voice wake widget to MIC pins will power up codec
> and codec can do the detection while rest of the system is sleeping
>
> Once the event is detected, codec should send an interrupt to host, thereby
> waking up the system. The driver will figure out that it is voice wake event
> and should set the alsa kcontrol status, on which usermode needs to listen.
>
> Now usermode knows the event occurred so lets read the data. During all the
> activation, wakeup etc some data would be buffered, in you case 2 secs. IMO
> PCM devices aren't suited for this, so you need to use a compressed device.
> The data can come from SPI port (we can get all the buffered data in one big
> burst". I know one vendor has implemented such a mechanism.
>
> The device here wont be any different than regular PCM/Compressed devices.
> Assume the data path is SPI rather than I2S so things would be modelled that
> way. You still need to setup SPI port, DMAs like you would do on I2S.

Ah, I hadn't thought about using the compressed framework.  I can see
how that would make this easier.  I'll take a closer look at it
tomorrow.

Thanks Vinod.

-Dylan

>
> HTH
> --
> ~Vinod
>

Re: Streaming wake-up buffer from DSP to SoC over SPI

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 791 bytes --]

On Tue, Dec 16, 2014 at 12:16:40AM -0800, Dylan Reid wrote:
> On Mon, Dec 15, 2014 at 8:41 PM, Vinod Koul <vinod.koul@intel.com> wrote:

> > The device here wont be any different than regular PCM/Compressed devices.
> > Assume the data path is SPI rather than I2S so things would be modelled that
> > way. You still need to setup SPI port, DMAs like you would do on I2S.

> Ah, I hadn't thought about using the compressed framework.  I can see
> how that would make this easier.  I'll take a closer look at it
> tomorrow.

You should be able to use a normal device as well, you need to either
arrange for the driver to buffer data until userspace starts the magic
stream or arrange for userspace to not mind if it blocks for extended
periods.  Either compressed or uncompressed should work.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Streaming wake-up buffer from DSP to SoC over SPI

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 684 bytes --]

On Mon, Dec 15, 2014 at 06:45:37PM -0800, Dylan Reid wrote:

> I was thinking of making a separate PCM device.  The odd things are
> that it would have to originate from soc/codecs, the codec would own
> the codec and cpu end of the link, and the hw_ptr position would need
> to be queried over SPI.  But it would make userspace access to it more
> standard looking.

That's all totally fine from a framework point of view, if the data
comes from the CODEC with no SoC side involvement other than the control
interface bus then it's a good way to model things.  The component model
should make this a more natural fit than it is currently, you can just
add a DAI to the CODEC device.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]