[RFC 0/4] ASoC DSP topology

[RFC 0/4] ASoC DSP topology

[Liam Girdwood]

It's been about 2 years since posting V1 of the ASoC dynamic firmware patches
and a lot has changed since then ...

This RFC series is being posted instead of a V2 patch series atm since we are
still completing some testing on the codebase and developing a new userspace
tool. Expect V2 once more testing is completed shortly.

ASoC topology (formerly called dynamic firmware) allows audio DSP topology to
be defined as part of the firmware package and not hard coded into the driver.
This allows DSP vendors to ship multiple different firmwares or firmwares with
programmable topologies for different devices (i.e. phones, tablets, TVs) using
a single DSP driver. The DSP topology can be contained as part of the firmware
file or exist as separate files. The topology core and file format is generic
and not tied to any single vendor or device. 

The topology core consists of kernel and userspace components :-

Kernel. The kernel core reads in the topology data and builds the driver
capabilities and audio topology based on this data. This includes building
the DAPM graph, setting PCM and DAI capabilities, registering kcontrols and
registering DAPM widgets.

The userspace tool allows the creation of binary DSP FW topology files based on
a topology text files.

Changes since V1:-

 o Name changed to ASoC Topology
 o Structures from soc.h and soc-dapm.h no longer exposed as UAPIs
 o Moved to component model from codec/platform model.
 o Added reserved fields in each UAPI structure.
 o UAPI structures packed and endianess defined for each structure.
 o Added support for PCM, DAI and codec <-> codec link capabilities.
 o Added support for client drivers manifests.
 o Added support for private data for each dynamic object type.
 o Single list of dynamic objects rather than list of each object type.

Liam

Re: [RFC 0/4] ASoC DSP topology

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 658 bytes --]

On Thu, Apr 16, 2015 at 09:48:05PM +0100, Liam Girdwood wrote:
> It's been about 2 years since posting V1 of the ASoC dynamic firmware patches
> and a lot has changed since then ...

:)

> Kernel. The kernel core reads in the topology data and builds the driver
> capabilities and audio topology based on this data. This includes building
> the DAPM graph, setting PCM and DAI capabilities, registering kcontrols and
> registering DAPM widgets.

I think the topology code itself looks good, I'll take another pass over
it though since there's a lot to get through.  The main thing is to make
sure the ABI makes sense though - we can fix up the kernel later.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC 0/4] ASoC DSP topology

[Peter Ujfalusi]

Liam,

On 04/16/2015 11:48 PM, Liam Girdwood wrote:
> It's been about 2 years since posting V1 of the ASoC dynamic firmware patches
> and a lot has changed since then ...
> 
> This RFC series is being posted instead of a V2 patch series atm since we are
> still completing some testing on the codebase and developing a new userspace
> tool. Expect V2 once more testing is completed shortly.
> 
> ASoC topology (formerly called dynamic firmware) allows audio DSP topology to
> be defined as part of the firmware package and not hard coded into the driver.
> This allows DSP vendors to ship multiple different firmwares or firmwares with
> programmable topologies for different devices (i.e. phones, tablets, TVs) using
> a single DSP driver. The DSP topology can be contained as part of the firmware
> file or exist as separate files. The topology core and file format is generic
> and not tied to any single vendor or device. 
> 
> The topology core consists of kernel and userspace components :-
> 
> Kernel. The kernel core reads in the topology data and builds the driver
> capabilities and audio topology based on this data. This includes building
> the DAPM graph, setting PCM and DAI capabilities, registering kcontrols and
> registering DAPM widgets.

It is not entirely clear for me based on the first look, but who is
responsible to initiate the topology load? Is it the component or machine
driver? We have had issues with deferred probing when the component driver was
in charge of loading the firmware. I got around this by initiating the FW load
from the machine driver and via callbacks I notified the component driver to
take over from that point. This fixed the probe order and I can also handle
cases when the filesystem does not have the firmware so I can fall back to
'legacy' mode.

> The userspace tool allows the creation of binary DSP FW topology files based on
> a topology text files.

Is it possible to get access to the user space part so if I have time I can
try to convert the OMAP4/5 AESS/ABE code to use tplg?

Thanks,
Péter

> 
> Changes since V1:-
> 
>  o Name changed to ASoC Topology
>  o Structures from soc.h and soc-dapm.h no longer exposed as UAPIs
>  o Moved to component model from codec/platform model.
>  o Added reserved fields in each UAPI structure.
>  o UAPI structures packed and endianess defined for each structure.
>  o Added support for PCM, DAI and codec <-> codec link capabilities.
>  o Added support for client drivers manifests.
>  o Added support for private data for each dynamic object type.
>  o Single list of dynamic objects rather than list of each object type.
> 
> Liam
> 
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
> 

Re: [RFC 0/4] ASoC DSP topology

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 901 bytes --]

On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:

> It is not entirely clear for me based on the first look, but who is
> responsible to initiate the topology load? Is it the component or machine
> driver? We have had issues with deferred probing when the component driver was
> in charge of loading the firmware. I got around this by initiating the FW load
> from the machine driver and via callbacks I notified the component driver to
> take over from that point. This fixed the probe order and I can also handle
> cases when the filesystem does not have the firmware so I can fall back to
> 'legacy' mode.

Could you expand on those issues please?  I'd *really* not expect the
machine driver to be involved in loading firmware for a component driver
(think how this is going to affect generic drivers) and it's not obvious
to me what impact this might have on deferred probe either.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC 0/4] ASoC DSP topology

[Liam Girdwood]

On Tue, 2015-04-21 at 10:28 +0100, Mark Brown wrote:
> On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:
> 
> > It is not entirely clear for me based on the first look, but who is
> > responsible to initiate the topology load? Is it the component or machine
> > driver? We have had issues with deferred probing when the component driver was
> > in charge of loading the firmware. I got around this by initiating the FW load
> > from the machine driver and via callbacks I notified the component driver to
> > take over from that point. This fixed the probe order and I can also handle
> > cases when the filesystem does not have the firmware so I can fall back to
> > 'legacy' mode.
> 
> Could you expand on those issues please?  I'd *really* not expect the
> machine driver to be involved in loading firmware for a component driver
> (think how this is going to affect generic drivers) and it's not obvious
> to me what impact this might have on deferred probe either.

The OMAP4 driver should not have any issues using the new topology code,
it's very generic :)

The topology code has moved to the component model so the ABE DSP driver
can load it's FW topology. If the FS is not available to load FW or
topology then you can wait until FS is ready (the upstream SST drivers
wait for FW to be ready before continuing with probing).

Liam

Re: [RFC 0/4] ASoC DSP topology

[Peter Ujfalusi]

On 04/21/2015 12:28 PM, Mark Brown wrote:
> On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:
> 
>> It is not entirely clear for me based on the first look, but who is
>> responsible to initiate the topology load? Is it the component or machine
>> driver? We have had issues with deferred probing when the component driver was
>> in charge of loading the firmware. I got around this by initiating the FW load
>> from the machine driver and via callbacks I notified the component driver to
>> take over from that point. This fixed the probe order and I can also handle
>> cases when the filesystem does not have the firmware so I can fall back to
>> 'legacy' mode.
> 
> Could you expand on those issues please?  I'd *really* not expect the
> machine driver to be involved in loading firmware for a component driver
> (think how this is going to affect generic drivers) and it's not obvious
> to me what impact this might have on deferred probe either.

The two issue were interconnected. We had cases when all components have been
loaded, but the machine driver was still in the deferred list, if we loaded
any non related module the card will suddenly got created.
It is due to the firmware loading. The firmware load will not kick the
deferred module list (we had a hack in place to kick the queue from the ABE
driver):
Components are loaded and registered in ASoC with the exception of the parts
needing the FW, since the dynamic FW contained widgets, routes, etc. So the
ABE driver loaded and requested the FW, probe exited with success. Machine
driver is still not able to form the card since the ABE widgets, PCMs are
missing, it goes to deferred probe. The ABE driver loads the dynamic FW,
parses it, DAPM widgets, PCMs, routes are created and registered. But the
machine driver is in deferred list and there is nothing to kick that queue,
which means that we have everything, but the card is not created.

But if we initiate the FW load from the machine driver and we pass the FW down
to the component driver(s) interested in handling the FW we are going to be
fine since we only register the card when the component drivers already parsed
and processed the FW image.

In this way we can have fall back for cases when the FW is not available on
the filesystem (some distro might want to go w/o binary blob).

We have discussed this with Liam in the past: in my view the DSP topology (or
Dynamic FW) should be represented in the machine level and it would be the
best if the same image could carry card level widgets routes and links. If you
have big enough change in the FW and it's provided widgets/PCMs you would need
separate machine driver or at least a way to have different set of machine
level routes, widgets, links, etc for different DSP topology file.

One of the issue with this: When we need to call the component driver's
firmware load function (to process the FW and create the widgets and then
register the component) the component is not registered (we need the firmware
to set up things) so generic snd_soc_* callback is not possible to use

-- 
Péter

Re: [RFC 0/4] ASoC DSP topology

[Liam Girdwood]

On Tue, 2015-04-21 at 15:30 +0300, Peter Ujfalusi wrote:
> On 04/21/2015 12:28 PM, Mark Brown wrote:
> > On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:
> > 
> >> It is not entirely clear for me based on the first look, but who is
> >> responsible to initiate the topology load? Is it the component or machine
> >> driver? We have had issues with deferred probing when the component driver was
> >> in charge of loading the firmware. I got around this by initiating the FW load
> >> from the machine driver and via callbacks I notified the component driver to
> >> take over from that point. This fixed the probe order and I can also handle
> >> cases when the filesystem does not have the firmware so I can fall back to
> >> 'legacy' mode.
> > 
> > Could you expand on those issues please?  I'd *really* not expect the
> > machine driver to be involved in loading firmware for a component driver
> > (think how this is going to affect generic drivers) and it's not obvious
> > to me what impact this might have on deferred probe either.
> 
> The two issue were interconnected. We had cases when all components have been
> loaded, but the machine driver was still in the deferred list, if we loaded
> any non related module the card will suddenly got created.
> It is due to the firmware loading. The firmware load will not kick the
> deferred module list (we had a hack in place to kick the queue from the ABE
> driver):

Yes, I remember that hack :)

Can you take a look at the work Jarkko did for FW and component loading
in sst-acpi.c. This should work for ABE too.

> Components are loaded and registered in ASoC with the exception of the parts
> needing the FW, since the dynamic FW contained widgets, routes, etc. So the
> ABE driver loaded and requested the FW, probe exited with success. Machine
> driver is still not able to form the card since the ABE widgets, PCMs are
> missing, it goes to deferred probe. The ABE driver loads the dynamic FW,
> parses it, DAPM widgets, PCMs, routes are created and registered. But the
> machine driver is in deferred list and there is nothing to kick that queue,
> which means that we have everything, but the card is not created.
> 
> But if we initiate the FW load from the machine driver and we pass the FW down
> to the component driver(s) interested in handling the FW we are going to be
> fine since we only register the card when the component drivers already parsed
> and processed the FW image.
> 
> In this way we can have fall back for cases when the FW is not available on
> the filesystem (some distro might want to go w/o binary blob).
> 
> We have discussed this with Liam in the past: in my view the DSP topology (or
> Dynamic FW) should be represented in the machine level and it would be the
> best if the same image could carry card level widgets routes and links. If you
> have big enough change in the FW and it's provided widgets/PCMs you would need
> separate machine driver or at least a way to have different set of machine
> level routes, widgets, links, etc for different DSP topology file.
> 

The component level allows us to target the physical component devices
that may have runtime definable topologies. This would include codecs
too, since some vendors are making codecs with built in FW (maybe TI
too ?). The machine level more represents the board HW topology and this
should be derived from ACPI or DT.

Fwiw, I've not been following ABE development so I'm not sure of the
current driver state in newer kernels. I dont see any problems if the
ABE driver used the same sst-acpi.c like probing method. Btw, there is
now also a non atomic trigger() that would simplify the ABE code too :)

Liam

Re: [RFC 0/4] ASoC DSP topology

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1309 bytes --]

On Tue, Apr 21, 2015 at 04:23:51PM +0100, Liam Girdwood wrote:
> On Tue, 2015-04-21 at 15:30 +0300, Peter Ujfalusi wrote:

> > We have discussed this with Liam in the past: in my view the DSP topology (or
> > Dynamic FW) should be represented in the machine level and it would be the
> > best if the same image could carry card level widgets routes and links. If you
> > have big enough change in the FW and it's provided widgets/PCMs you would need
> > separate machine driver or at least a way to have different set of machine
> > level routes, widgets, links, etc for different DSP topology file.

> The component level allows us to target the physical component devices
> that may have runtime definable topologies. This would include codecs
> too, since some vendors are making codecs with built in FW (maybe TI
> too ?). The machine level more represents the board HW topology and this
> should be derived from ACPI or DT.

I tend to agree.  We should let each vendor provide their own topology
information - if they need to do something with this (which does seem
unlikely) system integrators should be on an equal footing with silicon
vendors here, and we shouldn't be encouraging systems where we need
per-board firmware definitions for the silicon components just because
the board has differences.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC 0/4] ASoC DSP topology

[Vinod Koul]

[-- Attachment #1.1: Type: text/plain, Size: 2014 bytes --]

On Tue, Apr 21, 2015 at 05:39:42PM +0100, Mark Brown wrote:
> On Tue, Apr 21, 2015 at 04:23:51PM +0100, Liam Girdwood wrote:
> > On Tue, 2015-04-21 at 15:30 +0300, Peter Ujfalusi wrote:
> 
> > > We have discussed this with Liam in the past: in my view the DSP topology (or
> > > Dynamic FW) should be represented in the machine level and it would be the
> > > best if the same image could carry card level widgets routes and links. If you
> > > have big enough change in the FW and it's provided widgets/PCMs you would need
> > > separate machine driver or at least a way to have different set of machine
> > > level routes, widgets, links, etc for different DSP topology file.
> 
> > The component level allows us to target the physical component devices
> > that may have runtime definable topologies. This would include codecs
> > too, since some vendors are making codecs with built in FW (maybe TI
> > too ?). The machine level more represents the board HW topology and this
> > should be derived from ACPI or DT.
> 
> I tend to agree.  We should let each vendor provide their own topology
> information - if they need to do something with this (which does seem
> unlikely) system integrators should be on an equal footing with silicon
> vendors here, and we shouldn't be encouraging systems where we need
> per-board firmware definitions for the silicon components just because
> the board has differences.
One case which comes to my mind worth discussing here is PCMs.

We are trying to move FE dialink PCMs to toplogy as that is SW defined and
IMO not linked to HW.
So machine driver will not have dailinks to register at probe so we can't get
sound card created (other info like widgets map can get get registered after
probe though)

I was thinking that in core, a component should register with a toplogy bin
as well. Core can load (or with a callback) the toplogy binary and then parse it for
PCMs register them, and create sound card...

thoughts...

-- 
~Vinod

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]