Re: [RFC 00/15] ALSA/ASoC: USB Audio Offload

[Cezary Rojewski <cezary.rojewski@intel.com>]

On 2025-04-10 12:24 PM, Takashi Iwai wrote:
> On Wed, 09 Apr 2025 13:07:15 +0200,
> Cezary Rojewski wrote:
>>
>> Note: this series is based on Mark's broonie/for-next. The xHCI
>> dependency is missing so it won't compile.  Goal of this RFC is
>> discussing the direction of sound/usb changes.
>>
>> Note #2: Why exclude xHCI?  Current form of xhci-sideband.c [1] does not
>> fare well with Intel's hardware design for USB Audio Offload feature.
>> The production shape for usb/xHCI subjects is being discussed with
>> Mathias.  Once we're ready, I'll share the rest.
>>
>> Note #3: this series does _NOT_ aim to block QCOM's equivalent series
>> [2].  The team does acknowledge that we came the "table" late.  At the
>> same time, we're prepared to help QCOM switch to the presented sound/usb
>> approach if that would benefit the framework and its users as a whole.
>> Make it part of this very series if need be.
>>
>>
>>
>> Apart from changes to sound/usb, the patchset contains exemplary
>> ASoC-based, offload-aware USB driver and a small sound card on the
>> avs-driver side to show how card/dai_link initialization looks like.
>>
>> In short, USB Audio Offload functionality lowers CPU usage when
>> streaming PCM over a USB Audio-Class device.  It has been first
>> introduced in xHCI 1.2 release and is described in section 7.9 [3].
>> Once hardware is prepared with hw_params(), all the USB endpoints
>> operations e.g.: start, stop, submitting URBs, are performed
>> internally, by the hardware and AudioDSP firmware.  Software driver
>> shall not intervene.
>>
>> Startup flow from:
>>
>> 	usb_pcm_open()
>> 	usb_pcm_hw_params()
>> 		snd_usb_endpoint_open()
>> 	usb_pcm_prepare()
>> 		usb_set_interface()
>> 		snd_usb_endpoint_start()
>> 	usb_pcm_trigger(cmd: START/STOP etc.)
>>
>> reduced to:
>>
>> 	usb_pcm_open()
>> 	usb_pcm_hw_params()
>> 		snd_usb_endpoint_open()
>> 	usb_pcm_prepare()
>> 		usb_set_interface()
> 
> Hmm, how can it be?  The start of EP at prepare stage is done only
> conditionally for non-lowlatency or implicit-feedback mode, for
> example.


On Intel architecture the AudioDSP utilizes internal channel to 
communicate with xHCI and perform all the transfer operations.  In 
essence, once SET_INTERFACE TRB is done, the usb-driver is not supposed 
to do anything.  The avs-driver (sound driver) would be the trigger here 
- sends the start/stop/etc. requests to the AudioDSP firmware, DSP does 
the rest.

By trigger I mean SET_PIPELINE_STATE IPC and avs_path_xxx() which are 
utilized for any transfer-type (sound/soc/intel/avs/pcm.c).

>> Handlers such as ack(), sync_stop(), pointer(), delay() are not used
>> here too.  The AudioDSP driver will handle pointer(), rest is
>> firmware/hardware responsibility.
>>
>>
>> There's a hefty number of limitations, most importantly:
>>
>> 1) typically 2 USB devices tops, rest go the classic (non-offload) path
>> 2) AUDIOSTREAMING interfaces only, MIDI not.  While not an interface
>>     type, Media (sound/usb/media) unsupported either
>> 3) simple PCM, UAC_FORMAT_TYPE_I_PCM only
>>
>> Current patchset shows this in form of 'udev->audsb_capable' field.
>> True if xHCI sideband reasource has been assigned to the device.  The
>> filtering is code is not part of the patchset.
>>
>> Important to highlight, 1) means both, offload-aware and offload-unaware
>> drivers could be utilized simultaneously on the system in runtime.
>> Opportunistically few devices would be controlled by the offload-aware
>> driver, whereas everything else by the offload-unaware one.  This
>> differs from existing HDAudio Controller driver situation where either
>> classic, snd_hda_intel driver takes complete control -or- the
>> offload-aware snd_soc_avs driver.
>> In short, once all Audio Sideband resources are depleted, classic
>> sound/usb/card.c driver manages whatever comes next:
>>
>> 	snd_usb_audio (offload un-aware, sound/usb/card.c)
>> 	snd_soc_usb_codec (offload aware, sound/soc/codecs/usb.c)
>>
>>
>> The design goals:
>> 	- make ASoC first class citizen of sound/usb
>> 	- re-use code found in sound/usb, mimic HDAudio integration in ASoC:
>> 	  small sound/soc/codecs/hda.c driver leveraging power of entire
>> 	  sound/pci/hda/
>> 	- no shared control over a USB device, either snd_usb_audio or
>> 	  its ASoC equivalent takes control of the device
>>
>> To do that, major tasks are identified:
>>
>> a) On ASoC side 'struct snd_card' is part of 'struct snd_soc_card' and
>> is managed by the framework.  Similar situation with 'struct snd_pcm'
>> and rtd->pcm.  To keep the teardown path sane, drop card->private_free()
>> and pcm->private_free() usage.
> 
> Well, this is a generic problem of ASoC framework.
> I believe this should be better handled in ASoC core side at first.
> e.g. the card object could be created at the very first step of the
> snd_soc_card creation, too (but without the actual slot assignment or
> device creation).


Well, in my opinion the card's tailing private context (extra size), 
private_free() and all that comes with them in sound/core brings 
unnecessary complexity to the ALSA framework.  devm_xxx() is enough. 
Polluting snd_card and sound/core/init.c with driver-specifics causes 
the teardown procedures on the ALSA framework side harder to read/maintain.

>> b) To initialize ASoC components/DAI properly, PCM capabilities should be
>> known up-front.  To do that, existing USB card probe() has to be split.
>>  From one-stage to two-stage process:
>>
>> 	- look ahead and parse usb_interface descriptors for PCM endpoints
>> 	  but do not create any PCMs (sound devices) yet
>> 	- create all PCMs based on obtained ->pcm_list and follow with
>> 	  MIDI/mixers/media
>>
>> Such approach allows to feed DAIs proper data even when a valid
>> sound-card pointer is not yet present - the initialization occurs before
>> snd_soc_bind_card() is called.
>   
> This one is another thing that is needed to adjust for ASoC
> framework.  But when snd_card object is available, this can be
> resolved automatically, too?  e.g. snd_pcm object or such can be
> created at that point.  The actual device registration is done anyway
> later via snd_device_register() call.


While I'm up for upgrading either framework in any area necessary to 
have proper UAO support in ASoC, not sure whether it's a good idea to 
make it a requirement for the feature.  This will enlarge the series - 
well, guess it will be separate series (dependency) entirely.

>> Point a) is scaled for all three "domains" of sound/usb: chip, stream
>> and quirks.  That's why there total of 6 commits doing that job.  First
>> implement, then switch.  Everything that follows is, in my opinion,
>> self-explanatory.  No need to repeat commit messages.
>>
>>
>> [1]: https://lore.kernel.org/all/20250319005141.312805-2-quic_wcheng@quicinc.com/
>> [2]: https://lore.kernel.org/all/20250319005141.312805-1-quic_wcheng@quicinc.com/
>> [3]: https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf