Re: [RFC 00/15] ALSA/ASoC: USB Audio Offload

[Takashi Iwai <tiwai@suse.de>]

On Wed, 09 Apr 2025 13:07:15 +0200,
Cezary Rojewski wrote:
> 
> Note: this series is based on Mark's broonie/for-next. The xHCI
> dependency is missing so it won't compile.  Goal of this RFC is
> discussing the direction of sound/usb changes.
> 
> Note #2: Why exclude xHCI?  Current form of xhci-sideband.c [1] does not
> fare well with Intel's hardware design for USB Audio Offload feature.
> The production shape for usb/xHCI subjects is being discussed with
> Mathias.  Once we're ready, I'll share the rest.
> 
> Note #3: this series does _NOT_ aim to block QCOM's equivalent series
> [2].  The team does acknowledge that we came the "table" late.  At the
> same time, we're prepared to help QCOM switch to the presented sound/usb
> approach if that would benefit the framework and its users as a whole.
> Make it part of this very series if need be.
> 
> 
> 
> Apart from changes to sound/usb, the patchset contains exemplary
> ASoC-based, offload-aware USB driver and a small sound card on the
> avs-driver side to show how card/dai_link initialization looks like.
> 
> In short, USB Audio Offload functionality lowers CPU usage when
> streaming PCM over a USB Audio-Class device.  It has been first
> introduced in xHCI 1.2 release and is described in section 7.9 [3].
> Once hardware is prepared with hw_params(), all the USB endpoints
> operations e.g.: start, stop, submitting URBs, are performed
> internally, by the hardware and AudioDSP firmware.  Software driver
> shall not intervene.
> 
> Startup flow from:
> 
> 	usb_pcm_open()
> 	usb_pcm_hw_params()
> 		snd_usb_endpoint_open()
> 	usb_pcm_prepare()
> 		usb_set_interface()
> 		snd_usb_endpoint_start()
> 	usb_pcm_trigger(cmd: START/STOP etc.)
> 
> reduced to:
> 
> 	usb_pcm_open()
> 	usb_pcm_hw_params()
> 		snd_usb_endpoint_open()
> 	usb_pcm_prepare()
> 		usb_set_interface()

Hmm, how can it be?  The start of EP at prepare stage is done only
conditionally for non-lowlatency or implicit-feedback mode, for
example.

> Handlers such as ack(), sync_stop(), pointer(), delay() are not used
> here too.  The AudioDSP driver will handle pointer(), rest is
> firmware/hardware responsibility.
> 
> 
> There's a hefty number of limitations, most importantly:
> 
> 1) typically 2 USB devices tops, rest go the classic (non-offload) path
> 2) AUDIOSTREAMING interfaces only, MIDI not.  While not an interface
>    type, Media (sound/usb/media) unsupported either
> 3) simple PCM, UAC_FORMAT_TYPE_I_PCM only
> 
> Current patchset shows this in form of 'udev->audsb_capable' field.
> True if xHCI sideband reasource has been assigned to the device.  The
> filtering is code is not part of the patchset.
> 
> Important to highlight, 1) means both, offload-aware and offload-unaware
> drivers could be utilized simultaneously on the system in runtime.
> Opportunistically few devices would be controlled by the offload-aware
> driver, whereas everything else by the offload-unaware one.  This
> differs from existing HDAudio Controller driver situation where either
> classic, snd_hda_intel driver takes complete control -or- the
> offload-aware snd_soc_avs driver.
> In short, once all Audio Sideband resources are depleted, classic
> sound/usb/card.c driver manages whatever comes next:
> 
> 	snd_usb_audio (offload un-aware, sound/usb/card.c)
> 	snd_soc_usb_codec (offload aware, sound/soc/codecs/usb.c)
> 
> 
> The design goals:
> 	- make ASoC first class citizen of sound/usb
> 	- re-use code found in sound/usb, mimic HDAudio integration in ASoC:
> 	  small sound/soc/codecs/hda.c driver leveraging power of entire
> 	  sound/pci/hda/
> 	- no shared control over a USB device, either snd_usb_audio or
> 	  its ASoC equivalent takes control of the device
> 
> To do that, major tasks are identified:
> 
> a) On ASoC side 'struct snd_card' is part of 'struct snd_soc_card' and
> is managed by the framework.  Similar situation with 'struct snd_pcm'
> and rtd->pcm.  To keep the teardown path sane, drop card->private_free()
> and pcm->private_free() usage.

Well, this is a generic problem of ASoC framework.
I believe this should be better handled in ASoC core side at first.
e.g. the card object could be created at the very first step of the
snd_soc_card creation, too (but without the actual slot assignment or
device creation).

> b) To initialize ASoC components/DAI properly, PCM capabilities should be
> known up-front.  To do that, existing USB card probe() has to be split.
> From one-stage to two-stage process:
> 
> 	- look ahead and parse usb_interface descriptors for PCM endpoints
> 	  but do not create any PCMs (sound devices) yet
> 	- create all PCMs based on obtained ->pcm_list and follow with
> 	  MIDI/mixers/media
> 
> Such approach allows to feed DAIs proper data even when a valid
> sound-card pointer is not yet present - the initialization occurs before
> snd_soc_bind_card() is called.
 
This one is another thing that is needed to adjust for ASoC
framework.  But when snd_card object is available, this can be
resolved automatically, too?  e.g. snd_pcm object or such can be
created at that point.  The actual device registration is done anyway
later via snd_device_register() call.


thanks,

Takashi

> Point a) is scaled for all three "domains" of sound/usb: chip, stream
> and quirks.  That's why there total of 6 commits doing that job.  First
> implement, then switch.  Everything that follows is, in my opinion,
> self-explanatory.  No need to repeat commit messages.
> 
> 
> [1]: https://lore.kernel.org/all/20250319005141.312805-2-quic_wcheng@quicinc.com/
> [2]: https://lore.kernel.org/all/20250319005141.312805-1-quic_wcheng@quicinc.com/
> [3]: https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf
> 
> 
> Cezary Rojewski (15):
>   ALSA: usb: Move media-filters to the media code
>   ALSA: usb: Drop private_free() usage for card and pcms
>   ALSA: usb: Relocate the usbaudio header file
>   ALSA: usb: Implement two-stage quirk applying mechanism
>   ALSA: usb: Implement two-stage stream creation mechanism
>   ALSA: usb: Implement two-stage chip probing mechanism
>   ALSA: usb: Switch to the two-stage chip probing
>   ALSA: usb: Switch to the two-stage stream creation
>   ALSA: usb: Switch to the two-stage quirk applying
>   ALSA: usb: Export PCM operations
>   ALSA: usb: Export usb_interface driver operations
>   ALSA: usb: Export card-naming procedure
>   ALSA: usb: Add getters to obtain endpoint information
>   ASoC: codecs: Add USB-Audio driver
>   ASoC: Intel: avs: Add USB machine board
> 
>  include/linux/usb.h                         |  15 +
>  include/linux/usb/ch9.h                     |  11 +
>  sound/usb/usbaudio.h => include/sound/usb.h |  53 +-
>  include/sound/usb_offload.h                 |  46 +
>  sound/soc/codecs/Kconfig                    |   6 +
>  sound/soc/codecs/Makefile                   |   2 +
>  sound/soc/codecs/usb.c                      | 441 +++++++++
>  sound/soc/intel/avs/boards/Kconfig          |   8 +
>  sound/soc/intel/avs/boards/Makefile         |   2 +
>  sound/soc/intel/avs/boards/usb.c            | 115 +++
>  sound/usb/caiaq/device.h                    |   2 +-
>  sound/usb/card.c                            | 946 +++++++++++++-------
>  sound/usb/card.h                            |   4 +-
>  sound/usb/clock.c                           |   2 +-
>  sound/usb/endpoint.c                        |   2 +-
>  sound/usb/format.c                          |   2 +-
>  sound/usb/helper.c                          |   2 +-
>  sound/usb/implicit.c                        |   2 +-
>  sound/usb/media.c                           |   8 +-
>  sound/usb/midi.c                            |   2 +-
>  sound/usb/midi.h                            |   2 +
>  sound/usb/midi2.c                           |   2 +-
>  sound/usb/midi2.h                           |   1 +
>  sound/usb/misc/ua101.c                      |   2 +-
>  sound/usb/mixer.c                           |   2 +-
>  sound/usb/mixer_quirks.c                    |   2 +-
>  sound/usb/mixer_s1810c.c                    |   2 +-
>  sound/usb/mixer_scarlett.c                  |   2 +-
>  sound/usb/mixer_scarlett2.c                 |   2 +-
>  sound/usb/mixer_us16x08.c                   |   2 +-
>  sound/usb/pcm.c                             | 161 +++-
>  sound/usb/power.c                           |   2 +-
>  sound/usb/proc.c                            |   2 +-
>  sound/usb/quirks.c                          | 203 +++--
>  sound/usb/quirks.h                          |   6 +
>  sound/usb/stream.c                          | 217 +++--
>  sound/usb/stream.h                          |   2 +
>  sound/usb/usx2y/us122l.c                    |   2 +-
>  sound/usb/usx2y/usX2Yhwdep.c                |   1 +
>  sound/usb/usx2y/usbusx2y.h                  |   2 +-
>  sound/usb/validate.c                        |   2 +-
>  41 files changed, 1749 insertions(+), 541 deletions(-)
>  rename sound/usb/usbaudio.h => include/sound/usb.h (82%)
>  create mode 100644 include/sound/usb_offload.h
>  create mode 100644 sound/soc/codecs/usb.c
>  create mode 100644 sound/soc/intel/avs/boards/usb.c
> 
> -- 
> 2.25.1
>