HDA HDMI PCM device allocation

HDA HDMI PCM device allocation

[Jaroslav Kysela]

Hi,

	I am trying to fix some issues in UCM for the HDA HDMI devices [1][2]. I 
would like to summary the current situation at first (correct me, if I miss 
something):

We have two methods to map the PINs in the HDA HDMI driver to the PCM devices 
(legacy/static - 1:1 mapping, dynamic - used for new devices with the MST 
capability). There is also set of converters in each HDMI codec and the number 
of simultaneously used PCM devices cannot go beyond this count of converters 
in hardware (otherwise -EBUSY error is returned). The count of converters is 3 
or 4 depending on the hardware.

Things to discuss:

It seems quite straight to limit the count of created PCMs to the count of 
converters. We cannot use more anyway and it does not help, if more PCM 
devices are allocated (and Jacks reported) to applications when they cannot be 
used simultaneously.

Legacy/static mapping should be converted to dynamic (unless the count of 
created PCM devices is equal to the count of codec converters).

There may be 1:1 mapping between the converter and the PCM device to make 
things easier.

There is a corner case, when more HDMI devices are connected than the count of 
converters. In this case, an extra method (a module parameter and/or a control 
element and/or procfs) may be used to filter unwanted HDMI devices. It may be 
a bit difficult to select the proper filtering key - it may be the PIN/MST 
device hash or so. The driver may report this key in eld#* files (procfs).

Impact to applications:

Those days, pulseaudio or pipewire servers are mostly used on the current 
hardware. Both servers share the legacy probe code for HDMI devices - they are 
trying to open PCM devices sequentially and check for the error code. There 
should not be a problem when the connected HDMI devices do not go beyond the 
count of converters. A minor issue is that the name of the used sink/port may 
be different (users may be forced to reselect the output path).

For other applications, the PCM device assigned to the connected HDMI device 
may be different (available in a different ALSA device name). I do not think 
that it's a big issue. It should be easy solvable with an updated software 
configuration.


Let me know about your opinion about this.

					Thank you,
						Jaroslav

[1] https://github.com/alsa-project/alsa-lib/issues/245
[2] https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2481

-- 
Jaroslav Kysela <perex@perex.cz>
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.

Re: HDA HDMI PCM device allocation

[Kai Vehmanen]

Hi,

On Mon, 19 Sep 2022, Jaroslav Kysela wrote:

> We have two methods to map the PINs in the HDA HDMI driver to the PCM devices
> (legacy/static - 1:1 mapping, dynamic - used for new devices with the MST
> capability). There is also set of converters in each HDMI codec and the number
> of simultaneously used PCM devices cannot go beyond this count of converters
> in hardware (otherwise -EBUSY error is returned). The count of converters is 3
> or 4 depending on the hardware.

roughly yes. There is some further details and variation in implementation 
the dynamic method if you look at patch_hdmi.c:generic_hdmi_build_pcms(). 
E.g. recently we added 'dyn_pcm_no_legacy' to start limiting the amount of 
PCM nodes created. But yeah, this is exactly to move to the direction you 
are proposing.

> Things to discuss:
> 
> It seems quite straight to limit the count of created PCMs to the count of
> converters. We cannot use more anyway and it does not help, if more PCM
> devices are allocated (and Jacks reported) to applications when they cannot be
> used simultaneously.

5~Agreed. When the amount of HDA pins (~= possible physical ports) was 
small, and before DP-MST (or when DP-MST was available only on a small 
subset of physical ports), there was clear value in having sticky 
PIN-to-PCMx mapping. If you plugged a monitor to certain HDMI or DP port, 
it would show up in the same PCM node. If you had a DP-MST hub, the 
monitor/receivers behind the hub, would also end up mapped to the 
familiar ALSA PCM nodes.

But especially with USB-C, the # of possible topologies has shot up 
(basicly any port can host a DP-MST hub), making this approach less and 
less practical.

With SOF, we had further constraints in integrating with ASoC, so we've 
basicly limited PCM (FE) nodes to number of converters (PCM BEs) from the 
start. Now with dyn_pcm_no_legacy, this starts to be used also with 
non-DSP usage via snd-hda-intel (with Intel Tiger Lake and newer). At 
least so far we've not got any negative feedback on this, so road seems 
clear to move ahead with this approach. There is certainly a lot of cruft 
in the code to maintain all the legacy options.

> There is a corner case, when more HDMI devices are connected than the count of
> converters. In this case, an extra method (a module parameter and/or a control
> element and/or procfs) may be used to filter unwanted HDMI devices. It may be

At least on Intel platforms this is not a problem. The number of 
converters is aligned with number of display pipes. So you'll never have 
more HDMI devices connected than the max number of converters.

> Impact to applications:
> 
> Those days, pulseaudio or pipewire servers are mostly used on the current
> hardware. Both servers share the legacy probe code for HDMI devices - they are
> trying to open PCM devices sequentially and check for the error code. There
5~> should not be a problem when the connected HDMI devices do not go 
beyond the
> count of converters. A minor issue is that the name of the used sink/port may
> be different (users may be forced to reselect the output path).
> 
> For other applications, the PCM device assigned to the connected HDMI device
> may be different (available in a different ALSA device name). I do not think
> that it's a big issue. It should be easy solvable with an updated software
> configuration.

Ack, and this model is already required for smooth integration. With 
snd-hda-intel, while PCM routing tries to maintain legacy PCM mapping, it 
cannot be guaranteed in all possible cases. On systems using SOF, there's 
no legacy mapping at all.

So the right (and robust) approach for apps to select the PCM for 
HDMI/DP audio is to use:
 - UCM tells which kcontrol to monitor for jack event
	- JackControl "HDMI/DP,pcm=3 Jack"
 - Jack control tells whether receiver is connected or not
	- "numid=22,iface=CARD,name='HDMI/DP,pcm=3 Jack'"
 - the ELD data describes receiver properties 
	- "numid=27,iface=PCM,name='ELD',device=3"

The above is done by Pulseaudio when UCM is used (and followed by 
Pipewire) and CRAS on ChromeOS. The above can give user sensible GUI 
information on where to route the audio, and provide enough tools for 
applications to provide persistancy (audio is routed to Foobar XYZ monitor 
always, indepedently of the order in which the display/receivers are 
discovered).

In simple setups (one HDMI/DP receiver with audio capability connected), 
the receiver is always connected to the first HDMI/DP PCM of the card.

Br, Kai

Re: HDA HDMI PCM device allocation

[Takashi Iwai]

Jaroslav,

thanks for bringing the issue.
I follow up onto Kai's reply.

On Mon, 19 Sep 2022 19:43:28 +0200,
Kai Vehmanen wrote:
> 
> Hi,
> 
> On Mon, 19 Sep 2022, Jaroslav Kysela wrote:
> 
> > We have two methods to map the PINs in the HDA HDMI driver to the PCM devices
> > (legacy/static - 1:1 mapping, dynamic - used for new devices with the MST
> > capability). There is also set of converters in each HDMI codec and the number
> > of simultaneously used PCM devices cannot go beyond this count of converters
> > in hardware (otherwise -EBUSY error is returned). The count of converters is 3
> > or 4 depending on the hardware.
> 
> roughly yes. There is some further details and variation in implementation 
> the dynamic method if you look at patch_hdmi.c:generic_hdmi_build_pcms(). 
> E.g. recently we added 'dyn_pcm_no_legacy' to start limiting the amount of 
> PCM nodes created. But yeah, this is exactly to move to the direction you 
> are proposing.
> 
> > Things to discuss:
> > 
> > It seems quite straight to limit the count of created PCMs to the count of
> > converters. We cannot use more anyway and it does not help, if more PCM
> > devices are allocated (and Jacks reported) to applications when they cannot be
> > used simultaneously.
> 
> 5~Agreed. When the amount of HDA pins (~= possible physical ports) was 
> small, and before DP-MST (or when DP-MST was available only on a small 
> subset of physical ports), there was clear value in having sticky 
> PIN-to-PCMx mapping. If you plugged a monitor to certain HDMI or DP port, 
> it would show up in the same PCM node. If you had a DP-MST hub, the 
> monitor/receivers behind the hub, would also end up mapped to the 
> familiar ALSA PCM nodes.
> 
> But especially with USB-C, the # of possible topologies has shot up 
> (basicly any port can host a DP-MST hub), making this approach less and 
> less practical.
> 
> With SOF, we had further constraints in integrating with ASoC, so we've 
> basicly limited PCM (FE) nodes to number of converters (PCM BEs) from the 
> start. Now with dyn_pcm_no_legacy, this starts to be used also with 
> non-DSP usage via snd-hda-intel (with Intel Tiger Lake and newer). At 
> least so far we've not got any negative feedback on this, so road seems 
> clear to move ahead with this approach. There is certainly a lot of cruft 
> in the code to maintain all the legacy options.

Yes, and I thought this flag should suffice and provide the dynamic
assignment feature, if it's set to every new model in future as
default (at least about the converters)?

> > There is a corner case, when more HDMI devices are connected than the count of
> > converters. In this case, an extra method (a module parameter and/or a control
> > element and/or procfs) may be used to filter unwanted HDMI devices. It may be
> 
> At least on Intel platforms this is not a problem. The number of 
> converters is aligned with number of display pipes. So you'll never have 
> more HDMI devices connected than the max number of converters.

I guess it's true in general that a converter corresponds to an
output for all graphics implementations, so far.  Of course, it's not
set on stone, so things may change.  But I'm afraid that it'll be
beyond the HD-audio codec itself.


> > Impact to applications:
> > 
> > Those days, pulseaudio or pipewire servers are mostly used on the current
> > hardware. Both servers share the legacy probe code for HDMI devices - they are
> > trying to open PCM devices sequentially and check for the error code. There
> 5~> should not be a problem when the connected HDMI devices do not go 
> beyond the
> > count of converters. A minor issue is that the name of the used sink/port may
> > be different (users may be forced to reselect the output path).
> > 
> > For other applications, the PCM device assigned to the connected HDMI device
> > may be different (available in a different ALSA device name). I do not think
> > that it's a big issue. It should be easy solvable with an updated software
> > configuration.
> 
> Ack, and this model is already required for smooth integration. With 
> snd-hda-intel, while PCM routing tries to maintain legacy PCM mapping, it 
> cannot be guaranteed in all possible cases. On systems using SOF, there's 
> no legacy mapping at all.
> 
> So the right (and robust) approach for apps to select the PCM for 
> HDMI/DP audio is to use:
>  - UCM tells which kcontrol to monitor for jack event
> 	- JackControl "HDMI/DP,pcm=3 Jack"
>  - Jack control tells whether receiver is connected or not
> 	- "numid=22,iface=CARD,name='HDMI/DP,pcm=3 Jack'"
>  - the ELD data describes receiver properties 
> 	- "numid=27,iface=PCM,name='ELD',device=3"
> 
> The above is done by Pulseaudio when UCM is used (and followed by 
> Pipewire) and CRAS on ChromeOS. The above can give user sensible GUI 
> information on where to route the audio, and provide enough tools for 
> applications to provide persistancy (audio is routed to Foobar XYZ monitor 
> always, indepedently of the order in which the display/receivers are 
> discovered).
> 
> In simple setups (one HDMI/DP receiver with audio capability connected), 
> the receiver is always connected to the first HDMI/DP PCM of the card.

Sounds reasonable to me.


thanks,

Takashi