Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Clark, Rob]

On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
>> Hello!
>>
>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
>> propagate events from the HDMI output (e.g., display getting
>> enabled/disabled/suspended). This for the users of the driver to react
>> to such events. For instance, if the display is disabled or disconected,
>> audio could be stopped, rerouted or whatever other decision the user
>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
>> audio will stall and the audio users will only see a "playback write
>> error (DMA or IRQ trouble?)"
>>
>> In my prototypes I have used snd_soc_jack for this purpose and I have
>> some questions:
>>
>> *I see snd_soc_jack is used mostly for headsets and microphones with
>> actual external mechanical connections. Strictly, in my case I propagate
>> events originated by the OMAP display driver (changes in the power
>> state), and not from external events. Some of these events are generated
>> from an actual HDMI cable connection/disconnection, though.
>>
>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
>> entity in charge of the audio policy should listen those events instead.
>>
>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is
>> feasible for an audio driver to report events from an AV output.
>>
>> I was wondering about how much sense does it make to you guys use a
>> snd_soc_jack in this case?
>
> How does DRM handle audio? I made a quick grep, but I see the drm
> drivers only enabling the audio in the HW, nothing else.

I confess to not knowing too much about audio/alsa, but what does
audio driver need from hdmi?  Just hotplug events?

>From a quick look, it seems most of what the existing drm drivers are
doing is detecting if the display supports audio, and then turning
on/off the hw.. (and some infoframe stuff in some cases).

Does ASoC support 'hotplug' of audio devices?  If so, maybe it makes
some sense to have some support in drm core.  At least all the edid
parsing stuff to determine if the display supports audio should be
generic and not driver specific.

BR,
-R

> If there's a common generic way to handle this, we should obviously use
> that. But if we need to choose between doing something custom or doing
> it in omapdrm driver, I think we should go for drm the only solution and
> forget about audio with omapfb.
>
>  Tomi
>

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 498 bytes --]

On Tue, Aug 21, 2012 at 07:39:55AM -0500, Clark, Rob wrote:

> Does ASoC support 'hotplug' of audio devices?  If so, maybe it makes
> some sense to have some support in drm core.  At least all the edid
> parsing stuff to determine if the display supports audio should be
> generic and not driver specific.

Not really (except on a card level), and it'd probably confuse most
userspaces.  What I'd expect to happen currently is that the current
state of the connector would get reported via extcon.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Ricardo Neri]

On 08/21/2012 07:39 AM, Clark, Rob wrote:
> On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
>> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
>>> Hello!
>>>
>>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
>>> propagate events from the HDMI output (e.g., display getting
>>> enabled/disabled/suspended). This for the users of the driver to react
>>> to such events. For instance, if the display is disabled or disconected,
>>> audio could be stopped, rerouted or whatever other decision the user
>>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
>>> audio will stall and the audio users will only see a "playback write
>>> error (DMA or IRQ trouble?)"
>>>
>>> In my prototypes I have used snd_soc_jack for this purpose and I have
>>> some questions:
>>>
>>> *I see snd_soc_jack is used mostly for headsets and microphones with
>>> actual external mechanical connections. Strictly, in my case I propagate
>>> events originated by the OMAP display driver (changes in the power
>>> state), and not from external events. Some of these events are generated
>>> from an actual HDMI cable connection/disconnection, though.
>>>
>>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
>>> entity in charge of the audio policy should listen those events instead.
>>>
>>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is
>>> feasible for an audio driver to report events from an AV output.
>>>
>>> I was wondering about how much sense does it make to you guys use a
>>> snd_soc_jack in this case?
>>
>> How does DRM handle audio? I made a quick grep, but I see the drm
>> drivers only enabling the audio in the HW, nothing else.
>
> I confess to not knowing too much about audio/alsa, but what does
> audio driver need from hdmi?  Just hotplug events?

At least for the case of the ASoC HDMI audio driver (but hopefully for 
other audio drivers as well), it needs to detect whether an HDMI output 
is available, if the display's current configuration supports audio 
(e.g., a 1080p display configured as VGA should be reported as not 
supporting audio). It also needs interfaces to 
configure/prepare/start/stop audio. Also, of course, needs to know if 
the display is off/on/suspended and when transitions occur. For OMAP, 
omapdss provide an interface for this functionality for ALSA or any 
other interested user.
>
>  From a quick look, it seems most of what the existing drm drivers are
> doing is detecting if the display supports audio, and then turning
> on/off the hw.. (and some infoframe stuff in some cases).

Yes, it seems to me that every driver makes its own audio 
implementation, mainly focused on configuration. I could not find any 
audio common interface so that users like ALSA can take advantage of.

Also, I could not see any ALSA driver using functionality provided by a 
drm driver.

Maybe the lack of audio support in drm is because the audio users should 
not talk to drm directly but to a lower level component (omapdrm, 
omapdss?). However, today there exists video technology supports audio 
as well, such as DisplayPort or HDMI. Could it make more sense now to 
provide audio support?

>
> Does ASoC support 'hotplug' of audio devices?  If so, maybe it makes
> some sense to have some support in drm core.  At least all the edid
> parsing stuff to determine if the display supports audio should be
> generic and not driver specific.

drm already supports generic edid parsing to check for audio support. 
This is a good example of how an audio driver could just use that 
functionality to probe audio support.

Ricardo
>
> BR,
> -R
>
>> If there's a common generic way to handle this, we should obviously use
>> that. But if we need to choose between doing something custom or doing
>> it in omapdrm driver, I think we should go for drm the only solution and
>> forget about audio with omapfb.
>>
>>   Tomi
>>

Re: [alsa-devel] [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Takashi Iwai]

At Tue, 21 Aug 2012 19:58:02 -0500,
Ricardo Neri wrote:
> 
> 
> 
> On 08/21/2012 07:39 AM, Clark, Rob wrote:
> > On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
> >> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
> >>> Hello!
> >>>
> >>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
> >>> propagate events from the HDMI output (e.g., display getting
> >>> enabled/disabled/suspended). This for the users of the driver to react
> >>> to such events. For instance, if the display is disabled or disconected,
> >>> audio could be stopped, rerouted or whatever other decision the user
> >>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
> >>> audio will stall and the audio users will only see a "playback write
> >>> error (DMA or IRQ trouble?)"
> >>>
> >>> In my prototypes I have used snd_soc_jack for this purpose and I have
> >>> some questions:
> >>>
> >>> *I see snd_soc_jack is used mostly for headsets and microphones with
> >>> actual external mechanical connections. Strictly, in my case I propagate
> >>> events originated by the OMAP display driver (changes in the power
> >>> state), and not from external events. Some of these events are generated
> >>> from an actual HDMI cable connection/disconnection, though.
> >>>
> >>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
> >>> entity in charge of the audio policy should listen those events instead.
> >>>
> >>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is
> >>> feasible for an audio driver to report events from an AV output.
> >>>
> >>> I was wondering about how much sense does it make to you guys use a
> >>> snd_soc_jack in this case?
> >>
> >> How does DRM handle audio? I made a quick grep, but I see the drm
> >> drivers only enabling the audio in the HW, nothing else.
> >
> > I confess to not knowing too much about audio/alsa, but what does
> > audio driver need from hdmi?  Just hotplug events?
> 
> At least for the case of the ASoC HDMI audio driver (but hopefully for 
> other audio drivers as well), it needs to detect whether an HDMI output 
> is available, if the display's current configuration supports audio 
> (e.g., a 1080p display configured as VGA should be reported as not 
> supporting audio). It also needs interfaces to 
> configure/prepare/start/stop audio. Also, of course, needs to know if 
> the display is off/on/suspended and when transitions occur. For OMAP, 
> omapdss provide an interface for this functionality for ALSA or any 
> other interested user.
> >
> >  From a quick look, it seems most of what the existing drm drivers are
> > doing is detecting if the display supports audio, and then turning
> > on/off the hw.. (and some infoframe stuff in some cases).
> 
> Yes, it seems to me that every driver makes its own audio 
> implementation, mainly focused on configuration. I could not find any 
> audio common interface so that users like ALSA can take advantage of.
> 
> Also, I could not see any ALSA driver using functionality provided by a 
> drm driver.
> 
> Maybe the lack of audio support in drm is because the audio users should 
> not talk to drm directly but to a lower level component (omapdrm, 
> omapdss?). However, today there exists video technology supports audio 
> as well, such as DisplayPort or HDMI. Could it make more sense now to 
> provide audio support?

The reason is that the audio and video handling are already separated
in the hardware level, at least, for desktop graphics.

The audio infoframe is passed via ELD to the audio controller part
upon plug/unplugging via HD-audio unsolicited event, and the audio
driver sets up the stuff according to the given ELD.  Thus no extra
interface between drm and ALSA was required in the kernel API level,
so far.


Takashi

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Ricardo Neri]

Hi Takashi,

On 08/22/2012 02:55 AM, Takashi Iwai wrote:
> At Tue, 21 Aug 2012 19:58:02 -0500,
> Ricardo Neri wrote:
>>
>>
>>
>> On 08/21/2012 07:39 AM, Clark, Rob wrote:
>>> On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
>>>> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
>>>>> Hello!
>>>>>
>>>>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
>>>>> propagate events from the HDMI output (e.g., display getting
>>>>> enabled/disabled/suspended). This for the users of the driver to react
>>>>> to such events. For instance, if the display is disabled or disconected,
>>>>> audio could be stopped, rerouted or whatever other decision the user
>>>>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
>>>>> audio will stall and the audio users will only see a "playback write
>>>>> error (DMA or IRQ trouble?)"
>>>>>
>>>>> In my prototypes I have used snd_soc_jack for this purpose and I have
>>>>> some questions:
>>>>>
>>>>> *I see snd_soc_jack is used mostly for headsets and microphones with
>>>>> actual external mechanical connections. Strictly, in my case I propagate
>>>>> events originated by the OMAP display driver (changes in the power
>>>>> state), and not from external events. Some of these events are generated
>>>>> from an actual HDMI cable connection/disconnection, though.
>>>>>
>>>>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
>>>>> entity in charge of the audio policy should listen those events instead.
>>>>>
>>>>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it ie · · Il y a 12 minutes ·

s
>>>>> feasible for an audio driver to report events from an AV output.
>>>>>
>>>>> I was wondering about how much sense does it make to you guys use a
>>>>> snd_soc_jack in this case?
>>>>
>>>> How does DRM handle audio? I made a quick grep, but I see the drm
>>>> drivers only enabling the audio in the HW, nothing else.
>>>
>>> I confess to not knowing too much about audio/alsa, but what does
>>> audio driver need from hdmi?  Just hotplug events?
>>
>> At least for the case of the ASoC HDMI audio driver (but hopefully for
>> other audio drivers as well), it needs to detect whether an HDMI output
>> is available, if the display's current configuration supports audio
>> (e.g., a 1080p display configured as VGA should be reported as not
>> supporting audio). It also needs interfaces to
>> configure/prepare/start/stop audio. Also, of course, needs to know if
>> the display is off/on/suspended and when transitions occur. For OMAP,
>> omapdss provide an interface for this functionality for ALSA or any
>> other interested user.
>>>
>>>   From a quick look, it seems most of what the existing drm drivers are
>>> doing is detecting if the display supports audio, and then turning
>>> on/off the hw.. (and some infoframe stuff in some cases).
>>
>> Yes, it seems to me that every driver makes its own audio
>> implementation, mainly focused on configuration. I could not find any
>> audio common interface so that users like ALSA can take advantage of.
>>
>> Also, I could not see any ALSA driver using functionality provided by a
>> drm driver.
>>
>> Maybe the lack of audio support in drm is because the audio users should
>> not talk to drm directly but to a lower level component (omapdrm,
>> omapdss?). However, today there exists video technology supports audio
>> as well, such as DisplayPort or HDMI. Could it make more sense now to
>> provide audio support?
>
> The reason is that the audio and video handling are already separated
> in the hardware level, at least, for desktop graphics.

Separated in what sense? Do they have separate register banks in 
independent power domains? Can audio an video work with complete 
independence. What happens if someone decides to power off video. Is the 
audio able to continue if required?
>
> The audio infoframe is passed via ELD to the audio controller part
> upon plug/unplugging via HD-audio unsolicited event, and the audio
> driver sets up the stuff according to the given ELD.  Thus no extra
> interface between drm and ALSA was required in the kernel API level,
> so far.

I see that the unsolicited event is used only to parse the EDID, 
correct? It also notifies about the jack status. Hence, if there the 
cable is disconnected the application will know and act accordingly. Is 
this understanding correct?

Also, I see that hda_pcm_ops does not have a trigger or start/stop 
functions, how does it know when to start/stop audio. Maybe with 
azx_pcm_trigger?

Sorry, I am not expert in HD-audio :)

Ricardo

>
>
> Takashi
>

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Stephen Warren]

On 08/23/2012 07:44 PM, Ricardo Neri wrote:
> On 08/22/2012 02:55 AM, Takashi Iwai wrote:
>> At Tue, 21 Aug 2012 19:58:02 -0500,
>> Ricardo Neri wrote:
...
>>> Maybe the lack of audio support in drm is because the audio users should
>>> not talk to drm directly but to a lower level component (omapdrm,
>>> omapdss?). However, today there exists video technology supports audio
>>> as well, such as DisplayPort or HDMI. Could it make more sense now to
>>> provide audio support?
>>
>> The reason is that the audio and video handling are already separated
>> in the hardware level, at least, for desktop graphics.
> 
> Separated in what sense? Do they have separate register banks in

For NVIDIA desktop GPUs, this is certainly true, and I think so for any
Intel Azalia/HDA controller. The separate register banks are in
different PCI functions on the chip. The Intel Azalia/HDA spec also
architects specific ways that the audio and video parts interact (i.e.
ELD representation of EDID data, unsolicited response messages when the
video state changes, etc.) Oh, I see Takashi mentioned this below.

> independent power domains?

Most likely yes.

> Can audio an video work with complete
> independence. What happens if someone decides to power off video. Is the
> audio able to continue if required?

I believe audio DMA isn't affect by the video state, but I'm not 100%
sure of that.

>> The audio infoframe is passed via ELD to the audio controller part
>> upon plug/unplugging via HD-audio unsolicited event, and the audio
>> driver sets up the stuff according to the given ELD.  Thus no extra
>> interface between drm and ALSA was required in the kernel API level,
>> so far.
> 
> I see that the unsolicited event is used only to parse the EDID,
> correct? It also notifies about the jack status. Hence, if there the
> cable is disconnected the application will know and act accordingly. Is
> this understanding correct?

The kernel will know, and then passes the information on to user-space.

Re: [alsa-devel] [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Takashi Iwai]

At Thu, 23 Aug 2012 20:44:32 -0500,
Ricardo Neri wrote:
> 
> Hi Takashi,
> 
> On 08/22/2012 02:55 AM, Takashi Iwai wrote:
> > At Tue, 21 Aug 2012 19:58:02 -0500,
> > Ricardo Neri wrote:
> >>
> >>
> >>
> >> On 08/21/2012 07:39 AM, Clark, Rob wrote:
> >>> On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
> >>>> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
> >>>>> Hello!
> >>>>>
> >>>>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
> >>>>> propagate events from the HDMI output (e.g., display getting
> >>>>> enabled/disabled/suspended). This for the users of the driver to react
> >>>>> to such events. For instance, if the display is disabled or disconected,
> >>>>> audio could be stopped, rerouted or whatever other decision the user
> >>>>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
> >>>>> audio will stall and the audio users will only see a "playback write
> >>>>> error (DMA or IRQ trouble?)"
> >>>>>
> >>>>> In my prototypes I have used snd_soc_jack for this purpose and I have
> >>>>> some questions:
> >>>>>
> >>>>> *I see snd_soc_jack is used mostly for headsets and microphones with
> >>>>> actual external mechanical connections. Strictly, in my case I propagate
> >>>>> events originated by the OMAP display driver (changes in the power
> >>>>> state), and not from external events. Some of these events are generated
> >>>>> from an actual HDMI cable connection/disconnection, though.
> >>>>>
> >>>>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
> >>>>> entity in charge of the audio policy should listen those events instead.
> >>>>>
> >>>>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it ie · · Il y a 12 minutes ·
> 
> s
> >>>>> feasible for an audio driver to report events from an AV output.
> >>>>>
> >>>>> I was wondering about how much sense does it make to you guys use a
> >>>>> snd_soc_jack in this case?
> >>>>
> >>>> How does DRM handle audio? I made a quick grep, but I see the drm
> >>>> drivers only enabling the audio in the HW, nothing else.
> >>>
> >>> I confess to not knowing too much about audio/alsa, but what does
> >>> audio driver need from hdmi?  Just hotplug events?
> >>
> >> At least for the case of the ASoC HDMI audio driver (but hopefully for
> >> other audio drivers as well), it needs to detect whether an HDMI output
> >> is available, if the display's current configuration supports audio
> >> (e.g., a 1080p display configured as VGA should be reported as not
> >> supporting audio). It also needs interfaces to
> >> configure/prepare/start/stop audio. Also, of course, needs to know if
> >> the display is off/on/suspended and when transitions occur. For OMAP,
> >> omapdss provide an interface for this functionality for ALSA or any
> >> other interested user.
> >>>
> >>>   From a quick look, it seems most of what the existing drm drivers are
> >>> doing is detecting if the display supports audio, and then turning
> >>> on/off the hw.. (and some infoframe stuff in some cases).
> >>
> >> Yes, it seems to me that every driver makes its own audio
> >> implementation, mainly focused on configuration. I could not find any
> >> audio common interface so that users like ALSA can take advantage of.
> >>
> >> Also, I could not see any ALSA driver using functionality provided by a
> >> drm driver.
> >>
> >> Maybe the lack of audio support in drm is because the audio users should
> >> not talk to drm directly but to a lower level component (omapdrm,
> >> omapdss?). However, today there exists video technology supports audio
> >> as well, such as DisplayPort or HDMI. Could it make more sense now to
> >> provide audio support?
> >
> > The reason is that the audio and video handling are already separated
> > in the hardware level, at least, for desktop graphics.
> 
> Separated in what sense? Do they have separate register banks in 
> independent power domains? Can audio an video work with complete 
> independence. What happens if someone decides to power off video. Is the 
> audio able to continue if required?

As Stephen already pointed, the functionality is independent in the
PCI hardware level.  The audio part even accepts and DMA is working
without connection to the actual output.

> > The audio infoframe is passed via ELD to the audio controller part
> > upon plug/unplugging via HD-audio unsolicited event, and the audio
> > driver sets up the stuff according to the given ELD.  Thus no extra
> > interface between drm and ALSA was required in the kernel API level,
> > so far.
> 
> I see that the unsolicited event is used only to parse the EDID, 
> correct? It also notifies about the jack status. Hence, if there the 
> cable is disconnected the application will know and act accordingly. Is 
> this understanding correct?

Correct.

> Also, I see that hda_pcm_ops does not have a trigger or start/stop 
> functions, how does it know when to start/stop audio. Maybe with 
> azx_pcm_trigger?

The start/stop isn't issued in the driver level because DMA is still
working.  For HD-audio, it's really just like a normal jack
plug/unplug.


Takashi
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html