Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Clemens Ladisch]

Laurent Pinchart wrote:
> A link is a point-to-point oriented connection between two pads, either
> on the same entity or on different entities. Data flows from a source
> pad to a sink pad.
> 
> Links are stored in the source entity.

In the descriptors of USB Audio and HDAudio devices, the links are
stored only in the sink.  But AFAICS this doesn't matter in the media
driver API.

> +Links have flags that describe the link capabilities and state.
> +
> +	MEDIA_LINK_FLAG_ACTIVE indicates that the link is active and can be
> +	used to transfer media data. When two or more links target a sink pad,
> +	only one of them can be active at a time.
> +	MEDIA_LINK_FLAG_IMMUTABLE indicates that the link active state can't
> +	be modified at runtime. If MEDIA_LINK_FLAG_IMMUTABLE is set, then
> +	MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is
> +	always active.

So the intended userspace API for controlling routing is to change the
ACTIVE flag of links?

In USB and HD audio devices, all links are immutable, and the routing
is controlled by 'selector' entities that activate exactly one of their
input pads.  In userspace, this entity shows up as a mixer control.
I guess it would be possible to map the ACTIVE flag onto these controls.

Alternatively, entities can have 'mute' mixer controls associated with
their pads.  In this case, multiple unmuted inputs would be mixed
together.

> +#define MEDIA_ENTITY_TYPE_MASK               0x00ff0000
> +#define MEDIA_ENTITY_SUBTYPE_MASK            0x0000ffff
> +
> +#define MEDIA_ENTITY_TYPE_NODE               (1 << MEDIA_ENTITY_TYPE_SHIFT)
> ...
> +#define MEDIA_ENTITY_TYPE_NODE_ALSA          (MEDIA_ENTITY_TYPE_NODE + 3)

ALSA has PCM and MIDI devices, and several types of mixer controls.
(It also has hardware dependent and timer devices, but I don't think
these would need topology information.)  So we need at least these:
  MEDIA_ENTITY_TYPE_NODE_ALSA_PCM
  MEDIA_ENTITY_TYPE_NODE_ALSA_MIDI
  MEDIA_ENTITY_TYPE_SUBDEV_ALSA_CONTROL

Furthermore, topology information is also needed for entities not
associated with a mixer control, such as microphones, speakers, jacks/
connectors, and effect units.  These entities are defined in the USB and
HD audio specifications, but are not yet handled by ALSA.

> +struct media_entity {
> ...
> +	union {
> +		/* Node specifications */
> +		struct {
> +			u32 major;
> +			u32 minor;
> +		} v4l;
> +		struct {
> +			u32 major;
> +			u32 minor;
> +		} fb;
> +		int alsa;
> +		int dvb;
> +
> +		/* Sub-device specifications */
> +		/* Nothing needed yet */
> +	};

ALSA devices are not addressed by their device node but with card/device/
subdevice numbers; mixer controls have numeric IDs, unique per card:

		struct {
			int card;
			int device;
			int subdevice;
		} alsa_device;
		struct {
			int card;
			int numid;
		} alsa_control;


Regards,
Clemens

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Mark Brown]

On Thu, Nov 25, 2010 at 10:38:05AM +0100, Clemens Ladisch wrote:

> In USB and HD audio devices, all links are immutable, and the routing
> is controlled by 'selector' entities that activate exactly one of their
> input pads.  In userspace, this entity shows up as a mixer control.
> I guess it would be possible to map the ACTIVE flag onto these controls.

Ditto for ASoC, mostly.

> Alternatively, entities can have 'mute' mixer controls associated with
> their pads.  In this case, multiple unmuted inputs would be mixed
> together.

> ALSA has PCM and MIDI devices, and several types of mixer controls.
> (It also has hardware dependent and timer devices, but I don't think
> these would need topology information.)  So we need at least these:
>   MEDIA_ENTITY_TYPE_NODE_ALSA_PCM
>   MEDIA_ENTITY_TYPE_NODE_ALSA_MIDI
>   MEDIA_ENTITY_TYPE_SUBDEV_ALSA_CONTROL

> Furthermore, topology information is also needed for entities not
> associated with a mixer control, such as microphones, speakers, jacks/
> connectors, and effect units.  These entities are defined in the USB and
> HD audio specifications, but are not yet handled by ALSA.

All this and more in the embedded case - digital audio link nodes and DSP
I/O nodes (could possibly do those as digital audio ones) spring to
mind.  Also bear in mind that embedded devices can get *very* large - a
mobile phone audio system can have of the order of 100 nodes in the
graph.

> ALSA devices are not addressed by their device node but with card/device/
> subdevice numbers; mixer controls have numeric IDs, unique per card:
> 
> 		struct {
> 			int card;
> 			int device;
> 			int subdevice;
> 		} alsa_device;
> 		struct {
> 			int card;
> 			int numid;
> 		} alsa_control;

For the embedded stuff we also have a bunch of stuff in the graph which
may not be visible to userspace at all at present and would just have a
string based identifier.

Re: [RFC/PATCH v6 03/12] [alsa-devel] media: Entities, pads and links

[Laurent Pinchart]

Hi Mark,

On Thursday 25 November 2010 14:41:35 Mark Brown wrote:
> On Thu, Nov 25, 2010 at 10:38:05AM +0100, Clemens Ladisch wrote:
> > In USB and HD audio devices, all links are immutable, and the routing
> > is controlled by 'selector' entities that activate exactly one of their
> > input pads.  In userspace, this entity shows up as a mixer control.
> > I guess it would be possible to map the ACTIVE flag onto these controls.
> 
> Ditto for ASoC, mostly.
> 
> > Alternatively, entities can have 'mute' mixer controls associated with
> > their pads.  In this case, multiple unmuted inputs would be mixed
> > together.
> > 
> > ALSA has PCM and MIDI devices, and several types of mixer controls.
> > (It also has hardware dependent and timer devices, but I don't think
> > 
> > these would need topology information.)  So we need at least these:
> >   MEDIA_ENTITY_TYPE_NODE_ALSA_PCM
> >   MEDIA_ENTITY_TYPE_NODE_ALSA_MIDI
> >   MEDIA_ENTITY_TYPE_SUBDEV_ALSA_CONTROL
> > 
> > Furthermore, topology information is also needed for entities not
> > associated with a mixer control, such as microphones, speakers, jacks/
> > connectors, and effect units.  These entities are defined in the USB and
> > HD audio specifications, but are not yet handled by ALSA.
> 
> All this and more in the embedded case - digital audio link nodes and DSP
> I/O nodes (could possibly do those as digital audio ones) spring to
> mind.  Also bear in mind that embedded devices can get *very* large - a
> mobile phone audio system can have of the order of 100 nodes in the
> graph.

It depends on how you define nodes. I can certainly imagine a graph with 100 
controls, but maybe several controls can be part of the same node ? On the 
video side we've decided to split entities depending on the possible data 
paths configurations. As I'm not a fluent ascii-art speaker, please have a 
look at pages 4 and 5 of http://www.ideasonboard.org/media/20101103-lpc-
media.pdf

Page 4 shows the internal topology of the OMAP3 ISP. The major blocks in that 
diagram are reported as entities. Page 5 shows the internal topology of one of 
the blocks, the OMAP3 ISP preview engine. As you can see the pipeline is made 
of sub-blocks that implement a single image processing function. As the 
pipeline is linear (don't worry about the non-linear part in the beginning, 
it's just there to take into account link configurability at the higher level) 
we don't export all the sub-blocks as entities, but we expose the controls on 
the preview engine entity instead.

> > ALSA devices are not addressed by their device node but with card/device/
> > 
> > subdevice numbers; mixer controls have numeric IDs, unique per card:
> > 		struct {
> > 		
> > 			int card;
> > 			int device;
> > 			int subdevice;
> > 		
> > 		} alsa_device;
> > 		struct {
> > 		
> > 			int card;
> > 			int numid;
> > 		
> > 		} alsa_control;
> 
> For the embedded stuff we also have a bunch of stuff in the graph which
> may not be visible to userspace at all at present and would just have a
> string based identifier.

That could be easily added (provided the string is not too long).

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Mark Brown]

On Thu, Nov 25, 2010 at 04:29:53PM +0100, Laurent Pinchart wrote:

> It depends on how you define nodes. I can certainly imagine a graph with 100 
> controls, but maybe several controls can be part of the same node ? On the 
> video side we've decided to split entities depending on the possible data 
> paths configurations. As I'm not a fluent ascii-art speaker, please have a 
> look at pages 4 and 5 of http://www.ideasonboard.org/media/20101103-lpc-
> media.pdf

Please take a look at:

	http://www.wolfsonmicro.com/products/WM8994

for an example of a modern smartphone CODEC.  This has in the ballpark
of 75-100 nodes (I've not counted exactly) represented in the routing
graph in the driver today.  This does not include any audio routing
available in other components such as the CPU.

Re: [RFC/PATCH v6 03/12] [alsa-devel] media: Entities, pads and links

[Laurent Pinchart]

Hi Clemens,

Thanks a lot for the review.

On Thursday 25 November 2010 10:38:05 Clemens Ladisch wrote:
> Laurent Pinchart wrote:
> > A link is a point-to-point oriented connection between two pads, either
> > on the same entity or on different entities. Data flows from a source
> > pad to a sink pad.
> > 
> > Links are stored in the source entity.
> 
> In the descriptors of USB Audio and HDAudio devices, the links are
> stored only in the sink.  But AFAICS this doesn't matter in the media
> driver API.

Same for USB Video (UVC). The reason for that is that UAC and UVC have only 1-
to-many connections (same for the media controller by the way), and it's 
easier to store that information in fixed-size structures if you store it in 
the sink entities.

On the kernel side, the media controller stores links in both source and sink 
entities for internal reasons. Links stored in the sink entity are called 
backlinks.

On the userspace side applications will only see "forward" links, stored in 
source entities. Applications will of course be free to store the links 
whereever they want in their internal data structures.

As you mentioned, I think it doesn't matter much anyway.

> > +Links have flags that describe the link capabilities and state.
> > +
> > +	MEDIA_LINK_FLAG_ACTIVE indicates that the link is active and can be
> > +	used to transfer media data. When two or more links target a sink pad,
> > +	only one of them can be active at a time.
> > +	MEDIA_LINK_FLAG_IMMUTABLE indicates that the link active state can't
> > +	be modified at runtime. If MEDIA_LINK_FLAG_IMMUTABLE is set, then
> > +	MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is
> > +	always active.
> 
> So the intended userspace API for controlling routing is to change the
> ACTIVE flag of links?

That's correct.

> In USB and HD audio devices, all links are immutable, and the routing
> is controlled by 'selector' entities that activate exactly one of their
> input pads.  In userspace, this entity shows up as a mixer control.
> I guess it would be possible to map the ACTIVE flag onto these controls.

Same for UVC once again. It would be quite easy to map selector units to link 
activation, but it doesn't even have to be done. We could keep the selector 
units in the graph with all links immutable, and use the existing APIs to 
control the selector units.

The alternative would have been to add selector units to the media controller. 
I've thought about it, but we have embedded hardware (at least on the video 
side) that have configurable links without selector units. The media 
controller would have had to create fake selector units.

Between creating fake selector and translating link activation to selector 
unit commands, I've decided to go for the second solution, as this will make 
the graph simpler. Once again we can decide to keep explicit selector units in 
the graph for UVC, UAC and HD audio devices. That's one I've done in a test 
implementation of the media controller API in the uvcvideo driver.

> Alternatively, entities can have 'mute' mixer controls associated with
> their pads.  In this case, multiple unmuted inputs would be mixed
> together.

I don't see any problem keeping those.

> > +#define MEDIA_ENTITY_TYPE_MASK               0x00ff0000
> > +#define MEDIA_ENTITY_SUBTYPE_MASK            0x0000ffff
> > +
> > +#define MEDIA_ENTITY_TYPE_NODE               (1 <<
> > MEDIA_ENTITY_TYPE_SHIFT) ...
> > +#define MEDIA_ENTITY_TYPE_NODE_ALSA          (MEDIA_ENTITY_TYPE_NODE +
> > 3)
> 
> ALSA has PCM and MIDI devices, and several types of mixer controls.
> (It also has hardware dependent and timer devices, but I don't think
> these would need topology information.)  So we need at least these:
>   MEDIA_ENTITY_TYPE_NODE_ALSA_PCM
>   MEDIA_ENTITY_TYPE_NODE_ALSA_MIDI
>   MEDIA_ENTITY_TYPE_SUBDEV_ALSA_CONTROL

I agree about PCM and MIDI, but I'm not sure about controls. If controls are 
part of an entity, the entity will be reported through the media controller 
API. If information about that entity can be queried through existing APIs 
(ALSA, V4L, ...) those APIs should be used. For instance, on the V4L side, 
V4L2 sub-devices are mapped to entities and also have a device node which can 
be used to enumerate the controls supported by the subdev. The media 
controller only reports the entity -> major:minor mapping to let applications 
find the device and query it directly.

I think we will need a new ioctl in the media controller API to report 
advanced information about an entity. This could be used to report controls 
implemented by an ALSA element if ALSA doesn't provide a way to do that 
directly. Another use case, which make me think that such an ioctl would be 
needed, is to report UVC extension units type (16-byte GUID) to userspace.

> Furthermore, topology information is also needed for entities not
> associated with a mixer control, such as microphones, speakers, jacks/
> connectors, and effect units.  These entities are defined in the USB and
> HD audio specifications, but are not yet handled by ALSA.

Agreed, we will need to add new entity types and subtypes for those. The 
reason they're not part of this submission is that I wanted real use cases 
instead of coming up with a made-up list of entities I think would be useful.

> > +struct media_entity {
> > ...
> > +	union {
> > +		/* Node specifications */
> > +		struct {
> > +			u32 major;
> > +			u32 minor;
> > +		} v4l;
> > +		struct {
> > +			u32 major;
> > +			u32 minor;
> > +		} fb;
> > +		int alsa;
> > +		int dvb;
> > +
> > +		/* Sub-device specifications */
> > +		/* Nothing needed yet */
> > +	};
> 
> ALSA devices are not addressed by their device node but with card/device/
> subdevice numbers; mixer controls have numeric IDs, unique per card:
> 
> 		struct {
> 			int card;
> 			int device;
> 			int subdevice;
> 		} alsa_device;

I will use this instead of 'int alsa'. Thanks a lot for the useful feedback.

> 		struct {
> 			int card;
> 			int numid;
> 		} alsa_control;

Regarding controls, let's first see how they should be reported by userspace, 
as explained above.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Mark Brown]

On Thu, Nov 25, 2010 at 04:21:38PM +0100, Laurent Pinchart wrote:
> On Thursday 25 November 2010 10:38:05 Clemens Ladisch wrote:

> > ALSA has PCM and MIDI devices, and several types of mixer controls.
> > (It also has hardware dependent and timer devices, but I don't think
> > these would need topology information.)  So we need at least these:
> >   MEDIA_ENTITY_TYPE_NODE_ALSA_PCM
> >   MEDIA_ENTITY_TYPE_NODE_ALSA_MIDI
> >   MEDIA_ENTITY_TYPE_SUBDEV_ALSA_CONTROL

> I agree about PCM and MIDI, but I'm not sure about controls. If controls are 
> part of an entity, the entity will be reported through the media controller 
> API. If information about that entity can be queried through existing APIs 
> (ALSA, V4L, ...) those APIs should be used. For instance, on the V4L side, 
> V4L2 sub-devices are mapped to entities and also have a device node which can 
> be used to enumerate the controls supported by the subdev. The media 
> controller only reports the entity -> major:minor mapping to let applications 
> find the device and query it directly.

For audio we don't currently have a sensible API for associating
controls with any sort of map of how the device is laid out, userspace
has to play guessing games.

> I think we will need a new ioctl in the media controller API to report 
> advanced information about an entity. This could be used to report controls 
> implemented by an ALSA element if ALSA doesn't provide a way to do that 
> directly. Another use case, which make me think that such an ioctl would be 
> needed, is to report UVC extension units type (16-byte GUID) to userspace.

That seems reasonable.

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Clemens Ladisch]

Laurent Pinchart wrote:
> On Thursday 25 November 2010 10:38:05 Clemens Ladisch wrote:
> >   MEDIA_ENTITY_TYPE_NODE_ALSA_PCM
> >   MEDIA_ENTITY_TYPE_NODE_ALSA_MIDI
> >   MEDIA_ENTITY_TYPE_SUBDEV_ALSA_CONTROL
> 
> I agree about PCM and MIDI, but I'm not sure about controls. If controls are
> part of an entity, the entity will be reported through the media controller
> API. If information about that entity can be queried through existing APIs
> (ALSA, V4L, ...) those APIs should be used.

At the moment, ALSA has no API for topology information.

> I can certainly imagine a graph with 100 controls, but maybe several
> controls can be part of the same node ?

There is indeed no strict 1:1 relation; e.g., volume and mute are often
part of the same node.  So it looks we need some kind of separate ALSA
node, which can be associated with mixer controls and/or other
information.

ALSA already has is a method to query arbitrary (TLV) metadata for mixer
controls; the entity/control relationship can be stored in the control.

> I think we will need a new ioctl in the media controller API to report
> advanced information about an entity. This could be used to report controls
> implemented by an ALSA element if ALSA doesn't provide a way to do that
> directly.

This advanced information would always be specific to the entity type,
so maybe this should be part of that subsystem's API.  Otherwise, the
media_entity would need a callback, or store a pointer to some memory
block (which assumes that the information is always constant).

> > Furthermore, topology information is also needed for entities not
> > associated with a mixer control, such as microphones, speakers, jacks/
> > connectors, and effect units.  These entities are defined in the USB and
> > HD audio specifications, but are not yet handled by ALSA.
> 
> Agreed, we will need to add new entity types and subtypes for those. The
> reason they're not part of this submission is that I wanted real use cases
> instead of coming up with a made-up list of entities I think would be useful.

The reason that I'm always mentioning the USB and HD audio specs is that
those already define entities that should cover practically all of the
audio needs.  (And, of course, we want to be able to report those
entities without having to do too many conversions.)


I'll see if I can draw up the ALSA-specific media stuff over the weekend.


Regards,
Clemens

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Clemens Ladisch]

I wrote:
> I'll see if I can draw up the ALSA-specific media stuff over the weekend.

Sorry, wrong weekend.

Anyway, below are some remarks and a patch.


* Entity types

TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a node
in a graph, which does not distinguish it from other entity types
because all entities are part of the topology graph.  I chose "device"
as this type describes entities that are visible as some device node to
other software.

TYPE_EXT describes entities that represent some interface to the
external world, TYPE_INT those that are internal to the entire device.
(I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
to be an even more meaningless name.)


ALSA mixer controls are not directly represented; a better fit for the
architecture of actual devices is that one or more mixer controls can be
associated with an entity.  (This can be done with a field of the mixer
control.)


* Entity properties

There needs to be a mechanism to associate meta-information (properties)
with entities.  This information should be optional and extensible, but,
when being handled inside the kernel, doesn't need to be more than
a read-only blob.  I think that something like ALSA's TLV format (used
for mixer controls) can be used here.  (I'm not mentioning the X-word
here, except to note that the "M" stands for "markup".)


* Entity subtypes

EXT_JACK_ANALOG represents any analog audio and/or video connector.
Properties for audio jacks would be jack type (TRS/RCA), color code,
line level, position, etc.

EXT_JACK_DIGITAL represents a digital connector like S/PDIF (coax/
TOSLINK), ADAT, TDIF, or MADI.

EXT_JACK_BUS represents a bus like FireWire and comes from the USB audio
spec.  (I doubt that any devices with this entitiy will ever exist.)

EXT_INSTRUMENT represents something like an e-guitar, keyboard, or MIDI
controller.  (Instrument entities are typically audio sources and MIDI
sources and sinks, but can also be audio sinks.)

EXT_SPEAKER also includes headphones; there might be made a case for
having those as a separate subtype.

EXT_PLAYER represents a device like a CD/DVD/tape player.  Recorders can
also write to that device, so "player" might not be an ideal name.

EXT_BROADCAST represents devices like TV tuners, satellite receivers,
cable tuners, or radios.

INT_SYNTHESIZER converts MIDI to audio.

INT_NOISE_SOURCE comes from the USB audio spec; this is not an attempt
to describe the characteristics of consumer-grade devices :-) , but
represents an internal noise source for level calibration or measurements.

INT_CONTROLS may have multiple independent controls (this is USB's
Feature Unit); INT_EFFECT may have multiple controls that affect one
single algorithm.

INT_CHANNEL_SPLIT/MERGE are needed for HDAudio devices, whose topology
information has only stereo links.


* Entity specifications

While TYPE_DEVICE entities can be identified by their device node, other
entities typcially have just a numeric ID.  For that, it would be useful
to make do without separate identification and let the driver choose the
entity ID.


Signed-off-by: Clemens Ladisch <clemens@ladisch.de>

--- linux/include/linux/media.h
+++ linux/include/linux/media.h
@@ -46,16 +46,36 @@ struct media_device_info {
 #define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
 #define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
 
-#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
-#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
-#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
-#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
-#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
+#define MEDIA_ENTITY_TYPE_DEVICE		(1 << MEDIA_ENTITY_TYPE_SHIFT)
+#define MEDIA_ENTITY_TYPE_DEVICE_V4L		(MEDIA_ENTITY_TYPE_DEVICE + 1)
+#define MEDIA_ENTITY_TYPE_DEVICE_FB		(MEDIA_ENTITY_TYPE_DEVICE + 2)
+#define MEDIA_ENTITY_TYPE_DEVICE_DVB		(MEDIA_ENTITY_TYPE_DEVICE + 3)
+#define MEDIA_ENTITY_TYPE_DEVICE_ALSA_PCM	(MEDIA_ENTITY_TYPE_DEVICE + 4)
+#define MEDIA_ENTITY_TYPE_DEVICE_ALSA_MIDI	(MEDIA_ENTITY_TYPE_DEVICE + 5)
 
-#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
-#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
-#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
-#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
+#define MEDIA_ENTITY_TYPE_EXT			(2 << MEDIA_ENTITY_TYPE_SHIFT)
+#define MEDIA_ENTITY_TYPE_EXT_SENSOR		(MEDIA_ENTITY_TYPE_EXT + 1)
+#define MEDIA_ENTITY_TYPE_EXT_FLASH		(MEDIA_ENTITY_TYPE_EXT + 2)
+#define MEDIA_ENTITY_TYPE_EXT_LENS		(MEDIA_ENTITY_TYPE_EXT + 3)
+#define MEDIA_ENTITY_TYPE_EXT_JACK_MIDI		(MEDIA_ENTITY_TYPE_EXT + 4)
+#define MEDIA_ENTITY_TYPE_EXT_JACK_ANALOG	(MEDIA_ENTITY_TYPE_EXT + 5)
+#define MEDIA_ENTITY_TYPE_EXT_JACK_DIGITAL	(MEDIA_ENTITY_TYPE_EXT + 6)
+#define MEDIA_ENTITY_TYPE_EXT_JACK_BUS		(MEDIA_ENTITY_TYPE_EXT + 7)
+#define MEDIA_ENTITY_TYPE_EXT_INSTRUMENT	(MEDIA_ENTITY_TYPE_EXT + 8)
+#define MEDIA_ENTITY_TYPE_EXT_SPEAKER		(MEDIA_ENTITY_TYPE_EXT + 9)
+#define MEDIA_ENTITY_TYPE_EXT_MICROPHONE	(MEDIA_ENTITY_TYPE_EXT + 10)
+#define MEDIA_ENTITY_TYPE_EXT_PLAYER		(MEDIA_ENTITY_TYPE_EXT + 11)
+#define MEDIA_ENTITY_TYPE_EXT_BROADCAST		(MEDIA_ENTITY_TYPE_EXT + 12)
+
+#define MEDIA_ENTITY_TYPE_INT			(3 << MEDIA_ENTITY_TYPE_SHIFT)
+#define MEDIA_ENTITY_TYPE_INT_SYNTHESIZER	(MEDIA_ENTITY_TYPE_INT + 1)
+#define MEDIA_ENTITY_TYPE_INT_NOISE_SOURCE	(MEDIA_ENTITY_TYPE_INT + 2)
+#define MEDIA_ENTITY_TYPE_INT_MIXER		(MEDIA_ENTITY_TYPE_INT + 3)
+#define MEDIA_ENTITY_TYPE_INT_SELECTOR		(MEDIA_ENTITY_TYPE_INT + 4)
+#define MEDIA_ENTITY_TYPE_INT_CONTROLS		(MEDIA_ENTITY_TYPE_INT + 5)
+#define MEDIA_ENTITY_TYPE_INT_EFFECT		(MEDIA_ENTITY_TYPE_INT + 6)
+#define MEDIA_ENTITY_TYPE_INT_CHANNEL_SPLIT	(MEDIA_ENTITY_TYPE_INT + 7)
+#define MEDIA_ENTITY_TYPE_INT_CHANNEL_MERGE	(MEDIA_ENTITY_TYPE_INT + 8)
 
 #define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
 
@@ -72,7 +92,7 @@ struct media_entity_desc {
 	__u32 reserved[4];
 
 	union {
-		/* Node specifications */
+		/* Device specifications */
 		struct {
 			__u32 major;
 			__u32 minor;
@@ -81,11 +101,15 @@ struct media_entity_desc {
 			__u32 major;
 			__u32 minor;
 		} fb;
-		int alsa;
+		struct {
+			__u32 card;
+			__u32 device;
+			__s32 subdevice;
+		} alsa;
 		int dvb;
 
 		/* Sub-device specifications */
 		/* Nothing needed yet */
 		__u8 raw[184];
 	};
 };

Re: [alsa-devel] [RFC/PATCH v6 03/12] media: Entities, pads and links

[Laurent Pinchart]

Hi Clemens,

On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> I wrote:
> > I'll see if I can draw up the ALSA-specific media stuff over the weekend.
> 
> Sorry, wrong weekend.
> 
> Anyway, below are some remarks and a patch.

Thank you. Please see my comments inline.

> * Entity types
> 
> TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a node
> in a graph, which does not distinguish it from other entity types
> because all entities are part of the topology graph.  I chose "device"
> as this type describes entities that are visible as some device node to
> other software.

What this type describes is a device node. Both NODE and DEVICE can be 
confusing in my opinion, but DEVICE_NODE is a bit long.

> TYPE_EXT describes entities that represent some interface to the
> external world, TYPE_INT those that are internal to the entire device.
> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
> to be an even more meaningless name.)

SUBDEV comes from the V4L2 world, and I agree that it might not be a very good 
name.

I'm not sure I would split entities in internal/external categories. I would 
create a category for connectors though.

> ALSA mixer controls are not directly represented; a better fit for the
> architecture of actual devices is that one or more mixer controls can be
> associated with an entity.  (This can be done with a field of the mixer
> control.)

Agreed.

> * Entity properties
> 
> There needs to be a mechanism to associate meta-information (properties)
> with entities.  This information should be optional and extensible, but,
> when being handled inside the kernel, doesn't need to be more than
> a read-only blob.  I think that something like ALSA's TLV format (used
> for mixer controls) can be used here.  (I'm not mentioning the X-word
> here, except to note that the "M" stands for "markup".)

I've been thinking of adding a new ioctl for that. It's something we need to 
draft. The UVC driver will need it, and I'm pretty sure other V4L2 drivers 
would find it useful as well.

> * Entity subtypes
> 
> EXT_JACK_ANALOG represents any analog audio and/or video connector.
> Properties for audio jacks would be jack type (TRS/RCA), color code,
> line level, position, etc.
> 
> EXT_JACK_DIGITAL represents a digital connector like S/PDIF (coax/
> TOSLINK), ADAT, TDIF, or MADI.
> 
> EXT_JACK_BUS represents a bus like FireWire and comes from the USB audio
> spec.  (I doubt that any devices with this entitiy will ever exist.)
> 
> EXT_INSTRUMENT represents something like an e-guitar, keyboard, or MIDI
> controller.  (Instrument entities are typically audio sources and MIDI
> sources and sinks, but can also be audio sinks.)
> 
> EXT_SPEAKER also includes headphones; there might be made a case for
> having those as a separate subtype.

Shouldn't headphones be represented by an EXT_JACK_ANALOG ?

> EXT_PLAYER represents a device like a CD/DVD/tape player.  Recorders can
> also write to that device, so "player" might not be an ideal name.
> 
> EXT_BROADCAST represents devices like TV tuners, satellite receivers,
> cable tuners, or radios.

There's clearly an overlap with V4L here. Hopefully someone from the linux-
media list can comment on this.

> INT_SYNTHESIZER converts MIDI to audio.
> 
> INT_NOISE_SOURCE comes from the USB audio spec; this is not an attempt
> to describe the characteristics of consumer-grade devices :-) , but
> represents an internal noise source for level calibration or measurements.
> 
> INT_CONTROLS may have multiple independent controls (this is USB's
> Feature Unit); INT_EFFECT may have multiple controls that affect one
> single algorithm.

I'd describe this as a feature unit/processing unit then.

> INT_CHANNEL_SPLIT/MERGE are needed for HDAudio devices, whose topology
> information has only stereo links.

Some of those INT entities could also be implemented in dedicated chips, so I 
really think the EXT/INT split doesn't make too much sense. Should we have an 
AUDIO category ?

> * Entity specifications
> 
> While TYPE_DEVICE entities can be identified by their device node, other
> entities typcially have just a numeric ID.

In V4L2 sub-devices have (or rather will have once the media controller 
patches will be integrated) device nodes as well, so exposing that information 
is required.

> For that, it would be useful to make do without separate identification and
> let the driver choose the entity ID.

How would drivers do that ? What if you have two instances of the same chip (a 
video sensor, audio mixer, ...) on the same board ?

> Signed-off-by: Clemens Ladisch <clemens@ladisch.de>
> 
> --- linux/include/linux/media.h
> +++ linux/include/linux/media.h
> @@ -46,16 +46,36 @@ struct media_device_info {
>  #define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
>  #define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
> 
> -#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
> -#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
> -#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
> -#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
> -#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
> +#define MEDIA_ENTITY_TYPE_DEVICE		(1 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_DEVICE_V4L		(MEDIA_ENTITY_TYPE_DEVICE + 1)
> +#define MEDIA_ENTITY_TYPE_DEVICE_FB		(MEDIA_ENTITY_TYPE_DEVICE + 2)
> +#define MEDIA_ENTITY_TYPE_DEVICE_DVB		(MEDIA_ENTITY_TYPE_DEVICE + 3)
> +#define MEDIA_ENTITY_TYPE_DEVICE_ALSA_PCM	(MEDIA_ENTITY_TYPE_DEVICE + 4)
> +#define MEDIA_ENTITY_TYPE_DEVICE_ALSA_MIDI	(MEDIA_ENTITY_TYPE_DEVICE + 5)
> 
> -#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
> -#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
> -#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
> -#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
> +#define MEDIA_ENTITY_TYPE_EXT			(2 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_EXT_SENSOR		(MEDIA_ENTITY_TYPE_EXT + 1)
> +#define MEDIA_ENTITY_TYPE_EXT_FLASH		(MEDIA_ENTITY_TYPE_EXT + 2)
> +#define MEDIA_ENTITY_TYPE_EXT_LENS		(MEDIA_ENTITY_TYPE_EXT + 3)
> +#define MEDIA_ENTITY_TYPE_EXT_JACK_MIDI		(MEDIA_ENTITY_TYPE_EXT + 4)
> +#define MEDIA_ENTITY_TYPE_EXT_JACK_ANALOG	(MEDIA_ENTITY_TYPE_EXT + 5)
> +#define MEDIA_ENTITY_TYPE_EXT_JACK_DIGITAL	(MEDIA_ENTITY_TYPE_EXT + 6)
> +#define MEDIA_ENTITY_TYPE_EXT_JACK_BUS		(MEDIA_ENTITY_TYPE_EXT + 7)
> +#define MEDIA_ENTITY_TYPE_EXT_INSTRUMENT	(MEDIA_ENTITY_TYPE_EXT + 8)
> +#define MEDIA_ENTITY_TYPE_EXT_SPEAKER		(MEDIA_ENTITY_TYPE_EXT + 9)
> +#define MEDIA_ENTITY_TYPE_EXT_MICROPHONE	(MEDIA_ENTITY_TYPE_EXT + 10)
> +#define MEDIA_ENTITY_TYPE_EXT_PLAYER		(MEDIA_ENTITY_TYPE_EXT + 11)
> +#define MEDIA_ENTITY_TYPE_EXT_BROADCAST		(MEDIA_ENTITY_TYPE_EXT + 12)
> +
> +#define MEDIA_ENTITY_TYPE_INT			(3 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_INT_SYNTHESIZER	(MEDIA_ENTITY_TYPE_INT + 1)
> +#define MEDIA_ENTITY_TYPE_INT_NOISE_SOURCE	(MEDIA_ENTITY_TYPE_INT + 2)
> +#define MEDIA_ENTITY_TYPE_INT_MIXER		(MEDIA_ENTITY_TYPE_INT + 3)
> +#define MEDIA_ENTITY_TYPE_INT_SELECTOR		(MEDIA_ENTITY_TYPE_INT + 4)
> +#define MEDIA_ENTITY_TYPE_INT_CONTROLS		(MEDIA_ENTITY_TYPE_INT + 5)
> +#define MEDIA_ENTITY_TYPE_INT_EFFECT		(MEDIA_ENTITY_TYPE_INT + 6)
> +#define MEDIA_ENTITY_TYPE_INT_CHANNEL_SPLIT	(MEDIA_ENTITY_TYPE_INT + 7)
> +#define MEDIA_ENTITY_TYPE_INT_CHANNEL_MERGE	(MEDIA_ENTITY_TYPE_INT + 8)
> 
>  #define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
> 
> @@ -72,7 +92,7 @@ struct media_entity_desc {
>  	__u32 reserved[4];
> 
>  	union {
> -		/* Node specifications */
> +		/* Device specifications */
>  		struct {
>  			__u32 major;
>  			__u32 minor;
> @@ -81,11 +101,15 @@ struct media_entity_desc {
>  			__u32 major;
>  			__u32 minor;
>  		} fb;
> -		int alsa;
> +		struct {
> +			__u32 card;
> +			__u32 device;
> +			__s32 subdevice;
> +		} alsa;

I will already incorporate this change, and I'll wait for other opinions on 
the types before changing them.

>  		int dvb;
> 
>  		/* Sub-device specifications */
>  		/* Nothing needed yet */
>  		__u8 raw[184];
>  	};
>  };

-- 
Regards,

Laurent Pinchart

Re: [alsa-devel] [RFC/PATCH v6 03/12] media: Entities, pads and links

[Hans Verkuil]

> Hi Clemens,
>
> On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
>> I wrote:
>> > I'll see if I can draw up the ALSA-specific media stuff over the
>> weekend.
>>
>> Sorry, wrong weekend.
>>
>> Anyway, below are some remarks and a patch.
>
> Thank you. Please see my comments inline.
>
>> * Entity types
>>
>> TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a node
>> in a graph, which does not distinguish it from other entity types
>> because all entities are part of the topology graph.  I chose "device"
>> as this type describes entities that are visible as some device node to
>> other software.
>
> What this type describes is a device node. Both NODE and DEVICE can be
> confusing in my opinion, but DEVICE_NODE is a bit long.

What about DEVNODE? I think that would be a good alternative.

>> TYPE_EXT describes entities that represent some interface to the
>> external world, TYPE_INT those that are internal to the entire device.
>> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
>> to be an even more meaningless name.)
>
> SUBDEV comes from the V4L2 world, and I agree that it might not be a very
> good
> name.

SUBDEV refers to a specific type of driver. Within the v4l world it is
well defined. So I prefer to keep this. Perhaps some additional comments
or documentation can be added to clarify this.

> I'm not sure I would split entities in internal/external categories. I
> would
> create a category for connectors though.

I agree. It was always the plan to eventually add connectors, but v4l
didn't really need it (it already has an API to enumerate connectors).

>> ALSA mixer controls are not directly represented; a better fit for the
>> architecture of actual devices is that one or more mixer controls can be
>> associated with an entity.  (This can be done with a field of the mixer
>> control.)
>
> Agreed.
>
>> * Entity properties
>>
>> There needs to be a mechanism to associate meta-information (properties)
>> with entities.  This information should be optional and extensible, but,
>> when being handled inside the kernel, doesn't need to be more than
>> a read-only blob.  I think that something like ALSA's TLV format (used
>> for mixer controls) can be used here.  (I'm not mentioning the X-word
>> here, except to note that the "M" stands for "markup".)
>
> I've been thinking of adding a new ioctl for that. It's something we need
> to
> draft. The UVC driver will need it, and I'm pretty sure other V4L2 drivers
> would find it useful as well.
>
>> * Entity subtypes
>>
>> EXT_JACK_ANALOG represents any analog audio and/or video connector.
>> Properties for audio jacks would be jack type (TRS/RCA), color code,
>> line level, position, etc.
>>
>> EXT_JACK_DIGITAL represents a digital connector like S/PDIF (coax/
>> TOSLINK), ADAT, TDIF, or MADI.
>>
>> EXT_JACK_BUS represents a bus like FireWire and comes from the USB audio
>> spec.  (I doubt that any devices with this entitiy will ever exist.)
>>
>> EXT_INSTRUMENT represents something like an e-guitar, keyboard, or MIDI
>> controller.  (Instrument entities are typically audio sources and MIDI
>> sources and sinks, but can also be audio sinks.)
>>
>> EXT_SPEAKER also includes headphones; there might be made a case for
>> having those as a separate subtype.
>
> Shouldn't headphones be represented by an EXT_JACK_ANALOG ?
>
>> EXT_PLAYER represents a device like a CD/DVD/tape player.  Recorders can
>> also write to that device, so "player" might not be an ideal name.
>>
>> EXT_BROADCAST represents devices like TV tuners, satellite receivers,
>> cable tuners, or radios.

I don't think it is right to talk about 'represents devices'. I'd rephrase
it to 'connects to devices'.

> There's clearly an overlap with V4L here. Hopefully someone from the
> linux-
> media list can comment on this.

I don't think this will be a problem. Initially we probably won't be
enumerating connectors for V4L since it already has its own API for that.

>> INT_SYNTHESIZER converts MIDI to audio.
>>
>> INT_NOISE_SOURCE comes from the USB audio spec; this is not an attempt
>> to describe the characteristics of consumer-grade devices :-) , but
>> represents an internal noise source for level calibration or
>> measurements.
>>
>> INT_CONTROLS may have multiple independent controls (this is USB's
>> Feature Unit); INT_EFFECT may have multiple controls that affect one
>> single algorithm.
>
> I'd describe this as a feature unit/processing unit then.
>
>> INT_CHANNEL_SPLIT/MERGE are needed for HDAudio devices, whose topology
>> information has only stereo links.
>
> Some of those INT entities could also be implemented in dedicated chips,
> so I
> really think the EXT/INT split doesn't make too much sense. Should we have
> an
> AUDIO category ?
>
>> * Entity specifications
>>
>> While TYPE_DEVICE entities can be identified by their device node, other
>> entities typcially have just a numeric ID.
>
> In V4L2 sub-devices have (or rather will have once the media controller
> patches will be integrated) device nodes as well, so exposing that
> information
> is required.
>
>> For that, it would be useful to make do without separate identification
>> and
>> let the driver choose the entity ID.
>
> How would drivers do that ? What if you have two instances of the same
> chip (a
> video sensor, audio mixer, ...) on the same board ?

Regards,

       Hans

-- 
Hans Verkuil - video4linux developer - sponsored by Cisco

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Laurent Pinchart]

Hi Hans,

On Tuesday 14 December 2010 13:40:21 Hans Verkuil wrote:
> > On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> >> * Entity types
> >> 
> >> TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a node
> >> in a graph, which does not distinguish it from other entity types
> >> because all entities are part of the topology graph.  I chose "device"
> >> as this type describes entities that are visible as some device node to
> >> other software.
> > 
> > What this type describes is a device node. Both NODE and DEVICE can be
> > confusing in my opinion, but DEVICE_NODE is a bit long.
> 
> What about DEVNODE? I think that would be a good alternative.

Fine with me. Clemens, any opinion on that ?

> >> TYPE_EXT describes entities that represent some interface to the
> >> external world, TYPE_INT those that are internal to the entire device.
> >> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
> >> to be an even more meaningless name.)
> > 
> > SUBDEV comes from the V4L2 world, and I agree that it might not be a very
> > good name.
> 
> SUBDEV refers to a specific type of driver. Within the v4l world it is
> well defined. So I prefer to keep this. Perhaps some additional comments
> or documentation can be added to clarify this.

Should this be clarified by using V4L2_SUBDEV instead then ? What about ALSA 
entities, should they use MEDIA_ENTITY_TYPE_ALSA_* ?

> > I'm not sure I would split entities in internal/external categories. I
> > would create a category for connectors though.
> 
> I agree. It was always the plan to eventually add connectors, but v4l
> didn't really need it (it already has an API to enumerate connectors).
> 
> >> ALSA mixer controls are not directly represented; a better fit for the
> >> architecture of actual devices is that one or more mixer controls can be
> >> associated with an entity.  (This can be done with a field of the mixer
> >> control.)
> > 
> > Agreed.
> > 
> >> * Entity properties
> >> 
> >> There needs to be a mechanism to associate meta-information (properties)
> >> with entities.  This information should be optional and extensible, but,
> >> when being handled inside the kernel, doesn't need to be more than
> >> a read-only blob.  I think that something like ALSA's TLV format (used
> >> for mixer controls) can be used here.  (I'm not mentioning the X-word
> >> here, except to note that the "M" stands for "markup".)
> > 
> > I've been thinking of adding a new ioctl for that. It's something we need
> > to draft. The UVC driver will need it, and I'm pretty sure other V4L2
> > drivers would find it useful as well.
> > 
> >> * Entity subtypes
> >> 
> >> EXT_JACK_ANALOG represents any analog audio and/or video connector.
> >> Properties for audio jacks would be jack type (TRS/RCA), color code,
> >> line level, position, etc.
> >> 
> >> EXT_JACK_DIGITAL represents a digital connector like S/PDIF (coax/
> >> TOSLINK), ADAT, TDIF, or MADI.
> >> 
> >> EXT_JACK_BUS represents a bus like FireWire and comes from the USB audio
> >> spec.  (I doubt that any devices with this entitiy will ever exist.)
> >> 
> >> EXT_INSTRUMENT represents something like an e-guitar, keyboard, or MIDI
> >> controller.  (Instrument entities are typically audio sources and MIDI
> >> sources and sinks, but can also be audio sinks.)
> >> 
> >> EXT_SPEAKER also includes headphones; there might be made a case for
> >> having those as a separate subtype.
> > 
> > Shouldn't headphones be represented by an EXT_JACK_ANALOG ?
> > 
> >> EXT_PLAYER represents a device like a CD/DVD/tape player.  Recorders can
> >> also write to that device, so "player" might not be an ideal name.
> >> 
> >> EXT_BROADCAST represents devices like TV tuners, satellite receivers,
> >> cable tuners, or radios.
> 
> I don't think it is right to talk about 'represents devices'. I'd rephrase
> it to 'connects to devices'.
> 
> > There's clearly an overlap with V4L here. Hopefully someone from the
> > linux-media list can comment on this.
> 
> I don't think this will be a problem. Initially we probably won't be
> enumerating connectors for V4L since it already has its own API for that.

My understanding is that EXT_BROADCAST really represents the TV tuners, ..., 
not the connector they connect to. Some (all ?) of them are definitely V4L2 
subdevs.

> >> INT_SYNTHESIZER converts MIDI to audio.
> >> 
> >> INT_NOISE_SOURCE comes from the USB audio spec; this is not an attempt
> >> to describe the characteristics of consumer-grade devices :-) , but
> >> represents an internal noise source for level calibration or
> >> measurements.
> >> 
> >> INT_CONTROLS may have multiple independent controls (this is USB's
> >> Feature Unit); INT_EFFECT may have multiple controls that affect one
> >> single algorithm.
> > 
> > I'd describe this as a feature unit/processing unit then.
> > 
> >> INT_CHANNEL_SPLIT/MERGE are needed for HDAudio devices, whose topology
> >> information has only stereo links.
> > 
> > Some of those INT entities could also be implemented in dedicated chips,
> > so I really think the EXT/INT split doesn't make too much sense. Should we
> > have an AUDIO category ?
> > 
> >> * Entity specifications
> >> 
> >> While TYPE_DEVICE entities can be identified by their device node, other
> >> entities typcially have just a numeric ID.
> > 
> > In V4L2 sub-devices have (or rather will have once the media controller
> > patches will be integrated) device nodes as well, so exposing that
> > information
> > is required.
> > 
> >> For that, it would be useful to make do without separate identification
> >> and let the driver choose the entity ID.
> > 
> > How would drivers do that ? What if you have two instances of the same
> > chip (a video sensor, audio mixer, ...) on the same board ?

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Clemens Ladisch]

Laurent Pinchart wrote:
> On Tuesday 14 December 2010 13:40:21 Hans Verkuil wrote:
>> > On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
>> >> * Entity types
>> >> 
>> >> TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a node
>> >> in a graph, which does not distinguish it from other entity types
>> >> because all entities are part of the topology graph.  I chose "device"
>> >> as this type describes entities that are visible as some device node to
>> >> other software.
>> > 
>> > What this type describes is a device node. Both NODE and DEVICE can be
>> > confusing in my opinion, but DEVICE_NODE is a bit long.
>> 
>> What about DEVNODE? I think that would be a good alternative.
> 
> Fine with me. Clemens, any opinion on that ?

Fine with me too.

> > >> TYPE_EXT describes entities that represent some interface to the
> > >> external world, TYPE_INT those that are internal to the entire device.
> > >> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
> > >> to be an even more meaningless name.)
> > > 
> > > SUBDEV comes from the V4L2 world, and I agree that it might not be a very
> > > good name.
> > 
> > SUBDEV refers to a specific type of driver. Within the v4l world it is
> > well defined. So I prefer to keep this. Perhaps some additional comments
> > or documentation can be added to clarify this.
> 
> Should this be clarified by using V4L2_SUBDEV instead then ?

If the "SUBDEV" concept doesn't exist outside V4L, that would indeed be
better.

I don't want to rename things that come out of existing frameworks; this
naming discussion makes sense only for those entity (sub)types that can
be shared between them.  Are there any, besides jacks?

> What about ALSA entities, should they use MEDIA_ENTITY_TYPE_ALSA_* ?

The entity types representing ALSA devices are already named "ALSA".


Regards,
Clemens

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Laurent Pinchart]

Hi Clemens,

On Tuesday 14 December 2010 14:49:15 Clemens Ladisch wrote:
> Laurent Pinchart wrote:
> > On Tuesday 14 December 2010 13:40:21 Hans Verkuil wrote:
> >> > On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> >> >> * Entity types
> >> >> 
> >> >> TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a
> >> >> node in a graph, which does not distinguish it from other entity
> >> >> types because all entities are part of the topology graph.  I chose
> >> >> "device" as this type describes entities that are visible as some
> >> >> device node to other software.
> >> > 
> >> > What this type describes is a device node. Both NODE and DEVICE can be
> >> > confusing in my opinion, but DEVICE_NODE is a bit long.
> >> 
> >> What about DEVNODE? I think that would be a good alternative.
> > 
> > Fine with me. Clemens, any opinion on that ?
> 
> Fine with me too.

OK I'll use that name.

> > > >> TYPE_EXT describes entities that represent some interface to the
> > > >> external world, TYPE_INT those that are internal to the entire
> > > >> device. (I'm not sure if that distinction is very useful, but
> > > >> TYPE_SUBDEV seems to be an even more meaningless name.)
> > > > 
> > > > SUBDEV comes from the V4L2 world, and I agree that it might not be a
> > > > very good name.
> > > 
> > > SUBDEV refers to a specific type of driver. Within the v4l world it is
> > > well defined. So I prefer to keep this. Perhaps some additional
> > > comments or documentation can be added to clarify this.
> > 
> > Should this be clarified by using V4L2_SUBDEV instead then ?
> 
> If the "SUBDEV" concept doesn't exist outside V4L, that would indeed be
> better.
> 
> I don't want to rename things that come out of existing frameworks; this
> naming discussion makes sense only for those entity (sub)types that can
> be shared between them.  Are there any, besides jacks?

Some entities like TV tuners play a dual audio/video role. I'm not sure how to 
handle them, I lack experience in that field.

> > What about ALSA entities, should they use MEDIA_ENTITY_TYPE_ALSA_* ?
> 
> The entity types representing ALSA devices are already named "ALSA".

I was talking about the INT_* types. They're ALSA-specific, but have no ALSA 
in the type name.

-- 
Regards,

Laurent Pinchart

Re: [alsa-devel] [RFC/PATCH v6 03/12] media: Entities, pads and links

[Hans Verkuil]

On Wednesday, December 15, 2010 00:50:44 Laurent Pinchart wrote:
> Hi Clemens,
> 
> On Tuesday 14 December 2010 14:49:15 Clemens Ladisch wrote:
> > Laurent Pinchart wrote:
> > > On Tuesday 14 December 2010 13:40:21 Hans Verkuil wrote:
> > >> > On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> > >> >> * Entity types
> > >> >> 
> > >> >> TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a
> > >> >> node in a graph, which does not distinguish it from other entity
> > >> >> types because all entities are part of the topology graph.  I chose
> > >> >> "device" as this type describes entities that are visible as some
> > >> >> device node to other software.
> > >> > 
> > >> > What this type describes is a device node. Both NODE and DEVICE can be
> > >> > confusing in my opinion, but DEVICE_NODE is a bit long.
> > >> 
> > >> What about DEVNODE? I think that would be a good alternative.
> > > 
> > > Fine with me. Clemens, any opinion on that ?
> > 
> > Fine with me too.
> 
> OK I'll use that name.
> 
> > > > >> TYPE_EXT describes entities that represent some interface to the
> > > > >> external world, TYPE_INT those that are internal to the entire
> > > > >> device. (I'm not sure if that distinction is very useful, but
> > > > >> TYPE_SUBDEV seems to be an even more meaningless name.)
> > > > > 
> > > > > SUBDEV comes from the V4L2 world, and I agree that it might not be a
> > > > > very good name.
> > > > 
> > > > SUBDEV refers to a specific type of driver. Within the v4l world it is
> > > > well defined. So I prefer to keep this. Perhaps some additional
> > > > comments or documentation can be added to clarify this.
> > > 
> > > Should this be clarified by using V4L2_SUBDEV instead then ?
> > 
> > If the "SUBDEV" concept doesn't exist outside V4L, that would indeed be
> > better.
> > 
> > I don't want to rename things that come out of existing frameworks; this
> > naming discussion makes sense only for those entity (sub)types that can
> > be shared between them.  Are there any, besides jacks?
> 
> Some entities like TV tuners play a dual audio/video role. I'm not sure how to 
> handle them, I lack experience in that field.

It is very important to distinguish between the actual tuner device and the
physical connector. ALSA doesn't program a tuner device, that's the domain
of V4L and DVB. ALSA just sees an input pin.

Regarding tuners there are roughly two types of hardware: one where the audio
goes to an output jack (and the user has to use a loopback cable to hook it up
to an audio input), or it goes to memory using DMA and an ALSA driver.

In the first scenario the MC would model a TV_ANTENNA connector and an AUDIO_OUT
connector. The TV_ANTENNA connector would typically link to a V4L2_SUBDEV_TUNER,
which would link to a V4L2_SUBDEV_AUDIO_DEMOD (in turn linked to the AUDIO_OUT
connector). The tuner would also link to a V4L2_SUBDEV_VIDEO_DIGITIZER (in turn
linked to a DEVNODE_V4L).

In the second scenario there is no AUDIO_OUT connector, instead there is a
DEVNODE_ALSA.

It can get more complex: in the case of MPEG encoders the audio from the tuner
goes to an audio demod, the video goes to a digitizer, and the output of those
subdevs both go into the same MPEG encoder subdev.

When modeling hardware like audio or video devices it is important to remember
to separate I/O pins from actually physical connectors. E.g. an audio device may
have many possible input pins, but how they are hooked up to which physical
connectors is something that is board specific and not part of the audio driver
itself.

Anyway, what we need is a 'connector' entity. And just like the other entities,
connectors can have multiple input pads so I don't see any problems in modeling
antenna connectors.

Regards,

	Hans

> > > What about ALSA entities, should they use MEDIA_ENTITY_TYPE_ALSA_* ?
> > 
> > The entity types representing ALSA devices are already named "ALSA".
> 
> I was talking about the INT_* types. They're ALSA-specific, but have no ALSA 
> in the type name.
> 
> 

-- 
Hans Verkuil - video4linux developer - sponsored by Cisco

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Clemens Ladisch]

Laurent Pinchart wrote:
> On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
>> TYPE_EXT describes entities that represent some interface to the
>> external world, TYPE_INT those that are internal to the entire device.
>> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
>> to be an even more meaningless name.)
> 
> SUBDEV comes from the V4L2 world, and I agree that it might not be a very good 
> name.
> 
> I'm not sure I would split entities in internal/external categories. I would
> create a category for connectors though.

I'm not disagreeing, but what is actually the distinction between types
and subtypes?  ;-)

>> * Entity properties
>> 
>> There needs to be a mechanism to associate meta-information (properties)
>> with entities.  This information should be optional and extensible, but,
>> when being handled inside the kernel, doesn't need to be more than
>> a read-only blob.  I think that something like ALSA's TLV format (used
>> for mixer controls) can be used here.  (I'm not mentioning the X-word
>> here, except to note that the "M" stands for "markup".)
> 
> I've been thinking of adding a new ioctl for that. It's something we need to
> draft. The UVC driver will need it, and I'm pretty sure other V4L2 drivers
> would find it useful as well.

I'm imagining a "read-the-properties" ioctl that just returns the
entity's blob.

>> EXT_SPEAKER also includes headphones; there might be made a case for
>> having those as a separate subtype.
> 
> Shouldn't headphones be represented by an EXT_JACK_ANALOG ?

Headphone jacks are jacks; there are also USB headphones.

>> EXT_BROADCAST represents devices like TV tuners, satellite receivers,
>> cable tuners, or radios.
> 
> There's clearly an overlap with V4L here.

These come from the USB audio spec.  Video devices are indeed likely to
be more detailed than just a single audio source. :)

>> INT_CONTROLS may have multiple independent controls (this is USB's
>> Feature Unit); INT_EFFECT may have multiple controls that affect one
>> single algorithm.
> 
> I'd describe this as a feature unit/processing unit then.

I was aiming for more descriptive names, but I agree that the original
names might be more useful.

> Should we have an AUDIO category ?

Probably not, because there are combined audio/video jacks, any maybe
other entities.

>> * Entity specifications
>> 
>> While TYPE_DEVICE entities can be identified by their device node, other
>> entities typcially have just a numeric ID.
> 
> In V4L2 sub-devices have (or rather will have once the media controller
> patches will be integrated) device nodes as well, so exposing that information
> is required.

USB and HDA entities already have numeric IDs.

>> For that, it would be useful to make do without separate identification and
>> let the driver choose the entity ID.
> 
> How would drivers do that ? What if you have two instances of the same chip
> (a video sensor, audio mixer, ...) on the same board ?

Then those would get different IDs; USB descriptors always describe the
entire device.


Regards,
Clemens

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Takashi Iwai]

At Tue, 14 Dec 2010 14:31:55 +0100,
Clemens Ladisch wrote:
> 
> > Should we have an AUDIO category ?
> 
> Probably not, because there are combined audio/video jacks, any maybe
> other entities.

Yes, nowadays HDMI / DP are pretty common, for example.


Takashi

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Laurent Pinchart]

Hi Clemens,

On Tuesday 14 December 2010 14:31:55 Clemens Ladisch wrote:
> Laurent Pinchart wrote:
> > On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> >> TYPE_EXT describes entities that represent some interface to the
> >> external world, TYPE_INT those that are internal to the entire device.
> >> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
> >> to be an even more meaningless name.)
> > 
> > SUBDEV comes from the V4L2 world, and I agree that it might not be a very
> > good name.
> > 
> > I'm not sure I would split entities in internal/external categories. I
> > would create a category for connectors though.
> 
> I'm not disagreeing, but what is actually the distinction between types
> and subtypes?  ;-)

The type is currently used to distinguish between entities that stream media 
data from/to memory and other entities. They need to be handled differently in 
the kernel for power management purposes for instance.

I'm not sure if we should create new types, or just remove the type/subtype 
distinction and add a flag somewhere.

> >> * Entity properties
> >> 
> >> There needs to be a mechanism to associate meta-information (properties)
> >> with entities.  This information should be optional and extensible, but,
> >> when being handled inside the kernel, doesn't need to be more than
> >> a read-only blob.  I think that something like ALSA's TLV format (used
> >> for mixer controls) can be used here.  (I'm not mentioning the X-word
> >> here, except to note that the "M" stands for "markup".)
> > 
> > I've been thinking of adding a new ioctl for that. It's something we need
> > to draft. The UVC driver will need it, and I'm pretty sure other V4L2
> > drivers would find it useful as well.
> 
> I'm imagining a "read-the-properties" ioctl that just returns the
> entity's blob.

Martin Rubli has already proposed something similar a while ago on the linux-
media mailing list. His proposal didn't use TLV though.

> >> EXT_SPEAKER also includes headphones; there might be made a case for
> >> having those as a separate subtype.
> > 
> > Shouldn't headphones be represented by an EXT_JACK_ANALOG ?
> 
> Headphone jacks are jacks; there are also USB headphones.

So EXT_SPEAKER are speakers not connected through a jack (USB, internal 
analog, ...) ?

> >> EXT_BROADCAST represents devices like TV tuners, satellite receivers,
> >> cable tuners, or radios.
> > 
> > There's clearly an overlap with V4L here.
> 
> These come from the USB audio spec.  Video devices are indeed likely to
> be more detailed than just a single audio source. :)

Does EXT_BROADCAST represent the TV tuner (or satellite receiver, cable tuner, 
radio tuner, ...) itself, or the connection between the tuner and the rest of 
the device ? Most TV tuner are currently handled by V4L2 and would thus turn 
up as V4L2 subdevs (I'm not sure if that's what we want in the long term, but 
it's at least the current situation).

> >> INT_CONTROLS may have multiple independent controls (this is USB's
> >> Feature Unit); INT_EFFECT may have multiple controls that affect one
> >> single algorithm.
> > 
> > I'd describe this as a feature unit/processing unit then.
> 
> I was aiming for more descriptive names, but I agree that the original
> names might be more useful.
> 
> > Should we have an AUDIO category ?
> 
> Probably not, because there are combined audio/video jacks, any maybe
> other entities.

> >> * Entity specifications
> >> 
> >> While TYPE_DEVICE entities can be identified by their device node, other
> >> entities typcially have just a numeric ID.
> > 
> > In V4L2 sub-devices have (or rather will have once the media controller
> > patches will be integrated) device nodes as well, so exposing that
> > information is required.
> 
> USB and HDA entities already have numeric IDs.

Right. Same for USB Video Class.

We could let drivers set the entity ID, and have the core fill it if the value 
is 0. Non-zero values would have to be checked for uniqueness though. Hans, a 
comment on that ?

> >> For that, it would be useful to make do without separate identification
> >> and let the driver choose the entity ID.
> > 
> > How would drivers do that ? What if you have two instances of the same
> > chip (a video sensor, audio mixer, ...) on the same board ?
> 
> Then those would get different IDs; USB descriptors always describe the
> entire device.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Clemens Ladisch]

Laurent Pinchart wrote:
> On Tuesday 14 December 2010 14:31:55 Clemens Ladisch wrote:
> > Laurent Pinchart wrote:
> > > On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> > >> EXT_SPEAKER also includes headphones; there might be made a case for
> > >> having those as a separate subtype.
> > > 
> > > Shouldn't headphones be represented by an EXT_JACK_ANALOG ?
> > 
> > Headphone jacks are jacks; there are also USB headphones.
> 
> So EXT_SPEAKER are speakers not connected through a jack (USB, internal
> analog, ...) ?

Yes.

When there is jack, the driver often does not know what is connected.

> > >> EXT_BROADCAST represents devices like TV tuners, satellite receivers,
> > >> cable tuners, or radios.
> > > 
> > > There's clearly an overlap with V4L here.
> > 
> > These come from the USB audio spec.  Video devices are indeed likely to
> > be more detailed than just a single audio source. :)
> 
> Does EXT_BROADCAST represent the TV tuner (or satellite receiver, cable tuner,
> radio tuner, ...) itself, or the connection between the tuner and the rest of
> the device ? Most TV tuner are currently handled by V4L2 and would thus turn
> up as V4L2 subdevs (I'm not sure if that's what we want in the long term, but
> it's at least the current situation).

>From the point of view of an audio device, this would be just some audio
source, much like a connector.  We don't need this if there is some
better V4L entitity that the USB audio entity can be mapped to.


Regards,
Clemens

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Raymond Yau]

2010/12/14 Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> Hi Clemens,
>
> On Tuesday 14 December 2010 14:31:55 Clemens Ladisch wrote:
> > Laurent Pinchart wrote:
> > > On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> > >> TYPE_EXT describes entities that represent some interface to the
> > >> external world, TYPE_INT those that are internal to the entire device.
> > >> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV
> seems
> > >> to be an even more meaningless name.)
> > >
> > > SUBDEV comes from the V4L2 world, and I agree that it might not be a
> very
> > > good name.
> > >
>

It is rather confusing since subdevice in ALSA has another specific meaning


 **** List of PLAYBACK Hardware Devices ****
card 0: Intel [HDA Intel], device 0: VT1702 Analog [VT1702 Analog]
  Subdevices: 1/2
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1

**** List of CAPTURE Hardware Devices ****
card 0: Intel [HDA Intel], device 0: VT1702 Analog [VT1702 Analog]
  Subdevices: 3/3
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2




>
> > >> EXT_SPEAKER also includes headphones; there might be made a case for
> > >> having those as a separate subtype.
> > >
> > > Shouldn't headphones be represented by an EXT_JACK_ANALOG ?
> >
> > Headphone jacks are jacks; there are also USB headphones.
>
> So EXT_SPEAKER are speakers not connected through a jack (USB, internal
> analog, ...) ?
>

How about internal microphone of those laptop ? ( mic array , ...)

How about those default device in HDA specification ( e.g. CD , AUX,
Telephony , Modem Line , Modem Headset)

Re: [alsa-devel] [RFC/PATCH v6 03/12] media: Entities, pads and links

[Hans Verkuil]

> Laurent Pinchart wrote:
>> On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
>>> TYPE_EXT describes entities that represent some interface to the
>>> external world, TYPE_INT those that are internal to the entire device.
>>> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
>>> to be an even more meaningless name.)
>>
>> SUBDEV comes from the V4L2 world, and I agree that it might not be a
>> very good
>> name.
>>
>> I'm not sure I would split entities in internal/external categories. I
>> would
>> create a category for connectors though.
>
> I'm not disagreeing, but what is actually the distinction between types
> and subtypes?  ;-)

The type tells what the behavior is of an entity. E.g., type DEVNODE
represents device node(s) in userspace, V4L2_SUBDEV represents a v4l2
sub-device, etc. The subtype tells whether a V4L2_SUBDEV is a sensor or a
receiver or whatever. Nice to know, but it doesn't change the way
sub-devices work.

In the case of connectors you would create a CONNECTOR type and have a
bunch of subtypes for all the variations of connectors.

That said, I'm not sure whether the distinction is useful for DEVNODEs.
You do need to know the subtype in order to interpret the union correctly.

Laurent, does the MC code test against the DEVNODE type? I.e., does the MC
code ignore the subtype of a DEVNODE, or does it always use it?

Regards,

        Hans

-- 
Hans Verkuil - video4linux developer - sponsored by Cisco

Re: [RFC/PATCH v6 03/12] media: Entities, pads and links

[Laurent Pinchart]

Hi Hans,

On Tuesday 14 December 2010 15:51:08 Hans Verkuil wrote:
> > Laurent Pinchart wrote:
> >> On Monday 13 December 2010 17:10:51 Clemens Ladisch wrote:
> >>> TYPE_EXT describes entities that represent some interface to the
> >>> external world, TYPE_INT those that are internal to the entire device.
> >>> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
> >>> to be an even more meaningless name.)
> >> 
> >> SUBDEV comes from the V4L2 world, and I agree that it might not be a
> >> very good
> >> name.
> >> 
> >> I'm not sure I would split entities in internal/external categories. I
> >> would
> >> create a category for connectors though.
> > 
> > I'm not disagreeing, but what is actually the distinction between types
> > and subtypes?  ;-)
> 
> The type tells what the behavior is of an entity. E.g., type DEVNODE
> represents device node(s) in userspace, V4L2_SUBDEV represents a v4l2
> sub-device, etc. The subtype tells whether a V4L2_SUBDEV is a sensor or a
> receiver or whatever. Nice to know, but it doesn't change the way
> sub-devices work.
> 
> In the case of connectors you would create a CONNECTOR type and have a
> bunch of subtypes for all the variations of connectors.
> 
> That said, I'm not sure whether the distinction is useful for DEVNODEs.
> You do need to know the subtype in order to interpret the union correctly.
> 
> Laurent, does the MC code test against the DEVNODE type? I.e., does the MC
> code ignore the subtype of a DEVNODE, or does it always use it?

The MC code uses the DEVNODE type, ignoring the subtype, for power management. 
When a device node is opened all entities in the chain need to be powered up.

-- 
Regards,

Laurent Pinchart

Re: [alsa-devel] [RFC/PATCH v6 03/12] media: Entities, pads and links

[Sakari Ailus]

Hi Clemens, Laurent, Hans & others!

Clemens Ladisch wrote:
> I wrote:
>> I'll see if I can draw up the ALSA-specific media stuff over the weekend.
> 
> Sorry, wrong weekend.
> 
> Anyway, below are some remarks and a patch.
> 
> 
> * Entity types
> 
> TYPE_NODE was renamed to TYPE_DEVICE because "node" sounds like a node
> in a graph, which does not distinguish it from other entity types
> because all entities are part of the topology graph.  I chose "device"
> as this type describes entities that are visible as some device node to
> other software.
> 
> TYPE_EXT describes entities that represent some interface to the
> external world, TYPE_INT those that are internal to the entire device.
> (I'm not sure if that distinction is very useful, but TYPE_SUBDEV seems
> to be an even more meaningless name.)
> 
> 
> ALSA mixer controls are not directly represented; a better fit for the
> architecture of actual devices is that one or more mixer controls can be
> associated with an entity.  (This can be done with a field of the mixer
> control.)
> 
> 
> * Entity properties
> 
> There needs to be a mechanism to associate meta-information (properties)
> with entities.  This information should be optional and extensible, but,
> when being handled inside the kernel, doesn't need to be more than
> a read-only blob.  I think that something like ALSA's TLV format (used
> for mixer controls) can be used here.  (I'm not mentioning the X-word
> here, except to note that the "M" stands for "markup".)
> 
> 
> * Entity subtypes
> 
> EXT_JACK_ANALOG represents any analog audio and/or video connector.
> Properties for audio jacks would be jack type (TRS/RCA), color code,
> line level, position, etc.
> 
> EXT_JACK_DIGITAL represents a digital connector like S/PDIF (coax/
> TOSLINK), ADAT, TDIF, or MADI.
> 
> EXT_JACK_BUS represents a bus like FireWire and comes from the USB audio
> spec.  (I doubt that any devices with this entitiy will ever exist.)
> 
> EXT_INSTRUMENT represents something like an e-guitar, keyboard, or MIDI
> controller.  (Instrument entities are typically audio sources and MIDI
> sources and sinks, but can also be audio sinks.)
> 
> EXT_SPEAKER also includes headphones; there might be made a case for
> having those as a separate subtype.
> 
> EXT_PLAYER represents a device like a CD/DVD/tape player.  Recorders can
> also write to that device, so "player" might not be an ideal name.
> 
> EXT_BROADCAST represents devices like TV tuners, satellite receivers,
> cable tuners, or radios.
> 
> INT_SYNTHESIZER converts MIDI to audio.
> 
> INT_NOISE_SOURCE comes from the USB audio spec; this is not an attempt
> to describe the characteristics of consumer-grade devices :-) , but
> represents an internal noise source for level calibration or measurements.
> 
> INT_CONTROLS may have multiple independent controls (this is USB's
> Feature Unit); INT_EFFECT may have multiple controls that affect one
> single algorithm.
> 
> INT_CHANNEL_SPLIT/MERGE are needed for HDAudio devices, whose topology
> information has only stereo links.

This naming already has been commented, but what do you think: should
the type explicitly tell what kind of interface, if any, is exported to
user space?

Only MEDIA_ENTITY_NODE_* types do this currently, and
MEDIA_ENTITY_TYPE_SUBDEV_* to some extent, but the way is not consistent
at the moment. MEDIA_ENTITY_NODE_* range has lost of different
interfaces whereas MEDIA_ENTITY_TYPE_SUBDEV_* are basically offering
v4l2_subdev and beyond that, suggesting what kind of controls might be
found from the nodes.

I would expect that the interfaces offered by the character devices
would be somewhat standardised in the end like v4l2_subdev user space
interface.

The types above are mostly describing the role of an entity, which might
be interesting as well.

Regards,

-- 
Sakari Ailus
sakari.ailus@maxwell.research.nokia.com