[PATCH v6 00/10] Renesas VSP1 driver

[PATCH v6 00/10] Renesas VSP1 driver

[Laurent Pinchart]

Hello,

Here's the sixth version of the VSP1 engine (a Video Signal Processor found
in several Renesas R-Car SoCs) driver. This version adds a videobuf2 mmap
locking fix and improves the documentation patch (thanks to Hans for the
discussion we had on IRC, hopefully we won't need another round now).

The VSP1 is a video processing engine that includes a blender, scalers,
filters and statistics computation. Configurable data path routing logic
allows ordering the internal blocks in a flexible way, making this driver a
prime candidate for the media controller API.

Due to the configurable nature of the pipeline the driver doesn't use the V4L2
mem-to-mem framework, even though the device usually operates in memory to
memory mode.

Only the read pixel formatters, up/down scalers, write pixel formatters and
LCDC interface are supported at this stage.

The patch series starts with a fix for the media controller graph traversal
code, three documentation fixes, a videobuf2 mmap locking fix  and new pixel
format and media bus code definitions. The last three patches finally add the
VSP1 driver and fix two issues (I haven't squashed the patches together to
keep proper attribution).

Changes since v1:

- Updated to the v3.11 media controller API changes
- Only add the LIF entity to the entities list when the LIF is present
- Added a MODULE_ALIAS()
- Fixed file descriptions in comment blocks
- Removed function prototypes for the unimplemented destroy functions
- Fixed a typo in the HST register name
- Fixed format propagation for the UDS entities
- Added v4l2_capability::device_caps support
- Prefix the device name with "platform:" in bus_info
- Zero the v4l2_pix_format priv field in the internal try format handler
- Use vb2_is_busy() instead of vb2_is_streaming() when setting the
  format
- Use the vb2_ioctl_* handlers where possible

Changes since v2:

- Use a bitmap to track visited entities during graph traversal
- Fixed a typo in the V4L2_MBUS_FMT_ARGB888_1X32 documentation
- Fix register macros that were missing a n argument
- Mask unused bits when clearing the interrupt status register
- Explain why stride alignment to 128 bytes is needed
- Use the aligned stride value when computing the image size
- Assorted cosmetic changes

Changes since v3:

- Handle timeout errors when resetting WPFs
- Use DECLARE_BITMAP
- Update the NV16M/NV61M documentation to mention the multi-planar API for
  NV61M

Changes since v4:

- Clarify the VIDIOC_CREATE_BUFS format requirements in the V4L2
  documentation
- Clarify vb2 queue_setup() and buf_prepare() usage documentation
- Remove duplicate printk's in devm_* error paths
- Implement VIDIOC_CREATE_BUFS and VIDIOC_PREPARE_BUF support
- Reject invalid buffers in the .buffer_queue() handler

Changes since v5:

- Use the vb2 fop helpers
- Reword the VIDIOC_CREATE_BUFS clarification
- Take the queue or device lock in vb2_fop_mmap()
- Accept custom sizeimage values in .queue_setup()

Katsuya Matsubara (2):
  vsp1: Fix lack of the sink entity registration for enabled links
  vsp1: Use the maximum number of entities defined in platform data

Laurent Pinchart (8):
  media: Add support for circular graph traversal
  Documentation: media: Clarify the VIDIOC_CREATE_BUFS format
    requirements
  media: vb2: Clarify queue_setup() and buf_prepare() usage
    documentation
  media: vb2: Take queue or device lock in vb2_fop_mmap()
  v4l: Fix V4L2_MBUS_FMT_YUV10_1X30 media bus pixel code value
  v4l: Add media format codes for ARGB8888 and AYUV8888 on 32-bit busses
  v4l: Add V4L2_PIX_FMT_NV16M and V4L2_PIX_FMT_NV61M formats
  v4l: Renesas R-Car VSP1 driver

 Documentation/DocBook/media/v4l/pixfmt-nv16m.xml   |  171 ++++
 Documentation/DocBook/media/v4l/pixfmt.xml         |    1 +
 Documentation/DocBook/media/v4l/subdev-formats.xml |  611 +++++------
 .../DocBook/media/v4l/vidioc-create-bufs.xml       |   41 +-
 Documentation/DocBook/media_api.tmpl               |    6 +
 drivers/media/media-entity.c                       |   14 +-
 drivers/media/platform/Kconfig                     |   10 +
 drivers/media/platform/Makefile                    |    2 +
 drivers/media/platform/vsp1/Makefile               |    5 +
 drivers/media/platform/vsp1/vsp1.h                 |   73 ++
 drivers/media/platform/vsp1/vsp1_drv.c             |  495 +++++++++
 drivers/media/platform/vsp1/vsp1_entity.c          |  181 ++++
 drivers/media/platform/vsp1/vsp1_entity.h          |   68 ++
 drivers/media/platform/vsp1/vsp1_lif.c             |  238 +++++
 drivers/media/platform/vsp1/vsp1_lif.h             |   37 +
 drivers/media/platform/vsp1/vsp1_regs.h            |  581 +++++++++++
 drivers/media/platform/vsp1/vsp1_rpf.c             |  209 ++++
 drivers/media/platform/vsp1/vsp1_rwpf.c            |  124 +++
 drivers/media/platform/vsp1/vsp1_rwpf.h            |   53 +
 drivers/media/platform/vsp1/vsp1_uds.c             |  346 +++++++
 drivers/media/platform/vsp1/vsp1_uds.h             |   40 +
 drivers/media/platform/vsp1/vsp1_video.c           | 1071 ++++++++++++++++++++
 drivers/media/platform/vsp1/vsp1_video.h           |  144 +++
 drivers/media/platform/vsp1/vsp1_wpf.c             |  233 +++++
 drivers/media/v4l2-core/videobuf2-core.c           |    9 +-
 include/linux/platform_data/vsp1.h                 |   25 +
 include/media/media-entity.h                       |    4 +
 include/media/videobuf2-core.h                     |   11 +-
 include/uapi/linux/v4l2-mediabus.h                 |    6 +-
 include/uapi/linux/videodev2.h                     |    2 +
 30 files changed, 4420 insertions(+), 391 deletions(-)
 create mode 100644 Documentation/DocBook/media/v4l/pixfmt-nv16m.xml
 create mode 100644 drivers/media/platform/vsp1/Makefile
 create mode 100644 drivers/media/platform/vsp1/vsp1.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_drv.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_entity.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_entity.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_lif.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_lif.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_regs.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_rpf.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_rwpf.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_rwpf.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_uds.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_uds.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_video.c
 create mode 100644 drivers/media/platform/vsp1/vsp1_video.h
 create mode 100644 drivers/media/platform/vsp1/vsp1_wpf.c
 create mode 100644 include/linux/platform_data/vsp1.h

-- 
Regards,

Laurent Pinchart

[PATCH v6 01/10] media: Add support for circular graph traversal

[Laurent Pinchart]

The graph traversal API (media_entity_graph_walk_*) doesn't support
cyclic graphs and will fail to correctly walk a graph when circular
links exist. Support circular graph traversal by checking whether an
entity has already been visited before pushing it to the stack.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/media-entity.c | 14 +++++++++++---
 include/media/media-entity.h |  4 ++++
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index cb30ffb..2c286c3 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -20,6 +20,7 @@
  * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
  */
 
+#include <linux/bitmap.h>
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <media/media-entity.h>
@@ -121,7 +122,6 @@ static struct media_entity *stack_pop(struct media_entity_graph *graph)
 	return entity;
 }
 
-#define stack_peek(en)	((en)->stack[(en)->top - 1].entity)
 #define link_top(en)	((en)->stack[(en)->top].link)
 #define stack_top(en)	((en)->stack[(en)->top].entity)
 
@@ -140,6 +140,12 @@ void media_entity_graph_walk_start(struct media_entity_graph *graph,
 {
 	graph->top = 0;
 	graph->stack[graph->top].entity = NULL;
+	bitmap_zero(graph->entities, MEDIA_ENTITY_ENUM_MAX_ID);
+
+	if (WARN_ON(entity->id >= MEDIA_ENTITY_ENUM_MAX_ID))
+		return;
+
+	__set_bit(entity->id, graph->entities);
 	stack_push(graph, entity);
 }
 EXPORT_SYMBOL_GPL(media_entity_graph_walk_start);
@@ -180,9 +186,11 @@ media_entity_graph_walk_next(struct media_entity_graph *graph)
 
 		/* Get the entity in the other end of the link . */
 		next = media_entity_other(entity, link);
+		if (WARN_ON(next->id >= MEDIA_ENTITY_ENUM_MAX_ID))
+			return NULL;
 
-		/* Was it the entity we came here from? */
-		if (next = stack_peek(graph)) {
+		/* Has the entity already been visited? */
+		if (__test_and_set_bit(next->id, graph->entities)) {
 			link_top(graph)++;
 			continue;
 		}
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 06bacf9..10df551 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -23,6 +23,7 @@
 #ifndef _MEDIA_ENTITY_H
 #define _MEDIA_ENTITY_H
 
+#include <linux/bitops.h>
 #include <linux/list.h>
 #include <linux/media.h>
 
@@ -113,12 +114,15 @@ static inline u32 media_entity_subtype(struct media_entity *entity)
 }
 
 #define MEDIA_ENTITY_ENUM_MAX_DEPTH	16
+#define MEDIA_ENTITY_ENUM_MAX_ID	64
 
 struct media_entity_graph {
 	struct {
 		struct media_entity *entity;
 		int link;
 	} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
+
+	DECLARE_BITMAP(entities, MEDIA_ENTITY_ENUM_MAX_ID);
 	int top;
 };
 
-- 
1.8.1.5

[PATCH v6 02/10] Documentation: media: Clarify the VIDIOC_CREATE_BUFS format requirements

[Laurent Pinchart]

The VIDIOC_CREATE_BUFS ioctl takes a format argument that must contain a
valid format supported by the driver. Clarify the documentation.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 .../DocBook/media/v4l/vidioc-create-bufs.xml       | 41 ++++++++++++++--------
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml b/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
index cd99436..9b700a5 100644
--- a/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
+++ b/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
@@ -62,18 +62,29 @@ addition to the <constant>VIDIOC_REQBUFS</constant> ioctl, when a tighter
 control over buffers is required. This ioctl can be called multiple times to
 create buffers of different sizes.</para>
 
-    <para>To allocate device buffers applications initialize relevant fields of
-the <structname>v4l2_create_buffers</structname> structure. They set the
-<structfield>type</structfield> field in the
-&v4l2-format; structure, embedded in this
-structure, to the respective stream or buffer type.
-<structfield>count</structfield> must be set to the number of required buffers.
-<structfield>memory</structfield> specifies the required I/O method. The
-<structfield>format</structfield> field shall typically be filled in using
-either the <constant>VIDIOC_TRY_FMT</constant> or
-<constant>VIDIOC_G_FMT</constant> ioctl(). Additionally, applications can adjust
-<structfield>sizeimage</structfield> fields to fit their specific needs. The
-<structfield>reserved</structfield> array must be zeroed.</para>
+    <para>To allocate the device buffers applications must initialize the
+relevant fields of the <structname>v4l2_create_buffers</structname> structure.
+The <structfield>count</structfield> field must be set to the number of
+requested buffers, the <structfield>memory</structfield> field specifies the
+requested I/O method and the <structfield>reserved</structfield> array must be
+zeroed.</para>
+
+    <para>The <structfield>format</structfield> field specifies the image format
+that the buffers must be able to handle. The application has to fill in this
+&v4l2-format;. Usually this will be done using the
+<constant>VIDIOC_TRY_FMT</constant> or <constant>VIDIOC_G_FMT</constant> ioctl()
+to ensure that the requested format is supported by the driver. Unsupported
+formats will result in an error.</para>
+
+    <para>The buffers created by this ioctl will have as minimum size the size
+defined by the <structfield>format.pix.sizeimage</structfield> field. If the
+<structfield>format.pix.sizeimage</structfield> field is less than the minimum
+required for the given format, then <structfield>sizeimage</structfield> will be
+increased by the driver to that minimum to allocate the buffers. If it is
+larger, then the value will be used as-is. The same applies to the
+<structfield>sizeimage</structfield> field of the
+<structname>v4l2_plane_pix_format</structname> structure in the case of
+multiplanar formats.</para>
 
     <para>When the ioctl is called with a pointer to this structure the driver
 will attempt to allocate up to the requested number of buffers and store the
@@ -144,9 +155,9 @@ mapped</link> I/O.</para>
       <varlistentry>
 	<term><errorcode>EINVAL</errorcode></term>
 	<listitem>
-	  <para>The buffer type (<structfield>type</structfield> field) or the
-requested I/O method (<structfield>memory</structfield>) is not
-supported.</para>
+	  <para>The buffer type (<structfield>format.type</structfield> field),
+requested I/O method (<structfield>memory</structfield>) or format
+(<structfield>format</structfield> field) is not valid.</para>
 	</listitem>
       </varlistentry>
     </variablelist>
-- 
1.8.1.5

Re: [PATCH v6 02/10] Documentation: media: Clarify the VIDIOC_CREATE_BUFS format requirements

[Hans Verkuil]

On Mon 5 August 2013 19:53:21 Laurent Pinchart wrote:
> The VIDIOC_CREATE_BUFS ioctl takes a format argument that must contain a
> valid format supported by the driver. Clarify the documentation.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

Acked-by: Hans Verkuil <hans.verkuil@cisco.com>

Regards,

	Hans

> ---
>  .../DocBook/media/v4l/vidioc-create-bufs.xml       | 41 ++++++++++++++--------
>  1 file changed, 26 insertions(+), 15 deletions(-)
> 
> diff --git a/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml b/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
> index cd99436..9b700a5 100644
> --- a/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
> +++ b/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml
> @@ -62,18 +62,29 @@ addition to the <constant>VIDIOC_REQBUFS</constant> ioctl, when a tighter
>  control over buffers is required. This ioctl can be called multiple times to
>  create buffers of different sizes.</para>
>  
> -    <para>To allocate device buffers applications initialize relevant fields of
> -the <structname>v4l2_create_buffers</structname> structure. They set the
> -<structfield>type</structfield> field in the
> -&v4l2-format; structure, embedded in this
> -structure, to the respective stream or buffer type.
> -<structfield>count</structfield> must be set to the number of required buffers.
> -<structfield>memory</structfield> specifies the required I/O method. The
> -<structfield>format</structfield> field shall typically be filled in using
> -either the <constant>VIDIOC_TRY_FMT</constant> or
> -<constant>VIDIOC_G_FMT</constant> ioctl(). Additionally, applications can adjust
> -<structfield>sizeimage</structfield> fields to fit their specific needs. The
> -<structfield>reserved</structfield> array must be zeroed.</para>
> +    <para>To allocate the device buffers applications must initialize the
> +relevant fields of the <structname>v4l2_create_buffers</structname> structure.
> +The <structfield>count</structfield> field must be set to the number of
> +requested buffers, the <structfield>memory</structfield> field specifies the
> +requested I/O method and the <structfield>reserved</structfield> array must be
> +zeroed.</para>
> +
> +    <para>The <structfield>format</structfield> field specifies the image format
> +that the buffers must be able to handle. The application has to fill in this
> +&v4l2-format;. Usually this will be done using the
> +<constant>VIDIOC_TRY_FMT</constant> or <constant>VIDIOC_G_FMT</constant> ioctl()
> +to ensure that the requested format is supported by the driver. Unsupported
> +formats will result in an error.</para>
> +
> +    <para>The buffers created by this ioctl will have as minimum size the size
> +defined by the <structfield>format.pix.sizeimage</structfield> field. If the
> +<structfield>format.pix.sizeimage</structfield> field is less than the minimum
> +required for the given format, then <structfield>sizeimage</structfield> will be
> +increased by the driver to that minimum to allocate the buffers. If it is
> +larger, then the value will be used as-is. The same applies to the
> +<structfield>sizeimage</structfield> field of the
> +<structname>v4l2_plane_pix_format</structname> structure in the case of
> +multiplanar formats.</para>
>  
>      <para>When the ioctl is called with a pointer to this structure the driver
>  will attempt to allocate up to the requested number of buffers and store the
> @@ -144,9 +155,9 @@ mapped</link> I/O.</para>
>        <varlistentry>
>  	<term><errorcode>EINVAL</errorcode></term>
>  	<listitem>
> -	  <para>The buffer type (<structfield>type</structfield> field) or the
> -requested I/O method (<structfield>memory</structfield>) is not
> -supported.</para>
> +	  <para>The buffer type (<structfield>format.type</structfield> field),
> +requested I/O method (<structfield>memory</structfield>) or format
> +(<structfield>format</structfield> field) is not valid.</para>
>  	</listitem>
>        </varlistentry>
>      </variablelist>
> 

[PATCH v6 03/10] media: vb2: Clarify queue_setup() and buf_prepare() usage documentation

[Laurent Pinchart]

Explain how the two operations must handle formats and validate buffer
sizes when used with VIDIOC_CREATE_BUFS.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 include/media/videobuf2-core.h | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index d88a098..6781258 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -219,8 +219,9 @@ struct vb2_buffer {
  *			configured format and *num_buffers is the total number
  *			of buffers, that are being allocated. When called from
  *			VIDIOC_CREATE_BUFS, fmt != NULL and it describes the
- *			target frame format. In this case *num_buffers are being
- *			allocated additionally to q->num_buffers.
+ *			target frame format (if the format isn't valid the
+ *			callback must return -EINVAL). In this case *num_buffers
+ *			are being allocated additionally to q->num_buffers.
  * @wait_prepare:	release any locks taken while calling vb2 functions;
  *			it is called before an ioctl needs to wait for a new
  *			buffer to arrive; required to avoid a deadlock in
@@ -236,8 +237,10 @@ struct vb2_buffer {
  * @buf_prepare:	called every time the buffer is queued from userspace
  *			and from the VIDIOC_PREPARE_BUF ioctl; drivers may
  *			perform any initialization required before each hardware
- *			operation in this callback; if an error is returned, the
- *			buffer will not be queued in driver; optional
+ *			operation in this callback; drivers that support
+ *			VIDIOC_CREATE_BUFS must also validate the buffer size;
+ *			if an error is returned, the buffer will not be queued
+ *			in driver; optional
  * @buf_finish:		called before every dequeue of the buffer back to
  *			userspace; drivers may perform any operations required
  *			before userspace accesses the buffer; optional
-- 
1.8.1.5

[PATCH v6 04/10] media: vb2: Take queue or device lock in vb2_fop_mmap()

[Laurent Pinchart]

The vb2_fop_mmap() function is a plug-in implementation of the mmap()
file operation that calls vb2_mmap() on the queue associated with the
video device. Neither the vb2_fop_mmap() function nor the v4l2_mmap()
mmap handler in the V4L2 core take any lock, leading to race conditions
between mmap() and other buffer-related ioctls such as VIDIOC_REQBUFS.

Fix it by taking the queue or device lock around the vb2_mmap() call.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/v4l2-core/videobuf2-core.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
index 9fc4bab..bd4bade 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
 int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
 {
 	struct video_device *vdev = video_devdata(file);
+	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
+	int err;
 
-	return vb2_mmap(vdev->queue, vma);
+	if (lock && mutex_lock_interruptible(lock))
+		return -ERESTARTSYS;
+	err = vb2_mmap(vdev->queue, vma);
+	if (lock)
+		mutex_unlock(lock);
+	return err;
 }
 EXPORT_SYMBOL_GPL(vb2_fop_mmap);
 
-- 
1.8.1.5

Re: [PATCH v6 04/10] media: vb2: Take queue or device lock in vb2_fop_mmap()

[Hans Verkuil]

On Mon 5 August 2013 19:53:23 Laurent Pinchart wrote:
> The vb2_fop_mmap() function is a plug-in implementation of the mmap()
> file operation that calls vb2_mmap() on the queue associated with the
> video device. Neither the vb2_fop_mmap() function nor the v4l2_mmap()
> mmap handler in the V4L2 core take any lock, leading to race conditions
> between mmap() and other buffer-related ioctls such as VIDIOC_REQBUFS.
> 
> Fix it by taking the queue or device lock around the vb2_mmap() call.

Hi Laurent,

Can you do the same for vb2_fop_get_unmapped_area()?

Thanks!

	Hans

> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/videobuf2-core.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
> index 9fc4bab..bd4bade 100644
> --- a/drivers/media/v4l2-core/videobuf2-core.c
> +++ b/drivers/media/v4l2-core/videobuf2-core.c
> @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
>  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
>  {
>  	struct video_device *vdev = video_devdata(file);
> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
> +	int err;
>  
> -	return vb2_mmap(vdev->queue, vma);
> +	if (lock && mutex_lock_interruptible(lock))
> +		return -ERESTARTSYS;
> +	err = vb2_mmap(vdev->queue, vma);
> +	if (lock)
> +		mutex_unlock(lock);
> +	return err;
>  }
>  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
>  
> 

Re: [PATCH v6 04/10] media: vb2: Take queue or device lock in vb2_fop_mmap()

[Laurent Pinchart]

Hi Hans,

On Tuesday 06 August 2013 12:39:27 Hans Verkuil wrote:
> On Mon 5 August 2013 19:53:23 Laurent Pinchart wrote:
> > The vb2_fop_mmap() function is a plug-in implementation of the mmap()
> > file operation that calls vb2_mmap() on the queue associated with the
> > video device. Neither the vb2_fop_mmap() function nor the v4l2_mmap()
> > mmap handler in the V4L2 core take any lock, leading to race conditions
> > between mmap() and other buffer-related ioctls such as VIDIOC_REQBUFS.
> > 
> > Fix it by taking the queue or device lock around the vb2_mmap() call.
> 
> Hi Laurent,
> 
> Can you do the same for vb2_fop_get_unmapped_area()?

Sure. I'll repost a v7 of this patch that fixes both mmap and 
get_unmapped_area.

> > Signed-off-by: Laurent Pinchart
> > <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> > 
> >  drivers/media/v4l2-core/videobuf2-core.c | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/v4l2-core/videobuf2-core.c
> > b/drivers/media/v4l2-core/videobuf2-core.c index 9fc4bab..bd4bade 100644
> > --- a/drivers/media/v4l2-core/videobuf2-core.c
> > +++ b/drivers/media/v4l2-core/videobuf2-core.c
> > @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
> > 
> >  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
> >  {
> >  
> >  	struct video_device *vdev = video_devdata(file);
> > 
> > +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev-
>lock;
> > +	int err;
> > 
> > -	return vb2_mmap(vdev->queue, vma);
> > +	if (lock && mutex_lock_interruptible(lock))
> > +		return -ERESTARTSYS;
> > +	err = vb2_mmap(vdev->queue, vma);
> > +	if (lock)
> > +		mutex_unlock(lock);
> > +	return err;
> > 
> >  }
> >  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
-- 
Regards,

Laurent Pinchart

[PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Laurent Pinchart]

The vb2_fop_mmap() and vb2_fop_get_unmapped_area() functions are plug-in
implementation of the mmap() and get_unmapped_area() file operations
that calls vb2_mmap() and vb2_get_unmapped_area() on the queue
associated with the video device. Neither the
vb2_fop_mmap/vb2_fop_get_unmapped_area nor the
v4l2_mmap/vb2_get_unmapped_area functions in the V4L2 core take any
lock, leading to race conditions between mmap/get_unmapped_area and
other buffer-related ioctls such as VIDIOC_REQBUFS.

Fix it by taking the queue or device lock around the vb2_mmap() and
vb2_get_unmapped_area() calls.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/v4l2-core/videobuf2-core.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
index 9fc4bab..c9b50c7 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
 int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
 {
 	struct video_device *vdev = video_devdata(file);
+	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
+	int err;
 
-	return vb2_mmap(vdev->queue, vma);
+	if (lock && mutex_lock_interruptible(lock))
+		return -ERESTARTSYS;
+	err = vb2_mmap(vdev->queue, vma);
+	if (lock)
+		mutex_unlock(lock);
+	return err;
 }
 EXPORT_SYMBOL_GPL(vb2_fop_mmap);
 
@@ -2685,8 +2692,15 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, unsigned long addr,
 		unsigned long len, unsigned long pgoff, unsigned long flags)
 {
 	struct video_device *vdev = video_devdata(file);
+	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
+	int ret;
 
-	return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
+	if (lock && mutex_lock_interruptible(lock))
+		return -ERESTARTSYS;
+	ret = vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
+	if (lock)
+		mutex_unlock(lock);
+	return ret;
 }
 EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
 #endif
-- 
Regards,

Laurent Pinchart

Re: [PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Hans Verkuil]

On 08/06/2013 10:10 PM, Laurent Pinchart wrote:
> The vb2_fop_mmap() and vb2_fop_get_unmapped_area() functions are plug-in
> implementation of the mmap() and get_unmapped_area() file operations
> that calls vb2_mmap() and vb2_get_unmapped_area() on the queue
> associated with the video device. Neither the
> vb2_fop_mmap/vb2_fop_get_unmapped_area nor the
> v4l2_mmap/vb2_get_unmapped_area functions in the V4L2 core take any
> lock, leading to race conditions between mmap/get_unmapped_area and
> other buffer-related ioctls such as VIDIOC_REQBUFS.
> 
> Fix it by taking the queue or device lock around the vb2_mmap() and
> vb2_get_unmapped_area() calls.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

Acked-by: Hans Verkuil <hans.verkuil@cisco.com>

Thanks!

	Hans

> ---
>  drivers/media/v4l2-core/videobuf2-core.c | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
> index 9fc4bab..c9b50c7 100644
> --- a/drivers/media/v4l2-core/videobuf2-core.c
> +++ b/drivers/media/v4l2-core/videobuf2-core.c
> @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
>  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
>  {
>  	struct video_device *vdev = video_devdata(file);
> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
> +	int err;
>  
> -	return vb2_mmap(vdev->queue, vma);
> +	if (lock && mutex_lock_interruptible(lock))
> +		return -ERESTARTSYS;
> +	err = vb2_mmap(vdev->queue, vma);
> +	if (lock)
> +		mutex_unlock(lock);
> +	return err;
>  }
>  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
>  
> @@ -2685,8 +2692,15 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, unsigned long addr,
>  		unsigned long len, unsigned long pgoff, unsigned long flags)
>  {
>  	struct video_device *vdev = video_devdata(file);
> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
> +	int ret;
>  
> -	return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> +	if (lock && mutex_lock_interruptible(lock))
> +		return -ERESTARTSYS;
> +	ret = vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> +	if (lock)
> +		mutex_unlock(lock);
> +	return ret;
>  }
>  EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
>  #endif
> 

Re: [PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Hans Verkuil]

Hi Laurent,

This patch caused a circular locking dependency as reported by Sasha Levin:

https://lkml.org/lkml/2014/5/5/366

The reason is that copy_to/from_user is called in video_usercopy() with the
core lock held. The copy functions can fault which takes the mmap_sem. If it
was just video_usercopy() then it would be fairly easy to solve this, but
the copy_to_/from_user functions are also called from read and write and they
can be used in other unexpected places.

I'm not sure if vb2_fop_get_unmapped_area() is a problem. I suspect (but I'm
not sure) that when that one is called the mmap_sem isn't taken, in which
case taking the lock is fine.

But taking the lock in vb2_fop_mmap() does cause lockdep problems.

Ideally I would like to drop taking that lock in vb2_fop_mmap and resolve the
race condition that it intended to fix in a different way.

Regards,

	Hans

On 08/06/2013 10:10 PM, Laurent Pinchart wrote:
> The vb2_fop_mmap() and vb2_fop_get_unmapped_area() functions are plug-in
> implementation of the mmap() and get_unmapped_area() file operations
> that calls vb2_mmap() and vb2_get_unmapped_area() on the queue
> associated with the video device. Neither the
> vb2_fop_mmap/vb2_fop_get_unmapped_area nor the
> v4l2_mmap/vb2_get_unmapped_area functions in the V4L2 core take any
> lock, leading to race conditions between mmap/get_unmapped_area and
> other buffer-related ioctls such as VIDIOC_REQBUFS.
> 
> Fix it by taking the queue or device lock around the vb2_mmap() and
> vb2_get_unmapped_area() calls.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/videobuf2-core.c | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
> index 9fc4bab..c9b50c7 100644
> --- a/drivers/media/v4l2-core/videobuf2-core.c
> +++ b/drivers/media/v4l2-core/videobuf2-core.c
> @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
>  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
>  {
>  	struct video_device *vdev = video_devdata(file);
> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
> +	int err;
>  
> -	return vb2_mmap(vdev->queue, vma);
> +	if (lock && mutex_lock_interruptible(lock))
> +		return -ERESTARTSYS;
> +	err = vb2_mmap(vdev->queue, vma);
> +	if (lock)
> +		mutex_unlock(lock);
> +	return err;
>  }
>  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
>  
> @@ -2685,8 +2692,15 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, unsigned long addr,
>  		unsigned long len, unsigned long pgoff, unsigned long flags)
>  {
>  	struct video_device *vdev = video_devdata(file);
> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
> +	int ret;
>  
> -	return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> +	if (lock && mutex_lock_interruptible(lock))
> +		return -ERESTARTSYS;
> +	ret = vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> +	if (lock)
> +		mutex_unlock(lock);
> +	return ret;
>  }
>  EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
>  #endif
> 

Re: [PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Laurent Pinchart]

Hi Hans,

On Friday 23 May 2014 15:54:29 Hans Verkuil wrote:
> Hi Laurent,
> 
> This patch caused a circular locking dependency as reported by Sasha Levin:
> 
> https://lkml.org/lkml/2014/5/5/366
> 
> The reason is that copy_to/from_user is called in video_usercopy() with the
> core lock held. The copy functions can fault which takes the mmap_sem. If it
> was just video_usercopy() then it would be fairly easy to solve this, but
> the copy_to_/from_user functions are also called from read and write and
> they can be used in other unexpected places.
> 
> I'm not sure if vb2_fop_get_unmapped_area() is a problem. I suspect (but I'm
> not sure) that when that one is called the mmap_sem isn't taken, in which
> case taking the lock is fine.
> 
> But taking the lock in vb2_fop_mmap() does cause lockdep problems.
> 
> Ideally I would like to drop taking that lock in vb2_fop_mmap and resolve
> the race condition that it intended to fix in a different way.

I'm of course fine with a different way to solve the race condition, if we can 
find a good one. Do you already know how you would like to solve this ?

> On 08/06/2013 10:10 PM, Laurent Pinchart wrote:
> > The vb2_fop_mmap() and vb2_fop_get_unmapped_area() functions are plug-in
> > implementation of the mmap() and get_unmapped_area() file operations
> > that calls vb2_mmap() and vb2_get_unmapped_area() on the queue
> > associated with the video device. Neither the
> > vb2_fop_mmap/vb2_fop_get_unmapped_area nor the
> > v4l2_mmap/vb2_get_unmapped_area functions in the V4L2 core take any
> > lock, leading to race conditions between mmap/get_unmapped_area and
> > other buffer-related ioctls such as VIDIOC_REQBUFS.
> > 
> > Fix it by taking the queue or device lock around the vb2_mmap() and
> > vb2_get_unmapped_area() calls.
> > 
> > Signed-off-by: Laurent Pinchart
> > <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> > 
> >  drivers/media/v4l2-core/videobuf2-core.c | 18 ++++++++++++++++--
> >  1 file changed, 16 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/media/v4l2-core/videobuf2-core.c
> > b/drivers/media/v4l2-core/videobuf2-core.c index 9fc4bab..c9b50c7 100644
> > --- a/drivers/media/v4l2-core/videobuf2-core.c
> > +++ b/drivers/media/v4l2-core/videobuf2-core.c
> > @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
> > 
> >  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
> >  {
> >  
> >  	struct video_device *vdev = video_devdata(file);
> > 
> > +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev-
>lock;
> > +	int err;
> > 
> > -	return vb2_mmap(vdev->queue, vma);
> > +	if (lock && mutex_lock_interruptible(lock))
> > +		return -ERESTARTSYS;
> > +	err = vb2_mmap(vdev->queue, vma);
> > +	if (lock)
> > +		mutex_unlock(lock);
> > +	return err;
> > 
> >  }
> >  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
> > 
> > @@ -2685,8 +2692,15 @@ unsigned long vb2_fop_get_unmapped_area(struct file
> > *file, unsigned long addr,> 
> >  		unsigned long len, unsigned long pgoff, unsigned long flags)
> >  
> >  {
> >  
> >  	struct video_device *vdev = video_devdata(file);
> > 
> > +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev-
>lock;
> > +	int ret;
> > 
> > -	return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> > +	if (lock && mutex_lock_interruptible(lock))
> > +		return -ERESTARTSYS;
> > +	ret = vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> > +	if (lock)
> > +		mutex_unlock(lock);
> > +	return ret;
> > 
> >  }
> >  EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
> >  #endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Sasha Levin]

Ping?

On 05/23/2014 09:54 AM, Hans Verkuil wrote:
> Hi Laurent,
> 
> This patch caused a circular locking dependency as reported by Sasha Levin:
> 
> https://lkml.org/lkml/2014/5/5/366
> 
> The reason is that copy_to/from_user is called in video_usercopy() with the
> core lock held. The copy functions can fault which takes the mmap_sem. If it
> was just video_usercopy() then it would be fairly easy to solve this, but
> the copy_to_/from_user functions are also called from read and write and they
> can be used in other unexpected places.
> 
> I'm not sure if vb2_fop_get_unmapped_area() is a problem. I suspect (but I'm
> not sure) that when that one is called the mmap_sem isn't taken, in which
> case taking the lock is fine.
> 
> But taking the lock in vb2_fop_mmap() does cause lockdep problems.
> 
> Ideally I would like to drop taking that lock in vb2_fop_mmap and resolve the
> race condition that it intended to fix in a different way.
> 
> Regards,
> 
> 	Hans
> 
> On 08/06/2013 10:10 PM, Laurent Pinchart wrote:
>> The vb2_fop_mmap() and vb2_fop_get_unmapped_area() functions are plug-in
>> implementation of the mmap() and get_unmapped_area() file operations
>> that calls vb2_mmap() and vb2_get_unmapped_area() on the queue
>> associated with the video device. Neither the
>> vb2_fop_mmap/vb2_fop_get_unmapped_area nor the
>> v4l2_mmap/vb2_get_unmapped_area functions in the V4L2 core take any
>> lock, leading to race conditions between mmap/get_unmapped_area and
>> other buffer-related ioctls such as VIDIOC_REQBUFS.
>>
>> Fix it by taking the queue or device lock around the vb2_mmap() and
>> vb2_get_unmapped_area() calls.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
>> ---
>>  drivers/media/v4l2-core/videobuf2-core.c | 18 ++++++++++++++++--
>>  1 file changed, 16 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
>> index 9fc4bab..c9b50c7 100644
>> --- a/drivers/media/v4l2-core/videobuf2-core.c
>> +++ b/drivers/media/v4l2-core/videobuf2-core.c
>> @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
>>  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
>>  {
>>  	struct video_device *vdev = video_devdata(file);
>> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
>> +	int err;
>>  
>> -	return vb2_mmap(vdev->queue, vma);
>> +	if (lock && mutex_lock_interruptible(lock))
>> +		return -ERESTARTSYS;
>> +	err = vb2_mmap(vdev->queue, vma);
>> +	if (lock)
>> +		mutex_unlock(lock);
>> +	return err;
>>  }
>>  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
>>  
>> @@ -2685,8 +2692,15 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, unsigned long addr,
>>  		unsigned long len, unsigned long pgoff, unsigned long flags)
>>  {
>>  	struct video_device *vdev = video_devdata(file);
>> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock;
>> +	int ret;
>>  
>> -	return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
>> +	if (lock && mutex_lock_interruptible(lock))
>> +		return -ERESTARTSYS;
>> +	ret = vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
>> +	if (lock)
>> +		mutex_unlock(lock);
>> +	return ret;
>>  }
>>  EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
>>  #endif
>>
> 

Re: [PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Laurent Pinchart]

On Wednesday 25 June 2014 12:58:03 Sasha Levin wrote:
> Ping?

Hans, I've replied to your previous e-mail with

I'm of course fine with a different way to solve the race condition, if we can 
find a good one. Do you already know how you would like to solve this ?

> On 05/23/2014 09:54 AM, Hans Verkuil wrote:
> > Hi Laurent,
> > 
> > This patch caused a circular locking dependency as reported by Sasha
> > Levin:
> > 
> > https://lkml.org/lkml/2014/5/5/366
> > 
> > The reason is that copy_to/from_user is called in video_usercopy() with
> > the
> > core lock held. The copy functions can fault which takes the mmap_sem. If
> > it was just video_usercopy() then it would be fairly easy to solve this,
> > but the copy_to_/from_user functions are also called from read and write
> > and they can be used in other unexpected places.
> > 
> > I'm not sure if vb2_fop_get_unmapped_area() is a problem. I suspect (but
> > I'm not sure) that when that one is called the mmap_sem isn't taken, in
> > which case taking the lock is fine.
> > 
> > But taking the lock in vb2_fop_mmap() does cause lockdep problems.
> > 
> > Ideally I would like to drop taking that lock in vb2_fop_mmap and resolve
> > the race condition that it intended to fix in a different way.
> > 
> > Regards,
> > 
> > 	Hans
> > 
> > On 08/06/2013 10:10 PM, Laurent Pinchart wrote:
> >> The vb2_fop_mmap() and vb2_fop_get_unmapped_area() functions are plug-in
> >> implementation of the mmap() and get_unmapped_area() file operations
> >> that calls vb2_mmap() and vb2_get_unmapped_area() on the queue
> >> associated with the video device. Neither the
> >> vb2_fop_mmap/vb2_fop_get_unmapped_area nor the
> >> v4l2_mmap/vb2_get_unmapped_area functions in the V4L2 core take any
> >> lock, leading to race conditions between mmap/get_unmapped_area and
> >> other buffer-related ioctls such as VIDIOC_REQBUFS.
> >> 
> >> Fix it by taking the queue or device lock around the vb2_mmap() and
> >> vb2_get_unmapped_area() calls.
> >> 
> >> Signed-off-by: Laurent Pinchart
> >> <laurent.pinchart+renesas@ideasonboard.com>
> >> ---
> >> 
> >>  drivers/media/v4l2-core/videobuf2-core.c | 18 ++++++++++++++++--
> >>  1 file changed, 16 insertions(+), 2 deletions(-)
> >> 
> >> diff --git a/drivers/media/v4l2-core/videobuf2-core.c
> >> b/drivers/media/v4l2-core/videobuf2-core.c index 9fc4bab..c9b50c7 100644
> >> --- a/drivers/media/v4l2-core/videobuf2-core.c
> >> +++ b/drivers/media/v4l2-core/videobuf2-core.c
> >> @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
> >>  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
> >>  {
> >>  	struct video_device *vdev = video_devdata(file);
> >> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock :
> >> vdev->lock;
> >> +	int err;
> >> 
> >> -	return vb2_mmap(vdev->queue, vma);
> >> +	if (lock && mutex_lock_interruptible(lock))
> >> +		return -ERESTARTSYS;
> >> +	err = vb2_mmap(vdev->queue, vma);
> >> +	if (lock)
> >> +		mutex_unlock(lock);
> >> +	return err;
> >>  }
> >>  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
> >> 
> >> @@ -2685,8 +2692,15 @@ unsigned long vb2_fop_get_unmapped_area(struct
> >> file *file, unsigned long addr,>> 
> >>  		unsigned long len, unsigned long pgoff, unsigned long flags)
> >>  {
> >>  	struct video_device *vdev = video_devdata(file);
> >> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock :
> >> vdev->lock;
> >> +	int ret;
> >> 
> >> -	return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> >> +	if (lock && mutex_lock_interruptible(lock))
> >> +		return -ERESTARTSYS;
> >> +	ret = vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
> >> +	if (lock)
> >> +		mutex_unlock(lock);
> >> +	return ret;
> >> 
> >>  }
> >>  EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
> >>  #endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Hans Verkuil]

For now it is -ENOTIME for me. It's on my TODO list, but there are a number of
other things I want to finish first.

Regards,

	Hans

On 07/01/2014 11:08 PM, Laurent Pinchart wrote:
> On Wednesday 25 June 2014 12:58:03 Sasha Levin wrote:
>> Ping?
> 
> Hans, I've replied to your previous e-mail with
> 
> I'm of course fine with a different way to solve the race condition, if we can 
> find a good one. Do you already know how you would like to solve this ?
> 
>> On 05/23/2014 09:54 AM, Hans Verkuil wrote:
>>> Hi Laurent,
>>>
>>> This patch caused a circular locking dependency as reported by Sasha
>>> Levin:
>>>
>>> https://lkml.org/lkml/2014/5/5/366
>>>
>>> The reason is that copy_to/from_user is called in video_usercopy() with
>>> the
>>> core lock held. The copy functions can fault which takes the mmap_sem. If
>>> it was just video_usercopy() then it would be fairly easy to solve this,
>>> but the copy_to_/from_user functions are also called from read and write
>>> and they can be used in other unexpected places.
>>>
>>> I'm not sure if vb2_fop_get_unmapped_area() is a problem. I suspect (but
>>> I'm not sure) that when that one is called the mmap_sem isn't taken, in
>>> which case taking the lock is fine.
>>>
>>> But taking the lock in vb2_fop_mmap() does cause lockdep problems.
>>>
>>> Ideally I would like to drop taking that lock in vb2_fop_mmap and resolve
>>> the race condition that it intended to fix in a different way.
>>>
>>> Regards,
>>>
>>> 	Hans
>>>
>>> On 08/06/2013 10:10 PM, Laurent Pinchart wrote:
>>>> The vb2_fop_mmap() and vb2_fop_get_unmapped_area() functions are plug-in
>>>> implementation of the mmap() and get_unmapped_area() file operations
>>>> that calls vb2_mmap() and vb2_get_unmapped_area() on the queue
>>>> associated with the video device. Neither the
>>>> vb2_fop_mmap/vb2_fop_get_unmapped_area nor the
>>>> v4l2_mmap/vb2_get_unmapped_area functions in the V4L2 core take any
>>>> lock, leading to race conditions between mmap/get_unmapped_area and
>>>> other buffer-related ioctls such as VIDIOC_REQBUFS.
>>>>
>>>> Fix it by taking the queue or device lock around the vb2_mmap() and
>>>> vb2_get_unmapped_area() calls.
>>>>
>>>> Signed-off-by: Laurent Pinchart
>>>> <laurent.pinchart+renesas@ideasonboard.com>
>>>> ---
>>>>
>>>>  drivers/media/v4l2-core/videobuf2-core.c | 18 ++++++++++++++++--
>>>>  1 file changed, 16 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/media/v4l2-core/videobuf2-core.c
>>>> b/drivers/media/v4l2-core/videobuf2-core.c index 9fc4bab..c9b50c7 100644
>>>> --- a/drivers/media/v4l2-core/videobuf2-core.c
>>>> +++ b/drivers/media/v4l2-core/videobuf2-core.c
>>>> @@ -2578,8 +2578,15 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf);
>>>>  int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma)
>>>>  {
>>>>  	struct video_device *vdev = video_devdata(file);
>>>> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock :
>>>> vdev->lock;
>>>> +	int err;
>>>>
>>>> -	return vb2_mmap(vdev->queue, vma);
>>>> +	if (lock && mutex_lock_interruptible(lock))
>>>> +		return -ERESTARTSYS;
>>>> +	err = vb2_mmap(vdev->queue, vma);
>>>> +	if (lock)
>>>> +		mutex_unlock(lock);
>>>> +	return err;
>>>>  }
>>>>  EXPORT_SYMBOL_GPL(vb2_fop_mmap);
>>>>
>>>> @@ -2685,8 +2692,15 @@ unsigned long vb2_fop_get_unmapped_area(struct
>>>> file *file, unsigned long addr,>> 
>>>>  		unsigned long len, unsigned long pgoff, unsigned long flags)
>>>>  {
>>>>  	struct video_device *vdev = video_devdata(file);
>>>> +	struct mutex *lock = vdev->queue->lock ? vdev->queue->lock :
>>>> vdev->lock;
>>>> +	int ret;
>>>>
>>>> -	return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
>>>> +	if (lock && mutex_lock_interruptible(lock))
>>>> +		return -ERESTARTSYS;
>>>> +	ret = vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags);
>>>> +	if (lock)
>>>> +		mutex_unlock(lock);
>>>> +	return ret;
>>>>
>>>>  }
>>>>  EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area);
>>>>  #endif
> 

Re: [PATCH v7] media: vb2: Take queue or device lock in mmap-related vb2 ioctl handlers

[Laurent Pinchart]

Hi Hans,

On Wednesday 02 July 2014 08:33:47 Hans Verkuil wrote:
> For now it is -ENOTIME for me. It's on my TODO list, but there are a number
> of other things I want to finish first.

No worries, I know how it feels. I just wanted to make sure you hadn't missed 
my e-mail.

-- 
Regards,

Laurent Pinchart

[PATCH v6 05/10] v4l: Fix V4L2_MBUS_FMT_YUV10_1X30 media bus pixel code value

[Laurent Pinchart]

The V4L2_MBUS_FMT_YUV10_1X30 code is documented as being equal to
0x2014, while the v4l2-mediabus.h header defines it as 0x2016. Fix the
documentation.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 Documentation/DocBook/media/v4l/subdev-formats.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/DocBook/media/v4l/subdev-formats.xml b/Documentation/DocBook/media/v4l/subdev-formats.xml
index adc6198..0c2b1f2 100644
--- a/Documentation/DocBook/media/v4l/subdev-formats.xml
+++ b/Documentation/DocBook/media/v4l/subdev-formats.xml
@@ -2574,7 +2574,7 @@
 	    </row>
 	    <row id="V4L2-MBUS-FMT-YUV10-1X30">
 	      <entry>V4L2_MBUS_FMT_YUV10_1X30</entry>
-	      <entry>0x2014</entry>
+	      <entry>0x2016</entry>
 	      <entry></entry>
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
-- 
1.8.1.5

[PATCH v6 06/10] v4l: Add media format codes for ARGB8888 and AYUV8888 on 32-bit busses

[Laurent Pinchart]

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 Documentation/DocBook/media/v4l/subdev-formats.xml | 609 +++++++++------------
 Documentation/DocBook/media_api.tmpl               |   6 +
 include/uapi/linux/v4l2-mediabus.h                 |   6 +-
 3 files changed, 254 insertions(+), 367 deletions(-)

diff --git a/Documentation/DocBook/media/v4l/subdev-formats.xml b/Documentation/DocBook/media/v4l/subdev-formats.xml
index 0c2b1f2..f72c1cc 100644
--- a/Documentation/DocBook/media/v4l/subdev-formats.xml
+++ b/Documentation/DocBook/media/v4l/subdev-formats.xml
@@ -97,31 +97,39 @@
 	  <colspec colname="id" align="left" />
 	  <colspec colname="code" align="center"/>
 	  <colspec colname="bit" />
-	  <colspec colnum="4" colname="b23" align="center" />
-	  <colspec colnum="5" colname="b22" align="center" />
-	  <colspec colnum="6" colname="b21" align="center" />
-	  <colspec colnum="7" colname="b20" align="center" />
-	  <colspec colnum="8" colname="b19" align="center" />
-	  <colspec colnum="9" colname="b18" align="center" />
-	  <colspec colnum="10" colname="b17" align="center" />
-	  <colspec colnum="11" colname="b16" align="center" />
-	  <colspec colnum="12" colname="b15" align="center" />
-	  <colspec colnum="13" colname="b14" align="center" />
-	  <colspec colnum="14" colname="b13" align="center" />
-	  <colspec colnum="15" colname="b12" align="center" />
-	  <colspec colnum="16" colname="b11" align="center" />
-	  <colspec colnum="17" colname="b10" align="center" />
-	  <colspec colnum="18" colname="b09" align="center" />
-	  <colspec colnum="19" colname="b08" align="center" />
-	  <colspec colnum="20" colname="b07" align="center" />
-	  <colspec colnum="21" colname="b06" align="center" />
-	  <colspec colnum="22" colname="b05" align="center" />
-	  <colspec colnum="23" colname="b04" align="center" />
-	  <colspec colnum="24" colname="b03" align="center" />
-	  <colspec colnum="25" colname="b02" align="center" />
-	  <colspec colnum="26" colname="b01" align="center" />
-	  <colspec colnum="27" colname="b00" align="center" />
-	  <spanspec namest="b23" nameend="b00" spanname="b0" />
+	  <colspec colnum="4" colname="b31" align="center" />
+	  <colspec colnum="5" colname="b20" align="center" />
+	  <colspec colnum="6" colname="b29" align="center" />
+	  <colspec colnum="7" colname="b28" align="center" />
+	  <colspec colnum="8" colname="b27" align="center" />
+	  <colspec colnum="9" colname="b26" align="center" />
+	  <colspec colnum="10" colname="b25" align="center" />
+	  <colspec colnum="11" colname="b24" align="center" />
+	  <colspec colnum="12" colname="b23" align="center" />
+	  <colspec colnum="13" colname="b22" align="center" />
+	  <colspec colnum="14" colname="b21" align="center" />
+	  <colspec colnum="15" colname="b20" align="center" />
+	  <colspec colnum="16" colname="b19" align="center" />
+	  <colspec colnum="17" colname="b18" align="center" />
+	  <colspec colnum="18" colname="b17" align="center" />
+	  <colspec colnum="19" colname="b16" align="center" />
+	  <colspec colnum="20" colname="b15" align="center" />
+	  <colspec colnum="21" colname="b14" align="center" />
+	  <colspec colnum="22" colname="b13" align="center" />
+	  <colspec colnum="23" colname="b12" align="center" />
+	  <colspec colnum="24" colname="b11" align="center" />
+	  <colspec colnum="25" colname="b10" align="center" />
+	  <colspec colnum="26" colname="b09" align="center" />
+	  <colspec colnum="27" colname="b08" align="center" />
+	  <colspec colnum="28" colname="b07" align="center" />
+	  <colspec colnum="29" colname="b06" align="center" />
+	  <colspec colnum="30" colname="b05" align="center" />
+	  <colspec colnum="31" colname="b04" align="center" />
+	  <colspec colnum="32" colname="b03" align="center" />
+	  <colspec colnum="33" colname="b02" align="center" />
+	  <colspec colnum="34" colname="b01" align="center" />
+	  <colspec colnum="35" colname="b00" align="center" />
+	  <spanspec namest="b31" nameend="b00" spanname="b0" />
 	  <thead>
 	    <row>
 	      <entry>Identifier</entry>
@@ -133,6 +141,14 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry>Bit</entry>
+	      <entry>31</entry>
+	      <entry>30</entry>
+	      <entry>29</entry>
+	      <entry>28</entry>
+	      <entry>27</entry>
+	      <entry>26</entry>
+	      <entry>25</entry>
+	      <entry>24</entry>
 	      <entry>23</entry>
 	      <entry>22</entry>
 	      <entry>21</entry>
@@ -164,7 +180,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB444_2X8_PADHI_BE</entry>
 	      <entry>0x1001</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>0</entry>
 	      <entry>0</entry>
 	      <entry>0</entry>
@@ -178,7 +194,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>3</subscript></entry>
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
@@ -192,7 +208,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB444_2X8_PADHI_LE</entry>
 	      <entry>0x1002</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>3</subscript></entry>
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
@@ -206,7 +222,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>0</entry>
 	      <entry>0</entry>
 	      <entry>0</entry>
@@ -220,7 +236,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB555_2X8_PADHI_BE</entry>
 	      <entry>0x1003</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>0</entry>
 	      <entry>r<subscript>4</subscript></entry>
 	      <entry>r<subscript>3</subscript></entry>
@@ -234,7 +250,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
 	      <entry>g<subscript>0</subscript></entry>
@@ -248,7 +264,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB555_2X8_PADHI_LE</entry>
 	      <entry>0x1004</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
 	      <entry>g<subscript>0</subscript></entry>
@@ -262,7 +278,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>0</entry>
 	      <entry>r<subscript>4</subscript></entry>
 	      <entry>r<subscript>3</subscript></entry>
@@ -276,7 +292,7 @@
 	      <entry>V4L2_MBUS_FMT_BGR565_2X8_BE</entry>
 	      <entry>0x1005</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>b<subscript>4</subscript></entry>
 	      <entry>b<subscript>3</subscript></entry>
 	      <entry>b<subscript>2</subscript></entry>
@@ -290,7 +306,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
 	      <entry>g<subscript>0</subscript></entry>
@@ -304,7 +320,7 @@
 	      <entry>V4L2_MBUS_FMT_BGR565_2X8_LE</entry>
 	      <entry>0x1006</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
 	      <entry>g<subscript>0</subscript></entry>
@@ -318,7 +334,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>b<subscript>4</subscript></entry>
 	      <entry>b<subscript>3</subscript></entry>
 	      <entry>b<subscript>2</subscript></entry>
@@ -332,7 +348,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB565_2X8_BE</entry>
 	      <entry>0x1007</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>r<subscript>4</subscript></entry>
 	      <entry>r<subscript>3</subscript></entry>
 	      <entry>r<subscript>2</subscript></entry>
@@ -346,7 +362,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
 	      <entry>g<subscript>0</subscript></entry>
@@ -360,7 +376,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB565_2X8_LE</entry>
 	      <entry>0x1008</entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
 	      <entry>g<subscript>0</subscript></entry>
@@ -374,7 +390,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-16;
+	      &dash-ent-24;
 	      <entry>r<subscript>4</subscript></entry>
 	      <entry>r<subscript>3</subscript></entry>
 	      <entry>r<subscript>2</subscript></entry>
@@ -388,12 +404,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB666_1X18</entry>
 	      <entry>0x1009</entry>
 	      <entry></entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-14;
 	      <entry>r<subscript>5</subscript></entry>
 	      <entry>r<subscript>4</subscript></entry>
 	      <entry>r<subscript>3</subscript></entry>
@@ -417,6 +428,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB888_1X24</entry>
 	      <entry>0x100a</entry>
 	      <entry></entry>
+	      &dash-ent-8;
 	      <entry>r<subscript>7</subscript></entry>
 	      <entry>r<subscript>6</subscript></entry>
 	      <entry>r<subscript>5</subscript></entry>
@@ -446,9 +458,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB888_2X12_BE</entry>
 	      <entry>0x100b</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-20;
 	      <entry>r<subscript>7</subscript></entry>
 	      <entry>r<subscript>6</subscript></entry>
 	      <entry>r<subscript>5</subscript></entry>
@@ -466,9 +476,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-20;
 	      <entry>g<subscript>3</subscript></entry>
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
@@ -486,9 +494,7 @@
 	      <entry>V4L2_MBUS_FMT_RGB888_2X12_LE</entry>
 	      <entry>0x100c</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-20;
 	      <entry>g<subscript>3</subscript></entry>
 	      <entry>g<subscript>2</subscript></entry>
 	      <entry>g<subscript>1</subscript></entry>
@@ -506,9 +512,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-20;
 	      <entry>r<subscript>7</subscript></entry>
 	      <entry>r<subscript>6</subscript></entry>
 	      <entry>r<subscript>5</subscript></entry>
@@ -522,6 +526,43 @@
 	      <entry>g<subscript>5</subscript></entry>
 	      <entry>g<subscript>4</subscript></entry>
 	    </row>
+	    <row id="V4L2-MBUS-FMT-ARGB888-1X32">
+	      <entry>V4L2_MBUS_FMT_ARGB888_1X32</entry>
+	      <entry>0x100d</entry>
+	      <entry></entry>
+	      <entry>a<subscript>7</subscript></entry>
+	      <entry>a<subscript>6</subscript></entry>
+	      <entry>a<subscript>5</subscript></entry>
+	      <entry>a<subscript>4</subscript></entry>
+	      <entry>a<subscript>3</subscript></entry>
+	      <entry>a<subscript>2</subscript></entry>
+	      <entry>a<subscript>1</subscript></entry>
+	      <entry>a<subscript>0</subscript></entry>
+	      <entry>r<subscript>7</subscript></entry>
+	      <entry>r<subscript>6</subscript></entry>
+	      <entry>r<subscript>5</subscript></entry>
+	      <entry>r<subscript>4</subscript></entry>
+	      <entry>r<subscript>3</subscript></entry>
+	      <entry>r<subscript>2</subscript></entry>
+	      <entry>r<subscript>1</subscript></entry>
+	      <entry>r<subscript>0</subscript></entry>
+	      <entry>g<subscript>7</subscript></entry>
+	      <entry>g<subscript>6</subscript></entry>
+	      <entry>g<subscript>5</subscript></entry>
+	      <entry>g<subscript>4</subscript></entry>
+	      <entry>g<subscript>3</subscript></entry>
+	      <entry>g<subscript>2</subscript></entry>
+	      <entry>g<subscript>1</subscript></entry>
+	      <entry>g<subscript>0</subscript></entry>
+	      <entry>b<subscript>7</subscript></entry>
+	      <entry>b<subscript>6</subscript></entry>
+	      <entry>b<subscript>5</subscript></entry>
+	      <entry>b<subscript>4</subscript></entry>
+	      <entry>b<subscript>3</subscript></entry>
+	      <entry>b<subscript>2</subscript></entry>
+	      <entry>b<subscript>1</subscript></entry>
+	      <entry>b<subscript>0</subscript></entry>
+	    </row>
 	  </tbody>
 	</tgroup>
       </table>
@@ -1149,6 +1190,7 @@
 	   <listitem><para>y<subscript>x</subscript> for luma component bit number x</para></listitem>
 	   <listitem><para>u<subscript>x</subscript> for blue chroma component bit number x</para></listitem>
 	   <listitem><para>v<subscript>x</subscript> for red chroma component bit number x</para></listitem>
+	   <listitem><para>a<subscript>x</subscript> for alpha component bit number x</para></listitem>
 	   <listitem><para>- for non-available bits (for positions higher than the bus width)</para></listitem>
 	   <listitem><para>d for dummy bits</para></listitem>
 	</itemizedlist>
@@ -1159,37 +1201,39 @@
 	  <colspec colname="id" align="left" />
 	  <colspec colname="code" align="center"/>
 	  <colspec colname="bit" />
-	  <colspec colnum="4" colname="b29" align="center" />
-	  <colspec colnum="5" colname="b28" align="center" />
-	  <colspec colnum="6" colname="b27" align="center" />
-	  <colspec colnum="7" colname="b26" align="center" />
-	  <colspec colnum="8" colname="b25" align="center" />
-	  <colspec colnum="9" colname="b24" align="center" />
-	  <colspec colnum="10" colname="b23" align="center" />
-	  <colspec colnum="11" colname="b22" align="center" />
-	  <colspec colnum="12" colname="b21" align="center" />
-	  <colspec colnum="13" colname="b20" align="center" />
-	  <colspec colnum="14" colname="b19" align="center" />
-	  <colspec colnum="15" colname="b18" align="center" />
-	  <colspec colnum="16" colname="b17" align="center" />
-	  <colspec colnum="17" colname="b16" align="center" />
-	  <colspec colnum="18" colname="b15" align="center" />
-	  <colspec colnum="19" colname="b14" align="center" />
-	  <colspec colnum="20" colname="b13" align="center" />
-	  <colspec colnum="21" colname="b12" align="center" />
-	  <colspec colnum="22" colname="b11" align="center" />
-	  <colspec colnum="23" colname="b10" align="center" />
-	  <colspec colnum="24" colname="b09" align="center" />
-	  <colspec colnum="25" colname="b08" align="center" />
-	  <colspec colnum="26" colname="b07" align="center" />
-	  <colspec colnum="27" colname="b06" align="center" />
-	  <colspec colnum="28" colname="b05" align="center" />
-	  <colspec colnum="29" colname="b04" align="center" />
-	  <colspec colnum="30" colname="b03" align="center" />
-	  <colspec colnum="31" colname="b02" align="center" />
-	  <colspec colnum="32" colname="b01" align="center" />
-	  <colspec colnum="33" colname="b00" align="center" />
-	  <spanspec namest="b29" nameend="b00" spanname="b0" />
+	  <colspec colnum="4" colname="b31" align="center" />
+	  <colspec colnum="5" colname="b20" align="center" />
+	  <colspec colnum="6" colname="b29" align="center" />
+	  <colspec colnum="7" colname="b28" align="center" />
+	  <colspec colnum="8" colname="b27" align="center" />
+	  <colspec colnum="9" colname="b26" align="center" />
+	  <colspec colnum="10" colname="b25" align="center" />
+	  <colspec colnum="11" colname="b24" align="center" />
+	  <colspec colnum="12" colname="b23" align="center" />
+	  <colspec colnum="13" colname="b22" align="center" />
+	  <colspec colnum="14" colname="b21" align="center" />
+	  <colspec colnum="15" colname="b20" align="center" />
+	  <colspec colnum="16" colname="b19" align="center" />
+	  <colspec colnum="17" colname="b18" align="center" />
+	  <colspec colnum="18" colname="b17" align="center" />
+	  <colspec colnum="19" colname="b16" align="center" />
+	  <colspec colnum="20" colname="b15" align="center" />
+	  <colspec colnum="21" colname="b14" align="center" />
+	  <colspec colnum="22" colname="b13" align="center" />
+	  <colspec colnum="23" colname="b12" align="center" />
+	  <colspec colnum="24" colname="b11" align="center" />
+	  <colspec colnum="25" colname="b10" align="center" />
+	  <colspec colnum="26" colname="b09" align="center" />
+	  <colspec colnum="27" colname="b08" align="center" />
+	  <colspec colnum="28" colname="b07" align="center" />
+	  <colspec colnum="29" colname="b06" align="center" />
+	  <colspec colnum="30" colname="b05" align="center" />
+	  <colspec colnum="31" colname="b04" align="center" />
+	  <colspec colnum="32" colname="b03" align="center" />
+	  <colspec colnum="33" colname="b02" align="center" />
+	  <colspec colnum="34" colname="b01" align="center" />
+	  <colspec colnum="35" colname="b00" align="center" />
+	  <spanspec namest="b31" nameend="b00" spanname="b0" />
 	  <thead>
 	    <row>
 	      <entry>Identifier</entry>
@@ -1201,6 +1245,8 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry>Bit</entry>
+	      <entry>31</entry>
+	      <entry>30</entry>
 	      <entry>29</entry>
 	      <entry>28</entry>
 	      <entry>27</entry>
@@ -1238,10 +1284,7 @@
 	      <entry>V4L2_MBUS_FMT_Y8_1X8</entry>
 	      <entry>0x2001</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1255,18 +1298,7 @@
 	      <entry>V4L2_MBUS_FMT_UV8_1X8</entry>
 	      <entry>0x2015</entry>
 	      <entry></entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1280,18 +1312,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1305,10 +1326,7 @@
 	      <entry>V4L2_MBUS_FMT_UYVY8_1_5X8</entry>
 	      <entry>0x2002</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1322,10 +1340,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1339,10 +1354,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1356,10 +1368,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1373,10 +1382,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1390,10 +1396,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1407,10 +1410,7 @@
 	      <entry>V4L2_MBUS_FMT_VYUY8_1_5X8</entry>
 	      <entry>0x2003</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1424,10 +1424,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1441,10 +1438,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1458,10 +1452,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1475,10 +1466,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1492,10 +1480,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1509,10 +1494,7 @@
 	      <entry>V4L2_MBUS_FMT_YUYV8_1_5X8</entry>
 	      <entry>0x2004</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1526,10 +1508,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1543,10 +1522,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1560,10 +1536,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1577,10 +1550,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1594,10 +1564,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1611,10 +1578,7 @@
 	      <entry>V4L2_MBUS_FMT_YVYU8_1_5X8</entry>
 	      <entry>0x2005</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1628,10 +1592,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1645,10 +1606,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1662,10 +1620,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1679,10 +1634,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1696,10 +1648,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1713,10 +1662,7 @@
 	      <entry>V4L2_MBUS_FMT_UYVY8_2X8</entry>
 	      <entry>0x2006</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1730,10 +1676,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1747,10 +1690,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1764,10 +1704,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1781,10 +1718,7 @@
 	      <entry>V4L2_MBUS_FMT_VYUY8_2X8</entry>
 	      <entry>0x2007</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1798,10 +1732,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1815,10 +1746,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1832,10 +1760,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1849,10 +1774,7 @@
 	      <entry>V4L2_MBUS_FMT_YUYV8_2X8</entry>
 	      <entry>0x2008</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1866,10 +1788,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1883,10 +1802,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1900,10 +1816,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1917,10 +1830,7 @@
 	      <entry>V4L2_MBUS_FMT_YVYU8_2X8</entry>
 	      <entry>0x2009</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1934,10 +1844,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -1951,10 +1858,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -1968,10 +1872,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-24;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -1985,8 +1886,7 @@
 	      <entry>V4L2_MBUS_FMT_Y10_1X10</entry>
 	      <entry>0x200a</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2002,8 +1902,7 @@
 	      <entry>V4L2_MBUS_FMT_YUYV10_2X10</entry>
 	      <entry>0x200b</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2019,8 +1918,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>u<subscript>9</subscript></entry>
 	      <entry>u<subscript>8</subscript></entry>
 	      <entry>u<subscript>7</subscript></entry>
@@ -2036,8 +1934,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2053,8 +1950,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>v<subscript>9</subscript></entry>
 	      <entry>v<subscript>8</subscript></entry>
 	      <entry>v<subscript>7</subscript></entry>
@@ -2070,8 +1966,7 @@
 	      <entry>V4L2_MBUS_FMT_YVYU10_2X10</entry>
 	      <entry>0x200c</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2087,8 +1982,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>v<subscript>9</subscript></entry>
 	      <entry>v<subscript>8</subscript></entry>
 	      <entry>v<subscript>7</subscript></entry>
@@ -2104,8 +1998,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2121,8 +2014,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      &dash-ent-10;
+	      &dash-ent-22;
 	      <entry>u<subscript>9</subscript></entry>
 	      <entry>u<subscript>8</subscript></entry>
 	      <entry>u<subscript>7</subscript></entry>
@@ -2138,15 +2030,7 @@
 	      <entry>V4L2_MBUS_FMT_Y12_1X12</entry>
 	      <entry>0x2013</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-20;
 	      <entry>y<subscript>11</subscript></entry>
 	      <entry>y<subscript>10</subscript></entry>
 	      <entry>y<subscript>9</subscript></entry>
@@ -2164,11 +2048,7 @@
 	      <entry>V4L2_MBUS_FMT_UYVY8_1X16</entry>
 	      <entry>0x200f</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -2190,11 +2070,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -2216,11 +2092,7 @@
 	      <entry>V4L2_MBUS_FMT_VYUY8_1X16</entry>
 	      <entry>0x2010</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>v<subscript>7</subscript></entry>
 	      <entry>v<subscript>6</subscript></entry>
 	      <entry>v<subscript>5</subscript></entry>
@@ -2242,11 +2114,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>u<subscript>7</subscript></entry>
 	      <entry>u<subscript>6</subscript></entry>
 	      <entry>u<subscript>5</subscript></entry>
@@ -2268,11 +2136,7 @@
 	      <entry>V4L2_MBUS_FMT_YUYV8_1X16</entry>
 	      <entry>0x2011</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2294,11 +2158,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2320,11 +2180,7 @@
 	      <entry>V4L2_MBUS_FMT_YVYU8_1X16</entry>
 	      <entry>0x2012</entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2346,11 +2202,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2372,10 +2224,7 @@
 	      <entry>V4L2_MBUS_FMT_YDYUYDYV8_1X16</entry>
 	      <entry>0x2014</entry>
 	      <entry></entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2397,10 +2246,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2422,10 +2268,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2447,10 +2290,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
-	      <entry>-</entry>
+	      &dash-ent-16;
 	      <entry>y<subscript>7</subscript></entry>
 	      <entry>y<subscript>6</subscript></entry>
 	      <entry>y<subscript>5</subscript></entry>
@@ -2472,7 +2312,7 @@
 	      <entry>V4L2_MBUS_FMT_YUYV10_1X20</entry>
 	      <entry>0x200d</entry>
 	      <entry></entry>
-	      &dash-ent-10;
+	      &dash-ent-12;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2498,7 +2338,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
+	      &dash-ent-12;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2524,7 +2364,7 @@
 	      <entry>V4L2_MBUS_FMT_YVYU10_1X20</entry>
 	      <entry>0x200e</entry>
 	      <entry></entry>
-	      &dash-ent-10;
+	      &dash-ent-12;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2550,7 +2390,7 @@
 	      <entry></entry>
 	      <entry></entry>
 	      <entry></entry>
-	      &dash-ent-10;
+	      &dash-ent-12;
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2576,6 +2416,8 @@
 	      <entry>V4L2_MBUS_FMT_YUV10_1X30</entry>
 	      <entry>0x2016</entry>
 	      <entry></entry>
+	      <entry>-</entry>
+	      <entry>-</entry>
 	      <entry>y<subscript>9</subscript></entry>
 	      <entry>y<subscript>8</subscript></entry>
 	      <entry>y<subscript>7</subscript></entry>
@@ -2607,6 +2449,43 @@
 	      <entry>v<subscript>1</subscript></entry>
 	      <entry>v<subscript>0</subscript></entry>
 	    </row>
+	    <row id="V4L2-MBUS-FMT-AYUV8-1X32">
+	      <entry>V4L2_MBUS_FMT_AYUV8_1X32</entry>
+	      <entry>0x2017</entry>
+	      <entry></entry>
+	      <entry>a<subscript>7</subscript></entry>
+	      <entry>a<subscript>6</subscript></entry>
+	      <entry>a<subscript>5</subscript></entry>
+	      <entry>a<subscript>4</subscript></entry>
+	      <entry>a<subscript>3</subscript></entry>
+	      <entry>a<subscript>2</subscript></entry>
+	      <entry>a<subscript>1</subscript></entry>
+	      <entry>a<subscript>0</subscript></entry>
+	      <entry>y<subscript>7</subscript></entry>
+	      <entry>y<subscript>6</subscript></entry>
+	      <entry>y<subscript>5</subscript></entry>
+	      <entry>y<subscript>4</subscript></entry>
+	      <entry>y<subscript>3</subscript></entry>
+	      <entry>y<subscript>2</subscript></entry>
+	      <entry>y<subscript>1</subscript></entry>
+	      <entry>y<subscript>0</subscript></entry>
+	      <entry>u<subscript>7</subscript></entry>
+	      <entry>u<subscript>6</subscript></entry>
+	      <entry>u<subscript>5</subscript></entry>
+	      <entry>u<subscript>4</subscript></entry>
+	      <entry>u<subscript>3</subscript></entry>
+	      <entry>u<subscript>2</subscript></entry>
+	      <entry>u<subscript>1</subscript></entry>
+	      <entry>u<subscript>0</subscript></entry>
+	      <entry>v<subscript>7</subscript></entry>
+	      <entry>v<subscript>6</subscript></entry>
+	      <entry>v<subscript>5</subscript></entry>
+	      <entry>v<subscript>4</subscript></entry>
+	      <entry>v<subscript>3</subscript></entry>
+	      <entry>v<subscript>2</subscript></entry>
+	      <entry>v<subscript>1</subscript></entry>
+	      <entry>v<subscript>0</subscript></entry>
+	    </row>
 	  </tbody>
 	</tgroup>
       </table>
diff --git a/Documentation/DocBook/media_api.tmpl b/Documentation/DocBook/media_api.tmpl
index 6a8b715..07e7eea 100644
--- a/Documentation/DocBook/media_api.tmpl
+++ b/Documentation/DocBook/media_api.tmpl
@@ -22,8 +22,14 @@
 
 <!-- LinuxTV v4l-dvb repository. -->
 <!ENTITY v4l-dvb		"<ulink url='http://linuxtv.org/repo/'>http://linuxtv.org/repo/</ulink>">
+<!ENTITY dash-ent-8             "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
 <!ENTITY dash-ent-10            "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
+<!ENTITY dash-ent-12            "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
+<!ENTITY dash-ent-14            "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
 <!ENTITY dash-ent-16            "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
+<!ENTITY dash-ent-20            "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
+<!ENTITY dash-ent-22            "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
+<!ENTITY dash-ent-24            "<entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry><entry>-</entry>">
 ]>
 
 <book id="media_api">
diff --git a/include/uapi/linux/v4l2-mediabus.h b/include/uapi/linux/v4l2-mediabus.h
index 6ee63d0..a960125 100644
--- a/include/uapi/linux/v4l2-mediabus.h
+++ b/include/uapi/linux/v4l2-mediabus.h
@@ -37,7 +37,7 @@
 enum v4l2_mbus_pixelcode {
 	V4L2_MBUS_FMT_FIXED = 0x0001,
 
-	/* RGB - next is 0x100d */
+	/* RGB - next is 0x100e */
 	V4L2_MBUS_FMT_RGB444_2X8_PADHI_BE = 0x1001,
 	V4L2_MBUS_FMT_RGB444_2X8_PADHI_LE = 0x1002,
 	V4L2_MBUS_FMT_RGB555_2X8_PADHI_BE = 0x1003,
@@ -50,8 +50,9 @@ enum v4l2_mbus_pixelcode {
 	V4L2_MBUS_FMT_RGB888_1X24 = 0x100a,
 	V4L2_MBUS_FMT_RGB888_2X12_BE = 0x100b,
 	V4L2_MBUS_FMT_RGB888_2X12_LE = 0x100c,
+	V4L2_MBUS_FMT_ARGB8888_1X32 = 0x100d,
 
-	/* YUV (including grey) - next is 0x2017 */
+	/* YUV (including grey) - next is 0x2018 */
 	V4L2_MBUS_FMT_Y8_1X8 = 0x2001,
 	V4L2_MBUS_FMT_UV8_1X8 = 0x2015,
 	V4L2_MBUS_FMT_UYVY8_1_5X8 = 0x2002,
@@ -74,6 +75,7 @@ enum v4l2_mbus_pixelcode {
 	V4L2_MBUS_FMT_YUYV10_1X20 = 0x200d,
 	V4L2_MBUS_FMT_YVYU10_1X20 = 0x200e,
 	V4L2_MBUS_FMT_YUV10_1X30 = 0x2016,
+	V4L2_MBUS_FMT_AYUV8_1X32 = 0x2017,
 
 	/* Bayer - next is 0x3019 */
 	V4L2_MBUS_FMT_SBGGR8_1X8 = 0x3001,
-- 
1.8.1.5

[PATCH v6 07/10] v4l: Add V4L2_PIX_FMT_NV16M and V4L2_PIX_FMT_NV61M formats

[Laurent Pinchart]

NV16M and NV61M are planar YCbCr 4:2:2 and YCrCb 4:2:2 formats with a
luma plane followed by an interleaved chroma plane. The planes are not
required to be contiguous in memory, and the formats can only be used
with the multi-planar formats API.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Reviewed-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Reviewed-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 Documentation/DocBook/media/v4l/pixfmt-nv16m.xml | 171 +++++++++++++++++++++++
 Documentation/DocBook/media/v4l/pixfmt.xml       |   1 +
 include/uapi/linux/videodev2.h                   |   2 +
 3 files changed, 174 insertions(+)
 create mode 100644 Documentation/DocBook/media/v4l/pixfmt-nv16m.xml

diff --git a/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml b/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml
new file mode 100644
index 0000000..c51d5a4
--- /dev/null
+++ b/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml
@@ -0,0 +1,171 @@
+    <refentry>
+      <refmeta>
+	<refentrytitle>V4L2_PIX_FMT_NV16M ('NM16'), V4L2_PIX_FMT_NV61M ('NM61')</refentrytitle>
+	&manvol;
+      </refmeta>
+      <refnamediv>
+	<refname id="V4L2-PIX-FMT-NV16M"><constant>V4L2_PIX_FMT_NV16M</constant></refname>
+	<refname id="V4L2-PIX-FMT-NV61M"><constant>V4L2_PIX_FMT_NV61M</constant></refname>
+	<refpurpose>Variation of <constant>V4L2_PIX_FMT_NV16</constant> and <constant>V4L2_PIX_FMT_NV61</constant> with planes
+	  non contiguous in memory. </refpurpose>
+      </refnamediv>
+      <refsect1>
+	<title>Description</title>
+
+	<para>This is a multi-planar, two-plane version of the YUV 4:2:0 format.
+The three components are separated into two sub-images or planes.
+<constant>V4L2_PIX_FMT_NV16M</constant> differs from <constant>V4L2_PIX_FMT_NV16
+</constant> in that the two planes are non-contiguous in memory, i.e. the chroma
+plane does not necessarily immediately follows the luma plane.
+The luminance data occupies the first plane. The Y plane has one byte per pixel.
+In the second plane there is chrominance data with alternating chroma samples.
+The CbCr plane is the same width and height, in bytes, as the Y plane.
+Each CbCr pair belongs to four pixels. For example,
+Cb<subscript>0</subscript>/Cr<subscript>0</subscript> belongs to
+Y'<subscript>00</subscript>, Y'<subscript>01</subscript>,
+Y'<subscript>10</subscript>, Y'<subscript>11</subscript>.
+<constant>V4L2_PIX_FMT_NV61M</constant> is the same as <constant>V4L2_PIX_FMT_NV16M</constant>
+except the Cb and Cr bytes are swapped, the CrCb plane starts with a Cr byte.</para>
+
+	<para><constant>V4L2_PIX_FMT_NV16M</constant> and
+<constant>V4L2_PIX_FMT_NV61M</constant> are intended to be used only in drivers
+and applications that support the multi-planar API, described in
+<xref linkend="planar-apis"/>. </para>
+
+	<example>
+	  <title><constant>V4L2_PIX_FMT_NV16M</constant> 4 &times; 4 pixel image</title>
+
+	  <formalpara>
+	    <title>Byte Order.</title>
+	    <para>Each cell is one byte.
+		<informaltable frame="none">
+		<tgroup cols="5" align="center">
+		  <colspec align="left" colwidth="2*" />
+		  <tbody valign="top">
+		    <row>
+		      <entry>start0&nbsp;+&nbsp;0:</entry>
+		      <entry>Y'<subscript>00</subscript></entry>
+		      <entry>Y'<subscript>01</subscript></entry>
+		      <entry>Y'<subscript>02</subscript></entry>
+		      <entry>Y'<subscript>03</subscript></entry>
+		    </row>
+		    <row>
+		      <entry>start0&nbsp;+&nbsp;4:</entry>
+		      <entry>Y'<subscript>10</subscript></entry>
+		      <entry>Y'<subscript>11</subscript></entry>
+		      <entry>Y'<subscript>12</subscript></entry>
+		      <entry>Y'<subscript>13</subscript></entry>
+		    </row>
+		    <row>
+		      <entry>start0&nbsp;+&nbsp;8:</entry>
+		      <entry>Y'<subscript>20</subscript></entry>
+		      <entry>Y'<subscript>21</subscript></entry>
+		      <entry>Y'<subscript>22</subscript></entry>
+		      <entry>Y'<subscript>23</subscript></entry>
+		    </row>
+		    <row>
+		      <entry>start0&nbsp;+&nbsp;12:</entry>
+		      <entry>Y'<subscript>30</subscript></entry>
+		      <entry>Y'<subscript>31</subscript></entry>
+		      <entry>Y'<subscript>32</subscript></entry>
+		      <entry>Y'<subscript>33</subscript></entry>
+		    </row>
+		    <row>
+		      <entry></entry>
+		    </row>
+		    <row>
+		      <entry>start1&nbsp;+&nbsp;0:</entry>
+		      <entry>Cb<subscript>00</subscript></entry>
+		      <entry>Cr<subscript>00</subscript></entry>
+		      <entry>Cb<subscript>02</subscript></entry>
+		      <entry>Cr<subscript>02</subscript></entry>
+		    </row>
+		    <row>
+		      <entry>start1&nbsp;+&nbsp;4:</entry>
+		      <entry>Cb<subscript>10</subscript></entry>
+		      <entry>Cr<subscript>10</subscript></entry>
+		      <entry>Cb<subscript>12</subscript></entry>
+		      <entry>Cr<subscript>12</subscript></entry>
+		    </row>
+		    <row>
+		      <entry>start1&nbsp;+&nbsp;8:</entry>
+		      <entry>Cb<subscript>20</subscript></entry>
+		      <entry>Cr<subscript>20</subscript></entry>
+		      <entry>Cb<subscript>22</subscript></entry>
+		      <entry>Cr<subscript>22</subscript></entry>
+		    </row>
+		    <row>
+		      <entry>start1&nbsp;+&nbsp;12:</entry>
+		      <entry>Cb<subscript>30</subscript></entry>
+		      <entry>Cr<subscript>30</subscript></entry>
+		      <entry>Cb<subscript>32</subscript></entry>
+		      <entry>Cr<subscript>32</subscript></entry>
+		    </row>
+		  </tbody>
+		</tgroup>
+		</informaltable>
+	      </para>
+	  </formalpara>
+
+	  <formalpara>
+	    <title>Color Sample Location.</title>
+	    <para>
+		<informaltable frame="none">
+		<tgroup cols="7" align="center">
+		  <tbody valign="top">
+		    <row>
+		      <entry></entry>
+		      <entry>0</entry><entry></entry><entry>1</entry><entry></entry>
+		      <entry>2</entry><entry></entry><entry>3</entry>
+		    </row>
+		    <row>
+		      <entry>0</entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry><entry></entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry>
+		    </row>
+		    <row>
+		      <entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry><entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry>
+		    </row>
+		    <row>
+		      <entry>1</entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry><entry></entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry>
+		    </row>
+		    <row>
+		      <entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry><entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry>
+		    </row>
+		    <row>
+		      <entry></entry>
+		    </row>
+		    <row>
+		      <entry>2</entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry><entry></entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry>
+		    </row>
+		    <row>
+		      <entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry><entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry>
+		    </row>
+		    <row>
+		      <entry>3</entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry><entry></entry>
+		      <entry>Y</entry><entry></entry><entry>Y</entry>
+		    </row>
+		    <row>
+		      <entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry><entry></entry>
+		      <entry></entry><entry>C</entry><entry></entry>
+		    </row>
+		  </tbody>
+		</tgroup>
+		</informaltable>
+	      </para>
+	  </formalpara>
+	</example>
+      </refsect1>
+    </refentry>
diff --git a/Documentation/DocBook/media/v4l/pixfmt.xml b/Documentation/DocBook/media/v4l/pixfmt.xml
index 99b8d2a..16db350 100644
--- a/Documentation/DocBook/media/v4l/pixfmt.xml
+++ b/Documentation/DocBook/media/v4l/pixfmt.xml
@@ -718,6 +718,7 @@ information.</para>
     &sub-nv12m;
     &sub-nv12mt;
     &sub-nv16;
+    &sub-nv16m;
     &sub-nv24;
     &sub-m420;
   </section>
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 95ef455..fec0c20 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -348,6 +348,8 @@ struct v4l2_pix_format {
 /* two non contiguous planes - one Y, one Cr + Cb interleaved  */
 #define V4L2_PIX_FMT_NV12M   v4l2_fourcc('N', 'M', '1', '2') /* 12  Y/CbCr 4:2:0  */
 #define V4L2_PIX_FMT_NV21M   v4l2_fourcc('N', 'M', '2', '1') /* 21  Y/CrCb 4:2:0  */
+#define V4L2_PIX_FMT_NV16M   v4l2_fourcc('N', 'M', '1', '6') /* 16  Y/CbCr 4:2:2  */
+#define V4L2_PIX_FMT_NV61M   v4l2_fourcc('N', 'M', '6', '1') /* 16  Y/CrCb 4:2:2  */
 #define V4L2_PIX_FMT_NV12MT  v4l2_fourcc('T', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 64x32 macroblocks */
 #define V4L2_PIX_FMT_NV12MT_16X16 v4l2_fourcc('V', 'M', '1', '2') /* 12  Y/CbCr 4:2:0 16x16 macroblocks */
 
-- 
1.8.1.5

[PATCH v6 09/10] vsp1: Fix lack of the sink entity registration for enabled links

[Laurent Pinchart]

From: Katsuya Matsubara <matsu@igel.co.jp>

Each source entity maintains a pointer to the counterpart sink
entity while an enabled link connects them. It should be managed by
the setup_link callback in the media controller framework at runtime.
However, enabled links which connect RPFs and WPFs that have an
equivalent index number are created during initialization.
This registers the pointer to a sink entity from the source entity
when an enabled link is created.

Signed-off-by: Katsuya Matsubara <matsu@igel.co.jp>
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vsp1/vsp1_drv.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index e58e49c..41dd891 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -101,6 +101,9 @@ static int vsp1_create_links(struct vsp1_device *vsp1, struct vsp1_entity *sink)
 						       entity, pad, flags);
 			if (ret < 0)
 				return ret;
+
+			if (flags & MEDIA_LNK_FL_ENABLED)
+				source->sink = entity;
 		}
 	}
 
-- 
1.8.1.5

[PATCH v6 10/10] vsp1: Use the maximum number of entities defined in platform data

[Laurent Pinchart]

From: Katsuya Matsubara <matsu@igel.co.jp>

The VSP1 driver allows to define the maximum number of each module
such as RPF, WPF, and UDS in a platform data definition.
This suppresses operations for nonexistent or unused modules.

Signed-off-by: Katsuya Matsubara <matsu@igel.co.jp>
Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vsp1/vsp1_drv.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index 41dd891..8700842 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -35,7 +35,7 @@ static irqreturn_t vsp1_irq_handler(int irq, void *data)
 	irqreturn_t ret = IRQ_NONE;
 	unsigned int i;
 
-	for (i = 0; i < VPS1_MAX_WPF; ++i) {
+	for (i = 0; i < vsp1->pdata->wpf_count; ++i) {
 		struct vsp1_rwpf *wpf = vsp1->wpf[i];
 		struct vsp1_pipeline *pipe;
 		u32 status;
@@ -243,7 +243,7 @@ static int vsp1_device_init(struct vsp1_device *vsp1)
 	/* Reset any channel that might be running. */
 	status = vsp1_read(vsp1, VI6_STATUS);
 
-	for (i = 0; i < VPS1_MAX_WPF; ++i) {
+	for (i = 0; i < vsp1->pdata->wpf_count; ++i) {
 		unsigned int timeout;
 
 		if (!(status & VI6_STATUS_SYS_ACT(i)))
@@ -267,10 +267,10 @@ static int vsp1_device_init(struct vsp1_device *vsp1)
 	vsp1_write(vsp1, VI6_CLK_DCSWT, (8 << VI6_CLK_DCSWT_CSTPW_SHIFT) |
 		   (8 << VI6_CLK_DCSWT_CSTRW_SHIFT));
 
-	for (i = 0; i < VPS1_MAX_RPF; ++i)
+	for (i = 0; i < vsp1->pdata->rpf_count; ++i)
 		vsp1_write(vsp1, VI6_DPR_RPF_ROUTE(i), VI6_DPR_NODE_UNUSED);
 
-	for (i = 0; i < VPS1_MAX_UDS; ++i)
+	for (i = 0; i < vsp1->pdata->uds_count; ++i)
 		vsp1_write(vsp1, VI6_DPR_UDS_ROUTE(i), VI6_DPR_NODE_UNUSED);
 
 	vsp1_write(vsp1, VI6_DPR_SRU_ROUTE, VI6_DPR_NODE_UNUSED);
-- 
1.8.1.5