[PATCH v2 0/2] Add pixel formats used in Synatpics SoC

[PATCH v2 0/2] Add pixel formats used in Synatpics SoC

[Hsia-Jun Li]

From: "Hsia-Jun(Randy) Li" <randy.li@synaptics.com>

Those pixel formats are used in Synaptics's VideoSmart series SoCs,
likes VS640, VS680. I just disclose the pixel formats used in the video
codecs and display pipeline this time. Actually any device connected to
the MTR module could support those tiled and compressed pixel formats.
The more detail about MTR module could be found in the first patch of
this serial of mail.

We may not be able to post any drivers here in a short time, the most of
work in this platform is done in the Trusted Execution Environment and
we didn't use the optee event its client framework.

Please notice that, the memory planes used for video codecs could be
one more than display case. That extra planes in the video codecs is
for the decoding internally usage, it can't append the luma or chroma
buffer as many other drivers do, because this buffer could be only
accessed by the video codecs itself, it requests a different memory
security attributes. There is not a proper place in v4l2 m2m to allocate
a large size buffer, we don't know when the users won't allocate more
graphics buffers. Although we could allocate it in a step likes
STREAMON, it would lead unusual delaying in starting of video playbacl.

https://synaptics.com/products/multimedia-solutions

Changlog
v2:
The DRM modifiers in the first draft is too simple, it can't tell
the tiles in group attribute in memory layout.
Removing the v4l2 fourcc. Adding a document for the future v4l2 extended
fmt.
v1:
first draft of DRM modifiers
Try to put basic tile formats into v4l2 fourcc

Hsia-Jun(Randy) Li (1):
  drm/fourcc: Add Synaptics VideoSmart tiled modifiers

Randy Li (1):
  media: docs: Add Synpatics tile modifiers

 .../media/v4l/pixfmt-synaptics.rst            | 80 +++++++++++++++++++
 .../userspace-api/media/v4l/pixfmt.rst        |  1 +
 include/uapi/drm/drm_fourcc.h                 | 75 +++++++++++++++++
 3 files changed, 156 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-synaptics.rst

-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 1/2] drm/fourcc: Add Synaptics VideoSmart tiled modifiers

[Hsia-Jun Li]

From: "Hsia-Jun(Randy) Li" <randy.li@synaptics.com>

Memory Traffic Reduction(MTR) is a module in Synaptics
VideoSmart platform could process lossless compression image
and cache the tile memory line.

Those modifiers only record the parameters would effort pixel
layout or memory layout. Whether physical memory page mapping
is used is not a part of format.

We would allocate the same size of memory for uncompressed
and compressed luma and chroma data, while the compressed buffer
would request two extra planes holding the metadata for
the decompression.

Signed-off-by: Hsia-Jun(Randy) Li <randy.li@synaptics.com>
---
 include/uapi/drm/drm_fourcc.h | 75 +++++++++++++++++++++++++++++++++++
 1 file changed, 75 insertions(+)

diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h
index bc056f2d537d..4b587a4694f7 100644
--- a/include/uapi/drm/drm_fourcc.h
+++ b/include/uapi/drm/drm_fourcc.h
@@ -407,6 +407,7 @@ extern "C" {
 #define DRM_FORMAT_MOD_VENDOR_ARM     0x08
 #define DRM_FORMAT_MOD_VENDOR_ALLWINNER 0x09
 #define DRM_FORMAT_MOD_VENDOR_AMLOGIC 0x0a
+#define DRM_FORMAT_MOD_VENDOR_SYNAPTICS 0x0b
 
 /* add more to the end as needed */
 
@@ -1507,6 +1508,80 @@ drm_fourcc_canonicalize_nvidia_format_mod(__u64 modifier)
 #define AMD_FMT_MOD_CLEAR(field) \
 	(~((__u64)AMD_FMT_MOD_##field##_MASK << AMD_FMT_MOD_##field##_SHIFT))
 
+/*
+ * Synaptics VideoSmart modifiers
+ *
+ * Tiles could be arranged in Groups of Tiles (GOTs), it is a small tile
+ * within a tile. GOT size and layout varies based on platform and
+ * performance concern. When the compression is applied, it is possible
+ * that we would have two tile type in the GOT, these parameters can't
+ * tell the secondary tile type.
+ *
+ * Besides, an 8 size 4 bytes arrary (32 bytes) would be need to store
+ * some compression parameters for a compression meta data plane.
+ *
+ *       Macro
+ * Bits  Param Description
+ * ----  ----- -----------------------------------------------------------------
+ *
+ *  7:0  f     Scan direction description.
+ *
+ *               0 = Invalid
+ *               1 = V4, the scan would always start from vertical for 4 pixel
+ *                   then move back to the start pixel of the next horizontal
+ *                   direction.
+ *               2 = Reserved for future use.
+ *
+ * 15:8  m     The times of pattern repeat in the right angle direction from
+ *             the first scan direction.
+ *
+ * 19:16 p     The padding bits after the whole scan, could be zero.
+ *
+ * 20:20 g     GOT packing flag.
+ *
+ * 23:21 -     Reserved for future use.  Must be zero.
+ *
+ * 27:24 h     log2(horizontal) of bytes, in GOTs.
+ *
+ * 31:28 v     log2(vertical) of bytes, in GOTs.
+ *
+ * 35:32 -     Reserved for future use.  Must be zero.
+ *
+ * 36:36 c     Compression flag.
+ *
+ * 55:37 -     Reserved for future use.  Must be zero.
+ *
+ */
+
+#define DRM_FORMAT_MOD_SYNA_V4_TILED		fourcc_mod_code(SYNAPTICS, 1)
+
+#define DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(f, m, p, g, h, v, c) \
+	fourcc_mod_code(SYNAPTICS, (((f) & 0xff) | \
+				 (((m) & 0xff) << 8) | \
+				 (((p) & 0xf) << 16) | \
+				 (((g) & 0x1) << 16) | \
+				 (((h) & 0xf) << 24) | \
+				 (((v) & 0xf) << 28) | \
+				 (((c) & 0x1) << 36)))
+
+#define DRM_FORMAT_MOD_SYNA_V4H1 \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 1, 0, 0, 0, 0, 0)
+
+#define DRM_FORMAT_MOD_SYNA_V4H3P8 \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 3, 8, 0, 0, 0, 0)
+
+#define DRM_FORMAT_MOD_SYNA_V4H1_64L4_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 1, 0, 1, 6, 2, 1)
+
+#define DRM_FORMAT_MOD_SYNA_V4H3P8_64L4_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 3, 8, 1, 6, 2, 1)
+
+#define DRM_FORMAT_MOD_SYNA_V4H1_128L128_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 1, 0, 1, 7, 7, 1)
+
+#define DRM_FORMAT_MOD_SYNA_V4H3P8_128L128_COMPRESSED \
+	DRM_FORMAT_MOD_SYNA_MTR_LINEAR_2D(1, 3, 8, 1, 7, 7, 1)
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/2] media: docs: Add Synpatics tile modifiers

[Hsia-Jun Li]

From: Randy Li <ayaka@soulik.info>

The pixel formats used in Synpatics video smart platform
are too many. It is impossible to store them in fourcc
namespace.

Signed-off-by: Randy Li <ayaka@soulik.info>
---
 .../media/v4l/pixfmt-synaptics.rst            | 80 +++++++++++++++++++
 .../userspace-api/media/v4l/pixfmt.rst        |  1 +
 2 files changed, 81 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-synaptics.rst

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-synaptics.rst b/Documentation/userspace-api/media/v4l/pixfmt-synaptics.rst
new file mode 100644
index 000000000000..bc86737febb7
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-synaptics.rst
@@ -0,0 +1,80 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _pixfmt-synaptics:
+
+***************************
+Synaptics Pixel Format Modifiers
+***************************
+
+The tiled pixel formats in synpatics video smart platform have
+many variants. Here just list the most widely pixel format modifiers
+here. The value here should be the same as the one defined in the
+``drm_fourcc.h`` file.
+
+.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.5cm}|
+
+.. raw:: latex
+
+	\small
+
+.. _reserved-formats:
+
+.. flat-table:: Synpatics Image Format Modifiers
+	:header-rows:  1
+	:stub-columns: 0
+	:widths:	   3 1 4
+
+	* - Identifier
+	  - Code
+	  - Details
+	* .. _V4L2-PIX-FMT-MOD-SYNA-V4H1:
+
+	  - ``V4L2_PIX_FMT_MOD_SYNA_V4H1``
+	  - '0x0b00000000000101'
+	  - The plain uncompressed 8bits tile format. It sounds similar to
+	Intel's Y-tile. but it won't take any pixel from the next X direction
+	in a tile group. The line stride and image height must be aligned to
+	a multiple of 16. The height of chrominance plane would plus 8.
+	This modifier current would be in conjunction with ``V4L2_PIX_FMT_NV12``
+	or ``V4L2_PIX_FMT_NV12M``.
+	* .. _V4L2-PIX-FMT-MOD-SYNA-V4H3P8:
+
+	  - ``V4L2_PIX_FMT_MOD_SYNA_V4H3P8``
+	  - '0x0b00000000080301'
+	  - The plain uncompressed 10bits tile format. It stores pixel in 2D
+	3x4 tiles with a 8bits padding to each of tiles. Then tile is in a
+	128 bytes cache line. This modifier would be in conjunction with
+	``V4L2_PIX_FMT_NV15``.
+	* .. _V4L2-PIX-FMT-MOD-SYNA-V4H1-64L4C:
+
+	  - ``V4L2_PIX_FMT_MOD_SYNA_V4H1_64L4C``
+	  - '0x0b00000026010101'
+	  - Compressed ``V4L2_PIX_FMT_MOD_SYNA_V4H1``. It stores 64x4 pixels
+	in 1x4 tiles. Each plane would request a meta plane (MTR plane) for
+	decompression. A MTR plane would have a 32 bytes parameters set.
+	* .. _V4L2-PIX-FMT-MOD-SYNA-V4H3P8-64L4C:
+
+	  - ``V4L2_PIX_FMT_MOD_SYNA_V4H3P8_64L4C``
+	  - '0x0b00000026090301'
+	  - Compressed ``V4L2_PIX_FMT_MOD_SYNA_V4H3``. It stores 64x4 pixels
+	in tiles. Each plane would request a meta plane (MTR plane) for
+	decompression. A MTR plane would have a 32 bytes parameters set.
+
+	* .. _V4L2-PIX-FMT-MOD-SYNA-V4H1-128L128C:
+
+	  - ``V4L2_PIX_FMT_MOD_SYNA_V4H1_128L128C``
+	  - '0x0b00000077010101'
+	  - Compressed ``V4L2_PIX_FMT_MOD_SYNA_V4H1``. It stores 128x128 pixels
+	in 1x4 tiles. Each plane would request a meta plane (MTR plane) for
+	decompression. A MTR plane would have a 32 bytes parameters set.
+	* .. _V4L2-PIX-FMT-MOD-SYNA-V4H3P8-128L128C:
+
+	  - ``V4L2_PIX_FMT_MOD_SYNA_V4H3P8_128L128C``
+	  - '0x0b00000077090301'
+	  - Compressed ``V4L2_PIX_FMT_MOD_SYNA_V4H3``. It stores 128x128 pixels
+	in tiles. Each plane would request a meta plane (MTR plane) for
+	decompression. A MTR plane would have a 32 bytes parameters set.
+
+.. raw:: latex
+
+	\normalsize
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..bfe4fdb52b6b 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
     colorspaces
     colorspaces-defs
     colorspaces-details
+	pixfmt-synaptics 
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel