* [PATCH v5 1/4] media: uapi: Add meta formats for PiSP FE config and stats
2024-09-10 8:07 [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE Tomi Valkeinen
@ 2024-09-10 8:07 ` Tomi Valkeinen
2024-09-10 8:07 ` [PATCH v5 2/4] dt-bindings: media: Add bindings for raspberrypi,rp1-cfe Tomi Valkeinen
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Tomi Valkeinen @ 2024-09-10 8:07 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Raspberry Pi Kernel Maintenance,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Florian Fainelli,
Broadcom internal kernel review list, Rob Herring,
Krzysztof Kozlowski
Cc: linux-media, linux-kernel, devicetree, linux-rpi-kernel,
linux-arm-kernel, Naushir Patuck, Laurent Pinchart, Sakari Ailus,
Jacopo Mondi, Kieran Bingham, Tomi Valkeinen
Add two meta formats for PiSP FE: V4L2_META_FMT_RPI_FE_CFG and
V4L2_META_FMT_RPI_FE_STATS. The former is used to provide configuration
for the FE and the latter is used to read the statistics from the FE.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../userspace-api/media/v4l/metafmt-pisp-fe.rst | 39 ++++++++++++++++++++++
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
include/uapi/linux/videodev2.h | 2 ++
4 files changed, 44 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index c6e56b5888bc..86ffb3bc8ade 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -16,6 +16,7 @@ These formats are used for the :ref:`metadata` interface only.
metafmt-generic
metafmt-intel-ipu3
metafmt-pisp-be
+ metafmt-pisp-fe
metafmt-rkisp1
metafmt-uvc
metafmt-vivid
diff --git a/Documentation/userspace-api/media/v4l/metafmt-pisp-fe.rst b/Documentation/userspace-api/media/v4l/metafmt-pisp-fe.rst
new file mode 100644
index 000000000000..fddeada83e4a
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/metafmt-pisp-fe.rst
@@ -0,0 +1,39 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _v4l2-meta-fmt-rpi-fe-cfg:
+
+************************
+V4L2_META_FMT_RPI_FE_CFG
+************************
+
+Raspberry Pi PiSP Front End configuration format
+================================================
+
+The Raspberry Pi PiSP Front End image signal processor is configured by
+userspace by providing a buffer of configuration parameters to the
+`rp1-cfe-fe-config` output video device node using the
+:c:type:`v4l2_meta_format` interface.
+
+The `Raspberry Pi PiSP technical specification
+<https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf>`_
+provide detailed description of the Front End configuration and programming
+model.
+
+.. _v4l2-meta-fmt-rpi-fe-stats:
+
+**************************
+V4L2_META_FMT_RPI_FE_STATS
+**************************
+
+Raspberry Pi PiSP Front End statistics format
+=============================================
+
+The Raspberry Pi PiSP Front End image signal processor provides statistics data
+by writing to a buffer provided via the `rp1-cfe-fe-stats` capture video device
+node using the
+:c:type:`v4l2_meta_format` interface.
+
+The `Raspberry Pi PiSP technical specification
+<https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf>`_
+provide detailed description of the Front End configuration and programming
+model.
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 5eb4d797d259..95c2c4a97966 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1466,6 +1466,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break;
case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break;
case V4L2_META_FMT_RPI_BE_CFG: descr = "RPi PiSP BE Config format"; break;
+ case V4L2_META_FMT_RPI_FE_CFG: descr = "RPi PiSP FE Config format"; break;
+ case V4L2_META_FMT_RPI_FE_STATS: descr = "RPi PiSP FE Statistics format"; break;
case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break;
case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8-bit Generic Meta, 10b CSI-2"; break;
case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8-bit Generic Meta, 12b CSI-2"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 4e91362da6da..9f68cd55248f 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -857,6 +857,8 @@ struct v4l2_pix_format {
/* Vendor specific - used for RaspberryPi PiSP */
#define V4L2_META_FMT_RPI_BE_CFG v4l2_fourcc('R', 'P', 'B', 'C') /* PiSP BE configuration */
+#define V4L2_META_FMT_RPI_FE_CFG v4l2_fourcc('R', 'P', 'F', 'C') /* PiSP FE configuration */
+#define V4L2_META_FMT_RPI_FE_STATS v4l2_fourcc('R', 'P', 'F', 'S') /* PiSP FE stats */
#ifdef __KERNEL__
/*
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v5 2/4] dt-bindings: media: Add bindings for raspberrypi,rp1-cfe
2024-09-10 8:07 [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE Tomi Valkeinen
2024-09-10 8:07 ` [PATCH v5 1/4] media: uapi: Add meta formats for PiSP FE config and stats Tomi Valkeinen
@ 2024-09-10 8:07 ` Tomi Valkeinen
2024-09-10 8:07 ` [PATCH v5 4/4] media: admin-guide: Document the Raspberry Pi CFE (rp1-cfe) Tomi Valkeinen
2024-09-26 7:13 ` [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE Tomi Valkeinen
3 siblings, 0 replies; 7+ messages in thread
From: Tomi Valkeinen @ 2024-09-10 8:07 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Raspberry Pi Kernel Maintenance,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Florian Fainelli,
Broadcom internal kernel review list, Rob Herring,
Krzysztof Kozlowski
Cc: linux-media, linux-kernel, devicetree, linux-rpi-kernel,
linux-arm-kernel, Naushir Patuck, Laurent Pinchart, Sakari Ailus,
Jacopo Mondi, Kieran Bingham, Tomi Valkeinen, Krzysztof Kozlowski
Add DT bindings for raspberrypi,rp1-cfe.
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
.../bindings/media/raspberrypi,rp1-cfe.yaml | 93 ++++++++++++++++++++++
1 file changed, 93 insertions(+)
diff --git a/Documentation/devicetree/bindings/media/raspberrypi,rp1-cfe.yaml b/Documentation/devicetree/bindings/media/raspberrypi,rp1-cfe.yaml
new file mode 100644
index 000000000000..eba5394719b9
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/raspberrypi,rp1-cfe.yaml
@@ -0,0 +1,93 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/raspberrypi,rp1-cfe.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Raspberry Pi PiSP Camera Front End
+
+maintainers:
+ - Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
+ - Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
+
+description: |
+ The Raspberry Pi PiSP Camera Front End is a module in Raspberrypi 5's RP1 I/O
+ controller, that contains:
+ - MIPI D-PHY
+ - MIPI CSI-2 receiver
+ - Simple image processor (called PiSP Front End, or FE)
+
+ The FE documentation is available at:
+ https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf
+
+ The PHY and CSI-2 receiver part have no public documentation.
+
+properties:
+ compatible:
+ items:
+ - const: raspberrypi,rp1-cfe
+
+ reg:
+ items:
+ - description: CSI-2 registers
+ - description: D-PHY registers
+ - description: MIPI CFG (a simple top-level mux) registers
+ - description: FE registers
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ maxItems: 1
+
+ port:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ additionalProperties: false
+ description: CSI-2 RX Port
+
+ properties:
+ endpoint:
+ $ref: video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ properties:
+ data-lanes:
+ minItems: 1
+ maxItems: 4
+
+ required:
+ - data-lanes
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+
+additionalProperties: false
+
+examples:
+ - |
+ rp1 {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ csi@110000 {
+ compatible = "raspberrypi,rp1-cfe";
+ reg = <0xc0 0x40110000 0x0 0x100>,
+ <0xc0 0x40114000 0x0 0x100>,
+ <0xc0 0x40120000 0x0 0x100>,
+ <0xc0 0x40124000 0x0 0x1000>;
+
+ interrupts = <42>;
+
+ clocks = <&rp1_clocks>;
+
+ port {
+ csi_ep: endpoint {
+ remote-endpoint = <&cam_endpoint>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+ };
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v5 4/4] media: admin-guide: Document the Raspberry Pi CFE (rp1-cfe)
2024-09-10 8:07 [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE Tomi Valkeinen
2024-09-10 8:07 ` [PATCH v5 1/4] media: uapi: Add meta formats for PiSP FE config and stats Tomi Valkeinen
2024-09-10 8:07 ` [PATCH v5 2/4] dt-bindings: media: Add bindings for raspberrypi,rp1-cfe Tomi Valkeinen
@ 2024-09-10 8:07 ` Tomi Valkeinen
2024-09-26 7:13 ` [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE Tomi Valkeinen
3 siblings, 0 replies; 7+ messages in thread
From: Tomi Valkeinen @ 2024-09-10 8:07 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Raspberry Pi Kernel Maintenance,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Florian Fainelli,
Broadcom internal kernel review list, Rob Herring,
Krzysztof Kozlowski
Cc: linux-media, linux-kernel, devicetree, linux-rpi-kernel,
linux-arm-kernel, Naushir Patuck, Laurent Pinchart, Sakari Ailus,
Jacopo Mondi, Kieran Bingham, Tomi Valkeinen
Add documentation for rp1-cfe driver.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
.../admin-guide/media/raspberrypi-rp1-cfe.dot | 27 ++++++++
.../admin-guide/media/raspberrypi-rp1-cfe.rst | 78 ++++++++++++++++++++++
Documentation/admin-guide/media/v4l-drivers.rst | 1 +
3 files changed, 106 insertions(+)
diff --git a/Documentation/admin-guide/media/raspberrypi-rp1-cfe.dot b/Documentation/admin-guide/media/raspberrypi-rp1-cfe.dot
new file mode 100644
index 000000000000..7717f2291049
--- /dev/null
+++ b/Documentation/admin-guide/media/raspberrypi-rp1-cfe.dot
@@ -0,0 +1,27 @@
+digraph board {
+ rankdir=TB
+ n00000001 [label="{{<port0> 0} | csi2\n/dev/v4l-subdev0 | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000001:port1 -> n00000011 [style=dashed]
+ n00000001:port1 -> n00000007:port0
+ n00000001:port2 -> n00000015
+ n00000001:port2 -> n00000007:port0 [style=dashed]
+ n00000001:port3 -> n00000019 [style=dashed]
+ n00000001:port3 -> n00000007:port0 [style=dashed]
+ n00000001:port4 -> n0000001d [style=dashed]
+ n00000001:port4 -> n00000007:port0 [style=dashed]
+ n00000007 [label="{{<port0> 0 | <port1> 1} | pisp-fe\n/dev/v4l-subdev1 | {<port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000007:port2 -> n00000021
+ n00000007:port3 -> n00000025 [style=dashed]
+ n00000007:port4 -> n00000029
+ n0000000d [label="{imx219 6-0010\n/dev/v4l-subdev2 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n0000000d:port0 -> n00000001:port0 [style=bold]
+ n00000011 [label="rp1-cfe-csi2-ch0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+ n00000015 [label="rp1-cfe-csi2-ch1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
+ n00000019 [label="rp1-cfe-csi2-ch2\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
+ n0000001d [label="rp1-cfe-csi2-ch3\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
+ n00000021 [label="rp1-cfe-fe-image0\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
+ n00000025 [label="rp1-cfe-fe-image1\n/dev/video5", shape=box, style=filled, fillcolor=yellow]
+ n00000029 [label="rp1-cfe-fe-stats\n/dev/video6", shape=box, style=filled, fillcolor=yellow]
+ n0000002d [label="rp1-cfe-fe-config\n/dev/video7", shape=box, style=filled, fillcolor=yellow]
+ n0000002d -> n00000007:port1
+}
diff --git a/Documentation/admin-guide/media/raspberrypi-rp1-cfe.rst b/Documentation/admin-guide/media/raspberrypi-rp1-cfe.rst
new file mode 100644
index 000000000000..668d978a9875
--- /dev/null
+++ b/Documentation/admin-guide/media/raspberrypi-rp1-cfe.rst
@@ -0,0 +1,78 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============================================
+Raspberry Pi PiSP Camera Front End (rp1-cfe)
+============================================
+
+The PiSP Camera Front End
+=========================
+
+The PiSP Camera Front End (CFE) is a module which combines a CSI-2 receiver with
+a simple ISP, called the Front End (FE).
+
+The CFE has four DMA engines and can write frames from four separate streams
+received from the CSI-2 to the memory. One of those streams can also be routed
+directly to the FE, which can do minimal image processing, write two versions
+(e.g. non-scaled and downscaled versions) of the received frames to memory and
+provide statistics of the received frames.
+
+The FE registers are documented in the `Raspberry Pi Image Signal Processor
+(ISP) Specification document
+<https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf>`_,
+and example code for FE can be found in `libpisp
+<https://github.com/raspberrypi/libpisp>`_.
+
+The rp1-cfe driver
+==================
+
+The Raspberry Pi PiSP Camera Front End (rp1-cfe) driver is located under
+drivers/media/platform/raspberrypi/rp1-cfe. It uses the `V4L2 API` to register
+a number of video capture and output devices, the `V4L2 subdev API` to register
+subdevices for the CSI-2 received and the FE that connects the video devices in
+a single media graph realized using the `Media Controller (MC) API`.
+
+The media topology registered by the `rp1-cfe` driver, in this particular
+example connected to an imx219 sensor, is the following one:
+
+.. _rp1-cfe-topology:
+
+.. kernel-figure:: raspberrypi-rp1-cfe.dot
+ :alt: Diagram of an example media pipeline topology
+ :align: center
+
+The media graph contains the following video device nodes:
+
+- rp1-cfe-csi2-ch0: capture device for the first CSI-2 stream
+- rp1-cfe-csi2-ch1: capture device for the second CSI-2 stream
+- rp1-cfe-csi2-ch2: capture device for the third CSI-2 stream
+- rp1-cfe-csi2-ch3: capture device for the fourth CSI-2 stream
+- rp1-cfe-fe-image0: capture device for the first FE output
+- rp1-cfe-fe-image1: capture device for the second FE output
+- rp1-cfe-fe-stats: capture device for the FE statistics
+- rp1-cfe-fe-config: output device for FE configuration
+
+rp1-cfe-csi2-chX
+----------------
+
+The rp1-cfe-csi2-chX capture devices are normal V4L2 capture devices which
+can be used to capture video frames or metadata received from the CSI-2.
+
+rp1-cfe-fe-image0, rp1-cfe-fe-image1
+------------------------------------
+
+The rp1-cfe-fe-image0 and rp1-cfe-fe-image1 capture devices are used to write
+the processed frames to memory.
+
+rp1-cfe-fe-stats
+----------------
+
+The format of the FE statistics buffer is defined by
+:c:type:`pisp_statistics` C structure and the meaning of each parameter is
+described in the `PiSP specification` document.
+
+rp1-cfe-fe-config
+-----------------
+
+The format of the FE configuration buffer is defined by
+:c:type:`pisp_fe_config` C structure and the meaning of each parameter is
+described in the `PiSP specification` document.
diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
index b6af448b9fe9..61da154e079a 100644
--- a/Documentation/admin-guide/media/v4l-drivers.rst
+++ b/Documentation/admin-guide/media/v4l-drivers.rst
@@ -26,6 +26,7 @@ Video4Linux (V4L) driver-specific documentation
raspberrypi-pisp-be
rcar-fdp1
rkisp1
+ raspberrypi-rp1-cfe
saa7134
si470x
si4713
--
2.43.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE
2024-09-10 8:07 [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE Tomi Valkeinen
` (2 preceding siblings ...)
2024-09-10 8:07 ` [PATCH v5 4/4] media: admin-guide: Document the Raspberry Pi CFE (rp1-cfe) Tomi Valkeinen
@ 2024-09-26 7:13 ` Tomi Valkeinen
2024-09-26 9:04 ` Jacopo Mondi
3 siblings, 1 reply; 7+ messages in thread
From: Tomi Valkeinen @ 2024-09-26 7:13 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Raspberry Pi Kernel Maintenance,
Broadcom internal kernel review list, Laurent Pinchart,
Sakari Ailus, Jacopo Mondi, Naushir Patuck
Cc: linux-media, linux-kernel, devicetree, linux-rpi-kernel,
linux-arm-kernel, Kieran Bingham, Krzysztof Kozlowski,
Rob Herring, Krzysztof Kozlowski, Florian Fainelli, Rob Herring,
Krzysztof Kozlowski, Conor Dooley
Hi,
On 10/09/2024 11:07, Tomi Valkeinen wrote:
> This series adds support to the CFE hardware block on RaspberryPi 5. The
> CFE (Camera Front End) contains a CSI-2 receiver and Front End, a small
> ISP.
>
> To run this, you need the basic RPi5 kernel support plus relevant dts
> changes to enable the cfe and camera. My work branch with everything
> needed to run CFE can be found from:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git rp1-cfe
>
> A few notes about the patches:
>
> - The original work was done by RaspberryPi, mostly by Naushir Patuck.
> - The second video node only sets V4L2_CAP_META_CAPTURE instead of both
> V4L2_CAP_META_CAPTURE and V4L2_CAP_META_CAPTURE like the other nodes.
> This is a temporary workaround for userspace (libcamera), and
> hopefully can be removed soon.
>
> I have tested this with:
> - A single IMX219 sensor connected to the RPi5's CSI-2 port
> - Arducam's UB960 FPD-Link board with four imx219 sensors connected
>
> Tomi
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> Changes in v5:
> - Add "depends on PM". The platforms that use CFE will always have PM in
> practice, and it's not worth supporting both the PM and !PM cases as
> it adds complexity to the driver.
> - Link to v4: https://lore.kernel.org/r/20240904-rp1-cfe-v4-0-f1b5b3d69c81@ideasonboard.com
Is this solution to the PM issue ok for everyone? It feels most sensible
to me. Any other comments?
Tomi
> Changes in v4:
> - Drop unnecessary clock-lanes from the DT bindings
> - Drop unnecessary linux-media from MAINTAINERS entry
> - Drop unnecessary conversion to bool with !!
> - Don't set cap->bus_info in cfe_querycap()
> - Make debugfs files not readable by the world
> - Check the return value of v4l2_fwnode_endpoint_parse()
> - Remove the code dealing with remote_ep_fwnode. Instead use
> v4l2_create_fwnode_links_to_pad() and media_pad_remote_pad_unique() to
> create the link and get the pad index.
> - Add cfe/csi2/fe/dphy argument to the respective dbg/info/err print
> macros.
> - Drop some debug prints and add a few, clarifying the prints for
> enabling and disabling the streams.
> - Some cosmetic changes (linefeed, drop unnecessary assignment, move a
> define)
> - Link to v3: https://lore.kernel.org/r/20240815-rp1-cfe-v3-0-e15a979db327@ideasonboard.com
>
> Changes in v3:
> - Based on v6.11-rc3. The PiSP BE series is now in upstream so no extra
> dependencies are needed.
> - Fixed cfe_remove() return value, as the .remove hook has changed
> - Added Krzysztof's Rb.
> - Link to v2: https://lore.kernel.org/r/20240620-rp1-cfe-v2-0-b8b48fdba3b3@ideasonboard.com
>
> Changes in v2:
> - Change the compatible string back to raspberrypi,rp1-cfe from raspberrypi,rpi5-rp1-cfe
> - Drop the references to rp1 headers in the DT binding example. This
> allows compiling the example without the rp1 support.
> - Fix missing remap lines for mono formats
> - Fix csi2_pad_set_fmt() so that the format can be changed back to the
> sink's format from 16-bit or compressed format.
> - Link to v1: https://lore.kernel.org/r/20240318-rp1-cfe-v1-0-ac6d960ff22d@ideasonboard.com
>
> ---
> Tomi Valkeinen (4):
> media: uapi: Add meta formats for PiSP FE config and stats
> dt-bindings: media: Add bindings for raspberrypi,rp1-cfe
> media: raspberrypi: Add support for RP1-CFE
> media: admin-guide: Document the Raspberry Pi CFE (rp1-cfe)
>
> .../admin-guide/media/raspberrypi-rp1-cfe.dot | 27 +
> .../admin-guide/media/raspberrypi-rp1-cfe.rst | 78 +
> Documentation/admin-guide/media/v4l-drivers.rst | 1 +
> .../bindings/media/raspberrypi,rp1-cfe.yaml | 93 +
> .../userspace-api/media/v4l/meta-formats.rst | 1 +
> .../userspace-api/media/v4l/metafmt-pisp-fe.rst | 39 +
> MAINTAINERS | 7 +
> drivers/media/platform/raspberrypi/Kconfig | 1 +
> drivers/media/platform/raspberrypi/Makefile | 1 +
> drivers/media/platform/raspberrypi/rp1-cfe/Kconfig | 15 +
> .../media/platform/raspberrypi/rp1-cfe/Makefile | 6 +
> .../media/platform/raspberrypi/rp1-cfe/cfe-fmts.h | 332 +++
> .../media/platform/raspberrypi/rp1-cfe/cfe-trace.h | 196 ++
> drivers/media/platform/raspberrypi/rp1-cfe/cfe.c | 2487 ++++++++++++++++++++
> drivers/media/platform/raspberrypi/rp1-cfe/cfe.h | 43 +
> drivers/media/platform/raspberrypi/rp1-cfe/csi2.c | 583 +++++
> drivers/media/platform/raspberrypi/rp1-cfe/csi2.h | 89 +
> drivers/media/platform/raspberrypi/rp1-cfe/dphy.c | 180 ++
> drivers/media/platform/raspberrypi/rp1-cfe/dphy.h | 27 +
> .../media/platform/raspberrypi/rp1-cfe/pisp-fe.c | 581 +++++
> .../media/platform/raspberrypi/rp1-cfe/pisp-fe.h | 53 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
> .../uapi/linux/media/raspberrypi/pisp_fe_config.h | 273 +++
> .../linux/media/raspberrypi/pisp_fe_statistics.h | 64 +
> include/uapi/linux/videodev2.h | 2 +
> 25 files changed, 5181 insertions(+)
> ---
> base-commit: 431c1646e1f86b949fa3685efc50b660a364c2b6
> change-id: 20240314-rp1-cfe-142b628b7214
>
> Best regards,
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE
2024-09-26 7:13 ` [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE Tomi Valkeinen
@ 2024-09-26 9:04 ` Jacopo Mondi
2024-09-26 9:15 ` Sakari Ailus
0 siblings, 1 reply; 7+ messages in thread
From: Jacopo Mondi @ 2024-09-26 9:04 UTC (permalink / raw)
To: Tomi Valkeinen
Cc: Mauro Carvalho Chehab, Raspberry Pi Kernel Maintenance,
Broadcom internal kernel review list, Laurent Pinchart,
Sakari Ailus, Jacopo Mondi, Naushir Patuck, linux-media,
linux-kernel, devicetree, linux-rpi-kernel, linux-arm-kernel,
Kieran Bingham, Krzysztof Kozlowski, Rob Herring,
Krzysztof Kozlowski, Florian Fainelli, Rob Herring,
Krzysztof Kozlowski, Conor Dooley
Hi Tomi
On Thu, Sep 26, 2024 at 10:13:29AM GMT, Tomi Valkeinen wrote:
> Hi,
>
> On 10/09/2024 11:07, Tomi Valkeinen wrote:
> > This series adds support to the CFE hardware block on RaspberryPi 5. The
> > CFE (Camera Front End) contains a CSI-2 receiver and Front End, a small
> > ISP.
> >
> > To run this, you need the basic RPi5 kernel support plus relevant dts
> > changes to enable the cfe and camera. My work branch with everything
> > needed to run CFE can be found from:
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git rp1-cfe
> >
> > A few notes about the patches:
> >
> > - The original work was done by RaspberryPi, mostly by Naushir Patuck.
> > - The second video node only sets V4L2_CAP_META_CAPTURE instead of both
> > V4L2_CAP_META_CAPTURE and V4L2_CAP_META_CAPTURE like the other nodes.
> > This is a temporary workaround for userspace (libcamera), and
> > hopefully can be removed soon.
> >
> > I have tested this with:
> > - A single IMX219 sensor connected to the RPi5's CSI-2 port
> > - Arducam's UB960 FPD-Link board with four imx219 sensors connected
> >
> > Tomi
> >
> > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > ---
> > Changes in v5:
> > - Add "depends on PM". The platforms that use CFE will always have PM in
> > practice, and it's not worth supporting both the PM and !PM cases as
> > it adds complexity to the driver.
> > - Link to v4: https://lore.kernel.org/r/20240904-rp1-cfe-v4-0-f1b5b3d69c81@ideasonboard.com
>
> Is this solution to the PM issue ok for everyone? It feels most sensible to
> me. Any other comments?
There was consensus at the media summit that platform drivers are
allowed to depend on PM.
I'll do the same for my latest PiSP BE series.
>
> Tomi
>
> > Changes in v4:
> > - Drop unnecessary clock-lanes from the DT bindings
> > - Drop unnecessary linux-media from MAINTAINERS entry
> > - Drop unnecessary conversion to bool with !!
> > - Don't set cap->bus_info in cfe_querycap()
> > - Make debugfs files not readable by the world
> > - Check the return value of v4l2_fwnode_endpoint_parse()
> > - Remove the code dealing with remote_ep_fwnode. Instead use
> > v4l2_create_fwnode_links_to_pad() and media_pad_remote_pad_unique() to
> > create the link and get the pad index.
> > - Add cfe/csi2/fe/dphy argument to the respective dbg/info/err print
> > macros.
> > - Drop some debug prints and add a few, clarifying the prints for
> > enabling and disabling the streams.
> > - Some cosmetic changes (linefeed, drop unnecessary assignment, move a
> > define)
> > - Link to v3: https://lore.kernel.org/r/20240815-rp1-cfe-v3-0-e15a979db327@ideasonboard.com
> >
> > Changes in v3:
> > - Based on v6.11-rc3. The PiSP BE series is now in upstream so no extra
> > dependencies are needed.
> > - Fixed cfe_remove() return value, as the .remove hook has changed
> > - Added Krzysztof's Rb.
> > - Link to v2: https://lore.kernel.org/r/20240620-rp1-cfe-v2-0-b8b48fdba3b3@ideasonboard.com
> >
> > Changes in v2:
> > - Change the compatible string back to raspberrypi,rp1-cfe from raspberrypi,rpi5-rp1-cfe
> > - Drop the references to rp1 headers in the DT binding example. This
> > allows compiling the example without the rp1 support.
> > - Fix missing remap lines for mono formats
> > - Fix csi2_pad_set_fmt() so that the format can be changed back to the
> > sink's format from 16-bit or compressed format.
> > - Link to v1: https://lore.kernel.org/r/20240318-rp1-cfe-v1-0-ac6d960ff22d@ideasonboard.com
> >
> > ---
> > Tomi Valkeinen (4):
> > media: uapi: Add meta formats for PiSP FE config and stats
> > dt-bindings: media: Add bindings for raspberrypi,rp1-cfe
> > media: raspberrypi: Add support for RP1-CFE
> > media: admin-guide: Document the Raspberry Pi CFE (rp1-cfe)
> >
> > .../admin-guide/media/raspberrypi-rp1-cfe.dot | 27 +
> > .../admin-guide/media/raspberrypi-rp1-cfe.rst | 78 +
> > Documentation/admin-guide/media/v4l-drivers.rst | 1 +
> > .../bindings/media/raspberrypi,rp1-cfe.yaml | 93 +
> > .../userspace-api/media/v4l/meta-formats.rst | 1 +
> > .../userspace-api/media/v4l/metafmt-pisp-fe.rst | 39 +
> > MAINTAINERS | 7 +
> > drivers/media/platform/raspberrypi/Kconfig | 1 +
> > drivers/media/platform/raspberrypi/Makefile | 1 +
> > drivers/media/platform/raspberrypi/rp1-cfe/Kconfig | 15 +
> > .../media/platform/raspberrypi/rp1-cfe/Makefile | 6 +
> > .../media/platform/raspberrypi/rp1-cfe/cfe-fmts.h | 332 +++
> > .../media/platform/raspberrypi/rp1-cfe/cfe-trace.h | 196 ++
> > drivers/media/platform/raspberrypi/rp1-cfe/cfe.c | 2487 ++++++++++++++++++++
> > drivers/media/platform/raspberrypi/rp1-cfe/cfe.h | 43 +
> > drivers/media/platform/raspberrypi/rp1-cfe/csi2.c | 583 +++++
> > drivers/media/platform/raspberrypi/rp1-cfe/csi2.h | 89 +
> > drivers/media/platform/raspberrypi/rp1-cfe/dphy.c | 180 ++
> > drivers/media/platform/raspberrypi/rp1-cfe/dphy.h | 27 +
> > .../media/platform/raspberrypi/rp1-cfe/pisp-fe.c | 581 +++++
> > .../media/platform/raspberrypi/rp1-cfe/pisp-fe.h | 53 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
> > .../uapi/linux/media/raspberrypi/pisp_fe_config.h | 273 +++
> > .../linux/media/raspberrypi/pisp_fe_statistics.h | 64 +
> > include/uapi/linux/videodev2.h | 2 +
> > 25 files changed, 5181 insertions(+)
> > ---
> > base-commit: 431c1646e1f86b949fa3685efc50b660a364c2b6
> > change-id: 20240314-rp1-cfe-142b628b7214
> >
> > Best regards,
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v5 0/4] media: raspberrypi: Support RPi5's CFE
2024-09-26 9:04 ` Jacopo Mondi
@ 2024-09-26 9:15 ` Sakari Ailus
0 siblings, 0 replies; 7+ messages in thread
From: Sakari Ailus @ 2024-09-26 9:15 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Tomi Valkeinen, Mauro Carvalho Chehab,
Raspberry Pi Kernel Maintenance,
Broadcom internal kernel review list, Laurent Pinchart,
Naushir Patuck, linux-media, linux-kernel, devicetree,
linux-rpi-kernel, linux-arm-kernel, Kieran Bingham,
Krzysztof Kozlowski, Rob Herring, Krzysztof Kozlowski,
Florian Fainelli, Rob Herring, Krzysztof Kozlowski, Conor Dooley
Hi Jacopo,
On Thu, Sep 26, 2024 at 11:04:36AM +0200, Jacopo Mondi wrote:
> Hi Tomi
>
> On Thu, Sep 26, 2024 at 10:13:29AM GMT, Tomi Valkeinen wrote:
> > Hi,
> >
> > On 10/09/2024 11:07, Tomi Valkeinen wrote:
> > > This series adds support to the CFE hardware block on RaspberryPi 5. The
> > > CFE (Camera Front End) contains a CSI-2 receiver and Front End, a small
> > > ISP.
> > >
> > > To run this, you need the basic RPi5 kernel support plus relevant dts
> > > changes to enable the cfe and camera. My work branch with everything
> > > needed to run CFE can be found from:
> > >
> > > git://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git rp1-cfe
> > >
> > > A few notes about the patches:
> > >
> > > - The original work was done by RaspberryPi, mostly by Naushir Patuck.
> > > - The second video node only sets V4L2_CAP_META_CAPTURE instead of both
> > > V4L2_CAP_META_CAPTURE and V4L2_CAP_META_CAPTURE like the other nodes.
> > > This is a temporary workaround for userspace (libcamera), and
> > > hopefully can be removed soon.
> > >
> > > I have tested this with:
> > > - A single IMX219 sensor connected to the RPi5's CSI-2 port
> > > - Arducam's UB960 FPD-Link board with four imx219 sensors connected
> > >
> > > Tomi
> > >
> > > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > > ---
> > > Changes in v5:
> > > - Add "depends on PM". The platforms that use CFE will always have PM in
> > > practice, and it's not worth supporting both the PM and !PM cases as
> > > it adds complexity to the driver.
> > > - Link to v4: https://lore.kernel.org/r/20240904-rp1-cfe-v4-0-f1b5b3d69c81@ideasonboard.com
> >
> > Is this solution to the PM issue ok for everyone? It feels most sensible to
> > me. Any other comments?
>
> There was consensus at the media summit that platform drivers are
> allowed to depend on PM.
I'd say you should have generally a good idea these devices aren't found in
systems without runtime PM support and that mostly applies to platform
drivers.
>
> I'll do the same for my latest PiSP BE series.
Ack.
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 7+ messages in thread