* [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver
@ 2024-11-25 9:21 Yuji Ishikawa
2024-11-25 9:21 ` [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver Yuji Ishikawa
` (5 more replies)
0 siblings, 6 replies; 34+ messages in thread
From: Yuji Ishikawa @ 2024-11-25 9:21 UTC (permalink / raw)
To: Laurent Pinchart, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Sakari Ailus, Hans Verkuil,
Nobuhiro Iwamatsu, Yuji Ishikawa
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
This series is the Video Input Interface driver
for Toshiba's ARM SoC, Visconti.
This provides DT binding documentation,
device driver, documentation and MAINTAINER files.
A visconti VIIF driver instance exposes
1 media control device file, 3 video device files for capture
and 2 video device files for controlling image signal processor.
Detailed HW/SW are described in documentation directory.
The VIIF hardware has CSI2 receiver,
image signal processor and video DMAC.
The device driver depends on two other drivers under development;
clock framework driver and IOMMU driver.
Corresponding features will be added later.
Best regards,
Yuji
Changelog v2:
- Resend v1 because a patch exceeds size limit.
Changelog v3:
- Add documentation to describe SW and HW
- Adapted to media control framework
- Introduced ISP subdevice, capture device
- Remove private IOCTLs and add vendor specific V4L2 controls
- Change function name avoiding camelcase and uppercase letters
Changelog v4:
- Split patches because a patch exceeds size limit
- fix dt-bindings document
- stop specifying ID numbers for driver instance explicitly at device tree
- use pm_runtime to trigger initialization of HW
along with open/close of device files.
- add a entry for a header file at MAINTAINERS file
Changelog v5:
- Fix coding style problem in viif.c (patch 2/6)
Changelog v6:
- add register definition of BUS-IF and MPU in dt-bindings
- add CSI2RX subdevice (separated from ISP subdevice)
- change directory layout (moved to media/platform/toshiba/visconti)
- change source file layout (removed hwd_xxxx.c)
- pointer to userland memory is removed from uAPI parameters
- change register access (from struct style to macro style)
- remove unused macros
Changelog v7:
- remove redundant "bindings" from header and description text
- fix multiline text of "description"
- change "compatible" to "visconti5-viif"
- explicitly define allowed properties for port::endpoint
- remove unused variables
- update kerneldoc comments
- update references to headers
Changelog v8:
- rename bindings description file
- remove/simplify items in bindings
- update operations around v4l2_async_notifier
- use v4l2_async_connection instead of v4l2_async_subdev
- use dev_err_probe()
- better error handling at probe
- remove redundant mutex
- add V4L2_CTRL_TYPE_VISCONTI_ISP constant
Changelog v9:
- dictionary ordering of dt-bindings properties
- applied sparse checker
- call div64_u64 for 64bit division
- rebase to media_staging tree
- fix warning for cast between ptr and dma_addr_t
Changelog v10:
- add an independent entry in MAINTAINERS
- add paddings to uAPI structs
- use parameter buffer to control ISP (instead of vendor specific controls)
Changelog v11:
- stop merging sensor's controls and capture device's
- fix strange indents at initializations
- remove feature VB2_USERPTR from viif_params and viif_stats
- fix usage in the document
Changelog v12:
- Separated CSI2RX driver and made it independent driver
- Add a bindings for CSI2RX driver
- Add description of parameter/statistics interface to v4l2-ioctl.c
- use PM_RUNTIME_OPS macro for power management routines
- use v4l2_subdev_enable_streams() to start streaming
- implement callback enable_streams and disable_streams,
instead of s_stream
- add spinlocks for variables shared among interrupt handlers
- use guard(spinlock)(locked_variable) macros
- call pm_runtime APIs at start/stop streaming,
instead of file handle callbacks
- add new "resizer" subdevice between ISP and Capture devices.
- update capability of sub path capture: capture only RAW8 or RAW16
- document: add description of CSI2RX driver
- document: add description of resizer subdevice
- document: add block diagrams of VIIF and ISP
- document: update usage of the driver
Yuji Ishikawa (8):
dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI
CSI-2 Receiver
dt-bindings: media: platform: visconti: Add Toshiba Visconti Video
Input Interface
media: uapi: add visconti viif meta buffer format
media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver
media: platform: visconti: Add Toshiba Visconti Video Input Interface
driver
media: platform: visconti: Add streaming interface for ISP parameters
and status
documentation: media: add documentation for Toshiba Visconti Video
Input Interface driver
MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface
.../admin-guide/media/v4l-drivers.rst | 1 +
.../admin-guide/media/visconti-viif.dot | 22 +
.../admin-guide/media/visconti-viif.rst | 435 ++++
.../media/toshiba,visconti5-csi2rx.yaml | 104 +
.../media/toshiba,visconti5-viif.yaml | 95 +
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../media/v4l/metafmt-visconti-viif.rst | 48 +
MAINTAINERS | 12 +
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/toshiba/Kconfig | 6 +
drivers/media/platform/toshiba/Makefile | 2 +
.../media/platform/toshiba/visconti/Kconfig | 34 +
.../media/platform/toshiba/visconti/Makefile | 10 +
.../platform/toshiba/visconti/csi2rx_drv.c | 791 +++++++
.../media/platform/toshiba/visconti/viif.c | 598 +++++
.../media/platform/toshiba/visconti/viif.h | 379 +++
.../platform/toshiba/visconti/viif_capture.c | 1285 +++++++++++
.../platform/toshiba/visconti/viif_capture.h | 21 +
.../platform/toshiba/visconti/viif_common.c | 239 ++
.../platform/toshiba/visconti/viif_common.h | 45 +
.../platform/toshiba/visconti/viif_isp.c | 911 ++++++++
.../platform/toshiba/visconti/viif_isp.h | 19 +
.../platform/toshiba/visconti/viif_params.c | 2034 +++++++++++++++++
.../platform/toshiba/visconti/viif_params.h | 24 +
.../platform/toshiba/visconti/viif_regs.h | 717 ++++++
.../platform/toshiba/visconti/viif_resizer.c | 491 ++++
.../platform/toshiba/visconti/viif_resizer.h | 18 +
.../platform/toshiba/visconti/viif_stats.c | 301 +++
.../platform/toshiba/visconti/viif_stats.h | 14 +
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
include/uapi/linux/videodev2.h | 4 +
include/uapi/linux/visconti_viif.h | 1921 ++++++++++++++++
33 files changed, 10586 insertions(+)
create mode 100644 Documentation/admin-guide/media/visconti-viif.dot
create mode 100644 Documentation/admin-guide/media/visconti-viif.rst
create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
create mode 100644 Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
create mode 100644 drivers/media/platform/toshiba/Kconfig
create mode 100644 drivers/media/platform/toshiba/Makefile
create mode 100644 drivers/media/platform/toshiba/visconti/Kconfig
create mode 100644 drivers/media/platform/toshiba/visconti/Makefile
create mode 100644 drivers/media/platform/toshiba/visconti/csi2rx_drv.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif.h
create mode 100644 drivers/media/platform/toshiba/visconti/viif_capture.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif_capture.h
create mode 100644 drivers/media/platform/toshiba/visconti/viif_common.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif_common.h
create mode 100644 drivers/media/platform/toshiba/visconti/viif_isp.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif_isp.h
create mode 100644 drivers/media/platform/toshiba/visconti/viif_params.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif_params.h
create mode 100644 drivers/media/platform/toshiba/visconti/viif_regs.h
create mode 100644 drivers/media/platform/toshiba/visconti/viif_resizer.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif_resizer.h
create mode 100644 drivers/media/platform/toshiba/visconti/viif_stats.c
create mode 100644 drivers/media/platform/toshiba/visconti/viif_stats.h
create mode 100644 include/uapi/linux/visconti_viif.h
--
2.25.1
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2024-11-25 9:21 [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver Yuji Ishikawa
@ 2024-11-25 9:21 ` Yuji Ishikawa
2024-11-25 10:11 ` Krzysztof Kozlowski
2025-01-02 9:29 ` Laurent Pinchart
2024-11-25 9:21 ` [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface Yuji Ishikawa
` (4 subsequent siblings)
5 siblings, 2 replies; 34+ messages in thread
From: Yuji Ishikawa @ 2024-11-25 9:21 UTC (permalink / raw)
To: Laurent Pinchart, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Sakari Ailus, Hans Verkuil,
Nobuhiro Iwamatsu, Yuji Ishikawa
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Adds the Device Tree binding documentation that allows to describe
the MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
Changelog v12:
- Newly add bindings for CSI2RX driver
.../media/toshiba,visconti5-csi2rx.yaml | 104 ++++++++++++++++++
1 file changed, 104 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
diff --git a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
new file mode 100644
index 000000000000..5488072bc82a
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
@@ -0,0 +1,104 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
+
+maintainers:
+ - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
+
+description: |-
+ Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI CSI-2 video
+ stream. Use with VIIF device. T.B.D
+
+properties:
+ compatible:
+ const: toshiba,visconti5-csi2rx
+
+ reg:
+ items:
+ - description: Registers for CSI2 receiver control
+
+ interrupts:
+ items:
+ - description: CSI2 Receiver Interrupt
+
+ ports:
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ properties:
+ port@0:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description:
+ Input port node, single endpoint describing the CSI-2 transmitter.
+
+ properties:
+ endpoint:
+ $ref: video-interfaces.yaml#
+ unevaluatedProperties: false
+
+ properties:
+ data-lanes:
+ description: CSI2 receiver supports 1, 2, 3 or 4 data lanes
+ minItems: 1
+ items:
+ - const: 1
+ - const: 2
+ - const: 3
+ - const: 4
+ required:
+ - data-lanes
+
+ port@1:
+ $ref: /schemas/graph.yaml#/properties/port
+ description:
+ Output port node, single endpoint describing the Visconti VIIF.
+
+ required:
+ - port@0
+ - port@1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - ports
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ csi2rx@1c008000 {
+ compatible = "toshiba,visconti5-csi2rx";
+ reg = <0 0x1c008000 0 0x400>;
+ interrupts = <GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ port@0 {
+ reg = <0>;
+ csi2rx_in0: endpoint {
+ data-lanes = <1 2>;
+ remote-endpoint = <&imx219_out0>;
+ };
+ };
+ port@1 {
+ reg = <1>;
+ csi2rx_out0: endpoint {
+ remote-endpoint = <&csi_in0>;
+ };
+ };
+ };
+ };
+ };
--
2.25.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-11-25 9:21 [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver Yuji Ishikawa
2024-11-25 9:21 ` [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver Yuji Ishikawa
@ 2024-11-25 9:21 ` Yuji Ishikawa
2024-11-25 10:07 ` Krzysztof Kozlowski
2025-01-02 9:56 ` Laurent Pinchart
2024-11-25 9:21 ` [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format Yuji Ishikawa
` (3 subsequent siblings)
5 siblings, 2 replies; 34+ messages in thread
From: Yuji Ishikawa @ 2024-11-25 9:21 UTC (permalink / raw)
To: Laurent Pinchart, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Sakari Ailus, Hans Verkuil,
Nobuhiro Iwamatsu, Yuji Ishikawa
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Adds the Device Tree binding documentation that allows to describe
the Video Input Interface found in Toshiba Visconti SoCs.
Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
Changelog v2:
- no change
Changelog v3:
- no change
Changelog v4:
- fix style problems at the v3 patch
- remove "index" member
- update example
Changelog v5:
- no change
Changelog v6:
- add register definition of BUS-IF and MPU
Changelog v7:
- remove trailing "bindings" from commit header message
- remove trailing "Device Tree Bindings" from title
- fix text wrapping of description
- change compatible to visconti5-viif
- explicitly define allowed properties for port::endpoint
Changelog v8:
- Suggestion from Krzysztof Kozlowski
- rename bindings description file
- use block style array instead of inline style
- remove clock-lane (as it is fixed at position 0)
- update sample node's name
- use lowercase hex for literals
- Suggestion from Laurent Pinchart
- update description message port::description
- remove port::endpoint::bus-type as it is fixed to <4>
- remove port::endpoint::clock-lanes from example
- add port::endpoint::data-lanes to required parameters list
- fix sequence of data-lanes: <1 2 3 4> because current driver does not support data reordering
- update port::endpoint::data-lanes::description
- remove redundant type definition for port::endpoint::data-lanes
Changelog v9:
- place "required" after "properties"
- dictionary ordering of properties
Changelog v10:
- no change
Changelog v11:
- no change
Changelog v12:
- remove property "clock-noncontinuous" as VIIF switches both modes automatically
- remove property "link-frequencies" as VIIF does not use the information
- remove reg[2] and interrupts[3] which are used for CSI2RX driver
- update example to refer csi2rx for remote-endpoint
.../media/toshiba,visconti5-viif.yaml | 95 +++++++++++++++++++
1 file changed, 95 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
diff --git a/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
new file mode 100644
index 000000000000..ef0452a47e98
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
@@ -0,0 +1,95 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/toshiba,visconti5-viif.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Toshiba Visconti5 SoC Video Input Interface
+
+maintainers:
+ - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
+
+description: |-
+ Toshiba Visconti5 SoC Video Input Interface (VIIF) receives videostream
+ from MIPI CSI-2 receiver device, processes the stream with image signal
+ processors (L1ISP, L2ISP), then stores pictures to main memory.
+
+properties:
+ compatible:
+ const: toshiba,visconti5-viif
+
+ reg:
+ items:
+ - description: Registers for capture control
+ - description: Registers for bus interface unit control
+ - description: Registers for Memory Protection Unit
+
+ interrupts:
+ items:
+ - description: Sync Interrupt
+ - description: Status (Error) Interrupt
+ - description: L1ISP Interrupt
+
+ port:
+ $ref: /schemas/graph.yaml#/$defs/port-base
+ unevaluatedProperties: false
+ description: CSI-2 input port, with a single endpoint connected to the CSI-2 transmitter.
+
+ properties:
+ endpoint:
+ $ref: video-interfaces.yaml#
+ additionalProperties: false
+
+ properties:
+ data-lanes:
+ description: VIIF supports 1, 2, 3 or 4 data lanes
+ minItems: 1
+ items:
+ - const: 1
+ - const: 2
+ - const: 3
+ - const: 4
+
+ remote-endpoint: true
+
+ required:
+ - data-lanes
+ - remote-endpoint
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - port
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/interrupt-controller/irq.h>
+
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ video@1c000000 {
+ compatible = "toshiba,visconti5-viif";
+ reg = <0 0x1c000000 0 0x6000>,
+ <0 0x1c00e000 0 0x1000>,
+ <0 0x2417a000 0 0x1000>;
+ interrupts = <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
+
+ port {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ csi_in0: endpoint {
+ data-lanes = <1 2>;
+ remote-endpoint = <&csi2rx_out0>;
+ };
+ };
+ };
+ };
--
2.25.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format
2024-11-25 9:21 [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver Yuji Ishikawa
2024-11-25 9:21 ` [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver Yuji Ishikawa
2024-11-25 9:21 ` [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface Yuji Ishikawa
@ 2024-11-25 9:21 ` Yuji Ishikawa
2025-01-02 13:10 ` Laurent Pinchart
2024-11-25 9:21 ` [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver Yuji Ishikawa
` (2 subsequent siblings)
5 siblings, 1 reply; 34+ messages in thread
From: Yuji Ishikawa @ 2024-11-25 9:21 UTC (permalink / raw)
To: Laurent Pinchart, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Sakari Ailus, Hans Verkuil,
Nobuhiro Iwamatsu, Yuji Ishikawa
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Adds the Toshiba Visconti VIIF specific metadata format
- V4L2_META_FMT_VISCONTI_VIIF_PARAMS for ISP parameters
- V4L2_META_FMT_VISCONTI_VIIF_STATS for ISP statistics
Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
---
Changelog v10:
- add entry for V4L2_META_FMT_VISCONTI_VIIF_PARAMS
- add entry for V4L2_META_FMT_VISCONTI_VIIF_STATS
Changelog v11:
- no change
Changelog v12:
- add description for meta formats at v4l2-ioctl.c
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
include/uapi/linux/videodev2.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 0304daa8471d..f7facb63b8ea 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1470,6 +1470,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_RPI_BE_CFG: descr = "RPi PiSP BE Config format"; break;
case V4L2_META_FMT_RPI_FE_CFG: descr = "RPi PiSP FE Config format"; break;
case V4L2_META_FMT_RPI_FE_STATS: descr = "RPi PiSP FE Statistics format"; break;
+ case V4L2_META_FMT_VISCONTI_VIIF_PARAMS: descr = "Visconti ISP Parameters"; break;
+ case V4L2_META_FMT_VISCONTI_VIIF_STATS: descr = "Visconti ISP Statistics"; break;
case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break;
case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8-bit Generic Meta, 10b CSI-2"; break;
case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8-bit Generic Meta, 12b CSI-2"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index a5418759e2ba..9e1f66fdf038 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -863,6 +863,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_RPI_FE_CFG v4l2_fourcc('R', 'P', 'F', 'C') /* PiSP FE configuration */
#define V4L2_META_FMT_RPI_FE_STATS v4l2_fourcc('R', 'P', 'F', 'S') /* PiSP FE stats */
+/* Vendor specific - used for Visconti VIIF sub-system */
+#define V4L2_META_FMT_VISCONTI_VIIF_PARAMS v4l2_fourcc('V', 'I', 'F', 'P') /* ISP Params */
+#define V4L2_META_FMT_VISCONTI_VIIF_STATS v4l2_fourcc('V', 'I', 'F', 'S') /* ISP Stats */
+
#ifdef __KERNEL__
/*
* Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
--
2.25.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver
2024-11-25 9:21 [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver Yuji Ishikawa
` (2 preceding siblings ...)
2024-11-25 9:21 ` [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format Yuji Ishikawa
@ 2024-11-25 9:21 ` Yuji Ishikawa
2025-01-02 13:08 ` Laurent Pinchart
2024-11-25 9:21 ` [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver Yuji Ishikawa
2024-11-25 9:21 ` [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface Yuji Ishikawa
5 siblings, 1 reply; 34+ messages in thread
From: Yuji Ishikawa @ 2024-11-25 9:21 UTC (permalink / raw)
To: Laurent Pinchart, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Sakari Ailus, Hans Verkuil,
Nobuhiro Iwamatsu, Yuji Ishikawa
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Add support to MIPI CSI-2 Receiver on Toshiba Visconti ARM SoCs.
This driver is used with Visconti Video Input Interface driver.
Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
---
Changelog v12:
- Separate CSI2RX driver and made it independent driver
- viif_csi2rx subdevice driver (in v11 patch) was removed.
- dictionary order at Kconfig and Makefile
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/toshiba/Kconfig | 6 +
drivers/media/platform/toshiba/Makefile | 2 +
.../media/platform/toshiba/visconti/Kconfig | 16 +
.../media/platform/toshiba/visconti/Makefile | 8 +
.../platform/toshiba/visconti/csi2rx_drv.c | 791 ++++++++++++++++++
7 files changed, 825 insertions(+)
create mode 100644 drivers/media/platform/toshiba/Kconfig
create mode 100644 drivers/media/platform/toshiba/Makefile
create mode 100644 drivers/media/platform/toshiba/visconti/Kconfig
create mode 100644 drivers/media/platform/toshiba/visconti/Makefile
create mode 100644 drivers/media/platform/toshiba/visconti/csi2rx_drv.c
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 85d2627776b6..761b15b07b90 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -86,6 +86,7 @@ source "drivers/media/platform/samsung/Kconfig"
source "drivers/media/platform/st/Kconfig"
source "drivers/media/platform/sunxi/Kconfig"
source "drivers/media/platform/ti/Kconfig"
+source "drivers/media/platform/toshiba/Kconfig"
source "drivers/media/platform/verisilicon/Kconfig"
source "drivers/media/platform/via/Kconfig"
source "drivers/media/platform/xilinx/Kconfig"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index ace4e34483dd..917145fe5171 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -29,6 +29,7 @@ obj-y += samsung/
obj-y += st/
obj-y += sunxi/
obj-y += ti/
+obj-y += toshiba/
obj-y += verisilicon/
obj-y += via/
obj-y += xilinx/
diff --git a/drivers/media/platform/toshiba/Kconfig b/drivers/media/platform/toshiba/Kconfig
new file mode 100644
index 000000000000..f02983f4fc97
--- /dev/null
+++ b/drivers/media/platform/toshiba/Kconfig
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+comment "Toshiba media platform drivers"
+
+source "drivers/media/platform/toshiba/visconti/Kconfig"
+
diff --git a/drivers/media/platform/toshiba/Makefile b/drivers/media/platform/toshiba/Makefile
new file mode 100644
index 000000000000..2bce85ef3b48
--- /dev/null
+++ b/drivers/media/platform/toshiba/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-y += visconti/
diff --git a/drivers/media/platform/toshiba/visconti/Kconfig b/drivers/media/platform/toshiba/visconti/Kconfig
new file mode 100644
index 000000000000..e5c92d598f8b
--- /dev/null
+++ b/drivers/media/platform/toshiba/visconti/Kconfig
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config VIDEO_VISCONTI_CSI2RX
+ tristate "Visconti MIPI CSI-2 Receiver driver"
+ depends on V4L_PLATFORM_DRIVERS
+ depends on VIDEO_DEV && OF
+ depends on ARCH_VISCONTI || COMPILE_TEST
+ select MEDIA_CONTROLLER
+ select VIDEO_V4L2_SUBDEV_API
+ select V4L2_FWNODE
+ help
+ Support for Toshiba Visconti MIPI CSI-2 receiver,
+ which is used with Visconti Camera Interface driver.
+
+ This driver yields 1 subdevice node for a hardware instance.
+ To compile this driver as a module, choose M here: the
+ module will be called visconti-csi2rx.
diff --git a/drivers/media/platform/toshiba/visconti/Makefile b/drivers/media/platform/toshiba/visconti/Makefile
new file mode 100644
index 000000000000..62a029376134
--- /dev/null
+++ b/drivers/media/platform/toshiba/visconti/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Visconti video input device driver
+#
+
+visconti-csi2rx-objs = csi2rx_drv.o
+
+obj-$(CONFIG_VIDEO_VISCONTI_CSI2RX) += visconti-csi2rx.o
diff --git a/drivers/media/platform/toshiba/visconti/csi2rx_drv.c b/drivers/media/platform/toshiba/visconti/csi2rx_drv.c
new file mode 100644
index 000000000000..94567963872a
--- /dev/null
+++ b/drivers/media/platform/toshiba/visconti/csi2rx_drv.c
@@ -0,0 +1,791 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/* Toshiba Visconti Video Capture Support
+ *
+ * (C) Copyright 2024 TOSHIBA CORPORATION
+ * (C) Copyright 2024 Toshiba Electronic Devices & Storage Corporation
+ */
+
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_graph.h>
+#include <linux/platform_device.h>
+
+#include <media/mipi-csi2.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-fwnode.h>
+#include <media/v4l2-subdev.h>
+
+/* CSI2HOST register space */
+#define REG_CSI2RX_NLANES 0x4
+#define REG_CSI2RX_RESETN 0x8
+#define REG_CSI2RX_INT_ST_MAIN 0xc
+#define REG_CSI2RX_DATA_IDS_1 0x10
+#define REG_CSI2RX_DATA_IDS_2 0x14
+#define REG_CSI2RX_PHY_SHUTDOWNZ 0x40
+#define REG_CSI2RX_PHY_RSTZ 0x44
+
+/* access to dphy external registers */
+#define REG_CSI2RX_PHY_TESTCTRL0 0x50
+#define BIT_TESTCTRL0_CLK_0 0
+#define BIT_TESTCTRL0_CLK_1 BIT(1)
+
+#define REG_CSI2RX_PHY_TESTCTRL1 0x54
+#define BIT_TESTCTRL1_ADDR BIT(16)
+#define MASK_TESTCTRL1_DIN 0xff
+#define MASK_TESTCTRL1_DOUT 0xff00
+
+#define REG_CSI2RX_INT_ST_PHY_FATAL 0xe0
+#define REG_CSI2RX_INT_MSK_PHY_FATAL 0xe4
+#define MASK_PHY_FATAL_ALL 0x0000000f
+
+#define REG_CSI2RX_INT_ST_PKT_FATAL 0xf0
+#define REG_CSI2RX_INT_MSK_PKT_FATAL 0xf4
+#define MASK_PKT_FATAL_ALL 0x0001000f
+
+#define REG_CSI2RX_INT_ST_FRAME_FATAL 0x100
+#define REG_CSI2RX_INT_MSK_FRAME_FATAL 0x104
+#define MASK_FRAME_FATAL_ALL 0x000f0f0f
+
+#define REG_CSI2RX_INT_ST_PHY 0x110
+#define REG_CSI2RX_INT_MSK_PHY 0x114
+#define MASK_PHY_ERROR_ALL 0x000f000f
+
+#define REG_CSI2RX_INT_ST_PKT 0x120
+#define REG_CSI2RX_INT_MSK_PKT 0x124
+#define MASK_PKT_ERROR_ALL 0x000f000f
+
+#define REG_CSI2RX_INT_ST_LINE 0x130
+#define REG_CSI2RX_INT_MSK_LINE 0x134
+#define MASK_LINE_ERROR_ALL 0x00ff00ff
+
+/* DPHY register space */
+enum dphy_testcode {
+ DIG_TESTCODE_EXT = 0,
+ DIG_SYS_0 = 0x001,
+ DIG_SYS_1 = 0x002,
+ DIG_SYS_3 = 0x004,
+ DIG_SYS_7 = 0x008,
+ DIG_RX_STARTUP_OVR_2 = 0x0e2,
+ DIG_RX_STARTUP_OVR_3 = 0x0e3,
+ DIG_RX_STARTUP_OVR_4 = 0x0e4,
+ DIG_RX_STARTUP_OVR_5 = 0x0e5,
+ DIG_CB_2 = 0x1ac,
+ DIG_TERM_CAL_0 = 0x220,
+ DIG_TERM_CAL_1 = 0x221,
+ DIG_TERM_CAL_2 = 0x222,
+ DIG_CLKLANE_LANE_6 = 0x307,
+ DIG_CLKLANE_OFFSET_CAL_0 = 0x39d,
+ DIG_LANE0_OFFSET_CAL_0 = 0x59f,
+ DIG_LANE0_DDL_0 = 0x5e0,
+ DIG_LANE1_OFFSET_CAL_0 = 0x79f,
+ DIG_LANE1_DDL_0 = 0x7e0,
+ DIG_LANE2_OFFSET_CAL_0 = 0x99f,
+ DIG_LANE2_DDL_0 = 0x9e0,
+ DIG_LANE3_OFFSET_CAL_0 = 0xb9f,
+ DIG_LANE3_DDL_0 = 0xbe0,
+};
+
+#define SYS_0_HSFREQRANGE_OVR BIT(5)
+#define SYS_3_NO_REXT BIT(4)
+#define SYS_7_RESERVED FIELD_PREP(0x1f, 0x0c)
+#define SYS_7_DESKEW_POL BIT(5)
+#define STARTUP_OVR_4_CNTVAL FIELD_PREP(0x70, 0x01)
+#define STARTUP_OVR_4_DDL_EN BIT(0)
+#define STARTUP_OVR_5_BYPASS BIT(0)
+#define CB_2_LPRX_BIAS BIT(6)
+#define CB_2_RESERVED FIELD_PREP(0x3f, 0x0b)
+#define CLKLANE_RXHS_PULL_LONG BIT(7)
+
+/* bit mask for calibration result registers */
+#define MASK_TERM_CAL_ERR 0
+#define MASK_TERM_CAL_DONE BIT(7)
+#define MASK_CLK_CAL_ERR BIT(4)
+#define MASK_CLK_CAL_DONE BIT(0)
+#define MASK_CAL_ERR BIT(2)
+#define MASK_CAL_DONE BIT(1)
+#define MASK_DDL_ERR BIT(1)
+#define MASK_DDL_DONE BIT(2)
+
+#define VISCONTI_CSI2RX_ERROR_MONITORS_NUM 8
+
+/**
+ * struct visconti_csi2rx_line_err_target
+ *
+ * Virtual Channel and Data Type pair for CSI2RX line error monitor
+ *
+ * When 0 is set to dt, line error detection is disabled.
+ *
+ * @vc: Virtual Channel to monitor; Range 0..3
+ * @dt: Data Type to monitor; Range 0, 0x10..0x3f
+ */
+struct visconti_csi2rx_line_err_target {
+ u32 vc[VISCONTI_CSI2RX_ERROR_MONITORS_NUM];
+ u32 dt[VISCONTI_CSI2RX_ERROR_MONITORS_NUM];
+};
+
+#define CSI2RX_MIN_DATA_RATE 80U
+#define CSI2RX_MAX_DATA_RATE 1500U
+
+#define VISCONTI_CSI2RX_PAD_SINK 0
+#define VISCONTI_CSI2RX_PAD_SRC 1
+#define VISCONTI_CSI2RX_PAD_NUM 2
+
+#define VISCONTI_CSI2RX_DEF_WIDTH 1920
+#define VISCONTI_CSI2RX_DEF_HEIGHT 1080
+#define VISCONTI_CSI2RX_MIN_WIDTH 640
+#define VISCONTI_CSI2RX_MAX_WIDTH 3840
+#define VISCONTI_CSI2RX_MIN_HEIGHT 480
+#define VISCONTI_CSI2RX_MAX_HEIGHT 2160
+
+struct visconti_csi2rx {
+ struct device *dev;
+ void __iomem *base;
+
+ struct v4l2_subdev subdev;
+ struct media_pad pads[VISCONTI_CSI2RX_PAD_NUM];
+ struct v4l2_async_notifier notifier;
+ struct v4l2_subdev *remote;
+ unsigned int remote_pad;
+
+ unsigned int lanes;
+
+ unsigned int irq;
+};
+
+static inline struct visconti_csi2rx *notifier_to_csi2(struct v4l2_async_notifier *n)
+{
+ return container_of(n, struct visconti_csi2rx, notifier);
+}
+
+static inline struct visconti_csi2rx *sd_to_csi2(struct v4l2_subdev *sd)
+{
+ return container_of(sd, struct visconti_csi2rx, subdev);
+}
+
+static inline void visconti_csi2rx_write(struct visconti_csi2rx *priv, u32 regid, u32 val)
+{
+ writel(val, priv->base + regid);
+}
+
+static inline u32 visconti_csi2rx_read(struct visconti_csi2rx *priv, u32 regid)
+{
+ return readl(priv->base + regid);
+}
+
+static inline void tick_testclk(struct visconti_csi2rx *priv)
+{
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, BIT_TESTCTRL0_CLK_1);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, BIT_TESTCTRL0_CLK_0);
+}
+
+static inline void set_dphy_addr(struct visconti_csi2rx *priv, u32 test_mode)
+{
+ /* select testcode Ex space with top 4bits of test_mode */
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
+ BIT_TESTCTRL1_ADDR | DIG_TESTCODE_EXT);
+ tick_testclk(priv);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1, FIELD_GET(0xf00, test_mode));
+ tick_testclk(priv);
+
+ /* set bottom 8bit of test_mode */
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
+ BIT_TESTCTRL1_ADDR | FIELD_GET(0xff, test_mode));
+ tick_testclk(priv);
+}
+
+static void write_dphy_param(struct visconti_csi2rx *priv, u32 test_mode, u8 test_in)
+{
+ set_dphy_addr(priv, test_mode);
+
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1, (u32)test_in);
+ tick_testclk(priv);
+}
+
+struct csi2rx_dphy_hs_info {
+ u32 rate;
+ u32 hsfreqrange;
+ u32 osc_freq_target;
+};
+
+static const struct csi2rx_dphy_hs_info dphy_hs_info[] = {
+ { 80, 0x0, 0x1cc }, { 85, 0x10, 0x1cc }, { 95, 0x20, 0x1cc }, { 105, 0x30, 0x1cc },
+ { 115, 0x1, 0x1cc }, { 125, 0x11, 0x1cc }, { 135, 0x21, 0x1cc }, { 145, 0x31, 0x1cc },
+ { 155, 0x2, 0x1cc }, { 165, 0x12, 0x1cc }, { 175, 0x22, 0x1cc }, { 185, 0x32, 0x1cc },
+ { 198, 0x3, 0x1cc }, { 213, 0x13, 0x1cc }, { 228, 0x23, 0x1cc }, { 243, 0x33, 0x1cc },
+ { 263, 0x4, 0x1cc }, { 288, 0x14, 0x1cc }, { 313, 0x25, 0x1cc }, { 338, 0x35, 0x1cc },
+ { 375, 0x5, 0x1cc }, { 425, 0x16, 0x1cc }, { 475, 0x26, 0x1cc }, { 525, 0x37, 0x1cc },
+ { 575, 0x7, 0x1cc }, { 625, 0x18, 0x1cc }, { 675, 0x28, 0x1cc }, { 725, 0x39, 0x1cc },
+ { 775, 0x9, 0x1cc }, { 825, 0x19, 0x1cc }, { 875, 0x29, 0x1cc }, { 925, 0x3a, 0x1cc },
+ { 975, 0xa, 0x1cc }, { 1025, 0x1a, 0x1cc }, { 1075, 0x2a, 0x1cc }, { 1125, 0x3b, 0x1cc },
+ { 1175, 0xb, 0x1cc }, { 1225, 0x1b, 0x1cc }, { 1275, 0x2b, 0x1cc }, { 1325, 0x3c, 0x1cc },
+ { 1375, 0xc, 0x1cc }, { 1425, 0x1c, 0x1cc }, { 1475, 0x2c, 0x1cc }
+};
+
+static void get_dphy_hs_transfer_info(u32 dphy_rate, u32 *hsfreqrange, u32 *osc_freq_target)
+{
+ unsigned int i;
+
+ for (i = 1; i < ARRAY_SIZE(dphy_hs_info); i++) {
+ if (dphy_rate < dphy_hs_info[i].rate) {
+ *hsfreqrange = dphy_hs_info[i - 1].hsfreqrange;
+ *osc_freq_target = dphy_hs_info[i - 1].osc_freq_target;
+ return;
+ }
+ }
+
+ /* not found; return the largest entry */
+ *hsfreqrange = dphy_hs_info[ARRAY_SIZE(dphy_hs_info) - 1].hsfreqrange;
+ *osc_freq_target = dphy_hs_info[ARRAY_SIZE(dphy_hs_info) - 1].osc_freq_target;
+}
+
+static void visconti_csi2rx_set_dphy_rate(struct visconti_csi2rx *priv, u32 dphy_rate)
+{
+ u32 hsfreqrange, osc_freq_target;
+
+ get_dphy_hs_transfer_info(dphy_rate, &hsfreqrange, &osc_freq_target);
+
+ write_dphy_param(priv, DIG_SYS_1, (u8)hsfreqrange);
+ write_dphy_param(priv, DIG_SYS_0, SYS_0_HSFREQRANGE_OVR);
+ write_dphy_param(priv, DIG_RX_STARTUP_OVR_5, STARTUP_OVR_5_BYPASS);
+ write_dphy_param(priv, DIG_RX_STARTUP_OVR_4, STARTUP_OVR_4_CNTVAL);
+ write_dphy_param(priv, DIG_CB_2, CB_2_LPRX_BIAS | CB_2_RESERVED);
+ write_dphy_param(priv, DIG_SYS_7, SYS_7_DESKEW_POL | SYS_7_RESERVED);
+ write_dphy_param(priv, DIG_CLKLANE_LANE_6, CLKLANE_RXHS_PULL_LONG);
+ write_dphy_param(priv, DIG_RX_STARTUP_OVR_2, FIELD_GET(0xff, osc_freq_target));
+ write_dphy_param(priv, DIG_RX_STARTUP_OVR_3, FIELD_GET(0xf00, osc_freq_target));
+ write_dphy_param(priv, DIG_RX_STARTUP_OVR_4, STARTUP_OVR_4_CNTVAL | STARTUP_OVR_4_DDL_EN);
+}
+
+static int visconti_csi2rx_initialize(struct visconti_csi2rx *priv, u32 num_lane, u32 dphy_rate,
+ const struct visconti_csi2rx_line_err_target *err_target)
+{
+ u32 val;
+
+ if (dphy_rate < CSI2RX_MIN_DATA_RATE || dphy_rate > CSI2RX_MAX_DATA_RATE) {
+ dev_err(priv->dev, "Unsupported PHY speed (%u Mbps)", dphy_rate);
+ return -ERANGE;
+ }
+
+ /* 1st phase of initialization */
+ visconti_csi2rx_write(priv, REG_CSI2RX_RESETN, 1);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 1);
+ ndelay(15U);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 0);
+
+ /* Configure D-PHY frequency range */
+ visconti_csi2rx_set_dphy_rate(priv, dphy_rate);
+
+ /* 2nd phase of initialization */
+ visconti_csi2rx_write(priv, REG_CSI2RX_NLANES, (num_lane - 1U));
+ ndelay(5U);
+
+ /* Release D-PHY from Reset */
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 1);
+ ndelay(5U);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 1);
+
+ /* configuration of line error target */
+ val = (err_target->vc[3] << 30U) | (err_target->dt[3] << 24U) | (err_target->vc[2] << 22U) |
+ (err_target->dt[2] << 16U) | (err_target->vc[1] << 14U) | (err_target->dt[1] << 8U) |
+ (err_target->vc[0] << 6U) | (err_target->dt[0]);
+ visconti_csi2rx_write(priv, REG_CSI2RX_DATA_IDS_1, val);
+ val = (err_target->vc[7] << 30U) | (err_target->dt[7] << 24U) | (err_target->vc[6] << 22U) |
+ (err_target->dt[6] << 16U) | (err_target->vc[5] << 14U) | (err_target->dt[5] << 8U) |
+ (err_target->vc[4] << 6U) | (err_target->dt[4]);
+ visconti_csi2rx_write(priv, REG_CSI2RX_DATA_IDS_2, val);
+
+ /* configuration of mask */
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY_FATAL, MASK_PHY_FATAL_ALL);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT_FATAL, MASK_PKT_FATAL_ALL);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL, MASK_FRAME_FATAL_ALL);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY, MASK_PHY_ERROR_ALL);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT, MASK_PKT_ERROR_ALL);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_LINE, MASK_LINE_ERROR_ALL);
+
+ return 0;
+}
+
+struct visconti_csi2rx_format {
+ u32 code;
+ unsigned int bpp;
+};
+
+static const struct visconti_csi2rx_format visconti_csi2rx_formats[] = {
+ { .code = MEDIA_BUS_FMT_RGB888_1X24, .bpp = 24 },
+ { .code = MEDIA_BUS_FMT_UYVY8_1X16, .bpp = 16 },
+ { .code = MEDIA_BUS_FMT_UYVY10_1X20, .bpp = 20 },
+ { .code = MEDIA_BUS_FMT_RGB565_1X16, .bpp = 16 },
+ { .code = MEDIA_BUS_FMT_SBGGR8_1X8, .bpp = 8 },
+ { .code = MEDIA_BUS_FMT_SGBRG8_1X8, .bpp = 8 },
+ { .code = MEDIA_BUS_FMT_SGRBG8_1X8, .bpp = 8 },
+ { .code = MEDIA_BUS_FMT_SRGGB8_1X8, .bpp = 8 },
+ { .code = MEDIA_BUS_FMT_SBGGR10_1X10, .bpp = 10 },
+ { .code = MEDIA_BUS_FMT_SGBRG10_1X10, .bpp = 10 },
+ { .code = MEDIA_BUS_FMT_SGRBG10_1X10, .bpp = 10 },
+ { .code = MEDIA_BUS_FMT_SRGGB10_1X10, .bpp = 10 },
+ { .code = MEDIA_BUS_FMT_SBGGR12_1X12, .bpp = 12 },
+ { .code = MEDIA_BUS_FMT_SGBRG12_1X12, .bpp = 12 },
+ { .code = MEDIA_BUS_FMT_SGRBG12_1X12, .bpp = 12 },
+ { .code = MEDIA_BUS_FMT_SRGGB12_1X12, .bpp = 12 },
+ { .code = MEDIA_BUS_FMT_SBGGR14_1X14, .bpp = 14 },
+ { .code = MEDIA_BUS_FMT_SGBRG14_1X14, .bpp = 14 },
+ { .code = MEDIA_BUS_FMT_SGRBG14_1X14, .bpp = 14 },
+ { .code = MEDIA_BUS_FMT_SRGGB14_1X14, .bpp = 14 },
+};
+
+static const struct visconti_csi2rx_format *fmt_for_mbus_code(unsigned int mbus_code)
+{
+ int i;
+
+ for (i = 0; ARRAY_SIZE(visconti_csi2rx_formats); i++)
+ if (visconti_csi2rx_formats[i].code == mbus_code)
+ return &visconti_csi2rx_formats[i];
+}
+
+static unsigned int bpp_for_mbus_code(unsigned int mbus_code)
+{
+ const struct visconti_csi2rx_format *fmt = fmt_for_mbus_code(mbus_code);
+
+ return fmt ? fmt->bpp : 0;
+}
+
+static int64_t get_pixelclock(struct v4l2_subdev *sd)
+{
+ struct v4l2_ctrl *ctrl;
+
+ ctrl = v4l2_ctrl_find(sd->ctrl_handler, V4L2_CID_PIXEL_RATE);
+ if (!ctrl)
+ return -EINVAL;
+
+ return v4l2_ctrl_g_ctrl_int64(ctrl);
+}
+
+static const struct visconti_csi2rx_line_err_target err_target_vc0_alldt = {
+ /* select VC=0 */
+ /* select all supported DataTypes */
+ .dt = {
+ MIPI_CSI2_DT_RGB565,
+ MIPI_CSI2_DT_YUV422_8B,
+ MIPI_CSI2_DT_YUV422_10B,
+ MIPI_CSI2_DT_RGB888,
+ MIPI_CSI2_DT_RAW8,
+ MIPI_CSI2_DT_RAW10,
+ MIPI_CSI2_DT_RAW12,
+ MIPI_CSI2_DT_RAW14,
+ }
+};
+
+static int visconti_csi2rx_start(struct visconti_csi2rx *priv, struct v4l2_subdev_state *state)
+{
+ struct v4l2_mbus_framefmt *sink_fmt;
+ int cur_bpp, dphy_rate;
+ s64 pixelclock;
+
+ /* get bpp for current format */
+ sink_fmt = v4l2_subdev_state_get_format(state, VISCONTI_CSI2RX_PAD_SINK);
+ cur_bpp = bpp_for_mbus_code(sink_fmt->code);
+
+ /* get pixel clock */
+ pixelclock = get_pixelclock(priv->remote);
+ if (pixelclock < 0)
+ return -EINVAL;
+
+ dphy_rate = div64_u64((u64)pixelclock * (u32)cur_bpp, priv->lanes * 1000000);
+
+ ndelay(15U);
+
+ return visconti_csi2rx_initialize(priv, priv->lanes, dphy_rate, &err_target_vc0_alldt);
+}
+
+static void visconti_csi2rx_stop(struct visconti_csi2rx *priv)
+{
+ /* disable interrupt -> make sure registers cleared -> wait for current handlers finish */
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY_FATAL, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT_FATAL, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_LINE, 0);
+ visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PHY_FATAL);
+ visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PKT_FATAL);
+ visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL);
+ visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PHY);
+ visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PKT);
+ visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_LINE);
+ synchronize_irq(priv->irq);
+
+ /* shutdown hardware */
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 0);
+ visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 1);
+ visconti_csi2rx_write(priv, REG_CSI2RX_RESETN, 0);
+}
+
+static int visconti_csi2rx_enable_streams(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
+ u32 pad, u64 streams_mask)
+{
+ struct visconti_csi2rx *priv = sd_to_csi2(sd);
+ struct v4l2_subdev *remote_sd;
+ struct media_pad *remote_pad;
+ int ret;
+
+ remote_pad = media_pad_remote_pad_first(&sd->entity.pads[VISCONTI_CSI2RX_PAD_SINK]);
+ if (!remote_pad)
+ return -ENODEV;
+ remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
+
+ /* enabling: turn on CSI2RX -> turn on sensor */
+ ret = visconti_csi2rx_start(priv, state);
+ if (ret)
+ return ret;
+
+ /* currently CSI2RX supports only stream0 in source pad */
+ ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index, BIT(0));
+ if (ret) {
+ visconti_csi2rx_stop(priv);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int visconti_csi2rx_disable_streams(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
+ u32 pad, u64 streams_mask)
+{
+ struct visconti_csi2rx *priv = sd_to_csi2(sd);
+ struct v4l2_subdev *remote_sd;
+ struct media_pad *remote_pad;
+
+ remote_pad = media_pad_remote_pad_first(&sd->entity.pads[VISCONTI_CSI2RX_PAD_SINK]);
+ if (!remote_pad)
+ return -ENODEV;
+ remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
+
+ /* disabling: turn off sensor -> turn off CSI2RX */
+ v4l2_subdev_disable_streams(remote_sd, remote_pad->index, BIT(0));
+ visconti_csi2rx_stop(priv);
+
+ return 0;
+}
+
+static int visconti_csi2rx_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ if (code->pad == VISCONTI_CSI2RX_PAD_SRC) {
+ const struct v4l2_mbus_framefmt *sink_fmt;
+
+ /* SRC pad supports exactly the same format as SINK pad */
+ if (code->index)
+ return -EINVAL;
+ sink_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SINK);
+ code->code = sink_fmt->code;
+ return 0;
+ }
+
+ if (code->index >= ARRAY_SIZE(visconti_csi2rx_formats))
+ return -EINVAL;
+ code->code = visconti_csi2rx_formats[code->index].code;
+
+ return 0;
+}
+
+static int visconti_csi2rx_init_state(struct v4l2_subdev *sd, struct v4l2_subdev_state *sd_state)
+{
+ struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
+
+ sink_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SINK);
+ src_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SRC);
+
+ sink_fmt->width = VISCONTI_CSI2RX_DEF_WIDTH;
+ sink_fmt->height = VISCONTI_CSI2RX_DEF_HEIGHT;
+ sink_fmt->field = V4L2_FIELD_NONE;
+ sink_fmt->code = visconti_csi2rx_formats[0].code;
+
+ *src_fmt = *sink_fmt;
+
+ return 0;
+}
+
+static int visconti_csi2rx_set_pad_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
+
+ /* SRC PAD has the same format as SINK PAD */
+ if (fmt->pad == 1)
+ return v4l2_subdev_get_fmt(sd, sd_state, fmt);
+
+ sink_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SINK);
+
+ *sink_fmt = fmt->format;
+ sink_fmt->width = clamp_t(u32, fmt->format.width, VISCONTI_CSI2RX_MIN_WIDTH,
+ VISCONTI_CSI2RX_MAX_WIDTH);
+ sink_fmt->height = clamp_t(u32, fmt->format.height, VISCONTI_CSI2RX_MIN_HEIGHT,
+ VISCONTI_CSI2RX_MAX_HEIGHT);
+ if (!fmt_for_mbus_code(sink_fmt->code))
+ sink_fmt->code = visconti_csi2rx_formats[0].code;
+ fmt->format = *sink_fmt;
+
+ /* source pad should have the same format */
+ src_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SRC);
+ *src_fmt = *sink_fmt;
+
+ return 0;
+}
+
+static const struct media_entity_operations visconti_csi2rx_entity_ops = {
+ .link_validate = v4l2_subdev_link_validate,
+};
+
+static const struct v4l2_subdev_video_ops visconti_csi2rx_video_ops = {
+ .s_stream = v4l2_subdev_s_stream_helper,
+};
+
+static const struct v4l2_subdev_pad_ops visconti_csi2rx_pad_ops = {
+ .enum_mbus_code = visconti_csi2rx_enum_mbus_code,
+ .disable_streams = visconti_csi2rx_disable_streams,
+ .enable_streams = visconti_csi2rx_enable_streams,
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = visconti_csi2rx_set_pad_format,
+};
+
+static const struct v4l2_subdev_ops visconti_csi2rx_subdev_ops = {
+ .video = &visconti_csi2rx_video_ops,
+ .pad = &visconti_csi2rx_pad_ops,
+};
+
+static const struct v4l2_subdev_internal_ops visconti_csi2rx_internal_ops = {
+ .init_state = visconti_csi2rx_init_state,
+};
+
+static int visconti_csi2rx_notify_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *subdev,
+ struct v4l2_async_connection *asc)
+{
+ struct visconti_csi2rx *priv = notifier_to_csi2(notifier);
+ int pad;
+
+ pad = media_entity_get_fwnode_pad(&subdev->entity, asc->match.fwnode, MEDIA_PAD_FL_SOURCE);
+ if (pad < 0) {
+ dev_err(priv->dev, "Failed to find pad for %s\n", subdev->name);
+ return pad;
+ }
+
+ priv->remote = subdev;
+ priv->remote_pad = pad;
+
+ return media_create_pad_link(&subdev->entity, pad, &priv->subdev.entity, 0,
+ MEDIA_LNK_FL_ENABLED);
+}
+
+static void visconti_csi2rx_notify_unbind(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *subdev,
+ struct v4l2_async_connection *asc)
+{
+ struct visconti_csi2rx *priv = notifier_to_csi2(notifier);
+
+ priv->remote = NULL;
+}
+
+static const struct v4l2_async_notifier_operations visconti_csi2rx_notify_ops = {
+ .bound = visconti_csi2rx_notify_bound,
+ .unbind = visconti_csi2rx_notify_unbind,
+};
+
+static int visconti_csi2rx_parse_v4l2(struct visconti_csi2rx *priv,
+ struct v4l2_fwnode_endpoint *vep)
+{
+ /* Only port 0 endpoint 0 is valid. */
+ if (vep->base.port || vep->base.id)
+ return -ENOTCONN;
+
+ priv->lanes = vep->bus.mipi_csi2.num_data_lanes;
+
+ /* got trouble */
+ if (vep->bus_type != V4L2_MBUS_CSI2_DPHY) {
+ dev_err(priv->dev, "Specified bus type is not supported\n");
+ return -EINVAL;
+ }
+
+ if (priv->lanes != 1 && priv->lanes != 2 && priv->lanes != 4) {
+ dev_err(priv->dev, "Unsupported number of data-lanes for D-PHY: %u\n", priv->lanes);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int visconti_csi2rx_parse_dt(struct visconti_csi2rx *priv)
+{
+ struct v4l2_async_connection *asc;
+ struct fwnode_handle *fwnode;
+ struct fwnode_handle *ep;
+ struct v4l2_fwnode_endpoint v4l2_ep = {
+ .bus_type = V4L2_MBUS_UNKNOWN,
+ };
+ int ret;
+
+ ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(priv->dev), 0, 0, 0);
+ if (!ep) {
+ dev_err(priv->dev, "Not connected to subdevice\n");
+ return -EINVAL;
+ }
+
+ ret = v4l2_fwnode_endpoint_parse(ep, &v4l2_ep);
+ if (ret) {
+ dev_err(priv->dev, "Could not parse v4l2 endpoint\n");
+ fwnode_handle_put(ep);
+ return -EINVAL;
+ }
+
+ ret = visconti_csi2rx_parse_v4l2(priv, &v4l2_ep);
+ if (ret) {
+ fwnode_handle_put(ep);
+ return ret;
+ }
+
+ fwnode = fwnode_graph_get_remote_endpoint(ep);
+ fwnode_handle_put(ep);
+
+ v4l2_async_subdev_nf_init(&priv->notifier, &priv->subdev);
+ priv->notifier.ops = &visconti_csi2rx_notify_ops;
+
+ asc = v4l2_async_nf_add_fwnode(&priv->notifier, fwnode, struct v4l2_async_connection);
+ fwnode_handle_put(fwnode);
+ if (IS_ERR(asc))
+ return PTR_ERR(asc);
+
+ ret = v4l2_async_nf_register(&priv->notifier);
+ if (ret)
+ v4l2_async_nf_cleanup(&priv->notifier);
+
+ return ret;
+}
+
+static irqreturn_t visconti_csi2rx_irq(int irq, void *dev_id)
+{
+ struct visconti_csi2rx *priv = dev_id;
+ u32 event;
+
+ event = visconti_csi2rx_read(priv, REG_CSI2RX_INT_ST_MAIN);
+ dev_err(priv->dev, "CSI2RX error 0x%x.\n", event);
+
+ return IRQ_HANDLED;
+}
+
+static const struct of_device_id visconti_csi2rx_of_table[] = {
+ {
+ .compatible = "toshiba,visconti5-csi2rx",
+ },
+ {},
+};
+
+static int visconti_csi2rx_probe(struct platform_device *pdev)
+{
+ struct visconti_csi2rx *priv;
+ int irq, ret;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->dev = &pdev->dev;
+
+ priv->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(priv->base)) {
+ dev_err(priv->dev, "Failed to get registers\n");
+ return PTR_ERR(priv->base);
+ }
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+ ret = devm_request_irq(&pdev->dev, irq, visconti_csi2rx_irq, 0, KBUILD_MODNAME, priv);
+ priv->irq = irq;
+ if (ret) {
+ dev_err(priv->dev, "request irq failed: %d\n", ret);
+ return ret;
+ }
+
+ platform_set_drvdata(pdev, priv);
+
+ ret = visconti_csi2rx_parse_dt(priv); /*this function does v4l2_async_nf_register */
+ if (ret)
+ return ret;
+
+ priv->subdev.owner = THIS_MODULE;
+ priv->subdev.dev = &pdev->dev;
+ v4l2_subdev_init(&priv->subdev, &visconti_csi2rx_subdev_ops);
+ v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
+ snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s", KBUILD_MODNAME,
+ dev_name(&pdev->dev));
+
+ priv->subdev.internal_ops = &visconti_csi2rx_internal_ops;
+ priv->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+ priv->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
+ priv->subdev.entity.ops = &visconti_csi2rx_entity_ops;
+
+ priv->pads[VISCONTI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+ priv->pads[VISCONTI_CSI2RX_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
+
+ ret = media_entity_pads_init(&priv->subdev.entity, VISCONTI_CSI2RX_PAD_NUM, priv->pads);
+ if (ret)
+ goto err_cleanup_async;
+
+ ret = v4l2_subdev_init_finalize(&priv->subdev);
+ if (ret)
+ goto err_cleanup_media_entity;
+
+ ret = v4l2_async_register_subdev(&priv->subdev);
+ if (ret < 0)
+ goto err_cleanup_subdev_state;
+
+ return 0;
+
+err_cleanup_subdev_state:
+ v4l2_subdev_cleanup(&priv->subdev);
+
+err_cleanup_media_entity:
+ media_entity_cleanup(&priv->subdev.entity);
+
+err_cleanup_async:
+ v4l2_async_nf_unregister(&priv->notifier);
+ v4l2_async_nf_cleanup(&priv->notifier);
+
+ return ret;
+}
+
+static void visconti_csi2rx_remove(struct platform_device *pdev)
+{
+ struct visconti_csi2rx *priv = platform_get_drvdata(pdev);
+
+ v4l2_async_nf_unregister(&priv->notifier);
+ v4l2_async_nf_cleanup(&priv->notifier);
+ v4l2_async_unregister_subdev(&priv->subdev);
+
+ v4l2_subdev_cleanup(&priv->subdev);
+ media_entity_cleanup(&priv->subdev.entity);
+}
+
+static struct platform_driver visconti_csi2rx_driver = {
+ .probe = visconti_csi2rx_probe,
+ .remove = visconti_csi2rx_remove,
+ .driver = {
+ .name = "visconti_csi2rx_dev",
+ .of_match_table = visconti_csi2rx_of_table,
+ },
+};
+
+module_platform_driver(visconti_csi2rx_driver);
+
+MODULE_AUTHOR("Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>");
+MODULE_DESCRIPTION("Toshiba Visconti CSI-2 receiver driver");
+MODULE_LICENSE("Dual BSD/GPL");
--
2.25.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver
2024-11-25 9:21 [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver Yuji Ishikawa
` (3 preceding siblings ...)
2024-11-25 9:21 ` [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver Yuji Ishikawa
@ 2024-11-25 9:21 ` Yuji Ishikawa
2025-01-02 21:26 ` Laurent Pinchart
2024-11-25 9:21 ` [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface Yuji Ishikawa
5 siblings, 1 reply; 34+ messages in thread
From: Yuji Ishikawa @ 2024-11-25 9:21 UTC (permalink / raw)
To: Laurent Pinchart, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Sakari Ailus, Hans Verkuil,
Nobuhiro Iwamatsu, Yuji Ishikawa
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Added description of Video Input Interface driver of
Toshiba Visconti architecture.
It includes hardware organization, structure of the driver
and metadata format for embedded image signal processor.
Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
---
Changelog v3:
- Newly add documentation to describe SW and HW
Changelog v4:
- no change
Changelog v5:
- no change
Changelog v6:
- add description of CSI2RX subdevice
- add ordering of ioctl(S_FMT) and ioctl(S_EXT_CTRLS)
Changelog v7:
- no change
Changelog v8:
- add usage of V4L2_CTRL_TYPE_VISCONTI_ISP
Changelog v9:
- fix warning: set reference target for keyword V4L2_CTRL_TYPE_VISCONTI_ISP
Changelog v10:
- use parameter buffers instead of compound control
- removed description of vendor specific compound control
- add description of parameter buffers for ISP control
- update directory structure
- remove documents under driver-api
- add documents to admin-guide, userspace-api
Changelog v11:
- update usage of the driver
Changelog v12:
- add description of CSI2RX driver
- description of resizer subdevice
- add block diagrams of VIIF and ISP
- update usage of the driver
.../admin-guide/media/v4l-drivers.rst | 1 +
.../admin-guide/media/visconti-viif.dot | 22 +
.../admin-guide/media/visconti-viif.rst | 435 ++++++++++++++++++
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../media/v4l/metafmt-visconti-viif.rst | 48 ++
5 files changed, 507 insertions(+)
create mode 100644 Documentation/admin-guide/media/visconti-viif.dot
create mode 100644 Documentation/admin-guide/media/visconti-viif.rst
create mode 100644 Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
index b6af448b9fe9..81054512e768 100644
--- a/Documentation/admin-guide/media/v4l-drivers.rst
+++ b/Documentation/admin-guide/media/v4l-drivers.rst
@@ -32,5 +32,6 @@ Video4Linux (V4L) driver-specific documentation
si476x
starfive_camss
vimc
+ visconti-viif
visl
vivid
diff --git a/Documentation/admin-guide/media/visconti-viif.dot b/Documentation/admin-guide/media/visconti-viif.dot
new file mode 100644
index 000000000000..cc75c73336fb
--- /dev/null
+++ b/Documentation/admin-guide/media/visconti-viif.dot
@@ -0,0 +1,22 @@
+digraph board {
+ rankdir=TB
+ n00000001 [label="{{<port0> 0 | <port4> 4} | visconti-viif:isp\n/dev/v4l-subdev0 | {<port1> 1 | <port2> 2 | <port3> 3 | <port5> 5}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000001:port1 -> n00000008:port0
+ n00000001:port2 -> n0000000b:port0
+ n00000001:port3 -> n00000016
+ n00000001:port5 -> n0000001e
+ n00000008 [label="{{<port0> 0} | visconti-viif:resizer0\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000008:port1 -> n0000000e
+ n0000000b [label="{{<port0> 0} | visconti-viif:resizer1\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n0000000b:port1 -> n00000012
+ n0000000e [label="viif_capture_post0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+ n00000012 [label="viif_capture_post1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
+ n00000016 [label="viif_capture_sub\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
+ n0000001a [label="viif_params\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
+ n0000001a -> n00000001:port4
+ n0000001e [label="viif_stats\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
+ n00000030 [label="{{<port0> 0} | visconti_csi2rx 1c008000.csi2rx\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000030:port1 -> n00000001:port0
+ n00000035 [label="{{} | imx219 1-0010\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000035:port0 -> n00000030:port0
+}
diff --git a/Documentation/admin-guide/media/visconti-viif.rst b/Documentation/admin-guide/media/visconti-viif.rst
new file mode 100644
index 000000000000..c2e85fb6f8c1
--- /dev/null
+++ b/Documentation/admin-guide/media/visconti-viif.rst
@@ -0,0 +1,435 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================================================
+Visconti Video Input Interface Driver (visconti-viif)
+======================================================
+
+Introduction
+============
+
+This file documents the driver for the Video Input Interface (VIIF) that is
+part of Toshiba Visconti SoCs.
+The driver is located under drivers/media/platform/toshiba/visconti and uses
+the Media-Controller API.
+
+The driver module is named visconti-viif,
+and is enabled through the CONFIG_VIDEO_VISCONTI_VIIF config option.
+The CSI-2 receiver part is controlled by another module named visconti-csi2rx,
+which is enabled through the CONFIG_VIDEO_VISCONTI_CSI2RX config option.
+
+The Visconti VIIF Hardware
+==========================
+
+The Visconti VIIF hardware is an internally developed video capture device.
+Following function modules are integrated:
+
+* MIPI CSI-2 receiver (CSI2RX)
+* L1 Image Signal Processor (L1ISP)
+
+ * Correction, enhancement, adjustment on bayer images.
+
+* L2 Image Signal Processor (L2ISP)
+
+ * Lens distortion correction
+ * Scaling & Cropping with up to 2 parameter sets
+ * Formatting picture (RGB, YUV, Grayscale, ...)
+ * Integrated DMAC: Writing picture into main memory
+
+* Video DMAC
+
+ * Writing picture into main memory
+
+Visconti5 SoC has two VIIF hardware instances.
+
+
+The hardware block diagram is shown below.::
+
+ The VIIF hardware
+ "POST0"
+ "RGB with scale 0"
+ +--------+ +----------+ +-----+ +-----+ +-----+ +--------+
+ | Sensor |--->| CSI2RX |--->| | | | | |--->| memory |
+ +--------+ +----------+ | | | | | | +--------+
+ | | | L1 | | L2 | "POST1"
+ | |--->| ISP |--->| ISP | "RGB with scale 1"
+ | | | | | | +--------+
+ | MUX | | | | |--->| memory |
+ | | +-----+ +-----+ +--------+
+ | | "SUB"
+ | | "RAW w/o scale"
+ | | +------------+ +--------+
+ | |------> | Video DMAC |--->| memory |
+ +-----+ +------------+ +--------+
+
+Topology
+========
+
+Graph
+-----
+
+.. _visconti_viif_topology_graph:
+
+.. kernel-figure:: visconti-viif.dot
+ :alt: Diagram of the default media pipeline topology
+ :align: center
+
+The driver has 3 video devices for capturing images:
+
+- viif_capture_post0: capture device for image.
+ - corresponds to L2ISP.
+- viif_capture_post1: capture device for image.
+ - corresponds to L2ISP.
+- viif_capture_sub: capture device for bayer image.
+ - corresponds to Video DMAC.
+
+The driver has 2 video devices for controlling ISP.
+
+- viif_params: a metadata output device that receives ISP parameters.
+ - corresponds to L1ISP and L2ISP.
+- viif_stats: a metadata capture device that sends statistics.
+ - corresponds to L1ISP and L2ISP.
+
+The driver has 2 subdevices:
+
+- visconti_csi2rx: CSI-2 receiver operation.
+ - corresponds to CSI2RX.
+- visconti-viif:isp: Image Signal Processor operation.
+ - corresponds to L1ISP and L2ISP.
+- visconti-viif:resizer: Scaling operation of Image Signal Processor.
+ - corresponds to L2ISP.
+
+visconti_csi2rx - CSI2 Receiver Subdevice Node
+---------------------------------------------------
+
+This subdevice node corresponds to a MIPI CSI2 receiver.
+It resides between an image sensor subdevice and the ISP subdevice.
+It controls CSI2 link configuration and training process.
+
+visconti-viif:isp - ISP Subdevice Node
+--------------------------------------
+
+This subdevice node corresponds to L1/L2 ISPs.
+It receives pictures from an sensor (via CSI2RX),
+applies multiple operations on pictures, then passes resulting images to capture nodes.
+
+ISP configurations/parameters are passed from userland via viif_params node.
+The status of ISP operations are passed to userland via viif_stats node.
+
+L1 ISP provides following operations:
+
+- Input: accepts 8, 10, 12, 14bit bayer format
+ - Operation selector; :c:type:`viif_l1_input_mode_config`
+ - HDR image / PWL (Piecewse Linear Compression) image
+ - with preprocessing / without preprocessing
+ - HDRE: HDR expansion (only for PWL image);
+ see :c:type:`viif_l1_hdre_config`
+- Preprocessing: generate intermediate data (24bit RAW)
+ - SLIC: Bit slicing (x3 12bit planes for preprocessing);
+ see :c:type:`viif_l1_img_extraction_config`
+ - ABPC/DPC: Blemish/Defect pixel correction :c:type:`viif_l1_dpc_config`
+ - PWHB: Preset white balance; see :c:type:`viif_l1_preset_white_balance_config`
+ - RCNR: RAW color noise reduction; see :c:type:`viif_l1_raw_color_noise_reduction_config`
+ - HDRS: HDR synthesis; see :c:type:`viif_l1_hdrs_config`
+- Processing on RAW image: Main Process (MPRO)
+ - BLVC: black level correction and normalization;
+ see :c:type:`viif_l1_black_level_correction_config`
+ - LSSC: Lens shading correction; see :c:type:`viif_l1_lsc_config`
+ - MPRO: digital amplifier; see :c:type:`viif_l1_main_process_config`
+ - MPRO: bayer demosaicing; see :c:type:`viif_l1_main_process_config`
+ - MPRO: color matrix correction; see :c:type:`viif_l1_main_process_config`
+ - HDRC: HDR compression;
+ see :c:type:`viif_l1_hdrc_config`, :c:type:`viif_l1_hdrc_ltm_config`,
+ :c:type:`viif_l1_rgb_to_y_coef_config`
+- Processing on RGB/YUV image: Video Process (VPRO)
+ - VPRO: gamma correction; see :c:type:`viif_l1_gamma_config`
+ - VPRO: RGB2YUV;
+ see :c:type:`viif_l1_rgb_to_y_coef_config`,
+ :c:type:`viif_l1_img_quality_adjustment_config`
+ - VPRO: image quality adjustment; see :c:type:`viif_l1_img_quality_adjustment_config`
+- Output: 16bit YUV
+- Feedback loop
+ - AWHB: auto white balance; see :c:type:`viif_l1_awb_config`,
+ :c:type:`viif_isp_capture_status`
+ - AEXP: auto exposure (average luminance calculation);
+ see :c:type:`viif_l1_avg_lum_generation_config`,
+ :c:type:`viif_l1_rgb_to_y_coef_config`, :c:type:`viif_isp_capture_status`
+ - AG: analog gain calculation;
+ see :c:type:`viif_l1_ag_mode_config`, :c:type:`viif_l1_ag_config`
+
+Below is the block diagram::
+
+ L1ISP::INPUT
+
+ +--------+ +-----+ +-----+
+ | Input |--------------->| |--------------------->| |
+ | 24bHDR | | | | |
+ +--------+ | 24b | | 24b |
+ | RAW | | RAW |
+ +--------+ +------+ | (0) | | (1) |
+ | Input |--->| HDRE |--->| | +------------+ | |
+ | 24bPWL | | | | |--->| preprocess |--->| |
+ +--------+ +------+ +-----+ +------------+ +-----+
+
+ L1ISP::INPUT::preprocess
+
+ +-----+ +-----+
+ | 24b | +------+ +------+ +------+ +------+ +------+ | 24b |
+ | RAW |--->| SLIC |--->| ABPC |--->| PWHB |--->| RCNR |--->| HDRS |--->| RAW |
+ | (0) | +------+ +------+ +------+ +------+ +------+ | (1) |
+ +-----+ +-----+
+
+ L1ISP::MainProcess(MPRO)
+
+ +-----+
+ | 24b | +------+ +------+
+ | RAW |--->| BLVC |--->| LSSC |---+
+ | (1) | +------+ +------+ |
+ +-----+ |
+ |
+ +------------------------------+
+ |
+ | +-----------+ +-------------+ +--------+ +-----+
+ +--->| MPRO | | MPRO | | MPRO | +------+ | 16b |
+ | Digital |--->| Demosaicing |----| Color |--->| HDRC |--->| RGB |
+ +--->| Amplifier | | | | Matrix | +------+ | |
+ | +-----------+ +-------------+ +--------+ +-----+
+ | | |
+ | +--------------+ | | +------+
+ +----| Auto |<----+ +--->| AEXP |---> Auto-Exposure statistics
+ | Whitebalance | +------+
+ +--------------+
+ |
+ +------------------------------> Auto-Whitebalance statistics
+
+ L1ISP::VideoProcess(VPRO)
+
+ +-----+ +------------+ +------------+ +---------------+ +--------+
+ | 16b |--->| Gamma |--->| RGB2YUV |--->| Image Quality |--->| Output |
+ | RGB | | Correction | | Conversion | | Adjustment | | 16b |
+ | | +------------+ +------------+ +---------------+ | YUV |
+ +-----+ +--------+
+
+ L1ISP::AnalogGain
+
+ statistics +-------------+ +------------------+
+ information ---> (user SW) --->| Analog Gain |--->| ABPC, RCNR, LSSC |
+ +-------------+ | MPRO, VPRO |
+ +------------------+
+
+L2 ISP provides following operations:
+
+- Input: accepts 16bit YUV / RGB
+- Operations:
+ - Lens undistortion; see :c:type:`viif_l2_undist`
+ - Scaling; see :c:type:`viif_l2_roi`
+ - Cropping; see :c:type:`viif_l2_roi`
+ - Gamma correction; see :c:type:`viif_l2_gamma_config`
+ - YUV2RGB
+- Output: RGB, YUV422, YUV444
+
+Below is the block diagram::
+
+ L2ISP
+
+ +-------+ +------------+ +--------------+ +---------+
+ | Input |--->| YUV2RGB |--->| Lens |--->| Scaling |---> |
+ | Image | | Conversion | | Undistortion | | |---> |
+ +-------+ +------------+ +--------------+ +---------+ |
+ |
+ +----------------------------------------------------------+
+ |
+ | +----------+ +------------+ +--------+ +--------+
+ +--->|Gamma |--->| Colorspace |--->| Data |--->| Output |
+ | |Correction| | Conversion | | Packer | | Image |
+ | +----------+ +------------+ +--------+ +--------+
+ |
+ | +----------+ +------------+ +--------+ +--------+
+ +--->|Gamma |--->| Colorspace |--->| Data |--->| Output |
+ |Correction| | Conversion | | Packer | | Image |
+ +----------+ +------------+ +--------+ +--------+
+
+visconti-viif:resizer - Resizer Subdevice Node
+----------------------------------------------
+
+The resizer subdevice resides between ISP subdevice and Capture device
+on a capture path for post0 and post1.
+It receives resize and crop parameters for the specific capture path
+and controls L2ISP HW.
+
+following selection rectangles can be passed at VIDIOC_S_SELECTION ioctl.
+
+- sink pads's compose rectangle (V4L2_SEL_TGT_COMPOSE) for scaling
+- source pad's crop rectangle (V4L2_SEL_TGT_CROP) for cropping
+
+
+viif_capture_post0, viif_capture_post1 - Processed Image Capture Video Node
+---------------------------------------------------------------------------
+
+These video nodes are used for capturing images processed at ISPs.
+Supported capture formats are as follows:
+
+- V4L2_PIX_FMT_RGB24
+- V4L2_PIX_FMT_ABGR32
+- V4L2_PIX_FMT_YUV422M
+- V4L2_PIX_FMT_YUV444M
+- V4L2_PIX_FMT_Y16
+
+Bayer format is not supported. Use viif_capture_sub instead.
+
+POST0 and POST1 can output images from the same input image
+using different cropping and scaling settings.
+
+viif_capture_sub - Raw Image Capture Video Node
+-----------------------------------------------
+
+This video node is used for capturing bayer image from the sensor.
+The output picture has exactly the same resolution and format as the sensor input.
+The pipeline does not edit pixel values.
+However, when writing pixel values to memory, they are shifted to the MSB
+to match either 8bit or 16bit.
+
+Therefore, resulting capture formats are as follows:
+
+- for 8bit RAW input:
+ - V4L2_PIX_FMT_SRGGB8
+ - V4L2_PIX_FMT_SGRBG8
+ - V4L2_PIX_FMT_SGBRG8
+ - V4L2_PIX_FMT_SBGGR8
+- for 10, 12, 14bit RAW input:
+ - V4L2_PIX_FMT_SRGGB16
+ - V4L2_PIX_FMT_SGRBG16
+ - V4L2_PIX_FMT_SGBRG16
+ - V4L2_PIX_FMT_SBGGR16
+
+.. _viif_params:
+
+viif_params - ISP Parameters Video Node
+---------------------------------------
+
+The viif_params video node receives a set of ISP parameters from userspace
+to be applied to the hardware during a video stream.
+
+The buffer format is defined by struct :c:type:`visconti_viif_isp_config`, and userspace should set
+:ref:`V4L2_META_FMT_VISCONTI_VIIF_PARAMS <v4l2-meta-fmt-visconti-viif-params>` as the data format.
+
+.. _viif_stats:
+
+viif_stats - Statistics Video Node
+----------------------------------
+
+The viif_stats video node provides current status of ISP.
+
+Following information is included:
+
+* statistics of auto white balance
+* average luminance information which can be used by auto exposure software impl.
+
+The buffer format is defined by struct :c:type:`visconti_viif_isp_stat`, and userspace should set
+:ref:`V4L2_META_FMT_VISCONTI_VIIF_STATS <v4l2-meta-fmt-visconti-viif-stats>` as the data format.
+
+Feedback Operations
+===================
+
+Among the so-called 3A functions, VIIF provides only auto-whitebalance and auto-exposure.
+Auto-whitebalance is a standalone hardware feature.
+Some status information is available through the ISP statistics interface.
+
+Auto-exposure is realized through a combination of hardware and userland software.
+VIIF provides weighted average luminance information through the ISP statistics interface.
+The userland application calculates the sensor gain, sensor exposure and ISP digital gain.
+The calculated parameters are then passed to sensor's controls and the ISP parameter interface.
+
+Among ISP parameters, there are parameters called AG (analog gain).
+Actually, AG parameters have nothing to do with auto-exposure.
+It controls "strength" in several signal correction algorithms.
+Below is the list of the functions which may be affected by AG parameters:
+
+- ABPC/DPC
+- RCNR
+- LSSC
+- MPRO: color matrix correction
+- VPRO
+
+Capturing Video Frames Example
+==============================
+
+In the following example,
+imx219 camera is connected to pad 0 of 'visconti_csi2rx' subdevice.
+
+The following commands yield three pictures with different zoom ratio:
+- main path 0: 1.5x zoom, crop 1920x1080, RGB picture
+- main path 1: 0.67x zoom, crop 640x480, RGB picture
+- sub path: 1920x1080 RAW picture
+
+.. code-block:: bash
+
+ # set the links
+ media-ctl -d platform:visconti-viif-0 -r
+ media-ctl -d platform:visconti-viif-0 -l '"imx219 1-0010":0 -> "visconti_csi2rx 1c008000.csi2rx":0 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"visconti_csi2rx 1c008000.csi2rx":1 -> "visconti-viif:isp":0 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":1 -> "visconti-viif:resizer0":0 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":2 -> "visconti-viif:resizer1":0 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":3 -> "viif_capture_sub":0 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer0":1 -> "viif_capture_post0":0 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer1":1 -> "viif_capture_post1":0 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"viif_params":0 -> "visconti-viif:isp":4 [1]'
+ media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":5 -> "viif_stats":0 [1]'
+
+ # set format for imx219
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"imx219 1-0010":0 [fmt:SRGGB10_1X10/1920x1080]'
+
+ # set format for csi2rx
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti_csi2rx 1c008000.csi2rx":0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range]'
+
+ # set format for isp sink pad
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":0 [fmt:SRGGB10_1X10/1920x1080]'
+
+ # set format for resizer pads
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer0":0 '"[fmt:YUV8_1X24/1920x1080 compose:(0,0)/2880x1620]"
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer0":1 '"[crop:(480,16)/1920x1080]"
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer1":0 '"[fmt:YUV8_1X24/1920x1080 compose:(0,0)/1280x720]"
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer1":1 '"[crop:(320,32)/640x480]"
+
+ media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":1 [fmt:YUV8_1X24/1024 crop:(640,0)/1024x1024]'
+
+ # set format for main path0
+ v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "width=1920,height=1080"
+ v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "pixelformat=RGB3"
+
+ # set format for main path1
+ v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "width=640,height=480"
+ v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "pixelformat=RGB3"
+
+ # start streaming
+ v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 --stream-mmap --stream-count 1000 &
+
+ # start streaming with other devices while viif_capture_post0 is running
+ v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post1 --stream-mmap --stream-count 10
+ v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_sub --stream-mmap --stream-count 10
+
+Use of coherent memory
+======================
+
+Visconti5 SoC has two independent DDR SDRAM controllers.
+Each controller is mapped to 36bit address space.
+
+Accelerator bus masters have two paths to access memory;
+one is directly connected to SDRAM controller,
+the another is connected via a cache coherency bus
+which keeps coherency among CPUs.
+
+From accelerators and CPUs, the address map is following:
+
+* 0x0_8000_0000 DDR0 direct access
+* 0x4_8000_0000 DDR0 coherency bus
+* 0x8_8000_0000 DDR1 direct access
+* 0xC_8000_0000 DDR1 coherency bus
+
+The base address can be specified with "memory" and "reserved-memory" elements
+in a device tree description.
+It's not recommended to mix direct address and coherent address.
+
+The Visconti5 VIIF driver always use only direct address to configure Video DMACs of the hardware.
+This design is to avoid great performance loss at coherency bus caused by massive memory access.
+You should not put the dma_coherent attribute to viif element in device tree.
+Cache operations are done automatically by videobuf2 driver.
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index 86ffb3bc8ade..2336842f0c26 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata` interface only.
metafmt-pisp-fe
metafmt-rkisp1
metafmt-uvc
+ metafmt-visconti-viif
metafmt-vivid
metafmt-vsp1-hgo
metafmt-vsp1-hgt
diff --git a/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
new file mode 100644
index 000000000000..dc4b31627fe1
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
@@ -0,0 +1,48 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _v4l2-meta-fmt-visconti-viif-params:
+
+.. _v4l2-meta-fmt-visconti-viif-stats:
+
+***************************************************************************************
+V4L2_META_FMT_VISCONTI_VIIF_PARAMS ('vifp'), V4L2_META_FMT_VISCONTI_VIIF_STATS ('vifs')
+***************************************************************************************
+
+Configuration parameters
+========================
+
+The configuration parameters are passed to the
+:ref:`viif_params <viif_params>` metadata output video node, using
+the :c:type:`v4l2_meta_format` interface. The buffer contains
+a single instance of the C structure :c:type:`visconti_viif_isp_config` defined in
+``visconti_viif.h``. So the structure can be obtained from the buffer by:
+
+.. code-block:: c
+
+ struct visconti_viif_isp_config *params = (struct visconti_viif_isp_config*) buffer;
+
+VIIF statistics
+===============
+
+The VIIF device collects different statistics over an input Bayer frame.
+Those statistics are obtained from the :ref:`viif_stats <viif_stats>`
+metadata capture video node,
+using the :c:type:`v4l2_meta_format` interface. The buffer contains a single
+instance of the C structure :c:type:`visconti_viif_isp_stat` defined in
+``visconti_viif.h``. So the structure can be obtained from the buffer by:
+
+.. code-block:: c
+
+ struct visconti_viif_isp_stat *stats = (struct visconti_viif_isp_stat*) buffer;
+
+The statistics collected are Exposure, AWB (auto white balance) and errors.
+See :c:type:`visconti_viif_isp_stat` for details of the statistics.
+
+The statistics and configuration parameters described here are usually
+consumed and produced by dedicated user space libraries that comprise the
+tuning tools using software control loop.
+
+visconti viif uAPI data types
+=============================
+
+.. kernel-doc:: include/uapi/linux/visconti_viif.h
--
2.25.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface
2024-11-25 9:21 [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver Yuji Ishikawa
` (4 preceding siblings ...)
2024-11-25 9:21 ` [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver Yuji Ishikawa
@ 2024-11-25 9:21 ` Yuji Ishikawa
2025-01-02 13:16 ` Laurent Pinchart
5 siblings, 1 reply; 34+ messages in thread
From: Yuji Ishikawa @ 2024-11-25 9:21 UTC (permalink / raw)
To: Laurent Pinchart, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Sakari Ailus, Hans Verkuil,
Nobuhiro Iwamatsu, Yuji Ishikawa
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Added entries for visconti Video Input Interface driver, including;
* device tree bindings
* source files
* documentation files
Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
Changelog v2:
- no change
Changelog v3:
- added entry for driver API documentation
Changelog v4:
- added entry for header file
Changelog v5:
- no change
Changelog v6:
- update path to VIIF driver source files
Changelog v7:
- no change
Changelog v8:
- rename bindings description file
Changelog v9:
- no change
Changelog v10:
- add a separate entry of VIIF driver
Changelog v11:
- no change
Changelog v12:
- add a bindings description of CSI2RX driver
MAINTAINERS | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index b878ddc99f94..b5c819e94e9b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -23430,6 +23430,18 @@ F: Documentation/devicetree/bindings/media/i2c/tc358743.txt
F: drivers/media/i2c/tc358743*
F: include/media/i2c/tc358743.h
+TOSHIBA VISCONTI VIIF DRIVER
+M: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
+M: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
+L: linux-media@vger.kernel.org
+S: Maintained
+F: Documentation/admin-guide/media/visconti-viif.*
+F: Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
+F: Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
+F: Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
+F: drivers/media/platform/toshiba/visconti/
+F: include/uapi/linux/visconti_viif.h
+
TOSHIBA WMI HOTKEYS DRIVER
M: Azael Avalos <coproscefalo@gmail.com>
L: platform-driver-x86@vger.kernel.org
--
2.25.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-11-25 9:21 ` [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface Yuji Ishikawa
@ 2024-11-25 10:07 ` Krzysztof Kozlowski
2024-12-17 0:00 ` yuji2.ishikawa
2025-01-02 9:56 ` Laurent Pinchart
1 sibling, 1 reply; 34+ messages in thread
From: Krzysztof Kozlowski @ 2024-11-25 10:07 UTC (permalink / raw)
To: Yuji Ishikawa, Laurent Pinchart, Mauro Carvalho Chehab,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Sakari Ailus,
Hans Verkuil, Nobuhiro Iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
On 25/11/2024 10:21, Yuji Ishikawa wrote:
> Adds the Device Tree binding documentation that allows to describe
> the Video Input Interface found in Toshiba Visconti SoCs.
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
Why this tag stayed and other was removed? What was the reason of tag
removal?
> ---
> Changelog v2:
> - no change
>
> Changelog v3:
> - no change
>
> Changelog v4:
> - fix style problems at the v3 patch
> - remove "index" member
> - update example
>
> Changelog v5:
> - no change
>
> Changelog v6:
> - add register definition of BUS-IF and MPU
>
> Changelog v7:
> - remove trailing "bindings" from commit header message
> - remove trailing "Device Tree Bindings" from title
> - fix text wrapping of description
> - change compatible to visconti5-viif
> - explicitly define allowed properties for port::endpoint
>
> Changelog v8:
> - Suggestion from Krzysztof Kozlowski
> - rename bindings description file
> - use block style array instead of inline style
> - remove clock-lane (as it is fixed at position 0)
> - update sample node's name
> - use lowercase hex for literals
> - Suggestion from Laurent Pinchart
> - update description message port::description
> - remove port::endpoint::bus-type as it is fixed to <4>
> - remove port::endpoint::clock-lanes from example
> - add port::endpoint::data-lanes to required parameters list
> - fix sequence of data-lanes: <1 2 3 4> because current driver does not support data reordering
> - update port::endpoint::data-lanes::description
> - remove redundant type definition for port::endpoint::data-lanes
>
> Changelog v9:
> - place "required" after "properties"
> - dictionary ordering of properties
>
> Changelog v10:
> - no change
>
> Changelog v11:
> - no change
>
> Changelog v12:
> - remove property "clock-noncontinuous" as VIIF switches both modes automatically
> - remove property "link-frequencies" as VIIF does not use the information
Driver does not use or hardware supports only one frequency?
> - remove reg[2] and interrupts[3] which are used for CSI2RX driver
> - update example to refer csi2rx for remote-endpoint
Were these the reasons?
I am really surprised that after 11 versions this binding still is being
totally reshaped and you need us to re-review.
Also, start using b4 tool, so:
1. your cover letter will have proper links to previous versions
2. b4 diff would work. Look, try by yourself:
b4 diff '<20241125092146.1561901-3-yuji2.ishikawa@toshiba.co.jp>'
Grabbing thread from
lore.kernel.org/all/20241125092146.1561901-3-yuji2.ishikawa@toshiba.co.jp/t.mbox.gz
Checking for older revisions
Grabbing search results from lore.kernel.org
Added from v11: 7 patches
---
Analyzing 144 messages in the thread
Looking for additional code-review trailers on lore.kernel.org
Analyzing 13 code-review messages
Preparing fake-am for v11: Add Toshiba Visconti Video Input Interface driver
range: e16e149ced15..4efb0d6768a6
Preparing fake-am for v12: dt-bindings: media: platform: visconti: Add
Toshiba Visconti MIPI CSI-2 Receiver
ERROR: v12 series incomplete; unable to create a fake-am range
---
Could not create fake-am range for upper series v12
How can I verify what happened here without too much effort?
>
> .../media/toshiba,visconti5-viif.yaml | 95 +++++++++++++++++++
> 1 file changed, 95 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> new file mode 100644
> index 000000000000..ef0452a47e98
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> @@ -0,0 +1,95 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/toshiba,visconti5-viif.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Toshiba Visconti5 SoC Video Input Interface
> +
> +maintainers:
> + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> +
> +description: |-
Since you ask for re-review, then:
Drop |-
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2024-11-25 9:21 ` [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver Yuji Ishikawa
@ 2024-11-25 10:11 ` Krzysztof Kozlowski
2024-12-16 23:57 ` yuji2.ishikawa
2025-01-02 9:29 ` Laurent Pinchart
1 sibling, 1 reply; 34+ messages in thread
From: Krzysztof Kozlowski @ 2024-11-25 10:11 UTC (permalink / raw)
To: Yuji Ishikawa, Laurent Pinchart, Mauro Carvalho Chehab,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Sakari Ailus,
Hans Verkuil, Nobuhiro Iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
On 25/11/2024 10:21, Yuji Ishikawa wrote:
> Adds the Device Tree binding documentation that allows to describe
> the MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
How newly added patch can have already Rb tag? Was this review really,
really performed internally or you just satisfy some internal managers
requirements and fake the stats?
> ---
>
> Changelog v12:
> - Newly add bindings for CSI2RX driver
>
> .../media/toshiba,visconti5-csi2rx.yaml | 104 ++++++++++++++++++
> 1 file changed, 104 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> new file mode 100644
> index 000000000000..5488072bc82a
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> @@ -0,0 +1,104 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
> +
> +maintainers:
> + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> +
> +description: |-
Drop |-
> + Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI CSI-2 video
> + stream. Use with VIIF device. T.B.D
> +
> +properties:
> + compatible:
> + const: toshiba,visconti5-csi2rx
Why this is called "RX"? Can you have a TX? I had impression that one
cannot.
> +
> + reg:
> + items:
> + - description: Registers for CSI2 receiver control
> +
> + interrupts:
> + items:
> + - description: CSI2 Receiver Interrupt
> +
> + ports:
> + $ref: /schemas/graph.yaml#/properties/ports
> +
> + properties:
> + port@0:
> + $ref: /schemas/graph.yaml#/$defs/port-base
> + unevaluatedProperties: false
> + description:
> + Input port node, single endpoint describing the CSI-2 transmitter.
> +
> + properties:
> + endpoint:
> + $ref: video-interfaces.yaml#
> + unevaluatedProperties: false
> +
> + properties:
> + data-lanes:
> + description: CSI2 receiver supports 1, 2, 3 or 4 data lanes
> + minItems: 1
> + items:
> + - const: 1
> + - const: 2
> + - const: 3
> + - const: 4
> + required:
> + - data-lanes
> +
> + port@1:
> + $ref: /schemas/graph.yaml#/properties/port
> + description:
> + Output port node, single endpoint describing the Visconti VIIF.
> +
> + required:
> + - port@0
> + - port@1
> +
> +required:
> + - compatible
> + - reg
> + - interrupts
> + - ports
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> + #include <dt-bindings/interrupt-controller/irq.h>
> +
> + soc {
> + #address-cells = <2>;
> + #size-cells = <2>;
> +
> + csi2rx@1c008000 {
csi@
Node names should be generic. See also an explanation and list of
examples (not exhaustive) in DT specification:
https://devicetree-specification.readthedocs.io/en/latest/chapter2-devicetree-basics.html#generic-names-recommendation
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2024-11-25 10:11 ` Krzysztof Kozlowski
@ 2024-12-16 23:57 ` yuji2.ishikawa
2024-12-17 5:44 ` Krzysztof Kozlowski
0 siblings, 1 reply; 34+ messages in thread
From: yuji2.ishikawa @ 2024-12-16 23:57 UTC (permalink / raw)
To: krzk, laurent.pinchart, mchehab, robh, krzk+dt, conor+dt,
sakari.ailus, hverkuil-cisco, nobuhiro1.iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Hello Krzysztof
Thank you for your review
> -----Original Message-----
> From: Krzysztof Kozlowski <krzk@kernel.org>
> Sent: Monday, November 25, 2024 7:11 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>; Laurent Pinchart
> <laurent.pinchart@ideasonboard.com>; Mauro Carvalho Chehab
> <mchehab@kernel.org>; Rob Herring <robh@kernel.org>; Krzysztof Kozlowski
> <krzk+dt@kernel.org>; Conor Dooley <conor+dt@kernel.org>; Sakari Ailus
> <sakari.ailus@linux.intel.com>; Hans Verkuil <hverkuil-cisco@xs4all.nl>;
> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
> <nobuhiro1.iwamatsu@toshiba.co.jp>
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add
> Toshiba Visconti MIPI CSI-2 Receiver
>
> On 25/11/2024 10:21, Yuji Ishikawa wrote:
> > Adds the Device Tree binding documentation that allows to describe the
> > MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>
>
> How newly added patch can have already Rb tag? Was this review really, really
> performed internally or you just satisfy some internal managers requirements
> and fake the stats?
>
I added this Reviewed-by tag because the patch was reviewed internally.
> > ---
> >
> > Changelog v12:
> > - Newly add bindings for CSI2RX driver
> >
> > .../media/toshiba,visconti5-csi2rx.yaml | 104
> ++++++++++++++++++
> > 1 file changed, 104 insertions(+)
> > create mode 100644
> > Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> >
> > diff --git
> > a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
> > l
> > b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
> > l
> > new file mode 100644
> > index 000000000000..5488072bc82a
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx
> > +++ .yaml
> > @@ -0,0 +1,104 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) %YAML 1.2
> > +---
> > +$id:
> > +http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
> > +
> > +maintainers:
> > + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > +
> > +description: |-
>
> Drop |-
>
I'll drop "|-"
> > + Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI
> > + CSI-2 video stream. Use with VIIF device. T.B.D
> > +
> > +properties:
> > + compatible:
> > + const: toshiba,visconti5-csi2rx
>
> Why this is called "RX"? Can you have a TX? I had impression that one cannot.
>
VIIF has only MIPI CSI2 receiver (CSI2RX). There's no TX for it.
Visconti also has VOIF (Video Output Interface) hardware which has MIPI CSI2 (not DSI) transmitter (CSI2TX).
> > +
> > + reg:
> > + items:
> > + - description: Registers for CSI2 receiver control
> > +
> > + interrupts:
> > + items:
> > + - description: CSI2 Receiver Interrupt
> > +
> > + ports:
> > + $ref: /schemas/graph.yaml#/properties/ports
> > +
> > + properties:
> > + port@0:
> > + $ref: /schemas/graph.yaml#/$defs/port-base
> > + unevaluatedProperties: false
> > + description:
> > + Input port node, single endpoint describing the CSI-2
> transmitter.
> > +
> > + properties:
> > + endpoint:
> > + $ref: video-interfaces.yaml#
> > + unevaluatedProperties: false
> > +
> > + properties:
> > + data-lanes:
> > + description: CSI2 receiver supports 1, 2, 3 or 4 data lanes
> > + minItems: 1
> > + items:
> > + - const: 1
> > + - const: 2
> > + - const: 3
> > + - const: 4
> > + required:
> > + - data-lanes
> > +
> > + port@1:
> > + $ref: /schemas/graph.yaml#/properties/port
> > + description:
> > + Output port node, single endpoint describing the Visconti VIIF.
> > +
> > + required:
> > + - port@0
> > + - port@1
> > +
> > +required:
> > + - compatible
> > + - reg
> > + - interrupts
> > + - ports
> > +
> > +additionalProperties: false
> > +
> > +examples:
> > + - |
> > + #include <dt-bindings/interrupt-controller/arm-gic.h>
> > + #include <dt-bindings/interrupt-controller/irq.h>
> > +
> > + soc {
> > + #address-cells = <2>;
> > + #size-cells = <2>;
> > +
> > + csi2rx@1c008000 {
>
>
> csi@
>
> Node names should be generic. See also an explanation and list of examples
> (not exhaustive) in DT specification:
> https://devicetree-specification.readthedocs.io/en/latest/chapter2-devicetre
> e-basics.html#generic-names-recommendation
>
I'll fix the node names.
>
> Best regards,
> Krzysztof
Best regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-11-25 10:07 ` Krzysztof Kozlowski
@ 2024-12-17 0:00 ` yuji2.ishikawa
2024-12-17 5:43 ` Krzysztof Kozlowski
0 siblings, 1 reply; 34+ messages in thread
From: yuji2.ishikawa @ 2024-12-17 0:00 UTC (permalink / raw)
To: krzk, laurent.pinchart, mchehab, robh, krzk+dt, conor+dt,
sakari.ailus, hverkuil-cisco, nobuhiro1.iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
Hello Krzysztof
Thank you for your review
> -----Original Message-----
> From: Krzysztof Kozlowski <krzk@kernel.org>
> Sent: Monday, November 25, 2024 7:08 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>; Laurent Pinchart
> <laurent.pinchart@ideasonboard.com>; Mauro Carvalho Chehab
> <mchehab@kernel.org>; Rob Herring <robh@kernel.org>; Krzysztof Kozlowski
> <krzk+dt@kernel.org>; Conor Dooley <conor+dt@kernel.org>; Sakari Ailus
> <sakari.ailus@linux.intel.com>; Hans Verkuil <hverkuil-cisco@xs4all.nl>;
> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
> <nobuhiro1.iwamatsu@toshiba.co.jp>
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add
> Toshiba Visconti Video Input Interface
>
> On 25/11/2024 10:21, Yuji Ishikawa wrote:
> > Adds the Device Tree binding documentation that allows to describe the
> > Video Input Interface found in Toshiba Visconti SoCs.
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>
> Why this tag stayed and other was removed? What was the reason of tag
> removal?
>
The stayed tag is due to internal review.
The removed tag is due to code's change (split of csi2rx part) after the last review.
If the code is largely changed following the instruction of another reviewer
after obtaining the tags, how should the tags be handled?
> > ---
> > Changelog v2:
> > - no change
> >
> > Changelog v3:
> > - no change
> >
> > Changelog v4:
> > - fix style problems at the v3 patch
> > - remove "index" member
> > - update example
> >
> > Changelog v5:
> > - no change
> >
> > Changelog v6:
> > - add register definition of BUS-IF and MPU
> >
> > Changelog v7:
> > - remove trailing "bindings" from commit header message
> > - remove trailing "Device Tree Bindings" from title
> > - fix text wrapping of description
> > - change compatible to visconti5-viif
> > - explicitly define allowed properties for port::endpoint
> >
> > Changelog v8:
> > - Suggestion from Krzysztof Kozlowski
> > - rename bindings description file
> > - use block style array instead of inline style
> > - remove clock-lane (as it is fixed at position 0)
> > - update sample node's name
> > - use lowercase hex for literals
> > - Suggestion from Laurent Pinchart
> > - update description message port::description
> > - remove port::endpoint::bus-type as it is fixed to <4>
> > - remove port::endpoint::clock-lanes from example
> > - add port::endpoint::data-lanes to required parameters list
> > - fix sequence of data-lanes: <1 2 3 4> because current driver does not
> support data reordering
> > - update port::endpoint::data-lanes::description
> > - remove redundant type definition for port::endpoint::data-lanes
> >
> > Changelog v9:
> > - place "required" after "properties"
> > - dictionary ordering of properties
> >
> > Changelog v10:
> > - no change
> >
> > Changelog v11:
> > - no change
> >
> > Changelog v12:
> > - remove property "clock-noncontinuous" as VIIF switches both modes
> > automatically
> > - remove property "link-frequencies" as VIIF does not use the
> > information
>
> Driver does not use or hardware supports only one frequency?
>
My comment was incorrect.
It should be "Driver does not use the information"
> > - remove reg[2] and interrupts[3] which are used for CSI2RX driver
> > - update example to refer csi2rx for remote-endpoint
>
> Were these the reasons?
>
> I am really surprised that after 11 versions this binding still is being totally
> reshaped and you need us to re-review.
>
Sorry for poor quality.
Several changes were brought about by the introduction of the CSI2RX driver.
The new driver was requested in the previous review.
The property "clock-noncontinuous" was pointed out as unnecessary (because HW supports both modes) in the previous review.
The property "link-frequencies" was dropped because It was found to be unused after re-check of the code.
> Also, start using b4 tool, so:
> 1. your cover letter will have proper links to previous versions 2. b4 diff would
> work. Look, try by yourself:
>
>
>
> b4 diff '<20241125092146.1561901-3-yuji2.ishikawa@toshiba.co.jp>'
> Grabbing thread from
> lore.kernel.org/all/20241125092146.1561901-3-yuji2.ishikawa@toshiba.co.jp/t
> .mbox.gz
> Checking for older revisions
> Grabbing search results from lore.kernel.org
> Added from v11: 7 patches
> ---
> Analyzing 144 messages in the thread
> Looking for additional code-review trailers on lore.kernel.org Analyzing 13
> code-review messages Preparing fake-am for v11: Add Toshiba Visconti Video
> Input Interface driver
> range: e16e149ced15..4efb0d6768a6
> Preparing fake-am for v12: dt-bindings: media: platform: visconti: Add Toshiba
> Visconti MIPI CSI-2 Receiver
> ERROR: v12 series incomplete; unable to create a fake-am range
> ---
> Could not create fake-am range for upper series v12
>
>
>
> How can I verify what happened here without too much effort?
>
>
I will use b4 tool for further submits.
Also I'll add links to previous versions in the cover letter.
I ran b4 diff with the HEAD of media_stage tree and confirmed that it produced non-error results.
I needed to fetch the tree to a certain depth to ensure everything worked smoothly.
$ git clone https://git.linuxtv.org/media_stage.git --depth 1
$ git fetch --shallow-exclude v6.4
$ git log --oneline | head -5
40384c840ea1 Linux 6.13-rc1
a14bf463e7df Merge tag 'i2c-for-6.13-rc1-part3' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
88862eeb4763 Merge tag 'trace-printf-v6.13' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
f788b5ef1ca9 Merge tag 'timers_urgent_for_v6.13_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
63f4993b792e Merge tag 'irq_urgent_for_v6.13_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
$ b4 diff '<20241125092146.1561901-3-yuji2.ishikawa@toshiba.co.jp>'
> >
> > .../media/toshiba,visconti5-viif.yaml | 95
> +++++++++++++++++++
> > 1 file changed, 95 insertions(+)
> > create mode 100644
> > Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> >
> > diff --git
> > a/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> > b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> > new file mode 100644
> > index 000000000000..ef0452a47e98
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.y
> > +++ aml
> > @@ -0,0 +1,95 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) %YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/media/toshiba,visconti5-viif.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Toshiba Visconti5 SoC Video Input Interface
> > +
> > +maintainers:
> > + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > +
> > +description: |-
>
> Since you ask for re-review, then:
>
> Drop |-
>
I'll drop "|-"
> Best regards,
> Krzysztof
Best regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-12-17 0:00 ` yuji2.ishikawa
@ 2024-12-17 5:43 ` Krzysztof Kozlowski
2024-12-17 9:45 ` Laurent Pinchart
2024-12-24 0:17 ` yuji2.ishikawa
0 siblings, 2 replies; 34+ messages in thread
From: Krzysztof Kozlowski @ 2024-12-17 5:43 UTC (permalink / raw)
To: yuji2.ishikawa, laurent.pinchart, mchehab, robh, krzk+dt,
conor+dt, sakari.ailus, hverkuil-cisco, nobuhiro1.iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
On 17/12/2024 01:00, yuji2.ishikawa@toshiba.co.jp wrote:
> Hello Krzysztof
>
> Thank you for your review
>
>> -----Original Message-----
>> From: Krzysztof Kozlowski <krzk@kernel.org>
>> Sent: Monday, November 25, 2024 7:08 PM
>> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
>> <yuji2.ishikawa@toshiba.co.jp>; Laurent Pinchart
>> <laurent.pinchart@ideasonboard.com>; Mauro Carvalho Chehab
>> <mchehab@kernel.org>; Rob Herring <robh@kernel.org>; Krzysztof Kozlowski
>> <krzk+dt@kernel.org>; Conor Dooley <conor+dt@kernel.org>; Sakari Ailus
>> <sakari.ailus@linux.intel.com>; Hans Verkuil <hverkuil-cisco@xs4all.nl>;
>> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
>> <nobuhiro1.iwamatsu@toshiba.co.jp>
>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
>> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add
>> Toshiba Visconti Video Input Interface
>>
>> On 25/11/2024 10:21, Yuji Ishikawa wrote:
>>> Adds the Device Tree binding documentation that allows to describe the
>>> Video Input Interface found in Toshiba Visconti SoCs.
>>>
>>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
>>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>>
>> Why this tag stayed and other was removed? What was the reason of tag
>> removal?
>>
>
> The stayed tag is due to internal review.
Did the internal review really happened? How is it that immediately new
version has internal review without any traces?
I have doubts this review happened in the context of reviewer's
statement of oversight.
> The removed tag is due to code's change (split of csi2rx part) after the last review.
> If the code is largely changed following the instruction of another reviewer
> after obtaining the tags, how should the tags be handled?
Drop all reviews and perform reviews on the list.
Such internal review appearing afterwards is rather a proof it you are
adding just the tags to satisfy your process. I have no way to even
verify whether that person performed any reasonable review or maybe just
acked your patch. I cannot even verify that that person understands the
reviewer's statement of oversight.
...
>>>
>>> Changelog v11:
>>> - no change
>>>
>>> Changelog v12:
>>> - remove property "clock-noncontinuous" as VIIF switches both modes
>>> automatically
>>> - remove property "link-frequencies" as VIIF does not use the
>>> information
>>
>> Driver does not use or hardware supports only one frequency?
>>
>
> My comment was incorrect.
> It should be "Driver does not use the information"
Then this is not that helping. Maybe hardware supports only one frequency?
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2024-12-16 23:57 ` yuji2.ishikawa
@ 2024-12-17 5:44 ` Krzysztof Kozlowski
2024-12-24 0:17 ` yuji2.ishikawa
2024-12-26 16:15 ` Laurent Pinchart
0 siblings, 2 replies; 34+ messages in thread
From: Krzysztof Kozlowski @ 2024-12-17 5:44 UTC (permalink / raw)
To: yuji2.ishikawa, laurent.pinchart, mchehab, robh, krzk+dt,
conor+dt, sakari.ailus, hverkuil-cisco, nobuhiro1.iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
On 17/12/2024 00:57, yuji2.ishikawa@toshiba.co.jp wrote:
>> On 25/11/2024 10:21, Yuji Ishikawa wrote:
>>> Adds the Device Tree binding documentation that allows to describe the
>>> MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
>>>
>>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
>>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>>
>>
>> How newly added patch can have already Rb tag? Was this review really, really
>> performed internally or you just satisfy some internal managers requirements
>> and fake the stats?
>>
>
> I added this Reviewed-by tag because the patch was reviewed internally.
What issues were identified by internal review, especially in the
context of bindings?
>
>>> ---
>>>
>>> Changelog v12:
>>> - Newly add bindings for CSI2RX driver
>>>
>>> .../media/toshiba,visconti5-csi2rx.yaml | 104
>> ++++++++++++++++++
>>> 1 file changed, 104 insertions(+)
>>> create mode 100644
>>> Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
>>>
>>> diff --git
>>> a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
>>> l
>>> b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
>>> l
>>> new file mode 100644
>>> index 000000000000..5488072bc82a
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx
>>> +++ .yaml
>>> @@ -0,0 +1,104 @@
>>> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) %YAML 1.2
>>> +---
>>> +$id:
>>> +http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
>>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>>> +
>>> +title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
>>> +
>>> +maintainers:
>>> + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>>> +
>>> +description: |-
>>
>> Drop |-
>>
>
> I'll drop "|-"
>
>>> + Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI
>>> + CSI-2 video stream. Use with VIIF device. T.B.D
>>> +
>>> +properties:
>>> + compatible:
>>> + const: toshiba,visconti5-csi2rx
>>
>> Why this is called "RX"? Can you have a TX? I had impression that one cannot.
>>
>
> VIIF has only MIPI CSI2 receiver (CSI2RX). There's no TX for it.
So this device cannot be anything else? Then drop rx.
> Visconti also has VOIF (Video Output Interface) hardware which has MIPI CSI2 (not DSI) transmitter (CSI2TX).
Or this can be something else? Confusing.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-12-17 5:43 ` Krzysztof Kozlowski
@ 2024-12-17 9:45 ` Laurent Pinchart
2024-12-17 12:51 ` Krzysztof Kozlowski
2024-12-24 0:17 ` yuji2.ishikawa
1 sibling, 1 reply; 34+ messages in thread
From: Laurent Pinchart @ 2024-12-17 9:45 UTC (permalink / raw)
To: Krzysztof Kozlowski
Cc: yuji2.ishikawa, mchehab, robh, krzk+dt, conor+dt, sakari.ailus,
hverkuil-cisco, nobuhiro1.iwamatsu, linux-media, linux-kernel,
linux-arm-kernel, devicetree
On Tue, Dec 17, 2024 at 06:43:22AM +0100, Krzysztof Kozlowski wrote:
> On 17/12/2024 01:00, yuji2.ishikawa@toshiba.co.jp wrote:
> > Hello Krzysztof
> >
> > Thank you for your review
> >
> >> -----Original Message-----
> >> From: Krzysztof Kozlowski <krzk@kernel.org>
> >> Sent: Monday, November 25, 2024 7:08 PM
> >> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> >> <yuji2.ishikawa@toshiba.co.jp>; Laurent Pinchart
> >> <laurent.pinchart@ideasonboard.com>; Mauro Carvalho Chehab
> >> <mchehab@kernel.org>; Rob Herring <robh@kernel.org>; Krzysztof Kozlowski
> >> <krzk+dt@kernel.org>; Conor Dooley <conor+dt@kernel.org>; Sakari Ailus
> >> <sakari.ailus@linux.intel.com>; Hans Verkuil <hverkuil-cisco@xs4all.nl>;
> >> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
> >> <nobuhiro1.iwamatsu@toshiba.co.jp>
> >> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> >> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> >> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add
> >> Toshiba Visconti Video Input Interface
> >>
> >> On 25/11/2024 10:21, Yuji Ishikawa wrote:
> >>> Adds the Device Tree binding documentation that allows to describe the
> >>> Video Input Interface found in Toshiba Visconti SoCs.
> >>>
> >>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> >>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>
> >> Why this tag stayed and other was removed? What was the reason of tag
> >> removal?
> >>
> >
> > The stayed tag is due to internal review.
>
> Did the internal review really happened? How is it that immediately new
> version has internal review without any traces?
>
> I have doubts this review happened in the context of reviewer's
> statement of oversight.
>
> > The removed tag is due to code's change (split of csi2rx part) after the last review.
> > If the code is largely changed following the instruction of another reviewer
> > after obtaining the tags, how should the tags be handled?
>
> Drop all reviews and perform reviews on the list.
>
> Such internal review appearing afterwards is rather a proof it you are
> adding just the tags to satisfy your process. I have no way to even
> verify whether that person performed any reasonable review or maybe just
> acked your patch.
How do you verify that for public reviews ?
> I cannot even verify that that person understands the
> reviewer's statement of oversight.
>
>
> ...
>
> >>>
> >>> Changelog v11:
> >>> - no change
> >>>
> >>> Changelog v12:
> >>> - remove property "clock-noncontinuous" as VIIF switches both modes
> >>> automatically
> >>> - remove property "link-frequencies" as VIIF does not use the
> >>> information
> >>
> >> Driver does not use or hardware supports only one frequency?
> >
> > My comment was incorrect.
> > It should be "Driver does not use the information"
>
> Then this is not that helping. Maybe hardware supports only one frequency?
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-12-17 9:45 ` Laurent Pinchart
@ 2024-12-17 12:51 ` Krzysztof Kozlowski
2025-01-02 9:41 ` Laurent Pinchart
0 siblings, 1 reply; 34+ messages in thread
From: Krzysztof Kozlowski @ 2024-12-17 12:51 UTC (permalink / raw)
To: Laurent Pinchart
Cc: yuji2.ishikawa, mchehab, robh, krzk+dt, conor+dt, sakari.ailus,
hverkuil-cisco, nobuhiro1.iwamatsu, linux-media, linux-kernel,
linux-arm-kernel, devicetree
On 17/12/2024 10:45, Laurent Pinchart wrote:
> On Tue, Dec 17, 2024 at 06:43:22AM +0100, Krzysztof Kozlowski wrote:
>> On 17/12/2024 01:00, yuji2.ishikawa@toshiba.co.jp wrote:
>>> Hello Krzysztof
>>>
>>> Thank you for your review
>>>
>>>> -----Original Message-----
>>>> From: Krzysztof Kozlowski <krzk@kernel.org>
>>>> Sent: Monday, November 25, 2024 7:08 PM
>>>> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
>>>> <yuji2.ishikawa@toshiba.co.jp>; Laurent Pinchart
>>>> <laurent.pinchart@ideasonboard.com>; Mauro Carvalho Chehab
>>>> <mchehab@kernel.org>; Rob Herring <robh@kernel.org>; Krzysztof Kozlowski
>>>> <krzk+dt@kernel.org>; Conor Dooley <conor+dt@kernel.org>; Sakari Ailus
>>>> <sakari.ailus@linux.intel.com>; Hans Verkuil <hverkuil-cisco@xs4all.nl>;
>>>> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
>>>> <nobuhiro1.iwamatsu@toshiba.co.jp>
>>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
>>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
>>>> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add
>>>> Toshiba Visconti Video Input Interface
>>>>
>>>> On 25/11/2024 10:21, Yuji Ishikawa wrote:
>>>>> Adds the Device Tree binding documentation that allows to describe the
>>>>> Video Input Interface found in Toshiba Visconti SoCs.
>>>>>
>>>>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
>>>>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>>>>
>>>> Why this tag stayed and other was removed? What was the reason of tag
>>>> removal?
>>>>
>>>
>>> The stayed tag is due to internal review.
>>
>> Did the internal review really happened? How is it that immediately new
>> version has internal review without any traces?
>>
>> I have doubts this review happened in the context of reviewer's
>> statement of oversight.
>>
>>> The removed tag is due to code's change (split of csi2rx part) after the last review.
>>> If the code is largely changed following the instruction of another reviewer
>>> after obtaining the tags, how should the tags be handled?
>>
>> Drop all reviews and perform reviews on the list.
>>
>> Such internal review appearing afterwards is rather a proof it you are
>> adding just the tags to satisfy your process. I have no way to even
>> verify whether that person performed any reasonable review or maybe just
>> acked your patch.
>
> How do you verify that for public reviews ?
By quality or amount of comments. Or timing. Or reviewing cover letter
without any feedback on individual patches.
There are many, many ways. Considering how many companies were adding
fake manager-review-tags in the past (or fake SoBs), I am pretty picky
on that.
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2024-12-17 5:44 ` Krzysztof Kozlowski
@ 2024-12-24 0:17 ` yuji2.ishikawa
2024-12-26 16:15 ` Laurent Pinchart
1 sibling, 0 replies; 34+ messages in thread
From: yuji2.ishikawa @ 2024-12-24 0:17 UTC (permalink / raw)
To: krzk, laurent.pinchart, mchehab, robh, krzk+dt, conor+dt,
sakari.ailus, hverkuil-cisco, nobuhiro1.iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
> -----Original Message-----
> From: Krzysztof Kozlowski <krzk@kernel.org>
> Sent: Tuesday, December 17, 2024 2:45 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>; laurent.pinchart@ideasonboard.com;
> mchehab@kernel.org; robh@kernel.org; krzk+dt@kernel.org;
> conor+dt@kernel.org; sakari.ailus@linux.intel.com; hverkuil-cisco@xs4all.nl;
> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
> <nobuhiro1.iwamatsu@toshiba.co.jp>
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add
> Toshiba Visconti MIPI CSI-2 Receiver
>
> On 17/12/2024 00:57, yuji2.ishikawa@toshiba.co.jp wrote:
> >> On 25/11/2024 10:21, Yuji Ishikawa wrote:
> >>> Adds the Device Tree binding documentation that allows to describe
> >>> the MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
> >>>
> >>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> >>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>
> >>
> >> How newly added patch can have already Rb tag? Was this review
> >> really, really performed internally or you just satisfy some internal
> >> managers requirements and fake the stats?
> >>
> >
> > I added this Reviewed-by tag because the patch was reviewed internally.
>
>
> What issues were identified by internal review, especially in the context of
> bindings?
>
The review was insufficient. We discussed the splitting of drivers, but overlooked the schema.
I should have been more careful not to add an inappropriate tag.
I apologize for not understanding the attention and respect that should be given to the Reviewed-by tag.
From now on, I will add the tag to reviews discussed in the open mailing list.
> >
> >>> ---
> >>>
> >>> Changelog v12:
> >>> - Newly add bindings for CSI2RX driver
> >>>
> >>> .../media/toshiba,visconti5-csi2rx.yaml | 104
> >> ++++++++++++++++++
> >>> 1 file changed, 104 insertions(+)
> >>> create mode 100644
> >>> Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
> >>> l
> >>>
> >>> diff --git
> >>> a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.y
> >>> am
> >>> l
> >>> b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.y
> >>> am
> >>> l
> >>> new file mode 100644
> >>> index 000000000000..5488072bc82a
> >>> --- /dev/null
> >>> +++
> b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2
> >>> +++ rx
> >>> +++ .yaml
> >>> @@ -0,0 +1,104 @@
> >>> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) %YAML
> 1.2
> >>> +---
> >>> +$id:
> >>> +http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
> >>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> >>> +
> >>> +title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
> >>> +
> >>> +maintainers:
> >>> + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>> +
> >>> +description: |-
> >>
> >> Drop |-
> >>
> >
> > I'll drop "|-"
> >
> >>> + Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI
> >>> + CSI-2 video stream. Use with VIIF device. T.B.D
> >>> +
> >>> +properties:
> >>> + compatible:
> >>> + const: toshiba,visconti5-csi2rx
> >>
> >> Why this is called "RX"? Can you have a TX? I had impression that one
> cannot.
> >>
> >
> > VIIF has only MIPI CSI2 receiver (CSI2RX). There's no TX for it.
>
> So this device cannot be anything else? Then drop rx.
>
I'll drop "rx". The new compatible string will be "toshiba,visconti5-csi2".
I'll also update the name of bindings document file.
> > Visconti also has VOIF (Video Output Interface) hardware which has MIPI
> CSI2 (not DSI) transmitter (CSI2TX).
>
> Or this can be something else? Confusing.
>
VOIF has only TX. There's no RX for it.
I just wanted to inform you that there is a node with similar name and function.
Please forget about it.
>
>
> Best regards,
> Krzysztof
Best regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-12-17 5:43 ` Krzysztof Kozlowski
2024-12-17 9:45 ` Laurent Pinchart
@ 2024-12-24 0:17 ` yuji2.ishikawa
1 sibling, 0 replies; 34+ messages in thread
From: yuji2.ishikawa @ 2024-12-24 0:17 UTC (permalink / raw)
To: krzk, laurent.pinchart, mchehab, robh, krzk+dt, conor+dt,
sakari.ailus, hverkuil-cisco, nobuhiro1.iwamatsu
Cc: linux-media, linux-kernel, linux-arm-kernel, devicetree
> -----Original Message-----
> From: Krzysztof Kozlowski <krzk@kernel.org>
> Sent: Tuesday, December 17, 2024 2:43 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>; laurent.pinchart@ideasonboard.com;
> mchehab@kernel.org; robh@kernel.org; krzk+dt@kernel.org;
> conor+dt@kernel.org; sakari.ailus@linux.intel.com; hverkuil-cisco@xs4all.nl;
> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
> <nobuhiro1.iwamatsu@toshiba.co.jp>
> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add
> Toshiba Visconti Video Input Interface
>
> On 17/12/2024 01:00, yuji2.ishikawa@toshiba.co.jp wrote:
> > Hello Krzysztof
> >
> > Thank you for your review
> >
> >> -----Original Message-----
> >> From: Krzysztof Kozlowski <krzk@kernel.org>
> >> Sent: Monday, November 25, 2024 7:08 PM
> >> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> >> <yuji2.ishikawa@toshiba.co.jp>; Laurent Pinchart
> >> <laurent.pinchart@ideasonboard.com>; Mauro Carvalho Chehab
> >> <mchehab@kernel.org>; Rob Herring <robh@kernel.org>; Krzysztof
> >> Kozlowski <krzk+dt@kernel.org>; Conor Dooley <conor+dt@kernel.org>;
> >> Sakari Ailus <sakari.ailus@linux.intel.com>; Hans Verkuil
> >> <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩ
> Ｔ○ＯＳＴ)
> >> <nobuhiro1.iwamatsu@toshiba.co.jp>
> >> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> >> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> >> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti:
> >> Add Toshiba Visconti Video Input Interface
> >>
> >> On 25/11/2024 10:21, Yuji Ishikawa wrote:
> >>> Adds the Device Tree binding documentation that allows to describe
> >>> the Video Input Interface found in Toshiba Visconti SoCs.
> >>>
> >>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> >>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>
> >> Why this tag stayed and other was removed? What was the reason of tag
> >> removal?
> >>
> >
> > The stayed tag is due to internal review.
>
> Did the internal review really happened? How is it that immediately new version
> has internal review without any traces?
>
> I have doubts this review happened in the context of reviewer's statement of
> oversight.
>
>
> > The removed tag is due to code's change (split of csi2rx part) after the last
> review.
> > If the code is largely changed following the instruction of another
> > reviewer after obtaining the tags, how should the tags be handled?
>
> Drop all reviews and perform reviews on the list.
>
> Such internal review appearing afterwards is rather a proof it you are adding
> just the tags to satisfy your process. I have no way to even verify whether that
> person performed any reasonable review or maybe just acked your patch. I
> cannot even verify that that person understands the reviewer's statement of
> oversight.
>
I understand the importance and usage of the Reviewed-by tag.
We will continue to conduct internal reviews, but from now on, I will add the tag to reviews discussed in the open mailing list.
>
> ...
>
> >>>
> >>> Changelog v11:
> >>> - no change
> >>>
> >>> Changelog v12:
> >>> - remove property "clock-noncontinuous" as VIIF switches both modes
> >>> automatically
> >>> - remove property "link-frequencies" as VIIF does not use the
> >>> information
> >>
> >> Driver does not use or hardware supports only one frequency?
> >>
> >
> > My comment was incorrect.
> > It should be "Driver does not use the information"
>
> Then this is not that helping. Maybe hardware supports only one frequency?
>
The reason for the removal is the hardware PLL is configured using information from the sensor's V4L2_CID_PIXEL_RATE control.
>
> Best regards,
> Krzysztof
Best regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2024-12-17 5:44 ` Krzysztof Kozlowski
2024-12-24 0:17 ` yuji2.ishikawa
@ 2024-12-26 16:15 ` Laurent Pinchart
1 sibling, 0 replies; 34+ messages in thread
From: Laurent Pinchart @ 2024-12-26 16:15 UTC (permalink / raw)
To: Krzysztof Kozlowski
Cc: yuji2.ishikawa, mchehab, robh, krzk+dt, conor+dt, sakari.ailus,
hverkuil-cisco, nobuhiro1.iwamatsu, linux-media, linux-kernel,
linux-arm-kernel, devicetree
On Tue, Dec 17, 2024 at 06:44:58AM +0100, Krzysztof Kozlowski wrote:
> On 17/12/2024 00:57, yuji2.ishikawa@toshiba.co.jp wrote:
> >> On 25/11/2024 10:21, Yuji Ishikawa wrote:
> >>> Adds the Device Tree binding documentation that allows to describe the
> >>> MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
> >>>
> >>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> >>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>
> >> How newly added patch can have already Rb tag? Was this review really, really
> >> performed internally or you just satisfy some internal managers requirements
> >> and fake the stats?
> >>
> >
> > I added this Reviewed-by tag because the patch was reviewed internally.
>
> What issues were identified by internal review, especially in the
> context of bindings?
>
> >>> ---
> >>>
> >>> Changelog v12:
> >>> - Newly add bindings for CSI2RX driver
> >>>
> >>> .../media/toshiba,visconti5-csi2rx.yaml | 104
> >> ++++++++++++++++++
> >>> 1 file changed, 104 insertions(+)
> >>> create mode 100644
> >>> Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> >>>
> >>> diff --git
> >>> a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
> >>> l
> >>> b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
> >>> l
> >>> new file mode 100644
> >>> index 000000000000..5488072bc82a
> >>> --- /dev/null
> >>> +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx
> >>> +++ .yaml
> >>> @@ -0,0 +1,104 @@
> >>> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) %YAML 1.2
> >>> +---
> >>> +$id:
> >>> +http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
> >>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> >>> +
> >>> +title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
> >>> +
> >>> +maintainers:
> >>> + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>> +
> >>> +description: |-
> >>
> >> Drop |-
> >>
> >
> > I'll drop "|-"
> >
> >>> + Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI
> >>> + CSI-2 video stream. Use with VIIF device. T.B.D
> >>> +
> >>> +properties:
> >>> + compatible:
> >>> + const: toshiba,visconti5-csi2rx
> >>
> >> Why this is called "RX"? Can you have a TX? I had impression that one cannot.
> >>
> >
> > VIIF has only MIPI CSI2 receiver (CSI2RX). There's no TX for it.
>
> So this device cannot be anything else? Then drop rx.
It's a compatible string, it identifies the IP core. As the SoC also has
a CSI-2 transmitter (as mentioned by Ishikawa-san), it makes sense to
name the CSI-2 receiver csi2rx.
> > Visconti also has VOIF (Video Output Interface) hardware which has
> > MIPI CSI2 (not DSI) transmitter (CSI2TX).
>
> Or this can be something else? Confusing.
In a camera capture pipeline the CSI-2 interface of the SoC is a CSI-2
receiver, but SoCs commonly have CSI-2 transmitters as well (even if
that's less common than receivers).
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2024-11-25 9:21 ` [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver Yuji Ishikawa
2024-11-25 10:11 ` Krzysztof Kozlowski
@ 2025-01-02 9:29 ` Laurent Pinchart
2025-01-20 0:10 ` yuji2.ishikawa
1 sibling, 1 reply; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-02 9:29 UTC (permalink / raw)
To: Yuji Ishikawa
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Sakari Ailus, Hans Verkuil, Nobuhiro Iwamatsu,
linux-media, linux-kernel, linux-arm-kernel, devicetree
Hello Ishikawa-san,
Thank you for the patch.
On Mon, Nov 25, 2024 at 06:21:39PM +0900, Yuji Ishikawa wrote:
> Adds the Device Tree binding documentation that allows to describe
> the MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> ---
>
> Changelog v12:
> - Newly add bindings for CSI2RX driver
>
> .../media/toshiba,visconti5-csi2rx.yaml | 104 ++++++++++++++++++
> 1 file changed, 104 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> new file mode 100644
> index 000000000000..5488072bc82a
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> @@ -0,0 +1,104 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
> +
> +maintainers:
> + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> +
> +description: |-
As Krzysztof mentioned, '|-' isn't needed. See
https://yaml-multiline.info/ for more information. The literal block
style indicator ('|') is only needed when line breaks need to be
preserved, e.g. when the description contains ASCII art.
> + Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI CSI-2 video
> + stream. Use with VIIF device. T.B.D
T.B.D ?
> +
> +properties:
> + compatible:
> + const: toshiba,visconti5-csi2rx
> +
> + reg:
> + items:
> + - description: Registers for CSI2 receiver control
> +
> + interrupts:
> + items:
> + - description: CSI2 Receiver Interrupt
> +
> + ports:
> + $ref: /schemas/graph.yaml#/properties/ports
> +
> + properties:
> + port@0:
> + $ref: /schemas/graph.yaml#/$defs/port-base
> + unevaluatedProperties: false
> + description:
> + Input port node, single endpoint describing the CSI-2 transmitter.
> +
> + properties:
> + endpoint:
> + $ref: video-interfaces.yaml#
Please use a full path for the ref:
$ref: /schemas/media/video-interfaces.yaml#
> + unevaluatedProperties: false
> +
> + properties:
> + data-lanes:
> + description: CSI2 receiver supports 1, 2, 3 or 4 data lanes
You can drop the description. The video-interfaces.yaml schema has a
more complete description, and the fact that the receiver supports
between 1 and 4 lanes is conveyed by minItems and items below.
> + minItems: 1
> + items:
> + - const: 1
> + - const: 2
> + - const: 3
> + - const: 4
> + required:
> + - data-lanes
> +
> + port@1:
> + $ref: /schemas/graph.yaml#/properties/port
> + description:
> + Output port node, single endpoint describing the Visconti VIIF.
> +
> + required:
> + - port@0
> + - port@1
> +
> +required:
> + - compatible
> + - reg
> + - interrupts
> + - ports
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> + #include <dt-bindings/interrupt-controller/irq.h>
> +
> + soc {
> + #address-cells = <2>;
> + #size-cells = <2>;
> +
> + csi2rx@1c008000 {
Node names should describe the function of the node, not the precise
model of the device. "csi2" would be a more appropriate name.
> + compatible = "toshiba,visconti5-csi2rx";
> + reg = <0 0x1c008000 0 0x400>;
> + interrupts = <GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
> +
> + ports {
> + #address-cells = <1>;
> + #size-cells = <0>;
> + port@0 {
> + reg = <0>;
> + csi2rx_in0: endpoint {
> + data-lanes = <1 2>;
> + remote-endpoint = <&imx219_out0>;
> + };
> + };
> + port@1 {
> + reg = <1>;
> + csi2rx_out0: endpoint {
> + remote-endpoint = <&csi_in0>;
> + };
> + };
> + };
> + };
> + };
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-12-17 12:51 ` Krzysztof Kozlowski
@ 2025-01-02 9:41 ` Laurent Pinchart
0 siblings, 0 replies; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-02 9:41 UTC (permalink / raw)
To: Krzysztof Kozlowski
Cc: yuji2.ishikawa, mchehab, robh, krzk+dt, conor+dt, sakari.ailus,
hverkuil-cisco, nobuhiro1.iwamatsu, linux-media, linux-kernel,
linux-arm-kernel, devicetree
On Tue, Dec 17, 2024 at 01:51:54PM +0100, Krzysztof Kozlowski wrote:
> On 17/12/2024 10:45, Laurent Pinchart wrote:
> > On Tue, Dec 17, 2024 at 06:43:22AM +0100, Krzysztof Kozlowski wrote:
> >> On 17/12/2024 01:00, yuji2.ishikawa@toshiba.co.jp wrote:
> >>> Hello Krzysztof
> >>>
> >>> Thank you for your review
> >>>
> >>>> -----Original Message-----
> >>>> From: Krzysztof Kozlowski <krzk@kernel.org>
> >>>> Sent: Monday, November 25, 2024 7:08 PM
> >>>> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> >>>> <yuji2.ishikawa@toshiba.co.jp>; Laurent Pinchart
> >>>> <laurent.pinchart@ideasonboard.com>; Mauro Carvalho Chehab
> >>>> <mchehab@kernel.org>; Rob Herring <robh@kernel.org>; Krzysztof Kozlowski
> >>>> <krzk+dt@kernel.org>; Conor Dooley <conor+dt@kernel.org>; Sakari Ailus
> >>>> <sakari.ailus@linux.intel.com>; Hans Verkuil <hverkuil-cisco@xs4all.nl>;
> >>>> iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ□ＤＩＴ○ＯＳＴ)
> >>>> <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>>> Cc: linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> >>>> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> >>>> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add
> >>>> Toshiba Visconti Video Input Interface
> >>>>
> >>>> On 25/11/2024 10:21, Yuji Ishikawa wrote:
> >>>>> Adds the Device Tree binding documentation that allows to describe the
> >>>>> Video Input Interface found in Toshiba Visconti SoCs.
> >>>>>
> >>>>> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> >>>>> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>>>
> >>>> Why this tag stayed and other was removed? What was the reason of tag
> >>>> removal?
> >>>>
> >>>
> >>> The stayed tag is due to internal review.
> >>
> >> Did the internal review really happened? How is it that immediately new
> >> version has internal review without any traces?
> >>
> >> I have doubts this review happened in the context of reviewer's
> >> statement of oversight.
> >>
> >>> The removed tag is due to code's change (split of csi2rx part) after the last review.
> >>> If the code is largely changed following the instruction of another reviewer
> >>> after obtaining the tags, how should the tags be handled?
> >>
> >> Drop all reviews and perform reviews on the list.
> >>
> >> Such internal review appearing afterwards is rather a proof it you are
> >> adding just the tags to satisfy your process. I have no way to even
> >> verify whether that person performed any reasonable review or maybe just
> >> acked your patch.
> >
> > How do you verify that for public reviews ?
>
> By quality or amount of comments. Or timing. Or reviewing cover letter
> without any feedback on individual patches.
>
> There are many, many ways. Considering how many companies were adding
> fake manager-review-tags in the past (or fake SoBs), I am pretty picky
> on that.
On the other hand I've heard numerous complains about patches being sent
by new developers from large companies to upstream lists without first
being reviewed internally by more experienced developers. You can't ask
people to review patches internally before submitting them upstream and
at the same time complain about R-b tags coming from internal reviews.
The value of a R-b tag, regardless of whether or not it comes from an
internal review, ultimately depends on the trust you have on the
reviewer. You can take that into account to decide when you consider a
patch to be ready to be merged, or to skip reviewing it if enough
reviewers you trust have looked at it first.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2024-11-25 9:21 ` [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface Yuji Ishikawa
2024-11-25 10:07 ` Krzysztof Kozlowski
@ 2025-01-02 9:56 ` Laurent Pinchart
2025-01-20 0:13 ` yuji2.ishikawa
1 sibling, 1 reply; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-02 9:56 UTC (permalink / raw)
To: Yuji Ishikawa
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Sakari Ailus, Hans Verkuil, Nobuhiro Iwamatsu,
linux-media, linux-kernel, linux-arm-kernel, devicetree
Hi Ishikawa-san,
Thank you for the patch.
On Mon, Nov 25, 2024 at 06:21:40PM +0900, Yuji Ishikawa wrote:
> Adds the Device Tree binding documentation that allows to describe
> the Video Input Interface found in Toshiba Visconti SoCs.
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> ---
> Changelog v2:
> - no change
>
> Changelog v3:
> - no change
>
> Changelog v4:
> - fix style problems at the v3 patch
> - remove "index" member
> - update example
>
> Changelog v5:
> - no change
>
> Changelog v6:
> - add register definition of BUS-IF and MPU
>
> Changelog v7:
> - remove trailing "bindings" from commit header message
> - remove trailing "Device Tree Bindings" from title
> - fix text wrapping of description
> - change compatible to visconti5-viif
> - explicitly define allowed properties for port::endpoint
>
> Changelog v8:
> - Suggestion from Krzysztof Kozlowski
> - rename bindings description file
> - use block style array instead of inline style
> - remove clock-lane (as it is fixed at position 0)
> - update sample node's name
> - use lowercase hex for literals
> - Suggestion from Laurent Pinchart
> - update description message port::description
> - remove port::endpoint::bus-type as it is fixed to <4>
> - remove port::endpoint::clock-lanes from example
> - add port::endpoint::data-lanes to required parameters list
> - fix sequence of data-lanes: <1 2 3 4> because current driver does not support data reordering
> - update port::endpoint::data-lanes::description
> - remove redundant type definition for port::endpoint::data-lanes
>
> Changelog v9:
> - place "required" after "properties"
> - dictionary ordering of properties
>
> Changelog v10:
> - no change
>
> Changelog v11:
> - no change
>
> Changelog v12:
> - remove property "clock-noncontinuous" as VIIF switches both modes automatically
> - remove property "link-frequencies" as VIIF does not use the information
> - remove reg[2] and interrupts[3] which are used for CSI2RX driver
> - update example to refer csi2rx for remote-endpoint
>
> .../media/toshiba,visconti5-viif.yaml | 95 +++++++++++++++++++
> 1 file changed, 95 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
>
> diff --git a/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> new file mode 100644
> index 000000000000..ef0452a47e98
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> @@ -0,0 +1,95 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/toshiba,visconti5-viif.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Toshiba Visconti5 SoC Video Input Interface
> +
> +maintainers:
> + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> +
> +description: |-
> + Toshiba Visconti5 SoC Video Input Interface (VIIF) receives videostream
> + from MIPI CSI-2 receiver device, processes the stream with image signal
> + processors (L1ISP, L2ISP), then stores pictures to main memory.
> +
> +properties:
> + compatible:
> + const: toshiba,visconti5-viif
> +
> + reg:
> + items:
> + - description: Registers for capture control
> + - description: Registers for bus interface unit control
> + - description: Registers for Memory Protection Unit
I'm a bit surprised by the lack of clocks.
> +
> + interrupts:
> + items:
> + - description: Sync Interrupt
> + - description: Status (Error) Interrupt
> + - description: L1ISP Interrupt
> +
> + port:
> + $ref: /schemas/graph.yaml#/$defs/port-base
> + unevaluatedProperties: false
> + description: CSI-2 input port, with a single endpoint connected to the CSI-2 transmitter.
> +
> + properties:
> + endpoint:
> + $ref: video-interfaces.yaml#
> + additionalProperties: false
> +
> + properties:
> + data-lanes:
> + description: VIIF supports 1, 2, 3 or 4 data lanes
> + minItems: 1
> + items:
> + - const: 1
> + - const: 2
> + - const: 3
> + - const: 4
Now that the CSI-2 receiver is modeled as a separate DT node, I don't
think data-lanes is applicable anymore. The interface between the CSI-2
receiver and the VIIF isn't a CSI-2 bus.
I think you can simplify the bindings by switching from port-base to
port, as you don't need to specify additional properties for the
endpoint:
port:
$ref: /schemas/graph.yaml#/$defs/port
description:
CSI-2 input port, with a single endpoint connected to the CSI-2
transmitter.
Please test this though (by running the DT bindings checks).
> +
> + remote-endpoint: true
> +
> + required:
> + - data-lanes
> + - remote-endpoint
> +
> +required:
> + - compatible
> + - reg
> + - interrupts
> + - port
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> + #include <dt-bindings/interrupt-controller/irq.h>
> +
> + soc {
> + #address-cells = <2>;
> + #size-cells = <2>;
> +
> + video@1c000000 {
> + compatible = "toshiba,visconti5-viif";
> + reg = <0 0x1c000000 0 0x6000>,
> + <0 0x1c00e000 0 0x1000>,
> + <0 0x2417a000 0 0x1000>;
> + interrupts = <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>,
> + <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
> + <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
> +
> + port {
> + #address-cells = <1>;
> + #size-cells = <0>;
> +
> + csi_in0: endpoint {
> + data-lanes = <1 2>;
> + remote-endpoint = <&csi2rx_out0>;
> + };
> + };
> + };
> + };
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver
2024-11-25 9:21 ` [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver Yuji Ishikawa
@ 2025-01-02 13:08 ` Laurent Pinchart
2025-01-20 0:21 ` yuji2.ishikawa
0 siblings, 1 reply; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-02 13:08 UTC (permalink / raw)
To: Yuji Ishikawa
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Sakari Ailus, Hans Verkuil, Nobuhiro Iwamatsu,
linux-media, linux-kernel, linux-arm-kernel, devicetree
Hello Ishikawa-san,
Thank you for the patch.
On Mon, Nov 25, 2024 at 06:21:42PM +0900, Yuji Ishikawa wrote:
> Add support to MIPI CSI-2 Receiver on Toshiba Visconti ARM SoCs.
> This driver is used with Visconti Video Input Interface driver.
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> ---
> Changelog v12:
> - Separate CSI2RX driver and made it independent driver
> - viif_csi2rx subdevice driver (in v11 patch) was removed.
> - dictionary order at Kconfig and Makefile
>
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/toshiba/Kconfig | 6 +
> drivers/media/platform/toshiba/Makefile | 2 +
> .../media/platform/toshiba/visconti/Kconfig | 16 +
> .../media/platform/toshiba/visconti/Makefile | 8 +
> .../platform/toshiba/visconti/csi2rx_drv.c | 791 ++++++++++++++++++
> 7 files changed, 825 insertions(+)
> create mode 100644 drivers/media/platform/toshiba/Kconfig
> create mode 100644 drivers/media/platform/toshiba/Makefile
> create mode 100644 drivers/media/platform/toshiba/visconti/Kconfig
> create mode 100644 drivers/media/platform/toshiba/visconti/Makefile
> create mode 100644 drivers/media/platform/toshiba/visconti/csi2rx_drv.c
>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index 85d2627776b6..761b15b07b90 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -86,6 +86,7 @@ source "drivers/media/platform/samsung/Kconfig"
> source "drivers/media/platform/st/Kconfig"
> source "drivers/media/platform/sunxi/Kconfig"
> source "drivers/media/platform/ti/Kconfig"
> +source "drivers/media/platform/toshiba/Kconfig"
> source "drivers/media/platform/verisilicon/Kconfig"
> source "drivers/media/platform/via/Kconfig"
> source "drivers/media/platform/xilinx/Kconfig"
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index ace4e34483dd..917145fe5171 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -29,6 +29,7 @@ obj-y += samsung/
> obj-y += st/
> obj-y += sunxi/
> obj-y += ti/
> +obj-y += toshiba/
> obj-y += verisilicon/
> obj-y += via/
> obj-y += xilinx/
> diff --git a/drivers/media/platform/toshiba/Kconfig b/drivers/media/platform/toshiba/Kconfig
> new file mode 100644
> index 000000000000..f02983f4fc97
> --- /dev/null
> +++ b/drivers/media/platform/toshiba/Kconfig
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +comment "Toshiba media platform drivers"
> +
> +source "drivers/media/platform/toshiba/visconti/Kconfig"
> +
> diff --git a/drivers/media/platform/toshiba/Makefile b/drivers/media/platform/toshiba/Makefile
> new file mode 100644
> index 000000000000..2bce85ef3b48
> --- /dev/null
> +++ b/drivers/media/platform/toshiba/Makefile
> @@ -0,0 +1,2 @@
> +# SPDX-License-Identifier: GPL-2.0-only
A blank line would be nice here.
> +obj-y += visconti/
> diff --git a/drivers/media/platform/toshiba/visconti/Kconfig b/drivers/media/platform/toshiba/visconti/Kconfig
> new file mode 100644
> index 000000000000..e5c92d598f8b
> --- /dev/null
> +++ b/drivers/media/platform/toshiba/visconti/Kconfig
> @@ -0,0 +1,16 @@
> +# SPDX-License-Identifier: GPL-2.0-only
A blank line would be nice here too.
> +config VIDEO_VISCONTI_CSI2RX
> + tristate "Visconti MIPI CSI-2 Receiver driver"
> + depends on V4L_PLATFORM_DRIVERS
> + depends on VIDEO_DEV && OF
> + depends on ARCH_VISCONTI || COMPILE_TEST
> + select MEDIA_CONTROLLER
> + select VIDEO_V4L2_SUBDEV_API
> + select V4L2_FWNODE
> + help
> + Support for Toshiba Visconti MIPI CSI-2 receiver,
> + which is used with Visconti Camera Interface driver.
> +
> + This driver yields 1 subdevice node for a hardware instance.
> + To compile this driver as a module, choose M here: the
> + module will be called visconti-csi2rx.
> diff --git a/drivers/media/platform/toshiba/visconti/Makefile b/drivers/media/platform/toshiba/visconti/Makefile
> new file mode 100644
> index 000000000000..62a029376134
> --- /dev/null
> +++ b/drivers/media/platform/toshiba/visconti/Makefile
> @@ -0,0 +1,8 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for the Visconti video input device driver
> +#
> +
> +visconti-csi2rx-objs = csi2rx_drv.o
> +
> +obj-$(CONFIG_VIDEO_VISCONTI_CSI2RX) += visconti-csi2rx.o
> diff --git a/drivers/media/platform/toshiba/visconti/csi2rx_drv.c b/drivers/media/platform/toshiba/visconti/csi2rx_drv.c
> new file mode 100644
> index 000000000000..94567963872a
> --- /dev/null
> +++ b/drivers/media/platform/toshiba/visconti/csi2rx_drv.c
> @@ -0,0 +1,791 @@
> +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
> +/* Toshiba Visconti Video Capture Support
/*
* Toshiba Visconti Video Capture Support
*
> + *
> + * (C) Copyright 2024 TOSHIBA CORPORATION
> + * (C) Copyright 2024 Toshiba Electronic Devices & Storage Corporation
> + */
> +
> +#include <linux/delay.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/of_graph.h>
You don't need those two headers. You however need to include
- linux/device.h (for devm_kzalloc)
- linux/property.h (for the fwnode_* API)
> +#include <linux/platform_device.h>
> +
> +#include <media/mipi-csi2.h>
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/v4l2-subdev.h>
> +
> +/* CSI2HOST register space */
> +#define REG_CSI2RX_NLANES 0x4
> +#define REG_CSI2RX_RESETN 0x8
> +#define REG_CSI2RX_INT_ST_MAIN 0xc
> +#define REG_CSI2RX_DATA_IDS_1 0x10
> +#define REG_CSI2RX_DATA_IDS_2 0x14
> +#define REG_CSI2RX_PHY_SHUTDOWNZ 0x40
> +#define REG_CSI2RX_PHY_RSTZ 0x44
> +
> +/* access to dphy external registers */
/* Access to dphy external registers */
Same in other comments below.
> +#define REG_CSI2RX_PHY_TESTCTRL0 0x50
> +#define BIT_TESTCTRL0_CLK_0 0
> +#define BIT_TESTCTRL0_CLK_1 BIT(1)
> +
> +#define REG_CSI2RX_PHY_TESTCTRL1 0x54
> +#define BIT_TESTCTRL1_ADDR BIT(16)
> +#define MASK_TESTCTRL1_DIN 0xff
> +#define MASK_TESTCTRL1_DOUT 0xff00
> +
> +#define REG_CSI2RX_INT_ST_PHY_FATAL 0xe0
> +#define REG_CSI2RX_INT_MSK_PHY_FATAL 0xe4
> +#define MASK_PHY_FATAL_ALL 0x0000000f
> +
> +#define REG_CSI2RX_INT_ST_PKT_FATAL 0xf0
> +#define REG_CSI2RX_INT_MSK_PKT_FATAL 0xf4
> +#define MASK_PKT_FATAL_ALL 0x0001000f
> +
> +#define REG_CSI2RX_INT_ST_FRAME_FATAL 0x100
> +#define REG_CSI2RX_INT_MSK_FRAME_FATAL 0x104
> +#define MASK_FRAME_FATAL_ALL 0x000f0f0f
> +
> +#define REG_CSI2RX_INT_ST_PHY 0x110
> +#define REG_CSI2RX_INT_MSK_PHY 0x114
> +#define MASK_PHY_ERROR_ALL 0x000f000f
> +
> +#define REG_CSI2RX_INT_ST_PKT 0x120
> +#define REG_CSI2RX_INT_MSK_PKT 0x124
> +#define MASK_PKT_ERROR_ALL 0x000f000f
> +
> +#define REG_CSI2RX_INT_ST_LINE 0x130
> +#define REG_CSI2RX_INT_MSK_LINE 0x134
> +#define MASK_LINE_ERROR_ALL 0x00ff00ff
> +
> +/* DPHY register space */
> +enum dphy_testcode {
> + DIG_TESTCODE_EXT = 0,
> + DIG_SYS_0 = 0x001,
> + DIG_SYS_1 = 0x002,
> + DIG_SYS_3 = 0x004,
> + DIG_SYS_7 = 0x008,
> + DIG_RX_STARTUP_OVR_2 = 0x0e2,
> + DIG_RX_STARTUP_OVR_3 = 0x0e3,
> + DIG_RX_STARTUP_OVR_4 = 0x0e4,
> + DIG_RX_STARTUP_OVR_5 = 0x0e5,
> + DIG_CB_2 = 0x1ac,
> + DIG_TERM_CAL_0 = 0x220,
> + DIG_TERM_CAL_1 = 0x221,
> + DIG_TERM_CAL_2 = 0x222,
> + DIG_CLKLANE_LANE_6 = 0x307,
> + DIG_CLKLANE_OFFSET_CAL_0 = 0x39d,
> + DIG_LANE0_OFFSET_CAL_0 = 0x59f,
> + DIG_LANE0_DDL_0 = 0x5e0,
> + DIG_LANE1_OFFSET_CAL_0 = 0x79f,
> + DIG_LANE1_DDL_0 = 0x7e0,
> + DIG_LANE2_OFFSET_CAL_0 = 0x99f,
> + DIG_LANE2_DDL_0 = 0x9e0,
> + DIG_LANE3_OFFSET_CAL_0 = 0xb9f,
> + DIG_LANE3_DDL_0 = 0xbe0,
> +};
> +
> +#define SYS_0_HSFREQRANGE_OVR BIT(5)
> +#define SYS_3_NO_REXT BIT(4)
> +#define SYS_7_RESERVED FIELD_PREP(0x1f, 0x0c)
> +#define SYS_7_DESKEW_POL BIT(5)
> +#define STARTUP_OVR_4_CNTVAL FIELD_PREP(0x70, 0x01)
> +#define STARTUP_OVR_4_DDL_EN BIT(0)
> +#define STARTUP_OVR_5_BYPASS BIT(0)
> +#define CB_2_LPRX_BIAS BIT(6)
> +#define CB_2_RESERVED FIELD_PREP(0x3f, 0x0b)
> +#define CLKLANE_RXHS_PULL_LONG BIT(7)
> +
> +/* bit mask for calibration result registers */
> +#define MASK_TERM_CAL_ERR 0
> +#define MASK_TERM_CAL_DONE BIT(7)
> +#define MASK_CLK_CAL_ERR BIT(4)
> +#define MASK_CLK_CAL_DONE BIT(0)
> +#define MASK_CAL_ERR BIT(2)
> +#define MASK_CAL_DONE BIT(1)
> +#define MASK_DDL_ERR BIT(1)
> +#define MASK_DDL_DONE BIT(2)
> +
> +#define VISCONTI_CSI2RX_ERROR_MONITORS_NUM 8
> +
> +/**
> + * struct visconti_csi2rx_line_err_target
> + *
> + * Virtual Channel and Data Type pair for CSI2RX line error monitor
> + *
> + * When 0 is set to dt, line error detection is disabled.
> + *
> + * @vc: Virtual Channel to monitor; Range 0..3
> + * @dt: Data Type to monitor; Range 0, 0x10..0x3f
> + */
> +struct visconti_csi2rx_line_err_target {
> + u32 vc[VISCONTI_CSI2RX_ERROR_MONITORS_NUM];
> + u32 dt[VISCONTI_CSI2RX_ERROR_MONITORS_NUM];
> +};
> +
> +#define CSI2RX_MIN_DATA_RATE 80U
> +#define CSI2RX_MAX_DATA_RATE 1500U
> +
> +#define VISCONTI_CSI2RX_PAD_SINK 0
> +#define VISCONTI_CSI2RX_PAD_SRC 1
> +#define VISCONTI_CSI2RX_PAD_NUM 2
> +
> +#define VISCONTI_CSI2RX_DEF_WIDTH 1920
> +#define VISCONTI_CSI2RX_DEF_HEIGHT 1080
> +#define VISCONTI_CSI2RX_MIN_WIDTH 640
> +#define VISCONTI_CSI2RX_MAX_WIDTH 3840
> +#define VISCONTI_CSI2RX_MIN_HEIGHT 480
> +#define VISCONTI_CSI2RX_MAX_HEIGHT 2160
> +
> +struct visconti_csi2rx {
> + struct device *dev;
> + void __iomem *base;
> +
> + struct v4l2_subdev subdev;
> + struct media_pad pads[VISCONTI_CSI2RX_PAD_NUM];
> + struct v4l2_async_notifier notifier;
> + struct v4l2_subdev *remote;
> + unsigned int remote_pad;
> +
> + unsigned int lanes;
> +
> + unsigned int irq;
> +};
> +
> +static inline struct visconti_csi2rx *notifier_to_csi2(struct v4l2_async_notifier *n)
> +{
> + return container_of(n, struct visconti_csi2rx, notifier);
> +}
> +
> +static inline struct visconti_csi2rx *sd_to_csi2(struct v4l2_subdev *sd)
> +{
> + return container_of(sd, struct visconti_csi2rx, subdev);
> +}
> +
> +static inline void visconti_csi2rx_write(struct visconti_csi2rx *priv, u32 regid, u32 val)
The media subsystem tries to limit line lengths to 80 columns, when it
doesn't hinder readability. For instance here you could write
static inline void visconti_csi2rx_write(struct visconti_csi2rx *priv,
u32 regid, u32 val)
If lines are just a few characters over 80 columns the need to break
them is less than if they approach 100 columns. It's a "soft limit"
policy to try and maximize readability.
> +{
> + writel(val, priv->base + regid);
> +}
> +
> +static inline u32 visconti_csi2rx_read(struct visconti_csi2rx *priv, u32 regid)
> +{
> + return readl(priv->base + regid);
> +}
> +
> +static inline void tick_testclk(struct visconti_csi2rx *priv)
Don't inline this or the next function, compilers are smart enough to
decide by themselves these days.
> +{
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, BIT_TESTCTRL0_CLK_1);
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, BIT_TESTCTRL0_CLK_0);
> +}
> +
> +static inline void set_dphy_addr(struct visconti_csi2rx *priv, u32 test_mode)
> +{
> + /* select testcode Ex space with top 4bits of test_mode */
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
> + BIT_TESTCTRL1_ADDR | DIG_TESTCODE_EXT);
> + tick_testclk(priv);
If writing to REG_CSI2RX_PHY_TESTCTRL1 always needs to be followed by a
call to tick_testclk(), I would create a visconti_csi2rx_dphy_write()
function:
static void visconti_csi2rx_dphy_write(struct visconti_csi2rx *priv, u32 data)
{
visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1, data);
visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, BIT_TESTCTRL0_CLK_1);
visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, BIT_TESTCTRL0_CLK_0);
}
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1, FIELD_GET(0xf00, test_mode));
And here
visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
FIELD_GET(0xf00, test_mode));
There are many other places below that go over 80 columns.
> + tick_testclk(priv);
> +
> + /* set bottom 8bit of test_mode */
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
> + BIT_TESTCTRL1_ADDR | FIELD_GET(0xff, test_mode));
> + tick_testclk(priv);
> +}
> +
> +static void write_dphy_param(struct visconti_csi2rx *priv, u32 test_mode, u8 test_in)
> +{
> + set_dphy_addr(priv, test_mode);
> +
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1, (u32)test_in);
Is the (u8) explicit cast needed ?
> + tick_testclk(priv);
> +}
> +
> +struct csi2rx_dphy_hs_info {
> + u32 rate;
> + u32 hsfreqrange;
> + u32 osc_freq_target;
> +};
> +
> +static const struct csi2rx_dphy_hs_info dphy_hs_info[] = {
> + { 80, 0x0, 0x1cc }, { 85, 0x10, 0x1cc }, { 95, 0x20, 0x1cc }, { 105, 0x30, 0x1cc },
> + { 115, 0x1, 0x1cc }, { 125, 0x11, 0x1cc }, { 135, 0x21, 0x1cc }, { 145, 0x31, 0x1cc },
> + { 155, 0x2, 0x1cc }, { 165, 0x12, 0x1cc }, { 175, 0x22, 0x1cc }, { 185, 0x32, 0x1cc },
> + { 198, 0x3, 0x1cc }, { 213, 0x13, 0x1cc }, { 228, 0x23, 0x1cc }, { 243, 0x33, 0x1cc },
> + { 263, 0x4, 0x1cc }, { 288, 0x14, 0x1cc }, { 313, 0x25, 0x1cc }, { 338, 0x35, 0x1cc },
> + { 375, 0x5, 0x1cc }, { 425, 0x16, 0x1cc }, { 475, 0x26, 0x1cc }, { 525, 0x37, 0x1cc },
> + { 575, 0x7, 0x1cc }, { 625, 0x18, 0x1cc }, { 675, 0x28, 0x1cc }, { 725, 0x39, 0x1cc },
> + { 775, 0x9, 0x1cc }, { 825, 0x19, 0x1cc }, { 875, 0x29, 0x1cc }, { 925, 0x3a, 0x1cc },
> + { 975, 0xa, 0x1cc }, { 1025, 0x1a, 0x1cc }, { 1075, 0x2a, 0x1cc }, { 1125, 0x3b, 0x1cc },
> + { 1175, 0xb, 0x1cc }, { 1225, 0x1b, 0x1cc }, { 1275, 0x2b, 0x1cc }, { 1325, 0x3c, 0x1cc },
> + { 1375, 0xc, 0x1cc }, { 1425, 0x1c, 0x1cc }, { 1475, 0x2c, 0x1cc }
osc_freq_target seems to always be 0x1cc, does it have to be stored in
this table ?
> +};
> +
> +static void get_dphy_hs_transfer_info(u32 dphy_rate, u32 *hsfreqrange, u32 *osc_freq_target)
> +{
> + unsigned int i;
> +
> + for (i = 1; i < ARRAY_SIZE(dphy_hs_info); i++) {
> + if (dphy_rate < dphy_hs_info[i].rate) {
> + *hsfreqrange = dphy_hs_info[i - 1].hsfreqrange;
> + *osc_freq_target = dphy_hs_info[i - 1].osc_freq_target;
> + return;
> + }
> + }
> +
> + /* not found; return the largest entry */
> + *hsfreqrange = dphy_hs_info[ARRAY_SIZE(dphy_hs_info) - 1].hsfreqrange;
> + *osc_freq_target = dphy_hs_info[ARRAY_SIZE(dphy_hs_info) - 1].osc_freq_target;
> +}
> +
> +static void visconti_csi2rx_set_dphy_rate(struct visconti_csi2rx *priv, u32 dphy_rate)
> +{
> + u32 hsfreqrange, osc_freq_target;
> +
> + get_dphy_hs_transfer_info(dphy_rate, &hsfreqrange, &osc_freq_target);
> +
> + write_dphy_param(priv, DIG_SYS_1, (u8)hsfreqrange);
I don't think the explicit cast is needed here either.
> + write_dphy_param(priv, DIG_SYS_0, SYS_0_HSFREQRANGE_OVR);
> + write_dphy_param(priv, DIG_RX_STARTUP_OVR_5, STARTUP_OVR_5_BYPASS);
> + write_dphy_param(priv, DIG_RX_STARTUP_OVR_4, STARTUP_OVR_4_CNTVAL);
> + write_dphy_param(priv, DIG_CB_2, CB_2_LPRX_BIAS | CB_2_RESERVED);
> + write_dphy_param(priv, DIG_SYS_7, SYS_7_DESKEW_POL | SYS_7_RESERVED);
> + write_dphy_param(priv, DIG_CLKLANE_LANE_6, CLKLANE_RXHS_PULL_LONG);
> + write_dphy_param(priv, DIG_RX_STARTUP_OVR_2, FIELD_GET(0xff, osc_freq_target));
> + write_dphy_param(priv, DIG_RX_STARTUP_OVR_3, FIELD_GET(0xf00, osc_freq_target));
> + write_dphy_param(priv, DIG_RX_STARTUP_OVR_4, STARTUP_OVR_4_CNTVAL | STARTUP_OVR_4_DDL_EN);
> +}
> +
> +static int visconti_csi2rx_initialize(struct visconti_csi2rx *priv, u32 num_lane, u32 dphy_rate,
> + const struct visconti_csi2rx_line_err_target *err_target)
> +{
> + u32 val;
> +
> + if (dphy_rate < CSI2RX_MIN_DATA_RATE || dphy_rate > CSI2RX_MAX_DATA_RATE) {
> + dev_err(priv->dev, "Unsupported PHY speed (%u Mbps)", dphy_rate);
> + return -ERANGE;
> + }
> +
> + /* 1st phase of initialization */
> + visconti_csi2rx_write(priv, REG_CSI2RX_RESETN, 1);
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 1);
> + ndelay(15U);
I don't mind much, but the U suffix here and in many other places seems
unneeded.
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 0);
> +
> + /* Configure D-PHY frequency range */
> + visconti_csi2rx_set_dphy_rate(priv, dphy_rate);
> +
> + /* 2nd phase of initialization */
> + visconti_csi2rx_write(priv, REG_CSI2RX_NLANES, (num_lane - 1U));
No need for the inner parentheses.
> + ndelay(5U);
> +
> + /* Release D-PHY from Reset */
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 1);
> + ndelay(5U);
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 1);
> +
> + /* configuration of line error target */
> + val = (err_target->vc[3] << 30U) | (err_target->dt[3] << 24U) | (err_target->vc[2] << 22U) |
> + (err_target->dt[2] << 16U) | (err_target->vc[1] << 14U) | (err_target->dt[1] << 8U) |
> + (err_target->vc[0] << 6U) | (err_target->dt[0]);
> + visconti_csi2rx_write(priv, REG_CSI2RX_DATA_IDS_1, val);
> + val = (err_target->vc[7] << 30U) | (err_target->dt[7] << 24U) | (err_target->vc[6] << 22U) |
> + (err_target->dt[6] << 16U) | (err_target->vc[5] << 14U) | (err_target->dt[5] << 8U) |
> + (err_target->vc[4] << 6U) | (err_target->dt[4]);
> + visconti_csi2rx_write(priv, REG_CSI2RX_DATA_IDS_2, val);
> +
> + /* configuration of mask */
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY_FATAL, MASK_PHY_FATAL_ALL);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT_FATAL, MASK_PKT_FATAL_ALL);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL, MASK_FRAME_FATAL_ALL);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY, MASK_PHY_ERROR_ALL);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT, MASK_PKT_ERROR_ALL);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_LINE, MASK_LINE_ERROR_ALL);
> +
> + return 0;
> +}
> +
> +struct visconti_csi2rx_format {
> + u32 code;
> + unsigned int bpp;
> +};
> +
> +static const struct visconti_csi2rx_format visconti_csi2rx_formats[] = {
> + { .code = MEDIA_BUS_FMT_RGB888_1X24, .bpp = 24 },
> + { .code = MEDIA_BUS_FMT_UYVY8_1X16, .bpp = 16 },
> + { .code = MEDIA_BUS_FMT_UYVY10_1X20, .bpp = 20 },
> + { .code = MEDIA_BUS_FMT_RGB565_1X16, .bpp = 16 },
> + { .code = MEDIA_BUS_FMT_SBGGR8_1X8, .bpp = 8 },
> + { .code = MEDIA_BUS_FMT_SGBRG8_1X8, .bpp = 8 },
> + { .code = MEDIA_BUS_FMT_SGRBG8_1X8, .bpp = 8 },
> + { .code = MEDIA_BUS_FMT_SRGGB8_1X8, .bpp = 8 },
> + { .code = MEDIA_BUS_FMT_SBGGR10_1X10, .bpp = 10 },
> + { .code = MEDIA_BUS_FMT_SGBRG10_1X10, .bpp = 10 },
> + { .code = MEDIA_BUS_FMT_SGRBG10_1X10, .bpp = 10 },
> + { .code = MEDIA_BUS_FMT_SRGGB10_1X10, .bpp = 10 },
> + { .code = MEDIA_BUS_FMT_SBGGR12_1X12, .bpp = 12 },
> + { .code = MEDIA_BUS_FMT_SGBRG12_1X12, .bpp = 12 },
> + { .code = MEDIA_BUS_FMT_SGRBG12_1X12, .bpp = 12 },
> + { .code = MEDIA_BUS_FMT_SRGGB12_1X12, .bpp = 12 },
> + { .code = MEDIA_BUS_FMT_SBGGR14_1X14, .bpp = 14 },
> + { .code = MEDIA_BUS_FMT_SGBRG14_1X14, .bpp = 14 },
> + { .code = MEDIA_BUS_FMT_SGRBG14_1X14, .bpp = 14 },
> + { .code = MEDIA_BUS_FMT_SRGGB14_1X14, .bpp = 14 },
> +};
> +
> +static const struct visconti_csi2rx_format *fmt_for_mbus_code(unsigned int mbus_code)
> +{
> + int i;
unsigned int
> +
> + for (i = 0; ARRAY_SIZE(visconti_csi2rx_formats); i++)
> + if (visconti_csi2rx_formats[i].code == mbus_code)
> + return &visconti_csi2rx_formats[i];
Please use curly braces for the 'for' statement.
No return when the look doesn't find a match ?
> +}
> +
> +static unsigned int bpp_for_mbus_code(unsigned int mbus_code)
> +{
> + const struct visconti_csi2rx_format *fmt = fmt_for_mbus_code(mbus_code);
> +
> + return fmt ? fmt->bpp : 0;
> +}
> +
> +static int64_t get_pixelclock(struct v4l2_subdev *sd)
> +{
> + struct v4l2_ctrl *ctrl;
> +
> + ctrl = v4l2_ctrl_find(sd->ctrl_handler, V4L2_CID_PIXEL_RATE);
> + if (!ctrl)
> + return -EINVAL;
> +
> + return v4l2_ctrl_g_ctrl_int64(ctrl);
> +}
> +
> +static const struct visconti_csi2rx_line_err_target err_target_vc0_alldt = {
> + /* select VC=0 */
> + /* select all supported DataTypes */
> + .dt = {
> + MIPI_CSI2_DT_RGB565,
> + MIPI_CSI2_DT_YUV422_8B,
> + MIPI_CSI2_DT_YUV422_10B,
> + MIPI_CSI2_DT_RGB888,
> + MIPI_CSI2_DT_RAW8,
> + MIPI_CSI2_DT_RAW10,
> + MIPI_CSI2_DT_RAW12,
> + MIPI_CSI2_DT_RAW14,
> + }
> +};
> +
> +static int visconti_csi2rx_start(struct visconti_csi2rx *priv, struct v4l2_subdev_state *state)
> +{
> + struct v4l2_mbus_framefmt *sink_fmt;
const
> + int cur_bpp, dphy_rate;
> + s64 pixelclock;
> +
> + /* get bpp for current format */
> + sink_fmt = v4l2_subdev_state_get_format(state, VISCONTI_CSI2RX_PAD_SINK);
> + cur_bpp = bpp_for_mbus_code(sink_fmt->code);
bpp = fmt_for_mbus_code(sink_fmt->code)->bpp;
and drop the bpp_for_mbus_code() function.
> +
> + /* get pixel clock */
> + pixelclock = get_pixelclock(priv->remote);
Use v4l2_get_link_freq() and drop the get_pixelclock() function.
> + if (pixelclock < 0)
> + return -EINVAL;
> +
> + dphy_rate = div64_u64((u64)pixelclock * (u32)cur_bpp, priv->lanes * 1000000);
> +
> + ndelay(15U);
> +
> + return visconti_csi2rx_initialize(priv, priv->lanes, dphy_rate, &err_target_vc0_alldt);
> +}
> +
> +static void visconti_csi2rx_stop(struct visconti_csi2rx *priv)
> +{
> + /* disable interrupt -> make sure registers cleared -> wait for current handlers finish */
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY_FATAL, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT_FATAL, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_LINE, 0);
> + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PHY_FATAL);
> + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PKT_FATAL);
> + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL);
> + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PHY);
> + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PKT);
> + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_LINE);
> + synchronize_irq(priv->irq);
> +
> + /* shutdown hardware */
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 0);
> + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 1);
> + visconti_csi2rx_write(priv, REG_CSI2RX_RESETN, 0);
> +}
> +
> +static int visconti_csi2rx_enable_streams(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> + u32 pad, u64 streams_mask)
> +{
> + struct visconti_csi2rx *priv = sd_to_csi2(sd);
> + struct v4l2_subdev *remote_sd;
> + struct media_pad *remote_pad;
> + int ret;
> +
> + remote_pad = media_pad_remote_pad_first(&sd->entity.pads[VISCONTI_CSI2RX_PAD_SINK]);
> + if (!remote_pad)
> + return -ENODEV;
Can't you use priv->remote and priv->remote_pad in this function instead
of getting the remote pad dynamically ? Same in
visconti_csi2rx_disable_streams().
> + remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> +
> + /* enabling: turn on CSI2RX -> turn on sensor */
> + ret = visconti_csi2rx_start(priv, state);
> + if (ret)
> + return ret;
> +
> + /* currently CSI2RX supports only stream0 in source pad */
> + ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index, BIT(0));
> + if (ret) {
> + visconti_csi2rx_stop(priv);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static int visconti_csi2rx_disable_streams(struct v4l2_subdev *sd, struct v4l2_subdev_state *state,
> + u32 pad, u64 streams_mask)
> +{
> + struct visconti_csi2rx *priv = sd_to_csi2(sd);
> + struct v4l2_subdev *remote_sd;
> + struct media_pad *remote_pad;
> +
> + remote_pad = media_pad_remote_pad_first(&sd->entity.pads[VISCONTI_CSI2RX_PAD_SINK]);
> + if (!remote_pad)
> + return -ENODEV;
> + remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> +
> + /* disabling: turn off sensor -> turn off CSI2RX */
> + v4l2_subdev_disable_streams(remote_sd, remote_pad->index, BIT(0));
> + visconti_csi2rx_stop(priv);
> +
> + return 0;
> +}
> +
> +static int visconti_csi2rx_enum_mbus_code(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_mbus_code_enum *code)
> +{
> + if (code->pad == VISCONTI_CSI2RX_PAD_SRC) {
> + const struct v4l2_mbus_framefmt *sink_fmt;
> +
> + /* SRC pad supports exactly the same format as SINK pad */
> + if (code->index)
> + return -EINVAL;
> + sink_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SINK);
> + code->code = sink_fmt->code;
> + return 0;
> + }
> +
> + if (code->index >= ARRAY_SIZE(visconti_csi2rx_formats))
> + return -EINVAL;
> + code->code = visconti_csi2rx_formats[code->index].code;
> +
> + return 0;
> +}
> +
> +static int visconti_csi2rx_init_state(struct v4l2_subdev *sd, struct v4l2_subdev_state *sd_state)
> +{
> + struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
> +
> + sink_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SINK);
> + src_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SRC);
> +
> + sink_fmt->width = VISCONTI_CSI2RX_DEF_WIDTH;
> + sink_fmt->height = VISCONTI_CSI2RX_DEF_HEIGHT;
> + sink_fmt->field = V4L2_FIELD_NONE;
> + sink_fmt->code = visconti_csi2rx_formats[0].code;
Please also initialize the colourspace fields. V4L2_COLORSPACE_RAW,
V4L2_XFER_FUNC_NONE, V4L2_YCBCR_ENC_601 and V4L2_QUANTIZATION_FULL_RANGE
should be appropriate defaults.
> +
> + *src_fmt = *sink_fmt;
> +
> + return 0;
> +}
> +
> +static int visconti_csi2rx_set_pad_format(struct v4l2_subdev *sd,
> + struct v4l2_subdev_state *sd_state,
> + struct v4l2_subdev_format *fmt)
> +{
> + struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
> +
> + /* SRC PAD has the same format as SINK PAD */
> + if (fmt->pad == 1)
if (fmt->pad == VISCONTI_CSI2RX_PAD_SRC)
> + return v4l2_subdev_get_fmt(sd, sd_state, fmt);
> +
> + sink_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SINK);
> +
> + *sink_fmt = fmt->format;
> + sink_fmt->width = clamp_t(u32, fmt->format.width, VISCONTI_CSI2RX_MIN_WIDTH,
> + VISCONTI_CSI2RX_MAX_WIDTH);
> + sink_fmt->height = clamp_t(u32, fmt->format.height, VISCONTI_CSI2RX_MIN_HEIGHT,
> + VISCONTI_CSI2RX_MAX_HEIGHT);
> + if (!fmt_for_mbus_code(sink_fmt->code))
> + sink_fmt->code = visconti_csi2rx_formats[0].code;
> + fmt->format = *sink_fmt;
> +
> + /* source pad should have the same format */
> + src_fmt = v4l2_subdev_state_get_format(sd_state, VISCONTI_CSI2RX_PAD_SRC);
> + *src_fmt = *sink_fmt;
> +
> + return 0;
> +}
> +
> +static const struct media_entity_operations visconti_csi2rx_entity_ops = {
> + .link_validate = v4l2_subdev_link_validate,
> +};
> +
> +static const struct v4l2_subdev_video_ops visconti_csi2rx_video_ops = {
> + .s_stream = v4l2_subdev_s_stream_helper,
As the only driver that will use this CSI-2 RX is the VIIF driver, and
that driver uses v4l2_subdev_enable_streams(), .s_stream() will never be
called. You can drop the v4l2_subdev_video_ops.
> +};
> +
> +static const struct v4l2_subdev_pad_ops visconti_csi2rx_pad_ops = {
> + .enum_mbus_code = visconti_csi2rx_enum_mbus_code,
You also need to implement .enum_frame_size()
> + .disable_streams = visconti_csi2rx_disable_streams,
> + .enable_streams = visconti_csi2rx_enable_streams,
> + .get_fmt = v4l2_subdev_get_fmt,
> + .set_fmt = visconti_csi2rx_set_pad_format,
> +};
> +
> +static const struct v4l2_subdev_ops visconti_csi2rx_subdev_ops = {
> + .video = &visconti_csi2rx_video_ops,
> + .pad = &visconti_csi2rx_pad_ops,
> +};
> +
> +static const struct v4l2_subdev_internal_ops visconti_csi2rx_internal_ops = {
> + .init_state = visconti_csi2rx_init_state,
> +};
> +
> +static int visconti_csi2rx_notify_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_connection *asc)
> +{
> + struct visconti_csi2rx *priv = notifier_to_csi2(notifier);
> + int pad;
> +
> + pad = media_entity_get_fwnode_pad(&subdev->entity, asc->match.fwnode, MEDIA_PAD_FL_SOURCE);
> + if (pad < 0) {
> + dev_err(priv->dev, "Failed to find pad for %s\n", subdev->name);
> + return pad;
> + }
> +
> + priv->remote = subdev;
> + priv->remote_pad = pad;
> +
> + return media_create_pad_link(&subdev->entity, pad, &priv->subdev.entity, 0,
> + MEDIA_LNK_FL_ENABLED);
Can you have multiple sources connected to the same CSI-2 receiver ? If
not, you can make the link to the source immutable.
> +}
> +
> +static void visconti_csi2rx_notify_unbind(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_connection *asc)
> +{
> + struct visconti_csi2rx *priv = notifier_to_csi2(notifier);
> +
> + priv->remote = NULL;
> +}
> +
> +static const struct v4l2_async_notifier_operations visconti_csi2rx_notify_ops = {
> + .bound = visconti_csi2rx_notify_bound,
> + .unbind = visconti_csi2rx_notify_unbind,
> +};
> +
> +static int visconti_csi2rx_parse_v4l2(struct visconti_csi2rx *priv,
> + struct v4l2_fwnode_endpoint *vep)
> +{
> + /* Only port 0 endpoint 0 is valid. */
> + if (vep->base.port || vep->base.id)
> + return -ENOTCONN;
You call fwnode_graph_get_endpoint_by_id() with port and endpoint set to
0, so I think you can drop this check.
> +
> + priv->lanes = vep->bus.mipi_csi2.num_data_lanes;
> +
> + /* got trouble */
> + if (vep->bus_type != V4L2_MBUS_CSI2_DPHY) {
> + dev_err(priv->dev, "Specified bus type is not supported\n");
If you only support D-PHY, then set the bus_type to V4L2_MBUS_CSI2_DPHY
instead of V4L2_MBUS_UNKNOWN in visconti_csi2rx_parse_dt().
v4l2_fwnode_endpoint_parse() will return an error if the bus type is not
D-PHY, and you can drop this check.
> + return -EINVAL;
> + }
> +
> + if (priv->lanes != 1 && priv->lanes != 2 && priv->lanes != 4) {
> + dev_err(priv->dev, "Unsupported number of data-lanes for D-PHY: %u\n", priv->lanes);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static int visconti_csi2rx_parse_dt(struct visconti_csi2rx *priv)
> +{
> + struct v4l2_async_connection *asc;
> + struct fwnode_handle *fwnode;
> + struct fwnode_handle *ep;
> + struct v4l2_fwnode_endpoint v4l2_ep = {
> + .bus_type = V4L2_MBUS_UNKNOWN,
> + };
> + int ret;
> +
> + ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(priv->dev), 0, 0, 0);
> + if (!ep) {
> + dev_err(priv->dev, "Not connected to subdevice\n");
> + return -EINVAL;
> + }
> +
> + ret = v4l2_fwnode_endpoint_parse(ep, &v4l2_ep);
> + if (ret) {
> + dev_err(priv->dev, "Could not parse v4l2 endpoint\n");
> + fwnode_handle_put(ep);
> + return -EINVAL;
> + }
> +
> + ret = visconti_csi2rx_parse_v4l2(priv, &v4l2_ep);
I would inline what is left of that function in here.
> + if (ret) {
> + fwnode_handle_put(ep);
> + return ret;
> + }
> +
> + fwnode = fwnode_graph_get_remote_endpoint(ep);
> + fwnode_handle_put(ep);
> +
> + v4l2_async_subdev_nf_init(&priv->notifier, &priv->subdev);
> + priv->notifier.ops = &visconti_csi2rx_notify_ops;
> +
> + asc = v4l2_async_nf_add_fwnode(&priv->notifier, fwnode, struct v4l2_async_connection);
> + fwnode_handle_put(fwnode);
> + if (IS_ERR(asc))
> + return PTR_ERR(asc);
> +
> + ret = v4l2_async_nf_register(&priv->notifier);
> + if (ret)
> + v4l2_async_nf_cleanup(&priv->notifier);
> +
> + return ret;
> +}
> +
> +static irqreturn_t visconti_csi2rx_irq(int irq, void *dev_id)
> +{
> + struct visconti_csi2rx *priv = dev_id;
> + u32 event;
> +
> + event = visconti_csi2rx_read(priv, REG_CSI2RX_INT_ST_MAIN);
> + dev_err(priv->dev, "CSI2RX error 0x%x.\n", event);
Should this be at least rate-limited ?
> +
> + return IRQ_HANDLED;
> +}
> +
> +static const struct of_device_id visconti_csi2rx_of_table[] = {
> + {
> + .compatible = "toshiba,visconti5-csi2rx",
> + },
> + {},
{ /* Sentinel */ },
is customary. You can also drop the trailing comma, as there should
never be any entry after this one.
> +};
> +
> +static int visconti_csi2rx_probe(struct platform_device *pdev)
> +{
> + struct visconti_csi2rx *priv;
> + int irq, ret;
> +
> + priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
> + if (!priv)
> + return -ENOMEM;
> +
> + priv->dev = &pdev->dev;
> +
> + priv->base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(priv->base)) {
> + dev_err(priv->dev, "Failed to get registers\n");
devm_platform_ioremap_resource() prints an error message already, you
can drop this one.
> + return PTR_ERR(priv->base);
> + }
> +
> + irq = platform_get_irq(pdev, 0);
> + if (irq < 0)
> + return irq;
Here, on the other hand, an error message would be useful. You can use
dev_err_probe():
return dev_err_probe(priv->dev, irq, "Failed to get IRQ\n");
> + ret = devm_request_irq(&pdev->dev, irq, visconti_csi2rx_irq, 0, KBUILD_MODNAME, priv);
> + priv->irq = irq;
> + if (ret) {
> + dev_err(priv->dev, "request irq failed: %d\n", ret);
> + return ret;
> + }
> +
> + platform_set_drvdata(pdev, priv);
> +
> + ret = visconti_csi2rx_parse_dt(priv); /*this function does v4l2_async_nf_register */
> + if (ret)
> + return ret;
> +
> + priv->subdev.owner = THIS_MODULE;
Not needed, this is handled by v4l2_async_register_subdev()
> + priv->subdev.dev = &pdev->dev;
> + v4l2_subdev_init(&priv->subdev, &visconti_csi2rx_subdev_ops);
> + v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
> + snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s", KBUILD_MODNAME,
> + dev_name(&pdev->dev));
> +
> + priv->subdev.internal_ops = &visconti_csi2rx_internal_ops;
> + priv->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> + priv->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
MEDIA_ENT_F_VID_IF_BRIDGE seems more appropriate.
> + priv->subdev.entity.ops = &visconti_csi2rx_entity_ops;
> +
> + priv->pads[VISCONTI_CSI2RX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> + priv->pads[VISCONTI_CSI2RX_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
> +
> + ret = media_entity_pads_init(&priv->subdev.entity, VISCONTI_CSI2RX_PAD_NUM, priv->pads);
ret = media_entity_pads_init(&priv->subdev.entity, ARRAY_SIZE(priv->pads),
priv->pads);
> + if (ret)
> + goto err_cleanup_async;
> +
> + ret = v4l2_subdev_init_finalize(&priv->subdev);
> + if (ret)
> + goto err_cleanup_media_entity;
> +
> + ret = v4l2_async_register_subdev(&priv->subdev);
> + if (ret < 0)
> + goto err_cleanup_subdev_state;
> +
> + return 0;
> +
> +err_cleanup_subdev_state:
> + v4l2_subdev_cleanup(&priv->subdev);
> +
> +err_cleanup_media_entity:
> + media_entity_cleanup(&priv->subdev.entity);
> +
> +err_cleanup_async:
> + v4l2_async_nf_unregister(&priv->notifier);
> + v4l2_async_nf_cleanup(&priv->notifier);
> +
> + return ret;
> +}
> +
> +static void visconti_csi2rx_remove(struct platform_device *pdev)
> +{
> + struct visconti_csi2rx *priv = platform_get_drvdata(pdev);
> +
> + v4l2_async_nf_unregister(&priv->notifier);
> + v4l2_async_nf_cleanup(&priv->notifier);
> + v4l2_async_unregister_subdev(&priv->subdev);
> +
> + v4l2_subdev_cleanup(&priv->subdev);
> + media_entity_cleanup(&priv->subdev.entity);
> +}
> +
> +static struct platform_driver visconti_csi2rx_driver = {
> + .probe = visconti_csi2rx_probe,
> + .remove = visconti_csi2rx_remove,
> + .driver = {
> + .name = "visconti_csi2rx_dev",
> + .of_match_table = visconti_csi2rx_of_table,
> + },
> +};
> +
> +module_platform_driver(visconti_csi2rx_driver);
> +
> +MODULE_AUTHOR("Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>");
> +MODULE_DESCRIPTION("Toshiba Visconti CSI-2 receiver driver");
> +MODULE_LICENSE("Dual BSD/GPL");
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format
2024-11-25 9:21 ` [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format Yuji Ishikawa
@ 2025-01-02 13:10 ` Laurent Pinchart
2025-01-20 0:15 ` yuji2.ishikawa
0 siblings, 1 reply; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-02 13:10 UTC (permalink / raw)
To: Yuji Ishikawa
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Sakari Ailus, Hans Verkuil, Nobuhiro Iwamatsu,
linux-media, linux-kernel, linux-arm-kernel, devicetree
Hi Ishikawa-san,
Thank you for the patch.
On Mon, Nov 25, 2024 at 06:21:41PM +0900, Yuji Ishikawa wrote:
> Adds the Toshiba Visconti VIIF specific metadata format
s/Adds/Add/
s/format/formats./
>
> - V4L2_META_FMT_VISCONTI_VIIF_PARAMS for ISP parameters
> - V4L2_META_FMT_VISCONTI_VIIF_STATS for ISP statistics
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
Assuming the documentation of the formats in subsequent patches is fine,
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changelog v10:
> - add entry for V4L2_META_FMT_VISCONTI_VIIF_PARAMS
> - add entry for V4L2_META_FMT_VISCONTI_VIIF_STATS
>
> Changelog v11:
> - no change
>
> Changelog v12:
> - add description for meta formats at v4l2-ioctl.c
>
> drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
> include/uapi/linux/videodev2.h | 4 ++++
> 2 files changed, 6 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 0304daa8471d..f7facb63b8ea 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1470,6 +1470,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_META_FMT_RPI_BE_CFG: descr = "RPi PiSP BE Config format"; break;
> case V4L2_META_FMT_RPI_FE_CFG: descr = "RPi PiSP FE Config format"; break;
> case V4L2_META_FMT_RPI_FE_STATS: descr = "RPi PiSP FE Statistics format"; break;
> + case V4L2_META_FMT_VISCONTI_VIIF_PARAMS: descr = "Visconti ISP Parameters"; break;
> + case V4L2_META_FMT_VISCONTI_VIIF_STATS: descr = "Visconti ISP Statistics"; break;
> case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break;
> case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8-bit Generic Meta, 10b CSI-2"; break;
> case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8-bit Generic Meta, 12b CSI-2"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index a5418759e2ba..9e1f66fdf038 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -863,6 +863,10 @@ struct v4l2_pix_format {
> #define V4L2_META_FMT_RPI_FE_CFG v4l2_fourcc('R', 'P', 'F', 'C') /* PiSP FE configuration */
> #define V4L2_META_FMT_RPI_FE_STATS v4l2_fourcc('R', 'P', 'F', 'S') /* PiSP FE stats */
>
> +/* Vendor specific - used for Visconti VIIF sub-system */
> +#define V4L2_META_FMT_VISCONTI_VIIF_PARAMS v4l2_fourcc('V', 'I', 'F', 'P') /* ISP Params */
> +#define V4L2_META_FMT_VISCONTI_VIIF_STATS v4l2_fourcc('V', 'I', 'F', 'S') /* ISP Stats */
> +
> #ifdef __KERNEL__
> /*
> * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface
2024-11-25 9:21 ` [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface Yuji Ishikawa
@ 2025-01-02 13:16 ` Laurent Pinchart
2025-02-05 12:31 ` yuji2.ishikawa
0 siblings, 1 reply; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-02 13:16 UTC (permalink / raw)
To: Yuji Ishikawa
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Sakari Ailus, Hans Verkuil, Nobuhiro Iwamatsu,
linux-media, linux-kernel, linux-arm-kernel, devicetree
Hi Ishikawa-san,
Thank you for the patch.
On Mon, Nov 25, 2024 at 06:21:46PM +0900, Yuji Ishikawa wrote:
> Added entries for visconti Video Input Interface driver, including;
Commit messages are written using the imperative mood, as if you were
giving orders to someone:
s/Added/Add/
> * device tree bindings
> * source files
> * documentation files
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
I would have added the MAINTAINERS entry in 1/8 with just the DT binding
file, and added more files in corresponding patches. That would be
easier to review and check if entries are missing. I don't mind much
though, so if you prefer keeping it this way:
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changelog v2:
> - no change
>
> Changelog v3:
> - added entry for driver API documentation
>
> Changelog v4:
> - added entry for header file
>
> Changelog v5:
> - no change
>
> Changelog v6:
> - update path to VIIF driver source files
>
> Changelog v7:
> - no change
>
> Changelog v8:
> - rename bindings description file
>
> Changelog v9:
> - no change
>
> Changelog v10:
> - add a separate entry of VIIF driver
>
> Changelog v11:
> - no change
>
> Changelog v12:
> - add a bindings description of CSI2RX driver
>
> MAINTAINERS | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b878ddc99f94..b5c819e94e9b 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -23430,6 +23430,18 @@ F: Documentation/devicetree/bindings/media/i2c/tc358743.txt
> F: drivers/media/i2c/tc358743*
> F: include/media/i2c/tc358743.h
>
> +TOSHIBA VISCONTI VIIF DRIVER
> +M: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> +M: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> +L: linux-media@vger.kernel.org
> +S: Maintained
> +F: Documentation/admin-guide/media/visconti-viif.*
> +F: Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> +F: Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> +F: Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> +F: drivers/media/platform/toshiba/visconti/
> +F: include/uapi/linux/visconti_viif.h
> +
> TOSHIBA WMI HOTKEYS DRIVER
> M: Azael Avalos <coproscefalo@gmail.com>
> L: platform-driver-x86@vger.kernel.org
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver
2024-11-25 9:21 ` [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver Yuji Ishikawa
@ 2025-01-02 21:26 ` Laurent Pinchart
2025-02-05 12:29 ` yuji2.ishikawa
0 siblings, 1 reply; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-02 21:26 UTC (permalink / raw)
To: Yuji Ishikawa
Cc: Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Sakari Ailus, Hans Verkuil, Nobuhiro Iwamatsu,
linux-media, linux-kernel, linux-arm-kernel, devicetree
Hi Ishikawa-san,
Thank you for the patch.
Overall the documentation looks quite good, it has significantly
improved compared to early versions.
On Mon, Nov 25, 2024 at 06:21:45PM +0900, Yuji Ishikawa wrote:
> Added description of Video Input Interface driver of
s/Added/Add/
> Toshiba Visconti architecture.
> It includes hardware organization, structure of the driver
> and metadata format for embedded image signal processor.
>
> Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> ---
> Changelog v3:
> - Newly add documentation to describe SW and HW
>
> Changelog v4:
> - no change
>
> Changelog v5:
> - no change
>
> Changelog v6:
> - add description of CSI2RX subdevice
> - add ordering of ioctl(S_FMT) and ioctl(S_EXT_CTRLS)
>
> Changelog v7:
> - no change
>
> Changelog v8:
> - add usage of V4L2_CTRL_TYPE_VISCONTI_ISP
>
> Changelog v9:
> - fix warning: set reference target for keyword V4L2_CTRL_TYPE_VISCONTI_ISP
>
> Changelog v10:
> - use parameter buffers instead of compound control
> - removed description of vendor specific compound control
> - add description of parameter buffers for ISP control
> - update directory structure
> - remove documents under driver-api
> - add documents to admin-guide, userspace-api
>
> Changelog v11:
> - update usage of the driver
>
> Changelog v12:
> - add description of CSI2RX driver
> - description of resizer subdevice
> - add block diagrams of VIIF and ISP
> - update usage of the driver
>
> .../admin-guide/media/v4l-drivers.rst | 1 +
> .../admin-guide/media/visconti-viif.dot | 22 +
> .../admin-guide/media/visconti-viif.rst | 435 ++++++++++++++++++
> .../userspace-api/media/v4l/meta-formats.rst | 1 +
> .../media/v4l/metafmt-visconti-viif.rst | 48 ++
> 5 files changed, 507 insertions(+)
> create mode 100644 Documentation/admin-guide/media/visconti-viif.dot
> create mode 100644 Documentation/admin-guide/media/visconti-viif.rst
> create mode 100644 Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
>
> diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
> index b6af448b9fe9..81054512e768 100644
> --- a/Documentation/admin-guide/media/v4l-drivers.rst
> +++ b/Documentation/admin-guide/media/v4l-drivers.rst
> @@ -32,5 +32,6 @@ Video4Linux (V4L) driver-specific documentation
> si476x
> starfive_camss
> vimc
> + visconti-viif
> visl
> vivid
> diff --git a/Documentation/admin-guide/media/visconti-viif.dot b/Documentation/admin-guide/media/visconti-viif.dot
> new file mode 100644
> index 000000000000..cc75c73336fb
> --- /dev/null
> +++ b/Documentation/admin-guide/media/visconti-viif.dot
> @@ -0,0 +1,22 @@
> +digraph board {
> + rankdir=TB
> + n00000001 [label="{{<port0> 0 | <port4> 4} | visconti-viif:isp\n/dev/v4l-subdev0 | {<port1> 1 | <port2> 2 | <port3> 3 | <port5> 5}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000001:port1 -> n00000008:port0
> + n00000001:port2 -> n0000000b:port0
> + n00000001:port3 -> n00000016
> + n00000001:port5 -> n0000001e
> + n00000008 [label="{{<port0> 0} | visconti-viif:resizer0\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000008:port1 -> n0000000e
> + n0000000b [label="{{<port0> 0} | visconti-viif:resizer1\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> + n0000000b:port1 -> n00000012
> + n0000000e [label="viif_capture_post0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> + n00000012 [label="viif_capture_post1\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> + n00000016 [label="viif_capture_sub\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
> + n0000001a [label="viif_params\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
> + n0000001a -> n00000001:port4
> + n0000001e [label="viif_stats\n/dev/video4", shape=box, style=filled, fillcolor=yellow]
> + n00000030 [label="{{<port0> 0} | visconti_csi2rx 1c008000.csi2rx\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000030:port1 -> n00000001:port0
> + n00000035 [label="{{} | imx219 1-0010\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> + n00000035:port0 -> n00000030:port0
> +}
> diff --git a/Documentation/admin-guide/media/visconti-viif.rst b/Documentation/admin-guide/media/visconti-viif.rst
> new file mode 100644
> index 000000000000..c2e85fb6f8c1
> --- /dev/null
> +++ b/Documentation/admin-guide/media/visconti-viif.rst
> @@ -0,0 +1,435 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +======================================================
> +Visconti Video Input Interface Driver (visconti-viif)
> +======================================================
> +
> +Introduction
> +============
> +
> +This file documents the driver for the Video Input Interface (VIIF) that is
> +part of Toshiba Visconti SoCs.
> +The driver is located under drivers/media/platform/toshiba/visconti and uses
> +the Media-Controller API.
If you intend to make this two paragraphs, you should have a blank line
in-between:
--------
This file documents the driver for the Video Input Interface (VIIF) that is
part of Toshiba Visconti SoCs.
The driver is located under drivers/media/platform/toshiba/visconti and uses
the Media-Controller API.
--------
If you intend to make this a single paragraph, there should be no line
break after the first sentence:
--------
This file documents the driver for the Video Input Interface (VIIF) that is
part of Toshiba Visconti SoCs. The driver is located under
drivers/media/platform/toshiba/visconti and uses the Media-Controller API.
--------
> +
> +The driver module is named visconti-viif,
> +and is enabled through the CONFIG_VIDEO_VISCONTI_VIIF config option.
> +The CSI-2 receiver part is controlled by another module named visconti-csi2rx,
> +which is enabled through the CONFIG_VIDEO_VISCONTI_CSI2RX config option.
Please reflow text to reach as close to 80 columns:
--------
The driver module is named visconti-viif, and is enabled through the
CONFIG_VIDEO_VISCONTI_VIIF config option. The CSI-2 receiver part is controlled
by another module named visconti-csi2rx, which is enabled through the
CONFIG_VIDEO_VISCONTI_CSI2RX config option.
--------
Same comment below where applicable.
> +
> +The Visconti VIIF Hardware
> +==========================
> +
> +The Visconti VIIF hardware is an internally developed video capture device.
> +Following function modules are integrated:
> +
> +* MIPI CSI-2 receiver (CSI2RX)
> +* L1 Image Signal Processor (L1ISP)
> +
> + * Correction, enhancement, adjustment on bayer images.
> +
> +* L2 Image Signal Processor (L2ISP)
> +
> + * Lens distortion correction
> + * Scaling & Cropping with up to 2 parameter sets
> + * Formatting picture (RGB, YUV, Grayscale, ...)
> + * Integrated DMAC: Writing picture into main memory
> +
> +* Video DMAC
> +
> + * Writing picture into main memory
> +
> +Visconti5 SoC has two VIIF hardware instances.
> +
> +
> +The hardware block diagram is shown below.::
> +
> + The VIIF hardware
> + "POST0"
> + "RGB with scale 0"
> + +--------+ +----------+ +-----+ +-----+ +-----+ +--------+
> + | Sensor |--->| CSI2RX |--->| | | | | |--->| memory |
> + +--------+ +----------+ | | | | | | +--------+
> + | | | L1 | | L2 | "POST1"
> + | |--->| ISP |--->| ISP | "RGB with scale 1"
> + | | | | | | +--------+
> + | MUX | | | | |--->| memory |
> + | | +-----+ +-----+ +--------+
> + | | "SUB"
> + | | "RAW w/o scale"
> + | | +------------+ +--------+
> + | |------> | Video DMAC |--->| memory |
> + +-----+ +------------+ +--------+
> +
> +Topology
> +========
> +
> +Graph
> +-----
> +
> +.. _visconti_viif_topology_graph:
> +
> +.. kernel-figure:: visconti-viif.dot
> + :alt: Diagram of the default media pipeline topology
> + :align: center
> +
> +The driver has 3 video devices for capturing images:
> +
> +- viif_capture_post0: capture device for image.
> + - corresponds to L2ISP.
> +- viif_capture_post1: capture device for image.
> + - corresponds to L2ISP.
> +- viif_capture_sub: capture device for bayer image.
> + - corresponds to Video DMAC.
> +
> +The driver has 2 video devices for controlling ISP.
> +
> +- viif_params: a metadata output device that receives ISP parameters.
> + - corresponds to L1ISP and L2ISP.
> +- viif_stats: a metadata capture device that sends statistics.
> + - corresponds to L1ISP and L2ISP.
> +
> +The driver has 2 subdevices:
I count three subdevs in the list below (and there are actually two
resizer instances, so that would be 4 subdevs).
> +
> +- visconti_csi2rx: CSI-2 receiver operation.
> + - corresponds to CSI2RX.
> +- visconti-viif:isp: Image Signal Processor operation.
> + - corresponds to L1ISP and L2ISP.
> +- visconti-viif:resizer: Scaling operation of Image Signal Processor.
> + - corresponds to L2ISP.
> +
> +visconti_csi2rx - CSI2 Receiver Subdevice Node
> +---------------------------------------------------
The title underline should have the same length as the title.
> +
> +This subdevice node corresponds to a MIPI CSI2 receiver.
> +It resides between an image sensor subdevice and the ISP subdevice.
> +It controls CSI2 link configuration and training process.
> +
> +visconti-viif:isp - ISP Subdevice Node
> +--------------------------------------
> +
> +This subdevice node corresponds to L1/L2 ISPs.
> +It receives pictures from an sensor (via CSI2RX),
s/an sensor/a sensor/
> +applies multiple operations on pictures, then passes resulting images to capture nodes.
> +
> +ISP configurations/parameters are passed from userland via viif_params node.
> +The status of ISP operations are passed to userland via viif_stats node.
"stats" stands for "statistics", so I would write "The statistics
computed by the ISP are passed to ...".
And after reading more of this document, I realize that the viif_stats
node captures both statistics and status information. You could write
"The statistics computed by the ISP and the frame processing status are
passed to ...".
> +
> +L1 ISP provides following operations:
> +
> +- Input: accepts 8, 10, 12, 14bit bayer format
> + - Operation selector; :c:type:`viif_l1_input_mode_config`
> + - HDR image / PWL (Piecewse Linear Compression) image
> + - with preprocessing / without preprocessing
> + - HDRE: HDR expansion (only for PWL image);
> + see :c:type:`viif_l1_hdre_config`
> +- Preprocessing: generate intermediate data (24bit RAW)
> + - SLIC: Bit slicing (x3 12bit planes for preprocessing);
> + see :c:type:`viif_l1_img_extraction_config`
> + - ABPC/DPC: Blemish/Defect pixel correction :c:type:`viif_l1_dpc_config`
> + - PWHB: Preset white balance; see :c:type:`viif_l1_preset_white_balance_config`
> + - RCNR: RAW color noise reduction; see :c:type:`viif_l1_raw_color_noise_reduction_config`
> + - HDRS: HDR synthesis; see :c:type:`viif_l1_hdrs_config`
> +- Processing on RAW image: Main Process (MPRO)
> + - BLVC: black level correction and normalization;
> + see :c:type:`viif_l1_black_level_correction_config`
> + - LSSC: Lens shading correction; see :c:type:`viif_l1_lsc_config`
> + - MPRO: digital amplifier; see :c:type:`viif_l1_main_process_config`
> + - MPRO: bayer demosaicing; see :c:type:`viif_l1_main_process_config`
> + - MPRO: color matrix correction; see :c:type:`viif_l1_main_process_config`
> + - HDRC: HDR compression;
If my understanding is correct, this implements global and local tone
mapping. I would mention it explicitly here, those terms are more common
than "HDR compression". You could write for instance
- HDRC: HDR compression (global and local tone mapping);
> + see :c:type:`viif_l1_hdrc_config`, :c:type:`viif_l1_hdrc_ltm_config`,
> + :c:type:`viif_l1_rgb_to_y_coef_config`
> +- Processing on RGB/YUV image: Video Process (VPRO)
> + - VPRO: gamma correction; see :c:type:`viif_l1_gamma_config`
> + - VPRO: RGB2YUV;
> + see :c:type:`viif_l1_rgb_to_y_coef_config`,
> + :c:type:`viif_l1_img_quality_adjustment_config`
> + - VPRO: image quality adjustment; see :c:type:`viif_l1_img_quality_adjustment_config`
> +- Output: 16bit YUV
> +- Feedback loop
> + - AWHB: auto white balance; see :c:type:`viif_l1_awb_config`,
> + :c:type:`viif_isp_capture_status`
Does this mean that the ISP can compute white balance gains
automatically, implementing AWB in hardware ? That's interesting, it's a
feature I haven't seen before.
> + - AEXP: auto exposure (average luminance calculation);
> + see :c:type:`viif_l1_avg_lum_generation_config`,
> + :c:type:`viif_l1_rgb_to_y_coef_config`, :c:type:`viif_isp_capture_status`
> + - AG: analog gain calculation;
> + see :c:type:`viif_l1_ag_mode_config`, :c:type:`viif_l1_ag_config`
> +
> +Below is the block diagram::
> +
> + L1ISP::INPUT
> +
> + +--------+ +-----+ +-----+
> + | Input |--------------->| |--------------------->| |
> + | 24bHDR | | | | |
> + +--------+ | 24b | | 24b |
> + | RAW | | RAW |
> + +--------+ +------+ | (0) | | (1) |
> + | Input |--->| HDRE |--->| | +------------+ | |
> + | 24bPWL | | | | |--->| preprocess |--->| |
> + +--------+ +------+ +-----+ +------------+ +-----+
I'm not entirely sure to understand this correctly, could you please
correct me where I'm mistaken ?
I understand that the PWL input is HDR contents merged/stitched on the
sensor side, and compressed to a smaller than 24bpp width to reduce bus
bandwidth using a PWL function. The HDRE block applies the inverse
function to recover linear 24 bit data. In that case, the "preprocess"
pipeline operates on a single channel (using one of the param_h, param_m
and param_l set of parameters - which one ?). The HDRS blocks should be
disabled (how ?).
The HDR input, on the other hand, provides 2 or 3 channels with
different sensitivities (high, middle and low, from sensors that
implement DOL or DCG HDR). The preprocess pipeline operates on those
channels with different sets of parameters, and the HDRS combines the
channels into a single 24 bit image.
I'm not entirely sure how the two or three channels are provided to the
ISP, with DOL sensors there's a delay before the sensor starts
outputting the short (and very short) lines, so line buffers are needed
to realign the lines. I don't see this in the block diagram.
It would be nice to expand the documentation a bit with such information
about HDR processing, as I assume other developers may face similar
questions.
> +
> + L1ISP::INPUT::preprocess
> +
> + +-----+ +-----+
> + | 24b | +------+ +------+ +------+ +------+ +------+ | 24b |
> + | RAW |--->| SLIC |--->| ABPC |--->| PWHB |--->| RCNR |--->| HDRS |--->| RAW |
> + | (0) | +------+ +------+ +------+ +------+ +------+ | (1) |
> + +-----+ +-----+
> +
> + L1ISP::MainProcess(MPRO)
> +
> + +-----+
> + | 24b | +------+ +------+
> + | RAW |--->| BLVC |--->| LSSC |---+
> + | (1) | +------+ +------+ |
> + +-----+ |
> + |
> + +------------------------------+
> + |
> + | +-----------+ +-------------+ +--------+ +-----+
> + +--->| MPRO | | MPRO | | MPRO | +------+ | 16b |
> + | Digital |--->| Demosaicing |----| Color |--->| HDRC |--->| RGB |
> + +--->| Amplifier | | | | Matrix | +------+ | |
> + | +-----------+ +-------------+ +--------+ +-----+
> + | | |
> + | +--------------+ | | +------+
> + +----| Auto |<----+ +--->| AEXP |---> Auto-Exposure statistics
> + | Whitebalance | +------+
> + +--------------+
> + |
> + +------------------------------> Auto-Whitebalance statistics
> +
> + L1ISP::VideoProcess(VPRO)
> +
> + +-----+ +------------+ +------------+ +---------------+ +--------+
> + | 16b |--->| Gamma |--->| RGB2YUV |--->| Image Quality |--->| Output |
> + | RGB | | Correction | | Conversion | | Adjustment | | 16b |
> + | | +------------+ +------------+ +---------------+ | YUV |
> + +-----+ +--------+
> +
> + L1ISP::AnalogGain
> +
> + statistics +-------------+ +------------------+
> + information ---> (user SW) --->| Analog Gain |--->| ABPC, RCNR, LSSC |
> + +-------------+ | MPRO, VPRO |
> + +------------------+
I'm a bit puzzled by "analog gain" here, as the ISP operates on digital
data only. Does the ISP need to be informed of the analog gain values
computed by the AEGC software algorithm and applied to the camera
sensor, for instance to estimate the noise level based on the analog
gain and adapt noise filtering accordingly ? Or is it something else ?
Edit: the text below answers this question :-)
> +
> +L2 ISP provides following operations:
> +
> +- Input: accepts 16bit YUV / RGB
> +- Operations:
> + - Lens undistortion; see :c:type:`viif_l2_undist`
The structure is named viif_l2_undist_config. Please make sure to
compile the documentation, you should then get warnings for C types that
are not found.
> + - Scaling; see :c:type:`viif_l2_roi`
viif_l2_roi_config
> + - Cropping; see :c:type:`viif_l2_roi`
Looking at the implementation of the resizer subdevs, I see that the
crop and compose rectangles can be set, but they don't seem to ne used
to configure the resizer. Instead, the viif_l2_roi_config parameters are used
to configure cropping on the resizer input and scaling. This discrepancy
isn't good. I see two options to fix it:
- Keep configuring the resizer through viif_l2_roi_config and drop the resizer
subdevs. This will simplify the driver. The main drawback is that it
won't be possible to implement digital zoom (by changing the resizer
configuration) asynchronously from the ISP parameters buffers, which
can be useful to lower the latency of digital zoom.
- Drop viif_l2_roi_config, and configure the resizer from the formats,
crop and selection rectangles on the resizer subdev pads. This makes
the driver more complex. The main advantage is that digital zoom will
be configurable with a smaller latency, but the drawback is that it
won't be possible (or it will be more difficult) to configure it
synchronously with other ISP parameters.
There are drivers in mainline that implement either of those options, so
you can pick the one you think is best.
An additional issue is that the hardware seems to implement cropping on
the output of the resizer only, not on the input. Given that the size of
the images output by the ISP pipeline must stay constant during video
capture (otherwise there would be a risk of buffer overflow), modifying
the crop rectangle on the output of the resizer is usually not allowed
during streaming. I think this could be worked around by allowing
modification only of the left and top coordinates during streaming, but
configuring everything through viif_l2_roi_config would likely be
easier. In that case, you should probably extend viif_l2_roi_config with
the crop offsets.
All of this reflects my current understanding of the ISP architecture,
based on this document and on the driver code, so please correct me if
there's anything I misunderstood. We can discuss the different options
further before you modify the driver and send a new version.
> + - Gamma correction; see :c:type:`viif_l2_gamma_config`
> + - YUV2RGB
> +- Output: RGB, YUV422, YUV444
> +
> +Below is the block diagram::
> +
> + L2ISP
> +
> + +-------+ +------------+ +--------------+ +---------+
> + | Input |--->| YUV2RGB |--->| Lens |--->| Scaling |---> |
> + | Image | | Conversion | | Undistortion | | |---> |
> + +-------+ +------------+ +--------------+ +---------+ |
Is the scaling configuration for the two outputs independent ? If so I
would move the scaling block just before gamma correction, in each of
the branches below.
> + |
> + +----------------------------------------------------------+
> + |
> + | +----------+ +------------+ +--------+ +--------+
> + +--->|Gamma |--->| Colorspace |--->| Data |--->| Output |
> + | |Correction| | Conversion | | Packer | | Image |
> + | +----------+ +------------+ +--------+ +--------+
> + |
> + | +----------+ +------------+ +--------+ +--------+
> + +--->|Gamma |--->| Colorspace |--->| Data |--->| Output |
> + |Correction| | Conversion | | Packer | | Image |
> + +----------+ +------------+ +--------+ +--------+
> +
> +visconti-viif:resizer - Resizer Subdevice Node
> +----------------------------------------------
> +
> +The resizer subdevice resides between ISP subdevice and Capture device
> +on a capture path for post0 and post1.
> +It receives resize and crop parameters for the specific capture path
> +and controls L2ISP HW.
> +
> +following selection rectangles can be passed at VIDIOC_S_SELECTION ioctl.
s/following/The following/
> +
> +- sink pads's compose rectangle (V4L2_SEL_TGT_COMPOSE) for scaling
> +- source pad's crop rectangle (V4L2_SEL_TGT_CROP) for cropping
> +
> +
> +viif_capture_post0, viif_capture_post1 - Processed Image Capture Video Node
> +---------------------------------------------------------------------------
> +
> +These video nodes are used for capturing images processed at ISPs.
> +Supported capture formats are as follows:
> +
> +- V4L2_PIX_FMT_RGB24
> +- V4L2_PIX_FMT_ABGR32
> +- V4L2_PIX_FMT_YUV422M
> +- V4L2_PIX_FMT_YUV444M
The hardware doesn't support semi-planar formats (NV12 or NV16) or
packed formats (YUYV) ?
> +- V4L2_PIX_FMT_Y16
> +
> +Bayer format is not supported. Use viif_capture_sub instead.
> +
> +POST0 and POST1 can output images from the same input image
> +using different cropping and scaling settings.
> +
> +viif_capture_sub - Raw Image Capture Video Node
> +-----------------------------------------------
> +
> +This video node is used for capturing bayer image from the sensor.
> +The output picture has exactly the same resolution and format as the sensor input.
> +The pipeline does not edit pixel values.
> +However, when writing pixel values to memory, they are shifted to the MSB
> +to match either 8bit or 16bit.
> +
> +Therefore, resulting capture formats are as follows:
> +
> +- for 8bit RAW input:
> + - V4L2_PIX_FMT_SRGGB8
> + - V4L2_PIX_FMT_SGRBG8
> + - V4L2_PIX_FMT_SGBRG8
> + - V4L2_PIX_FMT_SBGGR8
> +- for 10, 12, 14bit RAW input:
> + - V4L2_PIX_FMT_SRGGB16
> + - V4L2_PIX_FMT_SGRBG16
> + - V4L2_PIX_FMT_SGBRG16
> + - V4L2_PIX_FMT_SBGGR16
> +
> +.. _viif_params:
> +
> +viif_params - ISP Parameters Video Node
> +---------------------------------------
> +
> +The viif_params video node receives a set of ISP parameters from userspace
> +to be applied to the hardware during a video stream.
> +
> +The buffer format is defined by struct :c:type:`visconti_viif_isp_config`, and userspace should set
> +:ref:`V4L2_META_FMT_VISCONTI_VIIF_PARAMS <v4l2-meta-fmt-visconti-viif-params>` as the data format.
> +
> +.. _viif_stats:
> +
> +viif_stats - Statistics Video Node
> +----------------------------------
> +
> +The viif_stats video node provides current status of ISP.
The viif_stats video node provides statistics computed by the ISP and
frame processing status.
> +
> +Following information is included:
> +
> +* statistics of auto white balance
> +* average luminance information which can be used by auto exposure software impl.
s/impl/implementation/
I would also add
* CSI-2 receiver calibration and error status
* ISP error status
It's quite uncommon to provide this kind of status through ISP stats
buffers, but it sounds like an interesting idea. Other drivers usually
keep error counters in the kernel and expose them through debugfs.
> +
> +The buffer format is defined by struct :c:type:`visconti_viif_isp_stat`, and userspace should set
> +:ref:`V4L2_META_FMT_VISCONTI_VIIF_STATS <v4l2-meta-fmt-visconti-viif-stats>` as the data format.
> +
> +Feedback Operations
> +===================
> +
> +Among the so-called 3A functions, VIIF provides only auto-whitebalance and auto-exposure.
> +Auto-whitebalance is a standalone hardware feature.
> +Some status information is available through the ISP statistics interface.
> +
> +Auto-exposure is realized through a combination of hardware and userland software.
> +VIIF provides weighted average luminance information through the ISP statistics interface.
> +The userland application calculates the sensor gain, sensor exposure and ISP digital gain.
> +The calculated parameters are then passed to sensor's controls and the ISP parameter interface.
> +
> +Among ISP parameters, there are parameters called AG (analog gain).
> +Actually, AG parameters have nothing to do with auto-exposure.
> +It controls "strength" in several signal correction algorithms.
> +Below is the list of the functions which may be affected by AG parameters:
> +
> +- ABPC/DPC
> +- RCNR
> +- LSSC
> +- MPRO: color matrix correction
> +- VPRO
> +
> +Capturing Video Frames Example
> +==============================
> +
> +In the following example,
> +imx219 camera is connected to pad 0 of 'visconti_csi2rx' subdevice.
> +
> +The following commands yield three pictures with different zoom ratio:
> +- main path 0: 1.5x zoom, crop 1920x1080, RGB picture
> +- main path 1: 0.67x zoom, crop 640x480, RGB picture
> +- sub path: 1920x1080 RAW picture
> +
> +.. code-block:: bash
> +
> + # set the links
> + media-ctl -d platform:visconti-viif-0 -r
> + media-ctl -d platform:visconti-viif-0 -l '"imx219 1-0010":0 -> "visconti_csi2rx 1c008000.csi2rx":0 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"visconti_csi2rx 1c008000.csi2rx":1 -> "visconti-viif:isp":0 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":1 -> "visconti-viif:resizer0":0 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":2 -> "visconti-viif:resizer1":0 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":3 -> "viif_capture_sub":0 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer0":1 -> "viif_capture_post0":0 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer1":1 -> "viif_capture_post1":0 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"viif_params":0 -> "visconti-viif:isp":4 [1]'
> + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":5 -> "viif_stats":0 [1]'
> +
> + # set format for imx219
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"imx219 1-0010":0 [fmt:SRGGB10_1X10/1920x1080]'
> +
> + # set format for csi2rx
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti_csi2rx 1c008000.csi2rx":0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range]'
> +
> + # set format for isp sink pad
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":0 [fmt:SRGGB10_1X10/1920x1080]'
> +
> + # set format for resizer pads
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer0":0 '"[fmt:YUV8_1X24/1920x1080 compose:(0,0)/2880x1620]"
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer0":1 '"[crop:(480,16)/1920x1080]"
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer1":0 '"[fmt:YUV8_1X24/1920x1080 compose:(0,0)/1280x720]"
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:resizer1":1 '"[crop:(320,32)/640x480]"
> +
> + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":1 [fmt:YUV8_1X24/1024 crop:(640,0)/1024x1024]'
> +
> + # set format for main path0
> + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "width=1920,height=1080"
> + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "pixelformat=RGB3"
> +
> + # set format for main path1
> + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "width=640,height=480"
> + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v "pixelformat=RGB3"
> +
> + # start streaming
> + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 --stream-mmap --stream-count 1000 &
> +
> + # start streaming with other devices while viif_capture_post0 is running
> + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post1 --stream-mmap --stream-count 10
> + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_sub --stream-mmap --stream-count 10
> +
> +Use of coherent memory
> +======================
> +
> +Visconti5 SoC has two independent DDR SDRAM controllers.
> +Each controller is mapped to 36bit address space.
> +
> +Accelerator bus masters have two paths to access memory;
> +one is directly connected to SDRAM controller,
> +the another is connected via a cache coherency bus
> +which keeps coherency among CPUs.
> +
> +From accelerators and CPUs, the address map is following:
> +
> +* 0x0_8000_0000 DDR0 direct access
> +* 0x4_8000_0000 DDR0 coherency bus
> +* 0x8_8000_0000 DDR1 direct access
> +* 0xC_8000_0000 DDR1 coherency bus
> +
> +The base address can be specified with "memory" and "reserved-memory" elements
> +in a device tree description.
> +It's not recommended to mix direct address and coherent address.
> +
> +The Visconti5 VIIF driver always use only direct address to configure Video DMACs of the hardware.
> +This design is to avoid great performance loss at coherency bus caused by massive memory access.
> +You should not put the dma_coherent attribute to viif element in device tree.
> +Cache operations are done automatically by videobuf2 driver.
> diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
> index 86ffb3bc8ade..2336842f0c26 100644
> --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
> @@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata` interface only.
> metafmt-pisp-fe
> metafmt-rkisp1
> metafmt-uvc
> + metafmt-visconti-viif
> metafmt-vivid
> metafmt-vsp1-hgo
> metafmt-vsp1-hgt
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> new file mode 100644
> index 000000000000..dc4b31627fe1
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> @@ -0,0 +1,48 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. _v4l2-meta-fmt-visconti-viif-params:
> +
> +.. _v4l2-meta-fmt-visconti-viif-stats:
> +
> +***************************************************************************************
> +V4L2_META_FMT_VISCONTI_VIIF_PARAMS ('vifp'), V4L2_META_FMT_VISCONTI_VIIF_STATS ('vifs')
> +***************************************************************************************
> +
> +Configuration parameters
> +========================
> +
> +The configuration parameters are passed to the
> +:ref:`viif_params <viif_params>` metadata output video node, using
> +the :c:type:`v4l2_meta_format` interface. The buffer contains
> +a single instance of the C structure :c:type:`visconti_viif_isp_config` defined in
> +``visconti_viif.h``. So the structure can be obtained from the buffer by:
> +
> +.. code-block:: c
> +
> + struct visconti_viif_isp_config *params = (struct visconti_viif_isp_config*) buffer;
> +
> +VIIF statistics
> +===============
> +
> +The VIIF device collects different statistics over an input Bayer frame.
> +Those statistics are obtained from the :ref:`viif_stats <viif_stats>`
> +metadata capture video node,
> +using the :c:type:`v4l2_meta_format` interface. The buffer contains a single
> +instance of the C structure :c:type:`visconti_viif_isp_stat` defined in
> +``visconti_viif.h``. So the structure can be obtained from the buffer by:
> +
> +.. code-block:: c
> +
> + struct visconti_viif_isp_stat *stats = (struct visconti_viif_isp_stat*) buffer;
> +
> +The statistics collected are Exposure, AWB (auto white balance) and errors.
> +See :c:type:`visconti_viif_isp_stat` for details of the statistics.
> +
> +The statistics and configuration parameters described here are usually
> +consumed and produced by dedicated user space libraries that comprise the
> +tuning tools using software control loop.
> +
> +visconti viif uAPI data types
> +=============================
> +
> +.. kernel-doc:: include/uapi/linux/visconti_viif.h
Assuming the documentation in the header file is adequate, the level of
detail here is fine.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver
2025-01-02 9:29 ` Laurent Pinchart
@ 2025-01-20 0:10 ` yuji2.ishikawa
0 siblings, 0 replies; 34+ messages in thread
From: yuji2.ishikawa @ 2025-01-20 0:10 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent
Thank you for your review comments.
> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Thursday, January 2, 2025 6:29 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add
> Toshiba Visconti MIPI CSI-2 Receiver
>
> Hello Ishikawa-san,
>
> Thank you for the patch.
>
> On Mon, Nov 25, 2024 at 06:21:39PM +0900, Yuji Ishikawa wrote:
> > Adds the Device Tree binding documentation that allows to describe the
> > MIPI CSI-2 Receiver found in Toshiba Visconti SoCs.
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > ---
> >
> > Changelog v12:
> > - Newly add bindings for CSI2RX driver
> >
> > .../media/toshiba,visconti5-csi2rx.yaml | 104
> ++++++++++++++++++
> > 1 file changed, 104 insertions(+)
> > create mode 100644
> > Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yaml
> >
> > diff --git
> > a/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
> > l
> > b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.yam
> > l
> > new file mode 100644
> > index 000000000000..5488072bc82a
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx
> > +++ .yaml
> > @@ -0,0 +1,104 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) %YAML 1.2
> > +---
> > +$id:
> > +http://devicetree.org/schemas/media/toshiba,visconti5-csi2rx.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Toshiba Visconti5 SoC MIPI CSI-2 receiver
> > +
> > +maintainers:
> > + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > +
> > +description: |-
>
> As Krzysztof mentioned, '|-' isn't needed. See https://yaml-multiline.info/ for
> more information. The literal block style indicator ('|') is only needed when line
> breaks need to be preserved, e.g. when the description contains ASCII art.
>
I'll remove "|-".
> > + Toshiba Visconti5 SoC MIPI CSI-2 receiver device receives MIPI
> > + CSI-2 video stream. Use with VIIF device. T.B.D
>
> T.B.D ?
>
This should have been removed.
I'll remove it.
> > +
> > +properties:
> > + compatible:
> > + const: toshiba,visconti5-csi2rx
> > +
> > + reg:
> > + items:
> > + - description: Registers for CSI2 receiver control
> > +
> > + interrupts:
> > + items:
> > + - description: CSI2 Receiver Interrupt
> > +
> > + ports:
> > + $ref: /schemas/graph.yaml#/properties/ports
> > +
> > + properties:
> > + port@0:
> > + $ref: /schemas/graph.yaml#/$defs/port-base
> > + unevaluatedProperties: false
> > + description:
> > + Input port node, single endpoint describing the CSI-2
> transmitter.
> > +
> > + properties:
> > + endpoint:
> > + $ref: video-interfaces.yaml#
>
> Please use a full path for the ref:
>
> $ref: /schemas/media/video-interfaces.yaml#
>
I'll use a full path description
> > + unevaluatedProperties: false
> > +
> > + properties:
> > + data-lanes:
> > + description: CSI2 receiver supports 1, 2, 3 or 4 data
> > + lanes
>
> You can drop the description. The video-interfaces.yaml schema has a more
> complete description, and the fact that the receiver supports between 1 and 4
> lanes is conveyed by minItems and items below.
>
I'll drop the description.
> > + minItems: 1
> > + items:
> > + - const: 1
> > + - const: 2
> > + - const: 3
> > + - const: 4
> > + required:
> > + - data-lanes
> > +
> > + port@1:
> > + $ref: /schemas/graph.yaml#/properties/port
> > + description:
> > + Output port node, single endpoint describing the Visconti VIIF.
> > +
> > + required:
> > + - port@0
> > + - port@1
> > +
> > +required:
> > + - compatible
> > + - reg
> > + - interrupts
> > + - ports
> > +
> > +additionalProperties: false
> > +
> > +examples:
> > + - |
> > + #include <dt-bindings/interrupt-controller/arm-gic.h>
> > + #include <dt-bindings/interrupt-controller/irq.h>
> > +
> > + soc {
> > + #address-cells = <2>;
> > + #size-cells = <2>;
> > +
> > + csi2rx@1c008000 {
>
> Node names should describe the function of the node, not the precise model of
> the device. "csi2" would be a more appropriate name.
>
I'll use "csi2" for the node's name.
> > + compatible = "toshiba,visconti5-csi2rx";
> > + reg = <0 0x1c008000 0 0x400>;
> > + interrupts = <GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
> > +
> > + ports {
> > + #address-cells = <1>;
> > + #size-cells = <0>;
> > + port@0 {
> > + reg = <0>;
> > + csi2rx_in0: endpoint {
> > + data-lanes = <1 2>;
> > + remote-endpoint = <&imx219_out0>;
> > + };
> > + };
> > + port@1 {
> > + reg = <1>;
> > + csi2rx_out0: endpoint {
> > + remote-endpoint = <&csi_in0>;
> > + };
> > + };
> > + };
> > + };
> > + };
>
> --
> Regards,
>
> Laurent Pinchart
Regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface
2025-01-02 9:56 ` Laurent Pinchart
@ 2025-01-20 0:13 ` yuji2.ishikawa
0 siblings, 0 replies; 34+ messages in thread
From: yuji2.ishikawa @ 2025-01-20 0:13 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent.
Thank you for your review comments.
> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Thursday, January 2, 2025 6:56 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add
> Toshiba Visconti Video Input Interface
>
> Hi Ishikawa-san,
>
> Thank you for the patch.
>
> On Mon, Nov 25, 2024 at 06:21:40PM +0900, Yuji Ishikawa wrote:
> > Adds the Device Tree binding documentation that allows to describe the
> > Video Input Interface found in Toshiba Visconti SoCs.
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > ---
> > Changelog v2:
> > - no change
> >
> > Changelog v3:
> > - no change
> >
> > Changelog v4:
> > - fix style problems at the v3 patch
> > - remove "index" member
> > - update example
> >
> > Changelog v5:
> > - no change
> >
> > Changelog v6:
> > - add register definition of BUS-IF and MPU
> >
> > Changelog v7:
> > - remove trailing "bindings" from commit header message
> > - remove trailing "Device Tree Bindings" from title
> > - fix text wrapping of description
> > - change compatible to visconti5-viif
> > - explicitly define allowed properties for port::endpoint
> >
> > Changelog v8:
> > - Suggestion from Krzysztof Kozlowski
> > - rename bindings description file
> > - use block style array instead of inline style
> > - remove clock-lane (as it is fixed at position 0)
> > - update sample node's name
> > - use lowercase hex for literals
> > - Suggestion from Laurent Pinchart
> > - update description message port::description
> > - remove port::endpoint::bus-type as it is fixed to <4>
> > - remove port::endpoint::clock-lanes from example
> > - add port::endpoint::data-lanes to required parameters list
> > - fix sequence of data-lanes: <1 2 3 4> because current driver does not
> support data reordering
> > - update port::endpoint::data-lanes::description
> > - remove redundant type definition for port::endpoint::data-lanes
> >
> > Changelog v9:
> > - place "required" after "properties"
> > - dictionary ordering of properties
> >
> > Changelog v10:
> > - no change
> >
> > Changelog v11:
> > - no change
> >
> > Changelog v12:
> > - remove property "clock-noncontinuous" as VIIF switches both modes
> > automatically
> > - remove property "link-frequencies" as VIIF does not use the
> > information
> > - remove reg[2] and interrupts[3] which are used for CSI2RX driver
> > - update example to refer csi2rx for remote-endpoint
> >
> > .../media/toshiba,visconti5-viif.yaml | 95
> +++++++++++++++++++
> > 1 file changed, 95 insertions(+)
> > create mode 100644
> > Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> >
> > diff --git
> > a/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> > b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yaml
> > new file mode 100644
> > index 000000000000..ef0452a47e98
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/toshiba,visconti5-viif.y
> > +++ aml
> > @@ -0,0 +1,95 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) %YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/media/toshiba,visconti5-viif.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Toshiba Visconti5 SoC Video Input Interface
> > +
> > +maintainers:
> > + - Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > +
> > +description: |-
> > + Toshiba Visconti5 SoC Video Input Interface (VIIF) receives
> > +videostream
> > + from MIPI CSI-2 receiver device, processes the stream with image
> > +signal
> > + processors (L1ISP, L2ISP), then stores pictures to main memory.
> > +
> > +properties:
> > + compatible:
> > + const: toshiba,visconti5-viif
> > +
> > + reg:
> > + items:
> > + - description: Registers for capture control
> > + - description: Registers for bus interface unit control
> > + - description: Registers for Memory Protection Unit
>
> I'm a bit surprised by the lack of clocks.
>
The VIIF HW requires clock input. However, since the clock controller driver is still in development and no corresponding clock source is available,
I did not describe the VIIF clock in the device tree and the binding.
I understand that a clock should be defined in the binding.
For the next patch, I'll add a clock definition to the binding.
> > +
> > + interrupts:
> > + items:
> > + - description: Sync Interrupt
> > + - description: Status (Error) Interrupt
> > + - description: L1ISP Interrupt
> > +
> > + port:
> > + $ref: /schemas/graph.yaml#/$defs/port-base
> > + unevaluatedProperties: false
> > + description: CSI-2 input port, with a single endpoint connected to the
> CSI-2 transmitter.
> > +
> > + properties:
> > + endpoint:
> > + $ref: video-interfaces.yaml#
> > + additionalProperties: false
> > +
> > + properties:
> > + data-lanes:
> > + description: VIIF supports 1, 2, 3 or 4 data lanes
> > + minItems: 1
> > + items:
> > + - const: 1
> > + - const: 2
> > + - const: 3
> > + - const: 4
>
> Now that the CSI-2 receiver is modeled as a separate DT node, I don't think
> data-lanes is applicable anymore. The interface between the CSI-2 receiver
> and the VIIF isn't a CSI-2 bus.
>
> I think you can simplify the bindings by switching from port-base to port, as you
> don't need to specify additional properties for the
> endpoint:
>
> port:
> $ref: /schemas/graph.yaml#/$defs/port
> description:
> CSI-2 input port, with a single endpoint connected to the CSI-2
> transmitter.
>
> Please test this though (by running the DT bindings checks).
>
I'll remove CSI-2 specific descriptions.
Also I'll use a reference to port.
> > +
> > + remote-endpoint: true
> > +
> > + required:
> > + - data-lanes
> > + - remote-endpoint
> > +
> > +required:
> > + - compatible
> > + - reg
> > + - interrupts
> > + - port
> > +
> > +additionalProperties: false
> > +
> > +examples:
> > + - |
> > + #include <dt-bindings/interrupt-controller/arm-gic.h>
> > + #include <dt-bindings/interrupt-controller/irq.h>
> > +
> > + soc {
> > + #address-cells = <2>;
> > + #size-cells = <2>;
> > +
> > + video@1c000000 {
> > + compatible = "toshiba,visconti5-viif";
> > + reg = <0 0x1c000000 0 0x6000>,
> > + <0 0x1c00e000 0 0x1000>,
> > + <0 0x2417a000 0 0x1000>;
> > + interrupts = <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>,
> > + <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
> > + <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
> > +
> > + port {
> > + #address-cells = <1>;
> > + #size-cells = <0>;
> > +
> > + csi_in0: endpoint {
> > + data-lanes = <1 2>;
> > + remote-endpoint = <&csi2rx_out0>;
> > + };
> > + };
> > + };
> > + };
>
> --
> Regards,
>
> Laurent Pinchart
Regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format
2025-01-02 13:10 ` Laurent Pinchart
@ 2025-01-20 0:15 ` yuji2.ishikawa
2025-01-20 1:49 ` Laurent Pinchart
0 siblings, 1 reply; 34+ messages in thread
From: yuji2.ishikawa @ 2025-01-20 0:15 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent.
Thank you for your review comments.
> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Thursday, January 2, 2025 10:10 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format
>
> Hi Ishikawa-san,
>
> Thank you for the patch.
>
>
> On Mon, Nov 25, 2024 at 06:21:41PM +0900, Yuji Ishikawa wrote:
> > Adds the Toshiba Visconti VIIF specific metadata format
>
> s/Adds/Add/
> s/format/formats./
>
I'll fix the commit message.
> >
> > - V4L2_META_FMT_VISCONTI_VIIF_PARAMS for ISP parameters
> > - V4L2_META_FMT_VISCONTI_VIIF_STATS for ISP statistics
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
>
> Assuming the documentation of the formats in subsequent patches is fine,
>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> > ---
> > Changelog v10:
> > - add entry for V4L2_META_FMT_VISCONTI_VIIF_PARAMS
> > - add entry for V4L2_META_FMT_VISCONTI_VIIF_STATS
> >
> > Changelog v11:
> > - no change
> >
> > Changelog v12:
> > - add description for meta formats at v4l2-ioctl.c
> >
> > drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
> > include/uapi/linux/videodev2.h | 4 ++++
> > 2 files changed, 6 insertions(+)
> >
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 0304daa8471d..f7facb63b8ea 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1470,6 +1470,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
> *fmt)
> > case V4L2_META_FMT_RPI_BE_CFG: descr = "RPi PiSP BE Config
> format"; break;
> > case V4L2_META_FMT_RPI_FE_CFG: descr = "RPi PiSP FE Config
> format"; break;
> > case V4L2_META_FMT_RPI_FE_STATS: descr = "RPi PiSP FE
> Statistics
> > format"; break;
> > + case V4L2_META_FMT_VISCONTI_VIIF_PARAMS: descr = "Visconti
> ISP Parameters"; break;
> > + case V4L2_META_FMT_VISCONTI_VIIF_STATS: descr = "Visconti ISP
> > +Statistics"; break;
The media-ci has reported the following errors.
Is it all right to leave these errors unfixed and keep the lines with the same style as other entries?
# Test checkpatch:./0003-media-uapi-add-visconti-viif-meta-buffer-format.patch
ERROR: trailing statements should be on next line
#26: FILE: drivers/media/v4l2-core/v4l2-ioctl.c:1473:
+case V4L2_META_FMT_VISCONTI_VIIF_PARAMS: descr = "Visconti ISP
+Parameters"; break;
ERROR: trailing statements should be on next line
#27: FILE: drivers/media/v4l2-core/v4l2-ioctl.c:1474:
+case V4L2_META_FMT_VISCONTI_VIIF_STATS: descr = "Visconti ISP
+Statistics"; break;
total: 2 errors, 0 warnings, 0 checks, 18 lines checked
> > case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic
> Metadata"; break;
> > case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8-bit
> Generic Meta, 10b CSI-2"; break;
> > case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8-bit
> Generic Meta, 12b CSI-2"; break;
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h index a5418759e2ba..9e1f66fdf038
> > 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -863,6 +863,10 @@ struct v4l2_pix_format {
> > #define V4L2_META_FMT_RPI_FE_CFG v4l2_fourcc('R', 'P', 'F', 'C') /*
> PiSP FE configuration */
> > #define V4L2_META_FMT_RPI_FE_STATS v4l2_fourcc('R', 'P', 'F', 'S') /*
> PiSP FE stats */
> >
> > +/* Vendor specific - used for Visconti VIIF sub-system */
> > +#define V4L2_META_FMT_VISCONTI_VIIF_PARAMS v4l2_fourcc('V', 'I', 'F',
> 'P') /* ISP Params */
> > +#define V4L2_META_FMT_VISCONTI_VIIF_STATS v4l2_fourcc('V', 'I', 'F',
> 'S') /* ISP Stats */
> > +
> > #ifdef __KERNEL__
> > /*
> > * Line-based metadata formats. Remember to update v4l_fill_fmtdesc()
> > when
>
> --
> Regards,
>
> Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver
2025-01-02 13:08 ` Laurent Pinchart
@ 2025-01-20 0:21 ` yuji2.ishikawa
0 siblings, 0 replies; 34+ messages in thread
From: yuji2.ishikawa @ 2025-01-20 0:21 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent.
Thank you for your review comments.
> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Thursday, January 2, 2025 10:08 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti
> CSI-2 Receiver driver
>
> Hello Ishikawa-san,
>
> Thank you for the patch.
>
> On Mon, Nov 25, 2024 at 06:21:42PM +0900, Yuji Ishikawa wrote:
> > Add support to MIPI CSI-2 Receiver on Toshiba Visconti ARM SoCs.
> > This driver is used with Visconti Video Input Interface driver.
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > ---
> > Changelog v12:
> > - Separate CSI2RX driver and made it independent driver
> > - viif_csi2rx subdevice driver (in v11 patch) was removed.
> > - dictionary order at Kconfig and Makefile
> >
> > drivers/media/platform/Kconfig | 1 +
> > drivers/media/platform/Makefile | 1 +
> > drivers/media/platform/toshiba/Kconfig | 6 +
> > drivers/media/platform/toshiba/Makefile | 2 +
> > .../media/platform/toshiba/visconti/Kconfig | 16 +
> > .../media/platform/toshiba/visconti/Makefile | 8 +
> > .../platform/toshiba/visconti/csi2rx_drv.c | 791
> ++++++++++++++++++
> > 7 files changed, 825 insertions(+)
> > create mode 100644 drivers/media/platform/toshiba/Kconfig
> > create mode 100644 drivers/media/platform/toshiba/Makefile
> > create mode 100644 drivers/media/platform/toshiba/visconti/Kconfig
> > create mode 100644 drivers/media/platform/toshiba/visconti/Makefile
> > create mode 100644
> > drivers/media/platform/toshiba/visconti/csi2rx_drv.c
> >
> > diff --git a/drivers/media/platform/Kconfig
> > b/drivers/media/platform/Kconfig index 85d2627776b6..761b15b07b90
> > 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -86,6 +86,7 @@ source "drivers/media/platform/samsung/Kconfig"
> > source "drivers/media/platform/st/Kconfig"
> > source "drivers/media/platform/sunxi/Kconfig"
> > source "drivers/media/platform/ti/Kconfig"
> > +source "drivers/media/platform/toshiba/Kconfig"
> > source "drivers/media/platform/verisilicon/Kconfig"
> > source "drivers/media/platform/via/Kconfig"
> > source "drivers/media/platform/xilinx/Kconfig"
> > diff --git a/drivers/media/platform/Makefile
> > b/drivers/media/platform/Makefile index ace4e34483dd..917145fe5171
> > 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -29,6 +29,7 @@ obj-y += samsung/
> > obj-y += st/
> > obj-y += sunxi/
> > obj-y += ti/
> > +obj-y += toshiba/
> > obj-y += verisilicon/
> > obj-y += via/
> > obj-y += xilinx/
> > diff --git a/drivers/media/platform/toshiba/Kconfig
> > b/drivers/media/platform/toshiba/Kconfig
> > new file mode 100644
> > index 000000000000..f02983f4fc97
> > --- /dev/null
> > +++ b/drivers/media/platform/toshiba/Kconfig
> > @@ -0,0 +1,6 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +
> > +comment "Toshiba media platform drivers"
> > +
> > +source "drivers/media/platform/toshiba/visconti/Kconfig"
> > +
> > diff --git a/drivers/media/platform/toshiba/Makefile
> > b/drivers/media/platform/toshiba/Makefile
> > new file mode 100644
> > index 000000000000..2bce85ef3b48
> > --- /dev/null
> > +++ b/drivers/media/platform/toshiba/Makefile
> > @@ -0,0 +1,2 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
>
> A blank line would be nice here.
>
I'll add a blank line.
> > +obj-y += visconti/
> > diff --git a/drivers/media/platform/toshiba/visconti/Kconfig
> > b/drivers/media/platform/toshiba/visconti/Kconfig
> > new file mode 100644
> > index 000000000000..e5c92d598f8b
> > --- /dev/null
> > +++ b/drivers/media/platform/toshiba/visconti/Kconfig
> > @@ -0,0 +1,16 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
>
> A blank line would be nice here too.
>
I'll add a blank line here too.
> > +config VIDEO_VISCONTI_CSI2RX
> > + tristate "Visconti MIPI CSI-2 Receiver driver"
> > + depends on V4L_PLATFORM_DRIVERS
> > + depends on VIDEO_DEV && OF
> > + depends on ARCH_VISCONTI || COMPILE_TEST
> > + select MEDIA_CONTROLLER
> > + select VIDEO_V4L2_SUBDEV_API
> > + select V4L2_FWNODE
> > + help
> > + Support for Toshiba Visconti MIPI CSI-2 receiver,
> > + which is used with Visconti Camera Interface driver.
> > +
> > + This driver yields 1 subdevice node for a hardware instance.
> > + To compile this driver as a module, choose M here: the
> > + module will be called visconti-csi2rx.
> > diff --git a/drivers/media/platform/toshiba/visconti/Makefile
> > b/drivers/media/platform/toshiba/visconti/Makefile
> > new file mode 100644
> > index 000000000000..62a029376134
> > --- /dev/null
> > +++ b/drivers/media/platform/toshiba/visconti/Makefile
> > @@ -0,0 +1,8 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# Makefile for the Visconti video input device driver #
> > +
> > +visconti-csi2rx-objs = csi2rx_drv.o
> > +
> > +obj-$(CONFIG_VIDEO_VISCONTI_CSI2RX) += visconti-csi2rx.o
> > diff --git a/drivers/media/platform/toshiba/visconti/csi2rx_drv.c
> > b/drivers/media/platform/toshiba/visconti/csi2rx_drv.c
> > new file mode 100644
> > index 000000000000..94567963872a
> > --- /dev/null
> > +++ b/drivers/media/platform/toshiba/visconti/csi2rx_drv.c
> > @@ -0,0 +1,791 @@
> > +// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
> > +/* Toshiba Visconti Video Capture Support
>
> /*
> * Toshiba Visconti Video Capture Support
> *
>
I'll fix it. Also I'll check for the similar cases.
> > + *
> > + * (C) Copyright 2024 TOSHIBA CORPORATION
> > + * (C) Copyright 2024 Toshiba Electronic Devices & Storage
> > + Corporation */
> > +
> > +#include <linux/delay.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/io.h>
> > +#include <linux/module.h>
> > +#include <linux/of.h>
> > +#include <linux/of_graph.h>
>
> You don't need those two headers. You however need to include
>
> - linux/device.h (for devm_kzalloc)
> - linux/property.h (for the fwnode_* API)
>
I'll update the include list.
> > +#include <linux/platform_device.h>
> > +
> > +#include <media/mipi-csi2.h>
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/v4l2-subdev.h>
> > +
> > +/* CSI2HOST register space */
> > +#define REG_CSI2RX_NLANES 0x4
> > +#define REG_CSI2RX_RESETN 0x8
> > +#define REG_CSI2RX_INT_ST_MAIN 0xc
> > +#define REG_CSI2RX_DATA_IDS_1 0x10
> > +#define REG_CSI2RX_DATA_IDS_2 0x14
> > +#define REG_CSI2RX_PHY_SHUTDOWNZ 0x40
> > +#define REG_CSI2RX_PHY_RSTZ 0x44
> > +
> > +/* access to dphy external registers */
>
> /* Access to dphy external registers */
>
> Same in other comments below.
>
I'll fix it. Also I'll check for the similar cases.
> > +#define REG_CSI2RX_PHY_TESTCTRL0 0x50
> > +#define BIT_TESTCTRL0_CLK_0 0
> > +#define BIT_TESTCTRL0_CLK_1 BIT(1)
> > +
> > +#define REG_CSI2RX_PHY_TESTCTRL1 0x54
> > +#define BIT_TESTCTRL1_ADDR BIT(16)
> > +#define MASK_TESTCTRL1_DIN 0xff
> > +#define MASK_TESTCTRL1_DOUT 0xff00
> > +
> > +#define REG_CSI2RX_INT_ST_PHY_FATAL 0xe0 #define
> > +REG_CSI2RX_INT_MSK_PHY_FATAL 0xe4
> > +#define MASK_PHY_FATAL_ALL 0x0000000f
> > +
> > +#define REG_CSI2RX_INT_ST_PKT_FATAL 0xf0 #define
> > +REG_CSI2RX_INT_MSK_PKT_FATAL 0xf4
> > +#define MASK_PKT_FATAL_ALL 0x0001000f
> > +
> > +#define REG_CSI2RX_INT_ST_FRAME_FATAL 0x100 #define
> > +REG_CSI2RX_INT_MSK_FRAME_FATAL 0x104
> > +#define MASK_FRAME_FATAL_ALL 0x000f0f0f
> > +
> > +#define REG_CSI2RX_INT_ST_PHY 0x110
> > +#define REG_CSI2RX_INT_MSK_PHY 0x114
> > +#define MASK_PHY_ERROR_ALL 0x000f000f
> > +
> > +#define REG_CSI2RX_INT_ST_PKT 0x120
> > +#define REG_CSI2RX_INT_MSK_PKT 0x124
> > +#define MASK_PKT_ERROR_ALL 0x000f000f
> > +
> > +#define REG_CSI2RX_INT_ST_LINE 0x130
> > +#define REG_CSI2RX_INT_MSK_LINE 0x134
> > +#define MASK_LINE_ERROR_ALL 0x00ff00ff
> > +
> > +/* DPHY register space */
> > +enum dphy_testcode {
> > + DIG_TESTCODE_EXT = 0,
> > + DIG_SYS_0 = 0x001,
> > + DIG_SYS_1 = 0x002,
> > + DIG_SYS_3 = 0x004,
> > + DIG_SYS_7 = 0x008,
> > + DIG_RX_STARTUP_OVR_2 = 0x0e2,
> > + DIG_RX_STARTUP_OVR_3 = 0x0e3,
> > + DIG_RX_STARTUP_OVR_4 = 0x0e4,
> > + DIG_RX_STARTUP_OVR_5 = 0x0e5,
> > + DIG_CB_2 = 0x1ac,
> > + DIG_TERM_CAL_0 = 0x220,
> > + DIG_TERM_CAL_1 = 0x221,
> > + DIG_TERM_CAL_2 = 0x222,
> > + DIG_CLKLANE_LANE_6 = 0x307,
> > + DIG_CLKLANE_OFFSET_CAL_0 = 0x39d,
> > + DIG_LANE0_OFFSET_CAL_0 = 0x59f,
> > + DIG_LANE0_DDL_0 = 0x5e0,
> > + DIG_LANE1_OFFSET_CAL_0 = 0x79f,
> > + DIG_LANE1_DDL_0 = 0x7e0,
> > + DIG_LANE2_OFFSET_CAL_0 = 0x99f,
> > + DIG_LANE2_DDL_0 = 0x9e0,
> > + DIG_LANE3_OFFSET_CAL_0 = 0xb9f,
> > + DIG_LANE3_DDL_0 = 0xbe0,
> > +};
> > +
> > +#define SYS_0_HSFREQRANGE_OVR BIT(5)
> > +#define SYS_3_NO_REXT BIT(4)
> > +#define SYS_7_RESERVED FIELD_PREP(0x1f, 0x0c)
> > +#define SYS_7_DESKEW_POL BIT(5)
> > +#define STARTUP_OVR_4_CNTVAL FIELD_PREP(0x70, 0x01)
> > +#define STARTUP_OVR_4_DDL_EN BIT(0)
> > +#define STARTUP_OVR_5_BYPASS BIT(0)
> > +#define CB_2_LPRX_BIAS BIT(6)
> > +#define CB_2_RESERVED FIELD_PREP(0x3f, 0x0b)
> > +#define CLKLANE_RXHS_PULL_LONG BIT(7)
> > +
> > +/* bit mask for calibration result registers */ #define
> > +MASK_TERM_CAL_ERR 0 #define MASK_TERM_CAL_DONE BIT(7)
> > +#define MASK_CLK_CAL_ERR BIT(4)
> > +#define MASK_CLK_CAL_DONE BIT(0)
> > +#define MASK_CAL_ERR BIT(2)
> > +#define MASK_CAL_DONE BIT(1)
> > +#define MASK_DDL_ERR BIT(1)
> > +#define MASK_DDL_DONE BIT(2)
> > +
> > +#define VISCONTI_CSI2RX_ERROR_MONITORS_NUM 8
> > +
> > +/**
> > + * struct visconti_csi2rx_line_err_target
> > + *
> > + * Virtual Channel and Data Type pair for CSI2RX line error monitor
> > + *
> > + * When 0 is set to dt, line error detection is disabled.
> > + *
> > + * @vc: Virtual Channel to monitor; Range 0..3
> > + * @dt: Data Type to monitor; Range 0, 0x10..0x3f */ struct
> > +visconti_csi2rx_line_err_target {
> > + u32 vc[VISCONTI_CSI2RX_ERROR_MONITORS_NUM];
> > + u32 dt[VISCONTI_CSI2RX_ERROR_MONITORS_NUM];
> > +};
> > +
> > +#define CSI2RX_MIN_DATA_RATE 80U
> > +#define CSI2RX_MAX_DATA_RATE 1500U
> > +
> > +#define VISCONTI_CSI2RX_PAD_SINK 0
> > +#define VISCONTI_CSI2RX_PAD_SRC 1
> > +#define VISCONTI_CSI2RX_PAD_NUM 2
> > +
> > +#define VISCONTI_CSI2RX_DEF_WIDTH 1920 #define
> > +VISCONTI_CSI2RX_DEF_HEIGHT 1080 #define
> VISCONTI_CSI2RX_MIN_WIDTH
> > +640 #define VISCONTI_CSI2RX_MAX_WIDTH 3840 #define
> > +VISCONTI_CSI2RX_MIN_HEIGHT 480 #define
> VISCONTI_CSI2RX_MAX_HEIGHT
> > +2160
> > +
> > +struct visconti_csi2rx {
> > + struct device *dev;
> > + void __iomem *base;
> > +
> > + struct v4l2_subdev subdev;
> > + struct media_pad pads[VISCONTI_CSI2RX_PAD_NUM];
> > + struct v4l2_async_notifier notifier;
> > + struct v4l2_subdev *remote;
> > + unsigned int remote_pad;
> > +
> > + unsigned int lanes;
> > +
> > + unsigned int irq;
> > +};
> > +
> > +static inline struct visconti_csi2rx *notifier_to_csi2(struct
> > +v4l2_async_notifier *n) {
> > + return container_of(n, struct visconti_csi2rx, notifier); }
> > +
> > +static inline struct visconti_csi2rx *sd_to_csi2(struct v4l2_subdev
> > +*sd) {
> > + return container_of(sd, struct visconti_csi2rx, subdev); }
> > +
> > +static inline void visconti_csi2rx_write(struct visconti_csi2rx
> > +*priv, u32 regid, u32 val)
>
> The media subsystem tries to limit line lengths to 80 columns, when it doesn't
> hinder readability. For instance here you could write
>
> static inline void visconti_csi2rx_write(struct visconti_csi2rx *priv,
> u32 regid, u32 val)
>
> If lines are just a few characters over 80 columns the need to break them is less
> than if they approach 100 columns. It's a "soft limit"
> policy to try and maximize readability.
>
I understand the policy.
I'll limit each line to around 80 columns.
> > +{
> > + writel(val, priv->base + regid);
> > +}
> > +
> > +static inline u32 visconti_csi2rx_read(struct visconti_csi2rx *priv,
> > +u32 regid) {
> > + return readl(priv->base + regid);
> > +}
> > +
> > +static inline void tick_testclk(struct visconti_csi2rx *priv)
>
> Don't inline this or the next function, compilers are smart enough to decide by
> themselves these days.
>
I'll remove inline specifier.
Do I also need to remove it from other multi-line functions?
> > +{
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0,
> BIT_TESTCTRL0_CLK_1);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0,
> > +BIT_TESTCTRL0_CLK_0); }
> > +
> > +static inline void set_dphy_addr(struct visconti_csi2rx *priv, u32
> > +test_mode) {
> > + /* select testcode Ex space with top 4bits of test_mode */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
> > + BIT_TESTCTRL1_ADDR |
> DIG_TESTCODE_EXT);
> > + tick_testclk(priv);
>
> If writing to REG_CSI2RX_PHY_TESTCTRL1 always needs to be followed by a
> call to tick_testclk(), I would create a visconti_csi2rx_dphy_write()
> function:
>
> static void visconti_csi2rx_dphy_write(struct visconti_csi2rx *priv, u32 data) {
> visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1, data);
> visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0,
> BIT_TESTCTRL0_CLK_1);
> visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0,
> BIT_TESTCTRL0_CLK_0); }
>
The writing to REG_CSI2RX_PHY_TESTCTRL1 is always followed by a call to tick_testclk(). This sequence can be merged.
I'll add visconti_csi2rx_dphy_write().
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
> > +FIELD_GET(0xf00, test_mode));
>
> And here
>
> visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
> FIELD_GET(0xf00, test_mode));
>
> There are many other places below that go over 80 columns.
>
I understand the policy.
I'll update the code not to go over 80 columns.
> > + tick_testclk(priv);
> > +
> > + /* set bottom 8bit of test_mode */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
> > + BIT_TESTCTRL1_ADDR | FIELD_GET(0xff,
> test_mode));
> > + tick_testclk(priv);
> > +}
> > +
> > +static void write_dphy_param(struct visconti_csi2rx *priv, u32
> > +test_mode, u8 test_in) {
> > + set_dphy_addr(priv, test_mode);
> > +
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL1,
> (u32)test_in);
>
> Is the (u8) explicit cast needed ?
>
There's no need for it. I'll remove the cast operator.
> > + tick_testclk(priv);
> > +}
> > +
> > +struct csi2rx_dphy_hs_info {
> > + u32 rate;
> > + u32 hsfreqrange;
> > + u32 osc_freq_target;
> > +};
> > +
> > +static const struct csi2rx_dphy_hs_info dphy_hs_info[] = {
> > + { 80, 0x0, 0x1cc }, { 85, 0x10, 0x1cc }, { 95, 0x20, 0x1cc }, { 105,
> 0x30, 0x1cc },
> > + { 115, 0x1, 0x1cc }, { 125, 0x11, 0x1cc }, { 135, 0x21, 0x1cc }, { 145,
> 0x31, 0x1cc },
> > + { 155, 0x2, 0x1cc }, { 165, 0x12, 0x1cc }, { 175, 0x22, 0x1cc }, { 185,
> 0x32, 0x1cc },
> > + { 198, 0x3, 0x1cc }, { 213, 0x13, 0x1cc }, { 228, 0x23, 0x1cc }, { 243,
> 0x33, 0x1cc },
> > + { 263, 0x4, 0x1cc }, { 288, 0x14, 0x1cc }, { 313, 0x25, 0x1cc }, { 338,
> 0x35, 0x1cc },
> > + { 375, 0x5, 0x1cc }, { 425, 0x16, 0x1cc }, { 475, 0x26, 0x1cc }, { 525,
> 0x37, 0x1cc },
> > + { 575, 0x7, 0x1cc }, { 625, 0x18, 0x1cc }, { 675, 0x28, 0x1cc }, { 725,
> 0x39, 0x1cc },
> > + { 775, 0x9, 0x1cc }, { 825, 0x19, 0x1cc }, { 875, 0x29, 0x1cc }, { 925,
> 0x3a, 0x1cc },
> > + { 975, 0xa, 0x1cc }, { 1025, 0x1a, 0x1cc }, { 1075, 0x2a, 0x1cc }, { 1125,
> 0x3b, 0x1cc },
> > + { 1175, 0xb, 0x1cc }, { 1225, 0x1b, 0x1cc }, { 1275, 0x2b, 0x1cc }, { 1325,
> 0x3c, 0x1cc },
> > + { 1375, 0xc, 0x1cc }, { 1425, 0x1c, 0x1cc }, { 1475, 0x2c, 0x1cc }
>
> osc_freq_target seems to always be 0x1cc, does it have to be stored in this
> table ?
>
I checked the document and found that osc_freq_target is always 0x1cc for tested cases.
I'll remove the member from the table and make it a constant value.
> > +};
> > +
> > +static void get_dphy_hs_transfer_info(u32 dphy_rate, u32
> > +*hsfreqrange, u32 *osc_freq_target) {
> > + unsigned int i;
> > +
> > + for (i = 1; i < ARRAY_SIZE(dphy_hs_info); i++) {
> > + if (dphy_rate < dphy_hs_info[i].rate) {
> > + *hsfreqrange = dphy_hs_info[i - 1].hsfreqrange;
> > + *osc_freq_target = dphy_hs_info[i -
> 1].osc_freq_target;
> > + return;
> > + }
> > + }
> > +
> > + /* not found; return the largest entry */
> > + *hsfreqrange = dphy_hs_info[ARRAY_SIZE(dphy_hs_info) -
> 1].hsfreqrange;
> > + *osc_freq_target = dphy_hs_info[ARRAY_SIZE(dphy_hs_info) -
> > +1].osc_freq_target; }
> > +
> > +static void visconti_csi2rx_set_dphy_rate(struct visconti_csi2rx
> > +*priv, u32 dphy_rate) {
> > + u32 hsfreqrange, osc_freq_target;
> > +
> > + get_dphy_hs_transfer_info(dphy_rate, &hsfreqrange,
> > +&osc_freq_target);
> > +
> > + write_dphy_param(priv, DIG_SYS_1, (u8)hsfreqrange);
>
> I don't think the explicit cast is needed here either.
>
I'll remove the cast operator.
> > + write_dphy_param(priv, DIG_SYS_0, SYS_0_HSFREQRANGE_OVR);
> > + write_dphy_param(priv, DIG_RX_STARTUP_OVR_5,
> STARTUP_OVR_5_BYPASS);
> > + write_dphy_param(priv, DIG_RX_STARTUP_OVR_4,
> STARTUP_OVR_4_CNTVAL);
> > + write_dphy_param(priv, DIG_CB_2, CB_2_LPRX_BIAS |
> CB_2_RESERVED);
> > + write_dphy_param(priv, DIG_SYS_7, SYS_7_DESKEW_POL |
> SYS_7_RESERVED);
> > + write_dphy_param(priv, DIG_CLKLANE_LANE_6,
> CLKLANE_RXHS_PULL_LONG);
> > + write_dphy_param(priv, DIG_RX_STARTUP_OVR_2, FIELD_GET(0xff,
> osc_freq_target));
> > + write_dphy_param(priv, DIG_RX_STARTUP_OVR_3, FIELD_GET(0xf00,
> osc_freq_target));
> > + write_dphy_param(priv, DIG_RX_STARTUP_OVR_4,
> STARTUP_OVR_4_CNTVAL |
> > +STARTUP_OVR_4_DDL_EN); }
> > +
> > +static int visconti_csi2rx_initialize(struct visconti_csi2rx *priv, u32
> num_lane, u32 dphy_rate,
> > + const struct
> visconti_csi2rx_line_err_target *err_target) {
> > + u32 val;
> > +
> > + if (dphy_rate < CSI2RX_MIN_DATA_RATE || dphy_rate >
> CSI2RX_MAX_DATA_RATE) {
> > + dev_err(priv->dev, "Unsupported PHY speed (%u Mbps)",
> dphy_rate);
> > + return -ERANGE;
> > + }
> > +
> > + /* 1st phase of initialization */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_RESETN, 1);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 1);
> > + ndelay(15U);
>
> I don't mind much, but the U suffix here and in many other places seems
> unneeded.
>
This suffix seems excessive. I'll remove U suffix in similar cases.
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 0);
> > +
> > + /* Configure D-PHY frequency range */
> > + visconti_csi2rx_set_dphy_rate(priv, dphy_rate);
> > +
> > + /* 2nd phase of initialization */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_NLANES, (num_lane - 1U));
>
> No need for the inner parentheses.
>
I'll remove them.
> > + ndelay(5U);
> > +
> > + /* Release D-PHY from Reset */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 1);
> > + ndelay(5U);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 1);
> > +
> > + /* configuration of line error target */
> > + val = (err_target->vc[3] << 30U) | (err_target->dt[3] << 24U) |
> (err_target->vc[2] << 22U) |
> > + (err_target->dt[2] << 16U) | (err_target->vc[1] << 14U) |
> (err_target->dt[1] << 8U) |
> > + (err_target->vc[0] << 6U) | (err_target->dt[0]);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_DATA_IDS_1, val);
> > + val = (err_target->vc[7] << 30U) | (err_target->dt[7] << 24U) |
> (err_target->vc[6] << 22U) |
> > + (err_target->dt[6] << 16U) | (err_target->vc[5] << 14U) |
> (err_target->dt[5] << 8U) |
> > + (err_target->vc[4] << 6U) | (err_target->dt[4]);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_DATA_IDS_2, val);
> > +
> > + /* configuration of mask */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY_FATAL,
> MASK_PHY_FATAL_ALL);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT_FATAL,
> MASK_PKT_FATAL_ALL);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL,
> MASK_FRAME_FATAL_ALL);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY,
> MASK_PHY_ERROR_ALL);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT,
> MASK_PKT_ERROR_ALL);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_LINE,
> > +MASK_LINE_ERROR_ALL);
> > +
> > + return 0;
> > +}
> > +
> > +struct visconti_csi2rx_format {
> > + u32 code;
> > + unsigned int bpp;
> > +};
> > +
> > +static const struct visconti_csi2rx_format visconti_csi2rx_formats[] = {
> > + { .code = MEDIA_BUS_FMT_RGB888_1X24, .bpp = 24 },
> > + { .code = MEDIA_BUS_FMT_UYVY8_1X16, .bpp = 16 },
> > + { .code = MEDIA_BUS_FMT_UYVY10_1X20, .bpp = 20 },
> > + { .code = MEDIA_BUS_FMT_RGB565_1X16, .bpp = 16 },
> > + { .code = MEDIA_BUS_FMT_SBGGR8_1X8, .bpp = 8 },
> > + { .code = MEDIA_BUS_FMT_SGBRG8_1X8, .bpp = 8 },
> > + { .code = MEDIA_BUS_FMT_SGRBG8_1X8, .bpp = 8 },
> > + { .code = MEDIA_BUS_FMT_SRGGB8_1X8, .bpp = 8 },
> > + { .code = MEDIA_BUS_FMT_SBGGR10_1X10, .bpp = 10 },
> > + { .code = MEDIA_BUS_FMT_SGBRG10_1X10, .bpp = 10 },
> > + { .code = MEDIA_BUS_FMT_SGRBG10_1X10, .bpp = 10 },
> > + { .code = MEDIA_BUS_FMT_SRGGB10_1X10, .bpp = 10 },
> > + { .code = MEDIA_BUS_FMT_SBGGR12_1X12, .bpp = 12 },
> > + { .code = MEDIA_BUS_FMT_SGBRG12_1X12, .bpp = 12 },
> > + { .code = MEDIA_BUS_FMT_SGRBG12_1X12, .bpp = 12 },
> > + { .code = MEDIA_BUS_FMT_SRGGB12_1X12, .bpp = 12 },
> > + { .code = MEDIA_BUS_FMT_SBGGR14_1X14, .bpp = 14 },
> > + { .code = MEDIA_BUS_FMT_SGBRG14_1X14, .bpp = 14 },
> > + { .code = MEDIA_BUS_FMT_SGRBG14_1X14, .bpp = 14 },
> > + { .code = MEDIA_BUS_FMT_SRGGB14_1X14, .bpp = 14 }, };
> > +
> > +static const struct visconti_csi2rx_format
> > +*fmt_for_mbus_code(unsigned int mbus_code) {
> > + int i;
>
> unsigned int
>
I'll fix it.
> > +
> > + for (i = 0; ARRAY_SIZE(visconti_csi2rx_formats); i++)
> > + if (visconti_csi2rx_formats[i].code == mbus_code)
> > + return &visconti_csi2rx_formats[i];
>
> Please use curly braces for the 'for' statement.
I'll add curly braces.
>
> No return when the look doesn't find a match ?
>
There should have been "return NULL" at the end of the function.
I'll fix it.
> > +}
> > +
> > +static unsigned int bpp_for_mbus_code(unsigned int mbus_code) {
> > + const struct visconti_csi2rx_format *fmt =
> > +fmt_for_mbus_code(mbus_code);
> > +
> > + return fmt ? fmt->bpp : 0;
> > +}
> > +
> > +static int64_t get_pixelclock(struct v4l2_subdev *sd) {
> > + struct v4l2_ctrl *ctrl;
> > +
> > + ctrl = v4l2_ctrl_find(sd->ctrl_handler, V4L2_CID_PIXEL_RATE);
> > + if (!ctrl)
> > + return -EINVAL;
> > +
> > + return v4l2_ctrl_g_ctrl_int64(ctrl); }
> > +
> > +static const struct visconti_csi2rx_line_err_target err_target_vc0_alldt = {
> > + /* select VC=0 */
> > + /* select all supported DataTypes */
> > + .dt = {
> > + MIPI_CSI2_DT_RGB565,
> > + MIPI_CSI2_DT_YUV422_8B,
> > + MIPI_CSI2_DT_YUV422_10B,
> > + MIPI_CSI2_DT_RGB888,
> > + MIPI_CSI2_DT_RAW8,
> > + MIPI_CSI2_DT_RAW10,
> > + MIPI_CSI2_DT_RAW12,
> > + MIPI_CSI2_DT_RAW14,
> > + }
> > +};
> > +
> > +static int visconti_csi2rx_start(struct visconti_csi2rx *priv, struct
> > +v4l2_subdev_state *state) {
> > + struct v4l2_mbus_framefmt *sink_fmt;
>
> const
>
I'll add a const qualifier.
> > + int cur_bpp, dphy_rate;
> > + s64 pixelclock;
> > +
> > + /* get bpp for current format */
> > + sink_fmt = v4l2_subdev_state_get_format(state,
> VISCONTI_CSI2RX_PAD_SINK);
> > + cur_bpp = bpp_for_mbus_code(sink_fmt->code);
>
> bpp = fmt_for_mbus_code(sink_fmt->code)->bpp;
>
> and drop the bpp_for_mbus_code() function.
>
I'll drop bpp_for_mbus_code().
Is NULL check for fmt_for_mbus_code(sink_fmt->code) required?
> > +
> > + /* get pixel clock */
> > + pixelclock = get_pixelclock(priv->remote);
>
> Use v4l2_get_link_freq() and drop the get_pixelclock() function.
>
I'll use v4l2_get_link_freq().
> > + if (pixelclock < 0)
> > + return -EINVAL;
> > +
> > + dphy_rate = div64_u64((u64)pixelclock * (u32)cur_bpp, priv->lanes *
> > +1000000);
> > +
> > + ndelay(15U);
> > +
> > + return visconti_csi2rx_initialize(priv, priv->lanes, dphy_rate,
> > +&err_target_vc0_alldt); }
> > +
> > +static void visconti_csi2rx_stop(struct visconti_csi2rx *priv) {
> > + /* disable interrupt -> make sure registers cleared -> wait for current
> handlers finish */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY_FATAL, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT_FATAL, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PHY, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_PKT, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_INT_MSK_LINE, 0);
> > + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PHY_FATAL);
> > + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PKT_FATAL);
> > + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_FRAME_FATAL);
> > + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PHY);
> > + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_PKT);
> > + visconti_csi2rx_read(priv, REG_CSI2RX_INT_MSK_LINE);
> > + synchronize_irq(priv->irq);
> > +
> > + /* shutdown hardware */
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_SHUTDOWNZ, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_RSTZ, 0);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_PHY_TESTCTRL0, 1);
> > + visconti_csi2rx_write(priv, REG_CSI2RX_RESETN, 0); }
> > +
> > +static int visconti_csi2rx_enable_streams(struct v4l2_subdev *sd, struct
> v4l2_subdev_state *state,
> > + u32 pad, u64 streams_mask)
> > +{
> > + struct visconti_csi2rx *priv = sd_to_csi2(sd);
> > + struct v4l2_subdev *remote_sd;
> > + struct media_pad *remote_pad;
> > + int ret;
> > +
> > + remote_pad =
> media_pad_remote_pad_first(&sd->entity.pads[VISCONTI_CSI2RX_PAD_SINK
> ]);
> > + if (!remote_pad)
> > + return -ENODEV;
>
> Can't you use priv->remote and priv->remote_pad in this function instead of
> getting the remote pad dynamically ? Same in
> visconti_csi2rx_disable_streams().
>
I'll use priv->remote, and priv->remote_pad.
> > + remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> > +
> > + /* enabling: turn on CSI2RX -> turn on sensor */
> > + ret = visconti_csi2rx_start(priv, state);
> > + if (ret)
> > + return ret;
> > +
> > + /* currently CSI2RX supports only stream0 in source pad */
> > + ret = v4l2_subdev_enable_streams(remote_sd, remote_pad->index,
> BIT(0));
> > + if (ret) {
> > + visconti_csi2rx_stop(priv);
> > + return ret;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int visconti_csi2rx_disable_streams(struct v4l2_subdev *sd, struct
> v4l2_subdev_state *state,
> > + u32 pad, u64 streams_mask)
> > +{
> > + struct visconti_csi2rx *priv = sd_to_csi2(sd);
> > + struct v4l2_subdev *remote_sd;
> > + struct media_pad *remote_pad;
> > +
> > + remote_pad =
> media_pad_remote_pad_first(&sd->entity.pads[VISCONTI_CSI2RX_PAD_SINK
> ]);
> > + if (!remote_pad)
> > + return -ENODEV;
> > + remote_sd = media_entity_to_v4l2_subdev(remote_pad->entity);
> > +
> > + /* disabling: turn off sensor -> turn off CSI2RX */
> > + v4l2_subdev_disable_streams(remote_sd, remote_pad->index,
> BIT(0));
> > + visconti_csi2rx_stop(priv);
> > +
> > + return 0;
> > +}
> > +
> > +static int visconti_csi2rx_enum_mbus_code(struct v4l2_subdev *sd,
> > + struct v4l2_subdev_state
> *sd_state,
> > + struct
> v4l2_subdev_mbus_code_enum *code) {
> > + if (code->pad == VISCONTI_CSI2RX_PAD_SRC) {
> > + const struct v4l2_mbus_framefmt *sink_fmt;
> > +
> > + /* SRC pad supports exactly the same format as SINK pad */
> > + if (code->index)
> > + return -EINVAL;
> > + sink_fmt = v4l2_subdev_state_get_format(sd_state,
> VISCONTI_CSI2RX_PAD_SINK);
> > + code->code = sink_fmt->code;
> > + return 0;
> > + }
> > +
> > + if (code->index >= ARRAY_SIZE(visconti_csi2rx_formats))
> > + return -EINVAL;
> > + code->code = visconti_csi2rx_formats[code->index].code;
> > +
> > + return 0;
> > +}
> > +
> > +static int visconti_csi2rx_init_state(struct v4l2_subdev *sd, struct
> > +v4l2_subdev_state *sd_state) {
> > + struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
> > +
> > + sink_fmt = v4l2_subdev_state_get_format(sd_state,
> VISCONTI_CSI2RX_PAD_SINK);
> > + src_fmt = v4l2_subdev_state_get_format(sd_state,
> > +VISCONTI_CSI2RX_PAD_SRC);
> > +
> > + sink_fmt->width = VISCONTI_CSI2RX_DEF_WIDTH;
> > + sink_fmt->height = VISCONTI_CSI2RX_DEF_HEIGHT;
> > + sink_fmt->field = V4L2_FIELD_NONE;
> > + sink_fmt->code = visconti_csi2rx_formats[0].code;
>
> Please also initialize the colourspace fields. V4L2_COLORSPACE_RAW,
> V4L2_XFER_FUNC_NONE, V4L2_YCBCR_ENC_601 and
> V4L2_QUANTIZATION_FULL_RANGE should be appropriate defaults.
>
I'll set the default value to colourspace fields.
> > +
> > + *src_fmt = *sink_fmt;
> > +
> > + return 0;
> > +}
> > +
> > +static int visconti_csi2rx_set_pad_format(struct v4l2_subdev *sd,
> > + struct v4l2_subdev_state
> *sd_state,
> > + struct v4l2_subdev_format *fmt) {
> > + struct v4l2_mbus_framefmt *sink_fmt, *src_fmt;
> > +
> > + /* SRC PAD has the same format as SINK PAD */
> > + if (fmt->pad == 1)
>
> if (fmt->pad == VISCONTI_CSI2RX_PAD_SRC)
>
I'll use the constant macro.
> > + return v4l2_subdev_get_fmt(sd, sd_state, fmt);
> > +
> > + sink_fmt = v4l2_subdev_state_get_format(sd_state,
> > +VISCONTI_CSI2RX_PAD_SINK);
> > +
> > + *sink_fmt = fmt->format;
> > + sink_fmt->width = clamp_t(u32, fmt->format.width,
> VISCONTI_CSI2RX_MIN_WIDTH,
> > + VISCONTI_CSI2RX_MAX_WIDTH);
> > + sink_fmt->height = clamp_t(u32, fmt->format.height,
> VISCONTI_CSI2RX_MIN_HEIGHT,
> > + VISCONTI_CSI2RX_MAX_HEIGHT);
> > + if (!fmt_for_mbus_code(sink_fmt->code))
> > + sink_fmt->code = visconti_csi2rx_formats[0].code;
> > + fmt->format = *sink_fmt;
> > +
> > + /* source pad should have the same format */
> > + src_fmt = v4l2_subdev_state_get_format(sd_state,
> VISCONTI_CSI2RX_PAD_SRC);
> > + *src_fmt = *sink_fmt;
> > +
> > + return 0;
> > +}
> > +
> > +static const struct media_entity_operations visconti_csi2rx_entity_ops = {
> > + .link_validate = v4l2_subdev_link_validate, };
> > +
> > +static const struct v4l2_subdev_video_ops visconti_csi2rx_video_ops = {
> > + .s_stream = v4l2_subdev_s_stream_helper,
>
> As the only driver that will use this CSI-2 RX is the VIIF driver, and that driver
> uses v4l2_subdev_enable_streams(), .s_stream() will never be called. You can
> drop the v4l2_subdev_video_ops.
>
I'll drop visconti_csi2rx_video_ops.
> > +};
> > +
> > +static const struct v4l2_subdev_pad_ops visconti_csi2rx_pad_ops = {
> > + .enum_mbus_code = visconti_csi2rx_enum_mbus_code,
>
> You also need to implement .enum_frame_size()
>
I'll implement it.
> > + .disable_streams = visconti_csi2rx_disable_streams,
> > + .enable_streams = visconti_csi2rx_enable_streams,
> > + .get_fmt = v4l2_subdev_get_fmt,
> > + .set_fmt = visconti_csi2rx_set_pad_format, };
> > +
> > +static const struct v4l2_subdev_ops visconti_csi2rx_subdev_ops = {
> > + .video = &visconti_csi2rx_video_ops,
> > + .pad = &visconti_csi2rx_pad_ops,
> > +};
> > +
> > +static const struct v4l2_subdev_internal_ops visconti_csi2rx_internal_ops =
> {
> > + .init_state = visconti_csi2rx_init_state, };
> > +
> > +static int visconti_csi2rx_notify_bound(struct v4l2_async_notifier *notifier,
> > + struct v4l2_subdev *subdev,
> > + struct v4l2_async_connection *asc) {
> > + struct visconti_csi2rx *priv = notifier_to_csi2(notifier);
> > + int pad;
> > +
> > + pad = media_entity_get_fwnode_pad(&subdev->entity,
> asc->match.fwnode, MEDIA_PAD_FL_SOURCE);
> > + if (pad < 0) {
> > + dev_err(priv->dev, "Failed to find pad for %s\n",
> subdev->name);
> > + return pad;
> > + }
> > +
> > + priv->remote = subdev;
> > + priv->remote_pad = pad;
> > +
> > + return media_create_pad_link(&subdev->entity, pad,
> &priv->subdev.entity, 0,
> > + MEDIA_LNK_FL_ENABLED);
>
> Can you have multiple sources connected to the same CSI-2 receiver ? If not,
> you can make the link to the source immutable.
>
Assuming that "sources" mean image sensors, the CSI-2 receiver does not support multiple sources.
I'll set MEDIA_LNK_FL_IMMUTABLE flag.
> > +}
> > +
> > +static void visconti_csi2rx_notify_unbind(struct v4l2_async_notifier
> *notifier,
> > + struct v4l2_subdev *subdev,
> > + struct v4l2_async_connection *asc)
> {
> > + struct visconti_csi2rx *priv = notifier_to_csi2(notifier);
> > +
> > + priv->remote = NULL;
> > +}
> > +
> > +static const struct v4l2_async_notifier_operations
> visconti_csi2rx_notify_ops = {
> > + .bound = visconti_csi2rx_notify_bound,
> > + .unbind = visconti_csi2rx_notify_unbind, };
> > +
> > +static int visconti_csi2rx_parse_v4l2(struct visconti_csi2rx *priv,
> > + struct v4l2_fwnode_endpoint *vep) {
> > + /* Only port 0 endpoint 0 is valid. */
> > + if (vep->base.port || vep->base.id)
> > + return -ENOTCONN;
>
> You call fwnode_graph_get_endpoint_by_id() with port and endpoint set to 0,
> so I think you can drop this check.
>
I'll drop the check.
> > +
> > + priv->lanes = vep->bus.mipi_csi2.num_data_lanes;
> > +
> > + /* got trouble */
> > + if (vep->bus_type != V4L2_MBUS_CSI2_DPHY) {
> > + dev_err(priv->dev, "Specified bus type is not supported\n");
>
> If you only support D-PHY, then set the bus_type to V4L2_MBUS_CSI2_DPHY
> instead of V4L2_MBUS_UNKNOWN in visconti_csi2rx_parse_dt().
> v4l2_fwnode_endpoint_parse() will return an error if the bus type is not D-PHY,
> and you can drop this check.
>
I'll update the query and drop this check.
> > + return -EINVAL;
> > + }
> > +
> > + if (priv->lanes != 1 && priv->lanes != 2 && priv->lanes != 4) {
> > + dev_err(priv->dev, "Unsupported number of data-lanes for
> D-PHY: %u\n", priv->lanes);
> > + return -EINVAL;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int visconti_csi2rx_parse_dt(struct visconti_csi2rx *priv) {
> > + struct v4l2_async_connection *asc;
> > + struct fwnode_handle *fwnode;
> > + struct fwnode_handle *ep;
> > + struct v4l2_fwnode_endpoint v4l2_ep = {
> > + .bus_type = V4L2_MBUS_UNKNOWN,
> > + };
> > + int ret;
> > +
> > + ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(priv->dev), 0, 0,
> 0);
> > + if (!ep) {
> > + dev_err(priv->dev, "Not connected to subdevice\n");
> > + return -EINVAL;
> > + }
> > +
> > + ret = v4l2_fwnode_endpoint_parse(ep, &v4l2_ep);
> > + if (ret) {
> > + dev_err(priv->dev, "Could not parse v4l2 endpoint\n");
> > + fwnode_handle_put(ep);
> > + return -EINVAL;
> > + }
> > +
> > + ret = visconti_csi2rx_parse_v4l2(priv, &v4l2_ep);
>
> I would inline what is left of that function in here.
>
I'll inline visconti_csi2rx_parse_v4l2()
> > + if (ret) {
> > + fwnode_handle_put(ep);
> > + return ret;
> > + }
> > +
> > + fwnode = fwnode_graph_get_remote_endpoint(ep);
> > + fwnode_handle_put(ep);
> > +
> > + v4l2_async_subdev_nf_init(&priv->notifier, &priv->subdev);
> > + priv->notifier.ops = &visconti_csi2rx_notify_ops;
> > +
> > + asc = v4l2_async_nf_add_fwnode(&priv->notifier, fwnode, struct
> v4l2_async_connection);
> > + fwnode_handle_put(fwnode);
> > + if (IS_ERR(asc))
> > + return PTR_ERR(asc);
> > +
> > + ret = v4l2_async_nf_register(&priv->notifier);
> > + if (ret)
> > + v4l2_async_nf_cleanup(&priv->notifier);
> > +
> > + return ret;
> > +}
> > +
> > +static irqreturn_t visconti_csi2rx_irq(int irq, void *dev_id) {
> > + struct visconti_csi2rx *priv = dev_id;
> > + u32 event;
> > +
> > + event = visconti_csi2rx_read(priv, REG_CSI2RX_INT_ST_MAIN);
> > + dev_err(priv->dev, "CSI2RX error 0x%x.\n", event);
>
> Should this be at least rate-limited ?
>
I'll use dev_err_ratelimited().
> > +
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static const struct of_device_id visconti_csi2rx_of_table[] = {
> > + {
> > + .compatible = "toshiba,visconti5-csi2rx",
> > + },
> > + {},
>
> { /* Sentinel */ },
>
> is customary. You can also drop the trailing comma, as there should never be
> any entry after this one.
>
I'll update the initializer.
> > +};
> > +
> > +static int visconti_csi2rx_probe(struct platform_device *pdev) {
> > + struct visconti_csi2rx *priv;
> > + int irq, ret;
> > +
> > + priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
> > + if (!priv)
> > + return -ENOMEM;
> > +
> > + priv->dev = &pdev->dev;
> > +
> > + priv->base = devm_platform_ioremap_resource(pdev, 0);
> > + if (IS_ERR(priv->base)) {
> > + dev_err(priv->dev, "Failed to get registers\n");
>
> devm_platform_ioremap_resource() prints an error message already, you can
> drop this one.
>
I'll drop the message output.
> > + return PTR_ERR(priv->base);
> > + }
> > +
> > + irq = platform_get_irq(pdev, 0);
> > + if (irq < 0)
> > + return irq;
>
> Here, on the other hand, an error message would be useful. You can use
> dev_err_probe():
>
> return dev_err_probe(priv->dev, irq, "Failed to get IRQ\n");
>
I'll keep it.
> > + ret = devm_request_irq(&pdev->dev, irq, visconti_csi2rx_irq, 0,
> KBUILD_MODNAME, priv);
> > + priv->irq = irq;
> > + if (ret) {
> > + dev_err(priv->dev, "request irq failed: %d\n", ret);
> > + return ret;
> > + }
> > +
> > + platform_set_drvdata(pdev, priv);
> > +
> > + ret = visconti_csi2rx_parse_dt(priv); /*this function does
> v4l2_async_nf_register */
> > + if (ret)
> > + return ret;
> > +
> > + priv->subdev.owner = THIS_MODULE;
>
> Not needed, this is handled by v4l2_async_register_subdev()
>
I'll drop it.
> > + priv->subdev.dev = &pdev->dev;
> > + v4l2_subdev_init(&priv->subdev, &visconti_csi2rx_subdev_ops);
> > + v4l2_set_subdevdata(&priv->subdev, &pdev->dev);
> > + snprintf(priv->subdev.name, sizeof(priv->subdev.name), "%s %s",
> KBUILD_MODNAME,
> > + dev_name(&pdev->dev));
> > +
> > + priv->subdev.internal_ops = &visconti_csi2rx_internal_ops;
> > + priv->subdev.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> > + priv->subdev.entity.function =
> > +MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER;
>
> MEDIA_ENT_F_VID_IF_BRIDGE seems more appropriate.
>
I'll use MEDIA_ENT_F_VID_IF_BRIDGE.
> > + priv->subdev.entity.ops = &visconti_csi2rx_entity_ops;
> > +
> > + priv->pads[VISCONTI_CSI2RX_PAD_SINK].flags =
> MEDIA_PAD_FL_SINK;
> > + priv->pads[VISCONTI_CSI2RX_PAD_SRC].flags =
> MEDIA_PAD_FL_SOURCE;
> > +
> > + ret = media_entity_pads_init(&priv->subdev.entity,
> > +VISCONTI_CSI2RX_PAD_NUM, priv->pads);
>
> ret = media_entity_pads_init(&priv->subdev.entity,
> ARRAY_SIZE(priv->pads),
> priv->pads);
>
>
I'll use ARRAY_SIZE.
> > + if (ret)
> > + goto err_cleanup_async;
> > +
> > + ret = v4l2_subdev_init_finalize(&priv->subdev);
> > + if (ret)
> > + goto err_cleanup_media_entity;
> > +
> > + ret = v4l2_async_register_subdev(&priv->subdev);
> > + if (ret < 0)
> > + goto err_cleanup_subdev_state;
> > +
> > + return 0;
> > +
> > +err_cleanup_subdev_state:
> > + v4l2_subdev_cleanup(&priv->subdev);
> > +
> > +err_cleanup_media_entity:
> > + media_entity_cleanup(&priv->subdev.entity);
> > +
> > +err_cleanup_async:
> > + v4l2_async_nf_unregister(&priv->notifier);
> > + v4l2_async_nf_cleanup(&priv->notifier);
> > +
> > + return ret;
> > +}
> > +
> > +static void visconti_csi2rx_remove(struct platform_device *pdev) {
> > + struct visconti_csi2rx *priv = platform_get_drvdata(pdev);
> > +
> > + v4l2_async_nf_unregister(&priv->notifier);
> > + v4l2_async_nf_cleanup(&priv->notifier);
> > + v4l2_async_unregister_subdev(&priv->subdev);
> > +
> > + v4l2_subdev_cleanup(&priv->subdev);
> > + media_entity_cleanup(&priv->subdev.entity);
> > +}
> > +
> > +static struct platform_driver visconti_csi2rx_driver = {
> > + .probe = visconti_csi2rx_probe,
> > + .remove = visconti_csi2rx_remove,
> > + .driver = {
> > + .name = "visconti_csi2rx_dev",
> > + .of_match_table = visconti_csi2rx_of_table,
> > + },
> > +};
> > +
> > +module_platform_driver(visconti_csi2rx_driver);
> > +
> > +MODULE_AUTHOR("Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>");
> > +MODULE_DESCRIPTION("Toshiba Visconti CSI-2 receiver driver");
> > +MODULE_LICENSE("Dual BSD/GPL");
>
> --
> Regards,
>
> Laurent Pinchart
Regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format
2025-01-20 0:15 ` yuji2.ishikawa
@ 2025-01-20 1:49 ` Laurent Pinchart
0 siblings, 0 replies; 34+ messages in thread
From: Laurent Pinchart @ 2025-01-20 1:49 UTC (permalink / raw)
To: yuji2.ishikawa
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
On Mon, Jan 20, 2025 at 12:15:43AM +0000, yuji2.ishikawa@toshiba.co.jp wrote:
> On Thursday, January 2, 2025 10:10 PM, Laurent Pinchart wrote:
> > On Mon, Nov 25, 2024 at 06:21:41PM +0900, Yuji Ishikawa wrote:
> > > Adds the Toshiba Visconti VIIF specific metadata format
> >
> > s/Adds/Add/
> > s/format/formats./
>
> I'll fix the commit message.
>
> > > - V4L2_META_FMT_VISCONTI_VIIF_PARAMS for ISP parameters
> > > - V4L2_META_FMT_VISCONTI_VIIF_STATS for ISP statistics
> > >
> > > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> >
> > Assuming the documentation of the formats in subsequent patches is fine,
> >
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >
> > > ---
> > > Changelog v10:
> > > - add entry for V4L2_META_FMT_VISCONTI_VIIF_PARAMS
> > > - add entry for V4L2_META_FMT_VISCONTI_VIIF_STATS
> > >
> > > Changelog v11:
> > > - no change
> > >
> > > Changelog v12:
> > > - add description for meta formats at v4l2-ioctl.c
> > >
> > > drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
> > > include/uapi/linux/videodev2.h | 4 ++++
> > > 2 files changed, 6 insertions(+)
> > >
> > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > index 0304daa8471d..f7facb63b8ea 100644
> > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > @@ -1470,6 +1470,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > > case V4L2_META_FMT_RPI_BE_CFG: descr = "RPi PiSP BE Config format"; break;
> > > case V4L2_META_FMT_RPI_FE_CFG: descr = "RPi PiSP FE Config format"; break;
> > > case V4L2_META_FMT_RPI_FE_STATS: descr = "RPi PiSP FE Statistics format"; break;
> > > + case V4L2_META_FMT_VISCONTI_VIIF_PARAMS: descr = "Visconti ISP Parameters"; break;
> > > + case V4L2_META_FMT_VISCONTI_VIIF_STATS: descr = "Visconti ISP Statistics"; break;
>
> The media-ci has reported the following errors.
> Is it all right to leave these errors unfixed and keep the lines with
> the same style as other entries?
Yes, you can ignore those issues.
> # Test checkpatch:./0003-media-uapi-add-visconti-viif-meta-buffer-format.patch
> ERROR: trailing statements should be on next line
> #26: FILE: drivers/media/v4l2-core/v4l2-ioctl.c:1473:
> +case V4L2_META_FMT_VISCONTI_VIIF_PARAMS: descr = "Visconti ISP
> +Parameters"; break;
>
> ERROR: trailing statements should be on next line
> #27: FILE: drivers/media/v4l2-core/v4l2-ioctl.c:1474:
> +case V4L2_META_FMT_VISCONTI_VIIF_STATS: descr = "Visconti ISP
> +Statistics"; break;
>
> total: 2 errors, 0 warnings, 0 checks, 18 lines checked
>
> > > case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break;
> > > case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8-bit Generic Meta, 10b CSI-2"; break;
> > > case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8-bit Generic Meta, 12b CSI-2"; break;
> > > diff --git a/include/uapi/linux/videodev2.h
> > > b/include/uapi/linux/videodev2.h index a5418759e2ba..9e1f66fdf038
> > > 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -863,6 +863,10 @@ struct v4l2_pix_format {
> > > #define V4L2_META_FMT_RPI_FE_CFG v4l2_fourcc('R', 'P', 'F', 'C') /* PiSP FE configuration */
> > > #define V4L2_META_FMT_RPI_FE_STATS v4l2_fourcc('R', 'P', 'F', 'S') /* PiSP FE stats */
> > >
> > > +/* Vendor specific - used for Visconti VIIF sub-system */
> > > +#define V4L2_META_FMT_VISCONTI_VIIF_PARAMS v4l2_fourcc('V', 'I', 'F', 'P') /* ISP Params */
> > > +#define V4L2_META_FMT_VISCONTI_VIIF_STATS v4l2_fourcc('V', 'I', 'F', 'S') /* ISP Stats */
> > > +
> > > #ifdef __KERNEL__
> > > /*
> > > * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver
2025-01-02 21:26 ` Laurent Pinchart
@ 2025-02-05 12:29 ` yuji2.ishikawa
2025-04-15 9:01 ` yuji2.ishikawa
0 siblings, 1 reply; 34+ messages in thread
From: yuji2.ishikawa @ 2025-02-05 12:29 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent
Thank you for your review comments.
> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Friday, January 3, 2025 6:26 AM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 7/8] documentation: media: add documentation for
> Toshiba Visconti Video Input Interface driver
>
> Hi Ishikawa-san,
>
> Thank you for the patch.
>
> Overall the documentation looks quite good, it has significantly improved
> compared to early versions.
>
> On Mon, Nov 25, 2024 at 06:21:45PM +0900, Yuji Ishikawa wrote:
> > Added description of Video Input Interface driver of
>
> s/Added/Add/
>
I'll fix it
> > Toshiba Visconti architecture.
> > It includes hardware organization, structure of the driver and
> > metadata format for embedded image signal processor.
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > ---
> > Changelog v3:
> > - Newly add documentation to describe SW and HW
> >
> > Changelog v4:
> > - no change
> >
> > Changelog v5:
> > - no change
> >
> > Changelog v6:
> > - add description of CSI2RX subdevice
> > - add ordering of ioctl(S_FMT) and ioctl(S_EXT_CTRLS)
> >
> > Changelog v7:
> > - no change
> >
> > Changelog v8:
> > - add usage of V4L2_CTRL_TYPE_VISCONTI_ISP
> >
> > Changelog v9:
> > - fix warning: set reference target for keyword
> > V4L2_CTRL_TYPE_VISCONTI_ISP
> >
> > Changelog v10:
> > - use parameter buffers instead of compound control
> > - removed description of vendor specific compound control
> > - add description of parameter buffers for ISP control
> > - update directory structure
> > - remove documents under driver-api
> > - add documents to admin-guide, userspace-api
> >
> > Changelog v11:
> > - update usage of the driver
> >
> > Changelog v12:
> > - add description of CSI2RX driver
> > - description of resizer subdevice
> > - add block diagrams of VIIF and ISP
> > - update usage of the driver
> >
> > .../admin-guide/media/v4l-drivers.rst | 1 +
> > .../admin-guide/media/visconti-viif.dot | 22 +
> > .../admin-guide/media/visconti-viif.rst | 435
> ++++++++++++++++++
> > .../userspace-api/media/v4l/meta-formats.rst | 1 +
> > .../media/v4l/metafmt-visconti-viif.rst | 48 ++
> > 5 files changed, 507 insertions(+)
> > create mode 100644 Documentation/admin-guide/media/visconti-viif.dot
> > create mode 100644 Documentation/admin-guide/media/visconti-viif.rst
> > create mode 100644
> > Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> >
> > diff --git a/Documentation/admin-guide/media/v4l-drivers.rst
> > b/Documentation/admin-guide/media/v4l-drivers.rst
> > index b6af448b9fe9..81054512e768 100644
> > --- a/Documentation/admin-guide/media/v4l-drivers.rst
> > +++ b/Documentation/admin-guide/media/v4l-drivers.rst
> > @@ -32,5 +32,6 @@ Video4Linux (V4L) driver-specific documentation
> > si476x
> > starfive_camss
> > vimc
> > + visconti-viif
> > visl
> > vivid
> > diff --git a/Documentation/admin-guide/media/visconti-viif.dot
> > b/Documentation/admin-guide/media/visconti-viif.dot
> > new file mode 100644
> > index 000000000000..cc75c73336fb
> > --- /dev/null
> > +++ b/Documentation/admin-guide/media/visconti-viif.dot
> > @@ -0,0 +1,22 @@
> > +digraph board {
> > + rankdir=TB
> > + n00000001 [label="{{<port0> 0 | <port4> 4} |
> visconti-viif:isp\n/dev/v4l-subdev0 | {<port1> 1 | <port2> 2 | <port3> 3 |
> <port5> 5}}", shape=Mrecord, style=filled, fillcolor=green]
> > + n00000001:port1 -> n00000008:port0
> > + n00000001:port2 -> n0000000b:port0
> > + n00000001:port3 -> n00000016
> > + n00000001:port5 -> n0000001e
> > + n00000008 [label="{{<port0> 0} |
> visconti-viif:resizer0\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord,
> style=filled, fillcolor=green]
> > + n00000008:port1 -> n0000000e
> > + n0000000b [label="{{<port0> 0} |
> visconti-viif:resizer1\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord,
> style=filled, fillcolor=green]
> > + n0000000b:port1 -> n00000012
> > + n0000000e [label="viif_capture_post0\n/dev/video0", shape=box,
> style=filled, fillcolor=yellow]
> > + n00000012 [label="viif_capture_post1\n/dev/video1", shape=box,
> style=filled, fillcolor=yellow]
> > + n00000016 [label="viif_capture_sub\n/dev/video2", shape=box,
> style=filled, fillcolor=yellow]
> > + n0000001a [label="viif_params\n/dev/video3", shape=box,
> style=filled, fillcolor=yellow]
> > + n0000001a -> n00000001:port4
> > + n0000001e [label="viif_stats\n/dev/video4", shape=box,
> style=filled, fillcolor=yellow]
> > + n00000030 [label="{{<port0> 0} | visconti_csi2rx
> 1c008000.csi2rx\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord,
> style=filled, fillcolor=green]
> > + n00000030:port1 -> n00000001:port0
> > + n00000035 [label="{{} | imx219 1-0010\n/dev/v4l-subdev4 |
> {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> > + n00000035:port0 -> n00000030:port0 }
> > diff --git a/Documentation/admin-guide/media/visconti-viif.rst
> > b/Documentation/admin-guide/media/visconti-viif.rst
> > new file mode 100644
> > index 000000000000..c2e85fb6f8c1
> > --- /dev/null
> > +++ b/Documentation/admin-guide/media/visconti-viif.rst
> > @@ -0,0 +1,435 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> >
> +================================================
> ======
> > +Visconti Video Input Interface Driver (visconti-viif)
> >
> +================================================
> ======
> > +
> > +Introduction
> > +============
> > +
> > +This file documents the driver for the Video Input Interface (VIIF)
> > +that is part of Toshiba Visconti SoCs.
> > +The driver is located under drivers/media/platform/toshiba/visconti
> > +and uses the Media-Controller API.
>
> If you intend to make this two paragraphs, you should have a blank line
> in-between:
>
> --------
> This file documents the driver for the Video Input Interface (VIIF) that is part of
> Toshiba Visconti SoCs.
>
> The driver is located under drivers/media/platform/toshiba/visconti and uses
> the Media-Controller API.
> --------
>
> If you intend to make this a single paragraph, there should be no line break
> after the first sentence:
>
> --------
> This file documents the driver for the Video Input Interface (VIIF) that is part of
> Toshiba Visconti SoCs. The driver is located under
> drivers/media/platform/toshiba/visconti and uses the Media-Controller API.
> --------
>
I meant that the introduction has a single paragraph.
I'll update the paragraph's style.
> > +
> > +The driver module is named visconti-viif, and is enabled through the
> > +CONFIG_VIDEO_VISCONTI_VIIF config option.
> > +The CSI-2 receiver part is controlled by another module named
> > +visconti-csi2rx, which is enabled through the
> CONFIG_VIDEO_VISCONTI_CSI2RX config option.
>
> Please reflow text to reach as close to 80 columns:
>
> --------
> The driver module is named visconti-viif, and is enabled through the
> CONFIG_VIDEO_VISCONTI_VIIF config option. The CSI-2 receiver part is
> controlled by another module named visconti-csi2rx, which is enabled through
> the CONFIG_VIDEO_VISCONTI_CSI2RX config option.
> --------
>
> Same comment below where applicable.
>
I'll fix it.
> > +
> > +The Visconti VIIF Hardware
> > +==========================
> > +
> > +The Visconti VIIF hardware is an internally developed video capture device.
> > +Following function modules are integrated:
> > +
> > +* MIPI CSI-2 receiver (CSI2RX)
> > +* L1 Image Signal Processor (L1ISP)
> > +
> > + * Correction, enhancement, adjustment on bayer images.
> > +
> > +* L2 Image Signal Processor (L2ISP)
> > +
> > + * Lens distortion correction
> > + * Scaling & Cropping with up to 2 parameter sets
> > + * Formatting picture (RGB, YUV, Grayscale, ...)
> > + * Integrated DMAC: Writing picture into main memory
> > +
> > +* Video DMAC
> > +
> > + * Writing picture into main memory
> > +
> > +Visconti5 SoC has two VIIF hardware instances.
> > +
> > +
> > +The hardware block diagram is shown below.::
> > +
> > + The VIIF hardware
> > +
> "POST0"
> > + "RGB
> with scale 0"
> > + +--------+ +----------+ +-----+ +-----+ +-----+
> +--------+
> > + | Sensor |--->| CSI2RX |--->| | | | | |--->|
> memory |
> > + +--------+ +----------+ | | | | | |
> +--------+
> > + | | | L1 | | L2 | "POST1"
> > + | |--->| ISP |--->| ISP | "RGB with
> scale 1"
> > + | | | | | |
> +--------+
> > + | MUX | | | | |--->|
> memory |
> > + | | +-----+ +-----+
> +--------+
> > + | | "SUB"
> > + | | "RAW
> w/o scale"
> > + | | +------------+
> +--------+
> > + | |------> | Video DMAC |--->|
> memory |
> > + +-----+ +------------+
> +--------+
> > +
> > +Topology
> > +========
> > +
> > +Graph
> > +-----
> > +
> > +.. _visconti_viif_topology_graph:
> > +
> > +.. kernel-figure:: visconti-viif.dot
> > + :alt: Diagram of the default media pipeline topology
> > + :align: center
> > +
> > +The driver has 3 video devices for capturing images:
> > +
> > +- viif_capture_post0: capture device for image.
> > + - corresponds to L2ISP.
> > +- viif_capture_post1: capture device for image.
> > + - corresponds to L2ISP.
> > +- viif_capture_sub: capture device for bayer image.
> > + - corresponds to Video DMAC.
> > +
> > +The driver has 2 video devices for controlling ISP.
> > +
> > +- viif_params: a metadata output device that receives ISP parameters.
> > + - corresponds to L1ISP and L2ISP.
> > +- viif_stats: a metadata capture device that sends statistics.
> > + - corresponds to L1ISP and L2ISP.
> > +
> > +The driver has 2 subdevices:
>
> I count three subdevs in the list below (and there are actually two resizer
> instances, so that would be 4 subdevs).
>
The number should have been updated.
> > +
> > +- visconti_csi2rx: CSI-2 receiver operation.
> > + - corresponds to CSI2RX.
> > +- visconti-viif:isp: Image Signal Processor operation.
> > + - corresponds to L1ISP and L2ISP.
> > +- visconti-viif:resizer: Scaling operation of Image Signal Processor.
> > + - corresponds to L2ISP.
> > +
> > +visconti_csi2rx - CSI2 Receiver Subdevice Node
> > +---------------------------------------------------
>
> The title underline should have the same length as the title.
>
I'll fix it.
> > +
> > +This subdevice node corresponds to a MIPI CSI2 receiver.
> > +It resides between an image sensor subdevice and the ISP subdevice.
> > +It controls CSI2 link configuration and training process.
> > +
> > +visconti-viif:isp - ISP Subdevice Node
> > +--------------------------------------
> > +
> > +This subdevice node corresponds to L1/L2 ISPs.
> > +It receives pictures from an sensor (via CSI2RX),
>
> s/an sensor/a sensor/
>
I'll check the use of articles in comments/documents again.
> > +applies multiple operations on pictures, then passes resulting images to
> capture nodes.
> > +
> > +ISP configurations/parameters are passed from userland via viif_params
> node.
> > +The status of ISP operations are passed to userland via viif_stats node.
>
> "stats" stands for "statistics", so I would write "The statistics computed by the
> ISP are passed to ...".
>
> And after reading more of this document, I realize that the viif_stats node
> captures both statistics and status information. You could write "The statistics
> computed by the ISP and the frame processing status are passed to ...".
>
Yes, the node captures both statistics and status information.
I'll update the sentence according to your suggestion.
> > +
> > +L1 ISP provides following operations:
> > +
> > +- Input: accepts 8, 10, 12, 14bit bayer format
> > + - Operation selector; :c:type:`viif_l1_input_mode_config`
> > + - HDR image / PWL (Piecewse Linear Compression) image
> > + - with preprocessing / without preprocessing
> > + - HDRE: HDR expansion (only for PWL image);
> > + see :c:type:`viif_l1_hdre_config`
> > +- Preprocessing: generate intermediate data (24bit RAW)
> > + - SLIC: Bit slicing (x3 12bit planes for preprocessing);
> > + see :c:type:`viif_l1_img_extraction_config`
> > + - ABPC/DPC: Blemish/Defect pixel
> correction :c:type:`viif_l1_dpc_config`
> > + - PWHB: Preset white balance;
> see :c:type:`viif_l1_preset_white_balance_config`
> > + - RCNR: RAW color noise reduction;
> see :c:type:`viif_l1_raw_color_noise_reduction_config`
> > + - HDRS: HDR synthesis; see :c:type:`viif_l1_hdrs_config`
> > +- Processing on RAW image: Main Process (MPRO)
> > + - BLVC: black level correction and normalization;
> > + see :c:type:`viif_l1_black_level_correction_config`
> > + - LSSC: Lens shading correction; see :c:type:`viif_l1_lsc_config`
> > + - MPRO: digital amplifier; see :c:type:`viif_l1_main_process_config`
> > + - MPRO: bayer demosaicing; see :c:type:`viif_l1_main_process_config`
> > + - MPRO: color matrix correction;
> see :c:type:`viif_l1_main_process_config`
> > + - HDRC: HDR compression;
>
> If my understanding is correct, this implements global and local tone mapping. I
> would mention it explicitly here, those terms are more common than "HDR
> compression". You could write for instance
>
> - HDRC: HDR compression (global and local tone mapping);
>
As you pointed out, VIIF provides global and local tone mapping.
I'll update the sentence according to your suggestion.
> > + see :c:type:`viif_l1_hdrc_config`, :c:type:`viif_l1_hdrc_ltm_config`,
> > + :c:type:`viif_l1_rgb_to_y_coef_config`
> > +- Processing on RGB/YUV image: Video Process (VPRO)
> > + - VPRO: gamma correction; see :c:type:`viif_l1_gamma_config`
> > + - VPRO: RGB2YUV;
> > + see :c:type:`viif_l1_rgb_to_y_coef_config`,
> > + :c:type:`viif_l1_img_quality_adjustment_config`
> > + - VPRO: image quality adjustment; see
> > +:c:type:`viif_l1_img_quality_adjustment_config`
> > +- Output: 16bit YUV
> > +- Feedback loop
> > + - AWHB: auto white balance; see :c:type:`viif_l1_awb_config`,
> > + :c:type:`viif_isp_capture_status`
>
> Does this mean that the ISP can compute white balance gains automatically,
> implementing AWB in hardware ? That's interesting, it's a feature I haven't seen
> before.
>
Sorry for the lack of explanation.
The AWB HW calculates the average values of U and V for the specified area of
the input image and adjust R-gain and B-gain so that the difference of the two
values approaches to zero. The software specifies the configurations through
viif_l1_awb_config but does not intervene in the processing of each frame.
The software can know the final gain obtained, the average values of U and V,
and whether the calculation has converged.
It is also possible to calculate the average value in hardware, then compute
the gain in software, and set the gain in the register. However, this is not
recommended due to significant control delay.
> > + - AEXP: auto exposure (average luminance calculation);
> > + see :c:type:`viif_l1_avg_lum_generation_config`,
> >
> + :c:type:`viif_l1_rgb_to_y_coef_config`, :c:type:`viif_isp_capture_status
> `
> > + - AG: analog gain calculation;
> > + see :c:type:`viif_l1_ag_mode_config`,
> > + :c:type:`viif_l1_ag_config`
> > +
> > +Below is the block diagram::
> > +
> > + L1ISP::INPUT
> > +
> > + +--------+ +-----+ +-----+
> > + | Input |--------------->| |--------------------->| |
> > + | 24bHDR | | | | |
> > + +--------+ | 24b | | 24b |
> > + | RAW | | RAW |
> > + +--------+ +------+ | (0) | | (1) |
> > + | Input |--->| HDRE |--->| | +------------+ | |
> > + | 24bPWL | | | | |--->| preprocess |--->| |
> > + +--------+ +------+ +-----+ +------------+ +-----+
>
> I'm not entirely sure to understand this correctly, could you please correct me
> where I'm mistaken ?
>
> I understand that the PWL input is HDR contents merged/stitched on the
> sensor side, and compressed to a smaller than 24bpp width to reduce bus
> bandwidth using a PWL function. The HDRE block applies the inverse function
> to recover linear 24 bit data. In that case, the "preprocess"
> pipeline operates on a single channel (using one of the param_h, param_m and
> param_l set of parameters - which one ?). The HDRS blocks should be disabled
> (how ?).
>
> The HDR input, on the other hand, provides 2 or 3 channels with different
> sensitivities (high, middle and low, from sensors that implement DOL or DCG
> HDR). The preprocess pipeline operates on those channels with different sets
> of parameters, and the HDRS combines the channels into a single 24 bit image.
>
> I'm not entirely sure how the two or three channels are provided to the ISP, with
> DOL sensors there's a delay before the sensor starts outputting the short (and
> very short) lines, so line buffers are needed to realign the lines. I don't see this
> in the block diagram.
>
> It would be nice to expand the documentation a bit with such information about
> HDR processing, as I assume other developers may face similar questions.
>
The L1ISP hardware supports following RAW formats.
* uncompressed single HDR image [8-24 bit]
* This driver supports 8-14bits, as 16-24bit is not fully tested.
* PWL compressed single HDR image [8-14 bit]
* multiple SDR images [8-12 bit, 1-3 planes]
* This driver does not support it for the following reasons.
As you pointed out, the PWL input is an 8-14 bit image, which is expanded to
a 24-bit image in the HDRE block. In the preprocess stage, the SLIC (bit slice)
block divides the 24-bit image into three images with different sensitivities
for processing by ABPC, PWHB, and RCNR. Finally, the HDRS block generates a 24-bit image.
The L1ISP hardware supports DOL HDR for multiple SDR inputs. In the document's
diagram, up to three images bypass the SLIC and are processed by the HDRS block.
However, as you pointed out, it is not possible to connect a typical DOL sensor
due to the lack of sufficient line buffer depth. Because it is difficult to
operate in practical cases, this driver does not support multiple SDR image inputs
nor DOL HDR.
Although this driver does not support memory-to-memory operation, the VIIF hardware
has an option to read in-memory images via a DMA reader. With that (possible) datapath,
properly organized line-interleaved images can be processed as DOL HDR input.
> > +
> > + L1ISP::INPUT::preprocess
> > +
> > + +-----+
> +-----+
> > + | 24b | +------+ +------+ +------+ +------+
> +------+ | 24b |
> > + | RAW |--->| SLIC |--->| ABPC |--->| PWHB |--->| RCNR |--->| HDRS
> |--->| RAW |
> > + | (0) | +------+ +------+ +------+ +------+ +------+
> | (1) |
> > + +-----+
> +-----+
> > +
> > + L1ISP::MainProcess(MPRO)
> > +
> > + +-----+
> > + | 24b | +------+ +------+
> > + | RAW |--->| BLVC |--->| LSSC |---+
> > + | (1) | +------+ +------+ |
> > + +-----+ |
> > + |
> > + +------------------------------+
> > + |
> > + | +-----------+ +-------------+ +--------+
> +-----+
> > + +--->| MPRO | | MPRO | | MPRO |
> +------+ | 16b |
> > + | Digital |--->| Demosaicing |----| Color |--->| HDRC
> |--->| RGB |
> > + +--->| Amplifier | | | | Matrix | +------+
> | |
> > + | +-----------+ +-------------+ +--------+
> +-----+
> > + | | |
> > + | +--------------+ | | +------+
> > + +----| Auto |<----+ +--->| AEXP |---> Auto-Exposure
> statistics
> > + | Whitebalance | +------+
> > + +--------------+
> > + |
> > + +------------------------------> Auto-Whitebalance
> > + statistics
> > +
> > + L1ISP::VideoProcess(VPRO)
> > +
> > + +-----+ +------------+ +------------+ +---------------+
> +--------+
> > + | 16b |--->| Gamma |--->| RGB2YUV |--->| Image Quality
> |--->| Output |
> > + | RGB | | Correction | | Conversion | | Adjustment | |
> 16b |
> > + | | +------------+ +------------+ +---------------+
> | YUV |
> > + +-----+
> +--------+
> > +
> > + L1ISP::AnalogGain
> > +
> > + statistics +-------------+
> +------------------+
> > + information ---> (user SW) --->| Analog Gain |--->| ABPC, RCNR, LSSC
> |
> > + +-------------+ | MPRO,
> VPRO |
> > +
> > + +------------------+
>
> I'm a bit puzzled by "analog gain" here, as the ISP operates on digital data only.
> Does the ISP need to be informed of the analog gain values computed by the
> AEGC software algorithm and applied to the camera sensor, for instance to
> estimate the noise level based on the analog gain and adapt noise filtering
> accordingly ? Or is it something else ?
>
> Edit: the text below answers this question :-)
>
The name "analog gain" was coined by the hardware team and is quite misleading.
It has nothing to do with the gain of the image sensor. Instead, it is a parameter
used to collectively change the processing intensity of several functional blocks
in the L1ISP. The intensity is calculated from the gain value provided by
viif_l1_ag_config and the linear transformation parameters (grad, ofst) provided
by viif_l1_ag_mode_config. Each of the blocks determines the coefficients and
thresholds from the passed intensity.
To avoid misunderstandings, it seems better to stop using the term 'analog gain'
and replace it with expressions like 'application gain' or 'algorithm gain,'
which maintain the abbreviation 'AG' that appears in register names
and structure member names."
> > +
> > +L2 ISP provides following operations:
> > +
> > +- Input: accepts 16bit YUV / RGB
> > +- Operations:
> > + - Lens undistortion; see :c:type:`viif_l2_undist`
>
> The structure is named viif_l2_undist_config. Please make sure to compile the
> documentation, you should then get warnings for C types that are not found.
>
It seems I missed error messages.
I'll fix it.
> > + - Scaling; see :c:type:`viif_l2_roi`
>
> viif_l2_roi_config
>
I'll fix it too.
> > + - Cropping; see :c:type:`viif_l2_roi`
>
> Looking at the implementation of the resizer subdevs, I see that the crop and
> compose rectangles can be set, but they don't seem to ne used to configure the
> resizer. Instead, the viif_l2_roi_config parameters are used to configure
> cropping on the resizer input and scaling. This discrepancy isn't good. I see two
> options to fix it:
>
> - Keep configuring the resizer through viif_l2_roi_config and drop the resizer
> subdevs. This will simplify the driver. The main drawback is that it
> won't be possible to implement digital zoom (by changing the resizer
> configuration) asynchronously from the ISP parameters buffers, which
> can be useful to lower the latency of digital zoom.
>
> - Drop viif_l2_roi_config, and configure the resizer from the formats,
> crop and selection rectangles on the resizer subdev pads. This makes
> the driver more complex. The main advantage is that digital zoom will
> be configurable with a smaller latency, but the drawback is that it
> won't be possible (or it will be more difficult) to configure it
> synchronously with other ISP parameters.
>
> There are drivers in mainline that implement either of those options, so you can
> pick the one you think is best.
>
> An additional issue is that the hardware seems to implement cropping on the
> output of the resizer only, not on the input. Given that the size of the images
> output by the ISP pipeline must stay constant during video capture (otherwise
> there would be a risk of buffer overflow), modifying the crop rectangle on the
> output of the resizer is usually not allowed during streaming. I think this could
> be worked around by allowing modification only of the left and top coordinates
> during streaming, but configuring everything through viif_l2_roi_config would
> likely be easier. In that case, you should probably extend viif_l2_roi_config with
> the crop offsets.
>
> All of this reflects my current understanding of the ISP architecture, based on
> this document and on the driver code, so please correct me if there's anything I
> misunderstood. We can discuss the different options further before you modify
> the driver and send a new version.
>
First of all, I apologize for attempting to make significant changes to the
driver structure without consulting you. The hardware specifications of VIIF
are unique, and the knowledge from the drivers I referred to (mainly rkisp1)
cannot be directly applied. As a result, I am still struggling with how to
write a driver that conforms to the V4L2 framework.
I am considering removing the resizer subdevice because viif_l2_roi_config
can set more detailed parameters compared to the resizer subdevice's pad.
To explain my idea and answer your questions, let me describe the processing of
the VIIF L2-ISP.
The central functions of the L2 ISP are lens undistortion and scaling. They are
integrated and executed simultaneously. To explain the coordinate transformation here,
several virtual images are defined.
* L1ISP output image
* the input of L2ISP
* geometry: {width, height}
* mapped to sink.format
* Input image
* geometry: {center_x, center_y, width, height}
* relative to the L1ISP output image
* mapped to sink.crop
* resizer subdevice assumes that sink.crop = sink.format
* Corrected image after undistortion
* geometry: {width, height}
* relative to the Input image
* Corrected image with scale
* geometry: {width, height}
* relative to the Input image
* mapped to sink.compose
* resizer subdevice assumes that scale factor = sink.compose / sink.format
* ROI image
* geometry: {left, top, width, height}
* relative to the Corrected image with scale
* mapped to source.crop
* Crop image
* the output of L2ISP.
* geometry: {left, top, width, height}
* relative to the Corrected image with scale
* mapped to source.crop
These images mentioned here do not have corresponding frame buffers. However,
there are control registers corresponding to the width and height of each image,
and appropriate values must be set. Otherwise, the output image may be corrupted,
or the L2ISP may abort.
Regarding your question of why sink.crop is not used.
According to the list above, sink.crop corresponds to the Input image. However,
the coordinates of the Input image must be set to align with the optical center
for undistortion, so it cannot be used to crop arbitrary rectangles.
Regarding digital zoom.
In the existing use cases of VIIF, the scale and crop are fixed during recording.
Therefore, I did not consider changing the selection parameter like digital zoom.
Regarding resizer subdevice.
The purpose of the resizer subdevice is to set the initial values of L2ISP
from pad's selection parameters. Most virtual image rectangles can be mapped to
the pad's parameters. Since the parameters of undistortion and the rectangle of
the Corrected image after undistortion are unknown, I assumed that undistortion
is disabled (pass-through) and the Corrected image after undistortion matches
the Input image. In this case, the scaling parameters can be calculated from
sink.compose and sink.input.
On the other hand, in cases where both undistortion and scaling are utilized,
it is necessary to set viif_l2_undist_config and viif_l2_roi_config. Therefore,
I made it possible to overwrite these parameters through the parameter buffer.
To configure cropping with the parameter buffer instead of pad's selection,
I'll add two members (left and top) to viif_l2_roi_config.
Otherwise, I'll newly add struct viif_l2_crop_config to the parameter buffer.
> > + - Gamma correction; see :c:type:`viif_l2_gamma_config`
> > + - YUV2RGB
> > +- Output: RGB, YUV422, YUV444
> > +
> > +Below is the block diagram::
> > +
> > + L2ISP
> > +
> > + +-------+ +------------+ +--------------+ +---------+
> > + | Input |--->| YUV2RGB |--->| Lens |--->| Scaling |---> |
> > + | Image | | Conversion | | Undistortion | | |---> |
> > + +-------+ +------------+ +--------------+ +---------+
> |
>
> Is the scaling configuration for the two outputs independent ? If so I would
> move the scaling block just before gamma correction, in each of the branches
> below.
>
Scaling configurations are independent of each other.
I'll update the diagram as follows:
+--------------+ +---------+ +------------+
| Lens | -+-> | Scaling | ---> | Gamma |
| Undistortion | | | | | Correction |
+--------------+ | +---------+ +------------+
|
| +---------+ +------------+
+-> | Scaling | ---> | Gamma |
| | | Correction |
+---------+ +------------+
> > +
> |
> > + +----------------------------------------------------------+
> > + |
> > + | +----------+ +------------+ +--------+
> +--------+
> > + +--->|Gamma |--->| Colorspace |--->| Data |--->|
> Output |
> > + | |Correction| | Conversion | | Packer | | Image |
> > + | +----------+ +------------+ +--------+
> +--------+
> > + |
> > + | +----------+ +------------+ +--------+
> +--------+
> > + +--->|Gamma |--->| Colorspace |--->| Data |--->|
> Output |
> > + |Correction| | Conversion | | Packer | | Image |
> > + +----------+ +------------+ +--------+
> +--------+
> > +
> > +visconti-viif:resizer - Resizer Subdevice Node
> > +----------------------------------------------
> > +
> > +The resizer subdevice resides between ISP subdevice and Capture
> > +device on a capture path for post0 and post1.
> > +It receives resize and crop parameters for the specific capture path
> > +and controls L2ISP HW.
> > +
> > +following selection rectangles can be passed at VIDIOC_S_SELECTION
> ioctl.
>
> s/following/The following/
>
I'll fix it.
> > +
> > +- sink pads's compose rectangle (V4L2_SEL_TGT_COMPOSE) for scaling
> > +- source pad's crop rectangle (V4L2_SEL_TGT_CROP) for cropping
> > +
> > +
> > +viif_capture_post0, viif_capture_post1 - Processed Image Capture
> > +Video Node
> > +---------------------------------------------------------------------
> > +------
> > +
> > +These video nodes are used for capturing images processed at ISPs.
> > +Supported capture formats are as follows:
> > +
> > +- V4L2_PIX_FMT_RGB24
> > +- V4L2_PIX_FMT_ABGR32
> > +- V4L2_PIX_FMT_YUV422M
> > +- V4L2_PIX_FMT_YUV444M
>
> The hardware doesn't support semi-planar formats (NV12 or NV16) or packed
> formats (YUYV) ?
>
The hardware does not support NV12 nor NV16.
For packed formats, UYVY is avaiable but not tested fully.
I'll check if there are any issues with adding UYVY.
> > +- V4L2_PIX_FMT_Y16
> > +
> > +Bayer format is not supported. Use viif_capture_sub instead.
> > +
> > +POST0 and POST1 can output images from the same input image using
> > +different cropping and scaling settings.
> > +
> > +viif_capture_sub - Raw Image Capture Video Node
> > +-----------------------------------------------
> > +
> > +This video node is used for capturing bayer image from the sensor.
> > +The output picture has exactly the same resolution and format as the
> sensor input.
> > +The pipeline does not edit pixel values.
> > +However, when writing pixel values to memory, they are shifted to the
> > +MSB to match either 8bit or 16bit.
> > +
> > +Therefore, resulting capture formats are as follows:
> > +
> > +- for 8bit RAW input:
> > + - V4L2_PIX_FMT_SRGGB8
> > + - V4L2_PIX_FMT_SGRBG8
> > + - V4L2_PIX_FMT_SGBRG8
> > + - V4L2_PIX_FMT_SBGGR8
> > +- for 10, 12, 14bit RAW input:
> > + - V4L2_PIX_FMT_SRGGB16
> > + - V4L2_PIX_FMT_SGRBG16
> > + - V4L2_PIX_FMT_SGBRG16
> > + - V4L2_PIX_FMT_SBGGR16
> > +
> > +.. _viif_params:
> > +
> > +viif_params - ISP Parameters Video Node
> > +---------------------------------------
> > +
> > +The viif_params video node receives a set of ISP parameters from
> > +userspace to be applied to the hardware during a video stream.
> > +
> > +The buffer format is defined by struct
> > +:c:type:`visconti_viif_isp_config`, and userspace should
> set :ref:`V4L2_META_FMT_VISCONTI_VIIF_PARAMS
> <v4l2-meta-fmt-visconti-viif-params>` as the data format.
> > +
> > +.. _viif_stats:
> > +
> > +viif_stats - Statistics Video Node
> > +----------------------------------
> > +
> > +The viif_stats video node provides current status of ISP.
>
> The viif_stats video node provides statistics computed by the ISP and frame
> processing status.
>
I'll update the sentence as you suggested.
> > +
> > +Following information is included:
> > +
> > +* statistics of auto white balance
> > +* average luminance information which can be used by auto exposure
> software impl.
>
> s/impl/implementation/
>
I should not have abbreviated a word.
I'll fix it.
>
> I would also add
>
> * CSI-2 receiver calibration and error status
> * ISP error status
>
> It's quite uncommon to provide this kind of status through ISP stats buffers, but
> it sounds like an interesting idea. Other drivers usually keep error counters in
> the kernel and expose them through debugfs.
>
These status information have existed since the initial implementation and must
be passed to the user space in some way. On the other hand, I think that these,
especially the CSI2 receiver error status, are not related to the
frame-by-frame processing of the ISP, so they should not be included in the
parameter buffer. I should classify the information and update the members of
the parameter buffer accordingly. I think some of the information should be
exposed through debugfs. However, I'm not sure how to classify them and how to
implement the debugfs interface. Could you tell me the source code that
should be referred to?
> > +
> > +The buffer format is defined by struct
> > +:c:type:`visconti_viif_isp_stat`, and userspace should
> set :ref:`V4L2_META_FMT_VISCONTI_VIIF_STATS
> <v4l2-meta-fmt-visconti-viif-stats>` as the data format.
> > +
> > +Feedback Operations
> > +===================
> > +
> > +Among the so-called 3A functions, VIIF provides only auto-whitebalance
> and auto-exposure.
> > +Auto-whitebalance is a standalone hardware feature.
> > +Some status information is available through the ISP statistics interface.
> > +
> > +Auto-exposure is realized through a combination of hardware and userland
> software.
> > +VIIF provides weighted average luminance information through the ISP
> statistics interface.
> > +The userland application calculates the sensor gain, sensor exposure and
> ISP digital gain.
> > +The calculated parameters are then passed to sensor's controls and the ISP
> parameter interface.
> > +
> > +Among ISP parameters, there are parameters called AG (analog gain).
> > +Actually, AG parameters have nothing to do with auto-exposure.
> > +It controls "strength" in several signal correction algorithms.
> > +Below is the list of the functions which may be affected by AG parameters:
> > +
> > +- ABPC/DPC
> > +- RCNR
> > +- LSSC
> > +- MPRO: color matrix correction
> > +- VPRO
> > +
> > +Capturing Video Frames Example
> > +==============================
> > +
> > +In the following example,
> > +imx219 camera is connected to pad 0 of 'visconti_csi2rx' subdevice.
> > +
> > +The following commands yield three pictures with different zoom ratio:
> > +- main path 0: 1.5x zoom, crop 1920x1080, RGB picture
> > +- main path 1: 0.67x zoom, crop 640x480, RGB picture
> > +- sub path: 1920x1080 RAW picture
> > +
> > +.. code-block:: bash
> > +
> > + # set the links
> > + media-ctl -d platform:visconti-viif-0 -r
> > + media-ctl -d platform:visconti-viif-0 -l '"imx219 1-0010":0 ->
> "visconti_csi2rx 1c008000.csi2rx":0 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"visconti_csi2rx
> 1c008000.csi2rx":1 -> "visconti-viif:isp":0 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":1 ->
> "visconti-viif:resizer0":0 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":2 ->
> "visconti-viif:resizer1":0 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":3 ->
> "viif_capture_sub":0 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer0":1 ->
> "viif_capture_post0":0 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer1":1 ->
> "viif_capture_post1":0 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"viif_params":0 ->
> "visconti-viif:isp":4 [1]'
> > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":5 ->
> "viif_stats":0 [1]'
> > +
> > + # set format for imx219
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"imx219 1-0010":0
> [fmt:SRGGB10_1X10/1920x1080]'
> > +
> > + # set format for csi2rx
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti_csi2rx
> 1c008000.csi2rx":0 [fmt:SRGGB10_1X10/1920x1080 field:none
> colorspace:raw xfer:none ycbcr:601 quantization:full-range]'
> > +
> > + # set format for isp sink pad
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":0
> [fmt:SRGGB10_1X10/1920x1080]'
> > +
> > + # set format for resizer pads
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> '"visconti-viif:resizer0":0 '"[fmt:YUV8_1X24/1920x1080
> compose:(0,0)/2880x1620]"
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> '"visconti-viif:resizer0":1 '"[crop:(480,16)/1920x1080]"
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> '"visconti-viif:resizer1":0 '"[fmt:YUV8_1X24/1920x1080
> compose:(0,0)/1280x720]"
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> '"visconti-viif:resizer1":1 '"[crop:(320,32)/640x480]"
> > +
> > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":1
> [fmt:YUV8_1X24/1024 crop:(640,0)/1024x1024]'
> > +
> > + # set format for main path0
> > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> "width=1920,height=1080"
> > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> "pixelformat=RGB3"
> > +
> > + # set format for main path1
> > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> "width=640,height=480"
> > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> "pixelformat=RGB3"
> > +
> > + # start streaming
> > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0
> > +--stream-mmap --stream-count 1000 &
> > +
> > + # start streaming with other devices while viif_capture_post0 is
> running
> > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post1
> --stream-mmap --stream-count 10
> > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_sub
> > +--stream-mmap --stream-count 10
> > +
> > +Use of coherent memory
> > +======================
> > +
> > +Visconti5 SoC has two independent DDR SDRAM controllers.
> > +Each controller is mapped to 36bit address space.
> > +
> > +Accelerator bus masters have two paths to access memory; one is
> > +directly connected to SDRAM controller, the another is connected via
> > +a cache coherency bus which keeps coherency among CPUs.
> > +
> > +From accelerators and CPUs, the address map is following:
> > +
> > +* 0x0_8000_0000 DDR0 direct access
> > +* 0x4_8000_0000 DDR0 coherency bus
> > +* 0x8_8000_0000 DDR1 direct access
> > +* 0xC_8000_0000 DDR1 coherency bus
> > +
> > +The base address can be specified with "memory" and "reserved-memory"
> > +elements in a device tree description.
> > +It's not recommended to mix direct address and coherent address.
> > +
> > +The Visconti5 VIIF driver always use only direct address to configure Video
> DMACs of the hardware.
> > +This design is to avoid great performance loss at coherency bus caused by
> massive memory access.
> > +You should not put the dma_coherent attribute to viif element in device
> tree.
> > +Cache operations are done automatically by videobuf2 driver.
> > diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst
> > b/Documentation/userspace-api/media/v4l/meta-formats.rst
> > index 86ffb3bc8ade..2336842f0c26 100644
> > --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
> > +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
> > @@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata` interface
> only.
> > metafmt-pisp-fe
> > metafmt-rkisp1
> > metafmt-uvc
> > + metafmt-visconti-viif
> > metafmt-vivid
> > metafmt-vsp1-hgo
> > metafmt-vsp1-hgt
> > diff --git
> > a/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > new file mode 100644
> > index 000000000000..dc4b31627fe1
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > @@ -0,0 +1,48 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +
> > +.. _v4l2-meta-fmt-visconti-viif-params:
> > +
> > +.. _v4l2-meta-fmt-visconti-viif-stats:
> > +
> > +*********************************************************************
> > +****************** V4L2_META_FMT_VISCONTI_VIIF_PARAMS ('vifp'),
> > +V4L2_META_FMT_VISCONTI_VIIF_STATS ('vifs')
> > +*********************************************************************
> > +******************
> > +
> > +Configuration parameters
> > +========================
> > +
> > +The configuration parameters are passed to the :ref:`viif_params
> > +<viif_params>` metadata output video node, using the
> > +:c:type:`v4l2_meta_format` interface. The buffer contains a single
> > +instance of the C structure :c:type:`visconti_viif_isp_config`
> > +defined in ``visconti_viif.h``. So the structure can be obtained from the
> buffer by:
> > +
> > +.. code-block:: c
> > +
> > + struct visconti_viif_isp_config *params = (struct
> > +visconti_viif_isp_config*) buffer;
> > +
> > +VIIF statistics
> > +===============
> > +
> > +The VIIF device collects different statistics over an input Bayer frame.
> > +Those statistics are obtained from the :ref:`viif_stats <viif_stats>`
> > +metadata capture video node, using the :c:type:`v4l2_meta_format`
> > +interface. The buffer contains a single instance of the C structure
> > +:c:type:`visconti_viif_isp_stat` defined in ``visconti_viif.h``. So
> > +the structure can be obtained from the buffer by:
> > +
> > +.. code-block:: c
> > +
> > + struct visconti_viif_isp_stat *stats = (struct
> > +visconti_viif_isp_stat*) buffer;
> > +
> > +The statistics collected are Exposure, AWB (auto white balance) and errors.
> > +See :c:type:`visconti_viif_isp_stat` for details of the statistics.
> > +
> > +The statistics and configuration parameters described here are
> > +usually consumed and produced by dedicated user space libraries that
> > +comprise the tuning tools using software control loop.
> > +
> > +visconti viif uAPI data types
> > +=============================
> > +
> > +.. kernel-doc:: include/uapi/linux/visconti_viif.h
>
> Assuming the documentation in the header file is adequate, the level of detail
> here is fine.
>
> --
> Regards,
>
> Laurent Pinchart
Regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface
2025-01-02 13:16 ` Laurent Pinchart
@ 2025-02-05 12:31 ` yuji2.ishikawa
0 siblings, 0 replies; 34+ messages in thread
From: yuji2.ishikawa @ 2025-02-05 12:31 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent,
Thank you for your review comments.
> -----Original Message-----
> From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Sent: Thursday, January 2, 2025 10:17 PM
> To: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> <yuji2.ishikawa@toshiba.co.jp>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: Re: [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti
> Video Input Interface
>
> Hi Ishikawa-san,
>
> Thank you for the patch.
>
> On Mon, Nov 25, 2024 at 06:21:46PM +0900, Yuji Ishikawa wrote:
> > Added entries for visconti Video Input Interface driver, including;
>
> Commit messages are written using the imperative mood, as if you were giving
> orders to someone:
>
> s/Added/Add/
>
Thank you for the advice. I'll update commit messages.
> > * device tree bindings
> > * source files
> > * documentation files
> >
> > Signed-off-by: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > Reviewed-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>
> I would have added the MAINTAINERS entry in 1/8 with just the DT binding file,
> and added more files in corresponding patches. That would be easier to review
> and check if entries are missing. I don't mind much though, so if you prefer
> keeping it this way:
>
From the next patch set, I'll update MAINTAINERS at the same time as new files are added.
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>
> > ---
> > Changelog v2:
> > - no change
> >
> > Changelog v3:
> > - added entry for driver API documentation
> >
> > Changelog v4:
> > - added entry for header file
> >
> > Changelog v5:
> > - no change
> >
> > Changelog v6:
> > - update path to VIIF driver source files
> >
> > Changelog v7:
> > - no change
> >
> > Changelog v8:
> > - rename bindings description file
> >
> > Changelog v9:
> > - no change
> >
> > Changelog v10:
> > - add a separate entry of VIIF driver
> >
> > Changelog v11:
> > - no change
> >
> > Changelog v12:
> > - add a bindings description of CSI2RX driver
> >
> > MAINTAINERS | 12 ++++++++++++
> > 1 file changed, 12 insertions(+)
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS index
> > b878ddc99f94..b5c819e94e9b 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -23430,6 +23430,18 @@ F:
> Documentation/devicetree/bindings/media/i2c/tc358743.txt
> > F: drivers/media/i2c/tc358743*
> > F: include/media/i2c/tc358743.h
> >
> > +TOSHIBA VISCONTI VIIF DRIVER
> > +M: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> > +M: Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>
> > +L: linux-media@vger.kernel.org
> > +S: Maintained
> > +F: Documentation/admin-guide/media/visconti-viif.*
> > +F:
> Documentation/devicetree/bindings/media/toshiba,visconti5-csi2rx.y
> aml
> > +F:
> Documentation/devicetree/bindings/media/toshiba,visconti5-viif.yam
> l
> > +F: Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > +F: drivers/media/platform/toshiba/visconti/
> > +F: include/uapi/linux/visconti_viif.h
> > +
> > TOSHIBA WMI HOTKEYS DRIVER
> > M: Azael Avalos <coproscefalo@gmail.com>
> > L: platform-driver-x86@vger.kernel.org
>
> --
> Regards,
>
> Laurent Pinchart
Regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver
2025-02-05 12:29 ` yuji2.ishikawa
@ 2025-04-15 9:01 ` yuji2.ishikawa
2025-06-18 1:30 ` yuji2.ishikawa
0 siblings, 1 reply; 34+ messages in thread
From: yuji2.ishikawa @ 2025-04-15 9:01 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent,
As I reconsider the implementation of the v12 driver, I have decided on how to
change the important parts pointed out in the review comments. On the other hand,
I'm still looking for better implementation methods in some other areas.
I would like to request your comments on these matters.
- Topic 1: Removing the resizer subdevice
- Topic 2: Adding new members to the parameter buffer
- Topic 3: Passing error information via debugfs
- Topic 4: Reconsidering the topology of the capture device (sub path)
> -----Original Message-----
> From: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> Sent: Wednesday, February 5, 2025 9:30 PM
> To: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> Subject: RE: [PATCH v12 7/8] documentation: media: add documentation for
> Toshiba Visconti Video Input Interface driver
>
> Hello Laurent
>
> Thank you for your review comments.
>
> > > + - Cropping; see :c:type:`viif_l2_roi`
> >
> > Looking at the implementation of the resizer subdevs, I see that the crop and
> > compose rectangles can be set, but they don't seem to ne used to configure
> the
> > resizer. Instead, the viif_l2_roi_config parameters are used to configure
> > cropping on the resizer input and scaling. This discrepancy isn't good. I see
> two
> > options to fix it:
> >
> > - Keep configuring the resizer through viif_l2_roi_config and drop the resizer
> > subdevs. This will simplify the driver. The main drawback is that it
> > won't be possible to implement digital zoom (by changing the resizer
> > configuration) asynchronously from the ISP parameters buffers, which
> > can be useful to lower the latency of digital zoom.
> >
> > - Drop viif_l2_roi_config, and configure the resizer from the formats,
> > crop and selection rectangles on the resizer subdev pads. This makes
> > the driver more complex. The main advantage is that digital zoom will
> > be configurable with a smaller latency, but the drawback is that it
> > won't be possible (or it will be more difficult) to configure it
> > synchronously with other ISP parameters.
> >
> > There are drivers in mainline that implement either of those options, so you
> can
> > pick the one you think is best.
> >
> > An additional issue is that the hardware seems to implement cropping on the
> > output of the resizer only, not on the input. Given that the size of the images
> > output by the ISP pipeline must stay constant during video capture
> (otherwise
> > there would be a risk of buffer overflow), modifying the crop rectangle on the
> > output of the resizer is usually not allowed during streaming. I think this could
> > be worked around by allowing modification only of the left and top
> coordinates
> > during streaming, but configuring everything through viif_l2_roi_config would
> > likely be easier. In that case, you should probably extend viif_l2_roi_config
> with
> > the crop offsets.
> >
> > All of this reflects my current understanding of the ISP architecture, based on
> > this document and on the driver code, so please correct me if there's anything
> I
> > misunderstood. We can discuss the different options further before you
> modify
> > the driver and send a new version.
> >
>
> First of all, I apologize for attempting to make significant changes to the
> driver structure without consulting you. The hardware specifications of VIIF
> are unique, and the knowledge from the drivers I referred to (mainly rkisp1)
> cannot be directly applied. As a result, I am still struggling with how to
> write a driver that conforms to the V4L2 framework.
>
> I am considering removing the resizer subdevice because viif_l2_roi_config
> can set more detailed parameters compared to the resizer subdevice's pad.
>
> To explain my idea and answer your questions, let me describe the processing
> of
> the VIIF L2-ISP.
>
> The central functions of the L2 ISP are lens undistortion and scaling. They are
> integrated and executed simultaneously. To explain the coordinate
> transformation here,
> several virtual images are defined.
>
> * L1ISP output image
> * the input of L2ISP
> * geometry: {width, height}
> * mapped to sink.format
> * Input image
> * geometry: {center_x, center_y, width, height}
> * relative to the L1ISP output image
> * mapped to sink.crop
> * resizer subdevice assumes that sink.crop = sink.format
> * Corrected image after undistortion
> * geometry: {width, height}
> * relative to the Input image
> * Corrected image with scale
> * geometry: {width, height}
> * relative to the Input image
> * mapped to sink.compose
> * resizer subdevice assumes that scale factor = sink.compose / sink.format
> * ROI image
> * geometry: {left, top, width, height}
> * relative to the Corrected image with scale
> * mapped to source.crop
> * Crop image
> * the output of L2ISP.
> * geometry: {left, top, width, height}
> * relative to the Corrected image with scale
> * mapped to source.crop
>
> These images mentioned here do not have corresponding frame buffers.
> However,
> there are control registers corresponding to the width and height of each image,
> and appropriate values must be set. Otherwise, the output image may be
> corrupted,
> or the L2ISP may abort.
>
> Regarding your question of why sink.crop is not used.
> According to the list above, sink.crop corresponds to the Input image. However,
> the coordinates of the Input image must be set to align with the optical center
> for undistortion, so it cannot be used to crop arbitrary rectangles.
>
> Regarding digital zoom.
> In the existing use cases of VIIF, the scale and crop are fixed during recording.
> Therefore, I did not consider changing the selection parameter like digital zoom.
>
> Regarding resizer subdevice.
> The purpose of the resizer subdevice is to set the initial values of L2ISP
> from pad's selection parameters. Most virtual image rectangles can be mapped
> to
> the pad's parameters. Since the parameters of undistortion and the rectangle of
> the Corrected image after undistortion are unknown, I assumed that
> undistortion
> is disabled (pass-through) and the Corrected image after undistortion matches
> the Input image. In this case, the scaling parameters can be calculated from
> sink.compose and sink.input.
> On the other hand, in cases where both undistortion and scaling are utilized,
> it is necessary to set viif_l2_undist_config and viif_l2_roi_config. Therefore,
> I made it possible to overwrite these parameters through the parameter buffer.
>
> To configure cropping with the parameter buffer instead of pad's selection,
> I'll add two members (left and top) to viif_l2_roi_config.
> Otherwise, I'll newly add struct viif_l2_crop_config to the parameter buffer.
>
I am removing the resizer sub-device. In the new implementation, the undistortion
and scaling functions of L2ISP are controlled using the l2_undist member and
l2_roi member of the parameter buffer. To control the cropping function of L2ISP,
the l2_crop_post0 and l2_crop_post1 members are added to the parameter buffer
to indicate the top-left coordinates for cropping.
These new members have the type struct l2_crop_config {__u32 left; __u32 top}.
The default values of the members are (0, 0).
ISP subdevice pads have the following specifications:
- The sink_pad::fmt and source_pad(post0)::fmt, source_pad(post1)::fmt can be set independently.
- Each pad cannot be set for crop and compose.
- The maximum size of source_pad::fmt is either the hardware limit of L2ISP or 2x sink_pad::fmt.
- The minimum size of source_pad::fmt is the hardware limit of L2ISP.
- Due to the inclusion relationship of the planes, the following constraint should be met:
l2_crop_post0::left + source_pad(post0)::fmt::width <= l2_roi::corrected_width(post0)
A similar constraint should be applied to the height and the post1.
There are two possible behaviors when constraints are not met. If the instructions
through the parameter buffer are not accepted, how should the driver behave?
- Increment the error counter (referred by debugfs) and ignore the instructions
from the parameter buffer.
- Modify l2_roi::corrected_width(post0) and l2_crop_post0::left and apply them
to the hardware. As a result, an L2ISP error might occur. The drawback is that
the user program cannot know the modified coordinates.
> > > +.. _viif_params:
> > > +
> > > +viif_params - ISP Parameters Video Node
> > > +---------------------------------------
> > > +
> > > +The viif_params video node receives a set of ISP parameters from
> > > +userspace to be applied to the hardware during a video stream.
> > > +
> > > +The buffer format is defined by struct
> > > +:c:type:`visconti_viif_isp_config`, and userspace should
> > set :ref:`V4L2_META_FMT_VISCONTI_VIIF_PARAMS
> > <v4l2-meta-fmt-visconti-viif-params>` as the data format.
> > > +
Please advise on how to handle unverified ISP functions in the parameter buffer.
The L1ISP hardware has a function to calculate the luminance histogram.
The software for this function is experimental and has not yet been developed
as a Linux driver. However, since the structure that the parameter buffer
handles comprehensively represents all ISP functions, I understand that it
would be difficult to add members for the histogram function later.
Should members corresponding to the histogram function be added to the parameter
buffer structure even if the implementation does not exist? Or, is it permissible
to add new members to the parameter buffer structure based on certain conditions?
> > > +.. _viif_stats:
> > > +
> > > +viif_stats - Statistics Video Node
> > > +----------------------------------
> > > +
> > > +The viif_stats video node provides current status of ISP.
> >
> > The viif_stats video node provides statistics computed by the ISP and frame
> > processing status.
> >
>
> I'll update the sentence as you suggested.
>
> > > +
> > > +Following information is included:
> > > +
> > > +* statistics of auto white balance
> > > +* average luminance information which can be used by auto exposure
> > software impl.
> >
> > s/impl/implementation/
> >
>
> I should not have abbreviated a word.
> I'll fix it.
>
> >
> > I would also add
> >
> > * CSI-2 receiver calibration and error status
> > * ISP error status
> >
> > It's quite uncommon to provide this kind of status through ISP stats buffers,
> but
> > it sounds like an interesting idea. Other drivers usually keep error counters in
> > the kernel and expose them through debugfs.
> >
>
> These status information have existed since the initial implementation and
> must
> be passed to the user space in some way. On the other hand, I think that these,
> especially the CSI2 receiver error status, are not related to the
> frame-by-frame processing of the ISP, so they should not be included in the
> parameter buffer. I should classify the information and update the members of
> the parameter buffer accordingly. I think some of the information should be
> exposed through debugfs. However, I'm not sure how to classify them and how
> to
> implement the debugfs interface. Could you tell me the source code that
> should be referred to?
>
In the new implementation, the following information will be passed to user
space using debugfs:
- Errors during capture (originally, struct viif_reported_errors)
- Counters will be prepared for each error cause. This will be implemented
using debugfs_create_ulong().
- The original specification was to "clear on read", but since it seems
difficult to implement concisely, it will be changed to a count-up operation.
- CSI2RX calibration status
- Information in three states (OK, Not-done, Error) for each signal line.
This will be represented using debugfs_create_x8() or debugfs_create_file()
in a human-readable format.
- CSI2RX error status
- Bit vector information for each error cause. This will be represented using
debugfs_create_x32().
The following information will remain in the parameter buffer's statistics interface:
- Average luminance information
- Auto white balance information
--------------------
I would like to reconsider the connection between the Capture device (sub) and
other subdevices. Could you provide some feedback?
Currently, the Capture device (sub) is connected to the ISP subdevice.
This topology is based on my best understanding of the media controller framework
at the time of design. However, the Capture device (sub) saves RAW images without
processing them, and its operation does not depend on ISP functions. Therefore,
for better maintenance in the future, I am considering disconnecting
the Capture device (sub) from the ISP subdevice.
Additionally, I want to prepare for improvements regarding streams. The CSI2RX
hardware can handle multiple virtual channels. Both the ISP hardware and
Capture hardware (sub) can each select and receive one virtual channel. Currently,
due to insufficient tests, both are limited to receiving virtual channel 0,
but in the future, I want to be able to handle multiple channels using the stream API.
I'm considering three options for the topology.
- Option 1: Connect the Capture device (sub) to the ISP subdevice as it is currently.
The switching of streams will be realized as a function of the ISP subdevice.
- Option 2: Create a new MUX subdevice and connect the ISP subdevice and
Capture device (sub) to it. When handling multiple streams in the future,
the switching function will be realized by the MUX subdevice. This will reduce
the complexity of the ISP subdevice implementation.
- Option 3: Connect both the ISP subdevice and Capture device (sub) to
the source port of CSI2RX. Is such a configuration really possible? The switching
of multiple streams will be handled by both the ISP subdevice and Capture device respectively.
[Option1: Current design]
CSI2RX|0 -- ISP|1 -- Capture(post0)
|2 -- Capture(post1)
|3 -- Capture(sub)
[Option2]
CSI2RX|0 -- MUX|1 -- ISP|1 -- Capture(post0)
| |2 -- Capture(post1)
|2 -- Capture(sub)
[Option3]
CSI2RX|0 --+-- ISP|1 -- Capture(post0)
| |2 -- Capture(post1)
+-- Capture(sub)
> > > +
> > > +The buffer format is defined by struct
> > > +:c:type:`visconti_viif_isp_stat`, and userspace should
> > set :ref:`V4L2_META_FMT_VISCONTI_VIIF_STATS
> > <v4l2-meta-fmt-visconti-viif-stats>` as the data format.
> > > +
> > > +Feedback Operations
> > > +===================
> > > +
> > > +Among the so-called 3A functions, VIIF provides only auto-whitebalance
> > and auto-exposure.
> > > +Auto-whitebalance is a standalone hardware feature.
> > > +Some status information is available through the ISP statistics interface.
> > > +
> > > +Auto-exposure is realized through a combination of hardware and
> userland
> > software.
> > > +VIIF provides weighted average luminance information through the ISP
> > statistics interface.
> > > +The userland application calculates the sensor gain, sensor exposure and
> > ISP digital gain.
> > > +The calculated parameters are then passed to sensor's controls and the
> ISP
> > parameter interface.
> > > +
> > > +Among ISP parameters, there are parameters called AG (analog gain).
> > > +Actually, AG parameters have nothing to do with auto-exposure.
> > > +It controls "strength" in several signal correction algorithms.
> > > +Below is the list of the functions which may be affected by AG
> parameters:
> > > +
> > > +- ABPC/DPC
> > > +- RCNR
> > > +- LSSC
> > > +- MPRO: color matrix correction
> > > +- VPRO
> > > +
> > > +Capturing Video Frames Example
> > > +==============================
> > > +
> > > +In the following example,
> > > +imx219 camera is connected to pad 0 of 'visconti_csi2rx' subdevice.
> > > +
> > > +The following commands yield three pictures with different zoom ratio:
> > > +- main path 0: 1.5x zoom, crop 1920x1080, RGB picture
> > > +- main path 1: 0.67x zoom, crop 640x480, RGB picture
> > > +- sub path: 1920x1080 RAW picture
> > > +
> > > +.. code-block:: bash
> > > +
> > > + # set the links
> > > + media-ctl -d platform:visconti-viif-0 -r
> > > + media-ctl -d platform:visconti-viif-0 -l '"imx219 1-0010":0 ->
> > "visconti_csi2rx 1c008000.csi2rx":0 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"visconti_csi2rx
> > 1c008000.csi2rx":1 -> "visconti-viif:isp":0 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":1 ->
> > "visconti-viif:resizer0":0 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":2 ->
> > "visconti-viif:resizer1":0 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":3 ->
> > "viif_capture_sub":0 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer0":1 ->
> > "viif_capture_post0":0 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:resizer1":1 ->
> > "viif_capture_post1":0 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"viif_params":0 ->
> > "visconti-viif:isp":4 [1]'
> > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":5 ->
> > "viif_stats":0 [1]'
> > > +
> > > + # set format for imx219
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"imx219 1-0010":0
> > [fmt:SRGGB10_1X10/1920x1080]'
> > > +
> > > + # set format for csi2rx
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti_csi2rx
> > 1c008000.csi2rx":0 [fmt:SRGGB10_1X10/1920x1080 field:none
> > colorspace:raw xfer:none ycbcr:601 quantization:full-range]'
> > > +
> > > + # set format for isp sink pad
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":0
> > [fmt:SRGGB10_1X10/1920x1080]'
> > > +
> > > + # set format for resizer pads
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > '"visconti-viif:resizer0":0 '"[fmt:YUV8_1X24/1920x1080
> > compose:(0,0)/2880x1620]"
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > '"visconti-viif:resizer0":1 '"[crop:(480,16)/1920x1080]"
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > '"visconti-viif:resizer1":0 '"[fmt:YUV8_1X24/1920x1080
> > compose:(0,0)/1280x720]"
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > '"visconti-viif:resizer1":1 '"[crop:(320,32)/640x480]"
> > > +
> > > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"visconti-viif:isp":1
> > [fmt:YUV8_1X24/1024 crop:(640,0)/1024x1024]'
> > > +
> > > + # set format for main path0
> > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > "width=1920,height=1080"
> > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > "pixelformat=RGB3"
> > > +
> > > + # set format for main path1
> > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > "width=640,height=480"
> > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > "pixelformat=RGB3"
> > > +
> > > + # start streaming
> > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0
> > > +--stream-mmap --stream-count 1000 &
> > > +
> > > + # start streaming with other devices while viif_capture_post0 is
> > running
> > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post1
> > --stream-mmap --stream-count 10
> > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_sub
> > > +--stream-mmap --stream-count 10
> > > +
> > > +Use of coherent memory
> > > +======================
> > > +
> > > +Visconti5 SoC has two independent DDR SDRAM controllers.
> > > +Each controller is mapped to 36bit address space.
> > > +
> > > +Accelerator bus masters have two paths to access memory; one is
> > > +directly connected to SDRAM controller, the another is connected via
> > > +a cache coherency bus which keeps coherency among CPUs.
> > > +
> > > +From accelerators and CPUs, the address map is following:
> > > +
> > > +* 0x0_8000_0000 DDR0 direct access
> > > +* 0x4_8000_0000 DDR0 coherency bus
> > > +* 0x8_8000_0000 DDR1 direct access
> > > +* 0xC_8000_0000 DDR1 coherency bus
> > > +
> > > +The base address can be specified with "memory" and
> "reserved-memory"
> > > +elements in a device tree description.
> > > +It's not recommended to mix direct address and coherent address.
> > > +
> > > +The Visconti5 VIIF driver always use only direct address to configure
> Video
> > DMACs of the hardware.
> > > +This design is to avoid great performance loss at coherency bus caused by
> > massive memory access.
> > > +You should not put the dma_coherent attribute to viif element in device
> > tree.
> > > +Cache operations are done automatically by videobuf2 driver.
> > > diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > b/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > index 86ffb3bc8ade..2336842f0c26 100644
> > > --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > @@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata`
> interface
> > only.
> > > metafmt-pisp-fe
> > > metafmt-rkisp1
> > > metafmt-uvc
> > > + metafmt-visconti-viif
> > > metafmt-vivid
> > > metafmt-vsp1-hgo
> > > metafmt-vsp1-hgt
> > > diff --git
> > > a/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > > b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > > new file mode 100644
> > > index 000000000000..dc4b31627fe1
> > > --- /dev/null
> > > +++
> b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > > @@ -0,0 +1,48 @@
> > > +.. SPDX-License-Identifier: GPL-2.0
> > > +
> > > +.. _v4l2-meta-fmt-visconti-viif-params:
> > > +
> > > +.. _v4l2-meta-fmt-visconti-viif-stats:
> > > +
> > > +*********************************************************************
> > > +****************** V4L2_META_FMT_VISCONTI_VIIF_PARAMS ('vifp'),
> > > +V4L2_META_FMT_VISCONTI_VIIF_STATS ('vifs')
> > > +*********************************************************************
> > > +******************
> > > +
> > > +Configuration parameters
> > > +========================
> > > +
> > > +The configuration parameters are passed to the :ref:`viif_params
> > > +<viif_params>` metadata output video node, using the
> > > +:c:type:`v4l2_meta_format` interface. The buffer contains a single
> > > +instance of the C structure :c:type:`visconti_viif_isp_config`
> > > +defined in ``visconti_viif.h``. So the structure can be obtained from the
> > buffer by:
> > > +
> > > +.. code-block:: c
> > > +
> > > + struct visconti_viif_isp_config *params = (struct
> > > +visconti_viif_isp_config*) buffer;
> > > +
> > > +VIIF statistics
> > > +===============
> > > +
> > > +The VIIF device collects different statistics over an input Bayer frame.
> > > +Those statistics are obtained from the :ref:`viif_stats <viif_stats>`
> > > +metadata capture video node, using the :c:type:`v4l2_meta_format`
> > > +interface. The buffer contains a single instance of the C structure
> > > +:c:type:`visconti_viif_isp_stat` defined in ``visconti_viif.h``. So
> > > +the structure can be obtained from the buffer by:
> > > +
> > > +.. code-block:: c
> > > +
> > > + struct visconti_viif_isp_stat *stats = (struct
> > > +visconti_viif_isp_stat*) buffer;
> > > +
> > > +The statistics collected are Exposure, AWB (auto white balance) and
> errors.
> > > +See :c:type:`visconti_viif_isp_stat` for details of the statistics.
> > > +
> > > +The statistics and configuration parameters described here are
> > > +usually consumed and produced by dedicated user space libraries that
> > > +comprise the tuning tools using software control loop.
> > > +
> > > +visconti viif uAPI data types
> > > +=============================
> > > +
> > > +.. kernel-doc:: include/uapi/linux/visconti_viif.h
> >
> > Assuming the documentation in the header file is adequate, the level of detail
> > here is fine.
> >
> > --
> > Regards,
> >
> > Laurent Pinchart
>
> Regards,
>
> Yuji Ishikawa
Regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
* RE: [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver
2025-04-15 9:01 ` yuji2.ishikawa
@ 2025-06-18 1:30 ` yuji2.ishikawa
0 siblings, 0 replies; 34+ messages in thread
From: yuji2.ishikawa @ 2025-06-18 1:30 UTC (permalink / raw)
To: laurent.pinchart
Cc: mchehab, robh, krzk+dt, conor+dt, sakari.ailus, hverkuil-cisco,
nobuhiro1.iwamatsu, linux-media, linux-kernel, linux-arm-kernel,
devicetree
Hello Laurent,
I will begin the revision work on the VIIF driver v13 patch according to the following revision policy proposal submitted on 4/15.
If there are more appropriate methods or points to consider, especially regarding structural-level changes to the driver, please let me know.
> -----Original Message-----
> From: ishikawa yuji(石川 悠司 □ＡＩＤＣ○ＥＡ開)
> Sent: Tuesday, April 15, 2025 6:01 PM
> To: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor Dooley
> <conor+dt@kernel.org>; Sakari Ailus <sakari.ailus@linux.intel.com>; Hans
> Verkuil <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 □ＤＩＴＣ
> ○ＣＰＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>; linux-media@vger.kernel.org;
> linux-kernel@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> devicetree@vger.kernel.org
> Subject: RE: [PATCH v12 7/8] documentation: media: add documentation for
> Toshiba Visconti Video Input Interface driver
>
> Hello Laurent,
>
> As I reconsider the implementation of the v12 driver, I have decided on how to
> change the important parts pointed out in the review comments. On the other
> hand, I'm still looking for better implementation methods in some other areas.
> I would like to request your comments on these matters.
>
> - Topic 1: Removing the resizer subdevice
> - Topic 2: Adding new members to the parameter buffer
> - Topic 3: Passing error information via debugfs
> - Topic 4: Reconsidering the topology of the capture device (sub path)
>
> > -----Original Message-----
> > From: ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴＣ○ＥＡ開)
> > Sent: Wednesday, February 5, 2025 9:30 PM
> > To: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Cc: Mauro Carvalho Chehab <mchehab@kernel.org>; Rob Herring
> > <robh@kernel.org>; Krzysztof Kozlowski <krzk+dt@kernel.org>; Conor
> > Dooley <conor+dt@kernel.org>; Sakari Ailus
> > <sakari.ailus@linux.intel.com>; Hans Verkuil
> > <hverkuil-cisco@xs4all.nl>; iwamatsu nobuhiro(岩松 信洋 ○ＤＩＴＣ
> > □ＤＩＴ○ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> > linux-media@vger.kernel.org; linux-kernel@vger.kernel.org;
> > linux-arm-kernel@lists.infradead.org; devicetree@vger.kernel.org
> > Subject: RE: [PATCH v12 7/8] documentation: media: add documentation
> > for Toshiba Visconti Video Input Interface driver
> >
> > Hello Laurent
> >
> > Thank you for your review comments.
> >
>
> > > > + - Cropping; see :c:type:`viif_l2_roi`
> > >
> > > Looking at the implementation of the resizer subdevs, I see that the
> > > crop and compose rectangles can be set, but they don't seem to ne
> > > used to configure
> > the
> > > resizer. Instead, the viif_l2_roi_config parameters are used to
> > > configure cropping on the resizer input and scaling. This
> > > discrepancy isn't good. I see
> > two
> > > options to fix it:
> > >
> > > - Keep configuring the resizer through viif_l2_roi_config and drop the
> resizer
> > > subdevs. This will simplify the driver. The main drawback is that it
> > > won't be possible to implement digital zoom (by changing the resizer
> > > configuration) asynchronously from the ISP parameters buffers, which
> > > can be useful to lower the latency of digital zoom.
> > >
> > > - Drop viif_l2_roi_config, and configure the resizer from the formats,
> > > crop and selection rectangles on the resizer subdev pads. This makes
> > > the driver more complex. The main advantage is that digital zoom will
> > > be configurable with a smaller latency, but the drawback is that it
> > > won't be possible (or it will be more difficult) to configure it
> > > synchronously with other ISP parameters.
> > >
> > > There are drivers in mainline that implement either of those
> > > options, so you
> > can
> > > pick the one you think is best.
> > >
> > > An additional issue is that the hardware seems to implement cropping
> > > on the output of the resizer only, not on the input. Given that the
> > > size of the images output by the ISP pipeline must stay constant
> > > during video capture
> > (otherwise
> > > there would be a risk of buffer overflow), modifying the crop
> > > rectangle on the output of the resizer is usually not allowed during
> > > streaming. I think this could be worked around by allowing
> > > modification only of the left and top
> > coordinates
> > > during streaming, but configuring everything through
> > > viif_l2_roi_config would likely be easier. In that case, you should
> > > probably extend viif_l2_roi_config
> > with
> > > the crop offsets.
> > >
> > > All of this reflects my current understanding of the ISP
> > > architecture, based on this document and on the driver code, so
> > > please correct me if there's anything
> > I
> > > misunderstood. We can discuss the different options further before
> > > you
> > modify
> > > the driver and send a new version.
> > >
> >
> > First of all, I apologize for attempting to make significant changes
> > to the driver structure without consulting you. The hardware
> > specifications of VIIF are unique, and the knowledge from the drivers
> > I referred to (mainly rkisp1) cannot be directly applied. As a result,
> > I am still struggling with how to write a driver that conforms to the V4L2
> framework.
> >
> > I am considering removing the resizer subdevice because
> > viif_l2_roi_config can set more detailed parameters compared to the resizer
> subdevice's pad.
> >
> > To explain my idea and answer your questions, let me describe the
> > processing of the VIIF L2-ISP.
> >
> > The central functions of the L2 ISP are lens undistortion and scaling.
> > They are integrated and executed simultaneously. To explain the
> > coordinate transformation here, several virtual images are defined.
> >
> > * L1ISP output image
> > * the input of L2ISP
> > * geometry: {width, height}
> > * mapped to sink.format
> > * Input image
> > * geometry: {center_x, center_y, width, height}
> > * relative to the L1ISP output image
> > * mapped to sink.crop
> > * resizer subdevice assumes that sink.crop = sink.format
> > * Corrected image after undistortion
> > * geometry: {width, height}
> > * relative to the Input image
> > * Corrected image with scale
> > * geometry: {width, height}
> > * relative to the Input image
> > * mapped to sink.compose
> > * resizer subdevice assumes that scale factor = sink.compose /
> > sink.format
> > * ROI image
> > * geometry: {left, top, width, height}
> > * relative to the Corrected image with scale
> > * mapped to source.crop
> > * Crop image
> > * the output of L2ISP.
> > * geometry: {left, top, width, height}
> > * relative to the Corrected image with scale
> > * mapped to source.crop
> >
> > These images mentioned here do not have corresponding frame buffers.
> > However,
> > there are control registers corresponding to the width and height of
> > each image, and appropriate values must be set. Otherwise, the output
> > image may be corrupted, or the L2ISP may abort.
> >
> > Regarding your question of why sink.crop is not used.
> > According to the list above, sink.crop corresponds to the Input image.
> > However, the coordinates of the Input image must be set to align with
> > the optical center for undistortion, so it cannot be used to crop arbitrary
> rectangles.
> >
> > Regarding digital zoom.
> > In the existing use cases of VIIF, the scale and crop are fixed during recording.
> > Therefore, I did not consider changing the selection parameter like digital
> zoom.
> >
> > Regarding resizer subdevice.
> > The purpose of the resizer subdevice is to set the initial values of
> > L2ISP from pad's selection parameters. Most virtual image rectangles
> > can be mapped to the pad's parameters. Since the parameters of
> > undistortion and the rectangle of the Corrected image after
> > undistortion are unknown, I assumed that undistortion is disabled
> > (pass-through) and the Corrected image after undistortion matches the
> > Input image. In this case, the scaling parameters can be calculated
> > from sink.compose and sink.input.
> > On the other hand, in cases where both undistortion and scaling are
> > utilized, it is necessary to set viif_l2_undist_config and
> > viif_l2_roi_config. Therefore, I made it possible to overwrite these parameters
> through the parameter buffer.
> >
> > To configure cropping with the parameter buffer instead of pad's
> > selection, I'll add two members (left and top) to viif_l2_roi_config.
> > Otherwise, I'll newly add struct viif_l2_crop_config to the parameter buffer.
> >
>
> I am removing the resizer sub-device. In the new implementation, the
> undistortion and scaling functions of L2ISP are controlled using the l2_undist
> member and l2_roi member of the parameter buffer. To control the cropping
> function of L2ISP, the l2_crop_post0 and l2_crop_post1 members are added to
> the parameter buffer to indicate the top-left coordinates for cropping.
> These new members have the type struct l2_crop_config {__u32 left; __u32 top}.
> The default values of the members are (0, 0).
>
> ISP subdevice pads have the following specifications:
> - The sink_pad::fmt and source_pad(post0)::fmt, source_pad(post1)::fmt can
> be set independently.
> - Each pad cannot be set for crop and compose.
> - The maximum size of source_pad::fmt is either the hardware limit of L2ISP or
> 2x sink_pad::fmt.
> - The minimum size of source_pad::fmt is the hardware limit of L2ISP.
> - Due to the inclusion relationship of the planes, the following constraint should
> be met:
>
> l2_crop_post0::left + source_pad(post0)::fmt::width <=
> l2_roi::corrected_width(post0)
>
> A similar constraint should be applied to the height and the post1.
>
> There are two possible behaviors when constraints are not met. If the
> instructions through the parameter buffer are not accepted, how should the
> driver behave?
>
> - Increment the error counter (referred by debugfs) and ignore the instructions
> from the parameter buffer.
> - Modify l2_roi::corrected_width(post0) and l2_crop_post0::left and apply them
> to the hardware. As a result, an L2ISP error might occur. The drawback is that
> the user program cannot know the modified coordinates.
>
> > > > +.. _viif_params:
> > > > +
> > > > +viif_params - ISP Parameters Video Node
> > > > +---------------------------------------
> > > > +
> > > > +The viif_params video node receives a set of ISP parameters from
> > > > +userspace to be applied to the hardware during a video stream.
> > > > +
> > > > +The buffer format is defined by struct
> > > > +:c:type:`visconti_viif_isp_config`, and userspace should
> > > set :ref:`V4L2_META_FMT_VISCONTI_VIIF_PARAMS
> > > <v4l2-meta-fmt-visconti-viif-params>` as the data format.
> > > > +
>
> Please advise on how to handle unverified ISP functions in the parameter
> buffer.
>
> The L1ISP hardware has a function to calculate the luminance histogram.
> The software for this function is experimental and has not yet been developed
> as a Linux driver. However, since the structure that the parameter buffer
> handles comprehensively represents all ISP functions, I understand that it
> would be difficult to add members for the histogram function later.
>
> Should members corresponding to the histogram function be added to the
> parameter buffer structure even if the implementation does not exist? Or, is it
> permissible to add new members to the parameter buffer structure based on
> certain conditions?
>
> > > > +.. _viif_stats:
> > > > +
> > > > +viif_stats - Statistics Video Node
> > > > +----------------------------------
> > > > +
> > > > +The viif_stats video node provides current status of ISP.
> > >
> > > The viif_stats video node provides statistics computed by the ISP
> > > and frame processing status.
> > >
> >
> > I'll update the sentence as you suggested.
> >
> > > > +
> > > > +Following information is included:
> > > > +
> > > > +* statistics of auto white balance
> > > > +* average luminance information which can be used by auto
> > > > +exposure
> > > software impl.
> > >
> > > s/impl/implementation/
> > >
> >
> > I should not have abbreviated a word.
> > I'll fix it.
> >
> > >
> > > I would also add
> > >
> > > * CSI-2 receiver calibration and error status
> > > * ISP error status
> > >
> > > It's quite uncommon to provide this kind of status through ISP stats
> > > buffers,
> > but
> > > it sounds like an interesting idea. Other drivers usually keep error
> > > counters in the kernel and expose them through debugfs.
> > >
> >
> > These status information have existed since the initial implementation
> > and must be passed to the user space in some way. On the other hand, I
> > think that these, especially the CSI2 receiver error status, are not
> > related to the frame-by-frame processing of the ISP, so they should
> > not be included in the parameter buffer. I should classify the
> > information and update the members of the parameter buffer
> > accordingly. I think some of the information should be exposed through
> > debugfs. However, I'm not sure how to classify them and how to
> > implement the debugfs interface. Could you tell me the source code
> > that should be referred to?
> >
>
> In the new implementation, the following information will be passed to user
> space using debugfs:
>
> - Errors during capture (originally, struct viif_reported_errors)
> - Counters will be prepared for each error cause. This will be implemented
> using debugfs_create_ulong().
> - The original specification was to "clear on read", but since it seems
> difficult to implement concisely, it will be changed to a count-up operation.
> - CSI2RX calibration status
> - Information in three states (OK, Not-done, Error) for each signal line.
> This will be represented using debugfs_create_x8() or
> debugfs_create_file()
> in a human-readable format.
> - CSI2RX error status
> - Bit vector information for each error cause. This will be represented using
> debugfs_create_x32().
>
> The following information will remain in the parameter buffer's statistics
> interface:
>
> - Average luminance information
> - Auto white balance information
>
> --------------------
>
> I would like to reconsider the connection between the Capture device (sub) and
> other subdevices. Could you provide some feedback?
>
> Currently, the Capture device (sub) is connected to the ISP subdevice.
> This topology is based on my best understanding of the media controller
> framework at the time of design. However, the Capture device (sub) saves RAW
> images without processing them, and its operation does not depend on ISP
> functions. Therefore, for better maintenance in the future, I am considering
> disconnecting the Capture device (sub) from the ISP subdevice.
>
> Additionally, I want to prepare for improvements regarding streams. The
> CSI2RX hardware can handle multiple virtual channels. Both the ISP hardware
> and Capture hardware (sub) can each select and receive one virtual channel.
> Currently, due to insufficient tests, both are limited to receiving virtual channel
> 0, but in the future, I want to be able to handle multiple channels using the
> stream API.
>
> I'm considering three options for the topology.
>
> - Option 1: Connect the Capture device (sub) to the ISP subdevice as it is
> currently.
> The switching of streams will be realized as a function of the ISP subdevice.
> - Option 2: Create a new MUX subdevice and connect the ISP subdevice and
> Capture device (sub) to it. When handling multiple streams in the future,
> the switching function will be realized by the MUX subdevice. This will
> reduce
> the complexity of the ISP subdevice implementation.
> - Option 3: Connect both the ISP subdevice and Capture device (sub) to
> the source port of CSI2RX. Is such a configuration really possible? The
> switching
> of multiple streams will be handled by both the ISP subdevice and Capture
> device respectively.
>
>
> [Option1: Current design]
> CSI2RX|0 -- ISP|1 -- Capture(post0)
> |2 -- Capture(post1)
> |3 -- Capture(sub)
>
> [Option2]
> CSI2RX|0 -- MUX|1 -- ISP|1 -- Capture(post0)
> | |2 -- Capture(post1)
> |2 -- Capture(sub)
>
> [Option3]
> CSI2RX|0 --+-- ISP|1 -- Capture(post0)
> | |2 -- Capture(post1)
> +-- Capture(sub)
>
> > > > +
> > > > +The buffer format is defined by struct
> > > > +:c:type:`visconti_viif_isp_stat`, and userspace should
> > > set :ref:`V4L2_META_FMT_VISCONTI_VIIF_STATS
> > > <v4l2-meta-fmt-visconti-viif-stats>` as the data format.
> > > > +
> > > > +Feedback Operations
> > > > +===================
> > > > +
> > > > +Among the so-called 3A functions, VIIF provides only
> > > > +auto-whitebalance
> > > and auto-exposure.
> > > > +Auto-whitebalance is a standalone hardware feature.
> > > > +Some status information is available through the ISP statistics
> interface.
> > > > +
> > > > +Auto-exposure is realized through a combination of hardware and
> > userland
> > > software.
> > > > +VIIF provides weighted average luminance information through the
> > > > +ISP
> > > statistics interface.
> > > > +The userland application calculates the sensor gain, sensor
> > > > +exposure and
> > > ISP digital gain.
> > > > +The calculated parameters are then passed to sensor's controls
> > > > +and the
> > ISP
> > > parameter interface.
> > > > +
> > > > +Among ISP parameters, there are parameters called AG (analog gain).
> > > > +Actually, AG parameters have nothing to do with auto-exposure.
> > > > +It controls "strength" in several signal correction algorithms.
> > > > +Below is the list of the functions which may be affected by AG
> > parameters:
> > > > +
> > > > +- ABPC/DPC
> > > > +- RCNR
> > > > +- LSSC
> > > > +- MPRO: color matrix correction
> > > > +- VPRO
> > > > +
> > > > +Capturing Video Frames Example
> > > > +==============================
> > > > +
> > > > +In the following example,
> > > > +imx219 camera is connected to pad 0 of 'visconti_csi2rx' subdevice.
> > > > +
> > > > +The following commands yield three pictures with different zoom ratio:
> > > > +- main path 0: 1.5x zoom, crop 1920x1080, RGB picture
> > > > +- main path 1: 0.67x zoom, crop 640x480, RGB picture
> > > > +- sub path: 1920x1080 RAW picture
> > > > +
> > > > +.. code-block:: bash
> > > > +
> > > > + # set the links
> > > > + media-ctl -d platform:visconti-viif-0 -r
> > > > + media-ctl -d platform:visconti-viif-0 -l '"imx219 1-0010":0 ->
> > > "visconti_csi2rx 1c008000.csi2rx":0 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l '"visconti_csi2rx
> > > 1c008000.csi2rx":1 -> "visconti-viif:isp":0 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":1
> > > > +->
> > > "visconti-viif:resizer0":0 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":2
> > > > +->
> > > "visconti-viif:resizer1":0 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":3
> > > > +->
> > > "viif_capture_sub":0 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l
> > > > +'"visconti-viif:resizer0":1 ->
> > > "viif_capture_post0":0 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l
> > > > +'"visconti-viif:resizer1":1 ->
> > > "viif_capture_post1":0 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l '"viif_params":0 ->
> > > "visconti-viif:isp":4 [1]'
> > > > + media-ctl -d platform:visconti-viif-0 -l '"visconti-viif:isp":5
> > > > +->
> > > "viif_stats":0 [1]'
> > > > +
> > > > + # set format for imx219
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2 '"imx219
> > > > +1-0010":0
> > > [fmt:SRGGB10_1X10/1920x1080]'
> > > > +
> > > > + # set format for csi2rx
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > > > +'"visconti_csi2rx
> > > 1c008000.csi2rx":0 [fmt:SRGGB10_1X10/1920x1080 field:none
> > > colorspace:raw xfer:none ycbcr:601 quantization:full-range]'
> > > > +
> > > > + # set format for isp sink pad
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > > > +'"visconti-viif:isp":0
> > > [fmt:SRGGB10_1X10/1920x1080]'
> > > > +
> > > > + # set format for resizer pads
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > > '"visconti-viif:resizer0":0 '"[fmt:YUV8_1X24/1920x1080
> > > compose:(0,0)/2880x1620]"
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > > '"visconti-viif:resizer0":1 '"[crop:(480,16)/1920x1080]"
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > > '"visconti-viif:resizer1":0 '"[fmt:YUV8_1X24/1920x1080
> > > compose:(0,0)/1280x720]"
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > > '"visconti-viif:resizer1":1 '"[crop:(320,32)/640x480]"
> > > > +
> > > > + media-ctl -d platform:visconti-viif-0 --set-v4l2
> > > > +'"visconti-viif:isp":1
> > > [fmt:YUV8_1X24/1024 crop:(640,0)/1024x1024]'
> > > > +
> > > > + # set format for main path0
> > > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > > "width=1920,height=1080"
> > > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > > "pixelformat=RGB3"
> > > > +
> > > > + # set format for main path1
> > > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > > "width=640,height=480"
> > > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0 -v
> > > "pixelformat=RGB3"
> > > > +
> > > > + # start streaming
> > > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post0
> > > > +--stream-mmap --stream-count 1000 &
> > > > +
> > > > + # start streaming with other devices while viif_capture_post0 is
> > > running
> > > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_post1
> > > --stream-mmap --stream-count 10
> > > > + v4l2-ctl -z platform:visconti-viif-0 -d viif_capture_sub
> > > > +--stream-mmap --stream-count 10
> > > > +
> > > > +Use of coherent memory
> > > > +======================
> > > > +
> > > > +Visconti5 SoC has two independent DDR SDRAM controllers.
> > > > +Each controller is mapped to 36bit address space.
> > > > +
> > > > +Accelerator bus masters have two paths to access memory; one is
> > > > +directly connected to SDRAM controller, the another is connected
> > > > +via a cache coherency bus which keeps coherency among CPUs.
> > > > +
> > > > +From accelerators and CPUs, the address map is following:
> > > > +
> > > > +* 0x0_8000_0000 DDR0 direct access
> > > > +* 0x4_8000_0000 DDR0 coherency bus
> > > > +* 0x8_8000_0000 DDR1 direct access
> > > > +* 0xC_8000_0000 DDR1 coherency bus
> > > > +
> > > > +The base address can be specified with "memory" and
> > "reserved-memory"
> > > > +elements in a device tree description.
> > > > +It's not recommended to mix direct address and coherent address.
> > > > +
> > > > +The Visconti5 VIIF driver always use only direct address to
> > > > +configure
> > Video
> > > DMACs of the hardware.
> > > > +This design is to avoid great performance loss at coherency bus
> > > > +caused by
> > > massive memory access.
> > > > +You should not put the dma_coherent attribute to viif element in
> > > > +device
> > > tree.
> > > > +Cache operations are done automatically by videobuf2 driver.
> > > > diff --git
> > > > a/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > > b/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > > index 86ffb3bc8ade..2336842f0c26 100644
> > > > --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
> > > > @@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata`
> > interface
> > > only.
> > > > metafmt-pisp-fe
> > > > metafmt-rkisp1
> > > > metafmt-uvc
> > > > + metafmt-visconti-viif
> > > > metafmt-vivid
> > > > metafmt-vsp1-hgo
> > > > metafmt-vsp1-hgt
> > > > diff --git
> > > > a/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > > > b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > > > new file mode 100644
> > > > index 000000000000..dc4b31627fe1
> > > > --- /dev/null
> > > > +++
> > b/Documentation/userspace-api/media/v4l/metafmt-visconti-viif.rst
> > > > @@ -0,0 +1,48 @@
> > > > +.. SPDX-License-Identifier: GPL-2.0
> > > > +
> > > > +.. _v4l2-meta-fmt-visconti-viif-params:
> > > > +
> > > > +.. _v4l2-meta-fmt-visconti-viif-stats:
> > > > +
> > > > +*****************************************************************
> > > > +****
> > > > +****************** V4L2_META_FMT_VISCONTI_VIIF_PARAMS ('vifp'),
> > > > +V4L2_META_FMT_VISCONTI_VIIF_STATS ('vifs')
> > > > +*****************************************************************
> > > > +****
> > > > +******************
> > > > +
> > > > +Configuration parameters
> > > > +========================
> > > > +
> > > > +The configuration parameters are passed to the :ref:`viif_params
> > > > +<viif_params>` metadata output video node, using the
> > > > +:c:type:`v4l2_meta_format` interface. The buffer contains a
> > > > +single instance of the C structure
> > > > +:c:type:`visconti_viif_isp_config`
> > > > +defined in ``visconti_viif.h``. So the structure can be obtained
> > > > +from the
> > > buffer by:
> > > > +
> > > > +.. code-block:: c
> > > > +
> > > > + struct visconti_viif_isp_config *params = (struct
> > > > +visconti_viif_isp_config*) buffer;
> > > > +
> > > > +VIIF statistics
> > > > +===============
> > > > +
> > > > +The VIIF device collects different statistics over an input Bayer frame.
> > > > +Those statistics are obtained from the :ref:`viif_stats
> > > > +<viif_stats>` metadata capture video node, using the
> > > > +:c:type:`v4l2_meta_format` interface. The buffer contains a
> > > > +single instance of the C structure
> > > > +:c:type:`visconti_viif_isp_stat` defined in ``visconti_viif.h``. So the
> structure can be obtained from the buffer by:
> > > > +
> > > > +.. code-block:: c
> > > > +
> > > > + struct visconti_viif_isp_stat *stats = (struct
> > > > +visconti_viif_isp_stat*) buffer;
> > > > +
> > > > +The statistics collected are Exposure, AWB (auto white balance)
> > > > +and
> > errors.
> > > > +See :c:type:`visconti_viif_isp_stat` for details of the statistics.
> > > > +
> > > > +The statistics and configuration parameters described here are
> > > > +usually consumed and produced by dedicated user space libraries
> > > > +that comprise the tuning tools using software control loop.
> > > > +
> > > > +visconti viif uAPI data types
> > > > +=============================
> > > > +
> > > > +.. kernel-doc:: include/uapi/linux/visconti_viif.h
> > >
> > > Assuming the documentation in the header file is adequate, the level
> > > of detail here is fine.
> > >
> > > --
> > > Regards,
> > >
> > > Laurent Pinchart
> >
> > Regards,
> >
> > Yuji Ishikawa
>
> Regards,
>
> Yuji Ishikawa
Regards,
Yuji Ishikawa
^ permalink raw reply [flat|nested] 34+ messages in thread
end of thread, other threads:[~2025-06-18 1:41 UTC | newest]
Thread overview: 34+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-11-25 9:21 [PATCH v12 0/8] Add Toshiba Visconti Video Input Interface driver Yuji Ishikawa
2024-11-25 9:21 ` [PATCH v12 1/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti MIPI CSI-2 Receiver Yuji Ishikawa
2024-11-25 10:11 ` Krzysztof Kozlowski
2024-12-16 23:57 ` yuji2.ishikawa
2024-12-17 5:44 ` Krzysztof Kozlowski
2024-12-24 0:17 ` yuji2.ishikawa
2024-12-26 16:15 ` Laurent Pinchart
2025-01-02 9:29 ` Laurent Pinchart
2025-01-20 0:10 ` yuji2.ishikawa
2024-11-25 9:21 ` [PATCH v12 2/8] dt-bindings: media: platform: visconti: Add Toshiba Visconti Video Input Interface Yuji Ishikawa
2024-11-25 10:07 ` Krzysztof Kozlowski
2024-12-17 0:00 ` yuji2.ishikawa
2024-12-17 5:43 ` Krzysztof Kozlowski
2024-12-17 9:45 ` Laurent Pinchart
2024-12-17 12:51 ` Krzysztof Kozlowski
2025-01-02 9:41 ` Laurent Pinchart
2024-12-24 0:17 ` yuji2.ishikawa
2025-01-02 9:56 ` Laurent Pinchart
2025-01-20 0:13 ` yuji2.ishikawa
2024-11-25 9:21 ` [PATCH v12 3/8] media: uapi: add visconti viif meta buffer format Yuji Ishikawa
2025-01-02 13:10 ` Laurent Pinchart
2025-01-20 0:15 ` yuji2.ishikawa
2025-01-20 1:49 ` Laurent Pinchart
2024-11-25 9:21 ` [PATCH v12 4/8] media: platform: visconti: Add Toshiba Visconti CSI-2 Receiver driver Yuji Ishikawa
2025-01-02 13:08 ` Laurent Pinchart
2025-01-20 0:21 ` yuji2.ishikawa
2024-11-25 9:21 ` [PATCH v12 7/8] documentation: media: add documentation for Toshiba Visconti Video Input Interface driver Yuji Ishikawa
2025-01-02 21:26 ` Laurent Pinchart
2025-02-05 12:29 ` yuji2.ishikawa
2025-04-15 9:01 ` yuji2.ishikawa
2025-06-18 1:30 ` yuji2.ishikawa
2024-11-25 9:21 ` [PATCH v12 8/8] MAINTAINERS: Add entries for Toshiba Visconti Video Input Interface Yuji Ishikawa
2025-01-02 13:16 ` Laurent Pinchart
2025-02-05 12:31 ` yuji2.ishikawa
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).