* [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface
@ 2023-10-16 9:00 Mehdi Djait
2023-10-16 9:00 ` [PATCH v8 1/3] media: dt-bindings: media: add bindings for Rockchip VIP Mehdi Djait
` (3 more replies)
0 siblings, 4 replies; 19+ messages in thread
From: Mehdi Djait @ 2023-10-16 9:00 UTC (permalink / raw)
To: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt,
conor+dt, ezequiel
Cc: linux-media, devicetree, linux-kernel, thomas.petazzoni,
alexandre.belloni, maxime.chevallier, paul.kocialkowski,
Mehdi Djait
Hello everyone,
V8 for basic support of the Video Input Processor found on the Rockchip PX30 SoC
The v6 is based on the fifth iteration of the series introducing the
driver: sent 29 Dec 2020 [1]
Most of this driver was written following the BSP driver from rockchip,
removing the parts that either didn't fit correctly the guidelines, or
that couldn't be tested.
In the BSP, this driver is known as the "cif" driver, but this was
renamed to "vip" to better fit the controller denomination in the
datasheet.
This version of the driver supports ONLY the parallel interface BT656
and was tested/implemented using an SDTV video decoder
media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235
V7 => V8:
vip/capture.c:
- fixed a warning: unused variable reported by the kernel test robot
V6 => V7:
vip/capture.c vip/dev.c vip/dev.h
- renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev
- added some error when rk_vip_get_buffer() returns NULL
- removed a WARN_ON
- made the irq NOT shared
- dropped of_match_ptr
- added the rk_vip_get_resource() function
rockchip,px30-vip.yaml:
- changed filename to match the compatible
- dropped the mention of the other rockchip SoC in the dt-binding
description and added a more detailed description of VIP
- removed unused labels in the example
V5 [1] => V6:
vip/capture.c vip/dev.c vip/dev.h
- added a video g_input_status subdev call, V4L2_IN_CAP_STD and the
supported stds in rk_vip_enum_input callback
- added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks
- added the supported video_device->tvnorms
- s_std will now update the format as this depends on the standard
NTSC/PAL (as suggested by Hans in [1])
- removed STD_ATSC
- moved the colorimetry information to come from the subdev
- removed the core s_power subdev calls
- dropped cropping in rk_vip_stream struct
rockchip-vip.yaml:
- fixed a mistake in the name of third clock plckin -> plck
- changed the reg maxItems 2 -> 1
[1] https://lore.kernel.org/linux-media/20201229161724.511102-1-maxime.chevallier@bootlin.com/
I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b
# v4l2-compliance
v4l2-compliance 1.25.0, 64 bits, 64-bit time_t
Compliance test for rk_vip device /dev/video0:
Driver Info:
Driver name : rk_vip
Card type : rk_vip
Bus info : platform:ff490000.vip
Driver version : 6.6.0
Capabilities : 0x84201000
Video Capture Multiplanar
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04201000
Video Capture Multiplanar
Streaming
Extended Pix Format
Media Driver Info:
Driver name : rk_vip
Model : rk_vip
Serial :
Bus info : platform:ff490000.vip
Media version : 6.6.0
Hardware revision: 0x00000000 (0)
Driver version : 6.6.0
Interface Info:
ID : 0x03000002
Type : V4L Video
Entity Info:
ID : 0x00000001 (1)
Name : video_rkvip
Function : V4L2 I/O
Pad 0x01000004 : 0: Sink
Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0
Mehdi Djait (3):
media: dt-bindings: media: add bindings for Rockchip VIP
media: rockchip: Add a driver for Rockhip's camera interface
arm64: dts: rockchip: Add the camera interface
.../bindings/media/rockchip,px30-vip.yaml | 93 ++
arch/arm64/boot/dts/rockchip/px30.dtsi | 12 +
drivers/media/platform/rockchip/Kconfig | 1 +
drivers/media/platform/rockchip/Makefile | 1 +
drivers/media/platform/rockchip/vip/Kconfig | 14 +
drivers/media/platform/rockchip/vip/Makefile | 3 +
drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++
drivers/media/platform/rockchip/vip/dev.c | 346 +++++
drivers/media/platform/rockchip/vip/dev.h | 163 +++
drivers/media/platform/rockchip/vip/regs.h | 260 ++++
10 files changed, 2103 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml
create mode 100644 drivers/media/platform/rockchip/vip/Kconfig
create mode 100644 drivers/media/platform/rockchip/vip/Makefile
create mode 100644 drivers/media/platform/rockchip/vip/capture.c
create mode 100644 drivers/media/platform/rockchip/vip/dev.c
create mode 100644 drivers/media/platform/rockchip/vip/dev.h
create mode 100644 drivers/media/platform/rockchip/vip/regs.h
--
2.41.0
^ permalink raw reply [flat|nested] 19+ messages in thread* [PATCH v8 1/3] media: dt-bindings: media: add bindings for Rockchip VIP 2023-10-16 9:00 [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface Mehdi Djait @ 2023-10-16 9:00 ` Mehdi Djait 2023-10-16 9:00 ` [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface Mehdi Djait ` (2 subsequent siblings) 3 siblings, 0 replies; 19+ messages in thread From: Mehdi Djait @ 2023-10-16 9:00 UTC (permalink / raw) To: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel Cc: linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier, paul.kocialkowski, Mehdi Djait, Rob Herring Add a documentation for the Rockchip Video Input Processor binding. The PX30 SoC is the only platform supported so far. Reviewed-by: Rob Herring <robh@kernel.org> Signed-off-by: Mehdi Djait <mehdi.djait@bootlin.com> --- .../bindings/media/rockchip,px30-vip.yaml | 93 +++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml diff --git a/Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml b/Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml new file mode 100644 index 000000000000..41f0cd58372d --- /dev/null +++ b/Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml @@ -0,0 +1,93 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/rockchip,px30-vip.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Rockchip VIP Camera Interface + +maintainers: + - Mehdi Djait <mehdi.djait@bootlin.com> + +description: |- + Rockchip VIP is the Video Input Processor of the rockchip PX30 SoC. It + receives the data from Camera sensor or CCIR656 encoder and transfers it into + system main memory by AXI bus. + +properties: + compatible: + const: rockchip,px30-vip + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: ACLK + - description: HCLK + - description: PCLK + + clock-names: + items: + - const: aclk + - const: hclk + - const: pclk + + resets: + items: + - description: AXI + - description: AHB + - description: PCLK IN + + reset-names: + items: + - const: axi + - const: ahb + - const: pclkin + + power-domains: + maxItems: 1 + + port: + $ref: /schemas/graph.yaml#/properties/port + description: A connection to a sensor or decoder + +required: + - compatible + - reg + - interrupts + - clocks + - port + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/clock/px30-cru.h> + #include <dt-bindings/power/px30-power.h> + + parent { + #address-cells = <2>; + #size-cells = <2>; + + vip@ff490000 { + compatible = "rockchip,px30-vip"; + reg = <0x0 0xff490000 0x0 0x200>; + interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_CIF>, <&cru HCLK_CIF>, <&cru PCLK_CIF>; + clock-names = "aclk", "hclk", "pclk"; + resets = <&cru SRST_CIF_A>, <&cru SRST_CIF_H>, <&cru SRST_CIF_PCLKIN>; + reset-names = "axi", "ahb", "pclkin"; + power-domains = <&power PX30_PD_VI>; + port { + endpoint { + remote-endpoint = <&tw9900_out>; + }; + }; + }; + }; +... -- 2.41.0 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-16 9:00 [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface Mehdi Djait 2023-10-16 9:00 ` [PATCH v8 1/3] media: dt-bindings: media: add bindings for Rockchip VIP Mehdi Djait @ 2023-10-16 9:00 ` Mehdi Djait 2023-10-19 15:40 ` Paul Kocialkowski ` (2 more replies) 2023-10-16 9:00 ` [PATCH v8 3/3] arm64: dts: rockchip: Add the " Mehdi Djait 2023-10-19 15:33 ` [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's " Paul Kocialkowski 3 siblings, 3 replies; 19+ messages in thread From: Mehdi Djait @ 2023-10-16 9:00 UTC (permalink / raw) To: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel Cc: linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier, paul.kocialkowski, Mehdi Djait Introduce a driver for the camera interface on some Rockchip platforms. This controller supports CSI2 and BT656 interfaces, but for now only the BT656 interface could be tested, hence it's the only one that's supported in the first version of this driver. This controller can be fond on PX30, RK1808, RK3128 and RK3288, but for now it's only been tested on PX30. Most of this driver was written following the BSP driver from rockchip, removing the parts that either didn't fit correctly the guidelines, or that couldn't be tested. In the BSP, this driver is known as the "cif" driver, but this was renamed to "vip" to better fit the controller denomination in the datasheet. This basic version doesn't support cropping nor scaling, and is only designed with one SDTV video decoder being attached to it a any time. This version uses the "pingpong" mode of the controller, which is a double-buffering mechanism. Signed-off-by: Mehdi Djait <mehdi.djait@bootlin.com> --- drivers/media/platform/rockchip/Kconfig | 1 + drivers/media/platform/rockchip/Makefile | 1 + drivers/media/platform/rockchip/vip/Kconfig | 14 + drivers/media/platform/rockchip/vip/Makefile | 3 + drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++ drivers/media/platform/rockchip/vip/dev.c | 346 +++++ drivers/media/platform/rockchip/vip/dev.h | 163 +++ drivers/media/platform/rockchip/vip/regs.h | 260 ++++ 8 files changed, 1998 insertions(+) create mode 100644 drivers/media/platform/rockchip/vip/Kconfig create mode 100644 drivers/media/platform/rockchip/vip/Makefile create mode 100644 drivers/media/platform/rockchip/vip/capture.c create mode 100644 drivers/media/platform/rockchip/vip/dev.c create mode 100644 drivers/media/platform/rockchip/vip/dev.h create mode 100644 drivers/media/platform/rockchip/vip/regs.h diff --git a/drivers/media/platform/rockchip/Kconfig b/drivers/media/platform/rockchip/Kconfig index b41d3960c1b4..f82a56d1431f 100644 --- a/drivers/media/platform/rockchip/Kconfig +++ b/drivers/media/platform/rockchip/Kconfig @@ -4,3 +4,4 @@ comment "Rockchip media platform drivers" source "drivers/media/platform/rockchip/rga/Kconfig" source "drivers/media/platform/rockchip/rkisp1/Kconfig" +source "drivers/media/platform/rockchip/vip/Kconfig" diff --git a/drivers/media/platform/rockchip/Makefile b/drivers/media/platform/rockchip/Makefile index 4f782b876ac9..3f1dade70103 100644 --- a/drivers/media/platform/rockchip/Makefile +++ b/drivers/media/platform/rockchip/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += rga/ obj-y += rkisp1/ +obj-y += vip/ diff --git a/drivers/media/platform/rockchip/vip/Kconfig b/drivers/media/platform/rockchip/vip/Kconfig new file mode 100644 index 000000000000..05953e12a2a8 --- /dev/null +++ b/drivers/media/platform/rockchip/vip/Kconfig @@ -0,0 +1,14 @@ +config VIDEO_ROCKCHIP_VIP + tristate "Rockchip VIP (Video InPut) Camera Interface" + depends on VIDEO_DEV + depends on ARCH_ROCKCHIP || COMPILE_TEST + select VIDEOBUF2_DMA_SG + select VIDEOBUF2_DMA_CONTIG + select V4L2_FWNODE + select V4L2_MEM2MEM_DEV + help + This is a v4l2 driver for Rockchip SOC Camera interface. It supports + BT.656 and CSI2 inputs. + + To compile this driver as a module choose m here : the module will + be called video_rkvip. diff --git a/drivers/media/platform/rockchip/vip/Makefile b/drivers/media/platform/rockchip/vip/Makefile new file mode 100644 index 000000000000..c239ee0bb0fe --- /dev/null +++ b/drivers/media/platform/rockchip/vip/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_VIDEO_ROCKCHIP_VIP) += video_rkvip.o +video_rkvip-objs += dev.o capture.o diff --git a/drivers/media/platform/rockchip/vip/capture.c b/drivers/media/platform/rockchip/vip/capture.c new file mode 100644 index 000000000000..e8f3480aacdb --- /dev/null +++ b/drivers/media/platform/rockchip/vip/capture.c @@ -0,0 +1,1210 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Rockchip VIP Camera Interface Driver + * + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. + * Copyright (C) 2020 Maxime Chevallier <maxime.chevallier@bootlin.com> + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> + */ + +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/pm_runtime.h> +#include <linux/reset.h> +#include <media/v4l2-common.h> +#include <media/v4l2-event.h> +#include <media/v4l2-fh.h> +#include <media/v4l2-fwnode.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-mc.h> +#include <media/v4l2-subdev.h> +#include <media/videobuf2-dma-contig.h> + +#include "dev.h" +#include "regs.h" + +#define VIP_REQ_BUFS_MIN 3 +#define VIP_MIN_WIDTH 64 +#define VIP_MIN_HEIGHT 64 +#define VIP_MAX_WIDTH 8192 +#define VIP_MAX_HEIGHT 8192 + +#define RK_VIP_PLANE_Y 0 +#define RK_VIP_PLANE_CBCR 1 + +#define VIP_FETCH_Y_LAST_LINE(VAL) ((VAL) & 0x1fff) + +static struct vip_output_fmt out_fmts[] = { + { + .fourcc = V4L2_PIX_FMT_NV16, + .fmt_val = VIP_FORMAT_YUV_OUTPUT_422 | + VIP_FORMAT_UV_STORAGE_ORDER_UVUV, + .cplanes = 2, + }, { + .fourcc = V4L2_PIX_FMT_NV61, + .fmt_val = VIP_FORMAT_YUV_OUTPUT_422 | + VIP_FORMAT_UV_STORAGE_ORDER_VUVU, + .cplanes = 2, + }, + { + .fourcc = V4L2_PIX_FMT_NV12, + .fmt_val = VIP_FORMAT_YUV_OUTPUT_420 | + VIP_FORMAT_UV_STORAGE_ORDER_UVUV, + .cplanes = 2, + }, + { + .fourcc = V4L2_PIX_FMT_NV21, + .fmt_val = VIP_FORMAT_YUV_OUTPUT_420 | + VIP_FORMAT_UV_STORAGE_ORDER_VUVU, + .cplanes = 2, + }, { + .fourcc = V4L2_PIX_FMT_RGB24, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_RGB565, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_BGR666, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB8, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG8, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG8, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR8, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB10, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG10, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG10, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR10, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SRGGB12, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SGRBG12, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SGBRG12, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR12, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_SBGGR16, + .cplanes = 1, + }, { + .fourcc = V4L2_PIX_FMT_Y16, + .cplanes = 1, + } +}; + +static const struct vip_input_fmt in_fmts[] = { + { + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_YUYV, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_YUYV, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_INTERLACED, + }, { + .mbus_code = MEDIA_BUS_FMT_YVYU8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_YVYU, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_YVYU8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_YVYU, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_INTERLACED, + }, { + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_UYVY, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_UYVY, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_INTERLACED, + }, { + .mbus_code = MEDIA_BUS_FMT_VYUY8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_VYUY, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_VYUY8_2X8, + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | + VIP_FORMAT_YUV_INPUT_ORDER_VYUY, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, + .fmt_type = VIP_FMT_TYPE_YUV, + .field = V4L2_FIELD_INTERLACED, + }, { + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_8, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_8, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_8, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_8, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_10, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_10, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_10, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_10, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_12, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_12, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_12, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_12, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_RGB888_1X24, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RGB888, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_Y8_1X8, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_8, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_Y10_1X10, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_10, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + }, { + .mbus_code = MEDIA_BUS_FMT_Y12_1X12, + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | + VIP_FORMAT_RAW_DATA_WIDTH_12, + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, + .fmt_type = VIP_FMT_TYPE_RAW, + .field = V4L2_FIELD_NONE, + } +}; + +static const struct +vip_input_fmt *get_input_fmt(struct v4l2_subdev *sd) +{ + struct v4l2_subdev_format fmt; + u32 i; + + fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; + fmt.pad = 0; + v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt); + + for (i = 0; i < ARRAY_SIZE(in_fmts); i++) + if (fmt.format.code == in_fmts[i].mbus_code && + fmt.format.field == in_fmts[i].field) + return &in_fmts[i]; + + v4l2_err(sd->v4l2_dev, "remote sensor mbus code not supported\n"); + return NULL; +} + +static struct +vip_output_fmt *find_output_fmt(struct rk_vip_stream *stream, u32 pixelfmt) +{ + struct vip_output_fmt *fmt; + u32 i; + + for (i = 0; i < ARRAY_SIZE(out_fmts); i++) { + fmt = &out_fmts[i]; + if (fmt->fourcc == pixelfmt) + return fmt; + } + + return NULL; +} + +static struct rk_vip_buffer *rk_vip_get_buffer(struct rk_vip_stream *stream) +{ + struct rk_vip_buffer *buff; + + lockdep_assert_held(&stream->vbq_lock); + + if (list_empty(&stream->buf_head)) + return NULL; + + buff = list_first_entry(&stream->buf_head, struct rk_vip_buffer, queue); + list_del(&buff->queue); + + return buff; +} + +static int rk_vip_init_buffers(struct rk_vip_stream *stream) +{ + struct rk_vip_device *vip_dev = stream->vipdev; + void __iomem *base = vip_dev->base_addr; + + spin_lock(&stream->vbq_lock); + + stream->buffs[0] = rk_vip_get_buffer(stream); + stream->buffs[1] = rk_vip_get_buffer(stream); + + if (!(stream->buffs[0]) || !(stream->buffs[1])) { + spin_unlock(&stream->vbq_lock); + return -EINVAL; + } + + spin_unlock(&stream->vbq_lock); + + write_vip_reg(base, VIP_FRM0_ADDR_Y, + stream->buffs[0]->buff_addr[RK_VIP_PLANE_Y]); + write_vip_reg(base, VIP_FRM0_ADDR_UV, + stream->buffs[0]->buff_addr[RK_VIP_PLANE_CBCR]); + + write_vip_reg(base, VIP_FRM1_ADDR_Y, + stream->buffs[1]->buff_addr[RK_VIP_PLANE_Y]); + write_vip_reg(base, VIP_FRM1_ADDR_UV, + stream->buffs[1]->buff_addr[RK_VIP_PLANE_CBCR]); + + return 0; +} + +static void rk_vip_assign_new_buffer_pingpong(struct rk_vip_stream *stream) +{ + struct rk_vip_scratch_buffer *scratch_buf = &stream->scratch_buf; + struct rk_vip_device *vip_dev = stream->vipdev; + struct rk_vip_buffer *buffer = NULL; + void __iomem *base = vip_dev->base_addr; + u32 frm_addr_y, frm_addr_uv; + + /* Set up an empty buffer for the next frame */ + spin_lock(&stream->vbq_lock); + + buffer = rk_vip_get_buffer(stream); + + stream->buffs[stream->frame_phase] = buffer; + + spin_unlock(&stream->vbq_lock); + + frm_addr_y = stream->frame_phase ? VIP_FRM1_ADDR_Y : VIP_FRM0_ADDR_Y; + frm_addr_uv = stream->frame_phase ? VIP_FRM1_ADDR_UV : VIP_FRM0_ADDR_UV; + + if (buffer) { + write_vip_reg(base, frm_addr_y, + buffer->buff_addr[RK_VIP_PLANE_Y]); + write_vip_reg(base, frm_addr_uv, + buffer->buff_addr[RK_VIP_PLANE_CBCR]); + } else { + write_vip_reg(base, frm_addr_y, scratch_buf->dma_addr); + write_vip_reg(base, frm_addr_uv, scratch_buf->dma_addr); + } +} + +static void rk_vip_stream_stop(struct rk_vip_stream *stream) +{ + struct rk_vip_device *vip_dev = stream->vipdev; + void __iomem *base = vip_dev->base_addr; + u32 val; + + val = read_vip_reg(base, VIP_CTRL); + write_vip_reg(base, VIP_CTRL, val & (~VIP_CTRL_ENABLE_CAPTURE)); + write_vip_reg(base, VIP_INTEN, 0x0); + write_vip_reg(base, VIP_INTSTAT, 0x3ff); + write_vip_reg(base, VIP_FRAME_STATUS, 0x0); + + stream->state = RK_VIP_STATE_READY; +} + +static int rk_vip_queue_setup(struct vb2_queue *queue, + unsigned int *num_buffers, + unsigned int *num_planes, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct rk_vip_stream *stream = queue->drv_priv; + const struct v4l2_plane_pix_format *plane_fmt; + const struct v4l2_pix_format_mplane *pixm; + + pixm = &stream->pixm; + + if (*num_planes) { + if (*num_planes != 1) + return -EINVAL; + + if (sizes[0] < pixm->plane_fmt[0].sizeimage) + return -EINVAL; + return 0; + } + + *num_planes = 1; + + plane_fmt = &pixm->plane_fmt[0]; + sizes[0] = plane_fmt->sizeimage; + + *num_buffers = VIP_REQ_BUFS_MIN; + + return 0; +} + +static void rk_vip_buf_queue(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct rk_vip_buffer *vipbuf = to_rk_vip_buffer(vbuf); + struct vb2_queue *queue = vb->vb2_queue; + struct rk_vip_stream *stream = queue->drv_priv; + struct v4l2_pix_format_mplane *pixm = &stream->pixm; + unsigned long lock_flags = 0; + int i; + + struct vip_output_fmt *fmt = stream->vip_fmt_out; + + memset(vipbuf->buff_addr, 0, sizeof(vipbuf->buff_addr)); + + vipbuf->buff_addr[0] = vb2_dma_contig_plane_dma_addr(vb, 0); + + for (i = 0; i < fmt->cplanes - 1; i++) + vipbuf->buff_addr[i + 1] = vipbuf->buff_addr[i] + + pixm->plane_fmt[i].bytesperline * pixm->height; + + spin_lock_irqsave(&stream->vbq_lock, lock_flags); + list_add_tail(&vipbuf->queue, &stream->buf_head); + spin_unlock_irqrestore(&stream->vbq_lock, lock_flags); +} + +static int rk_vip_create_scratch_buf(struct rk_vip_stream *stream) +{ + struct rk_vip_scratch_buffer *scratch_buf = &stream->scratch_buf; + struct rk_vip_device *vip_dev = stream->vipdev; + + /* get a maximum plane size */ + scratch_buf->size = max(stream->pixm.plane_fmt[0].sizeimage, + stream->pixm.plane_fmt[1].sizeimage); + + scratch_buf->vaddr = dma_alloc_coherent(vip_dev->dev, scratch_buf->size, + &scratch_buf->dma_addr, + GFP_KERNEL); + if (!scratch_buf->vaddr) + return -ENOMEM; + + return 0; +} + +static void rk_vip_destroy_scratch_buf(struct rk_vip_stream *stream) +{ + struct rk_vip_scratch_buffer *scratch_buf = &stream->scratch_buf; + struct rk_vip_device *vip_dev = stream->vipdev; + + dma_free_coherent(vip_dev->dev, scratch_buf->size, + scratch_buf->vaddr, scratch_buf->dma_addr); +} + +static void rk_vip_stop_streaming(struct vb2_queue *queue) +{ + struct rk_vip_stream *stream = queue->drv_priv; + struct rk_vip_device *vip_dev = stream->vipdev; + struct rk_vip_buffer *buf; + struct v4l2_subdev *sd; + int ret; + + stream->stopping = true; + ret = wait_event_timeout(stream->wq_stopped, + stream->state != RK_VIP_STATE_STREAMING, + msecs_to_jiffies(1000)); + if (!ret) { + rk_vip_stream_stop(stream); + stream->stopping = false; + } + pm_runtime_put(vip_dev->dev); + + /* stop the sub device*/ + sd = vip_dev->sensor.sd; + v4l2_subdev_call(sd, video, s_stream, 0); + + /* release buffers */ + if (stream->buffs[0]) { + list_add_tail(&stream->buffs[0]->queue, &stream->buf_head); + stream->buffs[0] = NULL; + } + + if (stream->buffs[1]) { + list_add_tail(&stream->buffs[1]->queue, &stream->buf_head); + stream->buffs[1] = NULL; + } + + while (!list_empty(&stream->buf_head)) { + buf = rk_vip_get_buffer(stream); + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); + } + + rk_vip_destroy_scratch_buf(stream); +} + +static u32 rk_vip_determine_input_mode(struct rk_vip_device *vip_dev) +{ + v4l2_std_id std = vip_dev->sensor.std; + + return (std == V4L2_STD_NTSC) ? + VIP_FORMAT_INPUT_MODE_NTSC : + VIP_FORMAT_INPUT_MODE_PAL; +} + +static inline u32 rk_vip_scl_ctl(struct rk_vip_stream *stream) +{ + u32 fmt_type = stream->vip_fmt_in->fmt_type; + + return (fmt_type == VIP_FMT_TYPE_YUV) ? + VIP_SCL_CTRL_ENABLE_YUV_16BIT_BYPASS : + VIP_SCL_CTRL_ENABLE_RAW_16BIT_BYPASS; +} + +static int rk_vip_stream_start(struct rk_vip_stream *stream) +{ + u32 val, mbus_flags, href_pol, vsync_pol, + xfer_mode = 0, yc_swap = 0, skip_top = 0; + struct rk_vip_device *vip_dev = stream->vipdev; + struct rk_vip_sensor_info *sensor_info; + void __iomem *base = vip_dev->base_addr; + int ret; + u32 input_mode; + + sensor_info = &vip_dev->sensor; + stream->frame_idx = 0; + input_mode = rk_vip_determine_input_mode(vip_dev); + + mbus_flags = sensor_info->mbus.bus.parallel.flags; + href_pol = (mbus_flags & V4L2_MBUS_HSYNC_ACTIVE_HIGH) ? + 0 : VIP_FORMAT_HSY_LOW_ACTIVE; + vsync_pol = (mbus_flags & V4L2_MBUS_VSYNC_ACTIVE_HIGH) ? + VIP_FORMAT_VSY_HIGH_ACTIVE : 0; + + val = vsync_pol | href_pol | input_mode | stream->vip_fmt_out->fmt_val | + stream->vip_fmt_in->dvp_fmt_val | xfer_mode | yc_swap; + write_vip_reg(base, VIP_FOR, val); + + val = stream->pixm.width; + if (stream->vip_fmt_in->fmt_type == VIP_FMT_TYPE_RAW) + val = stream->pixm.width * 2; + + write_vip_reg(base, VIP_VIR_LINE_WIDTH, val); + write_vip_reg(base, VIP_SET_SIZE, + stream->pixm.width | (stream->pixm.height << 16)); + + v4l2_subdev_call(sensor_info->sd, sensor, g_skip_top_lines, &skip_top); + + write_vip_reg(base, VIP_CROP, skip_top << VIP_CROP_Y_SHIFT); + write_vip_reg(base, VIP_FRAME_STATUS, VIP_FRAME_STAT_CLS); + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_CLS); + write_vip_reg(base, VIP_SCL_CTRL, rk_vip_scl_ctl(stream)); + + ret = rk_vip_init_buffers(stream); + if (ret) + return ret; + + write_vip_reg(base, VIP_INTEN, VIP_INTEN_FRAME_END_EN | + VIP_INTEN_LINE_ERR_EN | + VIP_INTEN_PST_INF_FRAME_END_EN); + + write_vip_reg(base, VIP_CTRL, VIP_CTRL_AXI_BURST_16 | + VIP_CTRL_MODE_PINGPONG | + VIP_CTRL_ENABLE_CAPTURE); + + stream->state = RK_VIP_STATE_STREAMING; + + return 0; +} + +static int rk_vip_start_streaming(struct vb2_queue *queue, unsigned int count) +{ + struct rk_vip_stream *stream = queue->drv_priv; + struct rk_vip_device *vip_dev = stream->vipdev; + struct v4l2_device *v4l2_dev = &vip_dev->v4l2_dev; + struct v4l2_subdev *sd; + int ret; + + if (stream->state != RK_VIP_STATE_READY) { + ret = -EBUSY; + v4l2_err(v4l2_dev, "Stream in busy state\n"); + goto destroy_buf; + } + + if (!vip_dev->sensor.sd) { + ret = -EINVAL; + v4l2_err(v4l2_dev, "No sensor subdev detected\n"); + goto destroy_buf; + } + + ret = rk_vip_create_scratch_buf(stream); + if (ret < 0) { + v4l2_err(v4l2_dev, "Failed to create scratch_buf, %d\n", ret); + goto destroy_buf; + } + + ret = pm_runtime_get_sync(vip_dev->dev); + if (ret < 0) { + v4l2_err(v4l2_dev, "Failed to get runtime pm, %d\n", ret); + goto destroy_scratch_buf; + } + + /* start sub-devices */ + sd = vip_dev->sensor.sd; + stream->vip_fmt_in = get_input_fmt(vip_dev->sensor.sd); + + ret = v4l2_subdev_call(sd, video, s_stream, 1); + if (ret < 0) + goto runtime_put; + + ret = rk_vip_stream_start(stream); + if (ret < 0) + goto stop_stream; + + return 0; + +stop_stream: + rk_vip_stream_stop(stream); +runtime_put: + pm_runtime_put(vip_dev->dev); +destroy_scratch_buf: + rk_vip_destroy_scratch_buf(stream); +destroy_buf: + while (!list_empty(&stream->buf_head)) { + struct rk_vip_buffer *buf; + + buf = rk_vip_get_buffer(stream); + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); + } + + return ret; +} + +static const struct vb2_ops rk_vip_vb2_ops = { + .queue_setup = rk_vip_queue_setup, + .buf_queue = rk_vip_buf_queue, + .wait_prepare = vb2_ops_wait_prepare, + .wait_finish = vb2_ops_wait_finish, + .stop_streaming = rk_vip_stop_streaming, + .start_streaming = rk_vip_start_streaming, +}; + +static int rk_vip_init_vb2_queue(struct vb2_queue *q, + struct rk_vip_stream *stream) +{ + q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + q->io_modes = VB2_MMAP | VB2_DMABUF; + q->drv_priv = stream; + q->ops = &rk_vip_vb2_ops; + q->mem_ops = &vb2_dma_contig_memops; + q->buf_struct_size = sizeof(struct rk_vip_buffer); + q->min_buffers_needed = VIP_REQ_BUFS_MIN; + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + q->lock = &stream->vlock; + q->dev = stream->vipdev->dev; + + return vb2_queue_init(q); +} + +static void rk_vip_update_pixm(struct rk_vip_stream *stream, + struct vip_output_fmt *fmt, + struct v4l2_pix_format_mplane *pixm) +{ + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; + struct v4l2_subdev_format sd_fmt; + struct v4l2_rect input_rect; + u32 width, height; + + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; + sd_fmt.pad = 0; + v4l2_subdev_call(sensor_info->sd, pad, get_fmt, NULL, &sd_fmt); + + input_rect.width = VIP_MAX_WIDTH; + input_rect.height = VIP_MAX_HEIGHT; + width = clamp_t(u32, sd_fmt.format.width, + VIP_MIN_WIDTH, input_rect.width); + height = clamp_t(u32, sd_fmt.format.height, + VIP_MIN_HEIGHT, input_rect.height); + + pixm->width = width; + pixm->height = height; + pixm->field = sd_fmt.format.field; + pixm->colorspace = sd_fmt.format.colorspace; + pixm->ycbcr_enc = sd_fmt.format.ycbcr_enc; + pixm->quantization = sd_fmt.format.quantization; + pixm->xfer_func = sd_fmt.format.xfer_func; + + v4l2_fill_pixfmt_mp(pixm, fmt->fourcc, pixm->width, pixm->height); +} + +static int rk_vip_set_fmt(struct rk_vip_stream *stream, + struct v4l2_pix_format_mplane *pixm) +{ + struct rk_vip_device *vip_dev = stream->vipdev; + struct v4l2_subdev_format sd_fmt; + struct vip_output_fmt *fmt; + int ret; + + if (stream->state == RK_VIP_STATE_STREAMING) + return -EBUSY; + + fmt = find_output_fmt(stream, pixm->pixelformat); + if (!fmt) + fmt = &out_fmts[0]; + + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; + sd_fmt.pad = 0; + sd_fmt.format.width = pixm->width; + sd_fmt.format.height = pixm->height; + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, set_fmt, NULL, &sd_fmt); + rk_vip_update_pixm(stream, fmt, pixm); + stream->pixm = *pixm; + stream->vip_fmt_out = fmt; + + return ret; +} + +void rk_vip_set_default_format(struct rk_vip_device *vip_dev) +{ + struct rk_vip_stream *stream = &vip_dev->stream; + struct v4l2_pix_format_mplane pixm; + + pixm.pixelformat = V4L2_PIX_FMT_NV12; + pixm.width = RK_VIP_DEFAULT_WIDTH; + pixm.height = RK_VIP_DEFAULT_HEIGHT; + rk_vip_set_fmt(stream, &pixm); +} + +void rk_vip_stream_init(struct rk_vip_device *vip_dev) +{ + struct rk_vip_stream *stream = &vip_dev->stream; + struct v4l2_pix_format_mplane pixm; + + memset(stream, 0, sizeof(*stream)); + memset(&pixm, 0, sizeof(pixm)); + stream->vipdev = vip_dev; + + INIT_LIST_HEAD(&stream->buf_head); + spin_lock_init(&stream->vbq_lock); + stream->state = RK_VIP_STATE_READY; + init_waitqueue_head(&stream->wq_stopped); +} + +static const struct v4l2_file_operations rk_vip_fops = { + .open = v4l2_fh_open, + .release = vb2_fop_release, + .unlocked_ioctl = video_ioctl2, + .poll = vb2_fop_poll, + .mmap = vb2_fop_mmap, +}; + +static int rk_vip_enum_input(struct file *file, void *priv, + struct v4l2_input *input) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct v4l2_subdev *sd = stream->vipdev->sensor.sd; + int ret; + + if (input->index > 0) + return -EINVAL; + + ret = v4l2_subdev_call(sd, video, g_input_status, &input->status); + if (ret) + return ret; + + strscpy(input->name, "Camera", sizeof(input->name)); + input->type = V4L2_INPUT_TYPE_CAMERA; + input->std = stream->vdev.tvnorms; + input->capabilities = V4L2_IN_CAP_STD; + + return 0; +} + +static int rk_vip_try_fmt_vid_cap_mplane(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct vip_output_fmt *fmt; + + fmt = find_output_fmt(stream, f->fmt.pix_mp.pixelformat); + if (!fmt) + fmt = &out_fmts[0]; + + rk_vip_update_pixm(stream, fmt, &f->fmt.pix_mp); + + return 0; +} + +static int rk_vip_g_std(struct file *file, void *fh, v4l2_std_id *norm) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; + + *norm = sensor_info->std; + + return 0; +} + +static int rk_vip_s_std(struct file *file, void *fh, v4l2_std_id norm) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; + int ret; + + if (norm == sensor_info->std) + return 0; + + if (stream->state == RK_VIP_STATE_STREAMING) + return -EBUSY; + + ret = v4l2_subdev_call(sensor_info->sd, video, s_std, norm); + if (ret) + return ret; + + sensor_info->std = norm; + + /* S_STD will update the format since that depends on the standard */ + rk_vip_update_pixm(stream, stream->vip_fmt_out, &stream->pixm); + + return 0; +} + +static int rk_vip_querystd(struct file *file, void *fh, v4l2_std_id *a) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; + + *a = V4L2_STD_UNKNOWN; + + return v4l2_subdev_call(sensor_info->sd, video, querystd, a); +} + +static int rk_vip_enum_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + struct vip_output_fmt *fmt = NULL; + + if (f->index >= ARRAY_SIZE(out_fmts)) + return -EINVAL; + + fmt = &out_fmts[f->index]; + f->pixelformat = fmt->fourcc; + + return 0; +} + +static int rk_vip_s_fmt_vid_cap_mplane(struct file *file, + void *priv, struct v4l2_format *f) +{ + struct rk_vip_stream *stream = video_drvdata(file); + int ret; + + if (stream->state == RK_VIP_STATE_STREAMING) + return -EBUSY; + + ret = rk_vip_set_fmt(stream, &f->fmt.pix_mp); + + return ret; +} + +static int rk_vip_g_fmt_vid_cap_mplane(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct rk_vip_stream *stream = video_drvdata(file); + + f->fmt.pix_mp = stream->pixm; + + return 0; +} + +static int rk_vip_querycap(struct file *file, void *priv, + struct v4l2_capability *cap) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct device *dev = stream->vipdev->dev; + + strscpy(cap->driver, dev->driver->name, sizeof(cap->driver)); + strscpy(cap->card, dev->driver->name, sizeof(cap->card)); + snprintf(cap->bus_info, sizeof(cap->bus_info), + "platform:%s", dev_name(dev)); + + return 0; +} + +static int rk_vip_enum_framesizes(struct file *file, void *fh, + struct v4l2_frmsizeenum *fsize) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct rk_vip_device *vip_dev = stream->vipdev; + struct v4l2_subdev_frame_size_enum fse = { + .index = fsize->index, + .which = V4L2_SUBDEV_FORMAT_ACTIVE, + }; + struct vip_output_fmt *fmt; + int ret; + + if (!vip_dev->sensor.sd) + return -EINVAL; + + fmt = find_output_fmt(stream, fsize->pixel_format); + if (!fmt) + return -EINVAL; + + fse.code = fmt->mbus; + + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, enum_frame_size, + NULL, &fse); + if (ret) + return ret; + + fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE; + fsize->discrete.width = fse.max_width; + fsize->discrete.height = fse.max_height; + + return 0; +} + +static int rk_vip_enum_frameintervals(struct file *file, void *fh, + struct v4l2_frmivalenum *fival) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct rk_vip_device *vip_dev = stream->vipdev; + struct v4l2_subdev_frame_interval_enum fie = { + .index = fival->index, + .width = fival->width, + .height = fival->height, + .which = V4L2_SUBDEV_FORMAT_ACTIVE, + }; + struct vip_output_fmt *fmt; + int ret; + + if (!vip_dev->sensor.sd) + return -EINVAL; + + fmt = find_output_fmt(stream, fival->pixel_format); + if (!fmt) + return -EINVAL; + + fie.code = fmt->mbus; + + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, enum_frame_interval, + NULL, &fie); + if (ret) + return ret; + + fival->type = V4L2_FRMSIZE_TYPE_DISCRETE; + fival->discrete = fie.interval; + + return 0; +} + +static int rk_vip_g_input(struct file *file, void *fh, unsigned int *i) +{ + *i = 0; + return 0; +} + +static int rk_vip_s_input(struct file *file, void *fh, unsigned int i) +{ + if (i) + return -EINVAL; + + return 0; +} + +static int rk_vip_g_parm(struct file *file, void *priv, + struct v4l2_streamparm *p) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct rk_vip_device *vip_dev = stream->vipdev; + + if (!vip_dev->sensor.sd) + return -EINVAL; + + return v4l2_g_parm_cap(video_devdata(file), vip_dev->sensor.sd, p); +} + +static int rk_vip_s_parm(struct file *file, void *priv, + struct v4l2_streamparm *p) +{ + struct rk_vip_stream *stream = video_drvdata(file); + struct rk_vip_device *vip_dev = stream->vipdev; + + if (!vip_dev->sensor.sd) + return -EINVAL; + + return v4l2_s_parm_cap(video_devdata(file), vip_dev->sensor.sd, p); +} + +static const struct v4l2_ioctl_ops rk_vip_v4l2_ioctl_ops = { + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_create_bufs = vb2_ioctl_create_bufs, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_expbuf = vb2_ioctl_expbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, + + .vidioc_g_std = rk_vip_g_std, + .vidioc_s_std = rk_vip_s_std, + .vidioc_querystd = rk_vip_querystd, + + .vidioc_enum_fmt_vid_cap = rk_vip_enum_fmt_vid_cap, + .vidioc_try_fmt_vid_cap_mplane = rk_vip_try_fmt_vid_cap_mplane, + .vidioc_s_fmt_vid_cap_mplane = rk_vip_s_fmt_vid_cap_mplane, + .vidioc_g_fmt_vid_cap_mplane = rk_vip_g_fmt_vid_cap_mplane, + .vidioc_querycap = rk_vip_querycap, + .vidioc_enum_framesizes = rk_vip_enum_framesizes, + .vidioc_enum_frameintervals = rk_vip_enum_frameintervals, + + .vidioc_enum_input = rk_vip_enum_input, + .vidioc_g_input = rk_vip_g_input, + .vidioc_s_input = rk_vip_s_input, + + .vidioc_g_parm = rk_vip_g_parm, + .vidioc_s_parm = rk_vip_s_parm, +}; + +void rk_vip_unregister_stream_vdev(struct rk_vip_device *vip_dev) +{ + struct rk_vip_stream *stream = &vip_dev->stream; + + media_entity_cleanup(&stream->vdev.entity); + video_unregister_device(&stream->vdev); +} + +int rk_vip_register_stream_vdev(struct rk_vip_device *vip_dev) +{ + struct rk_vip_stream *stream = &vip_dev->stream; + struct v4l2_device *v4l2_dev = &vip_dev->v4l2_dev; + struct video_device *vdev = &stream->vdev; + int ret; + + strscpy(vdev->name, VIP_VIDEODEVICE_NAME, sizeof(vdev->name)); + mutex_init(&stream->vlock); + + vdev->ioctl_ops = &rk_vip_v4l2_ioctl_ops; + vdev->release = video_device_release_empty; + vdev->fops = &rk_vip_fops; + vdev->minor = -1; + vdev->v4l2_dev = v4l2_dev; + vdev->lock = &stream->vlock; + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_STREAMING; + vdev->tvnorms = V4L2_STD_NTSC | V4L2_STD_PAL; + video_set_drvdata(vdev, stream); + vdev->vfl_dir = VFL_DIR_RX; + stream->pad.flags = MEDIA_PAD_FL_SINK; + + rk_vip_init_vb2_queue(&stream->buf_queue, stream); + + vdev->queue = &stream->buf_queue; + strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name)); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret < 0) { + v4l2_err(v4l2_dev, + "video_register_device failed with error %d\n", ret); + return ret; + } + + ret = media_entity_pads_init(&vdev->entity, 1, &stream->pad); + if (ret < 0) + goto unreg; + + return 0; +unreg: + video_unregister_device(vdev); + return ret; +} + +static void rk_vip_vb_done(struct rk_vip_stream *stream, + struct vb2_v4l2_buffer *vb_done) +{ + vb2_set_plane_payload(&vb_done->vb2_buf, 0, + stream->pixm.plane_fmt[0].sizeimage); + vb_done->vb2_buf.timestamp = ktime_get_ns(); + vb_done->sequence = stream->frame_idx; + vb2_buffer_done(&vb_done->vb2_buf, VB2_BUF_STATE_DONE); +} + +static void rk_vip_reset_stream(struct rk_vip_device *vip_dev) +{ + void __iomem *base = vip_dev->base_addr; + u32 ctl = read_vip_reg(base, VIP_CTRL); + + write_vip_reg(base, VIP_CTRL, ctl & (~VIP_CTRL_ENABLE_CAPTURE)); + write_vip_reg(base, VIP_CTRL, ctl | VIP_CTRL_ENABLE_CAPTURE); +} + +irqreturn_t rk_vip_irq_pingpong(int irq, void *ctx) +{ + struct device *dev = ctx; + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); + struct rk_vip_stream *stream = &vip_dev->stream; + void __iomem *base = vip_dev->base_addr; + unsigned int intstat; + + u32 lastline, lastpix, ctl, vip_frmst; + + intstat = read_vip_reg(base, VIP_INTSTAT); + vip_frmst = read_vip_reg(base, VIP_FRAME_STATUS); + lastline = VIP_FETCH_Y_LAST_LINE(read_vip_reg(base, VIP_LAST_LINE)); + lastpix = VIP_FETCH_Y_LAST_LINE(read_vip_reg(base, VIP_LAST_PIX)); + ctl = read_vip_reg(base, VIP_CTRL); + + /* There are two irqs enabled: + * - PST_INF_FRAME_END: vip FIFO is ready, + * this is prior to FRAME_END + * - FRAME_END: vip has saved frame to memory, + * a frame ready + */ + + if ((intstat & VIP_INTSTAT_PST_INF_FRAME_END)) { + write_vip_reg(base, VIP_INTSTAT, + VIP_INTSTAT_PST_INF_FRAME_END_CLR); + if (stream->stopping) + /* To stop VIP ASAP, before FRAME_END irq */ + write_vip_reg(base, VIP_CTRL, + ctl & (~VIP_CTRL_ENABLE_CAPTURE)); + } + + if ((intstat & VIP_INTSTAT_PRE_INF_FRAME_END)) + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_PRE_INF_FRAME_END); + + if (intstat & (VIP_INTSTAT_LINE_ERR | VIP_INTSTAT_PIX_ERR)) { + v4l2_err(&vip_dev->v4l2_dev, "LINE_ERR OR PIX_ERR"); + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_LINE_ERR | + VIP_INTSTAT_PIX_ERR); + rk_vip_reset_stream(vip_dev); + } + + if ((intstat & VIP_INTSTAT_FRAME_END)) { + struct vb2_v4l2_buffer *vb_done = NULL; + + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_FRAME_END_CLR | + VIP_INTSTAT_LINE_END_CLR); + + if (stream->stopping) { + rk_vip_stream_stop(stream); + stream->stopping = false; + wake_up(&stream->wq_stopped); + return IRQ_HANDLED; + } + + if (lastline != stream->pixm.height) { + v4l2_err(&vip_dev->v4l2_dev, + "Bad frame, irq:0x%x frmst:0x%x size:%dx%d\n", + intstat, vip_frmst, lastpix, lastline); + + rk_vip_reset_stream(vip_dev); + } + + if (vip_frmst & VIP_INTSTAT_F0_READY) + stream->frame_phase = 0; + else if (vip_frmst & VIP_INTSTAT_F1_READY) + stream->frame_phase = 1; + else + return IRQ_HANDLED; + + if (stream->buffs[stream->frame_phase]) + vb_done = &stream->buffs[stream->frame_phase]->vb; + + rk_vip_assign_new_buffer_pingpong(stream); + + if (vb_done) + rk_vip_vb_done(stream, vb_done); + + stream->frame_idx++; + } + + return IRQ_HANDLED; +} diff --git a/drivers/media/platform/rockchip/vip/dev.c b/drivers/media/platform/rockchip/vip/dev.c new file mode 100644 index 000000000000..c3931e5accbd --- /dev/null +++ b/drivers/media/platform/rockchip/vip/dev.c @@ -0,0 +1,346 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Rockchip VIP Camera Interface Driver + * + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. + * Copyright (C) 2020 Maxime Chevallier <maxime.chevallier@bootlin.com> + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> + */ + +#include <linux/clk.h> +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/module.h> +#include <linux/of.h> +#include <linux/of_graph.h> +#include <linux/of_platform.h> +#include <linux/of_reserved_mem.h> +#include <linux/reset.h> +#include <linux/pm_runtime.h> +#include <linux/pinctrl/consumer.h> +#include <media/v4l2-fwnode.h> + +#include "dev.h" +#include "regs.h" + +#define RK_VIP_VERNO_LEN 10 + +struct vip_match_data { + int chip_id; + const char * const *clks; + const char * const *rsts; + int clks_num; + int rsts_num; +}; + +static int rk_vip_create_links(struct rk_vip_device *vip_dev) +{ + struct v4l2_subdev *sd = vip_dev->sensor.sd; + int ret; + + ret = media_create_pad_link(&sd->entity, 0, + &vip_dev->stream.vdev.entity, 0, + MEDIA_LNK_FL_ENABLED); + if (ret) { + dev_err(vip_dev->dev, "failed to create link"); + return ret; + } + + return 0; +} + +static int subdev_notifier_complete(struct v4l2_async_notifier *notifier) +{ + struct rk_vip_device *vip_dev; + int ret; + + vip_dev = container_of(notifier, struct rk_vip_device, notifier); + + mutex_lock(&vip_dev->media_dev.graph_mutex); + + ret = v4l2_device_register_subdev_nodes(&vip_dev->v4l2_dev); + if (ret < 0) + goto unlock; + + ret = rk_vip_create_links(vip_dev); + if (ret < 0) + goto unlock; + +unlock: + mutex_unlock(&vip_dev->media_dev.graph_mutex); + return ret; +} + +static int subdev_notifier_bound(struct v4l2_async_notifier *notifier, + struct v4l2_subdev *subdev, + struct v4l2_async_connection *asd) +{ + struct rk_vip_device *vip_dev = container_of(notifier, + struct rk_vip_device, notifier); + + int pad; + + vip_dev->sensor.sd = subdev; + pad = media_entity_get_fwnode_pad(&subdev->entity, subdev->fwnode, + MEDIA_PAD_FL_SOURCE); + if (pad < 0) + return pad; + + vip_dev->sensor.pad = pad; + + return 0; +} + +static const struct v4l2_async_notifier_operations subdev_notifier_ops = { + .bound = subdev_notifier_bound, + .complete = subdev_notifier_complete, +}; + +static int vip_subdev_notifier(struct rk_vip_device *vip_dev) +{ + struct v4l2_async_notifier *ntf = &vip_dev->notifier; + struct device *dev = vip_dev->dev; + struct v4l2_async_connection *asd; + struct v4l2_fwnode_endpoint vep = { + .bus_type = V4L2_MBUS_PARALLEL, + }; + struct fwnode_handle *ep; + int ret; + + v4l2_async_nf_init(ntf, &vip_dev->v4l2_dev); + + ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), 0, 0, + FWNODE_GRAPH_ENDPOINT_NEXT); + if (!ep) + return -EINVAL; + + ret = v4l2_fwnode_endpoint_parse(ep, &vep); + if (ret) + return ret; + + asd = v4l2_async_nf_add_fwnode_remote(ntf, ep, + struct v4l2_async_connection); + if (IS_ERR(asd)) { + ret = PTR_ERR(asd); + return ret; + } + + ntf->ops = &subdev_notifier_ops; + + fwnode_handle_put(ep); + + ret = v4l2_async_nf_register(ntf); + return ret; +} + +static int rk_vip_register_platform_subdevs(struct rk_vip_device *vip_dev) +{ + int ret; + + ret = rk_vip_register_stream_vdev(vip_dev); + if (ret < 0) + return ret; + + ret = vip_subdev_notifier(vip_dev); + if (ret < 0) { + v4l2_err(&vip_dev->v4l2_dev, + "Failed to register subdev notifier(%d)\n", ret); + rk_vip_unregister_stream_vdev(vip_dev); + } + + return 0; +} + +static const char * const px30_vip_clks[] = { + "aclk", + "hclk", + "pclk", +}; + +static const struct vip_match_data px30_vip_match_data = { + .chip_id = CHIP_PX30_VIP, + .clks = px30_vip_clks, + .clks_num = ARRAY_SIZE(px30_vip_clks), +}; + +static const struct of_device_id rk_vip_plat_of_match[] = { + { + .compatible = "rockchip,px30-vip", + .data = &px30_vip_match_data, + }, + {}, +}; + +void rk_vip_soft_reset(struct rk_vip_device *vip_dev) +{ + reset_control_assert(vip_dev->vip_rst); + + udelay(5); + + reset_control_deassert(vip_dev->vip_rst); +} + +static int rk_vip_get_resource(struct platform_device *pdev, + struct rk_vip_device *vip_dev) +{ + struct resource *res; + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) { + dev_err(&pdev->dev, + "Unable to allocate resources for device\n"); + return -ENODEV; + } + + vip_dev->base_addr = devm_ioremap_resource(&pdev->dev, res); + if (IS_ERR(vip_dev->base_addr)) + return PTR_ERR(vip_dev->base_addr); + + return 0; +} + +static int rk_vip_plat_probe(struct platform_device *pdev) +{ + const struct of_device_id *match; + struct device_node *node = pdev->dev.of_node; + struct device *dev = &pdev->dev; + struct v4l2_device *v4l2_dev; + struct rk_vip_device *vip_dev; + const struct vip_match_data *data; + int i, ret, irq; + + match = of_match_node(rk_vip_plat_of_match, node); + if (!match) + return -ENODEV; + + vip_dev = devm_kzalloc(dev, sizeof(*vip_dev), GFP_KERNEL); + if (!vip_dev) + return -ENOMEM; + + dev_set_drvdata(dev, vip_dev); + vip_dev->dev = dev; + + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return irq; + + ret = devm_request_irq(dev, irq, rk_vip_irq_pingpong, 0, + dev_driver_string(dev), dev); + if (ret < 0) + return dev_err_probe(dev, ret, "request irq failed\n"); + + vip_dev->irq = irq; + data = match->data; + vip_dev->chip_id = data->chip_id; + + ret = rk_vip_get_resource(pdev, vip_dev); + if (ret) + return ret; + + for (i = 0; i < data->clks_num; i++) + vip_dev->clks[i].id = data->clks[i]; + + vip_dev->num_clk = data->clks_num; + + ret = devm_clk_bulk_get(dev, vip_dev->num_clk, vip_dev->clks); + if (ret) + return ret; + + vip_dev->vip_rst = devm_reset_control_array_get(dev, false, false); + if (IS_ERR(vip_dev->vip_rst)) + return PTR_ERR(vip_dev->vip_rst); + + /* Initialize the stream */ + rk_vip_stream_init(vip_dev); + strscpy(vip_dev->media_dev.model, "rk_vip", + sizeof(vip_dev->media_dev.model)); + vip_dev->media_dev.dev = &pdev->dev; + v4l2_dev = &vip_dev->v4l2_dev; + v4l2_dev->mdev = &vip_dev->media_dev; + strscpy(v4l2_dev->name, "rk_vip", sizeof(v4l2_dev->name)); + + ret = v4l2_device_register(vip_dev->dev, &vip_dev->v4l2_dev); + if (ret < 0) + return ret; + + media_device_init(&vip_dev->media_dev); + + ret = media_device_register(&vip_dev->media_dev); + if (ret < 0) { + v4l2_err(v4l2_dev, "Failed to register media device: %d\n", + ret); + goto err_unreg_v4l2_dev; + } + + /* create & register platform subdev (from of_node) */ + ret = rk_vip_register_platform_subdevs(vip_dev); + if (ret < 0) + goto err_unreg_media_dev; + + vip_dev->sensor.std = V4L2_STD_NTSC; + rk_vip_set_default_format(vip_dev); + pm_runtime_enable(&pdev->dev); + + return 0; + +err_unreg_media_dev: + media_device_unregister(&vip_dev->media_dev); +err_unreg_v4l2_dev: + v4l2_device_unregister(&vip_dev->v4l2_dev); + return ret; +} + +static int rk_vip_plat_remove(struct platform_device *pdev) +{ + struct rk_vip_device *vip_dev = platform_get_drvdata(pdev); + + pm_runtime_disable(&pdev->dev); + + media_device_unregister(&vip_dev->media_dev); + v4l2_device_unregister(&vip_dev->v4l2_dev); + rk_vip_unregister_stream_vdev(vip_dev); + + return 0; +} + +static int __maybe_unused rk_vip_runtime_suspend(struct device *dev) +{ + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); + + clk_bulk_disable_unprepare(vip_dev->num_clk, vip_dev->clks); + + return pinctrl_pm_select_sleep_state(dev); +} + +static int __maybe_unused rk_vip_runtime_resume(struct device *dev) +{ + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); + int ret; + + ret = pinctrl_pm_select_default_state(dev); + if (ret < 0) + return ret; + + return clk_bulk_prepare_enable(vip_dev->num_clk, vip_dev->clks); +} + +static const struct dev_pm_ops rk_vip_plat_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, + pm_runtime_force_resume) + SET_RUNTIME_PM_OPS(rk_vip_runtime_suspend, rk_vip_runtime_resume, NULL) +}; + +static struct platform_driver rk_vip_plat_drv = { + .driver = { + .name = VIP_DRIVER_NAME, + .of_match_table = rk_vip_plat_of_match, + .pm = &rk_vip_plat_pm_ops, + }, + .probe = rk_vip_plat_probe, + .remove = rk_vip_plat_remove, +}; +module_platform_driver(rk_vip_plat_drv); + +MODULE_AUTHOR("Rockchip Camera/ISP team"); +MODULE_DESCRIPTION("Rockchip VIP platform driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/media/platform/rockchip/vip/dev.h b/drivers/media/platform/rockchip/vip/dev.h new file mode 100644 index 000000000000..eb9cd8f20ffc --- /dev/null +++ b/drivers/media/platform/rockchip/vip/dev.h @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Rockchip VIP Driver + * + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> + */ + +#ifndef _RK_VIP_DEV_H +#define _RK_VIP_DEV_H + +#include <linux/clk.h> +#include <linux/mutex.h> +#include <media/media-device.h> +#include <media/media-entity.h> +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/videobuf2-v4l2.h> + +#define VIP_DRIVER_NAME "rk_vip" +#define VIP_VIDEODEVICE_NAME "stream_vip" + +#define RK_VIP_MAX_BUS_CLK 8 +#define RK_VIP_MAX_SENSOR 2 +#define RK_VIP_MAX_RESET 5 +#define RK_VIP_MAX_CSI_CHANNEL 4 + +#define RK_VIP_DEFAULT_WIDTH 640 +#define RK_VIP_DEFAULT_HEIGHT 480 + +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) +#define read_vip_reg(base, addr) readl((addr) + (base)) + +enum rk_vip_state { + RK_VIP_STATE_DISABLED, + RK_VIP_STATE_READY, + RK_VIP_STATE_STREAMING +}; + +enum rk_vip_chip_id { + CHIP_PX30_VIP, + CHIP_RK1808_VIP, + CHIP_RK3128_VIP, + CHIP_RK3288_VIP +}; + +enum host_type_t { + RK_CSI_RXHOST, + RK_DSI_RXHOST +}; + +struct rk_vip_buffer { + struct vb2_v4l2_buffer vb; + struct list_head queue; + union { + u32 buff_addr[VIDEO_MAX_PLANES]; + void *vaddr[VIDEO_MAX_PLANES]; + }; +}; + +struct rk_vip_scratch_buffer { + void *vaddr; + dma_addr_t dma_addr; + u32 size; +}; + +static inline struct rk_vip_buffer *to_rk_vip_buffer(struct vb2_v4l2_buffer *vb) +{ + return container_of(vb, struct rk_vip_buffer, vb); +} + +struct rk_vip_sensor_info { + struct v4l2_subdev *sd; + int pad; + struct v4l2_mbus_config mbus; + int lanes; + v4l2_std_id std; +}; + +struct vip_output_fmt { + u32 fourcc; + u32 mbus; + u32 fmt_val; + u8 cplanes; +}; + +enum vip_fmt_type { + VIP_FMT_TYPE_YUV = 0, + VIP_FMT_TYPE_RAW, +}; + +struct vip_input_fmt { + u32 mbus_code; + u32 dvp_fmt_val; + u32 csi_fmt_val; + enum vip_fmt_type fmt_type; + enum v4l2_field field; +}; + +struct rk_vip_stream { + struct rk_vip_device *vipdev; + enum rk_vip_state state; + bool stopping; + wait_queue_head_t wq_stopped; + int frame_idx; + int frame_phase; + + /* lock between irq and buf_queue */ + spinlock_t vbq_lock; + struct vb2_queue buf_queue; + struct list_head buf_head; + struct rk_vip_scratch_buffer scratch_buf; + struct rk_vip_buffer *buffs[2]; + + /* vfd lock */ + struct mutex vlock; + struct video_device vdev; + struct media_pad pad; + + struct vip_output_fmt *vip_fmt_out; + const struct vip_input_fmt *vip_fmt_in; + struct v4l2_pix_format_mplane pixm; +}; + +static inline struct rk_vip_stream *to_rk_vip_stream(struct video_device *vdev) +{ + return container_of(vdev, struct rk_vip_stream, vdev); +} + +struct rk_vip_device { + struct list_head list; + struct device *dev; + int irq; + void __iomem *base_addr; + void __iomem *csi_base; + struct clk_bulk_data clks[RK_VIP_MAX_BUS_CLK]; + int num_clk; + struct vb2_alloc_ctx *alloc_ctx; + bool iommu_en; + struct iommu_domain *domain; + struct reset_control *vip_rst; + + struct v4l2_device v4l2_dev; + struct media_device media_dev; + struct v4l2_ctrl_handler ctrl_handler; + struct v4l2_async_notifier notifier; + struct v4l2_async_connection asd; + struct rk_vip_sensor_info sensor; + + struct rk_vip_stream stream; + + int chip_id; +}; + +void rk_vip_unregister_stream_vdev(struct rk_vip_device *dev); +int rk_vip_register_stream_vdev(struct rk_vip_device *dev); +void rk_vip_stream_init(struct rk_vip_device *dev); +void rk_vip_set_default_format(struct rk_vip_device *dev); + +irqreturn_t rk_vip_irq_pingpong(int irq, void *ctx); +void rk_vip_soft_reset(struct rk_vip_device *vip_dev); + +#endif diff --git a/drivers/media/platform/rockchip/vip/regs.h b/drivers/media/platform/rockchip/vip/regs.h new file mode 100644 index 000000000000..ccf10ffbbff8 --- /dev/null +++ b/drivers/media/platform/rockchip/vip/regs.h @@ -0,0 +1,260 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Rockchip VIP Driver + * + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> + */ + +#ifndef _RK_VIP_REGS_H +#define _RK_VIP_REGS_H + +/* VIP Reg Offset */ +#define VIP_CTRL 0x00 +#define VIP_INTEN 0x04 +#define VIP_INTSTAT 0x08 +#define VIP_FOR 0x0c +#define VIP_LINE_NUM_ADDR 0x10 +#define VIP_FRM0_ADDR_Y 0x14 +#define VIP_FRM0_ADDR_UV 0x18 +#define VIP_FRM1_ADDR_Y 0x1c +#define VIP_FRM1_ADDR_UV 0x20 +#define VIP_VIR_LINE_WIDTH 0x24 +#define VIP_SET_SIZE 0x28 +#define VIP_SCM_ADDR_Y 0x2c +#define VIP_SCM_ADDR_U 0x30 +#define VIP_SCM_ADDR_V 0x34 +#define VIP_WB_UP_FILTER 0x38 +#define VIP_WB_LOW_FILTER 0x3c +#define VIP_WBC_CNT 0x40 +#define VIP_CROP 0x44 +#define VIP_SCL_CTRL 0x48 +#define VIP_SCL_DST 0x4c +#define VIP_SCL_FCT 0x50 +#define VIP_SCL_VALID_NUM 0x54 +#define VIP_LINE_LOOP_CTR 0x58 +#define VIP_FRAME_STATUS 0x60 +#define VIP_CUR_DST 0x64 +#define VIP_LAST_LINE 0x68 +#define VIP_LAST_PIX 0x6c + +/* RK1808 VIP CSI Registers Offset */ +#define VIP_CSI_ID0_CTRL0 0x80 +#define VIP_CSI_ID0_CTRL1 0x84 +#define VIP_CSI_ID1_CTRL0 0x88 +#define VIP_CSI_ID1_CTRL1 0x8c +#define VIP_CSI_ID2_CTRL0 0x90 +#define VIP_CSI_ID2_CTRL1 0x94 +#define VIP_CSI_ID3_CTRL0 0x98 +#define VIP_CSI_ID3_CTRL1 0x9c +#define VIP_CSI_WATER_LINE 0xa0 +#define VIP_CSI_FRM0_ADDR_Y_ID0 0xa4 +#define VIP_CSI_FRM1_ADDR_Y_ID0 0xa8 +#define VIP_CSI_FRM0_ADDR_UV_ID0 0xac +#define VIP_CSI_FRM1_ADDR_UV_ID0 0xb0 +#define VIP_CSI_FRM0_VLW_Y_ID0 0xb4 +#define VIP_CSI_FRM1_VLW_Y_ID0 0xb8 +#define VIP_CSI_FRM0_VLW_UV_ID0 0xbc +#define VIP_CSI_FRM1_VLW_UV_ID0 0xc0 +#define VIP_CSI_FRM0_ADDR_Y_ID1 0xc4 +#define VIP_CSI_FRM1_ADDR_Y_ID1 0xc8 +#define VIP_CSI_FRM0_ADDR_UV_ID1 0xcc +#define VIP_CSI_FRM1_ADDR_UV_ID1 0xd0 +#define VIP_CSI_FRM0_VLW_Y_ID1 0xd4 +#define VIP_CSI_FRM1_VLW_Y_ID1 0xd8 +#define VIP_CSI_FRM0_VLW_UV_ID1 0xdc +#define VIP_CSI_FRM1_VLW_UV_ID1 0xe0 +#define VIP_CSI_FRM0_ADDR_Y_ID2 0xe4 +#define VIP_CSI_FRM1_ADDR_Y_ID2 0xe8 +#define VIP_CSI_FRM0_ADDR_UV_ID2 0xec +#define VIP_CSI_FRM1_ADDR_UV_ID2 0xf0 +#define VIP_CSI_FRM0_VLW_Y_ID2 0xf4 +#define VIP_CSI_FRM1_VLW_Y_ID2 0xf8 +#define VIP_CSI_FRM0_VLW_UV_ID2 0xfc +#define VIP_CSI_FRM1_VLW_UV_ID2 0x100 +#define VIP_CSI_FRM0_ADDR_Y_ID3 0x104 +#define VIP_CSI_FRM1_ADDR_Y_ID3 0x108 +#define VIP_CSI_FRM0_ADDR_UV_ID3 0x10c +#define VIP_CSI_FRM1_ADDR_UV_ID3 0x110 +#define VIP_CSI_FRM0_VLW_Y_ID3 0x114 +#define VIP_CSI_FRM1_VLW_Y_ID3 0x118 +#define VIP_CSI_FRM0_VLW_UV_ID3 0x11c +#define VIP_CSI_FRM1_VLW_UV_ID3 0x120 +#define VIP_CSI_INTEN 0x124 +#define VIP_CSI_INTSTAT 0x128 +#define VIP_CSI_LINE_INT_NUM_ID0_1 0x12c +#define VIP_CSI_LINE_INT_NUM_ID2_3 0x130 +#define VIP_CSI_LINE_CNT_ID0_1 0x134 +#define VIP_CSI_LINE_CNT_ID2_3 0x138 +#define VIP_CSI_ID0_CROP_START 0x13c +#define VIP_CSI_ID1_CROP_START 0x140 +#define VIP_CSI_ID2_CROP_START 0x144 +#define VIP_CSI_ID3_CROP_START 0x148 + +/* The key register bit description */ + +/* VIP_CTRL Reg */ +#define VIP_CTRL_ENABLE_CAPTURE BIT(0) +#define VIP_CTRL_MODE_PINGPONG BIT(1) +#define VIP_CTRL_MODE_LINELOOP BIT(2) +#define VIP_CTRL_AXI_BURST_16 (0xF << 12) + +/* VIP_INTEN */ +#define VIP_INTEN_FRAME_END_EN BIT(0) +#define VIP_INTEN_LINE_ERR_EN BIT(2) +#define VIP_INTEN_BUS_ERR_EN BIT(6) +#define VIP_INTEN_SCL_ERR_EN BIT(7) +#define VIP_INTEN_PST_INF_FRAME_END_EN BIT(9) + +/* VIP INTSTAT */ +#define VIP_INTSTAT_CLS 0x3FF +#define VIP_INTSTAT_FRAME_END BIT(0) +#define VIP_INTSTAT_LINE_END BIT(1) +#define VIP_INTSTAT_LINE_ERR BIT(2) +#define VIP_INTSTAT_PIX_ERR BIT(3) +#define VIP_INTSTAT_DFIFO_OF BIT(5) +#define VIP_INTSTAT_BUS_ERR BIT(6) +#define VIP_INTSTAT_PRE_INF_FRAME_END BIT(8) +#define VIP_INTSTAT_PST_INF_FRAME_END BIT(9) +#define VIP_INTSTAT_FRAME_END_CLR BIT(0) +#define VIP_INTSTAT_LINE_END_CLR BIT(1) +#define VIP_INTSTAT_LINE_ERR_CLR BIT(2) +#define VIP_INTSTAT_PST_INF_FRAME_END_CLR BIT(9) +#define VIP_INTSTAT_ERR 0xFC + +/* VIP_FRAME STATUS */ +#define VIP_FRAME_STAT_CLS 0x00 +/* write 0 to clear frame 0 */ +#define VIP_FRAME_FRM0_STAT_CLS 0x20 + +/* VIP_FORMAT */ +#define VIP_FORMAT_VSY_HIGH_ACTIVE BIT(0) +#define VIP_FORMAT_HSY_LOW_ACTIVE BIT(1) + +#define VIP_FORMAT_INPUT_MODE_YUV (0x00 << 2) +#define VIP_FORMAT_INPUT_MODE_PAL (0x02 << 2) +#define VIP_FORMAT_INPUT_MODE_NTSC (0x03 << 2) +#define VIP_FORMAT_INPUT_MODE_BT1120 (0x07 << 2) +#define VIP_FORMAT_INPUT_MODE_RAW (0x04 << 2) +#define VIP_FORMAT_INPUT_MODE_JPEG (0x05 << 2) +#define VIP_FORMAT_INPUT_MODE_MIPI (0x06 << 2) + +#define VIP_FORMAT_YUV_INPUT_ORDER_UYVY (0x00 << 5) +#define VIP_FORMAT_YUV_INPUT_ORDER_YVYU BIT(5) +#define VIP_FORMAT_YUV_INPUT_ORDER_VYUY (0x10 << 5) +#define VIP_FORMAT_YUV_INPUT_ORDER_YUYV (0x03 << 5) +#define VIP_FORMAT_YUV_INPUT_422 (0x00 << 7) +#define VIP_FORMAT_YUV_INPUT_420 BIT(7) + +#define VIP_FORMAT_INPUT_420_ORDER_ODD BIT(8) + +#define VIP_FORMAT_CCIR_INPUT_ORDER_EVEN BIT(9) + +#define VIP_FORMAT_RAW_DATA_WIDTH_8 (0x00 << 11) +#define VIP_FORMAT_RAW_DATA_WIDTH_10 BIT(11) +#define VIP_FORMAT_RAW_DATA_WIDTH_12 (0x02 << 11) + +#define VIP_FORMAT_YUV_OUTPUT_422 (0x00 << 16) +#define VIP_FORMAT_YUV_OUTPUT_420 BIT(16) + +#define VIP_FORMAT_OUTPUT_420_ORDER_EVEN (0x00 << 17) +#define VIP_FORMAT_OUTPUT_420_ORDER_ODD BIT(17) + +#define VIP_FORMAT_RAWD_DATA_LITTLE_ENDIAN (0x00 << 18) +#define VIP_FORMAT_RAWD_DATA_BIG_ENDIAN BIT(18) + +#define VIP_FORMAT_UV_STORAGE_ORDER_UVUV (0x00 << 19) +#define VIP_FORMAT_UV_STORAGE_ORDER_VUVU BIT(19) + +#define VIP_FORMAT_BT1120_CLOCK_SINGLE_EDGES (0x00 << 24) +#define VIP_FORMAT_BT1120_CLOCK_DOUBLE_EDGES BIT(24) +#define VIP_FORMAT_BT1120_TRANSMIT_INTERFACE (0x00 << 25) +#define VIP_FORMAT_BT1120_TRANSMIT_PROGRESS BIT(25) +#define VIP_FORMAT_BT1120_YC_SWAP BIT(26) + +/* VIP_SCL_CTRL */ +#define VIP_SCL_CTRL_ENABLE_SCL_DOWN BIT(0) +#define VIP_SCL_CTRL_ENABLE_SCL_UP BIT(1) +#define VIP_SCL_CTRL_ENABLE_YUV_16BIT_BYPASS BIT(4) +#define VIP_SCL_CTRL_ENABLE_RAW_16BIT_BYPASS BIT(5) +#define VIP_SCL_CTRL_ENABLE_32BIT_BYPASS BIT(6) +#define VIP_SCL_CTRL_DISABLE_32BIT_BYPASS (0x00 << 6) + +/* VIP_INTSTAT */ +#define VIP_INTSTAT_F0_READY BIT(0) +#define VIP_INTSTAT_F1_READY BIT(1) + +/* VIP_CROP */ +#define VIP_CROP_Y_SHIFT 16 +#define VIP_CROP_X_SHIFT 0 + +/* VIP_CSI_ID_CTRL0 */ +#define VIP_CSI_ENABLE_CAPTURE BIT(0) +#define VIP_CSI_WRDDR_TYPE_RAW8 (0x0 << 1) +#define VIP_CSI_WRDDR_TYPE_RAW10 BIT(1) +#define VIP_CSI_WRDDR_TYPE_RAW12 (0x2 << 1) +#define VIP_CSI_WRDDR_TYPE_RGB888 (0x3 << 1) +#define VIP_CSI_WRDDR_TYPE_YUV422 (0x4 << 1) +#define VIP_CSI_ENABLE_COMMAND_MODE BIT(4) +#define VIP_CSI_ENABLE_CROP BIT(5) + +/* VIP_CSI_INTEN */ +#define VIP_CSI_FRAME0_START_INTEN(id) (0x1 << ((id) * 2)) +#define VIP_CSI_FRAME1_START_INTEN(id) (0x1 << ((id) * 2 + 1)) +#define VIP_CSI_FRAME0_END_INTEN(id) (0x1 << ((id) * 2 + 8)) +#define VIP_CSI_FRAME1_END_INTEN(id) (0x1 << ((id) * 2 + 9)) +#define VIP_CSI_DMA_Y_FIFO_OVERFLOW_INTEN BIT(16) +#define VIP_CSI_DMA_UV_FIFO_OVERFLOW_INTEN BIT(17) +#define VIP_CSI_CONFIG_FIFO_OVERFLOW_INTEN BIT(18) +#define VIP_CSI_BANDWIDTH_LACK_INTEN BIT(19) +#define VIP_CSI_RX_FIFO_OVERFLOW_INTEN BIT(20) +#define VIP_CSI_ALL_FRAME_START_INTEN (0xff << 0) +#define VIP_CSI_ALL_FRAME_END_INTEN (0xff << 8) +#define VIP_CSI_ALL_ERROR_INTEN (0x1f << 16) + +/* VIP_CSI_INTSTAT */ +#define VIP_CSI_FRAME0_START_ID0 BIT(0) +#define VIP_CSI_FRAME1_START_ID0 BIT(1) +#define VIP_CSI_FRAME0_START_ID1 BIT(2) +#define VIP_CSI_FRAME1_START_ID1 BIT(3) +#define VIP_CSI_FRAME0_START_ID2 BIT(4) +#define VIP_CSI_FRAME1_START_ID2 BIT(5) +#define VIP_CSI_FRAME0_START_ID3 BIT(6) +#define VIP_CSI_FRAME1_START_ID3 BIT(7) +#define VIP_CSI_FRAME0_END_ID0 BIT(8) +#define VIP_CSI_FRAME1_END_ID0 BIT(9) +#define VIP_CSI_FRAME0_END_ID1 BIT(10) +#define VIP_CSI_FRAME1_END_ID1 BIT(11) +#define VIP_CSI_FRAME0_END_ID2 BIT(12) +#define VIP_CSI_FRAME1_END_ID2 BIT(13) +#define VIP_CSI_FRAME0_END_ID3 BIT(14) +#define VIP_CSI_FRAME1_END_ID3 BIT(15) +#define VIP_CSI_DMA_Y_FIFO_OVERFLOW BIT(16) +#define VIP_CSI_DMA_UV_FIFO_OVERFLOW BIT(17) +#define VIP_CSI_CONFIG_FIFO_OVERFLOW BIT(18) +#define VIP_CSI_BANDWIDTH_LACK BIT(19) +#define VIP_CSI_RX_FIFO_OVERFLOW BIT(20) + +#define VIP_CSI_FIFO_OVERFLOW (VIP_CSI_DMA_Y_FIFO_OVERFLOW | \ + VIP_CSI_DMA_UV_FIFO_OVERFLOW | \ + VIP_CSI_CONFIG_FIFO_OVERFLOW | \ + VIP_CSI_RX_FIFO_OVERFLOW) + +/* CSI Host Registers Define */ +#define VIP_CSIHOST_N_LANES 0x04 +#define VIP_CSIHOST_PHY_RSTZ 0x0c +#define VIP_CSIHOST_RESETN 0x10 +#define VIP_CSIHOST_ERR1 0x20 +#define VIP_CSIHOST_ERR2 0x24 +#define VIP_CSIHOST_MSK1 0x28 +#define VIP_CSIHOST_MSK2 0x2c +#define VIP_CSIHOST_CONTROL 0x40 + +#define VIP_SW_CPHY_EN(x) ((x) << 0) +#define VIP_SW_DSI_EN(x) ((x) << 4) +#define VIP_SW_DATATYPE_FS(x) ((x) << 8) +#define VIP_SW_DATATYPE_FE(x) ((x) << 14) +#define VIP_SW_DATATYPE_LS(x) ((x) << 20) +#define VIP_SW_DATATYPE_LE(x) ((x) << 26) + +#endif -- 2.41.0 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-16 9:00 ` [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface Mehdi Djait @ 2023-10-19 15:40 ` Paul Kocialkowski 2023-10-20 15:38 ` Paul Kocialkowski 2023-10-23 13:28 ` Michael Riesch 2 siblings, 0 replies; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-19 15:40 UTC (permalink / raw) To: Mehdi Djait Cc: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 67064 bytes --] Hi, On Mon 16 Oct 23, 11:00, Mehdi Djait wrote: > This controller supports CSI2 and BT656 interfaces, but for Just a quick comment here: CSI2 is usually understood as MIPI CSI-2, which is not supported by the unit but is instead usually supported by the ISP1 unit. These are two separate units and we already have a driver for ISP1. I want to make it very clear that this driver is *not* a duplicate of rkisp1. To clarify, you could indicate that the unit supports parallel (aka DVP) interfaces such as BT656 (but not CSI2). The only exception (that I know of) is the RK1808 which has these same CIF registers but also has MIPI CSI-2 support in the same range. But for now we don't really need to care about this, besides making sure that the bindings are flexible enough to support a second input port. For now the driver is video-node centric and maybe this would call for a MC centric approach. But there will always be time to make this transition later on so we can focus on the driver as it is today. Cheers, Paul > now only the BT656 interface could be tested, hence it's the only one > that's supported in the first version of this driver. > > This controller can be fond on PX30, RK1808, RK3128 and RK3288, > but for now it's only been tested on PX30. > > Most of this driver was written following the BSP driver from rockchip, > removing the parts that either didn't fit correctly the guidelines, or > that couldn't be tested. > > In the BSP, this driver is known as the "cif" driver, but this was > renamed to "vip" to better fit the controller denomination in the > datasheet. > > This basic version doesn't support cropping nor scaling, and is only > designed with one SDTV video decoder being attached to it a any time. > > This version uses the "pingpong" mode of the controller, which is a > double-buffering mechanism. > > Signed-off-by: Mehdi Djait <mehdi.djait@bootlin.com> > --- > drivers/media/platform/rockchip/Kconfig | 1 + > drivers/media/platform/rockchip/Makefile | 1 + > drivers/media/platform/rockchip/vip/Kconfig | 14 + > drivers/media/platform/rockchip/vip/Makefile | 3 + > drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++ > drivers/media/platform/rockchip/vip/dev.c | 346 +++++ > drivers/media/platform/rockchip/vip/dev.h | 163 +++ > drivers/media/platform/rockchip/vip/regs.h | 260 ++++ > 8 files changed, 1998 insertions(+) > create mode 100644 drivers/media/platform/rockchip/vip/Kconfig > create mode 100644 drivers/media/platform/rockchip/vip/Makefile > create mode 100644 drivers/media/platform/rockchip/vip/capture.c > create mode 100644 drivers/media/platform/rockchip/vip/dev.c > create mode 100644 drivers/media/platform/rockchip/vip/dev.h > create mode 100644 drivers/media/platform/rockchip/vip/regs.h > > diff --git a/drivers/media/platform/rockchip/Kconfig b/drivers/media/platform/rockchip/Kconfig > index b41d3960c1b4..f82a56d1431f 100644 > --- a/drivers/media/platform/rockchip/Kconfig > +++ b/drivers/media/platform/rockchip/Kconfig > @@ -4,3 +4,4 @@ comment "Rockchip media platform drivers" > > source "drivers/media/platform/rockchip/rga/Kconfig" > source "drivers/media/platform/rockchip/rkisp1/Kconfig" > +source "drivers/media/platform/rockchip/vip/Kconfig" > diff --git a/drivers/media/platform/rockchip/Makefile b/drivers/media/platform/rockchip/Makefile > index 4f782b876ac9..3f1dade70103 100644 > --- a/drivers/media/platform/rockchip/Makefile > +++ b/drivers/media/platform/rockchip/Makefile > @@ -1,3 +1,4 @@ > # SPDX-License-Identifier: GPL-2.0-only > obj-y += rga/ > obj-y += rkisp1/ > +obj-y += vip/ > diff --git a/drivers/media/platform/rockchip/vip/Kconfig b/drivers/media/platform/rockchip/vip/Kconfig > new file mode 100644 > index 000000000000..05953e12a2a8 > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/Kconfig > @@ -0,0 +1,14 @@ > +config VIDEO_ROCKCHIP_VIP > + tristate "Rockchip VIP (Video InPut) Camera Interface" > + depends on VIDEO_DEV > + depends on ARCH_ROCKCHIP || COMPILE_TEST > + select VIDEOBUF2_DMA_SG > + select VIDEOBUF2_DMA_CONTIG > + select V4L2_FWNODE > + select V4L2_MEM2MEM_DEV > + help > + This is a v4l2 driver for Rockchip SOC Camera interface. It supports > + BT.656 and CSI2 inputs. > + > + To compile this driver as a module choose m here : the module will > + be called video_rkvip. > diff --git a/drivers/media/platform/rockchip/vip/Makefile b/drivers/media/platform/rockchip/vip/Makefile > new file mode 100644 > index 000000000000..c239ee0bb0fe > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/Makefile > @@ -0,0 +1,3 @@ > +# SPDX-License-Identifier: GPL-2.0 > +obj-$(CONFIG_VIDEO_ROCKCHIP_VIP) += video_rkvip.o > +video_rkvip-objs += dev.o capture.o > diff --git a/drivers/media/platform/rockchip/vip/capture.c b/drivers/media/platform/rockchip/vip/capture.c > new file mode 100644 > index 000000000000..e8f3480aacdb > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/capture.c > @@ -0,0 +1,1210 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Rockchip VIP Camera Interface Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2020 Maxime Chevallier <maxime.chevallier@bootlin.com> > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#include <linux/delay.h> > +#include <linux/interrupt.h> > +#include <linux/pm_runtime.h> > +#include <linux/reset.h> > +#include <media/v4l2-common.h> > +#include <media/v4l2-event.h> > +#include <media/v4l2-fh.h> > +#include <media/v4l2-fwnode.h> > +#include <media/v4l2-ioctl.h> > +#include <media/v4l2-mc.h> > +#include <media/v4l2-subdev.h> > +#include <media/videobuf2-dma-contig.h> > + > +#include "dev.h" > +#include "regs.h" > + > +#define VIP_REQ_BUFS_MIN 3 > +#define VIP_MIN_WIDTH 64 > +#define VIP_MIN_HEIGHT 64 > +#define VIP_MAX_WIDTH 8192 > +#define VIP_MAX_HEIGHT 8192 > + > +#define RK_VIP_PLANE_Y 0 > +#define RK_VIP_PLANE_CBCR 1 > + > +#define VIP_FETCH_Y_LAST_LINE(VAL) ((VAL) & 0x1fff) > + > +static struct vip_output_fmt out_fmts[] = { > + { > + .fourcc = V4L2_PIX_FMT_NV16, > + .fmt_val = VIP_FORMAT_YUV_OUTPUT_422 | > + VIP_FORMAT_UV_STORAGE_ORDER_UVUV, > + .cplanes = 2, > + }, { > + .fourcc = V4L2_PIX_FMT_NV61, > + .fmt_val = VIP_FORMAT_YUV_OUTPUT_422 | > + VIP_FORMAT_UV_STORAGE_ORDER_VUVU, > + .cplanes = 2, > + }, > + { > + .fourcc = V4L2_PIX_FMT_NV12, > + .fmt_val = VIP_FORMAT_YUV_OUTPUT_420 | > + VIP_FORMAT_UV_STORAGE_ORDER_UVUV, > + .cplanes = 2, > + }, > + { > + .fourcc = V4L2_PIX_FMT_NV21, > + .fmt_val = VIP_FORMAT_YUV_OUTPUT_420 | > + VIP_FORMAT_UV_STORAGE_ORDER_VUVU, > + .cplanes = 2, > + }, { > + .fourcc = V4L2_PIX_FMT_RGB24, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_RGB565, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_BGR666, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SRGGB8, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SGRBG8, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SGBRG8, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SBGGR8, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SRGGB10, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SGRBG10, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SGBRG10, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SBGGR10, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SRGGB12, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SGRBG12, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SGBRG12, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SBGGR12, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_SBGGR16, > + .cplanes = 1, > + }, { > + .fourcc = V4L2_PIX_FMT_Y16, > + .cplanes = 1, > + } > +}; > + > +static const struct vip_input_fmt in_fmts[] = { > + { > + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_YUYV, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_YUYV8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_YUYV, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_INTERLACED, > + }, { > + .mbus_code = MEDIA_BUS_FMT_YVYU8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_YVYU, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_YVYU8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_YVYU, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_INTERLACED, > + }, { > + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_UYVY, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_UYVY8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_UYVY, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_INTERLACED, > + }, { > + .mbus_code = MEDIA_BUS_FMT_VYUY8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_VYUY, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_VYUY8_2X8, > + .dvp_fmt_val = VIP_FORMAT_YUV_INPUT_422 | > + VIP_FORMAT_YUV_INPUT_ORDER_VYUY, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_YUV422, > + .fmt_type = VIP_FMT_TYPE_YUV, > + .field = V4L2_FIELD_INTERLACED, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SBGGR8_1X8, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_8, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SGBRG8_1X8, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_8, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SGRBG8_1X8, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_8, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SRGGB8_1X8, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_8, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SBGGR10_1X10, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_10, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SGBRG10_1X10, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_10, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SGRBG10_1X10, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_10, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SRGGB10_1X10, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_10, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SBGGR12_1X12, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_12, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SGBRG12_1X12, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_12, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SGRBG12_1X12, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_12, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_SRGGB12_1X12, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_12, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_RGB888_1X24, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RGB888, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_Y8_1X8, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_8, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW8, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_Y10_1X10, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_10, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW10, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + }, { > + .mbus_code = MEDIA_BUS_FMT_Y12_1X12, > + .dvp_fmt_val = VIP_FORMAT_INPUT_MODE_RAW | > + VIP_FORMAT_RAW_DATA_WIDTH_12, > + .csi_fmt_val = VIP_CSI_WRDDR_TYPE_RAW12, > + .fmt_type = VIP_FMT_TYPE_RAW, > + .field = V4L2_FIELD_NONE, > + } > +}; > + > +static const struct > +vip_input_fmt *get_input_fmt(struct v4l2_subdev *sd) > +{ > + struct v4l2_subdev_format fmt; > + u32 i; > + > + fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; > + fmt.pad = 0; > + v4l2_subdev_call(sd, pad, get_fmt, NULL, &fmt); > + > + for (i = 0; i < ARRAY_SIZE(in_fmts); i++) > + if (fmt.format.code == in_fmts[i].mbus_code && > + fmt.format.field == in_fmts[i].field) > + return &in_fmts[i]; > + > + v4l2_err(sd->v4l2_dev, "remote sensor mbus code not supported\n"); > + return NULL; > +} > + > +static struct > +vip_output_fmt *find_output_fmt(struct rk_vip_stream *stream, u32 pixelfmt) > +{ > + struct vip_output_fmt *fmt; > + u32 i; > + > + for (i = 0; i < ARRAY_SIZE(out_fmts); i++) { > + fmt = &out_fmts[i]; > + if (fmt->fourcc == pixelfmt) > + return fmt; > + } > + > + return NULL; > +} > + > +static struct rk_vip_buffer *rk_vip_get_buffer(struct rk_vip_stream *stream) > +{ > + struct rk_vip_buffer *buff; > + > + lockdep_assert_held(&stream->vbq_lock); > + > + if (list_empty(&stream->buf_head)) > + return NULL; > + > + buff = list_first_entry(&stream->buf_head, struct rk_vip_buffer, queue); > + list_del(&buff->queue); > + > + return buff; > +} > + > +static int rk_vip_init_buffers(struct rk_vip_stream *stream) > +{ > + struct rk_vip_device *vip_dev = stream->vipdev; > + void __iomem *base = vip_dev->base_addr; > + > + spin_lock(&stream->vbq_lock); > + > + stream->buffs[0] = rk_vip_get_buffer(stream); > + stream->buffs[1] = rk_vip_get_buffer(stream); > + > + if (!(stream->buffs[0]) || !(stream->buffs[1])) { > + spin_unlock(&stream->vbq_lock); > + return -EINVAL; > + } > + > + spin_unlock(&stream->vbq_lock); > + > + write_vip_reg(base, VIP_FRM0_ADDR_Y, > + stream->buffs[0]->buff_addr[RK_VIP_PLANE_Y]); > + write_vip_reg(base, VIP_FRM0_ADDR_UV, > + stream->buffs[0]->buff_addr[RK_VIP_PLANE_CBCR]); > + > + write_vip_reg(base, VIP_FRM1_ADDR_Y, > + stream->buffs[1]->buff_addr[RK_VIP_PLANE_Y]); > + write_vip_reg(base, VIP_FRM1_ADDR_UV, > + stream->buffs[1]->buff_addr[RK_VIP_PLANE_CBCR]); > + > + return 0; > +} > + > +static void rk_vip_assign_new_buffer_pingpong(struct rk_vip_stream *stream) > +{ > + struct rk_vip_scratch_buffer *scratch_buf = &stream->scratch_buf; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct rk_vip_buffer *buffer = NULL; > + void __iomem *base = vip_dev->base_addr; > + u32 frm_addr_y, frm_addr_uv; > + > + /* Set up an empty buffer for the next frame */ > + spin_lock(&stream->vbq_lock); > + > + buffer = rk_vip_get_buffer(stream); > + > + stream->buffs[stream->frame_phase] = buffer; > + > + spin_unlock(&stream->vbq_lock); > + > + frm_addr_y = stream->frame_phase ? VIP_FRM1_ADDR_Y : VIP_FRM0_ADDR_Y; > + frm_addr_uv = stream->frame_phase ? VIP_FRM1_ADDR_UV : VIP_FRM0_ADDR_UV; > + > + if (buffer) { > + write_vip_reg(base, frm_addr_y, > + buffer->buff_addr[RK_VIP_PLANE_Y]); > + write_vip_reg(base, frm_addr_uv, > + buffer->buff_addr[RK_VIP_PLANE_CBCR]); > + } else { > + write_vip_reg(base, frm_addr_y, scratch_buf->dma_addr); > + write_vip_reg(base, frm_addr_uv, scratch_buf->dma_addr); > + } > +} > + > +static void rk_vip_stream_stop(struct rk_vip_stream *stream) > +{ > + struct rk_vip_device *vip_dev = stream->vipdev; > + void __iomem *base = vip_dev->base_addr; > + u32 val; > + > + val = read_vip_reg(base, VIP_CTRL); > + write_vip_reg(base, VIP_CTRL, val & (~VIP_CTRL_ENABLE_CAPTURE)); > + write_vip_reg(base, VIP_INTEN, 0x0); > + write_vip_reg(base, VIP_INTSTAT, 0x3ff); > + write_vip_reg(base, VIP_FRAME_STATUS, 0x0); > + > + stream->state = RK_VIP_STATE_READY; > +} > + > +static int rk_vip_queue_setup(struct vb2_queue *queue, > + unsigned int *num_buffers, > + unsigned int *num_planes, > + unsigned int sizes[], > + struct device *alloc_devs[]) > +{ > + struct rk_vip_stream *stream = queue->drv_priv; > + const struct v4l2_plane_pix_format *plane_fmt; > + const struct v4l2_pix_format_mplane *pixm; > + > + pixm = &stream->pixm; > + > + if (*num_planes) { > + if (*num_planes != 1) > + return -EINVAL; > + > + if (sizes[0] < pixm->plane_fmt[0].sizeimage) > + return -EINVAL; > + return 0; > + } > + > + *num_planes = 1; > + > + plane_fmt = &pixm->plane_fmt[0]; > + sizes[0] = plane_fmt->sizeimage; > + > + *num_buffers = VIP_REQ_BUFS_MIN; > + > + return 0; > +} > + > +static void rk_vip_buf_queue(struct vb2_buffer *vb) > +{ > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); > + struct rk_vip_buffer *vipbuf = to_rk_vip_buffer(vbuf); > + struct vb2_queue *queue = vb->vb2_queue; > + struct rk_vip_stream *stream = queue->drv_priv; > + struct v4l2_pix_format_mplane *pixm = &stream->pixm; > + unsigned long lock_flags = 0; > + int i; > + > + struct vip_output_fmt *fmt = stream->vip_fmt_out; > + > + memset(vipbuf->buff_addr, 0, sizeof(vipbuf->buff_addr)); > + > + vipbuf->buff_addr[0] = vb2_dma_contig_plane_dma_addr(vb, 0); > + > + for (i = 0; i < fmt->cplanes - 1; i++) > + vipbuf->buff_addr[i + 1] = vipbuf->buff_addr[i] + > + pixm->plane_fmt[i].bytesperline * pixm->height; > + > + spin_lock_irqsave(&stream->vbq_lock, lock_flags); > + list_add_tail(&vipbuf->queue, &stream->buf_head); > + spin_unlock_irqrestore(&stream->vbq_lock, lock_flags); > +} > + > +static int rk_vip_create_scratch_buf(struct rk_vip_stream *stream) > +{ > + struct rk_vip_scratch_buffer *scratch_buf = &stream->scratch_buf; > + struct rk_vip_device *vip_dev = stream->vipdev; > + > + /* get a maximum plane size */ > + scratch_buf->size = max(stream->pixm.plane_fmt[0].sizeimage, > + stream->pixm.plane_fmt[1].sizeimage); > + > + scratch_buf->vaddr = dma_alloc_coherent(vip_dev->dev, scratch_buf->size, > + &scratch_buf->dma_addr, > + GFP_KERNEL); > + if (!scratch_buf->vaddr) > + return -ENOMEM; > + > + return 0; > +} > + > +static void rk_vip_destroy_scratch_buf(struct rk_vip_stream *stream) > +{ > + struct rk_vip_scratch_buffer *scratch_buf = &stream->scratch_buf; > + struct rk_vip_device *vip_dev = stream->vipdev; > + > + dma_free_coherent(vip_dev->dev, scratch_buf->size, > + scratch_buf->vaddr, scratch_buf->dma_addr); > +} > + > +static void rk_vip_stop_streaming(struct vb2_queue *queue) > +{ > + struct rk_vip_stream *stream = queue->drv_priv; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct rk_vip_buffer *buf; > + struct v4l2_subdev *sd; > + int ret; > + > + stream->stopping = true; > + ret = wait_event_timeout(stream->wq_stopped, > + stream->state != RK_VIP_STATE_STREAMING, > + msecs_to_jiffies(1000)); > + if (!ret) { > + rk_vip_stream_stop(stream); > + stream->stopping = false; > + } > + pm_runtime_put(vip_dev->dev); > + > + /* stop the sub device*/ > + sd = vip_dev->sensor.sd; > + v4l2_subdev_call(sd, video, s_stream, 0); > + > + /* release buffers */ > + if (stream->buffs[0]) { > + list_add_tail(&stream->buffs[0]->queue, &stream->buf_head); > + stream->buffs[0] = NULL; > + } > + > + if (stream->buffs[1]) { > + list_add_tail(&stream->buffs[1]->queue, &stream->buf_head); > + stream->buffs[1] = NULL; > + } > + > + while (!list_empty(&stream->buf_head)) { > + buf = rk_vip_get_buffer(stream); > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); > + } > + > + rk_vip_destroy_scratch_buf(stream); > +} > + > +static u32 rk_vip_determine_input_mode(struct rk_vip_device *vip_dev) > +{ > + v4l2_std_id std = vip_dev->sensor.std; > + > + return (std == V4L2_STD_NTSC) ? > + VIP_FORMAT_INPUT_MODE_NTSC : > + VIP_FORMAT_INPUT_MODE_PAL; > +} > + > +static inline u32 rk_vip_scl_ctl(struct rk_vip_stream *stream) > +{ > + u32 fmt_type = stream->vip_fmt_in->fmt_type; > + > + return (fmt_type == VIP_FMT_TYPE_YUV) ? > + VIP_SCL_CTRL_ENABLE_YUV_16BIT_BYPASS : > + VIP_SCL_CTRL_ENABLE_RAW_16BIT_BYPASS; > +} > + > +static int rk_vip_stream_start(struct rk_vip_stream *stream) > +{ > + u32 val, mbus_flags, href_pol, vsync_pol, > + xfer_mode = 0, yc_swap = 0, skip_top = 0; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct rk_vip_sensor_info *sensor_info; > + void __iomem *base = vip_dev->base_addr; > + int ret; > + u32 input_mode; > + > + sensor_info = &vip_dev->sensor; > + stream->frame_idx = 0; > + input_mode = rk_vip_determine_input_mode(vip_dev); > + > + mbus_flags = sensor_info->mbus.bus.parallel.flags; > + href_pol = (mbus_flags & V4L2_MBUS_HSYNC_ACTIVE_HIGH) ? > + 0 : VIP_FORMAT_HSY_LOW_ACTIVE; > + vsync_pol = (mbus_flags & V4L2_MBUS_VSYNC_ACTIVE_HIGH) ? > + VIP_FORMAT_VSY_HIGH_ACTIVE : 0; > + > + val = vsync_pol | href_pol | input_mode | stream->vip_fmt_out->fmt_val | > + stream->vip_fmt_in->dvp_fmt_val | xfer_mode | yc_swap; > + write_vip_reg(base, VIP_FOR, val); > + > + val = stream->pixm.width; > + if (stream->vip_fmt_in->fmt_type == VIP_FMT_TYPE_RAW) > + val = stream->pixm.width * 2; > + > + write_vip_reg(base, VIP_VIR_LINE_WIDTH, val); > + write_vip_reg(base, VIP_SET_SIZE, > + stream->pixm.width | (stream->pixm.height << 16)); > + > + v4l2_subdev_call(sensor_info->sd, sensor, g_skip_top_lines, &skip_top); > + > + write_vip_reg(base, VIP_CROP, skip_top << VIP_CROP_Y_SHIFT); > + write_vip_reg(base, VIP_FRAME_STATUS, VIP_FRAME_STAT_CLS); > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_CLS); > + write_vip_reg(base, VIP_SCL_CTRL, rk_vip_scl_ctl(stream)); > + > + ret = rk_vip_init_buffers(stream); > + if (ret) > + return ret; > + > + write_vip_reg(base, VIP_INTEN, VIP_INTEN_FRAME_END_EN | > + VIP_INTEN_LINE_ERR_EN | > + VIP_INTEN_PST_INF_FRAME_END_EN); > + > + write_vip_reg(base, VIP_CTRL, VIP_CTRL_AXI_BURST_16 | > + VIP_CTRL_MODE_PINGPONG | > + VIP_CTRL_ENABLE_CAPTURE); > + > + stream->state = RK_VIP_STATE_STREAMING; > + > + return 0; > +} > + > +static int rk_vip_start_streaming(struct vb2_queue *queue, unsigned int count) > +{ > + struct rk_vip_stream *stream = queue->drv_priv; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_device *v4l2_dev = &vip_dev->v4l2_dev; > + struct v4l2_subdev *sd; > + int ret; > + > + if (stream->state != RK_VIP_STATE_READY) { > + ret = -EBUSY; > + v4l2_err(v4l2_dev, "Stream in busy state\n"); > + goto destroy_buf; > + } > + > + if (!vip_dev->sensor.sd) { > + ret = -EINVAL; > + v4l2_err(v4l2_dev, "No sensor subdev detected\n"); > + goto destroy_buf; > + } > + > + ret = rk_vip_create_scratch_buf(stream); > + if (ret < 0) { > + v4l2_err(v4l2_dev, "Failed to create scratch_buf, %d\n", ret); > + goto destroy_buf; > + } > + > + ret = pm_runtime_get_sync(vip_dev->dev); > + if (ret < 0) { > + v4l2_err(v4l2_dev, "Failed to get runtime pm, %d\n", ret); > + goto destroy_scratch_buf; > + } > + > + /* start sub-devices */ > + sd = vip_dev->sensor.sd; > + stream->vip_fmt_in = get_input_fmt(vip_dev->sensor.sd); > + > + ret = v4l2_subdev_call(sd, video, s_stream, 1); > + if (ret < 0) > + goto runtime_put; > + > + ret = rk_vip_stream_start(stream); > + if (ret < 0) > + goto stop_stream; > + > + return 0; > + > +stop_stream: > + rk_vip_stream_stop(stream); > +runtime_put: > + pm_runtime_put(vip_dev->dev); > +destroy_scratch_buf: > + rk_vip_destroy_scratch_buf(stream); > +destroy_buf: > + while (!list_empty(&stream->buf_head)) { > + struct rk_vip_buffer *buf; > + > + buf = rk_vip_get_buffer(stream); > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); > + } > + > + return ret; > +} > + > +static const struct vb2_ops rk_vip_vb2_ops = { > + .queue_setup = rk_vip_queue_setup, > + .buf_queue = rk_vip_buf_queue, > + .wait_prepare = vb2_ops_wait_prepare, > + .wait_finish = vb2_ops_wait_finish, > + .stop_streaming = rk_vip_stop_streaming, > + .start_streaming = rk_vip_start_streaming, > +}; > + > +static int rk_vip_init_vb2_queue(struct vb2_queue *q, > + struct rk_vip_stream *stream) > +{ > + q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; > + q->io_modes = VB2_MMAP | VB2_DMABUF; > + q->drv_priv = stream; > + q->ops = &rk_vip_vb2_ops; > + q->mem_ops = &vb2_dma_contig_memops; > + q->buf_struct_size = sizeof(struct rk_vip_buffer); > + q->min_buffers_needed = VIP_REQ_BUFS_MIN; > + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; > + q->lock = &stream->vlock; > + q->dev = stream->vipdev->dev; > + > + return vb2_queue_init(q); > +} > + > +static void rk_vip_update_pixm(struct rk_vip_stream *stream, > + struct vip_output_fmt *fmt, > + struct v4l2_pix_format_mplane *pixm) > +{ > + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; > + struct v4l2_subdev_format sd_fmt; > + struct v4l2_rect input_rect; > + u32 width, height; > + > + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; > + sd_fmt.pad = 0; > + v4l2_subdev_call(sensor_info->sd, pad, get_fmt, NULL, &sd_fmt); > + > + input_rect.width = VIP_MAX_WIDTH; > + input_rect.height = VIP_MAX_HEIGHT; > + width = clamp_t(u32, sd_fmt.format.width, > + VIP_MIN_WIDTH, input_rect.width); > + height = clamp_t(u32, sd_fmt.format.height, > + VIP_MIN_HEIGHT, input_rect.height); > + > + pixm->width = width; > + pixm->height = height; > + pixm->field = sd_fmt.format.field; > + pixm->colorspace = sd_fmt.format.colorspace; > + pixm->ycbcr_enc = sd_fmt.format.ycbcr_enc; > + pixm->quantization = sd_fmt.format.quantization; > + pixm->xfer_func = sd_fmt.format.xfer_func; > + > + v4l2_fill_pixfmt_mp(pixm, fmt->fourcc, pixm->width, pixm->height); > +} > + > +static int rk_vip_set_fmt(struct rk_vip_stream *stream, > + struct v4l2_pix_format_mplane *pixm) > +{ > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_subdev_format sd_fmt; > + struct vip_output_fmt *fmt; > + int ret; > + > + if (stream->state == RK_VIP_STATE_STREAMING) > + return -EBUSY; > + > + fmt = find_output_fmt(stream, pixm->pixelformat); > + if (!fmt) > + fmt = &out_fmts[0]; > + > + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; > + sd_fmt.pad = 0; > + sd_fmt.format.width = pixm->width; > + sd_fmt.format.height = pixm->height; > + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, set_fmt, NULL, &sd_fmt); > + rk_vip_update_pixm(stream, fmt, pixm); > + stream->pixm = *pixm; > + stream->vip_fmt_out = fmt; > + > + return ret; > +} > + > +void rk_vip_set_default_format(struct rk_vip_device *vip_dev) > +{ > + struct rk_vip_stream *stream = &vip_dev->stream; > + struct v4l2_pix_format_mplane pixm; > + > + pixm.pixelformat = V4L2_PIX_FMT_NV12; > + pixm.width = RK_VIP_DEFAULT_WIDTH; > + pixm.height = RK_VIP_DEFAULT_HEIGHT; > + rk_vip_set_fmt(stream, &pixm); > +} > + > +void rk_vip_stream_init(struct rk_vip_device *vip_dev) > +{ > + struct rk_vip_stream *stream = &vip_dev->stream; > + struct v4l2_pix_format_mplane pixm; > + > + memset(stream, 0, sizeof(*stream)); > + memset(&pixm, 0, sizeof(pixm)); > + stream->vipdev = vip_dev; > + > + INIT_LIST_HEAD(&stream->buf_head); > + spin_lock_init(&stream->vbq_lock); > + stream->state = RK_VIP_STATE_READY; > + init_waitqueue_head(&stream->wq_stopped); > +} > + > +static const struct v4l2_file_operations rk_vip_fops = { > + .open = v4l2_fh_open, > + .release = vb2_fop_release, > + .unlocked_ioctl = video_ioctl2, > + .poll = vb2_fop_poll, > + .mmap = vb2_fop_mmap, > +}; > + > +static int rk_vip_enum_input(struct file *file, void *priv, > + struct v4l2_input *input) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct v4l2_subdev *sd = stream->vipdev->sensor.sd; > + int ret; > + > + if (input->index > 0) > + return -EINVAL; > + > + ret = v4l2_subdev_call(sd, video, g_input_status, &input->status); > + if (ret) > + return ret; > + > + strscpy(input->name, "Camera", sizeof(input->name)); > + input->type = V4L2_INPUT_TYPE_CAMERA; > + input->std = stream->vdev.tvnorms; > + input->capabilities = V4L2_IN_CAP_STD; > + > + return 0; > +} > + > +static int rk_vip_try_fmt_vid_cap_mplane(struct file *file, void *fh, > + struct v4l2_format *f) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct vip_output_fmt *fmt; > + > + fmt = find_output_fmt(stream, f->fmt.pix_mp.pixelformat); > + if (!fmt) > + fmt = &out_fmts[0]; > + > + rk_vip_update_pixm(stream, fmt, &f->fmt.pix_mp); > + > + return 0; > +} > + > +static int rk_vip_g_std(struct file *file, void *fh, v4l2_std_id *norm) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; > + > + *norm = sensor_info->std; > + > + return 0; > +} > + > +static int rk_vip_s_std(struct file *file, void *fh, v4l2_std_id norm) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; > + int ret; > + > + if (norm == sensor_info->std) > + return 0; > + > + if (stream->state == RK_VIP_STATE_STREAMING) > + return -EBUSY; > + > + ret = v4l2_subdev_call(sensor_info->sd, video, s_std, norm); > + if (ret) > + return ret; > + > + sensor_info->std = norm; > + > + /* S_STD will update the format since that depends on the standard */ > + rk_vip_update_pixm(stream, stream->vip_fmt_out, &stream->pixm); > + > + return 0; > +} > + > +static int rk_vip_querystd(struct file *file, void *fh, v4l2_std_id *a) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; > + > + *a = V4L2_STD_UNKNOWN; > + > + return v4l2_subdev_call(sensor_info->sd, video, querystd, a); > +} > + > +static int rk_vip_enum_fmt_vid_cap(struct file *file, void *priv, > + struct v4l2_fmtdesc *f) > +{ > + struct vip_output_fmt *fmt = NULL; > + > + if (f->index >= ARRAY_SIZE(out_fmts)) > + return -EINVAL; > + > + fmt = &out_fmts[f->index]; > + f->pixelformat = fmt->fourcc; > + > + return 0; > +} > + > +static int rk_vip_s_fmt_vid_cap_mplane(struct file *file, > + void *priv, struct v4l2_format *f) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + int ret; > + > + if (stream->state == RK_VIP_STATE_STREAMING) > + return -EBUSY; > + > + ret = rk_vip_set_fmt(stream, &f->fmt.pix_mp); > + > + return ret; > +} > + > +static int rk_vip_g_fmt_vid_cap_mplane(struct file *file, void *fh, > + struct v4l2_format *f) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + > + f->fmt.pix_mp = stream->pixm; > + > + return 0; > +} > + > +static int rk_vip_querycap(struct file *file, void *priv, > + struct v4l2_capability *cap) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct device *dev = stream->vipdev->dev; > + > + strscpy(cap->driver, dev->driver->name, sizeof(cap->driver)); > + strscpy(cap->card, dev->driver->name, sizeof(cap->card)); > + snprintf(cap->bus_info, sizeof(cap->bus_info), > + "platform:%s", dev_name(dev)); > + > + return 0; > +} > + > +static int rk_vip_enum_framesizes(struct file *file, void *fh, > + struct v4l2_frmsizeenum *fsize) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_subdev_frame_size_enum fse = { > + .index = fsize->index, > + .which = V4L2_SUBDEV_FORMAT_ACTIVE, > + }; > + struct vip_output_fmt *fmt; > + int ret; > + > + if (!vip_dev->sensor.sd) > + return -EINVAL; > + > + fmt = find_output_fmt(stream, fsize->pixel_format); > + if (!fmt) > + return -EINVAL; > + > + fse.code = fmt->mbus; > + > + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, enum_frame_size, > + NULL, &fse); > + if (ret) > + return ret; > + > + fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE; > + fsize->discrete.width = fse.max_width; > + fsize->discrete.height = fse.max_height; > + > + return 0; > +} > + > +static int rk_vip_enum_frameintervals(struct file *file, void *fh, > + struct v4l2_frmivalenum *fival) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_subdev_frame_interval_enum fie = { > + .index = fival->index, > + .width = fival->width, > + .height = fival->height, > + .which = V4L2_SUBDEV_FORMAT_ACTIVE, > + }; > + struct vip_output_fmt *fmt; > + int ret; > + > + if (!vip_dev->sensor.sd) > + return -EINVAL; > + > + fmt = find_output_fmt(stream, fival->pixel_format); > + if (!fmt) > + return -EINVAL; > + > + fie.code = fmt->mbus; > + > + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, enum_frame_interval, > + NULL, &fie); > + if (ret) > + return ret; > + > + fival->type = V4L2_FRMSIZE_TYPE_DISCRETE; > + fival->discrete = fie.interval; > + > + return 0; > +} > + > +static int rk_vip_g_input(struct file *file, void *fh, unsigned int *i) > +{ > + *i = 0; > + return 0; > +} > + > +static int rk_vip_s_input(struct file *file, void *fh, unsigned int i) > +{ > + if (i) > + return -EINVAL; > + > + return 0; > +} > + > +static int rk_vip_g_parm(struct file *file, void *priv, > + struct v4l2_streamparm *p) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_device *vip_dev = stream->vipdev; > + > + if (!vip_dev->sensor.sd) > + return -EINVAL; > + > + return v4l2_g_parm_cap(video_devdata(file), vip_dev->sensor.sd, p); > +} > + > +static int rk_vip_s_parm(struct file *file, void *priv, > + struct v4l2_streamparm *p) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_device *vip_dev = stream->vipdev; > + > + if (!vip_dev->sensor.sd) > + return -EINVAL; > + > + return v4l2_s_parm_cap(video_devdata(file), vip_dev->sensor.sd, p); > +} > + > +static const struct v4l2_ioctl_ops rk_vip_v4l2_ioctl_ops = { > + .vidioc_reqbufs = vb2_ioctl_reqbufs, > + .vidioc_querybuf = vb2_ioctl_querybuf, > + .vidioc_create_bufs = vb2_ioctl_create_bufs, > + .vidioc_qbuf = vb2_ioctl_qbuf, > + .vidioc_expbuf = vb2_ioctl_expbuf, > + .vidioc_dqbuf = vb2_ioctl_dqbuf, > + .vidioc_prepare_buf = vb2_ioctl_prepare_buf, > + .vidioc_streamon = vb2_ioctl_streamon, > + .vidioc_streamoff = vb2_ioctl_streamoff, > + > + .vidioc_g_std = rk_vip_g_std, > + .vidioc_s_std = rk_vip_s_std, > + .vidioc_querystd = rk_vip_querystd, > + > + .vidioc_enum_fmt_vid_cap = rk_vip_enum_fmt_vid_cap, > + .vidioc_try_fmt_vid_cap_mplane = rk_vip_try_fmt_vid_cap_mplane, > + .vidioc_s_fmt_vid_cap_mplane = rk_vip_s_fmt_vid_cap_mplane, > + .vidioc_g_fmt_vid_cap_mplane = rk_vip_g_fmt_vid_cap_mplane, > + .vidioc_querycap = rk_vip_querycap, > + .vidioc_enum_framesizes = rk_vip_enum_framesizes, > + .vidioc_enum_frameintervals = rk_vip_enum_frameintervals, > + > + .vidioc_enum_input = rk_vip_enum_input, > + .vidioc_g_input = rk_vip_g_input, > + .vidioc_s_input = rk_vip_s_input, > + > + .vidioc_g_parm = rk_vip_g_parm, > + .vidioc_s_parm = rk_vip_s_parm, > +}; > + > +void rk_vip_unregister_stream_vdev(struct rk_vip_device *vip_dev) > +{ > + struct rk_vip_stream *stream = &vip_dev->stream; > + > + media_entity_cleanup(&stream->vdev.entity); > + video_unregister_device(&stream->vdev); > +} > + > +int rk_vip_register_stream_vdev(struct rk_vip_device *vip_dev) > +{ > + struct rk_vip_stream *stream = &vip_dev->stream; > + struct v4l2_device *v4l2_dev = &vip_dev->v4l2_dev; > + struct video_device *vdev = &stream->vdev; > + int ret; > + > + strscpy(vdev->name, VIP_VIDEODEVICE_NAME, sizeof(vdev->name)); > + mutex_init(&stream->vlock); > + > + vdev->ioctl_ops = &rk_vip_v4l2_ioctl_ops; > + vdev->release = video_device_release_empty; > + vdev->fops = &rk_vip_fops; > + vdev->minor = -1; > + vdev->v4l2_dev = v4l2_dev; > + vdev->lock = &stream->vlock; > + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE | > + V4L2_CAP_STREAMING; > + vdev->tvnorms = V4L2_STD_NTSC | V4L2_STD_PAL; > + video_set_drvdata(vdev, stream); > + vdev->vfl_dir = VFL_DIR_RX; > + stream->pad.flags = MEDIA_PAD_FL_SINK; > + > + rk_vip_init_vb2_queue(&stream->buf_queue, stream); > + > + vdev->queue = &stream->buf_queue; > + strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name)); > + > + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); > + if (ret < 0) { > + v4l2_err(v4l2_dev, > + "video_register_device failed with error %d\n", ret); > + return ret; > + } > + > + ret = media_entity_pads_init(&vdev->entity, 1, &stream->pad); > + if (ret < 0) > + goto unreg; > + > + return 0; > +unreg: > + video_unregister_device(vdev); > + return ret; > +} > + > +static void rk_vip_vb_done(struct rk_vip_stream *stream, > + struct vb2_v4l2_buffer *vb_done) > +{ > + vb2_set_plane_payload(&vb_done->vb2_buf, 0, > + stream->pixm.plane_fmt[0].sizeimage); > + vb_done->vb2_buf.timestamp = ktime_get_ns(); > + vb_done->sequence = stream->frame_idx; > + vb2_buffer_done(&vb_done->vb2_buf, VB2_BUF_STATE_DONE); > +} > + > +static void rk_vip_reset_stream(struct rk_vip_device *vip_dev) > +{ > + void __iomem *base = vip_dev->base_addr; > + u32 ctl = read_vip_reg(base, VIP_CTRL); > + > + write_vip_reg(base, VIP_CTRL, ctl & (~VIP_CTRL_ENABLE_CAPTURE)); > + write_vip_reg(base, VIP_CTRL, ctl | VIP_CTRL_ENABLE_CAPTURE); > +} > + > +irqreturn_t rk_vip_irq_pingpong(int irq, void *ctx) > +{ > + struct device *dev = ctx; > + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); > + struct rk_vip_stream *stream = &vip_dev->stream; > + void __iomem *base = vip_dev->base_addr; > + unsigned int intstat; > + > + u32 lastline, lastpix, ctl, vip_frmst; > + > + intstat = read_vip_reg(base, VIP_INTSTAT); > + vip_frmst = read_vip_reg(base, VIP_FRAME_STATUS); > + lastline = VIP_FETCH_Y_LAST_LINE(read_vip_reg(base, VIP_LAST_LINE)); > + lastpix = VIP_FETCH_Y_LAST_LINE(read_vip_reg(base, VIP_LAST_PIX)); > + ctl = read_vip_reg(base, VIP_CTRL); > + > + /* There are two irqs enabled: > + * - PST_INF_FRAME_END: vip FIFO is ready, > + * this is prior to FRAME_END > + * - FRAME_END: vip has saved frame to memory, > + * a frame ready > + */ > + > + if ((intstat & VIP_INTSTAT_PST_INF_FRAME_END)) { > + write_vip_reg(base, VIP_INTSTAT, > + VIP_INTSTAT_PST_INF_FRAME_END_CLR); > + if (stream->stopping) > + /* To stop VIP ASAP, before FRAME_END irq */ > + write_vip_reg(base, VIP_CTRL, > + ctl & (~VIP_CTRL_ENABLE_CAPTURE)); > + } > + > + if ((intstat & VIP_INTSTAT_PRE_INF_FRAME_END)) > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_PRE_INF_FRAME_END); > + > + if (intstat & (VIP_INTSTAT_LINE_ERR | VIP_INTSTAT_PIX_ERR)) { > + v4l2_err(&vip_dev->v4l2_dev, "LINE_ERR OR PIX_ERR"); > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_LINE_ERR | > + VIP_INTSTAT_PIX_ERR); > + rk_vip_reset_stream(vip_dev); > + } > + > + if ((intstat & VIP_INTSTAT_FRAME_END)) { > + struct vb2_v4l2_buffer *vb_done = NULL; > + > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_FRAME_END_CLR | > + VIP_INTSTAT_LINE_END_CLR); > + > + if (stream->stopping) { > + rk_vip_stream_stop(stream); > + stream->stopping = false; > + wake_up(&stream->wq_stopped); > + return IRQ_HANDLED; > + } > + > + if (lastline != stream->pixm.height) { > + v4l2_err(&vip_dev->v4l2_dev, > + "Bad frame, irq:0x%x frmst:0x%x size:%dx%d\n", > + intstat, vip_frmst, lastpix, lastline); > + > + rk_vip_reset_stream(vip_dev); > + } > + > + if (vip_frmst & VIP_INTSTAT_F0_READY) > + stream->frame_phase = 0; > + else if (vip_frmst & VIP_INTSTAT_F1_READY) > + stream->frame_phase = 1; > + else > + return IRQ_HANDLED; > + > + if (stream->buffs[stream->frame_phase]) > + vb_done = &stream->buffs[stream->frame_phase]->vb; > + > + rk_vip_assign_new_buffer_pingpong(stream); > + > + if (vb_done) > + rk_vip_vb_done(stream, vb_done); > + > + stream->frame_idx++; > + } > + > + return IRQ_HANDLED; > +} > diff --git a/drivers/media/platform/rockchip/vip/dev.c b/drivers/media/platform/rockchip/vip/dev.c > new file mode 100644 > index 000000000000..c3931e5accbd > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/dev.c > @@ -0,0 +1,346 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Rockchip VIP Camera Interface Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2020 Maxime Chevallier <maxime.chevallier@bootlin.com> > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#include <linux/clk.h> > +#include <linux/delay.h> > +#include <linux/interrupt.h> > +#include <linux/module.h> > +#include <linux/of.h> > +#include <linux/of_graph.h> > +#include <linux/of_platform.h> > +#include <linux/of_reserved_mem.h> > +#include <linux/reset.h> > +#include <linux/pm_runtime.h> > +#include <linux/pinctrl/consumer.h> > +#include <media/v4l2-fwnode.h> > + > +#include "dev.h" > +#include "regs.h" > + > +#define RK_VIP_VERNO_LEN 10 > + > +struct vip_match_data { > + int chip_id; > + const char * const *clks; > + const char * const *rsts; > + int clks_num; > + int rsts_num; > +}; > + > +static int rk_vip_create_links(struct rk_vip_device *vip_dev) > +{ > + struct v4l2_subdev *sd = vip_dev->sensor.sd; > + int ret; > + > + ret = media_create_pad_link(&sd->entity, 0, > + &vip_dev->stream.vdev.entity, 0, > + MEDIA_LNK_FL_ENABLED); > + if (ret) { > + dev_err(vip_dev->dev, "failed to create link"); > + return ret; > + } > + > + return 0; > +} > + > +static int subdev_notifier_complete(struct v4l2_async_notifier *notifier) > +{ > + struct rk_vip_device *vip_dev; > + int ret; > + > + vip_dev = container_of(notifier, struct rk_vip_device, notifier); > + > + mutex_lock(&vip_dev->media_dev.graph_mutex); > + > + ret = v4l2_device_register_subdev_nodes(&vip_dev->v4l2_dev); > + if (ret < 0) > + goto unlock; > + > + ret = rk_vip_create_links(vip_dev); > + if (ret < 0) > + goto unlock; > + > +unlock: > + mutex_unlock(&vip_dev->media_dev.graph_mutex); > + return ret; > +} > + > +static int subdev_notifier_bound(struct v4l2_async_notifier *notifier, > + struct v4l2_subdev *subdev, > + struct v4l2_async_connection *asd) > +{ > + struct rk_vip_device *vip_dev = container_of(notifier, > + struct rk_vip_device, notifier); > + > + int pad; > + > + vip_dev->sensor.sd = subdev; > + pad = media_entity_get_fwnode_pad(&subdev->entity, subdev->fwnode, > + MEDIA_PAD_FL_SOURCE); > + if (pad < 0) > + return pad; > + > + vip_dev->sensor.pad = pad; > + > + return 0; > +} > + > +static const struct v4l2_async_notifier_operations subdev_notifier_ops = { > + .bound = subdev_notifier_bound, > + .complete = subdev_notifier_complete, > +}; > + > +static int vip_subdev_notifier(struct rk_vip_device *vip_dev) > +{ > + struct v4l2_async_notifier *ntf = &vip_dev->notifier; > + struct device *dev = vip_dev->dev; > + struct v4l2_async_connection *asd; > + struct v4l2_fwnode_endpoint vep = { > + .bus_type = V4L2_MBUS_PARALLEL, > + }; > + struct fwnode_handle *ep; > + int ret; > + > + v4l2_async_nf_init(ntf, &vip_dev->v4l2_dev); > + > + ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), 0, 0, > + FWNODE_GRAPH_ENDPOINT_NEXT); > + if (!ep) > + return -EINVAL; > + > + ret = v4l2_fwnode_endpoint_parse(ep, &vep); > + if (ret) > + return ret; > + > + asd = v4l2_async_nf_add_fwnode_remote(ntf, ep, > + struct v4l2_async_connection); > + if (IS_ERR(asd)) { > + ret = PTR_ERR(asd); > + return ret; > + } > + > + ntf->ops = &subdev_notifier_ops; > + > + fwnode_handle_put(ep); > + > + ret = v4l2_async_nf_register(ntf); > + return ret; > +} > + > +static int rk_vip_register_platform_subdevs(struct rk_vip_device *vip_dev) > +{ > + int ret; > + > + ret = rk_vip_register_stream_vdev(vip_dev); > + if (ret < 0) > + return ret; > + > + ret = vip_subdev_notifier(vip_dev); > + if (ret < 0) { > + v4l2_err(&vip_dev->v4l2_dev, > + "Failed to register subdev notifier(%d)\n", ret); > + rk_vip_unregister_stream_vdev(vip_dev); > + } > + > + return 0; > +} > + > +static const char * const px30_vip_clks[] = { > + "aclk", > + "hclk", > + "pclk", > +}; > + > +static const struct vip_match_data px30_vip_match_data = { > + .chip_id = CHIP_PX30_VIP, > + .clks = px30_vip_clks, > + .clks_num = ARRAY_SIZE(px30_vip_clks), > +}; > + > +static const struct of_device_id rk_vip_plat_of_match[] = { > + { > + .compatible = "rockchip,px30-vip", > + .data = &px30_vip_match_data, > + }, > + {}, > +}; > + > +void rk_vip_soft_reset(struct rk_vip_device *vip_dev) > +{ > + reset_control_assert(vip_dev->vip_rst); > + > + udelay(5); > + > + reset_control_deassert(vip_dev->vip_rst); > +} > + > +static int rk_vip_get_resource(struct platform_device *pdev, > + struct rk_vip_device *vip_dev) > +{ > + struct resource *res; > + > + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); > + if (!res) { > + dev_err(&pdev->dev, > + "Unable to allocate resources for device\n"); > + return -ENODEV; > + } > + > + vip_dev->base_addr = devm_ioremap_resource(&pdev->dev, res); > + if (IS_ERR(vip_dev->base_addr)) > + return PTR_ERR(vip_dev->base_addr); > + > + return 0; > +} > + > +static int rk_vip_plat_probe(struct platform_device *pdev) > +{ > + const struct of_device_id *match; > + struct device_node *node = pdev->dev.of_node; > + struct device *dev = &pdev->dev; > + struct v4l2_device *v4l2_dev; > + struct rk_vip_device *vip_dev; > + const struct vip_match_data *data; > + int i, ret, irq; > + > + match = of_match_node(rk_vip_plat_of_match, node); > + if (!match) > + return -ENODEV; > + > + vip_dev = devm_kzalloc(dev, sizeof(*vip_dev), GFP_KERNEL); > + if (!vip_dev) > + return -ENOMEM; > + > + dev_set_drvdata(dev, vip_dev); > + vip_dev->dev = dev; > + > + irq = platform_get_irq(pdev, 0); > + if (irq < 0) > + return irq; > + > + ret = devm_request_irq(dev, irq, rk_vip_irq_pingpong, 0, > + dev_driver_string(dev), dev); > + if (ret < 0) > + return dev_err_probe(dev, ret, "request irq failed\n"); > + > + vip_dev->irq = irq; > + data = match->data; > + vip_dev->chip_id = data->chip_id; > + > + ret = rk_vip_get_resource(pdev, vip_dev); > + if (ret) > + return ret; > + > + for (i = 0; i < data->clks_num; i++) > + vip_dev->clks[i].id = data->clks[i]; > + > + vip_dev->num_clk = data->clks_num; > + > + ret = devm_clk_bulk_get(dev, vip_dev->num_clk, vip_dev->clks); > + if (ret) > + return ret; > + > + vip_dev->vip_rst = devm_reset_control_array_get(dev, false, false); > + if (IS_ERR(vip_dev->vip_rst)) > + return PTR_ERR(vip_dev->vip_rst); > + > + /* Initialize the stream */ > + rk_vip_stream_init(vip_dev); > + strscpy(vip_dev->media_dev.model, "rk_vip", > + sizeof(vip_dev->media_dev.model)); > + vip_dev->media_dev.dev = &pdev->dev; > + v4l2_dev = &vip_dev->v4l2_dev; > + v4l2_dev->mdev = &vip_dev->media_dev; > + strscpy(v4l2_dev->name, "rk_vip", sizeof(v4l2_dev->name)); > + > + ret = v4l2_device_register(vip_dev->dev, &vip_dev->v4l2_dev); > + if (ret < 0) > + return ret; > + > + media_device_init(&vip_dev->media_dev); > + > + ret = media_device_register(&vip_dev->media_dev); > + if (ret < 0) { > + v4l2_err(v4l2_dev, "Failed to register media device: %d\n", > + ret); > + goto err_unreg_v4l2_dev; > + } > + > + /* create & register platform subdev (from of_node) */ > + ret = rk_vip_register_platform_subdevs(vip_dev); > + if (ret < 0) > + goto err_unreg_media_dev; > + > + vip_dev->sensor.std = V4L2_STD_NTSC; > + rk_vip_set_default_format(vip_dev); > + pm_runtime_enable(&pdev->dev); > + > + return 0; > + > +err_unreg_media_dev: > + media_device_unregister(&vip_dev->media_dev); > +err_unreg_v4l2_dev: > + v4l2_device_unregister(&vip_dev->v4l2_dev); > + return ret; > +} > + > +static int rk_vip_plat_remove(struct platform_device *pdev) > +{ > + struct rk_vip_device *vip_dev = platform_get_drvdata(pdev); > + > + pm_runtime_disable(&pdev->dev); > + > + media_device_unregister(&vip_dev->media_dev); > + v4l2_device_unregister(&vip_dev->v4l2_dev); > + rk_vip_unregister_stream_vdev(vip_dev); > + > + return 0; > +} > + > +static int __maybe_unused rk_vip_runtime_suspend(struct device *dev) > +{ > + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); > + > + clk_bulk_disable_unprepare(vip_dev->num_clk, vip_dev->clks); > + > + return pinctrl_pm_select_sleep_state(dev); > +} > + > +static int __maybe_unused rk_vip_runtime_resume(struct device *dev) > +{ > + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); > + int ret; > + > + ret = pinctrl_pm_select_default_state(dev); > + if (ret < 0) > + return ret; > + > + return clk_bulk_prepare_enable(vip_dev->num_clk, vip_dev->clks); > +} > + > +static const struct dev_pm_ops rk_vip_plat_pm_ops = { > + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, > + pm_runtime_force_resume) > + SET_RUNTIME_PM_OPS(rk_vip_runtime_suspend, rk_vip_runtime_resume, NULL) > +}; > + > +static struct platform_driver rk_vip_plat_drv = { > + .driver = { > + .name = VIP_DRIVER_NAME, > + .of_match_table = rk_vip_plat_of_match, > + .pm = &rk_vip_plat_pm_ops, > + }, > + .probe = rk_vip_plat_probe, > + .remove = rk_vip_plat_remove, > +}; > +module_platform_driver(rk_vip_plat_drv); > + > +MODULE_AUTHOR("Rockchip Camera/ISP team"); > +MODULE_DESCRIPTION("Rockchip VIP platform driver"); > +MODULE_LICENSE("GPL"); > diff --git a/drivers/media/platform/rockchip/vip/dev.h b/drivers/media/platform/rockchip/vip/dev.h > new file mode 100644 > index 000000000000..eb9cd8f20ffc > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/dev.h > @@ -0,0 +1,163 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * Rockchip VIP Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#ifndef _RK_VIP_DEV_H > +#define _RK_VIP_DEV_H > + > +#include <linux/clk.h> > +#include <linux/mutex.h> > +#include <media/media-device.h> > +#include <media/media-entity.h> > +#include <media/v4l2-ctrls.h> > +#include <media/v4l2-device.h> > +#include <media/videobuf2-v4l2.h> > + > +#define VIP_DRIVER_NAME "rk_vip" > +#define VIP_VIDEODEVICE_NAME "stream_vip" > + > +#define RK_VIP_MAX_BUS_CLK 8 > +#define RK_VIP_MAX_SENSOR 2 > +#define RK_VIP_MAX_RESET 5 > +#define RK_VIP_MAX_CSI_CHANNEL 4 > + > +#define RK_VIP_DEFAULT_WIDTH 640 > +#define RK_VIP_DEFAULT_HEIGHT 480 > + > +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) > +#define read_vip_reg(base, addr) readl((addr) + (base)) > + > +enum rk_vip_state { > + RK_VIP_STATE_DISABLED, > + RK_VIP_STATE_READY, > + RK_VIP_STATE_STREAMING > +}; > + > +enum rk_vip_chip_id { > + CHIP_PX30_VIP, > + CHIP_RK1808_VIP, > + CHIP_RK3128_VIP, > + CHIP_RK3288_VIP > +}; > + > +enum host_type_t { > + RK_CSI_RXHOST, > + RK_DSI_RXHOST > +}; > + > +struct rk_vip_buffer { > + struct vb2_v4l2_buffer vb; > + struct list_head queue; > + union { > + u32 buff_addr[VIDEO_MAX_PLANES]; > + void *vaddr[VIDEO_MAX_PLANES]; > + }; > +}; > + > +struct rk_vip_scratch_buffer { > + void *vaddr; > + dma_addr_t dma_addr; > + u32 size; > +}; > + > +static inline struct rk_vip_buffer *to_rk_vip_buffer(struct vb2_v4l2_buffer *vb) > +{ > + return container_of(vb, struct rk_vip_buffer, vb); > +} > + > +struct rk_vip_sensor_info { > + struct v4l2_subdev *sd; > + int pad; > + struct v4l2_mbus_config mbus; > + int lanes; > + v4l2_std_id std; > +}; > + > +struct vip_output_fmt { > + u32 fourcc; > + u32 mbus; > + u32 fmt_val; > + u8 cplanes; > +}; > + > +enum vip_fmt_type { > + VIP_FMT_TYPE_YUV = 0, > + VIP_FMT_TYPE_RAW, > +}; > + > +struct vip_input_fmt { > + u32 mbus_code; > + u32 dvp_fmt_val; > + u32 csi_fmt_val; > + enum vip_fmt_type fmt_type; > + enum v4l2_field field; > +}; > + > +struct rk_vip_stream { > + struct rk_vip_device *vipdev; > + enum rk_vip_state state; > + bool stopping; > + wait_queue_head_t wq_stopped; > + int frame_idx; > + int frame_phase; > + > + /* lock between irq and buf_queue */ > + spinlock_t vbq_lock; > + struct vb2_queue buf_queue; > + struct list_head buf_head; > + struct rk_vip_scratch_buffer scratch_buf; > + struct rk_vip_buffer *buffs[2]; > + > + /* vfd lock */ > + struct mutex vlock; > + struct video_device vdev; > + struct media_pad pad; > + > + struct vip_output_fmt *vip_fmt_out; > + const struct vip_input_fmt *vip_fmt_in; > + struct v4l2_pix_format_mplane pixm; > +}; > + > +static inline struct rk_vip_stream *to_rk_vip_stream(struct video_device *vdev) > +{ > + return container_of(vdev, struct rk_vip_stream, vdev); > +} > + > +struct rk_vip_device { > + struct list_head list; > + struct device *dev; > + int irq; > + void __iomem *base_addr; > + void __iomem *csi_base; > + struct clk_bulk_data clks[RK_VIP_MAX_BUS_CLK]; > + int num_clk; > + struct vb2_alloc_ctx *alloc_ctx; > + bool iommu_en; > + struct iommu_domain *domain; > + struct reset_control *vip_rst; > + > + struct v4l2_device v4l2_dev; > + struct media_device media_dev; > + struct v4l2_ctrl_handler ctrl_handler; > + struct v4l2_async_notifier notifier; > + struct v4l2_async_connection asd; > + struct rk_vip_sensor_info sensor; > + > + struct rk_vip_stream stream; > + > + int chip_id; > +}; > + > +void rk_vip_unregister_stream_vdev(struct rk_vip_device *dev); > +int rk_vip_register_stream_vdev(struct rk_vip_device *dev); > +void rk_vip_stream_init(struct rk_vip_device *dev); > +void rk_vip_set_default_format(struct rk_vip_device *dev); > + > +irqreturn_t rk_vip_irq_pingpong(int irq, void *ctx); > +void rk_vip_soft_reset(struct rk_vip_device *vip_dev); > + > +#endif > diff --git a/drivers/media/platform/rockchip/vip/regs.h b/drivers/media/platform/rockchip/vip/regs.h > new file mode 100644 > index 000000000000..ccf10ffbbff8 > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/regs.h > @@ -0,0 +1,260 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * Rockchip VIP Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#ifndef _RK_VIP_REGS_H > +#define _RK_VIP_REGS_H > + > +/* VIP Reg Offset */ > +#define VIP_CTRL 0x00 > +#define VIP_INTEN 0x04 > +#define VIP_INTSTAT 0x08 > +#define VIP_FOR 0x0c > +#define VIP_LINE_NUM_ADDR 0x10 > +#define VIP_FRM0_ADDR_Y 0x14 > +#define VIP_FRM0_ADDR_UV 0x18 > +#define VIP_FRM1_ADDR_Y 0x1c > +#define VIP_FRM1_ADDR_UV 0x20 > +#define VIP_VIR_LINE_WIDTH 0x24 > +#define VIP_SET_SIZE 0x28 > +#define VIP_SCM_ADDR_Y 0x2c > +#define VIP_SCM_ADDR_U 0x30 > +#define VIP_SCM_ADDR_V 0x34 > +#define VIP_WB_UP_FILTER 0x38 > +#define VIP_WB_LOW_FILTER 0x3c > +#define VIP_WBC_CNT 0x40 > +#define VIP_CROP 0x44 > +#define VIP_SCL_CTRL 0x48 > +#define VIP_SCL_DST 0x4c > +#define VIP_SCL_FCT 0x50 > +#define VIP_SCL_VALID_NUM 0x54 > +#define VIP_LINE_LOOP_CTR 0x58 > +#define VIP_FRAME_STATUS 0x60 > +#define VIP_CUR_DST 0x64 > +#define VIP_LAST_LINE 0x68 > +#define VIP_LAST_PIX 0x6c > + > +/* RK1808 VIP CSI Registers Offset */ > +#define VIP_CSI_ID0_CTRL0 0x80 > +#define VIP_CSI_ID0_CTRL1 0x84 > +#define VIP_CSI_ID1_CTRL0 0x88 > +#define VIP_CSI_ID1_CTRL1 0x8c > +#define VIP_CSI_ID2_CTRL0 0x90 > +#define VIP_CSI_ID2_CTRL1 0x94 > +#define VIP_CSI_ID3_CTRL0 0x98 > +#define VIP_CSI_ID3_CTRL1 0x9c > +#define VIP_CSI_WATER_LINE 0xa0 > +#define VIP_CSI_FRM0_ADDR_Y_ID0 0xa4 > +#define VIP_CSI_FRM1_ADDR_Y_ID0 0xa8 > +#define VIP_CSI_FRM0_ADDR_UV_ID0 0xac > +#define VIP_CSI_FRM1_ADDR_UV_ID0 0xb0 > +#define VIP_CSI_FRM0_VLW_Y_ID0 0xb4 > +#define VIP_CSI_FRM1_VLW_Y_ID0 0xb8 > +#define VIP_CSI_FRM0_VLW_UV_ID0 0xbc > +#define VIP_CSI_FRM1_VLW_UV_ID0 0xc0 > +#define VIP_CSI_FRM0_ADDR_Y_ID1 0xc4 > +#define VIP_CSI_FRM1_ADDR_Y_ID1 0xc8 > +#define VIP_CSI_FRM0_ADDR_UV_ID1 0xcc > +#define VIP_CSI_FRM1_ADDR_UV_ID1 0xd0 > +#define VIP_CSI_FRM0_VLW_Y_ID1 0xd4 > +#define VIP_CSI_FRM1_VLW_Y_ID1 0xd8 > +#define VIP_CSI_FRM0_VLW_UV_ID1 0xdc > +#define VIP_CSI_FRM1_VLW_UV_ID1 0xe0 > +#define VIP_CSI_FRM0_ADDR_Y_ID2 0xe4 > +#define VIP_CSI_FRM1_ADDR_Y_ID2 0xe8 > +#define VIP_CSI_FRM0_ADDR_UV_ID2 0xec > +#define VIP_CSI_FRM1_ADDR_UV_ID2 0xf0 > +#define VIP_CSI_FRM0_VLW_Y_ID2 0xf4 > +#define VIP_CSI_FRM1_VLW_Y_ID2 0xf8 > +#define VIP_CSI_FRM0_VLW_UV_ID2 0xfc > +#define VIP_CSI_FRM1_VLW_UV_ID2 0x100 > +#define VIP_CSI_FRM0_ADDR_Y_ID3 0x104 > +#define VIP_CSI_FRM1_ADDR_Y_ID3 0x108 > +#define VIP_CSI_FRM0_ADDR_UV_ID3 0x10c > +#define VIP_CSI_FRM1_ADDR_UV_ID3 0x110 > +#define VIP_CSI_FRM0_VLW_Y_ID3 0x114 > +#define VIP_CSI_FRM1_VLW_Y_ID3 0x118 > +#define VIP_CSI_FRM0_VLW_UV_ID3 0x11c > +#define VIP_CSI_FRM1_VLW_UV_ID3 0x120 > +#define VIP_CSI_INTEN 0x124 > +#define VIP_CSI_INTSTAT 0x128 > +#define VIP_CSI_LINE_INT_NUM_ID0_1 0x12c > +#define VIP_CSI_LINE_INT_NUM_ID2_3 0x130 > +#define VIP_CSI_LINE_CNT_ID0_1 0x134 > +#define VIP_CSI_LINE_CNT_ID2_3 0x138 > +#define VIP_CSI_ID0_CROP_START 0x13c > +#define VIP_CSI_ID1_CROP_START 0x140 > +#define VIP_CSI_ID2_CROP_START 0x144 > +#define VIP_CSI_ID3_CROP_START 0x148 > + > +/* The key register bit description */ > + > +/* VIP_CTRL Reg */ > +#define VIP_CTRL_ENABLE_CAPTURE BIT(0) > +#define VIP_CTRL_MODE_PINGPONG BIT(1) > +#define VIP_CTRL_MODE_LINELOOP BIT(2) > +#define VIP_CTRL_AXI_BURST_16 (0xF << 12) > + > +/* VIP_INTEN */ > +#define VIP_INTEN_FRAME_END_EN BIT(0) > +#define VIP_INTEN_LINE_ERR_EN BIT(2) > +#define VIP_INTEN_BUS_ERR_EN BIT(6) > +#define VIP_INTEN_SCL_ERR_EN BIT(7) > +#define VIP_INTEN_PST_INF_FRAME_END_EN BIT(9) > + > +/* VIP INTSTAT */ > +#define VIP_INTSTAT_CLS 0x3FF > +#define VIP_INTSTAT_FRAME_END BIT(0) > +#define VIP_INTSTAT_LINE_END BIT(1) > +#define VIP_INTSTAT_LINE_ERR BIT(2) > +#define VIP_INTSTAT_PIX_ERR BIT(3) > +#define VIP_INTSTAT_DFIFO_OF BIT(5) > +#define VIP_INTSTAT_BUS_ERR BIT(6) > +#define VIP_INTSTAT_PRE_INF_FRAME_END BIT(8) > +#define VIP_INTSTAT_PST_INF_FRAME_END BIT(9) > +#define VIP_INTSTAT_FRAME_END_CLR BIT(0) > +#define VIP_INTSTAT_LINE_END_CLR BIT(1) > +#define VIP_INTSTAT_LINE_ERR_CLR BIT(2) > +#define VIP_INTSTAT_PST_INF_FRAME_END_CLR BIT(9) > +#define VIP_INTSTAT_ERR 0xFC > + > +/* VIP_FRAME STATUS */ > +#define VIP_FRAME_STAT_CLS 0x00 > +/* write 0 to clear frame 0 */ > +#define VIP_FRAME_FRM0_STAT_CLS 0x20 > + > +/* VIP_FORMAT */ > +#define VIP_FORMAT_VSY_HIGH_ACTIVE BIT(0) > +#define VIP_FORMAT_HSY_LOW_ACTIVE BIT(1) > + > +#define VIP_FORMAT_INPUT_MODE_YUV (0x00 << 2) > +#define VIP_FORMAT_INPUT_MODE_PAL (0x02 << 2) > +#define VIP_FORMAT_INPUT_MODE_NTSC (0x03 << 2) > +#define VIP_FORMAT_INPUT_MODE_BT1120 (0x07 << 2) > +#define VIP_FORMAT_INPUT_MODE_RAW (0x04 << 2) > +#define VIP_FORMAT_INPUT_MODE_JPEG (0x05 << 2) > +#define VIP_FORMAT_INPUT_MODE_MIPI (0x06 << 2) > + > +#define VIP_FORMAT_YUV_INPUT_ORDER_UYVY (0x00 << 5) > +#define VIP_FORMAT_YUV_INPUT_ORDER_YVYU BIT(5) > +#define VIP_FORMAT_YUV_INPUT_ORDER_VYUY (0x10 << 5) > +#define VIP_FORMAT_YUV_INPUT_ORDER_YUYV (0x03 << 5) > +#define VIP_FORMAT_YUV_INPUT_422 (0x00 << 7) > +#define VIP_FORMAT_YUV_INPUT_420 BIT(7) > + > +#define VIP_FORMAT_INPUT_420_ORDER_ODD BIT(8) > + > +#define VIP_FORMAT_CCIR_INPUT_ORDER_EVEN BIT(9) > + > +#define VIP_FORMAT_RAW_DATA_WIDTH_8 (0x00 << 11) > +#define VIP_FORMAT_RAW_DATA_WIDTH_10 BIT(11) > +#define VIP_FORMAT_RAW_DATA_WIDTH_12 (0x02 << 11) > + > +#define VIP_FORMAT_YUV_OUTPUT_422 (0x00 << 16) > +#define VIP_FORMAT_YUV_OUTPUT_420 BIT(16) > + > +#define VIP_FORMAT_OUTPUT_420_ORDER_EVEN (0x00 << 17) > +#define VIP_FORMAT_OUTPUT_420_ORDER_ODD BIT(17) > + > +#define VIP_FORMAT_RAWD_DATA_LITTLE_ENDIAN (0x00 << 18) > +#define VIP_FORMAT_RAWD_DATA_BIG_ENDIAN BIT(18) > + > +#define VIP_FORMAT_UV_STORAGE_ORDER_UVUV (0x00 << 19) > +#define VIP_FORMAT_UV_STORAGE_ORDER_VUVU BIT(19) > + > +#define VIP_FORMAT_BT1120_CLOCK_SINGLE_EDGES (0x00 << 24) > +#define VIP_FORMAT_BT1120_CLOCK_DOUBLE_EDGES BIT(24) > +#define VIP_FORMAT_BT1120_TRANSMIT_INTERFACE (0x00 << 25) > +#define VIP_FORMAT_BT1120_TRANSMIT_PROGRESS BIT(25) > +#define VIP_FORMAT_BT1120_YC_SWAP BIT(26) > + > +/* VIP_SCL_CTRL */ > +#define VIP_SCL_CTRL_ENABLE_SCL_DOWN BIT(0) > +#define VIP_SCL_CTRL_ENABLE_SCL_UP BIT(1) > +#define VIP_SCL_CTRL_ENABLE_YUV_16BIT_BYPASS BIT(4) > +#define VIP_SCL_CTRL_ENABLE_RAW_16BIT_BYPASS BIT(5) > +#define VIP_SCL_CTRL_ENABLE_32BIT_BYPASS BIT(6) > +#define VIP_SCL_CTRL_DISABLE_32BIT_BYPASS (0x00 << 6) > + > +/* VIP_INTSTAT */ > +#define VIP_INTSTAT_F0_READY BIT(0) > +#define VIP_INTSTAT_F1_READY BIT(1) > + > +/* VIP_CROP */ > +#define VIP_CROP_Y_SHIFT 16 > +#define VIP_CROP_X_SHIFT 0 > + > +/* VIP_CSI_ID_CTRL0 */ > +#define VIP_CSI_ENABLE_CAPTURE BIT(0) > +#define VIP_CSI_WRDDR_TYPE_RAW8 (0x0 << 1) > +#define VIP_CSI_WRDDR_TYPE_RAW10 BIT(1) > +#define VIP_CSI_WRDDR_TYPE_RAW12 (0x2 << 1) > +#define VIP_CSI_WRDDR_TYPE_RGB888 (0x3 << 1) > +#define VIP_CSI_WRDDR_TYPE_YUV422 (0x4 << 1) > +#define VIP_CSI_ENABLE_COMMAND_MODE BIT(4) > +#define VIP_CSI_ENABLE_CROP BIT(5) > + > +/* VIP_CSI_INTEN */ > +#define VIP_CSI_FRAME0_START_INTEN(id) (0x1 << ((id) * 2)) > +#define VIP_CSI_FRAME1_START_INTEN(id) (0x1 << ((id) * 2 + 1)) > +#define VIP_CSI_FRAME0_END_INTEN(id) (0x1 << ((id) * 2 + 8)) > +#define VIP_CSI_FRAME1_END_INTEN(id) (0x1 << ((id) * 2 + 9)) > +#define VIP_CSI_DMA_Y_FIFO_OVERFLOW_INTEN BIT(16) > +#define VIP_CSI_DMA_UV_FIFO_OVERFLOW_INTEN BIT(17) > +#define VIP_CSI_CONFIG_FIFO_OVERFLOW_INTEN BIT(18) > +#define VIP_CSI_BANDWIDTH_LACK_INTEN BIT(19) > +#define VIP_CSI_RX_FIFO_OVERFLOW_INTEN BIT(20) > +#define VIP_CSI_ALL_FRAME_START_INTEN (0xff << 0) > +#define VIP_CSI_ALL_FRAME_END_INTEN (0xff << 8) > +#define VIP_CSI_ALL_ERROR_INTEN (0x1f << 16) > + > +/* VIP_CSI_INTSTAT */ > +#define VIP_CSI_FRAME0_START_ID0 BIT(0) > +#define VIP_CSI_FRAME1_START_ID0 BIT(1) > +#define VIP_CSI_FRAME0_START_ID1 BIT(2) > +#define VIP_CSI_FRAME1_START_ID1 BIT(3) > +#define VIP_CSI_FRAME0_START_ID2 BIT(4) > +#define VIP_CSI_FRAME1_START_ID2 BIT(5) > +#define VIP_CSI_FRAME0_START_ID3 BIT(6) > +#define VIP_CSI_FRAME1_START_ID3 BIT(7) > +#define VIP_CSI_FRAME0_END_ID0 BIT(8) > +#define VIP_CSI_FRAME1_END_ID0 BIT(9) > +#define VIP_CSI_FRAME0_END_ID1 BIT(10) > +#define VIP_CSI_FRAME1_END_ID1 BIT(11) > +#define VIP_CSI_FRAME0_END_ID2 BIT(12) > +#define VIP_CSI_FRAME1_END_ID2 BIT(13) > +#define VIP_CSI_FRAME0_END_ID3 BIT(14) > +#define VIP_CSI_FRAME1_END_ID3 BIT(15) > +#define VIP_CSI_DMA_Y_FIFO_OVERFLOW BIT(16) > +#define VIP_CSI_DMA_UV_FIFO_OVERFLOW BIT(17) > +#define VIP_CSI_CONFIG_FIFO_OVERFLOW BIT(18) > +#define VIP_CSI_BANDWIDTH_LACK BIT(19) > +#define VIP_CSI_RX_FIFO_OVERFLOW BIT(20) > + > +#define VIP_CSI_FIFO_OVERFLOW (VIP_CSI_DMA_Y_FIFO_OVERFLOW | \ > + VIP_CSI_DMA_UV_FIFO_OVERFLOW | \ > + VIP_CSI_CONFIG_FIFO_OVERFLOW | \ > + VIP_CSI_RX_FIFO_OVERFLOW) > + > +/* CSI Host Registers Define */ > +#define VIP_CSIHOST_N_LANES 0x04 > +#define VIP_CSIHOST_PHY_RSTZ 0x0c > +#define VIP_CSIHOST_RESETN 0x10 > +#define VIP_CSIHOST_ERR1 0x20 > +#define VIP_CSIHOST_ERR2 0x24 > +#define VIP_CSIHOST_MSK1 0x28 > +#define VIP_CSIHOST_MSK2 0x2c > +#define VIP_CSIHOST_CONTROL 0x40 > + > +#define VIP_SW_CPHY_EN(x) ((x) << 0) > +#define VIP_SW_DSI_EN(x) ((x) << 4) > +#define VIP_SW_DATATYPE_FS(x) ((x) << 8) > +#define VIP_SW_DATATYPE_FE(x) ((x) << 14) > +#define VIP_SW_DATATYPE_LS(x) ((x) << 20) > +#define VIP_SW_DATATYPE_LE(x) ((x) << 26) > + > +#endif > -- > 2.41.0 > -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-16 9:00 ` [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface Mehdi Djait 2023-10-19 15:40 ` Paul Kocialkowski @ 2023-10-20 15:38 ` Paul Kocialkowski 2023-10-23 13:28 ` Michael Riesch 2 siblings, 0 replies; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-20 15:38 UTC (permalink / raw) To: Mehdi Djait Cc: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 45513 bytes --] Hi Mehdi. On Mon 16 Oct 23, 11:00, Mehdi Djait wrote: > Introduce a driver for the camera interface on some Rockchip platforms. Here is a more thorough review of the driver. > This controller supports CSI2 and BT656 interfaces, but for > now only the BT656 interface could be tested, hence it's the only one > that's supported in the first version of this driver. Please mention the fact that this driver is video node-centric and not media controller-centric. > This controller can be fond on PX30, RK1808, RK3128 and RK3288, Typo: found. Also please update the list based on my previous email. > In the BSP, this driver is known as the "cif" driver, but this was > renamed to "vip" to better fit the controller denomination in the > datasheet. Please keep the cif name, as it is more representative of the unit. > +++ b/drivers/media/platform/rockchip/vip/Kconfig > @@ -0,0 +1,14 @@ > +config VIDEO_ROCKCHIP_VIP > + tristate "Rockchip VIP (Video InPut) Camera Interface" > + depends on VIDEO_DEV Add a depends on V4L_PLATFORM_DRIVERS too. Add a depends on PM && COMMON_CLK too. > + depends on ARCH_ROCKCHIP || COMPILE_TEST > + select VIDEOBUF2_DMA_SG I don't see where scatter-gather is used in the driver. If it's not used you should remove this. > + select VIDEOBUF2_DMA_CONTIG > + select V4L2_FWNODE Add a select on VIDEO_V4L2_SUBDEV_API. > + select V4L2_MEM2MEM_DEV I don't think this driver is using m2m so this should be removed. > + help > + This is a v4l2 driver for Rockchip SOC Camera interface. It supports > + BT.656 and CSI2 inputs. Please correct this to clarify that it supports parallel interfaces, often called "DVP" in rockchip terminology such as BT.656 but not CSI2. You can mention that this is called CIF and VIP here. Also it's pretty redundant that this is a v4l2 driver. > + To compile this driver as a module choose m here : the module will > + be called video_rkvip. This is generally redundant information, you can just remove it. > diff --git a/drivers/media/platform/rockchip/vip/Makefile b/drivers/media/platform/rockchip/vip/Makefile > new file mode 100644 > index 000000000000..c239ee0bb0fe > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/Makefile > @@ -0,0 +1,3 @@ > +# SPDX-License-Identifier: GPL-2.0 > +obj-$(CONFIG_VIDEO_ROCKCHIP_VIP) += video_rkvip.o > +video_rkvip-objs += dev.o capture.o That name is a bit unusual. You should call it "rockchip-cif" to be consistent with the rkisp1 and rga module names. > diff --git a/drivers/media/platform/rockchip/vip/capture.c b/drivers/media/platform/rockchip/vip/capture.c > new file mode 100644 > index 000000000000..e8f3480aacdb > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/capture.c > @@ -0,0 +1,1210 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Rockchip VIP Camera Interface Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2020 Maxime Chevallier <maxime.chevallier@bootlin.com> > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#include <linux/delay.h> > +#include <linux/interrupt.h> > +#include <linux/pm_runtime.h> > +#include <linux/reset.h> > +#include <media/v4l2-common.h> > +#include <media/v4l2-event.h> Not sure you need v4l2-event.h > +#include <media/v4l2-fh.h> > +#include <media/v4l2-fwnode.h> > +#include <media/v4l2-ioctl.h> > +#include <media/v4l2-mc.h> > +#include <media/v4l2-subdev.h> > +#include <media/videobuf2-dma-contig.h> > + > +#include "dev.h" > +#include "regs.h" > + > +#define VIP_REQ_BUFS_MIN 3 What's the justification for 3 buffers? In a regular double-buffering mechanism it should be able to operate with just two. > +#define VIP_MIN_WIDTH 64 > +#define VIP_MIN_HEIGHT 64 > +#define VIP_MAX_WIDTH 8192 > +#define VIP_MAX_HEIGHT 8192 > + > +#define RK_VIP_PLANE_Y 0 > +#define RK_VIP_PLANE_CBCR 1 Keep the prefix consistent: VIP/CIF instead of RK_VIP/RK_CIF. The same issue exists with a bunch of functions too (vip vs rk_vip) prefix and write_vip_reg/read_vip_reg which puts the prefix in the middle. Please go through this and make it consistent. Also name the latter "UV" insyead of "CBCR" since CbCr is most often used in analogue setups. > + > +#define VIP_FETCH_Y_LAST_LINE(VAL) ((VAL) & 0x1fff) Seems like this should be part of the register defines. > +static int rk_vip_init_buffers(struct rk_vip_stream *stream) > +{ > + struct rk_vip_device *vip_dev = stream->vipdev; > + void __iomem *base = vip_dev->base_addr; > + > + spin_lock(&stream->vbq_lock); You should probably irqsave/irqrestore here. > + > + stream->buffs[0] = rk_vip_get_buffer(stream); > + stream->buffs[1] = rk_vip_get_buffer(stream); > + > + if (!(stream->buffs[0]) || !(stream->buffs[1])) { > + spin_unlock(&stream->vbq_lock); > + return -EINVAL; > + } > + > + spin_unlock(&stream->vbq_lock); > + > + write_vip_reg(base, VIP_FRM0_ADDR_Y, > + stream->buffs[0]->buff_addr[RK_VIP_PLANE_Y]); > + write_vip_reg(base, VIP_FRM0_ADDR_UV, > + stream->buffs[0]->buff_addr[RK_VIP_PLANE_CBCR]); > + > + write_vip_reg(base, VIP_FRM1_ADDR_Y, > + stream->buffs[1]->buff_addr[RK_VIP_PLANE_Y]); > + write_vip_reg(base, VIP_FRM1_ADDR_UV, > + stream->buffs[1]->buff_addr[RK_VIP_PLANE_CBCR]); > + > + return 0; > +} > + > +static void rk_vip_assign_new_buffer_pingpong(struct rk_vip_stream *stream) > +{ > + struct rk_vip_scratch_buffer *scratch_buf = &stream->scratch_buf; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct rk_vip_buffer *buffer = NULL; > + void __iomem *base = vip_dev->base_addr; > + u32 frm_addr_y, frm_addr_uv; > + > + /* Set up an empty buffer for the next frame */ Add an ending dot. > + spin_lock(&stream->vbq_lock); > + > + buffer = rk_vip_get_buffer(stream); > + > + stream->buffs[stream->frame_phase] = buffer; > + > + spin_unlock(&stream->vbq_lock); > + > + frm_addr_y = stream->frame_phase ? VIP_FRM1_ADDR_Y : VIP_FRM0_ADDR_Y; > + frm_addr_uv = stream->frame_phase ? VIP_FRM1_ADDR_UV : VIP_FRM0_ADDR_UV; > + > + if (buffer) { > + write_vip_reg(base, frm_addr_y, > + buffer->buff_addr[RK_VIP_PLANE_Y]); > + write_vip_reg(base, frm_addr_uv, > + buffer->buff_addr[RK_VIP_PLANE_CBCR]); > + } else { > + write_vip_reg(base, frm_addr_y, scratch_buf->dma_addr); > + write_vip_reg(base, frm_addr_uv, scratch_buf->dma_addr); > + } In principle you should not need a scratch buffer for this. The general theory of operation is that you only switch the active buffer when you have a new one available. If userspace is too slow to requeue buffers, you just keep using the same one (don't return it to userspace) and overwrite it. > +static void rk_vip_stop_streaming(struct vb2_queue *queue) > +{ > + struct rk_vip_stream *stream = queue->drv_priv; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct rk_vip_buffer *buf; > + struct v4l2_subdev *sd; > + int ret; > + > + stream->stopping = true; > + ret = wait_event_timeout(stream->wq_stopped, > + stream->state != RK_VIP_STATE_STREAMING, > + msecs_to_jiffies(1000)); > + if (!ret) { > + rk_vip_stream_stop(stream); > + stream->stopping = false; > + } > + pm_runtime_put(vip_dev->dev); > + > + /* stop the sub device*/ First letter uppercase, dot at the end, space before the asterisk. > + sd = vip_dev->sensor.sd; > + v4l2_subdev_call(sd, video, s_stream, 0); In principle you should first stop the remove subdev and then stop this interface, not the other way round. > + > + /* release buffers */ Same cosmetics. > + if (stream->buffs[0]) { > + list_add_tail(&stream->buffs[0]->queue, &stream->buf_head); > + stream->buffs[0] = NULL; > + } > + > + if (stream->buffs[1]) { > + list_add_tail(&stream->buffs[1]->queue, &stream->buf_head); > + stream->buffs[1] = NULL; > + } It's a bit weird to re-add these buffers to the intermediate queue just to remove them after that. You could have a simple helper to call vb2_buffer_done from a struct rk_vip_buffer argument and then pass stream->buffs[0] and stream->buffs[1] to it... > + while (!list_empty(&stream->buf_head)) { > + buf = rk_vip_get_buffer(stream); > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); ... as well as passing it each remaining buf here. > + } > + > + rk_vip_destroy_scratch_buf(stream); > +} > + > +static u32 rk_vip_determine_input_mode(struct rk_vip_device *vip_dev) This is only used once so it should be inline in the calling function. > +{ > + v4l2_std_id std = vip_dev->sensor.std; > + > + return (std == V4L2_STD_NTSC) ? > + VIP_FORMAT_INPUT_MODE_NTSC : > + VIP_FORMAT_INPUT_MODE_PAL; > +} > + > +static inline u32 rk_vip_scl_ctl(struct rk_vip_stream *stream) Same comment here. > +{ > + u32 fmt_type = stream->vip_fmt_in->fmt_type; > + > + return (fmt_type == VIP_FMT_TYPE_YUV) ? > + VIP_SCL_CTRL_ENABLE_YUV_16BIT_BYPASS : > + VIP_SCL_CTRL_ENABLE_RAW_16BIT_BYPASS; > +} > + > +static int rk_vip_stream_start(struct rk_vip_stream *stream) > +{ > + u32 val, mbus_flags, href_pol, vsync_pol, > + xfer_mode = 0, yc_swap = 0, skip_top = 0; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct rk_vip_sensor_info *sensor_info; > + void __iomem *base = vip_dev->base_addr; > + int ret; > + u32 input_mode; > + > + sensor_info = &vip_dev->sensor; > + stream->frame_idx = 0; > + input_mode = rk_vip_determine_input_mode(vip_dev); > + > + mbus_flags = sensor_info->mbus.bus.parallel.flags; > + href_pol = (mbus_flags & V4L2_MBUS_HSYNC_ACTIVE_HIGH) ? > + 0 : VIP_FORMAT_HSY_LOW_ACTIVE; > + vsync_pol = (mbus_flags & V4L2_MBUS_VSYNC_ACTIVE_HIGH) ? > + VIP_FORMAT_VSY_HIGH_ACTIVE : 0; > + > + val = vsync_pol | href_pol | input_mode | stream->vip_fmt_out->fmt_val | > + stream->vip_fmt_in->dvp_fmt_val | xfer_mode | yc_swap; > + write_vip_reg(base, VIP_FOR, val); > + > + val = stream->pixm.width; > + if (stream->vip_fmt_in->fmt_type == VIP_FMT_TYPE_RAW) > + val = stream->pixm.width * 2; > + > + write_vip_reg(base, VIP_VIR_LINE_WIDTH, val); > + write_vip_reg(base, VIP_SET_SIZE, > + stream->pixm.width | (stream->pixm.height << 16)); > + > + v4l2_subdev_call(sensor_info->sd, sensor, g_skip_top_lines, &skip_top); Check return code and act accordingly for error cases, also handle -EOPNOTSUPP gracefully (fallback to skip_top = 0). > + > + write_vip_reg(base, VIP_CROP, skip_top << VIP_CROP_Y_SHIFT); > + write_vip_reg(base, VIP_FRAME_STATUS, VIP_FRAME_STAT_CLS); > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_CLS); > + write_vip_reg(base, VIP_SCL_CTRL, rk_vip_scl_ctl(stream)); > + > + ret = rk_vip_init_buffers(stream); > + if (ret) > + return ret; > + > + write_vip_reg(base, VIP_INTEN, VIP_INTEN_FRAME_END_EN | > + VIP_INTEN_LINE_ERR_EN | > + VIP_INTEN_PST_INF_FRAME_END_EN); > + > + write_vip_reg(base, VIP_CTRL, VIP_CTRL_AXI_BURST_16 | > + VIP_CTRL_MODE_PINGPONG | > + VIP_CTRL_ENABLE_CAPTURE); > + > + stream->state = RK_VIP_STATE_STREAMING; This variable is not needed in general, we have vb2_is_streaming to do the job as well as vb2_is_busy if you need to check if some buffers are already allocated (meaning we have already settled on a specific config). > + > + return 0; > +} > + > +static int rk_vip_start_streaming(struct vb2_queue *queue, unsigned int count) > +{ > + struct rk_vip_stream *stream = queue->drv_priv; > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_device *v4l2_dev = &vip_dev->v4l2_dev; > + struct v4l2_subdev *sd; > + int ret; > + > + if (stream->state != RK_VIP_STATE_READY) { Ditto. > + ret = -EBUSY; > + v4l2_err(v4l2_dev, "Stream in busy state\n"); > + goto destroy_buf; > + } > + > + if (!vip_dev->sensor.sd) { > + ret = -EINVAL; Probably -ENODEV instead. > + v4l2_err(v4l2_dev, "No sensor subdev detected\n"); > + goto destroy_buf; > + } > + > + ret = rk_vip_create_scratch_buf(stream); > + if (ret < 0) { > + v4l2_err(v4l2_dev, "Failed to create scratch_buf, %d\n", ret); > + goto destroy_buf; > + } > + > + ret = pm_runtime_get_sync(vip_dev->dev); > + if (ret < 0) { > + v4l2_err(v4l2_dev, "Failed to get runtime pm, %d\n", ret); > + goto destroy_scratch_buf; > + } > + > + /* start sub-devices */ > + sd = vip_dev->sensor.sd; > + stream->vip_fmt_in = get_input_fmt(vip_dev->sensor.sd); > + > + ret = v4l2_subdev_call(sd, video, s_stream, 1); > + if (ret < 0) > + goto runtime_put; > + > + ret = rk_vip_stream_start(stream); > + if (ret < 0) > + goto stop_stream; Normally you would do it the other way round: start the receiving interface first and then start the sensor. Otherwise you cannot exclude that the first frame will be sent while the camera interface is not yet ready. It will at best be lost and might confuse the receiver hardware. > + > + return 0; > + > +stop_stream: > + rk_vip_stream_stop(stream); > +runtime_put: > + pm_runtime_put(vip_dev->dev); > +destroy_scratch_buf: > + rk_vip_destroy_scratch_buf(stream); > +destroy_buf: > + while (!list_empty(&stream->buf_head)) { > + struct rk_vip_buffer *buf; > + > + buf = rk_vip_get_buffer(stream); > + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); > + } > + > + return ret; > +} > > +static int rk_vip_init_vb2_queue(struct vb2_queue *q, > + struct rk_vip_stream *stream) > +{ > + q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; You report mplane support but only support non-mplane pixel formats and assume that only one vb2 plane is used. You can either add support for mplane formats too or report non-mplane formats but please keep both consistent. > + q->io_modes = VB2_MMAP | VB2_DMABUF; > + q->drv_priv = stream; > + q->ops = &rk_vip_vb2_ops; > + q->mem_ops = &vb2_dma_contig_memops; > + q->buf_struct_size = sizeof(struct rk_vip_buffer); > + q->min_buffers_needed = VIP_REQ_BUFS_MIN; > + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; > + q->lock = &stream->vlock; > + q->dev = stream->vipdev->dev; > + > + return vb2_queue_init(q); > +} > + > +static void rk_vip_update_pixm(struct rk_vip_stream *stream, > + struct vip_output_fmt *fmt, > + struct v4l2_pix_format_mplane *pixm) > +{ > + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; > + struct v4l2_subdev_format sd_fmt; > + struct v4l2_rect input_rect; > + u32 width, height; > + > + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; > + sd_fmt.pad = 0; > + v4l2_subdev_call(sensor_info->sd, pad, get_fmt, NULL, &sd_fmt); > + > + input_rect.width = VIP_MAX_WIDTH; > + input_rect.height = VIP_MAX_HEIGHT; > + width = clamp_t(u32, sd_fmt.format.width, > + VIP_MIN_WIDTH, input_rect.width); > + height = clamp_t(u32, sd_fmt.format.height, > + VIP_MIN_HEIGHT, input_rect.height); I don't get the point of declaring this input_rect variable. Just use the defines directly. > + > + pixm->width = width; > + pixm->height = height; > + pixm->field = sd_fmt.format.field; > + pixm->colorspace = sd_fmt.format.colorspace; > + pixm->ycbcr_enc = sd_fmt.format.ycbcr_enc; > + pixm->quantization = sd_fmt.format.quantization; > + pixm->xfer_func = sd_fmt.format.xfer_func; > + > + v4l2_fill_pixfmt_mp(pixm, fmt->fourcc, pixm->width, pixm->height); > +} > + > +static int rk_vip_set_fmt(struct rk_vip_stream *stream, > + struct v4l2_pix_format_mplane *pixm) > +{ > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_subdev_format sd_fmt; > + struct vip_output_fmt *fmt; > + int ret; > + > + if (stream->state == RK_VIP_STATE_STREAMING) > + return -EBUSY; Same comment about using the existing helpers. > + > + fmt = find_output_fmt(stream, pixm->pixelformat); > + if (!fmt) > + fmt = &out_fmts[0]; > + > + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; > + sd_fmt.pad = 0; > + sd_fmt.format.width = pixm->width; > + sd_fmt.format.height = pixm->height; Newline here. > + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, set_fmt, NULL, &sd_fmt); And newline here. > + rk_vip_update_pixm(stream, fmt, pixm); > + stream->pixm = *pixm; > + stream->vip_fmt_out = fmt; > + > + return ret; > +} > > +static int rk_vip_try_fmt_vid_cap_mplane(struct file *file, void *fh, > + struct v4l2_format *f) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct vip_output_fmt *fmt; > + > + fmt = find_output_fmt(stream, f->fmt.pix_mp.pixelformat); > + if (!fmt) > + fmt = &out_fmts[0]; > + > + rk_vip_update_pixm(stream, fmt, &f->fmt.pix_mp); I don't think this will work since the function will use the subdev's currently active format while try_fmt should be a way to check that the provided format can work. So you probably need to call teh subdev's try_fmt here. > + > + return 0; > +} > + > +static int rk_vip_s_std(struct file *file, void *fh, v4l2_std_id norm) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_sensor_info *sensor_info = &stream->vipdev->sensor; > + int ret; > + > + if (norm == sensor_info->std) > + return 0; > + > + if (stream->state == RK_VIP_STATE_STREAMING) Same comment about using the existing helper. Also you should refuse this in case buffers are allocated, not just when streaming. > + return -EBUSY; > + > + ret = v4l2_subdev_call(sensor_info->sd, video, s_std, norm); > + if (ret) > + return ret; > + > + sensor_info->std = norm; > + > + /* S_STD will update the format since that depends on the standard */ Dot at the end. > + rk_vip_update_pixm(stream, stream->vip_fmt_out, &stream->pixm); > + > + return 0; > +} > + > +static int rk_vip_s_fmt_vid_cap_mplane(struct file *file, > + void *priv, struct v4l2_format *f) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + int ret; > + > + if (stream->state == RK_VIP_STATE_STREAMING) > + return -EBUSY; This is not enough: you must forbid changes if buffers were already allocated (typically checked with vb2_is_busy). > + > + ret = rk_vip_set_fmt(stream, &f->fmt.pix_mp); > + > + return ret; > +} > + > +static int rk_vip_enum_framesizes(struct file *file, void *fh, > + struct v4l2_frmsizeenum *fsize) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_subdev_frame_size_enum fse = { > + .index = fsize->index, > + .which = V4L2_SUBDEV_FORMAT_ACTIVE, > + }; > + struct vip_output_fmt *fmt; > + int ret; > + > + if (!vip_dev->sensor.sd) > + return -EINVAL; Probably more -ENODEV. > + > + fmt = find_output_fmt(stream, fsize->pixel_format); > + if (!fmt) > + return -EINVAL; > + > + fse.code = fmt->mbus; > + > + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, enum_frame_size, > + NULL, &fse); > + if (ret) > + return ret; > + > + fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE; > + fsize->discrete.width = fse.max_width; > + fsize->discrete.height = fse.max_height; > + > + return 0; > +} > + > +static int rk_vip_enum_frameintervals(struct file *file, void *fh, > + struct v4l2_frmivalenum *fival) > +{ > + struct rk_vip_stream *stream = video_drvdata(file); > + struct rk_vip_device *vip_dev = stream->vipdev; > + struct v4l2_subdev_frame_interval_enum fie = { > + .index = fival->index, > + .width = fival->width, > + .height = fival->height, > + .which = V4L2_SUBDEV_FORMAT_ACTIVE, > + }; > + struct vip_output_fmt *fmt; > + int ret; > + > + if (!vip_dev->sensor.sd) > + return -EINVAL; Probably more -ENODEV. > + > + fmt = find_output_fmt(stream, fival->pixel_format); > + if (!fmt) > + return -EINVAL; > + > + fie.code = fmt->mbus; > + > + ret = v4l2_subdev_call(vip_dev->sensor.sd, pad, enum_frame_interval, > + NULL, &fie); > + if (ret) > + return ret; > + > + fival->type = V4L2_FRMSIZE_TYPE_DISCRETE; > + fival->discrete = fie.interval; > + > + return 0; > +} > + > +int rk_vip_register_stream_vdev(struct rk_vip_device *vip_dev) > +{ > + struct rk_vip_stream *stream = &vip_dev->stream; > + struct v4l2_device *v4l2_dev = &vip_dev->v4l2_dev; > + struct video_device *vdev = &stream->vdev; > + int ret; > + > + strscpy(vdev->name, VIP_VIDEODEVICE_NAME, sizeof(vdev->name)); > + mutex_init(&stream->vlock); > + > + vdev->ioctl_ops = &rk_vip_v4l2_ioctl_ops; > + vdev->release = video_device_release_empty; > + vdev->fops = &rk_vip_fops; > + vdev->minor = -1; > + vdev->v4l2_dev = v4l2_dev; > + vdev->lock = &stream->vlock; > + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE | Same comment as before about mplane support. > + V4L2_CAP_STREAMING; > + vdev->tvnorms = V4L2_STD_NTSC | V4L2_STD_PAL; > + video_set_drvdata(vdev, stream); > + vdev->vfl_dir = VFL_DIR_RX; > + stream->pad.flags = MEDIA_PAD_FL_SINK; > + > + rk_vip_init_vb2_queue(&stream->buf_queue, stream); > + > + vdev->queue = &stream->buf_queue; > + strscpy(vdev->name, KBUILD_MODNAME, sizeof(vdev->name)); > + > + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); > + if (ret < 0) { > + v4l2_err(v4l2_dev, > + "video_register_device failed with error %d\n", ret); > + return ret; > + } > + > + ret = media_entity_pads_init(&vdev->entity, 1, &stream->pad); > + if (ret < 0) > + goto unreg; Please intialize the pads before registering the video device. > + > + return 0; > +unreg: > + video_unregister_device(vdev); > + return ret; > +} > + > +irqreturn_t rk_vip_irq_pingpong(int irq, void *ctx) > +{ > + struct device *dev = ctx; > + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); > + struct rk_vip_stream *stream = &vip_dev->stream; > + void __iomem *base = vip_dev->base_addr; > + unsigned int intstat; > + > + u32 lastline, lastpix, ctl, vip_frmst; > + > + intstat = read_vip_reg(base, VIP_INTSTAT); > + vip_frmst = read_vip_reg(base, VIP_FRAME_STATUS); > + lastline = VIP_FETCH_Y_LAST_LINE(read_vip_reg(base, VIP_LAST_LINE)); > + lastpix = VIP_FETCH_Y_LAST_LINE(read_vip_reg(base, VIP_LAST_PIX)); > + ctl = read_vip_reg(base, VIP_CTRL); > + > + /* There are two irqs enabled: Start the comment on the next line. > + * - PST_INF_FRAME_END: vip FIFO is ready, > + * this is prior to FRAME_END > + * - FRAME_END: vip has saved frame to memory, > + * a frame ready > + */ > + > + if ((intstat & VIP_INTSTAT_PST_INF_FRAME_END)) { No need for (( ... )). > + write_vip_reg(base, VIP_INTSTAT, > + VIP_INTSTAT_PST_INF_FRAME_END_CLR); Newline here. > + if (stream->stopping) > + /* To stop VIP ASAP, before FRAME_END irq */ Dot at the end. > + write_vip_reg(base, VIP_CTRL, > + ctl & (~VIP_CTRL_ENABLE_CAPTURE)); > + } > + > + if ((intstat & VIP_INTSTAT_PRE_INF_FRAME_END)) Ditto. > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_PRE_INF_FRAME_END); > + > + if (intstat & (VIP_INTSTAT_LINE_ERR | VIP_INTSTAT_PIX_ERR)) { > + v4l2_err(&vip_dev->v4l2_dev, "LINE_ERR OR PIX_ERR"); Please make this a more articulate sentence. > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_LINE_ERR | > + VIP_INTSTAT_PIX_ERR); > + rk_vip_reset_stream(vip_dev); > + } > + > + if ((intstat & VIP_INTSTAT_FRAME_END)) { Dtto. > + struct vb2_v4l2_buffer *vb_done = NULL; > + > + write_vip_reg(base, VIP_INTSTAT, VIP_INTSTAT_FRAME_END_CLR | > + VIP_INTSTAT_LINE_END_CLR); > + > + if (stream->stopping) { > + rk_vip_stream_stop(stream); > + stream->stopping = false; > + wake_up(&stream->wq_stopped); > + return IRQ_HANDLED; > + } > + > + if (lastline != stream->pixm.height) { > + v4l2_err(&vip_dev->v4l2_dev, > + "Bad frame, irq:0x%x frmst:0x%x size:%dx%d\n", > + intstat, vip_frmst, lastpix, lastline); Note that you can use %#x instead of 0x%x. > + > + rk_vip_reset_stream(vip_dev); > + } > + > + if (vip_frmst & VIP_INTSTAT_F0_READY) > + stream->frame_phase = 0; > + else if (vip_frmst & VIP_INTSTAT_F1_READY) > + stream->frame_phase = 1; Would be good to double-check that "ready" really means ready for the CPU side to write to this register, not "ready" as in "hardware has just finished writing new data to this buffer". Please add a comment to clarify how it should be understood. > + else > + return IRQ_HANDLED; > + > + if (stream->buffs[stream->frame_phase]) > + vb_done = &stream->buffs[stream->frame_phase]->vb; > + > + rk_vip_assign_new_buffer_pingpong(stream); > + > + if (vb_done) > + rk_vip_vb_done(stream, vb_done); > + > + stream->frame_idx++; > + } > + > + return IRQ_HANDLED; > +} > diff --git a/drivers/media/platform/rockchip/vip/dev.c b/drivers/media/platform/rockchip/vip/dev.c > new file mode 100644 > index 000000000000..c3931e5accbd > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/dev.c > @@ -0,0 +1,346 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Rockchip VIP Camera Interface Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2020 Maxime Chevallier <maxime.chevallier@bootlin.com> > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#include <linux/clk.h> > +#include <linux/delay.h> > +#include <linux/interrupt.h> > +#include <linux/module.h> > +#include <linux/of.h> > +#include <linux/of_graph.h> > +#include <linux/of_platform.h> > +#include <linux/of_reserved_mem.h> > +#include <linux/reset.h> > +#include <linux/pm_runtime.h> > +#include <linux/pinctrl/consumer.h> > +#include <media/v4l2-fwnode.h> > + > +#include "dev.h" > +#include "regs.h" > + > +#define RK_VIP_VERNO_LEN 10 > + > +struct vip_match_data { > + int chip_id; > + const char * const *clks; > + const char * const *rsts; > + int clks_num; > + int rsts_num; > +}; > + > +static int rk_vip_create_links(struct rk_vip_device *vip_dev) You can fold this function into the calling one since it's only used once. > +{ > + struct v4l2_subdev *sd = vip_dev->sensor.sd; > + int ret; > + > + ret = media_create_pad_link(&sd->entity, 0, > + &vip_dev->stream.vdev.entity, 0, > + MEDIA_LNK_FL_ENABLED); > + if (ret) { > + dev_err(vip_dev->dev, "failed to create link"); > + return ret; > + } > + > + return 0; > +} > + > +static int subdev_notifier_complete(struct v4l2_async_notifier *notifier) > +{ > + struct rk_vip_device *vip_dev; > + int ret; > + > + vip_dev = container_of(notifier, struct rk_vip_device, notifier); > + > + mutex_lock(&vip_dev->media_dev.graph_mutex); > + > + ret = v4l2_device_register_subdev_nodes(&vip_dev->v4l2_dev); > + if (ret < 0) > + goto unlock; > + > + ret = rk_vip_create_links(vip_dev); > + if (ret < 0) > + goto unlock; > + > +unlock: > + mutex_unlock(&vip_dev->media_dev.graph_mutex); > + return ret; > +} > + > +static int vip_subdev_notifier(struct rk_vip_device *vip_dev) > +{ > + struct v4l2_async_notifier *ntf = &vip_dev->notifier; > + struct device *dev = vip_dev->dev; > + struct v4l2_async_connection *asd; > + struct v4l2_fwnode_endpoint vep = { > + .bus_type = V4L2_MBUS_PARALLEL, > + }; > + struct fwnode_handle *ep; > + int ret; > + > + v4l2_async_nf_init(ntf, &vip_dev->v4l2_dev); > + > + ep = fwnode_graph_get_endpoint_by_id(dev_fwnode(dev), 0, 0, > + FWNODE_GRAPH_ENDPOINT_NEXT); > + if (!ep) > + return -EINVAL; > + > + ret = v4l2_fwnode_endpoint_parse(ep, &vep); > + if (ret) > + return ret; > + > + asd = v4l2_async_nf_add_fwnode_remote(ntf, ep, > + struct v4l2_async_connection); > + if (IS_ERR(asd)) { > + ret = PTR_ERR(asd); > + return ret; > + } > + > + ntf->ops = &subdev_notifier_ops; > + > + fwnode_handle_put(ep); > + > + ret = v4l2_async_nf_register(ntf); > + return ret; > +} > + > +static int rk_vip_register_platform_subdevs(struct rk_vip_device *vip_dev) Doesn't feel necessary to have this function, just include the two calls into the calling one. > +{ > + int ret; > + > + ret = rk_vip_register_stream_vdev(vip_dev); > + if (ret < 0) > + return ret; > + > + ret = vip_subdev_notifier(vip_dev); > + if (ret < 0) { > + v4l2_err(&vip_dev->v4l2_dev, > + "Failed to register subdev notifier(%d)\n", ret); > + rk_vip_unregister_stream_vdev(vip_dev); > + } > + > + return 0; > +} > + > +static const char * const px30_vip_clks[] = { Not sure it's really useful to have const specified twice here. > + "aclk", > + "hclk", > + "pclk", > +}; > + > +static const struct vip_match_data px30_vip_match_data = { > + .chip_id = CHIP_PX30_VIP, This is not used anywhere, I think you can just get rid of it. Feels like something remaining from the BSP code. > + .clks = px30_vip_clks, > + .clks_num = ARRAY_SIZE(px30_vip_clks), > +}; > + > +static const struct of_device_id rk_vip_plat_of_match[] = { > + { > + .compatible = "rockchip,px30-vip", > + .data = &px30_vip_match_data, > + }, > + {}, > +}; > + > +void rk_vip_soft_reset(struct rk_vip_device *vip_dev) > +{ > + reset_control_assert(vip_dev->vip_rst); > + > + udelay(5); > + > + reset_control_deassert(vip_dev->vip_rst); There is a reset toggle function in the reset API. Better use it unless you absolutely need a specific delay that's not handled by the reset controller (in which case there should be a comment to explain it). > +} > + > +static int rk_vip_get_resource(struct platform_device *pdev, > + struct rk_vip_device *vip_dev) > +{ > + struct resource *res; > + > + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); > + if (!res) { > + dev_err(&pdev->dev, > + "Unable to allocate resources for device\n"); > + return -ENODEV; > + } > + > + vip_dev->base_addr = devm_ioremap_resource(&pdev->dev, res); > + if (IS_ERR(vip_dev->base_addr)) > + return PTR_ERR(vip_dev->base_addr); > + > + return 0; > +} > + > +static int rk_vip_plat_probe(struct platform_device *pdev) > +{ > + const struct of_device_id *match; > + struct device_node *node = pdev->dev.of_node; > + struct device *dev = &pdev->dev; > + struct v4l2_device *v4l2_dev; > + struct rk_vip_device *vip_dev; > + const struct vip_match_data *data; > + int i, ret, irq; > + > + match = of_match_node(rk_vip_plat_of_match, node); > + if (!match) > + return -ENODEV; Just use of_device_get_match_data to get match->data directly. > + > + vip_dev = devm_kzalloc(dev, sizeof(*vip_dev), GFP_KERNEL); > + if (!vip_dev) > + return -ENOMEM; > + > + dev_set_drvdata(dev, vip_dev); Use platform_set_drvdata instead for good measure. > + vip_dev->dev = dev; > + > + irq = platform_get_irq(pdev, 0); > + if (irq < 0) > + return irq; > + > + ret = devm_request_irq(dev, irq, rk_vip_irq_pingpong, 0, > + dev_driver_string(dev), dev); > + if (ret < 0) Any ret != 0 is invalid, not just negative ones. > + return dev_err_probe(dev, ret, "request irq failed\n"); > + > + vip_dev->irq = irq; > + data = match->data; > + vip_dev->chip_id = data->chip_id; > + > + ret = rk_vip_get_resource(pdev, vip_dev); > + if (ret) > + return ret; > + > + for (i = 0; i < data->clks_num; i++) > + vip_dev->clks[i].id = data->clks[i]; > + > + vip_dev->num_clk = data->clks_num; Maybe you could directly define this in the match data (and keeping it around) instead of making a copy. The match data is not going to change or disappear. > + > + ret = devm_clk_bulk_get(dev, vip_dev->num_clk, vip_dev->clks); > + if (ret) > + return ret; > + > + vip_dev->vip_rst = devm_reset_control_array_get(dev, false, false); > + if (IS_ERR(vip_dev->vip_rst)) > + return PTR_ERR(vip_dev->vip_rst); > + > + /* Initialize the stream */ Ending dot here. > + rk_vip_stream_init(vip_dev); > + strscpy(vip_dev->media_dev.model, "rk_vip", > + sizeof(vip_dev->media_dev.model)); > + vip_dev->media_dev.dev = &pdev->dev; > + v4l2_dev = &vip_dev->v4l2_dev; > + v4l2_dev->mdev = &vip_dev->media_dev; > + strscpy(v4l2_dev->name, "rk_vip", sizeof(v4l2_dev->name)); Use the same thing that you're taking for the video device. > + > + ret = v4l2_device_register(vip_dev->dev, &vip_dev->v4l2_dev); > + if (ret < 0) > + return ret; > + > + media_device_init(&vip_dev->media_dev); > + > + ret = media_device_register(&vip_dev->media_dev); > + if (ret < 0) { > + v4l2_err(v4l2_dev, "Failed to register media device: %d\n", > + ret); You didn't print messages for other errors before here so either do it always or never, but this one call has nothing special compared to others. > + goto err_unreg_v4l2_dev; > + } > + > + /* create & register platform subdev (from of_node) */ Capital letter and ending dot here. The of_node part is not relevant. > + ret = rk_vip_register_platform_subdevs(vip_dev); > + if (ret < 0) > + goto err_unreg_media_dev; > + > + vip_dev->sensor.std = V4L2_STD_NTSC; You can have this inside the next call. > + rk_vip_set_default_format(vip_dev); > + pm_runtime_enable(&pdev->dev); > + > + return 0; > + > +err_unreg_media_dev: > + media_device_unregister(&vip_dev->media_dev); > +err_unreg_v4l2_dev: > + v4l2_device_unregister(&vip_dev->v4l2_dev); > + return ret; > +} > + > +static int rk_vip_plat_remove(struct platform_device *pdev) > +{ > + struct rk_vip_device *vip_dev = platform_get_drvdata(pdev); > + > + pm_runtime_disable(&pdev->dev); > + > + media_device_unregister(&vip_dev->media_dev); > + v4l2_device_unregister(&vip_dev->v4l2_dev); > + rk_vip_unregister_stream_vdev(vip_dev); > + > + return 0; > +} > + > +static int __maybe_unused rk_vip_runtime_suspend(struct device *dev) > +{ > + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); > + > + clk_bulk_disable_unprepare(vip_dev->num_clk, vip_dev->clks); > + > + return pinctrl_pm_select_sleep_state(dev); > +} > + > +static int __maybe_unused rk_vip_runtime_resume(struct device *dev) > +{ > + struct rk_vip_device *vip_dev = dev_get_drvdata(dev); > + int ret; > + > + ret = pinctrl_pm_select_default_state(dev); > + if (ret < 0) > + return ret; > + > + return clk_bulk_prepare_enable(vip_dev->num_clk, vip_dev->clks); > +} > + > +static const struct dev_pm_ops rk_vip_plat_pm_ops = { > + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, > + pm_runtime_force_resume) > + SET_RUNTIME_PM_OPS(rk_vip_runtime_suspend, rk_vip_runtime_resume, NULL) You can just depend on PM and avoid these conditional macros. > +}; > + > +static struct platform_driver rk_vip_plat_drv = { > + .driver = { > + .name = VIP_DRIVER_NAME, > + .of_match_table = rk_vip_plat_of_match, > + .pm = &rk_vip_plat_pm_ops, > + }, > + .probe = rk_vip_plat_probe, > + .remove = rk_vip_plat_remove, > +}; > +module_platform_driver(rk_vip_plat_drv); > + > +MODULE_AUTHOR("Rockchip Camera/ISP team"); > +MODULE_DESCRIPTION("Rockchip VIP platform driver"); > +MODULE_LICENSE("GPL"); > diff --git a/drivers/media/platform/rockchip/vip/dev.h b/drivers/media/platform/rockchip/vip/dev.h > new file mode 100644 > index 000000000000..eb9cd8f20ffc > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/dev.h > @@ -0,0 +1,163 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * Rockchip VIP Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#ifndef _RK_VIP_DEV_H > +#define _RK_VIP_DEV_H > + > +#include <linux/clk.h> > +#include <linux/mutex.h> > +#include <media/media-device.h> > +#include <media/media-entity.h> > +#include <media/v4l2-ctrls.h> > +#include <media/v4l2-device.h> > +#include <media/videobuf2-v4l2.h> > + > +#define VIP_DRIVER_NAME "rk_vip" > +#define VIP_VIDEODEVICE_NAME "stream_vip" Use "rockchip-cif" for both. > + > +#define RK_VIP_MAX_BUS_CLK 8 > +#define RK_VIP_MAX_SENSOR 2 Looks like only one is supported though. > +#define RK_VIP_MAX_RESET 5 > +#define RK_VIP_MAX_CSI_CHANNEL 4 > + > +#define RK_VIP_DEFAULT_WIDTH 640 > +#define RK_VIP_DEFAULT_HEIGHT 480 Should be next to the other size-related definitions. > + > +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) > +#define read_vip_reg(base, addr) readl((addr) + (base)) See comment about naming consistency in general. > +enum rk_vip_state { > + RK_VIP_STATE_DISABLED, > + RK_VIP_STATE_READY, > + RK_VIP_STATE_STREAMING > +}; I think you can get rid of this... > +enum rk_vip_chip_id { > + CHIP_PX30_VIP, > + CHIP_RK1808_VIP, > + CHIP_RK3128_VIP, > + CHIP_RK3288_VIP > +}; ... and this. > + > +enum host_type_t { > + RK_CSI_RXHOST, > + RK_DSI_RXHOST > +}; This is not used anywhere, please get rid of it. > + > +struct rk_vip_buffer { > + struct vb2_v4l2_buffer vb; > + struct list_head queue; > + union { > + u32 buff_addr[VIDEO_MAX_PLANES]; > + void *vaddr[VIDEO_MAX_PLANES]; > + }; Why the union? I think you can get rid of vaddr. > +}; > + > +struct rk_vip_scratch_buffer { > + void *vaddr; > + dma_addr_t dma_addr; > + u32 size; > +}; Scratch buffer should not be needed. > +struct rk_vip_stream { > + struct rk_vip_device *vipdev; Here you are tab-aligning the name of members but not in other places. Please make this consistent (here and everywhere else in this file). > + enum rk_vip_state state; > + bool stopping; > + wait_queue_head_t wq_stopped; > + int frame_idx; > + int frame_phase; > + > + /* lock between irq and buf_queue */ Capital letter and ending dot. > + spinlock_t vbq_lock; > + struct vb2_queue buf_queue; > + struct list_head buf_head; > + struct rk_vip_scratch_buffer scratch_buf; > + struct rk_vip_buffer *buffs[2]; > + > + /* vfd lock */ Ditto. > + struct mutex vlock; > + struct video_device vdev; > + struct media_pad pad; > + > + struct vip_output_fmt *vip_fmt_out; > + const struct vip_input_fmt *vip_fmt_in; > + struct v4l2_pix_format_mplane pixm; > +}; > + > +static inline struct rk_vip_stream *to_rk_vip_stream(struct video_device *vdev) > +{ > + return container_of(vdev, struct rk_vip_stream, vdev); > +} > + > +struct rk_vip_device { > + struct list_head list; > + struct device *dev; > + int irq; > + void __iomem *base_addr; > + void __iomem *csi_base; > + struct clk_bulk_data clks[RK_VIP_MAX_BUS_CLK]; > + int num_clk; I think we can have those in the match data directly and grab a pointer to it in this structure. > + struct vb2_alloc_ctx *alloc_ctx; > + bool iommu_en; > + struct iommu_domain *domain; All three above are not used. > + struct reset_control *vip_rst; > + > + struct v4l2_device v4l2_dev; > + struct media_device media_dev; > + struct v4l2_ctrl_handler ctrl_handler; This is not used, you can get rid of it. I think we no longer expect the video devices to provide controls when subdev nodes are available (but maybe this is still the case for video node-centric devices, I'm not sure). Maybe someone with more precise knowledge can answer here? > + struct v4l2_async_notifier notifier; > + struct v4l2_async_connection asd; > + struct rk_vip_sensor_info sensor; > + > + struct rk_vip_stream stream; > + > + int chip_id; Not used/needed. > +}; > + > +void rk_vip_unregister_stream_vdev(struct rk_vip_device *dev); > +int rk_vip_register_stream_vdev(struct rk_vip_device *dev); > +void rk_vip_stream_init(struct rk_vip_device *dev); > +void rk_vip_set_default_format(struct rk_vip_device *dev); > + > +irqreturn_t rk_vip_irq_pingpong(int irq, void *ctx); > +void rk_vip_soft_reset(struct rk_vip_device *vip_dev); > + > +#endif > diff --git a/drivers/media/platform/rockchip/vip/regs.h b/drivers/media/platform/rockchip/vip/regs.h > new file mode 100644 > index 000000000000..ccf10ffbbff8 > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/regs.h > @@ -0,0 +1,260 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * Rockchip VIP Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#ifndef _RK_VIP_REGS_H > +#define _RK_VIP_REGS_H > + > +/* VIP Reg Offset */ > +#define VIP_CTRL 0x00 > +#define VIP_INTEN 0x04 > +#define VIP_INTSTAT 0x08 > +#define VIP_FOR 0x0c > +#define VIP_LINE_NUM_ADDR 0x10 > +#define VIP_FRM0_ADDR_Y 0x14 > +#define VIP_FRM0_ADDR_UV 0x18 > +#define VIP_FRM1_ADDR_Y 0x1c > +#define VIP_FRM1_ADDR_UV 0x20 > +#define VIP_VIR_LINE_WIDTH 0x24 > +#define VIP_SET_SIZE 0x28 > +#define VIP_SCM_ADDR_Y 0x2c > +#define VIP_SCM_ADDR_U 0x30 > +#define VIP_SCM_ADDR_V 0x34 > +#define VIP_WB_UP_FILTER 0x38 > +#define VIP_WB_LOW_FILTER 0x3c > +#define VIP_WBC_CNT 0x40 > +#define VIP_CROP 0x44 > +#define VIP_SCL_CTRL 0x48 > +#define VIP_SCL_DST 0x4c > +#define VIP_SCL_FCT 0x50 > +#define VIP_SCL_VALID_NUM 0x54 > +#define VIP_LINE_LOOP_CTR 0x58 > +#define VIP_FRAME_STATUS 0x60 > +#define VIP_CUR_DST 0x64 > +#define VIP_LAST_LINE 0x68 > +#define VIP_LAST_PIX 0x6c > + > +/* RK1808 VIP CSI Registers Offset */ Get rid of RK1808, it's not supported yet and it's the weird one out there. > +#define VIP_CSI_ID0_CTRL0 0x80 > +#define VIP_CSI_ID0_CTRL1 0x84 > +#define VIP_CSI_ID1_CTRL0 0x88 > +#define VIP_CSI_ID1_CTRL1 0x8c > +#define VIP_CSI_ID2_CTRL0 0x90 > +#define VIP_CSI_ID2_CTRL1 0x94 > +#define VIP_CSI_ID3_CTRL0 0x98 > +#define VIP_CSI_ID3_CTRL1 0x9c > +#define VIP_CSI_WATER_LINE 0xa0 > +#define VIP_CSI_FRM0_ADDR_Y_ID0 0xa4 > +#define VIP_CSI_FRM1_ADDR_Y_ID0 0xa8 > +#define VIP_CSI_FRM0_ADDR_UV_ID0 0xac > +#define VIP_CSI_FRM1_ADDR_UV_ID0 0xb0 > +#define VIP_CSI_FRM0_VLW_Y_ID0 0xb4 > +#define VIP_CSI_FRM1_VLW_Y_ID0 0xb8 > +#define VIP_CSI_FRM0_VLW_UV_ID0 0xbc > +#define VIP_CSI_FRM1_VLW_UV_ID0 0xc0 > +#define VIP_CSI_FRM0_ADDR_Y_ID1 0xc4 > +#define VIP_CSI_FRM1_ADDR_Y_ID1 0xc8 > +#define VIP_CSI_FRM0_ADDR_UV_ID1 0xcc > +#define VIP_CSI_FRM1_ADDR_UV_ID1 0xd0 > +#define VIP_CSI_FRM0_VLW_Y_ID1 0xd4 > +#define VIP_CSI_FRM1_VLW_Y_ID1 0xd8 > +#define VIP_CSI_FRM0_VLW_UV_ID1 0xdc > +#define VIP_CSI_FRM1_VLW_UV_ID1 0xe0 > +#define VIP_CSI_FRM0_ADDR_Y_ID2 0xe4 > +#define VIP_CSI_FRM1_ADDR_Y_ID2 0xe8 > +#define VIP_CSI_FRM0_ADDR_UV_ID2 0xec > +#define VIP_CSI_FRM1_ADDR_UV_ID2 0xf0 > +#define VIP_CSI_FRM0_VLW_Y_ID2 0xf4 > +#define VIP_CSI_FRM1_VLW_Y_ID2 0xf8 > +#define VIP_CSI_FRM0_VLW_UV_ID2 0xfc > +#define VIP_CSI_FRM1_VLW_UV_ID2 0x100 > +#define VIP_CSI_FRM0_ADDR_Y_ID3 0x104 > +#define VIP_CSI_FRM1_ADDR_Y_ID3 0x108 > +#define VIP_CSI_FRM0_ADDR_UV_ID3 0x10c > +#define VIP_CSI_FRM1_ADDR_UV_ID3 0x110 > +#define VIP_CSI_FRM0_VLW_Y_ID3 0x114 > +#define VIP_CSI_FRM1_VLW_Y_ID3 0x118 > +#define VIP_CSI_FRM0_VLW_UV_ID3 0x11c > +#define VIP_CSI_FRM1_VLW_UV_ID3 0x120 > +#define VIP_CSI_INTEN 0x124 > +#define VIP_CSI_INTSTAT 0x128 > +#define VIP_CSI_LINE_INT_NUM_ID0_1 0x12c > +#define VIP_CSI_LINE_INT_NUM_ID2_3 0x130 > +#define VIP_CSI_LINE_CNT_ID0_1 0x134 > +#define VIP_CSI_LINE_CNT_ID2_3 0x138 > +#define VIP_CSI_ID0_CROP_START 0x13c > +#define VIP_CSI_ID1_CROP_START 0x140 > +#define VIP_CSI_ID2_CROP_START 0x144 > +#define VIP_CSI_ID3_CROP_START 0x148 > + > +/* The key register bit description */ This comment is not useful, remove it. > + > +/* VIP_CTRL Reg */ All these heading comments are not useful either. > +#define VIP_CTRL_ENABLE_CAPTURE BIT(0) > +#define VIP_CTRL_MODE_PINGPONG BIT(1) > +#define VIP_CTRL_MODE_LINELOOP BIT(2) > +#define VIP_CTRL_AXI_BURST_16 (0xF << 12) Please make sure you stay consistent with hex values: have them all lowercase and prefixed with a zero when < 0x10 (or something else you might prefer, but keep it consistent). > + > +/* VIP_INTEN */ > +#define VIP_INTEN_FRAME_END_EN BIT(0) > +#define VIP_INTEN_LINE_ERR_EN BIT(2) > +#define VIP_INTEN_BUS_ERR_EN BIT(6) > +#define VIP_INTEN_SCL_ERR_EN BIT(7) > +#define VIP_INTEN_PST_INF_FRAME_END_EN BIT(9) > + > +/* VIP INTSTAT */ > +#define VIP_INTSTAT_CLS 0x3FF > +#define VIP_INTSTAT_FRAME_END BIT(0) > +#define VIP_INTSTAT_LINE_END BIT(1) > +#define VIP_INTSTAT_LINE_ERR BIT(2) > +#define VIP_INTSTAT_PIX_ERR BIT(3) > +#define VIP_INTSTAT_DFIFO_OF BIT(5) > +#define VIP_INTSTAT_BUS_ERR BIT(6) > +#define VIP_INTSTAT_PRE_INF_FRAME_END BIT(8) > +#define VIP_INTSTAT_PST_INF_FRAME_END BIT(9) > +#define VIP_INTSTAT_FRAME_END_CLR BIT(0) > +#define VIP_INTSTAT_LINE_END_CLR BIT(1) > +#define VIP_INTSTAT_LINE_ERR_CLR BIT(2) > +#define VIP_INTSTAT_PST_INF_FRAME_END_CLR BIT(9) > +#define VIP_INTSTAT_ERR 0xFC > + > +/* VIP_FRAME STATUS */ > +#define VIP_FRAME_STAT_CLS 0x00 > +/* write 0 to clear frame 0 */ This comment is a bit cryptic. Either improve it or remove it. Cheers, Paul -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-16 9:00 ` [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface Mehdi Djait 2023-10-19 15:40 ` Paul Kocialkowski 2023-10-20 15:38 ` Paul Kocialkowski @ 2023-10-23 13:28 ` Michael Riesch 2023-10-25 8:49 ` Paul Kocialkowski 2 siblings, 1 reply; 19+ messages in thread From: Michael Riesch @ 2023-10-23 13:28 UTC (permalink / raw) To: Mehdi Djait, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel Cc: linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier, paul.kocialkowski Hi Mehdi, Typo in the subject: "Rockhip's" -> "Rockchip's" I think this typo has been in there for a while now ;-) On 10/16/23 11:00, Mehdi Djait wrote: > Introduce a driver for the camera interface on some Rockchip platforms. > > This controller supports CSI2 and BT656 interfaces, but for > now only the BT656 interface could be tested, hence it's the only one > that's supported in the first version of this driver. "CSI2" -> "MIPI CSI-2" ? "BT656" -> "BT.656" ? Also, additional interfaces are supported by some units, e.g., the RK3568 VICAP also supports BT.1120. But most likely it becomes too complex to list everything, and it would be better if you simply described the unit in the PX30. I think this would clarify the commit message a lot. > This controller can be fond on PX30, RK1808, RK3128 and RK3288, > but for now it's only been tested on PX30. > > Most of this driver was written following the BSP driver from rockchip, "rockchip" -> "Rockchip" > removing the parts that either didn't fit correctly the guidelines, or > that couldn't be tested. > > In the BSP, this driver is known as the "cif" driver, but this was > renamed to "vip" to better fit the controller denomination in the > datasheet. > > This basic version doesn't support cropping nor scaling, and is only > designed with one SDTV video decoder being attached to it a any time. > > This version uses the "pingpong" mode of the controller, which is a > double-buffering mechanism. > > Signed-off-by: Mehdi Djait <mehdi.djait@bootlin.com> Two things below: >[...] > diff --git a/drivers/media/platform/rockchip/vip/dev.h b/drivers/media/platform/rockchip/vip/dev.h > new file mode 100644 > index 000000000000..eb9cd8f20ffc > --- /dev/null > +++ b/drivers/media/platform/rockchip/vip/dev.h > @@ -0,0 +1,163 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * Rockchip VIP Driver > + * > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > + */ > + > +#ifndef _RK_VIP_DEV_H > +#define _RK_VIP_DEV_H > + > +#include <linux/clk.h> > +#include <linux/mutex.h> > +#include <media/media-device.h> > +#include <media/media-entity.h> > +#include <media/v4l2-ctrls.h> > +#include <media/v4l2-device.h> > +#include <media/videobuf2-v4l2.h> > + > +#define VIP_DRIVER_NAME "rk_vip" > +#define VIP_VIDEODEVICE_NAME "stream_vip" > + > +#define RK_VIP_MAX_BUS_CLK 8 > +#define RK_VIP_MAX_SENSOR 2 > +#define RK_VIP_MAX_RESET 5 > +#define RK_VIP_MAX_CSI_CHANNEL 4 > + > +#define RK_VIP_DEFAULT_WIDTH 640 > +#define RK_VIP_DEFAULT_HEIGHT 480 > + > +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) > +#define read_vip_reg(base, addr) readl((addr) + (base)) Please provide those helpers as proper inline functions. As to the naming, the "_reg" suffix seems unnecessary. Alternatively, you could consider converting the driver to use regmap. > + > +enum rk_vip_state { > + RK_VIP_STATE_DISABLED, > + RK_VIP_STATE_READY, > + RK_VIP_STATE_STREAMING > +}; > + > +enum rk_vip_chip_id { > + CHIP_PX30_VIP, > + CHIP_RK1808_VIP, > + CHIP_RK3128_VIP, > + CHIP_RK3288_VIP > +}; > + > +enum host_type_t { > + RK_CSI_RXHOST, > + RK_DSI_RXHOST > +}; > + > +struct rk_vip_buffer { > + struct vb2_v4l2_buffer vb; > + struct list_head queue; > + union { > + u32 buff_addr[VIDEO_MAX_PLANES]; > + void *vaddr[VIDEO_MAX_PLANES]; > + }; > +}; > + > +struct rk_vip_scratch_buffer { > + void *vaddr; > + dma_addr_t dma_addr; > + u32 size; > +}; > + > +static inline struct rk_vip_buffer *to_rk_vip_buffer(struct vb2_v4l2_buffer *vb) > +{ > + return container_of(vb, struct rk_vip_buffer, vb); > +} > + > +struct rk_vip_sensor_info { > + struct v4l2_subdev *sd; > + int pad; > + struct v4l2_mbus_config mbus; > + int lanes; > + v4l2_std_id std; > +}; > + > +struct vip_output_fmt { > + u32 fourcc; > + u32 mbus; > + u32 fmt_val; > + u8 cplanes; > +}; > + > +enum vip_fmt_type { > + VIP_FMT_TYPE_YUV = 0, > + VIP_FMT_TYPE_RAW, > +}; > + > +struct vip_input_fmt { > + u32 mbus_code; > + u32 dvp_fmt_val; > + u32 csi_fmt_val; > + enum vip_fmt_type fmt_type; > + enum v4l2_field field; > +}; > + > +struct rk_vip_stream { > + struct rk_vip_device *vipdev; > + enum rk_vip_state state; > + bool stopping; > + wait_queue_head_t wq_stopped; > + int frame_idx; > + int frame_phase; > + > + /* lock between irq and buf_queue */ > + spinlock_t vbq_lock; > + struct vb2_queue buf_queue; > + struct list_head buf_head; > + struct rk_vip_scratch_buffer scratch_buf; > + struct rk_vip_buffer *buffs[2]; > + > + /* vfd lock */ > + struct mutex vlock; > + struct video_device vdev; > + struct media_pad pad; > + > + struct vip_output_fmt *vip_fmt_out; > + const struct vip_input_fmt *vip_fmt_in; > + struct v4l2_pix_format_mplane pixm; > +}; > + > +static inline struct rk_vip_stream *to_rk_vip_stream(struct video_device *vdev) > +{ > + return container_of(vdev, struct rk_vip_stream, vdev); > +} > + > +struct rk_vip_device { > + struct list_head list; > + struct device *dev; > + int irq; > + void __iomem *base_addr; > + void __iomem *csi_base; > + struct clk_bulk_data clks[RK_VIP_MAX_BUS_CLK]; > + int num_clk; > + struct vb2_alloc_ctx *alloc_ctx; > + bool iommu_en; > + struct iommu_domain *domain; > + struct reset_control *vip_rst; > + > + struct v4l2_device v4l2_dev; > + struct media_device media_dev; > + struct v4l2_ctrl_handler ctrl_handler; > + struct v4l2_async_notifier notifier; > + struct v4l2_async_connection asd; > + struct rk_vip_sensor_info sensor; Using "sensor" as name does not seem correct. As pointed out above it could be a video decoder just as well. Something with "subdevice" maybe? Thanks and best regards, Michael > [...] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-23 13:28 ` Michael Riesch @ 2023-10-25 8:49 ` Paul Kocialkowski 2023-10-25 9:38 ` Michael Riesch 0 siblings, 1 reply; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-25 8:49 UTC (permalink / raw) To: Michael Riesch Cc: Mehdi Djait, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 7039 bytes --] Hi, On Mon 23 Oct 23, 15:28, Michael Riesch wrote: > Typo in the subject: "Rockhip's" -> "Rockchip's" > I think this typo has been in there for a while now ;-) Great hips make for great dancing! > On 10/16/23 11:00, Mehdi Djait wrote: > > Introduce a driver for the camera interface on some Rockchip platforms. > > > > This controller supports CSI2 and BT656 interfaces, but for > > now only the BT656 interface could be tested, hence it's the only one > > that's supported in the first version of this driver. > > "CSI2" -> "MIPI CSI-2" ? > "BT656" -> "BT.656" ? > Also, additional interfaces are supported by some units, e.g., the > RK3568 VICAP also supports BT.1120. > > But most likely it becomes too complex to list everything, and it would > be better if you simply described the unit in the PX30. I think this > would clarify the commit message a lot. For now I would just stick to mentionning parallel (aka DVP). Indeed we don't need to list every possible parallel setup and MIPI CSI-2 is not supported in the current version of the driver. > > This controller can be fond on PX30, RK1808, RK3128 and RK3288, > > but for now it's only been tested on PX30. > > > > Most of this driver was written following the BSP driver from rockchip, > > "rockchip" -> "Rockchip" > > > removing the parts that either didn't fit correctly the guidelines, or > > that couldn't be tested. > > > > In the BSP, this driver is known as the "cif" driver, but this was > > renamed to "vip" to better fit the controller denomination in the > > datasheet. > > > > This basic version doesn't support cropping nor scaling, and is only > > designed with one SDTV video decoder being attached to it a any time. > > > > This version uses the "pingpong" mode of the controller, which is a > > double-buffering mechanism. > > > > Signed-off-by: Mehdi Djait <mehdi.djait@bootlin.com> > > Two things below: > > >[...] > > diff --git a/drivers/media/platform/rockchip/vip/dev.h b/drivers/media/platform/rockchip/vip/dev.h > > new file mode 100644 > > index 000000000000..eb9cd8f20ffc > > --- /dev/null > > +++ b/drivers/media/platform/rockchip/vip/dev.h > > @@ -0,0 +1,163 @@ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > +/* > > + * Rockchip VIP Driver > > + * > > + * Copyright (C) 2018 Rockchip Electronics Co., Ltd. > > + * Copyright (C) 2023 Mehdi Djait <mehdi.djait@bootlin.com> > > + */ > > + > > +#ifndef _RK_VIP_DEV_H > > +#define _RK_VIP_DEV_H > > + > > +#include <linux/clk.h> > > +#include <linux/mutex.h> > > +#include <media/media-device.h> > > +#include <media/media-entity.h> > > +#include <media/v4l2-ctrls.h> > > +#include <media/v4l2-device.h> > > +#include <media/videobuf2-v4l2.h> > > + > > +#define VIP_DRIVER_NAME "rk_vip" > > +#define VIP_VIDEODEVICE_NAME "stream_vip" > > + > > +#define RK_VIP_MAX_BUS_CLK 8 > > +#define RK_VIP_MAX_SENSOR 2 > > +#define RK_VIP_MAX_RESET 5 > > +#define RK_VIP_MAX_CSI_CHANNEL 4 > > + > > +#define RK_VIP_DEFAULT_WIDTH 640 > > +#define RK_VIP_DEFAULT_HEIGHT 480 > > + > > +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) > > +#define read_vip_reg(base, addr) readl((addr) + (base)) > > Please provide those helpers as proper inline functions. As to the > naming, the "_reg" suffix seems unnecessary. > > Alternatively, you could consider converting the driver to use regmap. Come to think of it, I feel like it would make more sense to have an inline function which is given a struct rk_vip_device instead of having to dereference it every time in the caller to access the base address. > > + > > +enum rk_vip_state { > > + RK_VIP_STATE_DISABLED, > > + RK_VIP_STATE_READY, > > + RK_VIP_STATE_STREAMING > > +}; > > + > > +enum rk_vip_chip_id { > > + CHIP_PX30_VIP, > > + CHIP_RK1808_VIP, > > + CHIP_RK3128_VIP, > > + CHIP_RK3288_VIP > > +}; > > + > > +enum host_type_t { > > + RK_CSI_RXHOST, > > + RK_DSI_RXHOST > > +}; > > + > > +struct rk_vip_buffer { > > + struct vb2_v4l2_buffer vb; > > + struct list_head queue; > > + union { > > + u32 buff_addr[VIDEO_MAX_PLANES]; > > + void *vaddr[VIDEO_MAX_PLANES]; > > + }; > > +}; > > + > > +struct rk_vip_scratch_buffer { > > + void *vaddr; > > + dma_addr_t dma_addr; > > + u32 size; > > +}; > > + > > +static inline struct rk_vip_buffer *to_rk_vip_buffer(struct vb2_v4l2_buffer *vb) > > +{ > > + return container_of(vb, struct rk_vip_buffer, vb); > > +} > > + > > +struct rk_vip_sensor_info { > > + struct v4l2_subdev *sd; > > + int pad; > > + struct v4l2_mbus_config mbus; > > + int lanes; > > + v4l2_std_id std; > > +}; > > + > > +struct vip_output_fmt { > > + u32 fourcc; > > + u32 mbus; > > + u32 fmt_val; > > + u8 cplanes; > > +}; > > + > > +enum vip_fmt_type { > > + VIP_FMT_TYPE_YUV = 0, > > + VIP_FMT_TYPE_RAW, > > +}; > > + > > +struct vip_input_fmt { > > + u32 mbus_code; > > + u32 dvp_fmt_val; > > + u32 csi_fmt_val; > > + enum vip_fmt_type fmt_type; > > + enum v4l2_field field; > > +}; > > + > > +struct rk_vip_stream { > > + struct rk_vip_device *vipdev; > > + enum rk_vip_state state; > > + bool stopping; > > + wait_queue_head_t wq_stopped; > > + int frame_idx; > > + int frame_phase; > > + > > + /* lock between irq and buf_queue */ > > + spinlock_t vbq_lock; > > + struct vb2_queue buf_queue; > > + struct list_head buf_head; > > + struct rk_vip_scratch_buffer scratch_buf; > > + struct rk_vip_buffer *buffs[2]; > > + > > + /* vfd lock */ > > + struct mutex vlock; > > + struct video_device vdev; > > + struct media_pad pad; > > + > > + struct vip_output_fmt *vip_fmt_out; > > + const struct vip_input_fmt *vip_fmt_in; > > + struct v4l2_pix_format_mplane pixm; > > +}; > > + > > +static inline struct rk_vip_stream *to_rk_vip_stream(struct video_device *vdev) > > +{ > > + return container_of(vdev, struct rk_vip_stream, vdev); > > +} > > + > > +struct rk_vip_device { > > + struct list_head list; > > + struct device *dev; > > + int irq; > > + void __iomem *base_addr; > > + void __iomem *csi_base; > > + struct clk_bulk_data clks[RK_VIP_MAX_BUS_CLK]; > > + int num_clk; > > + struct vb2_alloc_ctx *alloc_ctx; > > + bool iommu_en; > > + struct iommu_domain *domain; > > + struct reset_control *vip_rst; > > + > > + struct v4l2_device v4l2_dev; > > + struct media_device media_dev; > > + struct v4l2_ctrl_handler ctrl_handler; > > + struct v4l2_async_notifier notifier; > > + struct v4l2_async_connection asd; > > + struct rk_vip_sensor_info sensor; > > Using "sensor" as name does not seem correct. As pointed out above it > could be a video decoder just as well. Something with "subdevice" maybe? Agreed. I suggest renaming the struct "rk_vip_sensor_info" -> "rk_cif_remote" and just calling the member "remote". Cheers, Paul -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-25 8:49 ` Paul Kocialkowski @ 2023-10-25 9:38 ` Michael Riesch 2023-10-25 9:48 ` Paul Kocialkowski 0 siblings, 1 reply; 19+ messages in thread From: Michael Riesch @ 2023-10-25 9:38 UTC (permalink / raw) To: Paul Kocialkowski Cc: Mehdi Djait, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier Hi Paul, On 10/25/23 10:49, Paul Kocialkowski wrote: > Hi, > > On Mon 23 Oct 23, 15:28, Michael Riesch wrote: >> Typo in the subject: "Rockhip's" -> "Rockchip's" >> I think this typo has been in there for a while now ;-) > > Great hips make for great dancing! ...to rock music, obviously. > [...] >>> +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) >>> +#define read_vip_reg(base, addr) readl((addr) + (base)) >> >> Please provide those helpers as proper inline functions. As to the >> naming, the "_reg" suffix seems unnecessary. >> >> Alternatively, you could consider converting the driver to use regmap. > > Come to think of it, I feel like it would make more sense to have an inline > function which is given a struct rk_vip_device instead of having to dereference > it every time in the caller to access the base address. Indeed. Either using regmap, e.g., int regmap_write(struct regmap *map, unsigned int reg, unsigned int val); or something equivalant static inline int cif_write(struct cif_device *device, unsigned int reg, unsigned int val); Not sure what you agreed on in terms of a method prefix. The Rockchip RGA driver uses "rga_something", the Rockchip ISP driver uses "rkisp1_something". This would mean either "cif_something" or "rkcif_something", right? > [...] >>> + struct rk_vip_sensor_info sensor; >> >> Using "sensor" as name does not seem correct. As pointed out above it >> could be a video decoder just as well. Something with "subdevice" maybe? > > Agreed. I suggest renaming the struct "rk_vip_sensor_info" -> "rk_cif_remote" > and just calling the member "remote". "remote" sounds reasonable. Prefix to be bikeshedded, see comment above. In the future, we may add an array with mipi_remotes that represents the subdevices attached wia MIPI CSI-2. Best regards, Michael > Cheers, > > Paul > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-25 9:38 ` Michael Riesch @ 2023-10-25 9:48 ` Paul Kocialkowski 2023-10-25 10:28 ` Mehdi Djait 0 siblings, 1 reply; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-25 9:48 UTC (permalink / raw) To: Michael Riesch Cc: Mehdi Djait, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 2404 bytes --] Michael, On Wed 25 Oct 23, 11:38, Michael Riesch wrote: > Hi Paul, > > On 10/25/23 10:49, Paul Kocialkowski wrote: > > Hi, > > > > On Mon 23 Oct 23, 15:28, Michael Riesch wrote: > >> Typo in the subject: "Rockhip's" -> "Rockchip's" > >> I think this typo has been in there for a while now ;-) > > > > Great hips make for great dancing! > > ...to rock music, obviously. :) > > [...] > >>> +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) > >>> +#define read_vip_reg(base, addr) readl((addr) + (base)) > >> > >> Please provide those helpers as proper inline functions. As to the > >> naming, the "_reg" suffix seems unnecessary. > >> > >> Alternatively, you could consider converting the driver to use regmap. > > > > Come to think of it, I feel like it would make more sense to have an inline > > function which is given a struct rk_vip_device instead of having to dereference > > it every time in the caller to access the base address. > > Indeed. Either using regmap, e.g., > > int regmap_write(struct regmap *map, unsigned int reg, unsigned int val); > > or something equivalant > > static inline int cif_write(struct cif_device *device, unsigned int reg, > unsigned int val); Looks good to me! > Not sure what you agreed on in terms of a method prefix. The Rockchip > RGA driver uses "rga_something", the Rockchip ISP driver uses > "rkisp1_something". This would mean either "cif_something" or > "rkcif_something", right? Yeah I don't really have strong opinions on this so I'll let Mehdi decide (as long as it's consistent everywhere in the code). I guess there is a slight readability advantage in using "cif_" instead of "rkcif_". > > [...] > >>> + struct rk_vip_sensor_info sensor; > >> > >> Using "sensor" as name does not seem correct. As pointed out above it > >> could be a video decoder just as well. Something with "subdevice" maybe? > > > > Agreed. I suggest renaming the struct "rk_vip_sensor_info" -> "rk_cif_remote" > > and just calling the member "remote". > > "remote" sounds reasonable. Prefix to be bikeshedded, see comment above. > > In the future, we may add an array with mipi_remotes that represents the > subdevices attached wia MIPI CSI-2. Sounds good! Thanks, Paul -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface 2023-10-25 9:48 ` Paul Kocialkowski @ 2023-10-25 10:28 ` Mehdi Djait 0 siblings, 0 replies; 19+ messages in thread From: Mehdi Djait @ 2023-10-25 10:28 UTC (permalink / raw) To: Paul Kocialkowski Cc: Michael Riesch, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier Hi Paul, Hi Michael Thank you for the review! On Wed, Oct 25, 2023 at 11:48:08AM +0200, Paul Kocialkowski wrote: > Michael, > > On Wed 25 Oct 23, 11:38, Michael Riesch wrote: > > Hi Paul, > > > > On 10/25/23 10:49, Paul Kocialkowski wrote: > > > Hi, > > > > > > On Mon 23 Oct 23, 15:28, Michael Riesch wrote: > > >> Typo in the subject: "Rockhip's" -> "Rockchip's" > > >> I think this typo has been in there for a while now ;-) > > > > > > Great hips make for great dancing! > > > > ...to rock music, obviously. > > :) > > > > [...] > > >>> +#define write_vip_reg(base, addr, val) writel(val, (addr) + (base)) > > >>> +#define read_vip_reg(base, addr) readl((addr) + (base)) > > >> > > >> Please provide those helpers as proper inline functions. As to the > > >> naming, the "_reg" suffix seems unnecessary. > > >> > > >> Alternatively, you could consider converting the driver to use regmap. > > > > > > Come to think of it, I feel like it would make more sense to have an inline > > > function which is given a struct rk_vip_device instead of having to dereference > > > it every time in the caller to access the base address. > > > > Indeed. Either using regmap, e.g., > > > > int regmap_write(struct regmap *map, unsigned int reg, unsigned int val); > > > > or something equivalant > > > > static inline int cif_write(struct cif_device *device, unsigned int reg, > > unsigned int val); > > Looks good to me! > Yes, I will change it ro cif_write() > > Not sure what you agreed on in terms of a method prefix. The Rockchip > > RGA driver uses "rga_something", the Rockchip ISP driver uses > > "rkisp1_something". This would mean either "cif_something" or > > "rkcif_something", right? I am going with cif_*() > > Yeah I don't really have strong opinions on this so I'll let Mehdi decide > (as long as it's consistent everywhere in the code). > > I guess there is a slight readability advantage in using "cif_" instead of > "rkcif_". > > > > [...] > > >>> + struct rk_vip_sensor_info sensor; > > >> > > >> Using "sensor" as name does not seem correct. As pointed out above it > > >> could be a video decoder just as well. Something with "subdevice" maybe? > > > > > > Agreed. I suggest renaming the struct "rk_vip_sensor_info" -> "rk_cif_remote" > > > and just calling the member "remote". Yes "remote" sounds right in this situation -- Kind Regards Mehdi Djait ^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v8 3/3] arm64: dts: rockchip: Add the camera interface 2023-10-16 9:00 [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface Mehdi Djait 2023-10-16 9:00 ` [PATCH v8 1/3] media: dt-bindings: media: add bindings for Rockchip VIP Mehdi Djait 2023-10-16 9:00 ` [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface Mehdi Djait @ 2023-10-16 9:00 ` Mehdi Djait 2023-10-20 14:10 ` Paul Kocialkowski 2023-10-19 15:33 ` [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's " Paul Kocialkowski 3 siblings, 1 reply; 19+ messages in thread From: Mehdi Djait @ 2023-10-16 9:00 UTC (permalink / raw) To: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel Cc: linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier, paul.kocialkowski, Mehdi Djait The PX30 has a camera interface, supporting CSI2 and BT656 modes. Add a DT description for this interface. Signed-off-by: Mehdi Djait <mehdi.djait@bootlin.com> --- arch/arm64/boot/dts/rockchip/px30.dtsi | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/boot/dts/rockchip/px30.dtsi b/arch/arm64/boot/dts/rockchip/px30.dtsi index 42ce78beb413..7aaa88a15d07 100644 --- a/arch/arm64/boot/dts/rockchip/px30.dtsi +++ b/arch/arm64/boot/dts/rockchip/px30.dtsi @@ -1281,6 +1281,18 @@ isp_mmu: iommu@ff4a8000 { #iommu-cells = <0>; }; + vip: vip@ff490000 { + compatible = "rockchip,px30-vip"; + reg = <0x0 0xff490000 0x0 0x200>; + interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_CIF>, <&cru HCLK_CIF>, <&cru PCLK_CIF>; + clock-names = "aclk", "hclk", "pclk"; + power-domains = <&power PX30_PD_VI>; + resets = <&cru SRST_CIF_A>, <&cru SRST_CIF_H>, <&cru SRST_CIF_PCLKIN>; + reset-names = "axi", "ahb", "pclkin"; + status = "disabled"; + }; + qos_gmac: qos@ff518000 { compatible = "rockchip,px30-qos", "syscon"; reg = <0x0 0xff518000 0x0 0x20>; -- 2.41.0 ^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v8 3/3] arm64: dts: rockchip: Add the camera interface 2023-10-16 9:00 ` [PATCH v8 3/3] arm64: dts: rockchip: Add the " Mehdi Djait @ 2023-10-20 14:10 ` Paul Kocialkowski 0 siblings, 0 replies; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-20 14:10 UTC (permalink / raw) To: Mehdi Djait Cc: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 1956 bytes --] Hi Mehdi, On Mon 16 Oct 23, 11:00, Mehdi Djait wrote: > The PX30 has a camera interface, supporting CSI2 and BT656 > modes. Add a DT description for this interface. The "vip" node name is not very standard but the generic names recommendation doesn't have anything that really fits video capture: https://devicetree-specification.readthedocs.io/en/v0.3/devicetree-basics.html#generic-names-recommendation You might want to call it "video-capture" or "camera-controller" which seems like a good middle-ground that could be included in the list. Please keep the name (vip/cif) as label though. Other than that, please rename vip to cif and this will be: Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com> Cheers, Paul > Signed-off-by: Mehdi Djait <mehdi.djait@bootlin.com> > --- > arch/arm64/boot/dts/rockchip/px30.dtsi | 12 ++++++++++++ > 1 file changed, 12 insertions(+) > > diff --git a/arch/arm64/boot/dts/rockchip/px30.dtsi b/arch/arm64/boot/dts/rockchip/px30.dtsi > index 42ce78beb413..7aaa88a15d07 100644 > --- a/arch/arm64/boot/dts/rockchip/px30.dtsi > +++ b/arch/arm64/boot/dts/rockchip/px30.dtsi > @@ -1281,6 +1281,18 @@ isp_mmu: iommu@ff4a8000 { > #iommu-cells = <0>; > }; > > + vip: vip@ff490000 { > + compatible = "rockchip,px30-vip"; > + reg = <0x0 0xff490000 0x0 0x200>; > + interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>; > + clocks = <&cru ACLK_CIF>, <&cru HCLK_CIF>, <&cru PCLK_CIF>; > + clock-names = "aclk", "hclk", "pclk"; > + power-domains = <&power PX30_PD_VI>; > + resets = <&cru SRST_CIF_A>, <&cru SRST_CIF_H>, <&cru SRST_CIF_PCLKIN>; > + reset-names = "axi", "ahb", "pclkin"; > + status = "disabled"; > + }; > + > qos_gmac: qos@ff518000 { > compatible = "rockchip,px30-qos", "syscon"; > reg = <0x0 0xff518000 0x0 0x20>; > -- > 2.41.0 > -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface 2023-10-16 9:00 [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface Mehdi Djait ` (2 preceding siblings ...) 2023-10-16 9:00 ` [PATCH v8 3/3] arm64: dts: rockchip: Add the " Mehdi Djait @ 2023-10-19 15:33 ` Paul Kocialkowski 2023-10-23 13:07 ` Michael Riesch 3 siblings, 1 reply; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-19 15:33 UTC (permalink / raw) To: Mehdi Djait Cc: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 9654 bytes --] Hi Mehdi, On Mon 16 Oct 23, 11:00, Mehdi Djait wrote: > In the BSP, this driver is known as the "cif" driver, but this was > renamed to "vip" to better fit the controller denomination in the > datasheet. So I have spent a bit of time trying to figure out the history of this unit and in which platform in was used. The takeaway is that the earliest Rockchip SoC that uses it is the RK3066 (2012) and the latest SoC is the RK3566/RK3568 (2020). Earlier SoCs (RK29) do mention VIP but seems quite clear that this is a whole different unit and the recent RK3588 (2021) has a new VICAP_DVP unit (mixed with VICAP_MIPI) which also seems significantly different. Over the course of the existence of this unit, it is most often referred to as CIF. Since this is also the name for the driver in the Rockchip tree, I feel like it is best to use CIF as the mainline terminology instead of VIP. Note that the unit is also called VICAP in RK3566/RK3568. Here is the detail of my research on the concerned chips. The + at the beginning of the line indicate support in Rockchip's 4.4 tree: - RK3566/RK3568 (2020): CIF pins + VICAP terminology + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers + PX30 (2017): VIP pins + VIP registers + RK3328 (2017): CIF pins + VIP terminology - RK3326 (2017): CIF pins + VIP terminology - RK3399 (2016): CIF pins - RK3368 (2015): CIF pins - PX2 (2014-11): CIF pins + CIF registers + RK3126/RK3128 (2014-10): CIF pins + registers + RK3288 (2014-05): CIF pins + VIP terminology - RK3026 (2013): CIF pins + CIF registers - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers - RK3066 (2012): CIF pins + CIF registers Note that there are a few variations over time (added/removed registers), but the offsets of crucial registers are always the same, so we can safely assume this is the same unit in different generations. Since the RK3066 is the first model starting the RK30 lineup I think we can safely use that for the "base" compatible to be used for e.g. the bindings document, instead of px30 which is just one of the many SoCs that use this unit. Cheers, Paul > This version of the driver supports ONLY the parallel interface BT656 > and was tested/implemented using an SDTV video decoder > > media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 > > V7 => V8: > vip/capture.c: > - fixed a warning: unused variable reported by the kernel test robot > > V6 => V7: > vip/capture.c vip/dev.c vip/dev.h > - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev > - added some error when rk_vip_get_buffer() returns NULL > - removed a WARN_ON > - made the irq NOT shared > - dropped of_match_ptr > - added the rk_vip_get_resource() function > > rockchip,px30-vip.yaml: > - changed filename to match the compatible > - dropped the mention of the other rockchip SoC in the dt-binding > description and added a more detailed description of VIP > - removed unused labels in the example > > > V5 [1] => V6: > vip/capture.c vip/dev.c vip/dev.h > - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the > supported stds in rk_vip_enum_input callback > - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks > - added the supported video_device->tvnorms > - s_std will now update the format as this depends on the standard > NTSC/PAL (as suggested by Hans in [1]) > - removed STD_ATSC > - moved the colorimetry information to come from the subdev > - removed the core s_power subdev calls > - dropped cropping in rk_vip_stream struct > > rockchip-vip.yaml: > - fixed a mistake in the name of third clock plckin -> plck > - changed the reg maxItems 2 -> 1 > > [1] https://lore.kernel.org/linux-media/20201229161724.511102-1-maxime.chevallier@bootlin.com/ > > I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b > > # v4l2-compliance > v4l2-compliance 1.25.0, 64 bits, 64-bit time_t > > Compliance test for rk_vip device /dev/video0: > > Driver Info: > Driver name : rk_vip > Card type : rk_vip > Bus info : platform:ff490000.vip > Driver version : 6.6.0 > Capabilities : 0x84201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04201000 > Video Capture Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : rk_vip > Model : rk_vip > Serial : > Bus info : platform:ff490000.vip > Media version : 6.6.0 > Hardware revision: 0x00000000 (0) > Driver version : 6.6.0 > Interface Info: > ID : 0x03000002 > Type : V4L Video > Entity Info: > ID : 0x00000001 (1) > Name : video_rkvip > Function : V4L2 I/O > Pad 0x01000004 : 0: Sink > Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled > > Required ioctls: > test MC information (see 'Media Driver Info' above): OK > test VIDIOC_QUERYCAP: OK > test invalid ioctls: OK > > Allow for multiple opens: > test second /dev/video0 open: OK > test VIDIOC_QUERYCAP: OK > test VIDIOC_G/S_PRIORITY: OK > test for unlimited opens: OK > > Debug ioctls: > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > test VIDIOC_LOG_STATUS: OK (Not Supported) > > Input ioctls: > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > test VIDIOC_ENUMAUDIO: OK (Not Supported) > test VIDIOC_G/S/ENUMINPUT: OK > test VIDIOC_G/S_AUDIO: OK (Not Supported) > Inputs: 1 Audio Inputs: 0 Tuners: 0 > > Output ioctls: > test VIDIOC_G/S_MODULATOR: OK (Not Supported) > test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > test VIDIOC_ENUMAUDOUT: OK (Not Supported) > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > test VIDIOC_G/S_AUDOUT: OK (Not Supported) > Outputs: 0 Audio Outputs: 0 Modulators: 0 > > Input/Output configuration ioctls: > test VIDIOC_ENUM/G/S/QUERY_STD: OK > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > test VIDIOC_G/S_EDID: OK (Not Supported) > > Control ioctls (Input 0): > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > test VIDIOC_QUERYCTRL: OK (Not Supported) > test VIDIOC_G/S_CTRL: OK (Not Supported) > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > Standard Controls: 0 Private Controls: 0 > > Format ioctls (Input 0): > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > test VIDIOC_G/S_PARM: OK (Not Supported) > test VIDIOC_G_FBUF: OK (Not Supported) > test VIDIOC_G_FMT: OK > test VIDIOC_TRY_FMT: OK > test VIDIOC_S_FMT: OK > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > test Cropping: OK (Not Supported) > test Composing: OK (Not Supported) > test Scaling: OK (Not Supported) > > Codec ioctls (Input 0): > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > test VIDIOC_G_ENC_INDEX: OK (Not Supported) > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > > Buffer ioctls (Input 0): > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > test VIDIOC_EXPBUF: OK > test Requests: OK (Not Supported) > > Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0 > > Mehdi Djait (3): > media: dt-bindings: media: add bindings for Rockchip VIP > media: rockchip: Add a driver for Rockhip's camera interface > arm64: dts: rockchip: Add the camera interface > > .../bindings/media/rockchip,px30-vip.yaml | 93 ++ > arch/arm64/boot/dts/rockchip/px30.dtsi | 12 + > drivers/media/platform/rockchip/Kconfig | 1 + > drivers/media/platform/rockchip/Makefile | 1 + > drivers/media/platform/rockchip/vip/Kconfig | 14 + > drivers/media/platform/rockchip/vip/Makefile | 3 + > drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++ > drivers/media/platform/rockchip/vip/dev.c | 346 +++++ > drivers/media/platform/rockchip/vip/dev.h | 163 +++ > drivers/media/platform/rockchip/vip/regs.h | 260 ++++ > 10 files changed, 2103 insertions(+) > create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml > create mode 100644 drivers/media/platform/rockchip/vip/Kconfig > create mode 100644 drivers/media/platform/rockchip/vip/Makefile > create mode 100644 drivers/media/platform/rockchip/vip/capture.c > create mode 100644 drivers/media/platform/rockchip/vip/dev.c > create mode 100644 drivers/media/platform/rockchip/vip/dev.h > create mode 100644 drivers/media/platform/rockchip/vip/regs.h > > -- > 2.41.0 > -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface 2023-10-19 15:33 ` [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's " Paul Kocialkowski @ 2023-10-23 13:07 ` Michael Riesch 2023-10-25 8:43 ` Paul Kocialkowski 0 siblings, 1 reply; 19+ messages in thread From: Michael Riesch @ 2023-10-23 13:07 UTC (permalink / raw) To: Paul Kocialkowski, Mehdi Djait Cc: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier Hi Mehdi and Paul, Let me begin by saying how great it is to see this series moving forward. After nearly three years of silence I was not expecting that :-) Thanks for your efforts! On 10/19/23 17:33, Paul Kocialkowski wrote: > Hi Mehdi, > > On Mon 16 Oct 23, 11:00, Mehdi Djait wrote: >> In the BSP, this driver is known as the "cif" driver, but this was >> renamed to "vip" to better fit the controller denomination in the >> datasheet. > > So I have spent a bit of time trying to figure out the history of this unit > and in which platform in was used. The takeaway is that the earliest Rockchip > SoC that uses it is the RK3066 (2012) and the latest SoC is the RK3566/RK3568 > (2020). Earlier SoCs (RK29) do mention VIP but seems quite clear that this is > a whole different unit and the recent RK3588 (2021) has a new VICAP_DVP unit > (mixed with VICAP_MIPI) which also seems significantly different. The RK3588 VICAP may be significantly more complex, but it is supported by the same driver in the downstream kernel [0]. The DVP part (which is in the scope of this series) should be more or less the same deal (although Rockchip mixed the register positions once again, the registers itself are similar to e.g., RK356X when it comes to DVP). > Over the course of the existence of this unit, it is most often referred to > as CIF. Since this is also the name for the driver in the Rockchip tree, > I feel like it is best to use CIF as the mainline terminology instead of VIP. > Note that the unit is also called VICAP in RK3566/RK3568. But even if we take the RK3588 VICAP unit into account, I'd agree that CIF seems to be the better name (precisly because the downstream driver is called "cif"). The individual compatibles can be named "rockchip,px30-vip", "rockchip,rk3568-vicap", ..., right? This would be in contrast to the downstream driver (which uses "-cif" for all units) but here the alignment with the respective data sheet comes into play (which will be helpful). > Here is the detail of my research on the concerned chips. The + at the beginning > of the line indicate support in Rockchip's 4.4 tree: > > - RK3566/RK3568 (2020): CIF pins + VICAP terminology > + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers > + PX30 (2017): VIP pins + VIP registers > + RK3328 (2017): CIF pins + VIP terminology > - RK3326 (2017): CIF pins + VIP terminology > - RK3399 (2016): CIF pins > - RK3368 (2015): CIF pins > - PX2 (2014-11): CIF pins + CIF registers > + RK3126/RK3128 (2014-10): CIF pins + registers > + RK3288 (2014-05): CIF pins + VIP terminology > - RK3026 (2013): CIF pins + CIF registers > - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers > - RK3066 (2012): CIF pins + CIF registers > > Note that there are a few variations over time (added/removed registers), but > the offsets of crucial registers are always the same, so we can safely > assume this is the same unit in different generations. > > Since the RK3066 is the first model starting the RK30 lineup I think we can > safely use that for the "base" compatible to be used for e.g. the bindings > document, instead of px30 which is just one of the many SoCs that use this unit. Once the name of the driver is defined and adjusted in v9, I can try to give the series a shot on my RK3568 board. First attempts to do so basing on Maxime's v5 showed that with a few modifications the DVP feature works fine. In a subsequent step, we could discuss the inclusion of the MIPI CSI-2 things in order to keep the driver sufficiently general. @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated. Best regards, Michael [0] https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968 > >> This version of the driver supports ONLY the parallel interface BT656 >> and was tested/implemented using an SDTV video decoder >> >> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 >> >> V7 => V8: >> vip/capture.c: >> - fixed a warning: unused variable reported by the kernel test robot >> >> V6 => V7: >> vip/capture.c vip/dev.c vip/dev.h >> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev >> - added some error when rk_vip_get_buffer() returns NULL >> - removed a WARN_ON >> - made the irq NOT shared >> - dropped of_match_ptr >> - added the rk_vip_get_resource() function >> >> rockchip,px30-vip.yaml: >> - changed filename to match the compatible >> - dropped the mention of the other rockchip SoC in the dt-binding >> description and added a more detailed description of VIP >> - removed unused labels in the example >> >> >> V5 [1] => V6: >> vip/capture.c vip/dev.c vip/dev.h >> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the >> supported stds in rk_vip_enum_input callback >> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks >> - added the supported video_device->tvnorms >> - s_std will now update the format as this depends on the standard >> NTSC/PAL (as suggested by Hans in [1]) >> - removed STD_ATSC >> - moved the colorimetry information to come from the subdev >> - removed the core s_power subdev calls >> - dropped cropping in rk_vip_stream struct >> >> rockchip-vip.yaml: >> - fixed a mistake in the name of third clock plckin -> plck >> - changed the reg maxItems 2 -> 1 >> >> [1] https://lore.kernel.org/linux-media/20201229161724.511102-1-maxime.chevallier@bootlin.com/ >> >> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b >> >> # v4l2-compliance >> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t >> >> Compliance test for rk_vip device /dev/video0: >> >> Driver Info: >> Driver name : rk_vip >> Card type : rk_vip >> Bus info : platform:ff490000.vip >> Driver version : 6.6.0 >> Capabilities : 0x84201000 >> Video Capture Multiplanar >> Streaming >> Extended Pix Format >> Device Capabilities >> Device Caps : 0x04201000 >> Video Capture Multiplanar >> Streaming >> Extended Pix Format >> Media Driver Info: >> Driver name : rk_vip >> Model : rk_vip >> Serial : >> Bus info : platform:ff490000.vip >> Media version : 6.6.0 >> Hardware revision: 0x00000000 (0) >> Driver version : 6.6.0 >> Interface Info: >> ID : 0x03000002 >> Type : V4L Video >> Entity Info: >> ID : 0x00000001 (1) >> Name : video_rkvip >> Function : V4L2 I/O >> Pad 0x01000004 : 0: Sink >> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled >> >> Required ioctls: >> test MC information (see 'Media Driver Info' above): OK >> test VIDIOC_QUERYCAP: OK >> test invalid ioctls: OK >> >> Allow for multiple opens: >> test second /dev/video0 open: OK >> test VIDIOC_QUERYCAP: OK >> test VIDIOC_G/S_PRIORITY: OK >> test for unlimited opens: OK >> >> Debug ioctls: >> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) >> test VIDIOC_LOG_STATUS: OK (Not Supported) >> >> Input ioctls: >> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) >> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) >> test VIDIOC_ENUMAUDIO: OK (Not Supported) >> test VIDIOC_G/S/ENUMINPUT: OK >> test VIDIOC_G/S_AUDIO: OK (Not Supported) >> Inputs: 1 Audio Inputs: 0 Tuners: 0 >> >> Output ioctls: >> test VIDIOC_G/S_MODULATOR: OK (Not Supported) >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) >> test VIDIOC_ENUMAUDOUT: OK (Not Supported) >> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) >> test VIDIOC_G/S_AUDOUT: OK (Not Supported) >> Outputs: 0 Audio Outputs: 0 Modulators: 0 >> >> Input/Output configuration ioctls: >> test VIDIOC_ENUM/G/S/QUERY_STD: OK >> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) >> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) >> test VIDIOC_G/S_EDID: OK (Not Supported) >> >> Control ioctls (Input 0): >> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) >> test VIDIOC_QUERYCTRL: OK (Not Supported) >> test VIDIOC_G/S_CTRL: OK (Not Supported) >> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) >> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) >> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) >> Standard Controls: 0 Private Controls: 0 >> >> Format ioctls (Input 0): >> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK >> test VIDIOC_G/S_PARM: OK (Not Supported) >> test VIDIOC_G_FBUF: OK (Not Supported) >> test VIDIOC_G_FMT: OK >> test VIDIOC_TRY_FMT: OK >> test VIDIOC_S_FMT: OK >> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) >> test Cropping: OK (Not Supported) >> test Composing: OK (Not Supported) >> test Scaling: OK (Not Supported) >> >> Codec ioctls (Input 0): >> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) >> test VIDIOC_G_ENC_INDEX: OK (Not Supported) >> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) >> >> Buffer ioctls (Input 0): >> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK >> test VIDIOC_EXPBUF: OK >> test Requests: OK (Not Supported) >> >> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0 >> >> Mehdi Djait (3): >> media: dt-bindings: media: add bindings for Rockchip VIP >> media: rockchip: Add a driver for Rockhip's camera interface >> arm64: dts: rockchip: Add the camera interface >> >> .../bindings/media/rockchip,px30-vip.yaml | 93 ++ >> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 + >> drivers/media/platform/rockchip/Kconfig | 1 + >> drivers/media/platform/rockchip/Makefile | 1 + >> drivers/media/platform/rockchip/vip/Kconfig | 14 + >> drivers/media/platform/rockchip/vip/Makefile | 3 + >> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++ >> drivers/media/platform/rockchip/vip/dev.c | 346 +++++ >> drivers/media/platform/rockchip/vip/dev.h | 163 +++ >> drivers/media/platform/rockchip/vip/regs.h | 260 ++++ >> 10 files changed, 2103 insertions(+) >> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml >> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig >> create mode 100644 drivers/media/platform/rockchip/vip/Makefile >> create mode 100644 drivers/media/platform/rockchip/vip/capture.c >> create mode 100644 drivers/media/platform/rockchip/vip/dev.c >> create mode 100644 drivers/media/platform/rockchip/vip/dev.h >> create mode 100644 drivers/media/platform/rockchip/vip/regs.h >> >> -- >> 2.41.0 >> > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface 2023-10-23 13:07 ` Michael Riesch @ 2023-10-25 8:43 ` Paul Kocialkowski 2023-10-25 9:17 ` Michael Riesch 0 siblings, 1 reply; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-25 8:43 UTC (permalink / raw) To: Michael Riesch Cc: Mehdi Djait, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 12255 bytes --] Hi Michael, On Mon 23 Oct 23, 15:07, Michael Riesch wrote: > > So I have spent a bit of time trying to figure out the history of this unit > > and in which platform in was used. The takeaway is that the earliest Rockchip > > SoC that uses it is the RK3066 (2012) and the latest SoC is the RK3566/RK3568 > > (2020). Earlier SoCs (RK29) do mention VIP but seems quite clear that this is > > a whole different unit and the recent RK3588 (2021) has a new VICAP_DVP unit > > (mixed with VICAP_MIPI) which also seems significantly different. > > The RK3588 VICAP may be significantly more complex, but it is supported > by the same driver in the downstream kernel [0]. The DVP part (which is > in the scope of this series) should be more or less the same deal > (although Rockchip mixed the register positions once again, the > registers itself are similar to e.g., RK356X when it comes to DVP). After a closer look I tend to agree with you: they moved some registers around but it still seems to be the same base unit. > > Over the course of the existence of this unit, it is most often referred to > > as CIF. Since this is also the name for the driver in the Rockchip tree, > > I feel like it is best to use CIF as the mainline terminology instead of VIP. > > Note that the unit is also called VICAP in RK3566/RK3568. > > But even if we take the RK3588 VICAP unit into account, I'd agree that > CIF seems to be the better name (precisly because the downstream driver > is called "cif"). > > The individual compatibles can be named "rockchip,px30-vip", > "rockchip,rk3568-vicap", ..., right? This would be in contrast to the > downstream driver (which uses "-cif" for all units) but here the > alignment with the respective data sheet comes into play (which will be > helpful). Yeah I think it would make sense to keep the SoC-specific terminology in the compatible, so I agree that "rockchip,px30-vip" and "rockchip,rk3568-vicap" feel like the most relevant choices here. > > Here is the detail of my research on the concerned chips. The + at the beginning > > of the line indicate support in Rockchip's 4.4 tree: > > > > - RK3566/RK3568 (2020): CIF pins + VICAP terminology > > + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers > > + PX30 (2017): VIP pins + VIP registers > > + RK3328 (2017): CIF pins + VIP terminology > > - RK3326 (2017): CIF pins + VIP terminology > > - RK3399 (2016): CIF pins > > - RK3368 (2015): CIF pins > > - PX2 (2014-11): CIF pins + CIF registers > > + RK3126/RK3128 (2014-10): CIF pins + registers > > + RK3288 (2014-05): CIF pins + VIP terminology > > - RK3026 (2013): CIF pins + CIF registers > > - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers > > - RK3066 (2012): CIF pins + CIF registers > > > > Note that there are a few variations over time (added/removed registers), but > > the offsets of crucial registers are always the same, so we can safely > > assume this is the same unit in different generations. > > > > Since the RK3066 is the first model starting the RK30 lineup I think we can > > safely use that for the "base" compatible to be used for e.g. the bindings > > document, instead of px30 which is just one of the many SoCs that use this unit. > > Once the name of the driver is defined and adjusted in v9, I can try to > give the series a shot on my RK3568 board. First attempts to do so > basing on Maxime's v5 showed that with a few modifications the DVP > feature works fine. In a subsequent step, we could discuss the inclusion > of the MIPI CSI-2 things in order to keep the driver sufficiently general. Nice! I guess there will be a need to introduce a variant structure associated to each compatible to express the differences betweens these different generations. Note that we will also probably need to convert the driver over to a MC-centric approach, but this is of course outside of the scope of this series. Cheers, Paul > @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated. > > Best regards, > Michael > > [0] > https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968 > > > > >> This version of the driver supports ONLY the parallel interface BT656 > >> and was tested/implemented using an SDTV video decoder > >> > >> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 > >> > >> V7 => V8: > >> vip/capture.c: > >> - fixed a warning: unused variable reported by the kernel test robot > >> > >> V6 => V7: > >> vip/capture.c vip/dev.c vip/dev.h > >> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev > >> - added some error when rk_vip_get_buffer() returns NULL > >> - removed a WARN_ON > >> - made the irq NOT shared > >> - dropped of_match_ptr > >> - added the rk_vip_get_resource() function > >> > >> rockchip,px30-vip.yaml: > >> - changed filename to match the compatible > >> - dropped the mention of the other rockchip SoC in the dt-binding > >> description and added a more detailed description of VIP > >> - removed unused labels in the example > >> > >> > >> V5 [1] => V6: > >> vip/capture.c vip/dev.c vip/dev.h > >> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the > >> supported stds in rk_vip_enum_input callback > >> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks > >> - added the supported video_device->tvnorms > >> - s_std will now update the format as this depends on the standard > >> NTSC/PAL (as suggested by Hans in [1]) > >> - removed STD_ATSC > >> - moved the colorimetry information to come from the subdev > >> - removed the core s_power subdev calls > >> - dropped cropping in rk_vip_stream struct > >> > >> rockchip-vip.yaml: > >> - fixed a mistake in the name of third clock plckin -> plck > >> - changed the reg maxItems 2 -> 1 > >> > >> [1] https://lore.kernel.org/linux-media/20201229161724.511102-1-maxime.chevallier@bootlin.com/ > >> > >> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b > >> > >> # v4l2-compliance > >> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t > >> > >> Compliance test for rk_vip device /dev/video0: > >> > >> Driver Info: > >> Driver name : rk_vip > >> Card type : rk_vip > >> Bus info : platform:ff490000.vip > >> Driver version : 6.6.0 > >> Capabilities : 0x84201000 > >> Video Capture Multiplanar > >> Streaming > >> Extended Pix Format > >> Device Capabilities > >> Device Caps : 0x04201000 > >> Video Capture Multiplanar > >> Streaming > >> Extended Pix Format > >> Media Driver Info: > >> Driver name : rk_vip > >> Model : rk_vip > >> Serial : > >> Bus info : platform:ff490000.vip > >> Media version : 6.6.0 > >> Hardware revision: 0x00000000 (0) > >> Driver version : 6.6.0 > >> Interface Info: > >> ID : 0x03000002 > >> Type : V4L Video > >> Entity Info: > >> ID : 0x00000001 (1) > >> Name : video_rkvip > >> Function : V4L2 I/O > >> Pad 0x01000004 : 0: Sink > >> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled > >> > >> Required ioctls: > >> test MC information (see 'Media Driver Info' above): OK > >> test VIDIOC_QUERYCAP: OK > >> test invalid ioctls: OK > >> > >> Allow for multiple opens: > >> test second /dev/video0 open: OK > >> test VIDIOC_QUERYCAP: OK > >> test VIDIOC_G/S_PRIORITY: OK > >> test for unlimited opens: OK > >> > >> Debug ioctls: > >> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > >> test VIDIOC_LOG_STATUS: OK (Not Supported) > >> > >> Input ioctls: > >> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > >> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > >> test VIDIOC_ENUMAUDIO: OK (Not Supported) > >> test VIDIOC_G/S/ENUMINPUT: OK > >> test VIDIOC_G/S_AUDIO: OK (Not Supported) > >> Inputs: 1 Audio Inputs: 0 Tuners: 0 > >> > >> Output ioctls: > >> test VIDIOC_G/S_MODULATOR: OK (Not Supported) > >> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > >> test VIDIOC_ENUMAUDOUT: OK (Not Supported) > >> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > >> test VIDIOC_G/S_AUDOUT: OK (Not Supported) > >> Outputs: 0 Audio Outputs: 0 Modulators: 0 > >> > >> Input/Output configuration ioctls: > >> test VIDIOC_ENUM/G/S/QUERY_STD: OK > >> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > >> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > >> test VIDIOC_G/S_EDID: OK (Not Supported) > >> > >> Control ioctls (Input 0): > >> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > >> test VIDIOC_QUERYCTRL: OK (Not Supported) > >> test VIDIOC_G/S_CTRL: OK (Not Supported) > >> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > >> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > >> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > >> Standard Controls: 0 Private Controls: 0 > >> > >> Format ioctls (Input 0): > >> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > >> test VIDIOC_G/S_PARM: OK (Not Supported) > >> test VIDIOC_G_FBUF: OK (Not Supported) > >> test VIDIOC_G_FMT: OK > >> test VIDIOC_TRY_FMT: OK > >> test VIDIOC_S_FMT: OK > >> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > >> test Cropping: OK (Not Supported) > >> test Composing: OK (Not Supported) > >> test Scaling: OK (Not Supported) > >> > >> Codec ioctls (Input 0): > >> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > >> test VIDIOC_G_ENC_INDEX: OK (Not Supported) > >> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > >> > >> Buffer ioctls (Input 0): > >> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > >> test VIDIOC_EXPBUF: OK > >> test Requests: OK (Not Supported) > >> > >> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0 > >> > >> Mehdi Djait (3): > >> media: dt-bindings: media: add bindings for Rockchip VIP > >> media: rockchip: Add a driver for Rockhip's camera interface > >> arm64: dts: rockchip: Add the camera interface > >> > >> .../bindings/media/rockchip,px30-vip.yaml | 93 ++ > >> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 + > >> drivers/media/platform/rockchip/Kconfig | 1 + > >> drivers/media/platform/rockchip/Makefile | 1 + > >> drivers/media/platform/rockchip/vip/Kconfig | 14 + > >> drivers/media/platform/rockchip/vip/Makefile | 3 + > >> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++ > >> drivers/media/platform/rockchip/vip/dev.c | 346 +++++ > >> drivers/media/platform/rockchip/vip/dev.h | 163 +++ > >> drivers/media/platform/rockchip/vip/regs.h | 260 ++++ > >> 10 files changed, 2103 insertions(+) > >> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml > >> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig > >> create mode 100644 drivers/media/platform/rockchip/vip/Makefile > >> create mode 100644 drivers/media/platform/rockchip/vip/capture.c > >> create mode 100644 drivers/media/platform/rockchip/vip/dev.c > >> create mode 100644 drivers/media/platform/rockchip/vip/dev.h > >> create mode 100644 drivers/media/platform/rockchip/vip/regs.h > >> > >> -- > >> 2.41.0 > >> > > -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface 2023-10-25 8:43 ` Paul Kocialkowski @ 2023-10-25 9:17 ` Michael Riesch 2023-10-25 9:54 ` Paul Kocialkowski 0 siblings, 1 reply; 19+ messages in thread From: Michael Riesch @ 2023-10-25 9:17 UTC (permalink / raw) To: Paul Kocialkowski Cc: Mehdi Djait, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier Hi Paul, On 10/25/23 10:43, Paul Kocialkowski wrote: > [...] >>> Here is the detail of my research on the concerned chips. The + at the beginning >>> of the line indicate support in Rockchip's 4.4 tree: >>> >>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology >>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers >>> + PX30 (2017): VIP pins + VIP registers >>> + RK3328 (2017): CIF pins + VIP terminology >>> - RK3326 (2017): CIF pins + VIP terminology >>> - RK3399 (2016): CIF pins >>> - RK3368 (2015): CIF pins >>> - PX2 (2014-11): CIF pins + CIF registers >>> + RK3126/RK3128 (2014-10): CIF pins + registers >>> + RK3288 (2014-05): CIF pins + VIP terminology >>> - RK3026 (2013): CIF pins + CIF registers >>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers >>> - RK3066 (2012): CIF pins + CIF registers >>> >>> Note that there are a few variations over time (added/removed registers), but >>> the offsets of crucial registers are always the same, so we can safely >>> assume this is the same unit in different generations. >>> >>> Since the RK3066 is the first model starting the RK30 lineup I think we can >>> safely use that for the "base" compatible to be used for e.g. the bindings >>> document, instead of px30 which is just one of the many SoCs that use this unit. >> >> Once the name of the driver is defined and adjusted in v9, I can try to >> give the series a shot on my RK3568 board. First attempts to do so >> basing on Maxime's v5 showed that with a few modifications the DVP >> feature works fine. In a subsequent step, we could discuss the inclusion >> of the MIPI CSI-2 things in order to keep the driver sufficiently general. > > Nice! I guess there will be a need to introduce a variant structure associated > to each compatible to express the differences betweens these different > generations. Indeed. If Mehdi and you suggest something, I'd be happy to review. Otherwise, I'll try to come up with something reasonable. IMHO it would make sense (as a first step) to have the clocks and the resets in this structure, as well as a sub-structure that describes the DVP. The latter consists of registers mainly, but maybe supported input/output formats and other things should go in there as well. Also, downstream code has a significant number of if (some condition including chip_id) A; else B; things that we should probably get rid of with this variant structure. As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X will have one of those, RK3588 will feature even six of them. So we should add a const array to the variant structure. > Note that we will also probably need to convert the driver over to a MC-centric > approach, but this is of course outside of the scope of this series. That would absolutely make sense. What is missing, though? (I was wondering that the driver calls media_device_(un)register but no /dev/mediaX device pops up.) Best regards, Michael > > Cheers, > > Paul > >> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated. >> >> Best regards, >> Michael >> >> [0] >> https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968 >> >>> >>>> This version of the driver supports ONLY the parallel interface BT656 >>>> and was tested/implemented using an SDTV video decoder >>>> >>>> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 >>>> >>>> V7 => V8: >>>> vip/capture.c: >>>> - fixed a warning: unused variable reported by the kernel test robot >>>> >>>> V6 => V7: >>>> vip/capture.c vip/dev.c vip/dev.h >>>> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev >>>> - added some error when rk_vip_get_buffer() returns NULL >>>> - removed a WARN_ON >>>> - made the irq NOT shared >>>> - dropped of_match_ptr >>>> - added the rk_vip_get_resource() function >>>> >>>> rockchip,px30-vip.yaml: >>>> - changed filename to match the compatible >>>> - dropped the mention of the other rockchip SoC in the dt-binding >>>> description and added a more detailed description of VIP >>>> - removed unused labels in the example >>>> >>>> >>>> V5 [1] => V6: >>>> vip/capture.c vip/dev.c vip/dev.h >>>> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the >>>> supported stds in rk_vip_enum_input callback >>>> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks >>>> - added the supported video_device->tvnorms >>>> - s_std will now update the format as this depends on the standard >>>> NTSC/PAL (as suggested by Hans in [1]) >>>> - removed STD_ATSC >>>> - moved the colorimetry information to come from the subdev >>>> - removed the core s_power subdev calls >>>> - dropped cropping in rk_vip_stream struct >>>> >>>> rockchip-vip.yaml: >>>> - fixed a mistake in the name of third clock plckin -> plck >>>> - changed the reg maxItems 2 -> 1 >>>> >>>> [1] https://lore.kernel.org/linux-media/20201229161724.511102-1-maxime.chevallier@bootlin.com/ >>>> >>>> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b >>>> >>>> # v4l2-compliance >>>> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t >>>> >>>> Compliance test for rk_vip device /dev/video0: >>>> >>>> Driver Info: >>>> Driver name : rk_vip >>>> Card type : rk_vip >>>> Bus info : platform:ff490000.vip >>>> Driver version : 6.6.0 >>>> Capabilities : 0x84201000 >>>> Video Capture Multiplanar >>>> Streaming >>>> Extended Pix Format >>>> Device Capabilities >>>> Device Caps : 0x04201000 >>>> Video Capture Multiplanar >>>> Streaming >>>> Extended Pix Format >>>> Media Driver Info: >>>> Driver name : rk_vip >>>> Model : rk_vip >>>> Serial : >>>> Bus info : platform:ff490000.vip >>>> Media version : 6.6.0 >>>> Hardware revision: 0x00000000 (0) >>>> Driver version : 6.6.0 >>>> Interface Info: >>>> ID : 0x03000002 >>>> Type : V4L Video >>>> Entity Info: >>>> ID : 0x00000001 (1) >>>> Name : video_rkvip >>>> Function : V4L2 I/O >>>> Pad 0x01000004 : 0: Sink >>>> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled >>>> >>>> Required ioctls: >>>> test MC information (see 'Media Driver Info' above): OK >>>> test VIDIOC_QUERYCAP: OK >>>> test invalid ioctls: OK >>>> >>>> Allow for multiple opens: >>>> test second /dev/video0 open: OK >>>> test VIDIOC_QUERYCAP: OK >>>> test VIDIOC_G/S_PRIORITY: OK >>>> test for unlimited opens: OK >>>> >>>> Debug ioctls: >>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) >>>> test VIDIOC_LOG_STATUS: OK (Not Supported) >>>> >>>> Input ioctls: >>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) >>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) >>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) >>>> test VIDIOC_ENUMAUDIO: OK (Not Supported) >>>> test VIDIOC_G/S/ENUMINPUT: OK >>>> test VIDIOC_G/S_AUDIO: OK (Not Supported) >>>> Inputs: 1 Audio Inputs: 0 Tuners: 0 >>>> >>>> Output ioctls: >>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported) >>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) >>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported) >>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) >>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported) >>>> Outputs: 0 Audio Outputs: 0 Modulators: 0 >>>> >>>> Input/Output configuration ioctls: >>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK >>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) >>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) >>>> test VIDIOC_G/S_EDID: OK (Not Supported) >>>> >>>> Control ioctls (Input 0): >>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) >>>> test VIDIOC_QUERYCTRL: OK (Not Supported) >>>> test VIDIOC_G/S_CTRL: OK (Not Supported) >>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) >>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) >>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) >>>> Standard Controls: 0 Private Controls: 0 >>>> >>>> Format ioctls (Input 0): >>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK >>>> test VIDIOC_G/S_PARM: OK (Not Supported) >>>> test VIDIOC_G_FBUF: OK (Not Supported) >>>> test VIDIOC_G_FMT: OK >>>> test VIDIOC_TRY_FMT: OK >>>> test VIDIOC_S_FMT: OK >>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) >>>> test Cropping: OK (Not Supported) >>>> test Composing: OK (Not Supported) >>>> test Scaling: OK (Not Supported) >>>> >>>> Codec ioctls (Input 0): >>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) >>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported) >>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) >>>> >>>> Buffer ioctls (Input 0): >>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK >>>> test VIDIOC_EXPBUF: OK >>>> test Requests: OK (Not Supported) >>>> >>>> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0 >>>> >>>> Mehdi Djait (3): >>>> media: dt-bindings: media: add bindings for Rockchip VIP >>>> media: rockchip: Add a driver for Rockhip's camera interface >>>> arm64: dts: rockchip: Add the camera interface >>>> >>>> .../bindings/media/rockchip,px30-vip.yaml | 93 ++ >>>> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 + >>>> drivers/media/platform/rockchip/Kconfig | 1 + >>>> drivers/media/platform/rockchip/Makefile | 1 + >>>> drivers/media/platform/rockchip/vip/Kconfig | 14 + >>>> drivers/media/platform/rockchip/vip/Makefile | 3 + >>>> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++ >>>> drivers/media/platform/rockchip/vip/dev.c | 346 +++++ >>>> drivers/media/platform/rockchip/vip/dev.h | 163 +++ >>>> drivers/media/platform/rockchip/vip/regs.h | 260 ++++ >>>> 10 files changed, 2103 insertions(+) >>>> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml >>>> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig >>>> create mode 100644 drivers/media/platform/rockchip/vip/Makefile >>>> create mode 100644 drivers/media/platform/rockchip/vip/capture.c >>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.c >>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.h >>>> create mode 100644 drivers/media/platform/rockchip/vip/regs.h >>>> >>>> -- >>>> 2.41.0 >>>> >>> > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface 2023-10-25 9:17 ` Michael Riesch @ 2023-10-25 9:54 ` Paul Kocialkowski 2023-10-25 10:33 ` Mehdi Djait 0 siblings, 1 reply; 19+ messages in thread From: Paul Kocialkowski @ 2023-10-25 9:54 UTC (permalink / raw) To: Michael Riesch Cc: Mehdi Djait, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier [-- Attachment #1: Type: text/plain, Size: 12878 bytes --] Michael, On Wed 25 Oct 23, 11:17, Michael Riesch wrote: > Hi Paul, > > On 10/25/23 10:43, Paul Kocialkowski wrote: > > [...] > >>> Here is the detail of my research on the concerned chips. The + at the beginning > >>> of the line indicate support in Rockchip's 4.4 tree: > >>> > >>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology > >>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers > >>> + PX30 (2017): VIP pins + VIP registers > >>> + RK3328 (2017): CIF pins + VIP terminology > >>> - RK3326 (2017): CIF pins + VIP terminology > >>> - RK3399 (2016): CIF pins > >>> - RK3368 (2015): CIF pins > >>> - PX2 (2014-11): CIF pins + CIF registers > >>> + RK3126/RK3128 (2014-10): CIF pins + registers > >>> + RK3288 (2014-05): CIF pins + VIP terminology > >>> - RK3026 (2013): CIF pins + CIF registers > >>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers > >>> - RK3066 (2012): CIF pins + CIF registers > >>> > >>> Note that there are a few variations over time (added/removed registers), but > >>> the offsets of crucial registers are always the same, so we can safely > >>> assume this is the same unit in different generations. > >>> > >>> Since the RK3066 is the first model starting the RK30 lineup I think we can > >>> safely use that for the "base" compatible to be used for e.g. the bindings > >>> document, instead of px30 which is just one of the many SoCs that use this unit. > >> > >> Once the name of the driver is defined and adjusted in v9, I can try to > >> give the series a shot on my RK3568 board. First attempts to do so > >> basing on Maxime's v5 showed that with a few modifications the DVP > >> feature works fine. In a subsequent step, we could discuss the inclusion > >> of the MIPI CSI-2 things in order to keep the driver sufficiently general. > > > > Nice! I guess there will be a need to introduce a variant structure associated > > to each compatible to express the differences betweens these different > > generations. > > Indeed. If Mehdi and you suggest something, I'd be happy to review. Well the be honest the scope of work on our side is really centered on PX30 and merging this first version. > Otherwise, I'll try to come up with something reasonable. IMHO it would > make sense (as a first step) to have the clocks and the resets in this > structure, as well as a sub-structure that describes the DVP. The latter > consists of registers mainly, but maybe supported input/output formats > and other things should go in there as well. Also, downstream code has a > significant number of > if (some condition including chip_id) A; else B; > things that we should probably get rid of with this variant structure. Indeed I think we want to try avoid that. Another common option is to define capability flags to represent differences between generations in a more practical and clean way than explicitly checking chip ids or so. > As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X > will have one of those, RK3588 will feature even six of them. So we > should add a const array to the variant structure. > > > Note that we will also probably need to convert the driver over to a MC-centric > > approach, but this is of course outside of the scope of this series. > > That would absolutely make sense. What is missing, though? (I was > wondering that the driver calls media_device_(un)register but no > /dev/mediaX device pops up.) Switching from video node-centric to MC-centric is more of a semantic change. In the first case we expect that subdevs are configured by the video device driver and userspace is not expected to change anything in the media topology or to configure media entities explicitly. In the latter case it's the opposite : the driver should never try to push configuration to a subdev and should instead validate that the current configuration makes sense. Still, I believe should be a media device registered and visible to userspace. Mehdi could you take a look at this? Do you see a media device in `media-ctl -p` and /dev/mediaX? Cheers, Paul > Best regards, > Michael > > > > > Cheers, > > > > Paul > > > >> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated. > >> > >> Best regards, > >> Michael > >> > >> [0] > >> https://github.com/rockchip-linux/kernel/blob/develop-5.10/drivers/media/platform/rockchip/cif/hw.c#L968 > >> > >>> > >>>> This version of the driver supports ONLY the parallel interface BT656 > >>>> and was tested/implemented using an SDTV video decoder > >>>> > >>>> media_tree, base-commit: 2c1bae27df787c9535e48cc27bbd11c3c3e0a235 > >>>> > >>>> V7 => V8: > >>>> vip/capture.c: > >>>> - fixed a warning: unused variable reported by the kernel test robot > >>>> > >>>> V6 => V7: > >>>> vip/capture.c vip/dev.c vip/dev.h > >>>> - renamed all struct rk_vip_dev dev => struct rk_vip_dev vip_dev > >>>> - added some error when rk_vip_get_buffer() returns NULL > >>>> - removed a WARN_ON > >>>> - made the irq NOT shared > >>>> - dropped of_match_ptr > >>>> - added the rk_vip_get_resource() function > >>>> > >>>> rockchip,px30-vip.yaml: > >>>> - changed filename to match the compatible > >>>> - dropped the mention of the other rockchip SoC in the dt-binding > >>>> description and added a more detailed description of VIP > >>>> - removed unused labels in the example > >>>> > >>>> > >>>> V5 [1] => V6: > >>>> vip/capture.c vip/dev.c vip/dev.h > >>>> - added a video g_input_status subdev call, V4L2_IN_CAP_STD and the > >>>> supported stds in rk_vip_enum_input callback > >>>> - added rk_vip_g_std, rk_vip_s_std and rk_vip_querystd callbacks > >>>> - added the supported video_device->tvnorms > >>>> - s_std will now update the format as this depends on the standard > >>>> NTSC/PAL (as suggested by Hans in [1]) > >>>> - removed STD_ATSC > >>>> - moved the colorimetry information to come from the subdev > >>>> - removed the core s_power subdev calls > >>>> - dropped cropping in rk_vip_stream struct > >>>> > >>>> rockchip-vip.yaml: > >>>> - fixed a mistake in the name of third clock plckin -> plck > >>>> - changed the reg maxItems 2 -> 1 > >>>> > >>>> [1] https://lore.kernel.org/linux-media/20201229161724.511102-1-maxime.chevallier@bootlin.com/ > >>>> > >>>> I used v4l-utils with HEAD: commit 1ee258e5bb91a12df378e19eb255c5219d6bc36b > >>>> > >>>> # v4l2-compliance > >>>> v4l2-compliance 1.25.0, 64 bits, 64-bit time_t > >>>> > >>>> Compliance test for rk_vip device /dev/video0: > >>>> > >>>> Driver Info: > >>>> Driver name : rk_vip > >>>> Card type : rk_vip > >>>> Bus info : platform:ff490000.vip > >>>> Driver version : 6.6.0 > >>>> Capabilities : 0x84201000 > >>>> Video Capture Multiplanar > >>>> Streaming > >>>> Extended Pix Format > >>>> Device Capabilities > >>>> Device Caps : 0x04201000 > >>>> Video Capture Multiplanar > >>>> Streaming > >>>> Extended Pix Format > >>>> Media Driver Info: > >>>> Driver name : rk_vip > >>>> Model : rk_vip > >>>> Serial : > >>>> Bus info : platform:ff490000.vip > >>>> Media version : 6.6.0 > >>>> Hardware revision: 0x00000000 (0) > >>>> Driver version : 6.6.0 > >>>> Interface Info: > >>>> ID : 0x03000002 > >>>> Type : V4L Video > >>>> Entity Info: > >>>> ID : 0x00000001 (1) > >>>> Name : video_rkvip > >>>> Function : V4L2 I/O > >>>> Pad 0x01000004 : 0: Sink > >>>> Link 0x02000009: from remote pad 0x1000006 of entity 'tw9900 2-0044' (Digital Video Decoder): Data, Enabled > >>>> > >>>> Required ioctls: > >>>> test MC information (see 'Media Driver Info' above): OK > >>>> test VIDIOC_QUERYCAP: OK > >>>> test invalid ioctls: OK > >>>> > >>>> Allow for multiple opens: > >>>> test second /dev/video0 open: OK > >>>> test VIDIOC_QUERYCAP: OK > >>>> test VIDIOC_G/S_PRIORITY: OK > >>>> test for unlimited opens: OK > >>>> > >>>> Debug ioctls: > >>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) > >>>> test VIDIOC_LOG_STATUS: OK (Not Supported) > >>>> > >>>> Input ioctls: > >>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) > >>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > >>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) > >>>> test VIDIOC_ENUMAUDIO: OK (Not Supported) > >>>> test VIDIOC_G/S/ENUMINPUT: OK > >>>> test VIDIOC_G/S_AUDIO: OK (Not Supported) > >>>> Inputs: 1 Audio Inputs: 0 Tuners: 0 > >>>> > >>>> Output ioctls: > >>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported) > >>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported) > >>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported) > >>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) > >>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported) > >>>> Outputs: 0 Audio Outputs: 0 Modulators: 0 > >>>> > >>>> Input/Output configuration ioctls: > >>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK > >>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) > >>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) > >>>> test VIDIOC_G/S_EDID: OK (Not Supported) > >>>> > >>>> Control ioctls (Input 0): > >>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) > >>>> test VIDIOC_QUERYCTRL: OK (Not Supported) > >>>> test VIDIOC_G/S_CTRL: OK (Not Supported) > >>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) > >>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) > >>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) > >>>> Standard Controls: 0 Private Controls: 0 > >>>> > >>>> Format ioctls (Input 0): > >>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK > >>>> test VIDIOC_G/S_PARM: OK (Not Supported) > >>>> test VIDIOC_G_FBUF: OK (Not Supported) > >>>> test VIDIOC_G_FMT: OK > >>>> test VIDIOC_TRY_FMT: OK > >>>> test VIDIOC_S_FMT: OK > >>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) > >>>> test Cropping: OK (Not Supported) > >>>> test Composing: OK (Not Supported) > >>>> test Scaling: OK (Not Supported) > >>>> > >>>> Codec ioctls (Input 0): > >>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) > >>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported) > >>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) > >>>> > >>>> Buffer ioctls (Input 0): > >>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK > >>>> test VIDIOC_EXPBUF: OK > >>>> test Requests: OK (Not Supported) > >>>> > >>>> Total for rk_vip device /dev/video0: 46, Succeeded: 46, Failed: 0, Warnings: 0 > >>>> > >>>> Mehdi Djait (3): > >>>> media: dt-bindings: media: add bindings for Rockchip VIP > >>>> media: rockchip: Add a driver for Rockhip's camera interface > >>>> arm64: dts: rockchip: Add the camera interface > >>>> > >>>> .../bindings/media/rockchip,px30-vip.yaml | 93 ++ > >>>> arch/arm64/boot/dts/rockchip/px30.dtsi | 12 + > >>>> drivers/media/platform/rockchip/Kconfig | 1 + > >>>> drivers/media/platform/rockchip/Makefile | 1 + > >>>> drivers/media/platform/rockchip/vip/Kconfig | 14 + > >>>> drivers/media/platform/rockchip/vip/Makefile | 3 + > >>>> drivers/media/platform/rockchip/vip/capture.c | 1210 +++++++++++++++++ > >>>> drivers/media/platform/rockchip/vip/dev.c | 346 +++++ > >>>> drivers/media/platform/rockchip/vip/dev.h | 163 +++ > >>>> drivers/media/platform/rockchip/vip/regs.h | 260 ++++ > >>>> 10 files changed, 2103 insertions(+) > >>>> create mode 100644 Documentation/devicetree/bindings/media/rockchip,px30-vip.yaml > >>>> create mode 100644 drivers/media/platform/rockchip/vip/Kconfig > >>>> create mode 100644 drivers/media/platform/rockchip/vip/Makefile > >>>> create mode 100644 drivers/media/platform/rockchip/vip/capture.c > >>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.c > >>>> create mode 100644 drivers/media/platform/rockchip/vip/dev.h > >>>> create mode 100644 drivers/media/platform/rockchip/vip/regs.h > >>>> > >>>> -- > >>>> 2.41.0 > >>>> > >>> > > -- Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface 2023-10-25 9:54 ` Paul Kocialkowski @ 2023-10-25 10:33 ` Mehdi Djait 2023-10-25 13:12 ` Michael Riesch 0 siblings, 1 reply; 19+ messages in thread From: Mehdi Djait @ 2023-10-25 10:33 UTC (permalink / raw) To: Paul Kocialkowski Cc: Michael Riesch, mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier Hi Paul, Hi Michael On Wed, Oct 25, 2023 at 11:54:27AM +0200, Paul Kocialkowski wrote: > Michael, > > On Wed 25 Oct 23, 11:17, Michael Riesch wrote: > > Hi Paul, > > > > On 10/25/23 10:43, Paul Kocialkowski wrote: > > > [...] > > >>> Here is the detail of my research on the concerned chips. The + at the beginning > > >>> of the line indicate support in Rockchip's 4.4 tree: > > >>> > > >>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology > > >>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers > > >>> + PX30 (2017): VIP pins + VIP registers > > >>> + RK3328 (2017): CIF pins + VIP terminology > > >>> - RK3326 (2017): CIF pins + VIP terminology > > >>> - RK3399 (2016): CIF pins > > >>> - RK3368 (2015): CIF pins > > >>> - PX2 (2014-11): CIF pins + CIF registers > > >>> + RK3126/RK3128 (2014-10): CIF pins + registers > > >>> + RK3288 (2014-05): CIF pins + VIP terminology > > >>> - RK3026 (2013): CIF pins + CIF registers > > >>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers > > >>> - RK3066 (2012): CIF pins + CIF registers > > >>> > > >>> Note that there are a few variations over time (added/removed registers), but > > >>> the offsets of crucial registers are always the same, so we can safely > > >>> assume this is the same unit in different generations. > > >>> > > >>> Since the RK3066 is the first model starting the RK30 lineup I think we can > > >>> safely use that for the "base" compatible to be used for e.g. the bindings > > >>> document, instead of px30 which is just one of the many SoCs that use this unit. > > >> > > >> Once the name of the driver is defined and adjusted in v9, I can try to > > >> give the series a shot on my RK3568 board. First attempts to do so This sounds good! > > >> basing on Maxime's v5 showed that with a few modifications the DVP > > >> feature works fine. In a subsequent step, we could discuss the inclusion > > >> of the MIPI CSI-2 things in order to keep the driver sufficiently general. > > > > > > Nice! I guess there will be a need to introduce a variant structure associated > > > to each compatible to express the differences betweens these different > > > generations. > > > > Indeed. If Mehdi and you suggest something, I'd be happy to review. > > Well the be honest the scope of work on our side is really centered on PX30 > and merging this first version. > > > Otherwise, I'll try to come up with something reasonable. IMHO it would > > make sense (as a first step) to have the clocks and the resets in this > > structure, as well as a sub-structure that describes the DVP. The latter > > consists of registers mainly, but maybe supported input/output formats > > and other things should go in there as well. Also, downstream code has a > > significant number of > > if (some condition including chip_id) A; else B; > > things that we should probably get rid of with this variant structure. > > Indeed I think we want to try avoid that. Another common option is to define > capability flags to represent differences between generations in a more > practical and clean way than explicitly checking chip ids or so. > > > As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X > > will have one of those, RK3588 will feature even six of them. So we > > should add a const array to the variant structure. > > > > > Note that we will also probably need to convert the driver over to a MC-centric > > > approach, but this is of course outside of the scope of this series. > > > > That would absolutely make sense. What is missing, though? (I was > > wondering that the driver calls media_device_(un)register but no > > /dev/mediaX device pops up.) > > Switching from video node-centric to MC-centric is more of a semantic change. > In the first case we expect that subdevs are configured by the video device > driver and userspace is not expected to change anything in the media topology > or to configure media entities explicitly. > > In the latter case it's the opposite : the driver should never try to push > configuration to a subdev and should instead validate that the current > configuration makes sense. > > Still, I believe should be a media device registered and visible to userspace. > Mehdi could you take a look at this? Do you see a media device in `media-ctl -p` > and /dev/mediaX? Yes I do have a media device media-ctl -p Media device information ------------------------ driver rockchip-cif model rk_cif serial bus info platform:ff490000.video-capture hw revision 0x0 driver version 6.6.0 Device topology - entity 1: rockchip_cif (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "tw9900 2-0044":0 [ENABLED] - entity 5: tw9900 2-0044 (1 pad, 1 link) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Source [fmt:UYVY8_2X8/720x480 field:none colorspace:smpte170m xfer:709 ycbcr:601] -> "rockchip_cif":0 [ENABLED] > > Cheers, > > Paul > > > Best regards, > > Michael > > > > > > > > Cheers, > > > > > > Paul > > > > > >> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated. Of course I will :) -- Kind Regards Mehdi Djait ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface 2023-10-25 10:33 ` Mehdi Djait @ 2023-10-25 13:12 ` Michael Riesch 0 siblings, 0 replies; 19+ messages in thread From: Michael Riesch @ 2023-10-25 13:12 UTC (permalink / raw) To: Mehdi Djait, Paul Kocialkowski Cc: mchehab, heiko, hverkuil-cisco, krzysztof.kozlowski+dt, robh+dt, conor+dt, ezequiel, linux-media, devicetree, linux-kernel, thomas.petazzoni, alexandre.belloni, maxime.chevallier Hi Mehdi, On 10/25/23 12:33, Mehdi Djait wrote: > Hi Paul, Hi Michael > > On Wed, Oct 25, 2023 at 11:54:27AM +0200, Paul Kocialkowski wrote: >> Michael, >> >> On Wed 25 Oct 23, 11:17, Michael Riesch wrote: >>> Hi Paul, >>> >>> On 10/25/23 10:43, Paul Kocialkowski wrote: >>>> [...] >>>>>> Here is the detail of my research on the concerned chips. The + at the beginning >>>>>> of the line indicate support in Rockchip's 4.4 tree: >>>>>> >>>>>> - RK3566/RK3568 (2020): CIF pins + VICAP terminology >>>>>> + RK1808 (2019): CIF pins + VIP registers + VIP_MIPI registers >>>>>> + PX30 (2017): VIP pins + VIP registers >>>>>> + RK3328 (2017): CIF pins + VIP terminology >>>>>> - RK3326 (2017): CIF pins + VIP terminology >>>>>> - RK3399 (2016): CIF pins >>>>>> - RK3368 (2015): CIF pins >>>>>> - PX2 (2014-11): CIF pins + CIF registers >>>>>> + RK3126/RK3128 (2014-10): CIF pins + registers >>>>>> + RK3288 (2014-05): CIF pins + VIP terminology >>>>>> - RK3026 (2013): CIF pins + CIF registers >>>>>> - RK3168/RK3188/PX3 (2012): CIF pins + CIF registers >>>>>> - RK3066 (2012): CIF pins + CIF registers >>>>>> >>>>>> Note that there are a few variations over time (added/removed registers), but >>>>>> the offsets of crucial registers are always the same, so we can safely >>>>>> assume this is the same unit in different generations. >>>>>> >>>>>> Since the RK3066 is the first model starting the RK30 lineup I think we can >>>>>> safely use that for the "base" compatible to be used for e.g. the bindings >>>>>> document, instead of px30 which is just one of the many SoCs that use this unit. >>>>> >>>>> Once the name of the driver is defined and adjusted in v9, I can try to >>>>> give the series a shot on my RK3568 board. First attempts to do so > > This sounds good! > >>>>> basing on Maxime's v5 showed that with a few modifications the DVP >>>>> feature works fine. In a subsequent step, we could discuss the inclusion >>>>> of the MIPI CSI-2 things in order to keep the driver sufficiently general. >>>> >>>> Nice! I guess there will be a need to introduce a variant structure associated >>>> to each compatible to express the differences betweens these different >>>> generations. >>> >>> Indeed. If Mehdi and you suggest something, I'd be happy to review. >> >> Well the be honest the scope of work on our side is really centered on PX30 >> and merging this first version. >> >>> Otherwise, I'll try to come up with something reasonable. IMHO it would >>> make sense (as a first step) to have the clocks and the resets in this >>> structure, as well as a sub-structure that describes the DVP. The latter >>> consists of registers mainly, but maybe supported input/output formats >>> and other things should go in there as well. Also, downstream code has a >>> significant number of >>> if (some condition including chip_id) A; else B; >>> things that we should probably get rid of with this variant structure. >> >> Indeed I think we want to try avoid that. Another common option is to define >> capability flags to represent differences between generations in a more >> practical and clean way than explicitly checking chip ids or so. >> >>> As next step, a sub-structure for MIPI CSI-2 could be defined. RK356X >>> will have one of those, RK3588 will feature even six of them. So we >>> should add a const array to the variant structure. >>> >>>> Note that we will also probably need to convert the driver over to a MC-centric >>>> approach, but this is of course outside of the scope of this series. >>> >>> That would absolutely make sense. What is missing, though? (I was >>> wondering that the driver calls media_device_(un)register but no >>> /dev/mediaX device pops up.) Sorry, had a false memory of this. The media device *does* pop up. >> Switching from video node-centric to MC-centric is more of a semantic change. >> In the first case we expect that subdevs are configured by the video device >> driver and userspace is not expected to change anything in the media topology >> or to configure media entities explicitly. >> >> In the latter case it's the opposite : the driver should never try to push >> configuration to a subdev and should instead validate that the current >> configuration makes sense. >> >> Still, I believe should be a media device registered and visible to userspace. >> Mehdi could you take a look at this? Do you see a media device in `media-ctl -p` >> and /dev/mediaX? > > Yes I do have a media device > > media-ctl -p > Media device information > ------------------------ > driver rockchip-cif > model rk_cif > serial > bus info platform:ff490000.video-capture > hw revision 0x0 > driver version 6.6.0 > > Device topology > - entity 1: rockchip_cif (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video0 > pad0: Sink > <- "tw9900 2-0044":0 [ENABLED] > > - entity 5: tw9900 2-0044 (1 pad, 1 link) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev0 > pad0: Source > [fmt:UYVY8_2X8/720x480 field:none colorspace:smpte170m xfer:709 ycbcr:601] > -> "rockchip_cif":0 [ENABLED] > >> >> Cheers, >> >> Paul >> >>> Best regards, >>> Michael >>> >>>> >>>> Cheers, >>>> >>>> Paul >>>> >>>>> @Mehdi: If you could Cc: me when you send out v9 it'd be much appreciated. > > Of course I will :) Cool, thanks! Looking forward to it! Best regards, Michael > > -- > Kind Regards > Mehdi Djait ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2023-10-25 13:12 UTC | newest] Thread overview: 19+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-10-16 9:00 [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's camera interface Mehdi Djait 2023-10-16 9:00 ` [PATCH v8 1/3] media: dt-bindings: media: add bindings for Rockchip VIP Mehdi Djait 2023-10-16 9:00 ` [PATCH v8 2/3] media: rockchip: Add a driver for Rockhip's camera interface Mehdi Djait 2023-10-19 15:40 ` Paul Kocialkowski 2023-10-20 15:38 ` Paul Kocialkowski 2023-10-23 13:28 ` Michael Riesch 2023-10-25 8:49 ` Paul Kocialkowski 2023-10-25 9:38 ` Michael Riesch 2023-10-25 9:48 ` Paul Kocialkowski 2023-10-25 10:28 ` Mehdi Djait 2023-10-16 9:00 ` [PATCH v8 3/3] arm64: dts: rockchip: Add the " Mehdi Djait 2023-10-20 14:10 ` Paul Kocialkowski 2023-10-19 15:33 ` [PATCH v8 0/3] media: rockchip: Add a driver for Rockchip's " Paul Kocialkowski 2023-10-23 13:07 ` Michael Riesch 2023-10-25 8:43 ` Paul Kocialkowski 2023-10-25 9:17 ` Michael Riesch 2023-10-25 9:54 ` Paul Kocialkowski 2023-10-25 10:33 ` Mehdi Djait 2023-10-25 13:12 ` Michael Riesch
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).