[PATCH v5 00/14] Add ISP 3A for StarFive

[PATCH v5 00/14] Add ISP 3A for StarFive

[Changhuang Liang]

This series implements the ISP 3A function to the Camera Subsystem on StarFive
JH7110 SoC. The series has been tested on the VisionFive 2 board.

This series is based on top of the master branch of media_stage repository,
which is tested with a v4l2-compliance compiled from the git repo
(git://linuxtv.org/v4l-utils.git).

changes since v4:
- Rebase on top of the master branch of media_stage repository.
- Insert a new PATCH 13
- PATCH 1:
1. Fix spelling error:
	- corsstakl -> crosstalk
	- balck -> black
	- magbitude -> magnitude
	- FALG -> FLAG
2. Exchange of the values of JH7110_ISP_SC_FLAG_AE_AF and JH7110_ISP_SC_FLAG_AWB

- PATCH 8:
1. Synchronous spelling error: FALG -> FLAG
2. Add:

if (*num_planes)
	return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL : 0;

- PATCH 11:
1. Drop read hook.
2. Add:

if (*num_planes)
	return sizes[0] < sizeof(struct jh7110_isp_params_buffer) ? -EINVAL : 0;

- PATCH 12:
1. Add:

ready_buf->vb.vb2_buf.timestamp = ktime_get_ns();
ready_buf->vb.sequence = output->buffers.sequence++;

to fix "v4l2-compliance -s" Failed.

v4: https://lore.kernel.org/all/20240402100011.13480-1-changhuang.liang@starfivetech.com/

changes since v3:
- Rebased on top of the master branch of media_stage repository and patch [1][2].
- Replaced "stf_get_isp_scd_type" with "stf_isp_get_scd_type" in PATCH 8.

v3: https://lore.kernel.org/all/20240205090424.40302-1-changhuang.liang@starfivetech.com/

changes since v2:
- Rebased on top of the master branch of media_stage repository.
- Added helper function "media_entity_remove_links" in PATCH 8 and PATCH 11 to cleanup links.

v2: https://lore.kernel.org/all/20240111084120.16685-1-changhuang.liang@starfivetech.com/

changes since v1:
- Rebased on top of the master branch of media_stage repository.
- Documentated ISP submodule.
- Added new ISP parameters format and document it.
- Added a meta output video device and link to ISP subdev.
- Replaced control with meta output video buffer for write ISP parameters.
- Separated buffer operation which can use for capture and output device.
- Droped ISP frame sync event.

v1: https://lore.kernel.org/all/20231214065027.28564-1-changhuang.liang@starfivetech.com/

The following are the media graph for the device and the v4l2-compliance
output.

==========================================================================================

[the media graph]:

digraph board {
	rankdir=TB
	n00000001 [label="{{<port0> 0 | <port1> 1} | stf_isp\n/dev/v4l-subdev0 | {<port2> 2 | <port3> 3}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000001:port2 -> n0000000e
	n00000001:port3 -> n00000012 [style=dashed]
	n00000006 [label="output_params\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
	n00000006 -> n00000001:port1 [style=dashed]
	n0000000a [label="capture_raw\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
	n0000000e [label="capture_yuv\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
	n00000012 [label="capture_scd\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
	n0000001c [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n/dev/v4l-subdev1 | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
	n0000001c:port1 -> n00000001:port0 [style=dashed]
	n0000001c:port1 -> n0000000a [style=dashed]
	n00000026 [label="{{} | imx219 6-0010\n/dev/v4l-subdev2 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000026:port0 -> n0000001c:port0 [style=bold]
}

[v4l2-compliance test]:

# ./v4l2-compliance -s -d /dev/video0
v4l2-compliance 1.27.0-5220, 64 bits, 64-bit time_t
v4l2-compliance SHA: 8387e3673837 2024-07-01 11:09:32

Compliance test for starfive-camss device /dev/video0:

Driver Info:
        Driver name      : starfive-camss
        Card type        : Starfive Camera Subsystem
        Bus info         : platform:19840000.isp
        Driver version   : 6.10.0
        Capabilities     : 0xac200000
                Metadata Output
                I/O MC
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x2c200000
                Metadata Output
                I/O MC
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : starfive-camss
        Model            : Starfive Camera Subsystem
        Serial           :
        Bus info         : platform:19840000.isp
        Media version    : 6.10.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.10.0
Interface Info:
        ID               : 0x03000008
        Type             : V4L Video
Entity Info:
        ID               : 0x00000006 (6)
        Name             : output_params
        Function         : V4L2 I/O
        Pad 0x01000007   : 0: Source
          Link 0x0200001a: to remote pad 0x1000003 of entity 'stf_isp' (Image Signal Processor): Data

Required ioctls:
        test MC information (see 'Media Driver Info' above): OK
        test VIDIOC_QUERYCAP: OK
        test invalid ioctls: OK

Allow for multiple opens:
        test second /dev/video0 open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 1 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Output 0):
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
        test VIDIOC_QUERYCTRL: OK (Not Supported)
        test VIDIOC_G/S_CTRL: OK (Not Supported)
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 0 Private Controls: 0

Format ioctls (Output 0):
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
        test VIDIOC_G/S_PARM: OK (Not Supported)
        test VIDIOC_G_FBUF: OK (Not Supported)
        test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
        test Cropping: OK (Not Supported)
        test Composing: OK (Not Supported)
        test Scaling: OK (Not Supported)

Codec ioctls (Output 0):
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Output 0):
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test CREATE_BUFS maximum buffers: OK
        test VIDIOC_REMOVE_BUFS: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)

Test output 0:

Streaming ioctls:
        test read/write: OK (Not Supported)
        test blocking wait: OK
        test MMAP (no poll): OK
        test MMAP (select): OK
        test MMAP (epoll): OK
        test USERPTR (no poll): OK (Not Supported)
        test USERPTR (select): OK (Not Supported)
        test DMABUF: Cannot test, specify --expbuf-device

Total for starfive-camss device /dev/video0: 55, Succeeded: 55, Failed: 0, Warnings: 0

# ./v4l2-compliance -s -d /dev/video3
v4l2-compliance 1.27.0-5220, 64 bits, 64-bit time_t
v4l2-compliance SHA: 8387e3673837 2024-07-01 11:09:32

Compliance test for starfive-camss device /dev/video3:

Driver Info:
        Driver name      : starfive-camss
        Card type        : Starfive Camera Subsystem
        Bus info         : platform:19840000.isp
        Driver version   : 6.10.0
        Capabilities     : 0xa4a00000
                Metadata Capture
                I/O MC
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24a00000
                Metadata Capture
                I/O MC
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : starfive-camss
        Model            : Starfive Camera Subsystem
        Serial           :
        Bus info         : platform:19840000.isp
        Media version    : 6.10.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 6.10.0
Interface Info:
        ID               : 0x03000014
        Type             : V4L Video
Entity Info:
        ID               : 0x00000012 (18)
        Name             : capture_scd
        Function         : V4L2 I/O
        Pad 0x01000013   : 0: Sink
          Link 0x02000018: from remote pad 0x1000005 of entity 'stf_isp' (Image Signal Processor): Data

Required ioctls:
        test MC information (see 'Media Driver Info' above): OK
        test VIDIOC_QUERYCAP: OK
        test invalid ioctls: OK

Allow for multiple opens:
        test second /dev/video3 open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
        test VIDIOC_QUERYCTRL: OK (Not Supported)
        test VIDIOC_G/S_CTRL: OK (Not Supported)
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 0 Private Controls: 0

Format ioctls (Input 0):
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
        test VIDIOC_G/S_PARM: OK (Not Supported)
        test VIDIOC_G_FBUF: OK (Not Supported)
        test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
        test Cropping: OK (Not Supported)
        test Composing: OK (Not Supported)
        test Scaling: OK (Not Supported)

Codec ioctls (Input 0):
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test CREATE_BUFS maximum buffers: OK
        test VIDIOC_REMOVE_BUFS: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
        test read/write: OK (Not Supported)
        test blocking wait: OK
        test MMAP (no poll): OK
        test MMAP (select): OK
        test MMAP (epoll): OK
        test USERPTR (no poll): OK (Not Supported)
        test USERPTR (select): OK (Not Supported)
        test DMABUF: Cannot test, specify --expbuf-device

Total for starfive-camss device /dev/video3: 55, Succeeded: 55, Failed: 0, Warnings: 0

Changhuang Liang (14):
  media: starfive: Add JH7110 ISP module definitions
  media: Documentation: Add description for StarFive ISP metadata
    formats
  media: videodev2.h, v4l2-ioctl: Add StarFive ISP meta buffer format
  staging: media: starfive: Add a params sink pad and a scd source pad
    for ISP
  staging: media: starfive: Separate buffer from ISP hardware operation
  staging: media: starfive: Separate buffer be a common file
  staging: media: starfive: Separate ISP hardware from capture device
  staging: media: starfive: Add for StarFive ISP 3A SC
  staging: media: starfive: Update ISP initialise config for 3A
  staging: media: starfive: Add V4L2_CAP_IO_MC capability
  staging: media: starfive: Add ISP params video device
  staging: media: starfive: Add ISP parameters hardware configure
  staging: media: starfive: Drop read support for video capture devices
  admin-guide: media: Update documents for StarFive Camera Subsystem

 .../admin-guide/media/starfive_camss.rst      |  11 +-
 .../media/starfive_camss_graph.dot            |  22 +-
 .../media/v4l/metafmt-starfive-isp.rst        |  75 ++
 MAINTAINERS                                   |   2 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 drivers/staging/media/starfive/camss/Makefile |   3 +
 .../staging/media/starfive/camss/stf-buffer.c | 166 ++++
 .../staging/media/starfive/camss/stf-buffer.h |  53 ++
 .../staging/media/starfive/camss/stf-camss.c  |  38 +-
 .../staging/media/starfive/camss/stf-camss.h  |  12 +
 .../media/starfive/camss/stf-capture.c        | 248 +-----
 .../media/starfive/camss/stf-capture.h        |  22 -
 .../media/starfive/camss/stf-isp-hw-ops.c     | 722 +++++++++++++++++
 .../media/starfive/camss/stf-isp-params.c     | 240 ++++++
 .../staging/media/starfive/camss/stf-isp.c    |  77 +-
 .../staging/media/starfive/camss/stf-isp.h    | 188 +++++
 .../staging/media/starfive/camss/stf-output.c |  83 ++
 .../staging/media/starfive/camss/stf-output.h |  22 +
 .../staging/media/starfive/camss/stf-video.c  | 147 +++-
 .../staging/media/starfive/camss/stf-video.h  |  11 +-
 include/uapi/linux/jh7110-isp.h               | 739 ++++++++++++++++++
 include/uapi/linux/videodev2.h                |   4 +
 22 files changed, 2608 insertions(+), 279 deletions(-)
 create mode 100644 Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
 create mode 100644 drivers/staging/media/starfive/camss/stf-buffer.c
 create mode 100644 drivers/staging/media/starfive/camss/stf-buffer.h
 create mode 100644 drivers/staging/media/starfive/camss/stf-isp-params.c
 create mode 100644 drivers/staging/media/starfive/camss/stf-output.c
 create mode 100644 drivers/staging/media/starfive/camss/stf-output.h
 create mode 100644 include/uapi/linux/jh7110-isp.h

--
2.25.1

[PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Changhuang Liang]

Add JH7110 ISP module definitions for user space.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
---
 MAINTAINERS                     |   1 +
 include/uapi/linux/jh7110-isp.h | 739 ++++++++++++++++++++++++++++++++
 2 files changed, 740 insertions(+)
 create mode 100644 include/uapi/linux/jh7110-isp.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 507f04a80499..890604eb0d64 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -21305,6 +21305,7 @@ S:	Maintained
 F:	Documentation/admin-guide/media/starfive_camss.rst
 F:	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
 F:	drivers/staging/media/starfive/camss
+F:	include/uapi/linux/jh7110-isp.h
 
 STARFIVE CRYPTO DRIVER
 M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
diff --git a/include/uapi/linux/jh7110-isp.h b/include/uapi/linux/jh7110-isp.h
new file mode 100644
index 000000000000..4939cd63e771
--- /dev/null
+++ b/include/uapi/linux/jh7110-isp.h
@@ -0,0 +1,739 @@
+/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
+/*
+ * jh7110-isp.h
+ *
+ * JH7110 ISP driver - user space header file.
+ *
+ * Copyright © 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Zejian Su <zejian.su@starfivetech.com>
+ *
+ */
+
+#ifndef __JH7110_ISP_H_
+#define __JH7110_ISP_H_
+
+/**
+ * ISP Module Diagram
+ * ------------------
+ *
+ *  Raw  +-----+    +------+    +------+    +----+
+ *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
+ *       +-----+    +------+    +------+    +----+     |
+ *                                                     |
+ *  +--------------------------------------------------+
+ *  |
+ *  |    +-----+    +-----+    +-----+    +-----+
+ *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
+ *       +-----+    +-----+    +-----+    +-----+      |
+ *                                                     |
+ *  +--------------------------------------------------+
+ *  |
+ *  |    +-----+    +--------+    +-----+    +------+
+ *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
+ *       +-----+    +--------+    +-----+    +------+  |
+ *                                                     |
+ *  +--------------------------------------------------+
+ *  |
+ *  |    +-------+    +-------+    +-----+    +----+
+ *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
+ *       +-------+    +-------+    +-----+    +----+
+ *
+ */
+
+/* Auto White Balance */
+#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
+/* Color Artifact Removal */
+#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
+/* Color Correction Matrix */
+#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
+/* Color Filter Arrays */
+#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
+/* Crosstalk Correction */
+#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
+/* Defect Bad Pixel Correction */
+#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
+/* Denoise YUV */
+#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
+/* RGB Gamma */
+#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
+/* Lens Correction Cosine Fourth */
+#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
+/* Optical Black Correction */
+#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
+/* Opto-Electric Conversion Function */
+#define JH7110_ISP_MODULE_OECF_SETTING			(1U << 10)
+/* RGB To YUV */
+#define JH7110_ISP_MODULE_R2Y_SETTING			(1U << 11)
+/* Saturation */
+#define JH7110_ISP_MODULE_SAT_SETTING			(1U << 12)
+/* Sharpen */
+#define JH7110_ISP_MODULE_SHARP_SETTING			(1U << 13)
+/* Y Curve */
+#define JH7110_ISP_MODULE_YCRV_SETTING			(1U << 14)
+/* Statistics Collection */
+#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)
+
+/**
+ * struct jh7110_isp_wb_gain - auto white balance gain
+ *
+ * @gain_r: gain value for red component.
+ * @gain_g: gain value for green component.
+ * @gain_b: gain value for blue component.
+ */
+struct jh7110_isp_wb_gain {
+	__u16 gain_r;
+	__u16 gain_g;
+	__u16 gain_b;
+};
+
+/**
+ * struct jh7110_isp_wb_setting - Configuration used by auto white balance gain
+ *
+ * @enabled: enabled setting flag.
+ * @gains: auto white balance gain setting.
+ */
+struct jh7110_isp_wb_setting {
+	__u32 enabled;
+	struct jh7110_isp_wb_gain gains;
+};
+
+/**
+ * struct jh7110_isp_car_setting - Configuration used by color artifact removal
+ *
+ * @enabled: enabled setting flag.
+ */
+struct jh7110_isp_car_setting {
+	__u32 enabled;
+};
+
+/**
+ * struct jh7110_isp_ccm_smlow - Color correction matrix
+ *
+ * @ccm: color transform matrix with size 3 by 3.
+ * @offsets: the offsets of R, G, B after the transform by the ccm.
+ */
+struct jh7110_isp_ccm_smlow {
+	__s32 ccm[3][3];
+	__s32 offsets[3];
+};
+
+/**
+ * struct jh7110_isp_ccm_setting - Configuration used by color correction matrix
+ *
+ * @enabled: enabled setting flag.
+ * @ccm_smlow: Color correction matrix.
+ */
+struct jh7110_isp_ccm_setting {
+	__u32 enabled;
+	struct jh7110_isp_ccm_smlow ccm_smlow;
+};
+
+/**
+ * struct jh7110_isp_cfa_params - demosaic parameters
+ *
+ * @hv_width: detail smooth factor
+ * @cross_cov: Cross covariance weighting.
+ */
+struct jh7110_isp_cfa_params {
+	__s32 hv_width;
+	__s32 cross_cov;
+};
+
+/**
+ * struct jh7110_isp_cfa_params - Configuration used by demosaic module
+ *
+ * @enabled: enabled setting flag.
+ * @settings: demosaic parameters.
+ */
+struct jh7110_isp_cfa_setting {
+	__u32 enabled;
+	struct jh7110_isp_cfa_params settings;
+};
+
+/**
+ * struct jh7110_isp_ctc_params - crosstalk remove parameters
+ *
+ * @saf_mode: smooth area filter mode.
+ * @daf_mode: detail area filter mode.
+ * @max_gt: the threshold for imbalance detection when pixel intensity is close to maximum.
+ * @min_gt: the threshold for imbalance detection when pixel intensity is close to 0.
+ */
+struct jh7110_isp_ctc_params {
+	__u8 saf_mode;
+	__u8 daf_mode;
+	__s32 max_gt;
+	__s32 min_gt;
+};
+
+/**
+ * struct jh7110_isp_ctc_params - Configuration used by crosstalk remove
+ *
+ * @enabled: enabled setting flag.
+ * @settings: corsstalk remove parameters.
+ */
+struct jh7110_isp_ctc_setting {
+	__u32 enabled;
+	struct jh7110_isp_ctc_params settings;
+};
+
+/**
+ * struct jh7110_isp_dbc_params - defect pixels correction parameters
+ *
+ * @bad_gt: bad pixel threshold for the green channel.
+ * @bad_xt: bad pixel threshold for the red and blue channels.
+ */
+struct jh7110_isp_dbc_params {
+	__s32 bad_gt;
+	__s32 bad_xt;
+};
+
+/**
+ * struct jh7110_isp_dbc_params - Configuration used by defect bad pixels correction
+ *
+ * @enabled: enabled setting flag.
+ * @settings: defect pixels correction parameters.
+ */
+struct jh7110_isp_dbc_setting {
+	__u32 enabled;
+	struct jh7110_isp_dbc_params settings;
+};
+
+/**
+ * struct jh7110_isp_dnyuv_params - yuv domain denoise parameters
+ *
+ * @y_sweight: ten coefficients of 7x7 spatial filter for Y channel.
+ * @y_curve: intensity difference (similarity) weight lookup table for Y channel.
+ * @uv_sweight: ten coefficients of 7x7 spatial filter for U and V channel.
+ * @uv_curve: intensity difference (similarity) weight lookup table for U and V channel.
+ */
+struct jh7110_isp_dnyuv_params {
+	__u8 y_sweight[10];
+	__u16 y_curve[7];
+	__u8 uv_sweight[10];
+	__u16 uv_curve[7];
+};
+
+/**
+ * struct jh7110_isp_dnyuv_params - Configuration used by yuv domain denoise
+ *
+ * @enabled: enabled setting flag.
+ * @settings: yuv domain denoise parameters.
+ */
+struct jh7110_isp_dnyuv_setting {
+	__u32 enabled;
+	struct jh7110_isp_dnyuv_params settings;
+};
+
+/**
+ * struct jh7110_isp_gmargb_point - RGB Gamma point
+ *
+ * @g_val: RGB gamma value.
+ * @sg_val: RGB gamma slope value.
+ */
+struct jh7110_isp_gmargb_point {
+	__u16 g_val;
+	__u16 sg_val;
+};
+
+/**
+ * struct jh7110_isp_gmargb_setting - Configuration used by RGB gamma
+ *
+ * @enabled: enabled setting flag.
+ * @curve: RGB Gamma point table.
+ */
+struct jh7110_isp_gmargb_setting {
+	__u32 enabled;
+	struct jh7110_isp_gmargb_point curve[15];
+};
+
+/**
+ * struct jh7110_isp_lccf_circle - len circle
+ *
+ * @center_x: center X distance from capture window.
+ * @center_y: center Y distance from capture window.
+ * @radius: len circle radius.
+ */
+struct jh7110_isp_lccf_circle {
+	__s16 center_x;
+	__s16 center_y;
+	__u8 radius;
+};
+
+/**
+ * struct jh7110_isp_lccf_curve_param - lens correction cosine fourth curve param
+ *
+ * @f1: F1 parameter.
+ * @f2: F2 parameter.
+ */
+struct jh7110_isp_lccf_curve_param {
+	__s16 f1;
+	__s16 f2;
+};
+
+/**
+ * struct jh7110_isp_lccf_setting - Configuration used by lens correction cosine fourth
+ *
+ * @enabled: enabled setting flag.
+ * @circle: len circle.
+ * @r_param: lens correction cosine fourth curve param for Bayer pattern R.
+ * @gr_param: lens correction cosine fourth curve param for Bayer pattern Gr.
+ * @gb_param: lens correction cosine fourth curve param for Bayer pattern Gb.
+ * @b_param: lens correction cosine fourth curve param for Bayer pattern B.
+ */
+struct jh7110_isp_lccf_setting {
+	__u32 enabled;
+	struct jh7110_isp_lccf_circle circle;
+	struct jh7110_isp_lccf_curve_param r_param;
+	struct jh7110_isp_lccf_curve_param gr_param;
+	struct jh7110_isp_lccf_curve_param gb_param;
+	struct jh7110_isp_lccf_curve_param b_param;
+};
+
+/**
+ * struct jh7110_isp_obc_win_size - optical black correction window size
+ *
+ * @width: window width.
+ * @height: window height.
+ */
+struct jh7110_isp_obc_win_size {
+	__u32 width;
+	__u32 height;
+};
+
+/**
+ * struct jh7110_isp_obc_gain - optical black correction symbol gain
+ *
+ * @tl_gain: gain at point A for symbol.
+ * @tr_gain: gain at point B for symbol.
+ * @bl_gain: gain at point C for symbol.
+ * @br_gain: gain at point D for symbol.
+ */
+struct jh7110_isp_obc_gain {
+	__u8 tl_gain;
+	__u8 tr_gain;
+	__u8 bl_gain;
+	__u8 br_gain;
+};
+
+/**
+ * struct jh7110_isp_obc_offset - optical black correction symbol offset
+ *
+ * @tl_offset: offset at point A for symbol.
+ * @tr_offset: offset at point B for symbol.
+ * @bl_offset: offset at point C for symbol.
+ * @br_offset: offset at point D for symbol.
+ */
+struct jh7110_isp_obc_offset {
+	__u8 tl_offset;
+	__u8 tr_offset;
+	__u8 bl_offset;
+	__u8 br_offset;
+};
+
+/**
+ * struct jh7110_isp_obc_setting - Configuration used by optical black correction
+ *
+ * @enabled: enabled setting flag.
+ * @win_size: optical black correction window size.
+ * @gain: optical black correction symbol gain.
+ * @offset: optical black correction symbol offset.
+ */
+struct jh7110_isp_obc_setting {
+	__u32 enabled;
+	struct jh7110_isp_obc_win_size win_size;
+	struct jh7110_isp_obc_gain gain[4];
+	struct jh7110_isp_obc_offset offset[4];
+};
+
+/**
+ * struct jh7110_isp_oecf_point - oecf curve
+ *
+ * @x: x coordinate.
+ * @y: y coordinate.
+ * @slope: the slope between this point and the next point.
+ */
+struct jh7110_isp_oecf_point {
+	__u16 x;
+	__u16 y;
+	__s16 slope;
+};
+
+/**
+ * struct jh7110_isp_oecf_setting - Configuration used by opto-electric conversion function
+ *
+ * @enabled: enabled setting flag.
+ * @r_curve: red pixel oecf curve.
+ * @gr_curve: green pixel oecf curve in GR line.
+ * @gb_curve: green pixel oecf curve in GB line.
+ * @b_curve: blue pixel oecf curve.
+ */
+struct jh7110_isp_oecf_setting {
+	__u32 enabled;
+	struct jh7110_isp_oecf_point r_curve[16];
+	struct jh7110_isp_oecf_point gr_curve[16];
+	struct jh7110_isp_oecf_point gb_curve[16];
+	struct jh7110_isp_oecf_point b_curve[16];
+};
+
+/**
+ * struct jh7110_isp_r2y_matrix - RGB to YUV color conversion matrix
+ *
+ * @m: The 3x3 color conversion matrix coefficient.
+ */
+struct jh7110_isp_r2y_matrix {
+	__s16 m[9];
+};
+
+/**
+ * struct jh7110_isp_r2y_setting - Configuration used by RGB To YUV
+ *
+ * @enabled: enabled setting flag.
+ * @matrix: RGB to YUV color conversion matrix.
+ */
+struct jh7110_isp_r2y_setting {
+	__u32 enabled;
+	struct jh7110_isp_r2y_matrix matrix;
+};
+
+/**
+ * struct jh7110_isp_sat_curve - Saturation curve
+ *
+ * @yi_min: the minimum input Y value.
+ * @yo_ir: the ratio of Y output range to input range.
+ * @yo_min: the minimum output Y value.
+ * @yo_max: the maximum output Y value.
+ */
+struct jh7110_isp_sat_curve {
+	__s16 yi_min;
+	__s16 yo_ir;
+	__s16 yo_min;
+	__s16 yo_max;
+};
+
+/**
+ * struct jh7110_isp_sat_hue_info - Chroma Saturation Hue Factor
+ *
+ * @cos: COS hue factor.
+ * @sin: SIN hue factor.
+ */
+struct jh7110_isp_sat_hue_info {
+	__s16 cos;
+	__s16 sin;
+};
+
+/**
+ * struct jh7110_isp_sat_info - Saturation information
+ *
+ * @gain_cmab: Chroma saturation magnitude amplification base for gain.
+ * @gain_cmmd: Chroma saturation magnitude amplification delta for gain.
+ * @threshold_cmb: Chroma saturation magnitude base threshold.
+ * @threshold_cmd: Chroma saturation magnitude delta threshold.
+ * @offset_u: Chroma saturation U offset.
+ * @offset_v: Chroma saturation V offset.
+ * @cmsf: Chroma saturation magnitude scaling factor.
+ */
+struct jh7110_isp_sat_info {
+	__s16 gain_cmab;
+	__s16 gain_cmmd;
+	__s16 threshold_cmb;
+	__s16 threshold_cmd;
+	__s16 offset_u;
+	__s16 offset_v;
+	__s16 cmsf;
+};
+
+/**
+ * struct jh7110_isp_sat_setting - Configuration used by Saturation
+ *
+ * @enabled: enabled setting flag.
+ * @curve: Saturation curve.
+ * @hue_info: Chroma Saturation Hue Factor.
+ * @sat_info: Saturation information.s
+ */
+struct jh7110_isp_sat_setting {
+	__u32 enabled;
+	struct jh7110_isp_sat_curve curve;
+	struct jh7110_isp_sat_hue_info hue_info;
+	struct jh7110_isp_sat_info sat_info;
+};
+
+/**
+ * struct jh7110_isp_sharp_weight - Sharpe weight
+ *
+ * @weight: Sharpen filter weight.
+ * @recip_wei_sum: Sharpen amplification filter weight normalization factor.
+ */
+struct jh7110_isp_sharp_weight {
+	__u8 weight[15];
+	__u32 recip_wei_sum;
+};
+
+/**
+ * struct jh7110_isp_sharp_strength - Sharpen strength
+ *
+ * @diff: Sharpen Edge amplification delta level.
+ * @f: Sharpen Edge amplification factor.
+ * @s: Sharpen Edge amplification factor slope.
+ */
+struct jh7110_isp_sharp_strength {
+	__s16 diff[4];
+	__s16 f[3];
+	__s32 s[3];
+};
+
+/**
+ * struct jh7110_isp_sharp_setting - Configuration used by Sharpen
+ *
+ * @enabled: enabled setting flag.
+ * @weight: Sharpe weight.
+ * @strength: Sharpen strength.
+ * @pdirf: Positive Factor Multiplier.
+ * @ndirf: Negative Factor Multiplier.
+ */
+struct jh7110_isp_sharp_setting {
+	__u32 enabled;
+	struct jh7110_isp_sharp_weight weight;
+	struct jh7110_isp_sharp_strength strength;
+	__s8 pdirf;
+	__s8 ndirf;
+};
+
+/**
+ * struct jh7110_isp_ycrv_curve - Y Curve parameters table
+ *
+ * @y: Y curve L parameters value.
+ */
+struct jh7110_isp_ycrv_curve {
+	__s16 y[64];
+};
+
+/**
+ * struct jh7110_isp_ycrv_setting - Configuration used by Y Curve
+ *
+ * @enabled: enabled setting flag.
+ * @curve: Y Curve parameters table.
+ */
+struct jh7110_isp_ycrv_setting {
+	__u32 enabled;
+	struct jh7110_isp_ycrv_curve curve;
+};
+
+/**
+ * struct jh7110_isp_sc_config - statistics collection crop configure
+ *
+ * @h_start: Horizontal starting point for frame cropping.
+ * @v_start: Vertical starting point for frame cropping.
+ * @sw_width: Width of statistics collection sub-window.
+ * @sw_height: Height of statistics collection sub-window.
+ * @hperiod: Horizontal period.
+ * @hkeep: Horizontal keep.
+ * @vperiod: Vertical period.
+ * @vkeep: Vertical keep.
+ */
+struct jh7110_isp_sc_config {
+	__u16 h_start;
+	__u16 v_start;
+	__u8 sw_width;
+	__u8 sw_height;
+	__u8 hperiod;
+	__u8 hkeep;
+	__u8 vperiod;
+	__u8 vkeep;
+};
+
+/**
+ * struct jh7110_isp_sc_af_config - statistics collection auto focus configure
+ *
+ * @es_hor_mode: Horizontal mode.
+ * @es_sum_mode: sum mode.
+ * @hor_en: Horizontal enable.
+ * @ver_en: Vertical enable.
+ * @es_ver_thr: Vertical threshold.
+ * @es_hor_thr: Horizontal threshold.
+ */
+struct jh7110_isp_sc_af_config {
+	__u8 es_hor_mode;
+	__u8 es_sum_mode;
+	__u8 hor_en;
+	__u8 ver_en;
+	__u8 es_ver_thr;
+	__u16 es_hor_thr;
+};
+
+/**
+ * struct jh7110_isp_sc_awb_ps - statistics collection auto white balance pixel sum
+ *
+ * @awb_ps_rl: Lower boundary of R value for pixel sum.
+ * @awb_ps_ru: Upper boundary of R value for pixel sum.
+ * @awb_ps_gl: Lower boundary of G value for pixel sum.
+ * @awb_ps_gu: Upper boundary of G value for pixel sum.
+ * @awb_ps_bl: Lower boundary of B value for pixel sum.
+ * @awb_ps_bu: Upper boundary of B value for pixel sum.
+ * @awb_ps_yl: Lower boundary of Y value for pixel sum.
+ * @awb_ps_yu: Upper boundary of Y value for pixel sum.
+ * @awb_ps_grl: Lower boundary of G/R ratio for pixel sum.
+ * @awb_ps_gru: Upper boundary of G/R ratio for pixel sum.
+ * @awb_ps_gbl: Lower boundary of G/B ratio for pixel sum.
+ * @awb_ps_gbu: Upper boundary of G/B ratio for pixel sum.
+ * @awb_ps_grbl: Lower boundary of (Gr/R + b/a * Gb/B) for pixel sum.
+ * @awb_ps_grbu: Upper boundary of (Gr/R + b/a * Gb/B) for pixel sum.
+ */
+struct jh7110_isp_sc_awb_ps {
+	__u8 awb_ps_rl;
+	__u8 awb_ps_ru;
+	__u8 awb_ps_gl;
+	__u8 awb_ps_gu;
+	__u8 awb_ps_bl;
+	__u8 awb_ps_bu;
+	__u8 awb_ps_yl;
+	__u8 awb_ps_yu;
+	__u16 awb_ps_grl;
+	__u16 awb_ps_gru;
+	__u16 awb_ps_gbl;
+	__u16 awb_ps_gbu;
+	__u16 awb_ps_grbl;
+	__u16 awb_ps_grbu;
+};
+
+/**
+ * struct jh7110_isp_sc_awb_ws - statistics collection auto white balance weight sum
+ *
+ * @awb_ws_rl: Lower boundary of R value for weight sum.
+ * @awb_ws_ru: Upper boundary of R value for weight sum.
+ * @awb_ws_grl: Lower boundary of Gr value for weight sum.
+ * @awb_ws_gru: Upper boundary of Gr value for weight sum.
+ * @awb_ws_gbl: Lower boundary of Gb value for weight sum.
+ * @awb_ws_gbu: Upper boundary of Gb value for weight sum.
+ * @awb_ws_bl: Lower boundary of B value for weight sum.
+ * @awb_ws_bu: Upper boundary of B value for weight sum.
+ */
+struct jh7110_isp_sc_awb_ws {
+	__u8 awb_ws_rl;
+	__u8 awb_ws_ru;
+	__u8 awb_ws_grl;
+	__u8 awb_ws_gru;
+	__u8 awb_ws_gbl;
+	__u8 awb_ws_gbu;
+	__u8 awb_ws_bl;
+	__u8 awb_ws_bu;
+};
+
+/**
+ * struct jh7110_isp_sc_awb_point - statistics collection auto white balance point
+ *
+ * @weight: Weighting value at point.
+ */
+struct jh7110_isp_sc_awb_point {
+	__u8 weight;
+};
+
+/**
+ * struct jh7110_isp_sc_awb_config - statistics collection auto white balance configure
+ *
+ * @ps_config: statistics collection auto white balance pixel sum.
+ * @awb_ps_grb_ba: auto white balance b/a value.
+ * @sel: input mux for statistics collection auto white balance.
+ * @ws_config: statistics collection auto white balance weight sum.
+ * @awb_cw: Weighting value at 13x13 point.
+ * @pts: statistics collection auto white balance point.
+ */
+struct jh7110_isp_sc_awb_config {
+	struct jh7110_isp_sc_awb_ps ps_config;
+	__u8 awb_ps_grb_ba;
+	__u8 sel;
+	struct jh7110_isp_sc_awb_ws ws_config;
+	__u8 awb_cw[169];
+	struct jh7110_isp_sc_awb_point pts[17];
+};
+
+/**
+ * struct jh7110_isp_sc_setting - Configuration used by statistics collection
+ *
+ * @enabled: enabled setting flag.
+ * @crop_config: statistics collection crop configure.
+ * @af_config: statistics collection auto focus configure.
+ * @awb_config: statistics collection auto white balance configure.
+ */
+struct jh7110_isp_sc_setting {
+	__u32 enabled;
+	struct jh7110_isp_sc_config crop_config;
+	struct jh7110_isp_sc_af_config af_config;
+	struct jh7110_isp_sc_awb_config awb_config;
+};
+
+/**
+ * struct jh7110_isp_params_buffer - StarFive JH7110 ISP Parameters Meta Data
+ *
+ * @enable_setting: enabled setting module (JH7110_ISP_MODULE_* definitions).
+ * @wb_setting: Configuration used by auto white balance gain.
+ * @car_setting: Configuration used by color artifact removal.
+ * @ccm_setting: Configuration used by color correction matrix.
+ * @cfa_setting: Configuration used by demosaic module.
+ * @ctc_setting: Configuration used by crosstalk remove.
+ * @dbc_setting: Configuration used by defect bad pixels correction.
+ * @dnyuv_setting: Configuration used by yuv domain denoise.
+ * @gmargb_setting: Configuration used by RGB gamma.
+ * @lccf_setting: Configuration used by lens correction cosine fourth.
+ * @obc_setting: Configuration used by optical black compensation.
+ * @oecf_setting: Configuration used by opto-electric conversion function.
+ * @r2y_setting: Configuration used by RGB To YUV.
+ * @sat_setting: Configuration used by Saturation.
+ * @sharp_setting: Configuration used by Sharpen.
+ * @ycrv_setting: Configuration used by Y Curve.
+ * @sc_setting: Configuration used by statistics collection.
+ */
+struct jh7110_isp_params_buffer {
+	__u32 enable_setting;
+	struct jh7110_isp_wb_setting wb_setting;
+	struct jh7110_isp_car_setting car_setting;
+	struct jh7110_isp_ccm_setting ccm_setting;
+	struct jh7110_isp_cfa_setting cfa_setting;
+	struct jh7110_isp_ctc_setting ctc_setting;
+	struct jh7110_isp_dbc_setting dbc_setting;
+	struct jh7110_isp_dnyuv_setting dnyuv_setting;
+	struct jh7110_isp_gmargb_setting gmargb_setting;
+	struct jh7110_isp_lccf_setting lccf_setting;
+	struct jh7110_isp_obc_setting obc_setting;
+	struct jh7110_isp_oecf_setting oecf_setting;
+	struct jh7110_isp_r2y_setting r2y_setting;
+	struct jh7110_isp_sat_setting sat_setting;
+	struct jh7110_isp_sharp_setting sharp_setting;
+	struct jh7110_isp_ycrv_setting ycrv_setting;
+	struct jh7110_isp_sc_setting sc_setting;
+};
+
+/**
+ * Statistics Collection Meta Data Flag
+ */
+#define JH7110_ISP_SC_FLAG_AWB			0x0
+#define JH7110_ISP_SC_FLAG_AE_AF		0xffff
+
+#pragma pack(1)
+
+/**
+ * struct jh7110_isp_sc_buffer - StarFive JH7110 ISP Statistics Collection Meta Data
+ *
+ * @y_histogram: Y histogram data for saturation control.
+ * @reserv0: reserve byte.
+ * @bright_sc: bright statistic. If flag is JH7110_ISP_SC_FLAG_AE_AF, This field is
+ *             saved auto exposure and auto focus. If flag is JH7110_ISP_SC_FLAG_AWB,
+ *             This field is saved auto white balance.
+ * @reserv1: reserve byte.
+ * @ae_hist_y: Y histogram for auto exposure.
+ * @reserv2: reserve byte.
+ * @flag: Statistics Collection Meta Data Flag (JH7110_ISP_SC_FLAG_* definitions)
+ */
+struct jh7110_isp_sc_buffer {
+	__u32 y_histogram[64];
+	__u32 reserv0[33];
+	__u32 bright_sc[4096];
+	__u32 reserv1[96];
+	__u32 ae_hist_y[128];
+	__u32 reserv2[511];
+	__u16 flag;
+};
+
+#pragma pack()
+
+#endif
-- 
2.25.1

[PATCH v5 02/14] media: Documentation: Add description for StarFive ISP metadata formats

[Changhuang Liang]

Add description for V4L2_META_FMT_STF_ISP_PARAMS and
V4L2_META_FMT_STF_ISP_STAT_3A meta data formats.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../media/v4l/metafmt-starfive-isp.rst        | 75 +++++++++++++++++++
 MAINTAINERS                                   |  1 +
 2 files changed, 76 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst

diff --git a/Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst b/Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
new file mode 100644
index 000000000000..ebb4291833d6
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
@@ -0,0 +1,75 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. _v4l2-meta-fmt-stf-isp-params:
+
+.. _v4l2-meta-fmt-stf-isp-stat-3a:
+
+*****************************************************************************
+V4L2_META_FMT_STF_ISP_PARAMS ('stfp'), V4L2_META_FMT_STF_ISP_STAT_3A ('stfs')
+*****************************************************************************
+
+.. jh7110_isp_params_buffer
+
+Configuration parameters
+========================
+
+The configuration parameters are passed to the "output_params" metadata output
+video node, using the :c:type:`v4l2_meta_format` interface. They are formatted
+as described by the :c:type:`jh7110_isp_params_buffer` structure.
+
+.. code-block:: c
+
+	struct jh7110_isp_params_buffer {
+		__u32 enable_setting;
+		struct jh7110_isp_wb_setting wb_setting;
+		struct jh7110_isp_car_setting car_setting;
+		struct jh7110_isp_ccm_setting ccm_setting;
+		struct jh7110_isp_cfa_setting cfa_setting;
+		struct jh7110_isp_ctc_setting ctc_setting;
+		struct jh7110_isp_dbc_setting dbc_setting;
+		struct jh7110_isp_dnyuv_setting dnyuv_setting;
+		struct jh7110_isp_gmargb_setting gmargb_setting;
+		struct jh7110_isp_lccf_setting lccf_setting;
+		struct jh7110_isp_obc_setting obc_setting;
+		struct jh7110_isp_oecf_setting oecf_setting;
+		struct jh7110_isp_r2y_setting r2y_setting;
+		struct jh7110_isp_sat_setting sat_setting;
+		struct jh7110_isp_sharp_setting sharp_setting;
+		struct jh7110_isp_ycrv_setting ycrv_setting;
+		struct jh7110_isp_sc_setting sc_setting;
+	};
+
+.. jh7110_isp_sc_buffer
+
+3A and histogram statistics
+===========================
+
+The ISP device collects different statistics over an input Bayer frame.
+Those statistics are obtained from the "capture_scd" metadata capture video
+node, using the :c:type:`v4l2_meta_format` interface. They are formatted as
+described by the :c:type:`jh7110_isp_sc_buffer` structure.
+
+.. code-block:: c
+
+	struct jh7110_isp_sc_buffer {
+		__u32 y_histogram[64];
+		__u32 reserv0[33];
+		__u32 bright_sc[4096];
+		__u32 reserv1[96];
+		__u32 ae_hist_y[128];
+		__u32 reserv2[511];
+		__u16 flag;
+	};
+
+The statistics collected are Auto Exposure, AWB (Auto-white balance), Histogram
+and AF (Auto-focus). See :c:type:`jh7110_isp_sc_buffer` for details of the
+statistics.
+
+The 3A statistics and configuration parameters described here are usually
+consumed and produced by dedicated user space libraries that comprise the
+important tuning tools using software control loop.
+
+JH7110 ISP uAPI data types
+======================
+
+.. kernel-doc:: include/uapi/linux/jh7110-isp.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 890604eb0d64..8fd613c93e62 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -21304,6 +21304,7 @@ L:	linux-media@vger.kernel.org
 S:	Maintained
 F:	Documentation/admin-guide/media/starfive_camss.rst
 F:	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
+F:	Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
 F:	drivers/staging/media/starfive/camss
 F:	include/uapi/linux/jh7110-isp.h
 
-- 
2.25.1

[PATCH v5 03/14] media: videodev2.h, v4l2-ioctl: Add StarFive ISP meta buffer format

[Changhuang Liang]

Add the StarFive ISP specific metadata format
V4L2_META_FMT_STF_ISP_PARAMS & V4L2_META_FMT_STF_ISP_STAT_3A for 3A.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
 include/uapi/linux/videodev2.h       | 4 ++++
 2 files changed, 6 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 4c76d17b4629..8770bfb31c5c 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1456,6 +1456,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
 	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
 	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
+	case V4L2_META_FMT_STF_ISP_PARAMS:	descr = "StarFive ISP 3A Parameters"; break;
+	case V4L2_META_FMT_STF_ISP_STAT_3A:	descr = "StarFive ISP 3A Statistics"; break;
 	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
 	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
 	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index fe6b67e83751..cfcbfe9bf973 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -841,6 +841,10 @@ struct v4l2_pix_format {
 #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
 #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
 
+/* Vendor specific - used for StarFive JH7110 ISP camera sub-system */
+#define V4L2_META_FMT_STF_ISP_PARAMS	v4l2_fourcc('S', 'T', 'F', 'P') /* StarFive ISP 3A Parameters */
+#define V4L2_META_FMT_STF_ISP_STAT_3A	v4l2_fourcc('S', 'T', 'F', 'S') /* StarFive ISP 3A Statistics */
+
 #ifdef __KERNEL__
 /*
  * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when
-- 
2.25.1

[PATCH v5 04/14] staging: media: starfive: Add a params sink pad and a scd source pad for ISP

[Changhuang Liang]

StarFive ISP can use params sink pad to transmit ISP parameters and use
scd source pad to capture statistics collection data.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../staging/media/starfive/camss/stf-isp.c    | 77 +++++++++++++++++--
 .../staging/media/starfive/camss/stf-isp.h    |  2 +
 2 files changed, 71 insertions(+), 8 deletions(-)

diff --git a/drivers/staging/media/starfive/camss/stf-isp.c b/drivers/staging/media/starfive/camss/stf-isp.c
index 4e6e26736852..0ebffd09842a 100644
--- a/drivers/staging/media/starfive/camss/stf-isp.c
+++ b/drivers/staging/media/starfive/camss/stf-isp.c
@@ -21,13 +21,23 @@ static const struct stf_isp_format isp_formats_sink[] = {
 	{ MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
 };
 
+static const struct stf_isp_format isp_formats_sink_params[] = {
+	{ MEDIA_BUS_FMT_METADATA_FIXED },
+};
+
 static const struct stf_isp_format isp_formats_source[] = {
 	{ MEDIA_BUS_FMT_YUYV8_1_5X8, 8 },
 };
 
+static const struct stf_isp_format isp_formats_source_scd[] = {
+	{ MEDIA_BUS_FMT_METADATA_FIXED },
+};
+
 static const struct stf_isp_format_table isp_formats_st7110[] = {
 	{ isp_formats_sink, ARRAY_SIZE(isp_formats_sink) },
+	{ isp_formats_sink_params, ARRAY_SIZE(isp_formats_sink_params) },
 	{ isp_formats_source, ARRAY_SIZE(isp_formats_source) },
+	{ isp_formats_source_scd, ARRAY_SIZE(isp_formats_source_scd) },
 };
 
 static const struct stf_isp_format *
@@ -93,13 +103,19 @@ static void isp_try_format(struct stf_isp_dev *isp_dev,
 
 	formats = &isp_dev->formats[pad];
 
-	fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
-			     STFCAMSS_FRAME_MAX_WIDTH);
-	fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
-			      STFCAMSS_FRAME_MAX_HEIGHT);
-	fmt->height &= ~0x1;
+	if (pad != STF_ISP_PAD_SRC_SCD && pad != STF_ISP_PAD_SINK_PARAMS) {
+		fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
+				     STFCAMSS_FRAME_MAX_WIDTH);
+		fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
+				      STFCAMSS_FRAME_MAX_HEIGHT);
+		fmt->height &= ~0x1;
+		fmt->colorspace = V4L2_COLORSPACE_SRGB;
+	} else {
+		fmt->width = 1;
+		fmt->height = 1;
+	}
+
 	fmt->field = V4L2_FIELD_NONE;
-	fmt->colorspace = V4L2_COLORSPACE_SRGB;
 	fmt->flags = 0;
 
 	if (!stf_g_fmt_by_mcode(formats, fmt->code))
@@ -119,7 +135,7 @@ static int isp_enum_mbus_code(struct v4l2_subdev *sd,
 
 		formats = &isp_dev->formats[code->pad];
 		code->code = formats->fmts[code->index].code;
-	} else {
+	} else if (code->pad == STF_ISP_PAD_SRC) {
 		struct v4l2_mbus_framefmt *sink_fmt;
 
 		if (code->index >= ARRAY_SIZE(isp_formats_source))
@@ -131,6 +147,10 @@ static int isp_enum_mbus_code(struct v4l2_subdev *sd,
 		code->code = sink_fmt->code;
 		if (!code->code)
 			return -EINVAL;
+	} else {
+		if (code->index > 0)
+			return -EINVAL;
+		code->code = MEDIA_BUS_FMT_METADATA_FIXED;
 	}
 	code->flags = 0;
 
@@ -151,6 +171,9 @@ static int isp_set_format(struct v4l2_subdev *sd,
 	isp_try_format(isp_dev, state, fmt->pad, &fmt->format);
 	*format = fmt->format;
 
+	if (fmt->pad == STF_ISP_PAD_SRC_SCD || fmt->pad == STF_ISP_PAD_SINK_PARAMS)
+		return 0;
+
 	isp_dev->current_fmt = stf_g_fmt_by_mcode(&isp_dev->formats[fmt->pad],
 						  fmt->format.code);
 
@@ -202,6 +225,9 @@ static int isp_get_selection(struct v4l2_subdev *sd,
 	struct v4l2_subdev_format fmt = { 0 };
 	struct v4l2_rect *rect;
 
+	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad == STF_ISP_PAD_SINK_PARAMS)
+		return -EINVAL;
+
 	switch (sel->target) {
 	case V4L2_SEL_TGT_CROP_BOUNDS:
 		if (sel->pad == STF_ISP_PAD_SINK) {
@@ -239,6 +265,9 @@ static int isp_set_selection(struct v4l2_subdev *sd,
 	struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
 	struct v4l2_rect *rect;
 
+	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad == STF_ISP_PAD_SINK_PARAMS)
+		return -EINVAL;
+
 	if (sel->target != V4L2_SEL_TGT_CROP)
 		return -EINVAL;
 
@@ -296,8 +325,38 @@ static int isp_init_formats(struct v4l2_subdev *sd,
 			.height = 1080
 		}
 	};
+	struct v4l2_subdev_format format_params = {
+		.pad = STF_ISP_PAD_SINK_PARAMS,
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+		.format = {
+			.code = MEDIA_BUS_FMT_METADATA_FIXED,
+			.width = 1,
+			.height = 1
+		}
+	};
+	struct v4l2_subdev_format format_scd = {
+		.pad = STF_ISP_PAD_SRC_SCD,
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+		.format = {
+			.code = MEDIA_BUS_FMT_METADATA_FIXED,
+			.width = 1,
+			.height = 1
+		}
+	};
+	int ret;
+
+	/* Init for STF_ISP_PAD_SINK and STF_ISP_PAD_SRC pad */
+	ret = isp_set_format(sd, sd_state, &format);
+	if (ret < 0)
+		return ret;
+
+	/* Init for STF_ISP_PAD_SINK_PARAMS pad */
+	ret = isp_set_format(sd, sd_state, &format_params);
+	if (ret < 0)
+		return ret;
 
-	return isp_set_format(sd, sd_state, &format);
+	/* Init for STF_ISP_PAD_SRC_SCD pad */
+	return isp_set_format(sd, sd_state, &format_scd);
 }
 
 static const struct v4l2_subdev_video_ops isp_video_ops = {
@@ -338,7 +397,9 @@ int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev)
 	v4l2_set_subdevdata(sd, isp_dev);
 
 	pads[STF_ISP_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+	pads[STF_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
 	pads[STF_ISP_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
+	pads[STF_ISP_PAD_SRC_SCD].flags = MEDIA_PAD_FL_SOURCE;
 
 	sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
 	sd->entity.ops = &isp_media_ops;
diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
index 955cbb048363..bc7e7b0736fa 100644
--- a/drivers/staging/media/starfive/camss/stf-isp.h
+++ b/drivers/staging/media/starfive/camss/stf-isp.h
@@ -392,7 +392,9 @@
 /* pad id for media framework */
 enum stf_isp_pad_id {
 	STF_ISP_PAD_SINK = 0,
+	STF_ISP_PAD_SINK_PARAMS,
 	STF_ISP_PAD_SRC,
+	STF_ISP_PAD_SRC_SCD,
 	STF_ISP_PAD_MAX
 };
 
-- 
2.25.1

[PATCH v5 05/14] staging: media: starfive: Separate buffer from ISP hardware operation

[Changhuang Liang]

Separate buffer from ISP hardware operation. Convenient to extract the
buffer be a common file.
Replace "while" with "if" in stf_buf_done helper function because one
interrupt signal only handle one video buffer.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../media/starfive/camss/stf-capture.c        | 63 +++++++++++--------
 1 file changed, 36 insertions(+), 27 deletions(-)

diff --git a/drivers/staging/media/starfive/camss/stf-capture.c b/drivers/staging/media/starfive/camss/stf-capture.c
index ec5169e7b391..494e56e731f0 100644
--- a/drivers/staging/media/starfive/camss/stf-capture.c
+++ b/drivers/staging/media/starfive/camss/stf-capture.c
@@ -368,7 +368,7 @@ static void stf_buf_flush(struct stf_v_buf *output, enum vb2_buffer_state state)
 	}
 }
 
-static void stf_buf_done(struct stf_v_buf *output)
+static struct stfcamss_buffer *stf_buf_done(struct stf_v_buf *output)
 {
 	struct stfcamss_buffer *ready_buf;
 	u64 ts = ktime_get_ns();
@@ -376,27 +376,27 @@ static void stf_buf_done(struct stf_v_buf *output)
 
 	if (output->state == STF_OUTPUT_OFF ||
 	    output->state == STF_OUTPUT_RESERVED)
-		return;
+		return NULL;
 
 	spin_lock_irqsave(&output->lock, flags);
 
-	while ((ready_buf = stf_buf_get_ready(output))) {
+	ready_buf = stf_buf_get_ready(output);
+	if (ready_buf) {
 		ready_buf->vb.vb2_buf.timestamp = ts;
 		ready_buf->vb.sequence = output->sequence++;
-
-		vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
 	}
 
 	spin_unlock_irqrestore(&output->lock, flags);
+
+	return ready_buf;
 }
 
-static void stf_change_buffer(struct stf_v_buf *output)
+static struct stfcamss_buffer *stf_change_buffer(struct stf_v_buf *output)
 {
 	struct stf_capture *cap = container_of(output, struct stf_capture,
 					       buffers);
 	struct stfcamss *stfcamss = cap->video.stfcamss;
 	struct stfcamss_buffer *ready_buf;
-	dma_addr_t *new_addr;
 	unsigned long flags;
 	u32 active_index;
 
@@ -404,7 +404,7 @@ static void stf_change_buffer(struct stf_v_buf *output)
 	    output->state == STF_OUTPUT_STOPPING ||
 	    output->state == STF_OUTPUT_RESERVED ||
 	    output->state == STF_OUTPUT_IDLE)
-		return;
+		return NULL;
 
 	spin_lock_irqsave(&output->lock, flags);
 
@@ -426,37 +426,37 @@ static void stf_change_buffer(struct stf_v_buf *output)
 
 	/* Get next buffer */
 	output->buf[active_index] = stf_buf_get_pending(output);
-	if (!output->buf[active_index]) {
-		new_addr = ready_buf->addr;
+	if (!output->buf[active_index])
 		stf_buf_update_on_last(output);
-	} else {
-		new_addr = output->buf[active_index]->addr;
+	else
 		stf_buf_update_on_next(output);
-	}
 
-	if (output->state == STF_OUTPUT_STOPPING) {
+	if (output->state == STF_OUTPUT_STOPPING)
 		output->last_buffer = ready_buf;
-	} else {
-		if (cap->type == STF_CAPTURE_RAW)
-			stf_set_raw_addr(stfcamss, new_addr[0]);
-		else if (cap->type == STF_CAPTURE_YUV)
-			stf_set_yuv_addr(stfcamss, new_addr[0], new_addr[1]);
-
+	else
 		stf_buf_add_ready(output, ready_buf);
-	}
 
 out_unlock:
 	spin_unlock_irqrestore(&output->lock, flags);
+
+	return output->buf[active_index];
 }
 
 irqreturn_t stf_wr_irq_handler(int irq, void *priv)
 {
 	struct stfcamss *stfcamss = priv;
 	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_RAW];
+	struct stfcamss_buffer *change_buf;
+	struct stfcamss_buffer *ready_buf;
 
 	if (atomic_dec_if_positive(&cap->buffers.frame_skip) < 0) {
-		stf_change_buffer(&cap->buffers);
-		stf_buf_done(&cap->buffers);
+		change_buf = stf_change_buffer(&cap->buffers);
+		if (change_buf)
+			stf_set_raw_addr(stfcamss, change_buf->addr[0]);
+
+		ready_buf = stf_buf_done(&cap->buffers);
+		if (ready_buf)
+			vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
 	}
 
 	stf_syscon_reg_set_bit(stfcamss, VIN_INRT_PIX_CFG, U0_VIN_INTR_CLEAN);
@@ -469,12 +469,16 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
 {
 	struct stfcamss *stfcamss = priv;
 	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stfcamss_buffer *ready_buf;
 	u32 status;
 
 	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
 	if (status & ISPC_ISP) {
-		if (status & ISPC_ENUO)
-			stf_buf_done(&cap->buffers);
+		if (status & ISPC_ENUO) {
+			ready_buf = stf_buf_done(&cap->buffers);
+			if (ready_buf)
+				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+		}
 
 		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
 				  (status & ~ISPC_INT_ALL_MASK) |
@@ -488,13 +492,18 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
 {
 	struct stfcamss *stfcamss = priv;
 	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stfcamss_buffer *change_buf;
 	u32 status;
 
 	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
 	if (status & ISPC_LINE) {
 		if (atomic_dec_if_positive(&cap->buffers.frame_skip) < 0) {
-			if ((status & ISPC_ENUO))
-				stf_change_buffer(&cap->buffers);
+			if ((status & ISPC_ENUO)) {
+				change_buf = stf_change_buffer(&cap->buffers);
+				if (change_buf)
+					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
+							 change_buf->addr[1]);
+			}
 		}
 
 		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS,
-- 
2.25.1

[PATCH v5 06/14] staging: media: starfive: Separate buffer be a common file

[Changhuang Liang]

Buffer operations is not only applicable to capture device, also
can use for output device. So separating it be a common file.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 drivers/staging/media/starfive/camss/Makefile |   1 +
 .../staging/media/starfive/camss/stf-buffer.c | 166 ++++++++++++++++++
 .../staging/media/starfive/camss/stf-buffer.h |  52 ++++++
 .../staging/media/starfive/camss/stf-camss.h  |   1 +
 .../media/starfive/camss/stf-capture.c        | 158 -----------------
 .../media/starfive/camss/stf-capture.h        |  22 ---
 .../staging/media/starfive/camss/stf-video.h  |  10 +-
 7 files changed, 222 insertions(+), 188 deletions(-)
 create mode 100644 drivers/staging/media/starfive/camss/stf-buffer.c
 create mode 100644 drivers/staging/media/starfive/camss/stf-buffer.h

diff --git a/drivers/staging/media/starfive/camss/Makefile b/drivers/staging/media/starfive/camss/Makefile
index 005790202e7b..411b45f3fb52 100644
--- a/drivers/staging/media/starfive/camss/Makefile
+++ b/drivers/staging/media/starfive/camss/Makefile
@@ -4,6 +4,7 @@
 #
 
 starfive-camss-objs += \
+		stf-buffer.o \
 		stf-camss.o \
 		stf-capture.o \
 		stf-isp.o \
diff --git a/drivers/staging/media/starfive/camss/stf-buffer.c b/drivers/staging/media/starfive/camss/stf-buffer.c
new file mode 100644
index 000000000000..7272b5ab9eb5
--- /dev/null
+++ b/drivers/staging/media/starfive/camss/stf-buffer.c
@@ -0,0 +1,166 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * StarFive Camera Subsystem - buffer common
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ */
+
+#include "stf-camss.h"
+
+void stf_buf_add_pending(struct stf_v_buf *output,
+			 struct stfcamss_buffer *buffer)
+{
+	INIT_LIST_HEAD(&buffer->queue);
+	list_add_tail(&buffer->queue, &output->pending_bufs);
+}
+
+struct stfcamss_buffer *stf_buf_get_pending(struct stf_v_buf *output)
+{
+	struct stfcamss_buffer *buffer = NULL;
+
+	if (!list_empty(&output->pending_bufs)) {
+		buffer = list_first_entry(&output->pending_bufs,
+					  struct stfcamss_buffer,
+					  queue);
+		list_del(&buffer->queue);
+	}
+
+	return buffer;
+}
+
+void stf_buf_add_ready(struct stf_v_buf *output,
+		       struct stfcamss_buffer *buffer)
+{
+	INIT_LIST_HEAD(&buffer->queue);
+	list_add_tail(&buffer->queue, &output->ready_bufs);
+}
+
+struct stfcamss_buffer *stf_buf_get_ready(struct stf_v_buf *output)
+{
+	struct stfcamss_buffer *buffer = NULL;
+
+	if (!list_empty(&output->ready_bufs)) {
+		buffer = list_first_entry(&output->ready_bufs,
+					  struct stfcamss_buffer,
+					  queue);
+		list_del(&buffer->queue);
+	}
+
+	return buffer;
+}
+
+static void stf_buf_update_on_last(struct stf_v_buf *output)
+{
+	switch (output->state) {
+	case STF_OUTPUT_CONTINUOUS:
+		output->state = STF_OUTPUT_SINGLE;
+		output->active_buf = !output->active_buf;
+		break;
+	case STF_OUTPUT_SINGLE:
+		output->state = STF_OUTPUT_STOPPING;
+		break;
+	default:
+		break;
+	}
+}
+
+static void stf_buf_update_on_next(struct stf_v_buf *output)
+{
+	switch (output->state) {
+	case STF_OUTPUT_CONTINUOUS:
+		output->active_buf = !output->active_buf;
+		break;
+	case STF_OUTPUT_SINGLE:
+	default:
+		break;
+	}
+}
+
+void stf_buf_flush(struct stf_v_buf *output, enum vb2_buffer_state state)
+{
+	struct stfcamss_buffer *buf;
+	struct stfcamss_buffer *t;
+
+	list_for_each_entry_safe(buf, t, &output->pending_bufs, queue) {
+		vb2_buffer_done(&buf->vb.vb2_buf, state);
+		list_del(&buf->queue);
+	}
+	list_for_each_entry_safe(buf, t, &output->ready_bufs, queue) {
+		vb2_buffer_done(&buf->vb.vb2_buf, state);
+		list_del(&buf->queue);
+	}
+}
+
+struct stfcamss_buffer *stf_change_buffer(struct stf_v_buf *output)
+{
+	struct stf_capture *cap = container_of(output, struct stf_capture,
+					       buffers);
+	struct stfcamss *stfcamss = cap->video.stfcamss;
+	struct stfcamss_buffer *ready_buf;
+	unsigned long flags;
+	u32 active_index;
+
+	if (output->state == STF_OUTPUT_OFF ||
+	    output->state == STF_OUTPUT_STOPPING ||
+	    output->state == STF_OUTPUT_RESERVED ||
+	    output->state == STF_OUTPUT_IDLE)
+		return NULL;
+
+	spin_lock_irqsave(&output->lock, flags);
+
+	active_index = output->active_buf;
+
+	ready_buf = output->buf[active_index];
+	if (!ready_buf) {
+		dev_dbg(stfcamss->dev, "missing ready buf %d %d.\n",
+			active_index, output->state);
+		active_index = !active_index;
+		ready_buf = output->buf[active_index];
+		if (!ready_buf) {
+			dev_dbg(stfcamss->dev,
+				"missing ready buf2 %d %d.\n",
+				active_index, output->state);
+			goto out_unlock;
+		}
+	}
+
+	/* Get next buffer */
+	output->buf[active_index] = stf_buf_get_pending(output);
+	if (!output->buf[active_index])
+		stf_buf_update_on_last(output);
+	else
+		stf_buf_update_on_next(output);
+
+	if (output->state == STF_OUTPUT_STOPPING)
+		output->last_buffer = ready_buf;
+	else
+		stf_buf_add_ready(output, ready_buf);
+
+out_unlock:
+	spin_unlock_irqrestore(&output->lock, flags);
+
+	return output->buf[active_index];
+}
+
+struct stfcamss_buffer *stf_buf_done(struct stf_v_buf *output)
+{
+	struct stfcamss_buffer *ready_buf;
+	u64 ts = ktime_get_ns();
+	unsigned long flags;
+
+	if (output->state == STF_OUTPUT_OFF ||
+	    output->state == STF_OUTPUT_RESERVED)
+		return NULL;
+
+	spin_lock_irqsave(&output->lock, flags);
+
+	ready_buf = stf_buf_get_ready(output);
+	if (ready_buf) {
+		ready_buf->vb.vb2_buf.timestamp = ts;
+		ready_buf->vb.sequence = output->sequence++;
+	}
+
+	spin_unlock_irqrestore(&output->lock, flags);
+
+	return ready_buf;
+}
diff --git a/drivers/staging/media/starfive/camss/stf-buffer.h b/drivers/staging/media/starfive/camss/stf-buffer.h
new file mode 100644
index 000000000000..9d1670fb05ed
--- /dev/null
+++ b/drivers/staging/media/starfive/camss/stf-buffer.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * StarFive Camera Subsystem - buffer common
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ */
+
+#ifndef STF_BUFFER_H
+#define STF_BUFFER_H
+
+#include <linux/list.h>
+#include <media/videobuf2-v4l2.h>
+
+enum stf_v_state {
+	STF_OUTPUT_OFF,
+	STF_OUTPUT_RESERVED,
+	STF_OUTPUT_SINGLE,
+	STF_OUTPUT_CONTINUOUS,
+	STF_OUTPUT_IDLE,
+	STF_OUTPUT_STOPPING
+};
+
+struct stfcamss_buffer {
+	struct vb2_v4l2_buffer vb;
+	dma_addr_t addr[2];
+	struct list_head queue;
+};
+
+struct stf_v_buf {
+	int active_buf;
+	struct stfcamss_buffer *buf[2];
+	struct stfcamss_buffer *last_buffer;
+	struct list_head pending_bufs;
+	struct list_head ready_bufs;
+	enum stf_v_state state;
+	unsigned int sequence;
+	/* protects the above member variables */
+	spinlock_t lock;
+	atomic_t frame_skip;
+};
+
+void stf_buf_add_pending(struct stf_v_buf *output,
+			 struct stfcamss_buffer *buffer);
+struct stfcamss_buffer *stf_buf_get_pending(struct stf_v_buf *output);
+void stf_buf_add_ready(struct stf_v_buf *output,
+		       struct stfcamss_buffer *buffer);
+struct stfcamss_buffer *stf_buf_get_ready(struct stf_v_buf *output);
+void stf_buf_flush(struct stf_v_buf *output, enum vb2_buffer_state state);
+struct stfcamss_buffer *stf_change_buffer(struct stf_v_buf *output);
+struct stfcamss_buffer *stf_buf_done(struct stf_v_buf *output);
+
+#endif /* STF_BUFFER_H */
diff --git a/drivers/staging/media/starfive/camss/stf-camss.h b/drivers/staging/media/starfive/camss/stf-camss.h
index e2b0cfb437bd..ae49c7031ab7 100644
--- a/drivers/staging/media/starfive/camss/stf-camss.h
+++ b/drivers/staging/media/starfive/camss/stf-camss.h
@@ -18,6 +18,7 @@
 #include <media/v4l2-async.h>
 #include <media/v4l2-device.h>
 
+#include "stf-buffer.h"
 #include "stf-isp.h"
 #include "stf-capture.h"
 
diff --git a/drivers/staging/media/starfive/camss/stf-capture.c b/drivers/staging/media/starfive/camss/stf-capture.c
index 494e56e731f0..696d79920713 100644
--- a/drivers/staging/media/starfive/camss/stf-capture.c
+++ b/drivers/staging/media/starfive/camss/stf-capture.c
@@ -93,20 +93,6 @@ static void stf_init_addrs(struct stfcamss_video *video)
 		stf_set_yuv_addr(video->stfcamss, addr0, addr1);
 }
 
-static struct stfcamss_buffer *stf_buf_get_pending(struct stf_v_buf *output)
-{
-	struct stfcamss_buffer *buffer = NULL;
-
-	if (!list_empty(&output->pending_bufs)) {
-		buffer = list_first_entry(&output->pending_bufs,
-					  struct stfcamss_buffer,
-					  queue);
-		list_del(&buffer->queue);
-	}
-
-	return buffer;
-}
-
 static void stf_cap_s_cfg(struct stfcamss_video *video)
 {
 	struct stf_capture *cap = to_stf_capture(video);
@@ -263,61 +249,6 @@ static void stf_capture_init(struct stfcamss *stfcamss, struct stf_capture *cap)
 	}
 }
 
-static void stf_buf_add_ready(struct stf_v_buf *output,
-			      struct stfcamss_buffer *buffer)
-{
-	INIT_LIST_HEAD(&buffer->queue);
-	list_add_tail(&buffer->queue, &output->ready_bufs);
-}
-
-static struct stfcamss_buffer *stf_buf_get_ready(struct stf_v_buf *output)
-{
-	struct stfcamss_buffer *buffer = NULL;
-
-	if (!list_empty(&output->ready_bufs)) {
-		buffer = list_first_entry(&output->ready_bufs,
-					  struct stfcamss_buffer,
-					  queue);
-		list_del(&buffer->queue);
-	}
-
-	return buffer;
-}
-
-static void stf_buf_add_pending(struct stf_v_buf *output,
-				struct stfcamss_buffer *buffer)
-{
-	INIT_LIST_HEAD(&buffer->queue);
-	list_add_tail(&buffer->queue, &output->pending_bufs);
-}
-
-static void stf_buf_update_on_last(struct stf_v_buf *output)
-{
-	switch (output->state) {
-	case STF_OUTPUT_CONTINUOUS:
-		output->state = STF_OUTPUT_SINGLE;
-		output->active_buf = !output->active_buf;
-		break;
-	case STF_OUTPUT_SINGLE:
-		output->state = STF_OUTPUT_STOPPING;
-		break;
-	default:
-		break;
-	}
-}
-
-static void stf_buf_update_on_next(struct stf_v_buf *output)
-{
-	switch (output->state) {
-	case STF_OUTPUT_CONTINUOUS:
-		output->active_buf = !output->active_buf;
-		break;
-	case STF_OUTPUT_SINGLE:
-	default:
-		break;
-	}
-}
-
 static void stf_buf_update_on_new(struct stfcamss_video *video,
 				  struct stfcamss_buffer *new_buf)
 {
@@ -353,95 +284,6 @@ static void stf_buf_update_on_new(struct stfcamss_video *video,
 	}
 }
 
-static void stf_buf_flush(struct stf_v_buf *output, enum vb2_buffer_state state)
-{
-	struct stfcamss_buffer *buf;
-	struct stfcamss_buffer *t;
-
-	list_for_each_entry_safe(buf, t, &output->pending_bufs, queue) {
-		vb2_buffer_done(&buf->vb.vb2_buf, state);
-		list_del(&buf->queue);
-	}
-	list_for_each_entry_safe(buf, t, &output->ready_bufs, queue) {
-		vb2_buffer_done(&buf->vb.vb2_buf, state);
-		list_del(&buf->queue);
-	}
-}
-
-static struct stfcamss_buffer *stf_buf_done(struct stf_v_buf *output)
-{
-	struct stfcamss_buffer *ready_buf;
-	u64 ts = ktime_get_ns();
-	unsigned long flags;
-
-	if (output->state == STF_OUTPUT_OFF ||
-	    output->state == STF_OUTPUT_RESERVED)
-		return NULL;
-
-	spin_lock_irqsave(&output->lock, flags);
-
-	ready_buf = stf_buf_get_ready(output);
-	if (ready_buf) {
-		ready_buf->vb.vb2_buf.timestamp = ts;
-		ready_buf->vb.sequence = output->sequence++;
-	}
-
-	spin_unlock_irqrestore(&output->lock, flags);
-
-	return ready_buf;
-}
-
-static struct stfcamss_buffer *stf_change_buffer(struct stf_v_buf *output)
-{
-	struct stf_capture *cap = container_of(output, struct stf_capture,
-					       buffers);
-	struct stfcamss *stfcamss = cap->video.stfcamss;
-	struct stfcamss_buffer *ready_buf;
-	unsigned long flags;
-	u32 active_index;
-
-	if (output->state == STF_OUTPUT_OFF ||
-	    output->state == STF_OUTPUT_STOPPING ||
-	    output->state == STF_OUTPUT_RESERVED ||
-	    output->state == STF_OUTPUT_IDLE)
-		return NULL;
-
-	spin_lock_irqsave(&output->lock, flags);
-
-	active_index = output->active_buf;
-
-	ready_buf = output->buf[active_index];
-	if (!ready_buf) {
-		dev_dbg(stfcamss->dev, "missing ready buf %d %d.\n",
-			active_index, output->state);
-		active_index = !active_index;
-		ready_buf = output->buf[active_index];
-		if (!ready_buf) {
-			dev_dbg(stfcamss->dev,
-				"missing ready buf2 %d %d.\n",
-				active_index, output->state);
-			goto out_unlock;
-		}
-	}
-
-	/* Get next buffer */
-	output->buf[active_index] = stf_buf_get_pending(output);
-	if (!output->buf[active_index])
-		stf_buf_update_on_last(output);
-	else
-		stf_buf_update_on_next(output);
-
-	if (output->state == STF_OUTPUT_STOPPING)
-		output->last_buffer = ready_buf;
-	else
-		stf_buf_add_ready(output, ready_buf);
-
-out_unlock:
-	spin_unlock_irqrestore(&output->lock, flags);
-
-	return output->buf[active_index];
-}
-
 irqreturn_t stf_wr_irq_handler(int irq, void *priv)
 {
 	struct stfcamss *stfcamss = priv;
diff --git a/drivers/staging/media/starfive/camss/stf-capture.h b/drivers/staging/media/starfive/camss/stf-capture.h
index 2f9740b7e500..fe2489d55090 100644
--- a/drivers/staging/media/starfive/camss/stf-capture.h
+++ b/drivers/staging/media/starfive/camss/stf-capture.h
@@ -48,28 +48,6 @@
 #define U0_VIN_P_I_MIPI_HAEDER_EN0_MASK		BIT(12)
 #define U0_VIN_PIX_NUM_MASK			GENMASK(16, 13)
 
-enum stf_v_state {
-	STF_OUTPUT_OFF,
-	STF_OUTPUT_RESERVED,
-	STF_OUTPUT_SINGLE,
-	STF_OUTPUT_CONTINUOUS,
-	STF_OUTPUT_IDLE,
-	STF_OUTPUT_STOPPING
-};
-
-struct stf_v_buf {
-	int active_buf;
-	struct stfcamss_buffer *buf[2];
-	struct stfcamss_buffer *last_buffer;
-	struct list_head pending_bufs;
-	struct list_head ready_bufs;
-	enum stf_v_state state;
-	unsigned int sequence;
-	/* protects the above member variables */
-	spinlock_t lock;
-	atomic_t frame_skip;
-};
-
 struct stf_capture {
 	struct stfcamss_video video;
 	struct stf_v_buf buffers;
diff --git a/drivers/staging/media/starfive/camss/stf-video.h b/drivers/staging/media/starfive/camss/stf-video.h
index 8052b77e3ad8..59799b65cbe5 100644
--- a/drivers/staging/media/starfive/camss/stf-video.h
+++ b/drivers/staging/media/starfive/camss/stf-video.h
@@ -10,13 +10,13 @@
 #ifndef STF_VIDEO_H
 #define STF_VIDEO_H
 
-#include <linux/list.h>
 #include <linux/mutex.h>
 #include <linux/videodev2.h>
 #include <media/v4l2-dev.h>
 #include <media/v4l2-fh.h>
 #include <media/v4l2-ioctl.h>
-#include <media/videobuf2-v4l2.h>
+
+#include "stf-buffer.h"
 
 #define STFCAMSS_FRAME_MIN_WIDTH		64
 #define STFCAMSS_FRAME_MAX_WIDTH		1920
@@ -40,12 +40,6 @@ enum stf_capture_type {
 	STF_CAPTURE_NUM,
 };
 
-struct stfcamss_buffer {
-	struct vb2_v4l2_buffer vb;
-	dma_addr_t addr[2];
-	struct list_head queue;
-};
-
 struct fract {
 	u8 numerator;
 	u8 denominator;
-- 
2.25.1

[PATCH v5 07/14] staging: media: starfive: Separate ISP hardware from capture device

[Changhuang Liang]

Some ISP hardware operations also used for output device, so separate
them from capture device, move them to ISP common file.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../media/starfive/camss/stf-capture.c        | 60 -------------------
 .../media/starfive/camss/stf-isp-hw-ops.c     | 60 +++++++++++++++++++
 .../staging/media/starfive/camss/stf-isp.h    |  3 +
 3 files changed, 63 insertions(+), 60 deletions(-)

diff --git a/drivers/staging/media/starfive/camss/stf-capture.c b/drivers/staging/media/starfive/camss/stf-capture.c
index 696d79920713..75f6ef405e61 100644
--- a/drivers/staging/media/starfive/camss/stf-capture.c
+++ b/drivers/staging/media/starfive/camss/stf-capture.c
@@ -66,13 +66,6 @@ static void stf_set_raw_addr(struct stfcamss *stfcamss, dma_addr_t addr)
 	stf_syscon_reg_write(stfcamss, VIN_START_ADDR_N, (long)addr);
 }
 
-static void stf_set_yuv_addr(struct stfcamss *stfcamss,
-			     dma_addr_t y_addr, dma_addr_t uv_addr)
-{
-	stf_isp_reg_write(stfcamss, ISP_REG_Y_PLANE_START_ADDR, y_addr);
-	stf_isp_reg_write(stfcamss, ISP_REG_UV_PLANE_START_ADDR, uv_addr);
-}
-
 static void stf_init_addrs(struct stfcamss_video *video)
 {
 	struct stf_capture *cap = to_stf_capture(video);
@@ -307,59 +300,6 @@ irqreturn_t stf_wr_irq_handler(int irq, void *priv)
 	return IRQ_HANDLED;
 }
 
-irqreturn_t stf_isp_irq_handler(int irq, void *priv)
-{
-	struct stfcamss *stfcamss = priv;
-	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
-	struct stfcamss_buffer *ready_buf;
-	u32 status;
-
-	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
-	if (status & ISPC_ISP) {
-		if (status & ISPC_ENUO) {
-			ready_buf = stf_buf_done(&cap->buffers);
-			if (ready_buf)
-				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
-		}
-
-		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
-				  (status & ~ISPC_INT_ALL_MASK) |
-				  ISPC_ISP | ISPC_CSI | ISPC_SC);
-	}
-
-	return IRQ_HANDLED;
-}
-
-irqreturn_t stf_line_irq_handler(int irq, void *priv)
-{
-	struct stfcamss *stfcamss = priv;
-	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
-	struct stfcamss_buffer *change_buf;
-	u32 status;
-
-	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
-	if (status & ISPC_LINE) {
-		if (atomic_dec_if_positive(&cap->buffers.frame_skip) < 0) {
-			if ((status & ISPC_ENUO)) {
-				change_buf = stf_change_buffer(&cap->buffers);
-				if (change_buf)
-					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
-							 change_buf->addr[1]);
-			}
-		}
-
-		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS,
-				    CSI_INTS_MASK, CSI_INTS(0x3));
-		stf_isp_reg_set_bit(stfcamss, ISP_REG_IESHD,
-				    SHAD_UP_M | SHAD_UP_EN, 0x3);
-
-		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
-				  (status & ~ISPC_INT_ALL_MASK) | ISPC_LINE);
-	}
-
-	return IRQ_HANDLED;
-}
-
 static int stf_queue_buffer(struct stfcamss_video *video,
 			    struct stfcamss_buffer *buf)
 {
diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
index c34631ff9422..6b3966ca18bf 100644
--- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
+++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
@@ -443,3 +443,63 @@ void stf_isp_stream_set(struct stf_isp_dev *isp_dev)
 	stf_isp_reg_write_delay(stfcamss, ISP_REG_CSI_INPUT_EN_AND_STATUS,
 				CSI_EN_S, 10);
 }
+
+void stf_set_yuv_addr(struct stfcamss *stfcamss,
+		      dma_addr_t y_addr, dma_addr_t uv_addr)
+{
+	stf_isp_reg_write(stfcamss, ISP_REG_Y_PLANE_START_ADDR, y_addr);
+	stf_isp_reg_write(stfcamss, ISP_REG_UV_PLANE_START_ADDR, uv_addr);
+}
+
+irqreturn_t stf_line_irq_handler(int irq, void *priv)
+{
+	struct stfcamss *stfcamss = priv;
+	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stfcamss_buffer *change_buf;
+	u32 status;
+
+	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
+	if (status & ISPC_LINE) {
+		if (atomic_dec_if_positive(&cap->buffers.frame_skip) < 0) {
+			if ((status & ISPC_ENUO)) {
+				change_buf = stf_change_buffer(&cap->buffers);
+				if (change_buf)
+					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
+							 change_buf->addr[1]);
+			}
+		}
+
+		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS,
+				    CSI_INTS_MASK, CSI_INTS(0x3));
+		stf_isp_reg_set_bit(stfcamss, ISP_REG_IESHD,
+				    SHAD_UP_M | SHAD_UP_EN, 0x3);
+
+		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
+				  (status & ~ISPC_INT_ALL_MASK) | ISPC_LINE);
+	}
+
+	return IRQ_HANDLED;
+}
+
+irqreturn_t stf_isp_irq_handler(int irq, void *priv)
+{
+	struct stfcamss *stfcamss = priv;
+	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stfcamss_buffer *ready_buf;
+	u32 status;
+
+	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
+	if (status & ISPC_ISP) {
+		if (status & ISPC_ENUO) {
+			ready_buf = stf_buf_done(&cap->buffers);
+			if (ready_buf)
+				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+		}
+
+		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
+				  (status & ~ISPC_INT_ALL_MASK) |
+				  ISPC_ISP | ISPC_CSI | ISPC_SC);
+	}
+
+	return IRQ_HANDLED;
+}
diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
index bc7e7b0736fa..fcda0502e3b0 100644
--- a/drivers/staging/media/starfive/camss/stf-isp.h
+++ b/drivers/staging/media/starfive/camss/stf-isp.h
@@ -427,4 +427,7 @@ int stf_isp_init(struct stfcamss *stfcamss);
 int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev);
 int stf_isp_unregister(struct stf_isp_dev *isp_dev);
 
+void stf_set_yuv_addr(struct stfcamss *stfcamss,
+		      dma_addr_t y_addr, dma_addr_t uv_addr);
+
 #endif /* STF_ISP_H */
-- 
2.25.1

[PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Changhuang Liang]

Register ISP 3A "capture_scd" video device to receive statistics
collection data.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../staging/media/starfive/camss/stf-buffer.h |   1 +
 .../staging/media/starfive/camss/stf-camss.c  |  15 ++
 .../media/starfive/camss/stf-capture.c        |  21 ++-
 .../media/starfive/camss/stf-isp-hw-ops.c     |  66 ++++++++
 .../staging/media/starfive/camss/stf-isp.h    |  23 +++
 .../staging/media/starfive/camss/stf-video.c  | 146 +++++++++++++++++-
 .../staging/media/starfive/camss/stf-video.h  |   1 +
 7 files changed, 264 insertions(+), 9 deletions(-)

diff --git a/drivers/staging/media/starfive/camss/stf-buffer.h b/drivers/staging/media/starfive/camss/stf-buffer.h
index 9d1670fb05ed..727d00617448 100644
--- a/drivers/staging/media/starfive/camss/stf-buffer.h
+++ b/drivers/staging/media/starfive/camss/stf-buffer.h
@@ -23,6 +23,7 @@ enum stf_v_state {
 struct stfcamss_buffer {
 	struct vb2_v4l2_buffer vb;
 	dma_addr_t addr[2];
+	void *vaddr;
 	struct list_head queue;
 };
 
diff --git a/drivers/staging/media/starfive/camss/stf-camss.c b/drivers/staging/media/starfive/camss/stf-camss.c
index fecd3e67c7a1..fafa3ce2f6da 100644
--- a/drivers/staging/media/starfive/camss/stf-camss.c
+++ b/drivers/staging/media/starfive/camss/stf-camss.c
@@ -8,6 +8,7 @@
  *
  * Author: Jack Zhu <jack.zhu@starfivetech.com>
  * Author: Changhuang Liang <changhuang.liang@starfivetech.com>
+ * Author: Keith Zhao <keith.zhao@starfivetech.com>
  *
  */
 #include <linux/module.h>
@@ -126,6 +127,7 @@ static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
 static int stfcamss_register_devs(struct stfcamss *stfcamss)
 {
 	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
 	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
 	int ret;
 
@@ -150,8 +152,18 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
 
 	cap_yuv->video.source_subdev = &isp_dev->subdev;
 
+	ret = media_create_pad_link(&isp_dev->subdev.entity, STF_ISP_PAD_SRC_SCD,
+				    &cap_scd->video.vdev.entity, 0, 0);
+	if (ret)
+		goto err_rm_links0;
+
+	cap_scd->video.source_subdev = &isp_dev->subdev;
+
 	return ret;
 
+err_rm_links0:
+	media_entity_remove_links(&isp_dev->subdev.entity);
+	media_entity_remove_links(&cap_yuv->video.vdev.entity);
 err_cap_unregister:
 	stf_capture_unregister(stfcamss);
 err_isp_unregister:
@@ -163,10 +175,12 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
 static void stfcamss_unregister_devs(struct stfcamss *stfcamss)
 {
 	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
 	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
 
 	media_entity_remove_links(&isp_dev->subdev.entity);
 	media_entity_remove_links(&cap_yuv->video.vdev.entity);
+	media_entity_remove_links(&cap_scd->video.vdev.entity);
 
 	stf_isp_unregister(&stfcamss->isp_dev);
 	stf_capture_unregister(stfcamss);
@@ -436,5 +450,6 @@ module_platform_driver(stfcamss_driver);
 
 MODULE_AUTHOR("Jack Zhu <jack.zhu@starfivetech.com>");
 MODULE_AUTHOR("Changhuang Liang <changhuang.liang@starfivetech.com>");
+MODULE_AUTHOR("Keith Zhao <keith.zhao@starfivetech.com>");
 MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
 MODULE_LICENSE("GPL");
diff --git a/drivers/staging/media/starfive/camss/stf-capture.c b/drivers/staging/media/starfive/camss/stf-capture.c
index 75f6ef405e61..328b8c6e351d 100644
--- a/drivers/staging/media/starfive/camss/stf-capture.c
+++ b/drivers/staging/media/starfive/camss/stf-capture.c
@@ -12,6 +12,7 @@
 static const char * const stf_cap_names[] = {
 	"capture_raw",
 	"capture_yuv",
+	"capture_scd",
 };
 
 static const struct stfcamss_format_info stf_wr_fmts[] = {
@@ -55,6 +56,14 @@ static const struct stfcamss_format_info stf_isp_fmts[] = {
 	},
 };
 
+/* 3A Statistics Collection Data */
+static const struct stfcamss_format_info stf_isp_scd_fmts[] = {
+	{
+		.code = MEDIA_BUS_FMT_METADATA_FIXED,
+		.pixelformat = V4L2_META_FMT_STF_ISP_STAT_3A,
+	},
+};
+
 static inline struct stf_capture *to_stf_capture(struct stfcamss_video *video)
 {
 	return container_of(video, struct stf_capture, video);
@@ -84,6 +93,8 @@ static void stf_init_addrs(struct stfcamss_video *video)
 		stf_set_raw_addr(video->stfcamss, addr0);
 	else if (cap->type == STF_CAPTURE_YUV)
 		stf_set_yuv_addr(video->stfcamss, addr0, addr1);
+	else
+		stf_set_scd_addr(video->stfcamss, addr0, addr1, TYPE_AWB);
 }
 
 static void stf_cap_s_cfg(struct stfcamss_video *video)
@@ -227,18 +238,24 @@ static void stf_capture_init(struct stfcamss *stfcamss, struct stf_capture *cap)
 	INIT_LIST_HEAD(&cap->buffers.ready_bufs);
 	spin_lock_init(&cap->buffers.lock);
 
-	cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	cap->video.stfcamss = stfcamss;
 	cap->video.bpl_alignment = 16 * 8;
 
 	if (cap->type == STF_CAPTURE_RAW) {
+		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 		cap->video.formats = stf_wr_fmts;
 		cap->video.nformats = ARRAY_SIZE(stf_wr_fmts);
 		cap->video.bpl_alignment = 8;
 	} else if (cap->type == STF_CAPTURE_YUV) {
+		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 		cap->video.formats = stf_isp_fmts;
 		cap->video.nformats = ARRAY_SIZE(stf_isp_fmts);
 		cap->video.bpl_alignment = 1;
+	} else {
+		cap->video.type = V4L2_BUF_TYPE_META_CAPTURE;
+		cap->video.formats = stf_isp_scd_fmts;
+		cap->video.nformats = ARRAY_SIZE(stf_isp_scd_fmts);
+		cap->video.bpl_alignment = 16 * 8;
 	}
 }
 
@@ -362,9 +379,11 @@ void stf_capture_unregister(struct stfcamss *stfcamss)
 {
 	struct stf_capture *cap_raw = &stfcamss->captures[STF_CAPTURE_RAW];
 	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
 
 	stf_capture_unregister_one(cap_raw);
 	stf_capture_unregister_one(cap_yuv);
+	stf_capture_unregister_one(cap_scd);
 }
 
 int stf_capture_register(struct stfcamss *stfcamss,
diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
index 6b3966ca18bf..3b18d09f2cc6 100644
--- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
+++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
@@ -451,11 +451,57 @@ void stf_set_yuv_addr(struct stfcamss *stfcamss,
 	stf_isp_reg_write(stfcamss, ISP_REG_UV_PLANE_START_ADDR, uv_addr);
 }
 
+static enum stf_isp_type_scd stf_isp_get_scd_type(struct stfcamss *stfcamss)
+{
+	int val;
+
+	val = stf_isp_reg_read(stfcamss, ISP_REG_SC_CFG_1);
+	return (enum stf_isp_type_scd)(val & ISP_SC_SEL_MASK) >> 30;
+}
+
+void stf_set_scd_addr(struct stfcamss *stfcamss,
+		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
+		      enum stf_isp_type_scd type_scd)
+{
+	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
+			    SEL_TYPE(type_scd));
+	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
+	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
+}
+
+static void stf_isp_fill_yhist(struct stfcamss *stfcamss, void *vaddr)
+{
+	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
+	u32 reg_addr = ISP_REG_YHIST_ACC_0;
+	u32 i;
+
+	for (i = 0; i < 64; i++, reg_addr += 4)
+		sc->y_histogram[i] = stf_isp_reg_read(stfcamss, reg_addr);
+}
+
+static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
+			      enum stf_isp_type_scd *type_scd)
+{
+	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
+
+	*type_scd = stf_isp_get_scd_type(stfcamss);
+	if (*type_scd == TYPE_AWB) {
+		sc->flag = JH7110_ISP_SC_FLAG_AWB;
+		*type_scd = TYPE_OECF;
+	} else {
+		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
+		*type_scd = TYPE_AWB;
+	}
+}
+
 irqreturn_t stf_line_irq_handler(int irq, void *priv)
 {
 	struct stfcamss *stfcamss = priv;
 	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
 	struct stfcamss_buffer *change_buf;
+	enum stf_isp_type_scd type_scd;
+	u32 value;
 	u32 status;
 
 	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
@@ -467,6 +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
 					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
 							 change_buf->addr[1]);
 			}
+
+			value = stf_isp_reg_read(stfcamss, ISP_REG_CSI_MODULE_CFG);
+			if (value & CSI_SC_EN) {
+				change_buf = stf_change_buffer(&cap_scd->buffers);
+				if (change_buf) {
+					stf_isp_fill_flag(stfcamss, change_buf->vaddr,
+							  &type_scd);
+					stf_set_scd_addr(stfcamss, change_buf->addr[0],
+							 change_buf->addr[1], type_scd);
+				}
+			}
 		}
 
 		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS,
@@ -485,6 +542,7 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
 {
 	struct stfcamss *stfcamss = priv;
 	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
+	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
 	struct stfcamss_buffer *ready_buf;
 	u32 status;
 
@@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
 				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
 		}
 
+		if (status & ISPC_SC) {
+			ready_buf = stf_buf_done(&cap_scd->buffers);
+			if (ready_buf) {
+				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
+				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+			}
+		}
+
 		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
 				  (status & ~ISPC_INT_ALL_MASK) |
 				  ISPC_ISP | ISPC_CSI | ISPC_SC);
diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
index fcda0502e3b0..0af7b367e57a 100644
--- a/drivers/staging/media/starfive/camss/stf-isp.h
+++ b/drivers/staging/media/starfive/camss/stf-isp.h
@@ -10,6 +10,7 @@
 #ifndef STF_ISP_H
 #define STF_ISP_H
 
+#include <linux/jh7110-isp.h>
 #include <media/v4l2-subdev.h>
 
 #include "stf-video.h"
@@ -107,6 +108,12 @@
 #define Y_COOR(y)				((y) << 16)
 #define X_COOR(x)				((x) << 0)
 
+#define ISP_REG_SCD_CFG_0			0x098
+
+#define ISP_REG_SC_CFG_1			0x0bc
+#define ISP_SC_SEL_MASK				GENMASK(31, 30)
+#define SEL_TYPE(n)				((n) << 30)
+
 #define ISP_REG_LCCF_CFG_2			0x0e0
 #define ISP_REG_LCCF_CFG_3			0x0e4
 #define ISP_REG_LCCF_CFG_4			0x0e8
@@ -305,6 +312,10 @@
 #define DNRM_F(n)				((n) << 16)
 #define CCM_M_DAT(n)				((n) << 0)
 
+#define ISP_REG_YHIST_CFG_4			0xcd8
+
+#define ISP_REG_YHIST_ACC_0			0xd00
+
 #define ISP_REG_GAMMA_VAL0			0xe00
 #define ISP_REG_GAMMA_VAL1			0xe04
 #define ISP_REG_GAMMA_VAL2			0xe08
@@ -389,6 +400,15 @@
 #define IMAGE_MAX_WIDTH				1920
 #define IMAGE_MAX_HEIGH				1080
 
+#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
+
+enum stf_isp_type_scd {
+	TYPE_DEC = 0,
+	TYPE_OBC,
+	TYPE_OECF,
+	TYPE_AWB,
+};
+
 /* pad id for media framework */
 enum stf_isp_pad_id {
 	STF_ISP_PAD_SINK = 0,
@@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev *isp_dev);
 
 void stf_set_yuv_addr(struct stfcamss *stfcamss,
 		      dma_addr_t y_addr, dma_addr_t uv_addr);
+void stf_set_scd_addr(struct stfcamss *stfcamss,
+		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
+		      enum stf_isp_type_scd type_scd);
 
 #endif /* STF_ISP_H */
diff --git a/drivers/staging/media/starfive/camss/stf-video.c b/drivers/staging/media/starfive/camss/stf-video.c
index 989b5e82bae9..2203605ec9c7 100644
--- a/drivers/staging/media/starfive/camss/stf-video.c
+++ b/drivers/staging/media/starfive/camss/stf-video.c
@@ -125,6 +125,14 @@ static int stf_video_init_format(struct stfcamss_video *video)
 	return 0;
 }
 
+static int stf_video_scd_init_format(struct stfcamss_video *video)
+{
+	video->active_fmt.fmt.meta.dataformat = video->formats[0].pixelformat;
+	video->active_fmt.fmt.meta.buffersize = sizeof(struct jh7110_isp_sc_buffer);
+
+	return 0;
+}
+
 /* -----------------------------------------------------------------------------
  * Video queue operations
  */
@@ -330,6 +338,78 @@ static const struct vb2_ops stf_video_vb2_q_ops = {
 	.stop_streaming  = video_stop_streaming,
 };
 
+static int video_scd_queue_setup(struct vb2_queue *q,
+				 unsigned int *num_buffers,
+				 unsigned int *num_planes,
+				 unsigned int sizes[],
+				 struct device *alloc_devs[])
+{
+	if (*num_planes)
+		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL : 0;
+
+	*num_planes = 1;
+	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
+
+	return 0;
+}
+
+static int video_scd_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
+	dma_addr_t *paddr;
+
+	paddr = vb2_plane_cookie(vb, 0);
+	buffer->addr[0] = *paddr;
+	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
+	buffer->vaddr = vb2_plane_vaddr(vb, 0);
+
+	return 0;
+}
+
+static int video_scd_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+	if (sizeof(struct jh7110_isp_sc_buffer) > vb2_plane_size(vb, 0))
+		return -EINVAL;
+
+	vb2_set_plane_payload(vb, 0, sizeof(struct jh7110_isp_sc_buffer));
+
+	vbuf->field = V4L2_FIELD_NONE;
+
+	return 0;
+}
+
+static int video_scd_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+	struct stfcamss_video *video = vb2_get_drv_priv(q);
+
+	video->ops->start_streaming(video);
+
+	return 0;
+}
+
+static void video_scd_stop_streaming(struct vb2_queue *q)
+{
+	struct stfcamss_video *video = vb2_get_drv_priv(q);
+
+	video->ops->stop_streaming(video);
+
+	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
+}
+
+static const struct vb2_ops stf_video_scd_vb2_q_ops = {
+	.queue_setup     = video_scd_queue_setup,
+	.wait_prepare    = vb2_ops_wait_prepare,
+	.wait_finish     = vb2_ops_wait_finish,
+	.buf_init        = video_scd_buf_init,
+	.buf_prepare     = video_scd_buf_prepare,
+	.buf_queue       = video_buf_queue,
+	.start_streaming = video_scd_start_streaming,
+	.stop_streaming  = video_scd_stop_streaming,
+};
+
 /* -----------------------------------------------------------------------------
  * V4L2 ioctls
  */
@@ -448,6 +528,37 @@ static const struct v4l2_ioctl_ops stf_vid_ioctl_ops = {
 	.vidioc_streamoff               = vb2_ioctl_streamoff,
 };
 
+static int video_scd_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct stfcamss_video *video = video_drvdata(file);
+	struct v4l2_meta_format *meta = &f->fmt.meta;
+
+	if (f->type != video->type)
+		return -EINVAL;
+
+	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
+	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops stf_vid_scd_ioctl_ops = {
+	.vidioc_querycap                = video_querycap,
+	.vidioc_enum_fmt_meta_cap       = video_enum_fmt,
+	.vidioc_g_fmt_meta_cap          = video_scd_g_fmt,
+	.vidioc_s_fmt_meta_cap          = video_scd_g_fmt,
+	.vidioc_try_fmt_meta_cap        = video_scd_g_fmt,
+	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
+	.vidioc_querybuf                = vb2_ioctl_querybuf,
+	.vidioc_qbuf                    = vb2_ioctl_qbuf,
+	.vidioc_expbuf                  = vb2_ioctl_expbuf,
+	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
+	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
+	.vidioc_streamon                = vb2_ioctl_streamon,
+	.vidioc_streamoff               = vb2_ioctl_streamoff,
+};
+
 /* -----------------------------------------------------------------------------
  * V4L2 file operations
  */
@@ -473,6 +584,9 @@ static int stf_link_validate(struct media_link *link)
 	struct stfcamss_video *video = video_get_drvdata(vdev);
 	int ret;
 
+	if (video->type == V4L2_BUF_TYPE_META_CAPTURE)
+		return 0;
+
 	ret = stf_video_check_format(video);
 
 	return ret;
@@ -506,7 +620,11 @@ int stf_video_register(struct stfcamss_video *video,
 	q = &video->vb2_q;
 	q->drv_priv = video;
 	q->mem_ops = &vb2_dma_contig_memops;
-	q->ops = &stf_video_vb2_q_ops;
+
+	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		q->ops = &stf_video_vb2_q_ops;
+	else
+		q->ops = &stf_video_scd_vb2_q_ops;
 	q->type = video->type;
 	q->io_modes = VB2_DMABUF | VB2_MMAP;
 	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
@@ -529,16 +647,28 @@ int stf_video_register(struct stfcamss_video *video,
 		goto err_mutex_destroy;
 	}
 
-	ret = stf_video_init_format(video);
-	if (ret < 0) {
-		dev_err(video->stfcamss->dev,
-			"Failed to init format: %d\n", ret);
-		goto err_media_cleanup;
+	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
+		ret = stf_video_init_format(video);
+		if (ret < 0) {
+			dev_err(video->stfcamss->dev,
+				"Failed to init format: %d\n", ret);
+			goto err_media_cleanup;
+		}
+		vdev->ioctl_ops = &stf_vid_ioctl_ops;
+		vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
+	} else {
+		ret = stf_video_scd_init_format(video);
+		if (ret < 0) {
+			dev_err(video->stfcamss->dev,
+				"Failed to init format: %d\n", ret);
+			goto err_media_cleanup;
+		}
+		vdev->ioctl_ops = &stf_vid_scd_ioctl_ops;
+		vdev->device_caps = V4L2_CAP_META_CAPTURE;
 	}
 
 	vdev->fops = &stf_vid_fops;
-	vdev->ioctl_ops = &stf_vid_ioctl_ops;
-	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+	vdev->device_caps |= V4L2_CAP_STREAMING;
 	vdev->entity.ops = &stf_media_ops;
 	vdev->vfl_dir = VFL_DIR_RX;
 	vdev->release = stf_video_release;
diff --git a/drivers/staging/media/starfive/camss/stf-video.h b/drivers/staging/media/starfive/camss/stf-video.h
index 59799b65cbe5..53a1cf4e59b7 100644
--- a/drivers/staging/media/starfive/camss/stf-video.h
+++ b/drivers/staging/media/starfive/camss/stf-video.h
@@ -37,6 +37,7 @@ enum stf_v_line_id {
 enum stf_capture_type {
 	STF_CAPTURE_RAW = 0,
 	STF_CAPTURE_YUV,
+	STF_CAPTURE_SCD,
 	STF_CAPTURE_NUM,
 };
 
-- 
2.25.1

[PATCH v5 09/14] staging: media: starfive: Update ISP initialise config for 3A

[Changhuang Liang]

Upadte ISP initialise for 3A statistics collection data.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../media/starfive/camss/stf-isp-hw-ops.c     | 23 +++++++++++++++++++
 .../staging/media/starfive/camss/stf-isp.h    | 21 +++++++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
index 3b18d09f2cc6..0bc5e36f952e 100644
--- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
+++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
@@ -300,6 +300,25 @@ static void stf_isp_config_sat(struct stfcamss *stfcamss)
 	stf_isp_reg_write(stfcamss, ISP_REG_YADJ1, YOMAX(0x3ff) | YOMIN(0x1));
 }
 
+static void stf_isp_config_sc(struct stfcamss *stfcamss)
+{
+	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_1, AXI_ID(0));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_0, HSTART(0) | VSTART(0xc));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_1,
+			  SC_WIDTH(0x1d) | SC_HEIGHT(0x15) |
+			  AWB_PS_GRB_BA(0x10) | SEL_TYPE(0x3));
+}
+
+static void stf_isp_config_yhist(struct stfcamss *stfcamss)
+{
+	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_0, 0);
+	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_1,
+			  YH_WIDTH(0x77f) | YH_HEIGHT(0x437));
+	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_2,
+			  YH_DEC_ETW(2) | YH_DEC_ETH(1));
+	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_3, 0);
+}
+
 int stf_isp_reset(struct stf_isp_dev *isp_dev)
 {
 	stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
@@ -332,7 +351,11 @@ void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
 	stf_isp_config_sharpen(isp_dev->stfcamss);
 	stf_isp_config_dnyuv(isp_dev->stfcamss);
 	stf_isp_config_sat(isp_dev->stfcamss);
+	stf_isp_config_sc(isp_dev->stfcamss);
+	stf_isp_config_yhist(isp_dev->stfcamss);
 
+	stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DUMP_CFG_1,
+			  DUMP_BURST_LEN(3) | DUMP_SD(0xb80));
 	stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
 			  CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
 			  CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
index 0af7b367e57a..eca3ba1ade75 100644
--- a/drivers/staging/media/starfive/camss/stf-isp.h
+++ b/drivers/staging/media/starfive/camss/stf-isp.h
@@ -110,9 +110,19 @@
 
 #define ISP_REG_SCD_CFG_0			0x098
 
+#define ISP_REG_SCD_CFG_1			0x09c
+#define AXI_ID(n)				((n) << 24)
+
+#define ISP_REG_SC_CFG_0			0x0b8
+#define VSTART(n)				((n) << 16)
+#define HSTART(n)				((n) << 0)
+
 #define ISP_REG_SC_CFG_1			0x0bc
 #define ISP_SC_SEL_MASK				GENMASK(31, 30)
 #define SEL_TYPE(n)				((n) << 30)
+#define AWB_PS_GRB_BA(n)			((n) << 16)
+#define SC_HEIGHT(n)				((n) << 8)
+#define SC_WIDTH(n)				((n) << 0)
 
 #define ISP_REG_LCCF_CFG_2			0x0e0
 #define ISP_REG_LCCF_CFG_3			0x0e4
@@ -312,6 +322,17 @@
 #define DNRM_F(n)				((n) << 16)
 #define CCM_M_DAT(n)				((n) << 0)
 
+#define ISP_REG_YHIST_CFG_0			0xcc8
+
+#define ISP_REG_YHIST_CFG_1			0xccc
+#define YH_HEIGHT(n)				((n) << 16)
+#define YH_WIDTH(n)				((n) << 0)
+
+#define ISP_REG_YHIST_CFG_2			0xcd0
+#define YH_DEC_ETH(n)				((n) << 16)
+#define YH_DEC_ETW(n)				((n) << 0)
+
+#define ISP_REG_YHIST_CFG_3			0xcd4
 #define ISP_REG_YHIST_CFG_4			0xcd8
 
 #define ISP_REG_YHIST_ACC_0			0xd00
-- 
2.25.1

[PATCH v5 10/14] staging: media: starfive: Add V4L2_CAP_IO_MC capability

[Changhuang Liang]

Add V4L2_CAP_IO_MC capabality for video device. User space can enumerate
formats by mbus code.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 drivers/staging/media/starfive/camss/stf-video.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/media/starfive/camss/stf-video.c b/drivers/staging/media/starfive/camss/stf-video.c
index 2203605ec9c7..8c6c45d8b7a1 100644
--- a/drivers/staging/media/starfive/camss/stf-video.c
+++ b/drivers/staging/media/starfive/camss/stf-video.c
@@ -668,7 +668,7 @@ int stf_video_register(struct stfcamss_video *video,
 	}
 
 	vdev->fops = &stf_vid_fops;
-	vdev->device_caps |= V4L2_CAP_STREAMING;
+	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
 	vdev->entity.ops = &stf_media_ops;
 	vdev->vfl_dir = VFL_DIR_RX;
 	vdev->release = stf_video_release;
-- 
2.25.1

[PATCH v5 11/14] staging: media: starfive: Add ISP params video device

[Changhuang Liang]

Add ISP params video device to write ISP parameters for 3A.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 drivers/staging/media/starfive/camss/Makefile |   2 +
 .../staging/media/starfive/camss/stf-camss.c  |  23 +-
 .../staging/media/starfive/camss/stf-camss.h  |   3 +
 .../media/starfive/camss/stf-isp-params.c     | 240 ++++++++++++++++++
 .../staging/media/starfive/camss/stf-isp.h    |   4 +
 .../staging/media/starfive/camss/stf-output.c |  83 ++++++
 .../staging/media/starfive/camss/stf-output.h |  22 ++
 7 files changed, 376 insertions(+), 1 deletion(-)
 create mode 100644 drivers/staging/media/starfive/camss/stf-isp-params.c
 create mode 100644 drivers/staging/media/starfive/camss/stf-output.c
 create mode 100644 drivers/staging/media/starfive/camss/stf-output.h

diff --git a/drivers/staging/media/starfive/camss/Makefile b/drivers/staging/media/starfive/camss/Makefile
index 411b45f3fb52..077165cbba7a 100644
--- a/drivers/staging/media/starfive/camss/Makefile
+++ b/drivers/staging/media/starfive/camss/Makefile
@@ -9,6 +9,8 @@ starfive-camss-objs += \
 		stf-capture.o \
 		stf-isp.o \
 		stf-isp-hw-ops.o \
+		stf-isp-params.o \
+		stf-output.o \
 		stf-video.o
 
 obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
diff --git a/drivers/staging/media/starfive/camss/stf-camss.c b/drivers/staging/media/starfive/camss/stf-camss.c
index fafa3ce2f6da..4b2288c3199c 100644
--- a/drivers/staging/media/starfive/camss/stf-camss.c
+++ b/drivers/staging/media/starfive/camss/stf-camss.c
@@ -128,6 +128,7 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
 {
 	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
 	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
+	struct stf_output *output = &stfcamss->output;
 	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
 	int ret;
 
@@ -138,13 +139,20 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
 		return ret;
 	}
 
-	ret = stf_capture_register(stfcamss, &stfcamss->v4l2_dev);
+	ret = stf_output_register(stfcamss, &stfcamss->v4l2_dev);
 	if (ret < 0) {
 		dev_err(stfcamss->dev,
 			"failed to register capture: %d\n", ret);
 		goto err_isp_unregister;
 	}
 
+	ret = stf_capture_register(stfcamss, &stfcamss->v4l2_dev);
+	if (ret < 0) {
+		dev_err(stfcamss->dev,
+			"failed to register capture: %d\n", ret);
+		goto err_out_unregister;
+	}
+
 	ret = media_create_pad_link(&isp_dev->subdev.entity, STF_ISP_PAD_SRC,
 				    &cap_yuv->video.vdev.entity, 0, 0);
 	if (ret)
@@ -159,13 +167,23 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
 
 	cap_scd->video.source_subdev = &isp_dev->subdev;
 
+	ret = media_create_pad_link(&output->video.vdev.entity, 0,
+				    &isp_dev->subdev.entity, STF_ISP_PAD_SINK_PARAMS,
+				    0);
+	if (ret)
+		goto err_rm_links1;
+
 	return ret;
 
+err_rm_links1:
+	media_entity_remove_links(&cap_scd->video.vdev.entity);
 err_rm_links0:
 	media_entity_remove_links(&isp_dev->subdev.entity);
 	media_entity_remove_links(&cap_yuv->video.vdev.entity);
 err_cap_unregister:
 	stf_capture_unregister(stfcamss);
+err_out_unregister:
+	stf_output_unregister(stfcamss);
 err_isp_unregister:
 	stf_isp_unregister(&stfcamss->isp_dev);
 
@@ -176,14 +194,17 @@ static void stfcamss_unregister_devs(struct stfcamss *stfcamss)
 {
 	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
 	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
+	struct stf_output *output = &stfcamss->output;
 	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
 
+	media_entity_remove_links(&output->video.vdev.entity);
 	media_entity_remove_links(&isp_dev->subdev.entity);
 	media_entity_remove_links(&cap_yuv->video.vdev.entity);
 	media_entity_remove_links(&cap_scd->video.vdev.entity);
 
 	stf_isp_unregister(&stfcamss->isp_dev);
 	stf_capture_unregister(stfcamss);
+	stf_output_unregister(stfcamss);
 }
 
 static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
diff --git a/drivers/staging/media/starfive/camss/stf-camss.h b/drivers/staging/media/starfive/camss/stf-camss.h
index ae49c7031ab7..3f84f1a1e997 100644
--- a/drivers/staging/media/starfive/camss/stf-camss.h
+++ b/drivers/staging/media/starfive/camss/stf-camss.h
@@ -21,6 +21,7 @@
 #include "stf-buffer.h"
 #include "stf-isp.h"
 #include "stf-capture.h"
+#include "stf-output.h"
 
 enum stf_port_num {
 	STF_PORT_DVP = 0,
@@ -55,6 +56,7 @@ struct stfcamss {
 	struct device *dev;
 	struct stf_isp_dev isp_dev;
 	struct stf_capture captures[STF_CAPTURE_NUM];
+	struct stf_output output;
 	struct v4l2_async_notifier notifier;
 	void __iomem *syscon_base;
 	void __iomem *isp_base;
@@ -132,4 +134,5 @@ static inline void stf_syscon_reg_clear_bit(struct stfcamss *stfcamss,
 	value = ioread32(stfcamss->syscon_base + reg);
 	iowrite32(value & ~bit_mask, stfcamss->syscon_base + reg);
 }
+
 #endif /* STF_CAMSS_H */
diff --git a/drivers/staging/media/starfive/camss/stf-isp-params.c b/drivers/staging/media/starfive/camss/stf-isp-params.c
new file mode 100644
index 000000000000..e015857815f0
--- /dev/null
+++ b/drivers/staging/media/starfive/camss/stf-isp-params.c
@@ -0,0 +1,240 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * stf-isp-params.c
+ *
+ * StarFive Camera Subsystem - V4L2 device node
+ *
+ * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+
+#include "stf-camss.h"
+#include "stf-video.h"
+
+static inline struct stfcamss_buffer *
+to_stfcamss_buffer(struct vb2_v4l2_buffer *vbuf)
+{
+	return container_of(vbuf, struct stfcamss_buffer, vb);
+}
+
+static int stf_isp_params_queue_setup(struct vb2_queue *q,
+				      unsigned int *num_buffers,
+				      unsigned int *num_planes,
+				      unsigned int sizes[],
+				      struct device *alloc_devs[])
+{
+	if (*num_planes)
+		return sizes[0] < sizeof(struct jh7110_isp_params_buffer) ? -EINVAL : 0;
+
+	*num_planes = 1;
+	sizes[0] = sizeof(struct jh7110_isp_params_buffer);
+
+	return 0;
+}
+
+static int stf_isp_params_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
+	dma_addr_t *paddr;
+
+	paddr = vb2_plane_cookie(vb, 0);
+	buffer->addr[0] = *paddr;
+	buffer->vaddr = vb2_plane_vaddr(vb, 0);
+
+	return 0;
+}
+
+static int stf_isp_params_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+	if (sizeof(struct jh7110_isp_params_buffer) > vb2_plane_size(vb, 0))
+		return -EINVAL;
+
+	vb2_set_plane_payload(vb, 0, sizeof(struct jh7110_isp_params_buffer));
+
+	vbuf->field = V4L2_FIELD_NONE;
+
+	return 0;
+}
+
+static void stf_isp_params_buf_queue(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
+	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
+
+	video->ops->queue_buffer(video, buffer);
+}
+
+static void stf_isp_params_stop_streaming(struct vb2_queue *q)
+{
+	struct stfcamss_video *video = vb2_get_drv_priv(q);
+
+	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
+}
+
+static const struct vb2_ops stf_isp_params_vb2_q_ops = {
+	.queue_setup     = stf_isp_params_queue_setup,
+	.wait_prepare    = vb2_ops_wait_prepare,
+	.wait_finish     = vb2_ops_wait_finish,
+	.buf_init        = stf_isp_params_buf_init,
+	.buf_prepare     = stf_isp_params_buf_prepare,
+	.buf_queue       = stf_isp_params_buf_queue,
+	.stop_streaming  = stf_isp_params_stop_streaming,
+};
+
+static int stf_isp_params_init_format(struct stfcamss_video *video)
+{
+	video->active_fmt.fmt.meta.dataformat = V4L2_META_FMT_STF_ISP_PARAMS;
+	video->active_fmt.fmt.meta.buffersize = sizeof(struct jh7110_isp_params_buffer);
+
+	return 0;
+}
+
+static int stf_isp_params_querycap(struct file *file, void *fh,
+				   struct v4l2_capability *cap)
+{
+	strscpy(cap->driver, "starfive-camss", sizeof(cap->driver));
+	strscpy(cap->card, "Starfive Camera Subsystem", sizeof(cap->card));
+
+	return 0;
+}
+
+static int stf_isp_params_enum_fmt(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	struct stfcamss_video *video = video_drvdata(file);
+
+	if (f->index > 0 || f->type != video->type)
+		return -EINVAL;
+
+	f->pixelformat = video->active_fmt.fmt.meta.dataformat;
+	return 0;
+}
+
+static int stf_isp_params_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+	struct stfcamss_video *video = video_drvdata(file);
+	struct v4l2_meta_format *meta = &f->fmt.meta;
+
+	if (f->type != video->type)
+		return -EINVAL;
+
+	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
+	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops stf_isp_params_ioctl_ops = {
+	.vidioc_querycap                = stf_isp_params_querycap,
+	.vidioc_enum_fmt_meta_out	= stf_isp_params_enum_fmt,
+	.vidioc_g_fmt_meta_out          = stf_isp_params_g_fmt,
+	.vidioc_s_fmt_meta_out          = stf_isp_params_g_fmt,
+	.vidioc_try_fmt_meta_out        = stf_isp_params_g_fmt,
+	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
+	.vidioc_querybuf                = vb2_ioctl_querybuf,
+	.vidioc_qbuf                    = vb2_ioctl_qbuf,
+	.vidioc_expbuf                  = vb2_ioctl_expbuf,
+	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
+	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
+	.vidioc_streamon                = vb2_ioctl_streamon,
+	.vidioc_streamoff               = vb2_ioctl_streamoff,
+};
+
+static const struct v4l2_file_operations stf_isp_params_fops = {
+	.owner          = THIS_MODULE,
+	.unlocked_ioctl = video_ioctl2,
+	.open           = v4l2_fh_open,
+	.release        = vb2_fop_release,
+	.poll           = vb2_fop_poll,
+	.mmap           = vb2_fop_mmap,
+};
+
+static void stf_isp_params_release(struct video_device *vdev)
+{
+	struct stfcamss_video *video = video_get_drvdata(vdev);
+
+	media_entity_cleanup(&vdev->entity);
+
+	mutex_destroy(&video->q_lock);
+	mutex_destroy(&video->lock);
+}
+
+int stf_isp_params_register(struct stfcamss_video *video,
+			    struct v4l2_device *v4l2_dev,
+			    const char *name)
+{
+	struct video_device *vdev = &video->vdev;
+	struct vb2_queue *q;
+	struct media_pad *pad = &video->pad;
+	int ret;
+
+	mutex_init(&video->q_lock);
+	mutex_init(&video->lock);
+
+	q = &video->vb2_q;
+	q->drv_priv = video;
+	q->mem_ops = &vb2_dma_contig_memops;
+	q->ops = &stf_isp_params_vb2_q_ops;
+	q->type = video->type;
+	q->io_modes = VB2_DMABUF | VB2_MMAP;
+	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+	q->buf_struct_size = sizeof(struct stfcamss_buffer);
+	q->dev = video->stfcamss->dev;
+	q->lock = &video->q_lock;
+	q->min_queued_buffers = STFCAMSS_MIN_BUFFERS;
+	ret = vb2_queue_init(q);
+	if (ret < 0) {
+		dev_err(video->stfcamss->dev,
+			"Failed to init vb2 queue: %d\n", ret);
+		goto err_mutex_destroy;
+	}
+
+	pad->flags = MEDIA_PAD_FL_SOURCE;
+	ret = media_entity_pads_init(&vdev->entity, 1, pad);
+	if (ret < 0) {
+		dev_err(video->stfcamss->dev,
+			"Failed to init video entity: %d\n", ret);
+		goto err_mutex_destroy;
+	}
+
+	ret = stf_isp_params_init_format(video);
+	if (ret < 0) {
+		dev_err(video->stfcamss->dev,
+			"Failed to init format: %d\n", ret);
+		goto err_media_cleanup;
+	}
+	vdev->ioctl_ops = &stf_isp_params_ioctl_ops;
+	vdev->device_caps = V4L2_CAP_META_OUTPUT;
+	vdev->fops = &stf_isp_params_fops;
+	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
+	vdev->vfl_dir = VFL_DIR_TX;
+	vdev->release = stf_isp_params_release;
+	vdev->v4l2_dev = v4l2_dev;
+	vdev->queue = &video->vb2_q;
+	vdev->lock = &video->lock;
+	strscpy(vdev->name, name, sizeof(vdev->name));
+
+	video_set_drvdata(vdev, video);
+
+	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+	if (ret < 0) {
+		dev_err(video->stfcamss->dev,
+			"Failed to register video device: %d\n", ret);
+		goto err_media_cleanup;
+	}
+
+	return 0;
+
+err_media_cleanup:
+	media_entity_cleanup(&vdev->entity);
+err_mutex_destroy:
+	mutex_destroy(&video->lock);
+	mutex_destroy(&video->q_lock);
+	return ret;
+}
diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
index eca3ba1ade75..76ea943bfe98 100644
--- a/drivers/staging/media/starfive/camss/stf-isp.h
+++ b/drivers/staging/media/starfive/camss/stf-isp.h
@@ -474,4 +474,8 @@ void stf_set_scd_addr(struct stfcamss *stfcamss,
 		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
 		      enum stf_isp_type_scd type_scd);
 
+int stf_isp_params_register(struct stfcamss_video *video,
+			    struct v4l2_device *v4l2_dev,
+			    const char *name);
+
 #endif /* STF_ISP_H */
diff --git a/drivers/staging/media/starfive/camss/stf-output.c b/drivers/staging/media/starfive/camss/stf-output.c
new file mode 100644
index 000000000000..8eaf4979cafa
--- /dev/null
+++ b/drivers/staging/media/starfive/camss/stf-output.c
@@ -0,0 +1,83 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * StarFive Camera Subsystem - output device
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ */
+
+#include "stf-camss.h"
+
+static inline struct stf_output *to_stf_output(struct stfcamss_video *video)
+{
+	return container_of(video, struct stf_output, video);
+}
+
+static int stf_output_queue_buffer(struct stfcamss_video *video,
+				   struct stfcamss_buffer *buf)
+{
+	struct stf_output *output = to_stf_output(video);
+	struct stf_v_buf *v_bufs = &output->buffers;
+	unsigned long flags;
+
+	spin_lock_irqsave(&v_bufs->lock, flags);
+	stf_buf_add_ready(v_bufs, buf);
+	spin_unlock_irqrestore(&v_bufs->lock, flags);
+
+	return 0;
+}
+
+static int stf_output_flush_buffers(struct stfcamss_video *video,
+				    enum vb2_buffer_state state)
+{
+	struct stf_output *output = to_stf_output(video);
+	struct stf_v_buf *v_bufs = &output->buffers;
+	unsigned long flags;
+
+	spin_lock_irqsave(&v_bufs->lock, flags);
+	stf_buf_flush(v_bufs, state);
+	spin_unlock_irqrestore(&v_bufs->lock, flags);
+
+	return 0;
+}
+
+static const struct stfcamss_video_ops stf_output_ops = {
+	.queue_buffer = stf_output_queue_buffer,
+	.flush_buffers = stf_output_flush_buffers,
+};
+
+static void stf_output_init(struct stfcamss *stfcamss, struct stf_output *out)
+{
+	out->buffers.state = STF_OUTPUT_OFF;
+	out->buffers.buf[0] = NULL;
+	out->buffers.buf[1] = NULL;
+	out->buffers.active_buf = 0;
+	INIT_LIST_HEAD(&out->buffers.pending_bufs);
+	INIT_LIST_HEAD(&out->buffers.ready_bufs);
+	spin_lock_init(&out->buffers.lock);
+
+	out->video.stfcamss = stfcamss;
+	out->video.type = V4L2_BUF_TYPE_META_OUTPUT;
+}
+
+void stf_output_unregister(struct stfcamss *stfcamss)
+{
+	struct stf_output *output = &stfcamss->output;
+
+	if (!video_is_registered(&output->video.vdev))
+		return;
+
+	media_entity_cleanup(&output->video.vdev.entity);
+	vb2_video_unregister_device(&output->video.vdev);
+}
+
+int stf_output_register(struct stfcamss *stfcamss,
+			struct v4l2_device *v4l2_dev)
+{
+	struct stf_output *output = &stfcamss->output;
+
+	output->video.ops = &stf_output_ops;
+	stf_output_init(stfcamss, output);
+	stf_isp_params_register(&output->video, v4l2_dev, "output_params");
+
+	return 0;
+}
diff --git a/drivers/staging/media/starfive/camss/stf-output.h b/drivers/staging/media/starfive/camss/stf-output.h
new file mode 100644
index 000000000000..d3591a0b609b
--- /dev/null
+++ b/drivers/staging/media/starfive/camss/stf-output.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Starfive Camera Subsystem driver
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ */
+
+#ifndef STF_OUTPUT_H
+#define STF_OUTPUT_H
+
+#include "stf-video.h"
+
+struct stf_output {
+	struct stfcamss_video video;
+	struct stf_v_buf buffers;
+};
+
+int stf_output_register(struct stfcamss *stfcamss,
+			struct v4l2_device *v4l2_dev);
+void stf_output_unregister(struct stfcamss *stfcamss);
+
+#endif
-- 
2.25.1

[PATCH v5 12/14] staging: media: starfive: Add ISP parameters hardware configure

[Changhuang Liang]

Add ISP parameters hardware configure.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../staging/media/starfive/camss/stf-camss.h  |   8 +
 .../media/starfive/camss/stf-isp-hw-ops.c     | 573 ++++++++++++++++++
 .../staging/media/starfive/camss/stf-isp.h    | 135 +++++
 3 files changed, 716 insertions(+)

diff --git a/drivers/staging/media/starfive/camss/stf-camss.h b/drivers/staging/media/starfive/camss/stf-camss.h
index 3f84f1a1e997..328318d61c6b 100644
--- a/drivers/staging/media/starfive/camss/stf-camss.h
+++ b/drivers/staging/media/starfive/camss/stf-camss.h
@@ -106,6 +106,14 @@ static inline void stf_isp_reg_set(struct stfcamss *stfcamss, u32 reg, u32 mask)
 		  stfcamss->isp_base + reg);
 }
 
+static inline void stf_isp_reg_fill_zero(struct stfcamss *stfcamss, u32 reg, u32 size)
+{
+	u32 i;
+
+	for (i = 0; i < size; i++, reg += 4)
+		iowrite32(0, stfcamss->isp_base + reg);
+}
+
 static inline u32 stf_syscon_reg_read(struct stfcamss *stfcamss, u32 reg)
 {
 	return ioread32(stfcamss->syscon_base + reg);
diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
index 0bc5e36f952e..e54368910906 100644
--- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
+++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
@@ -10,6 +10,25 @@
 
 #include "stf-camss.h"
 
+static const struct stf_isp_module_info mod_info[] = {
+	{ ISP_REG_CSI_MODULE_CFG, 2 },
+	{ ISP_REG_CSI_MODULE_CFG, 4 },
+	{ ISP_REG_CSI_MODULE_CFG, 6 },
+	{ ISP_REG_CSI_MODULE_CFG, 7 },
+	{ ISP_REG_CSI_MODULE_CFG, 17 },
+	{ ISP_REG_ISP_CTRL_1, 1 },
+	{ ISP_REG_ISP_CTRL_1, 2 },
+	{ ISP_REG_ISP_CTRL_1, 3 },
+	{ ISP_REG_ISP_CTRL_1, 4 },
+	{ ISP_REG_ISP_CTRL_1, 5 },
+	{ ISP_REG_ISP_CTRL_1, 7 },
+	{ ISP_REG_ISP_CTRL_1, 8 },
+	{ ISP_REG_ISP_CTRL_1, 17 },
+	{ ISP_REG_ISP_CTRL_1, 19 },
+	{ ISP_REG_ISP_CTRL_1, 21 },
+	{ ISP_REG_ISP_CTRL_1, 22 },
+};
+
 static void stf_isp_config_obc(struct stfcamss *stfcamss)
 {
 	u32 reg_val, reg_add;
@@ -517,6 +536,59 @@ static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
 	}
 }
 
+static void stf_isp_set_params(struct stfcamss *stfcamss, void *vaddr)
+{
+	struct jh7110_isp_params_buffer *params = (struct jh7110_isp_params_buffer *)vaddr;
+
+	if (params->enable_setting & JH7110_ISP_MODULE_WB_SETTING)
+		isp_set_ctrl_wb(stfcamss, &params->wb_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_CAR_SETTING)
+		isp_set_ctrl_car(stfcamss, &params->car_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_CCM_SETTING)
+		isp_set_ctrl_ccm(stfcamss, &params->ccm_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_CFA_SETTING)
+		isp_set_ctrl_cfa(stfcamss, &params->cfa_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_CTC_SETTING)
+		isp_set_ctrl_ctc(stfcamss, &params->ctc_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_DBC_SETTING)
+		isp_set_ctrl_dbc(stfcamss, &params->dbc_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_DNYUV_SETTING)
+		isp_set_ctrl_dnyuv(stfcamss, &params->dnyuv_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_GMARGB_SETTING)
+		isp_set_ctrl_gmargb(stfcamss, &params->gmargb_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_LCCF_SETTING)
+		isp_set_ctrl_lccf(stfcamss, &params->lccf_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_OBC_SETTING)
+		isp_set_ctrl_obc(stfcamss, &params->obc_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_OECF_SETTING)
+		isp_set_ctrl_oecf(stfcamss, &params->oecf_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_R2Y_SETTING)
+		isp_set_ctrl_r2y(stfcamss, &params->r2y_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_SAT_SETTING)
+		isp_set_ctrl_sat(stfcamss, &params->sat_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_SHARP_SETTING)
+		isp_set_ctrl_sharp(stfcamss, &params->sharp_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_YCRV_SETTING)
+		isp_set_ctrl_ycrv(stfcamss, &params->ycrv_setting);
+
+	if (params->enable_setting & JH7110_ISP_MODULE_SC_SETTING)
+		isp_set_ctrl_sc(stfcamss, &params->sc_setting);
+}
+
 irqreturn_t stf_line_irq_handler(int irq, void *priv)
 {
 	struct stfcamss *stfcamss = priv;
@@ -566,11 +638,20 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
 	struct stfcamss *stfcamss = priv;
 	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
 	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
+	struct stf_output *output = &stfcamss->output;
 	struct stfcamss_buffer *ready_buf;
 	u32 status;
 
 	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
 	if (status & ISPC_ISP) {
+		ready_buf = stf_buf_get_ready(&output->buffers);
+		if (ready_buf) {
+			stf_isp_set_params(stfcamss, ready_buf->vaddr);
+			ready_buf->vb.vb2_buf.timestamp = ktime_get_ns();
+			ready_buf->vb.sequence = output->buffers.sequence++;
+			vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+		}
+
 		if (status & ISPC_ENUO) {
 			ready_buf = stf_buf_done(&cap->buffers);
 			if (ready_buf)
@@ -591,4 +672,496 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
 	}
 
 	return IRQ_HANDLED;
+};
+
+int isp_set_ctrl_wb(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_awb];
+	const struct jh7110_isp_wb_setting *setting =
+		(const struct jh7110_isp_wb_setting *)value;
+	const struct jh7110_isp_wb_gain *gains = &setting->gains;
+
+	stf_isp_reg_fill_zero(stfcamss, ISP_REG_AWB_X0_CFG_0, 16);
+
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S0_CFG_0,
+			  AWB_S_SYMBOL_H(gains->gain_r) | AWB_S_SYMBOL_L(gains->gain_r));
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S0_CFG_1,
+			  AWB_S_SYMBOL_H(gains->gain_r) | AWB_S_SYMBOL_L(gains->gain_r));
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S1_CFG_0,
+			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S1_CFG_1,
+			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S2_CFG_0,
+			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S2_CFG_1,
+			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S3_CFG_0,
+			  AWB_S_SYMBOL_H(gains->gain_b) | AWB_S_SYMBOL_L(gains->gain_b));
+	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S3_CFG_1,
+			  AWB_S_SYMBOL_H(gains->gain_b) | AWB_S_SYMBOL_L(gains->gain_b));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_car(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_car];
+	const struct jh7110_isp_car_setting *setting =
+		(const struct jh7110_isp_car_setting *)value;
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_ccm(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_ccm];
+	const struct jh7110_isp_ccm_setting *setting =
+		(const struct jh7110_isp_ccm_setting *)value;
+	const struct jh7110_isp_ccm_smlow *ccm = &setting->ccm_smlow;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_0, DNRM_F(6));
+	stf_isp_reg_fill_zero(stfcamss, ISP_REG_ICAMD_1, 11);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_12, ccm->ccm[0][0]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_13, ccm->ccm[0][1]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_14, ccm->ccm[0][2]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_15, ccm->ccm[1][0]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_16, ccm->ccm[1][1]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_17, ccm->ccm[1][2]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_18, ccm->ccm[2][0]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_19, ccm->ccm[2][1]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_20, ccm->ccm[2][2]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_21, ccm->offsets[0]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_22, ccm->offsets[1]);
+	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_23, ccm->offsets[2]);
+	stf_isp_reg_fill_zero(stfcamss, ISP_REG_ICAMD_24, 2);
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_cfa(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_cfa];
+	const struct jh7110_isp_cfa_setting *setting =
+		(const struct jh7110_isp_cfa_setting *)value;
+	const struct jh7110_isp_cfa_params *cfa = &setting->settings;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_ICFAM,
+			  HV_W(cfa->hv_width) | CROSS_COV(cfa->cross_cov));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_ctc(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_ctc];
+	const struct jh7110_isp_ctc_setting *setting =
+		(const struct jh7110_isp_ctc_setting *)value;
+	const struct jh7110_isp_ctc_params *ctc = &setting->settings;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_ICTC,
+			  MINGT(ctc->min_gt) | MAXGT(ctc->max_gt) |
+			  GF_MODE(ctc->saf_mode | ctc->daf_mode));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_dbc(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_dbc];
+	const struct jh7110_isp_dbc_setting *setting =
+		(const struct jh7110_isp_dbc_setting *)value;
+	const struct jh7110_isp_dbc_params *dbc = &setting->settings;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_IDBC,
+			  BADXT(dbc->bad_xt) | BADGT(dbc->bad_gt));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_dnyuv(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_dnyuv];
+	const struct jh7110_isp_dnyuv_setting *setting =
+		(const struct jh7110_isp_dnyuv_setting *)value;
+	const struct jh7110_isp_dnyuv_params *cfg = &setting->settings;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR0,
+			  YUVSW0(cfg->y_sweight[0]) | YUVSW1(cfg->y_sweight[1]) |
+			  YUVSW2(cfg->y_sweight[2]) | YUVSW3(cfg->y_sweight[3]) |
+			  YUVSW4(cfg->y_sweight[4]) | YUVSW5(cfg->y_sweight[5]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR1,
+			  YUVSW0(cfg->y_sweight[6]) | YUVSW1(cfg->y_sweight[7]) |
+			  YUVSW2(cfg->y_sweight[8]) | YUVSW3(cfg->y_sweight[9]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR0,
+			  YUVSW0(cfg->uv_sweight[0]) | YUVSW1(cfg->uv_sweight[1]) |
+			  YUVSW2(cfg->uv_sweight[2]) | YUVSW3(cfg->uv_sweight[3]) |
+			  YUVSW4(cfg->uv_sweight[4]) | YUVSW5(cfg->uv_sweight[5]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR1,
+			  YUVSW0(cfg->uv_sweight[6]) | YUVSW1(cfg->uv_sweight[7]) |
+			  YUVSW2(cfg->uv_sweight[8]) | YUVSW3(cfg->uv_sweight[9]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR0,
+			  CURVE_D_L(cfg->y_curve[0]) | CURVE_D_H(cfg->y_curve[1]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR1,
+			  CURVE_D_L(cfg->y_curve[2]) | CURVE_D_H(cfg->y_curve[3]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR2,
+			  CURVE_D_L(cfg->y_curve[4]) | CURVE_D_H(cfg->y_curve[5]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR0,
+			  CURVE_D_L(cfg->uv_curve[0]) | CURVE_D_H(cfg->uv_curve[1]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR1,
+			  CURVE_D_L(cfg->uv_curve[2]) | CURVE_D_H(cfg->uv_curve[3]));
+	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR2,
+			  CURVE_D_L(cfg->uv_curve[4]) | CURVE_D_H(cfg->uv_curve[5]));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_gmargb(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_gmargb];
+	const struct jh7110_isp_gmargb_setting *setting =
+		(const struct jh7110_isp_gmargb_setting *)value;
+	const struct jh7110_isp_gmargb_point *curve = setting->curve;
+	u32 reg_addr = ISP_REG_GAMMA_VAL0;
+	u32 i;
+
+	for (i = 0; i < 15; i++, reg_addr += 4)
+		stf_isp_reg_write(stfcamss, reg_addr,
+				  GAMMA_S_VAL(curve[i].sg_val) | GAMMA_VAL(curve[i].g_val));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_lccf(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_lccf];
+	const struct jh7110_isp_lccf_setting *setting =
+		(const struct jh7110_isp_lccf_setting *)value;
+	const struct jh7110_isp_lccf_circle *circle = &setting->circle;
+	const struct jh7110_isp_lccf_curve_param *r_param = &setting->r_param;
+	const struct jh7110_isp_lccf_curve_param *gr_param = &setting->gr_param;
+	const struct jh7110_isp_lccf_curve_param *gb_param = &setting->gb_param;
+	const struct jh7110_isp_lccf_curve_param *b_param = &setting->b_param;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_0,
+			  Y_DISTANCE(circle->center_y) | X_DISTANCE(circle->center_x));
+	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_1,
+			  LCCF_MAX_DIS(circle->radius));
+	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_2,
+			  LCCF_F1_PAR(r_param->f1) | LCCF_F2_PAR(r_param->f2));
+	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_3,
+			  LCCF_F1_PAR(gr_param->f1) | LCCF_F2_PAR(gr_param->f2));
+	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_4,
+			  LCCF_F1_PAR(gb_param->f1) | LCCF_F2_PAR(gb_param->f2));
+	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_5,
+			  LCCF_F1_PAR(b_param->f1) | LCCF_F2_PAR(b_param->f2));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_obc(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_obc];
+	const struct jh7110_isp_obc_setting *setting =
+		(const struct jh7110_isp_obc_setting *)value;
+	const struct jh7110_isp_obc_win_size *win_size = &setting->win_size;
+	const struct jh7110_isp_obc_gain *gain = setting->gain;
+	const struct jh7110_isp_obc_offset *offset = setting->offset;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_OBC_CFG,
+			  OBC_W_W(win_size->width) | OBC_W_H(win_size->height));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_0,
+			  GAIN_A_POINT(gain[0].tl_gain) | GAIN_B_POINT(gain[0].tr_gain) |
+			  GAIN_C_POINT(gain[0].bl_gain) | GAIN_D_POINT(gain[0].br_gain));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_1,
+			  GAIN_A_POINT(gain[1].tl_gain) | GAIN_B_POINT(gain[1].tr_gain) |
+			  GAIN_C_POINT(gain[1].bl_gain) | GAIN_D_POINT(gain[1].br_gain));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_2,
+			  GAIN_A_POINT(gain[2].tl_gain) | GAIN_B_POINT(gain[2].tr_gain) |
+			  GAIN_C_POINT(gain[2].bl_gain) | GAIN_D_POINT(gain[2].br_gain));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_3,
+			  GAIN_A_POINT(gain[3].tl_gain) | GAIN_B_POINT(gain[3].tr_gain) |
+			  GAIN_C_POINT(gain[3].bl_gain) | GAIN_D_POINT(gain[3].br_gain));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_0,
+			  OFFSET_A_POINT(offset[0].tl_offset) |
+			  OFFSET_B_POINT(offset[0].tr_offset) |
+			  OFFSET_C_POINT(offset[0].bl_offset) |
+			  OFFSET_D_POINT(offset[0].br_offset));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_1,
+			  OFFSET_A_POINT(offset[1].tl_offset) |
+			  OFFSET_B_POINT(offset[1].tr_offset) |
+			  OFFSET_C_POINT(offset[1].bl_offset) |
+			  OFFSET_D_POINT(offset[1].br_offset));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_2,
+			  OFFSET_A_POINT(offset[2].tl_offset) |
+			  OFFSET_B_POINT(offset[2].tr_offset) |
+			  OFFSET_C_POINT(offset[2].bl_offset) |
+			  OFFSET_D_POINT(offset[2].br_offset));
+	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_3,
+			  OFFSET_A_POINT(offset[3].tl_offset) |
+			  OFFSET_B_POINT(offset[3].tr_offset) |
+			  OFFSET_C_POINT(offset[3].bl_offset) |
+			  OFFSET_D_POINT(offset[3].br_offset));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_oecf(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_oecf];
+	const struct jh7110_isp_oecf_setting *setting =
+		(const struct jh7110_isp_oecf_setting *)value;
+	const struct jh7110_isp_oecf_point *oecf = setting->r_curve;
+	u32 reg_x_addr = ISP_REG_OECF_X0_CFG0;
+	u32 reg_y_addr = ISP_REG_OECF_Y0_CFG0;
+	u32 reg_s_addr = ISP_REG_OECF_S0_CFG0;
+	u32 i;
+
+	for (i = 0; i < 64; i += 2, reg_x_addr += 4, reg_y_addr += 4, reg_s_addr += 4) {
+		stf_isp_reg_write(stfcamss, reg_x_addr,
+				  OCEF_PAR_L(oecf[i].x) | OCEF_PAR_H(oecf[i + 1].x));
+		stf_isp_reg_write(stfcamss, reg_y_addr,
+				  OCEF_PAR_L(oecf[i].y) | OCEF_PAR_H(oecf[i + 1].y));
+		stf_isp_reg_write(stfcamss, reg_s_addr,
+				  OCEF_PAR_L(oecf[i].slope) | OCEF_PAR_H(oecf[i + 1].slope));
+	}
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_r2y(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_r2y];
+	const struct jh7110_isp_r2y_setting *setting =
+		(const struct jh7110_isp_r2y_setting *)value;
+	const struct jh7110_isp_r2y_matrix *matrix = &setting->matrix;
+	u32 reg_addr = ISP_REG_R2Y_0;
+	u32 i;
+
+	for (i = 0; i < 9; i++, reg_addr += 4)
+		stf_isp_reg_write(stfcamss, reg_addr, matrix->m[i]);
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_sat(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_sat];
+	const struct jh7110_isp_sat_setting *setting =
+		(const struct jh7110_isp_sat_setting *)value;
+	const struct jh7110_isp_sat_info *sat = &setting->sat_info;
+	const struct jh7110_isp_sat_hue_info *hue = &setting->hue_info;
+	const struct jh7110_isp_sat_curve *curve = &setting->curve;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_CS_GAIN,
+			  CMAB(sat->gain_cmab) | CMAD(sat->gain_cmmd));
+	stf_isp_reg_write(stfcamss, ISP_REG_CS_THRESHOLD,
+			  CMB(sat->threshold_cmb) | CMD(sat->threshold_cmd));
+	stf_isp_reg_write(stfcamss, ISP_REG_CS_OFFSET,
+			  UOFF(sat->offset_u) | VOFF(sat->offset_v));
+	stf_isp_reg_write(stfcamss, ISP_REG_CS_SCALE, sat->cmsf);
+	stf_isp_reg_write(stfcamss, ISP_REG_CS_HUE_F,
+			  COS(hue->cos) | SIN(hue->sin));
+	stf_isp_reg_write(stfcamss, ISP_REG_YADJ0,
+			  YIMIN(curve->yi_min) | YOIR(curve->yo_ir));
+	stf_isp_reg_write(stfcamss, ISP_REG_YADJ1,
+			  YOMIN(curve->yo_min) | YOMAX(curve->yo_max));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_sharp(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_shrp];
+	const struct jh7110_isp_sharp_setting *setting =
+		(const struct jh7110_isp_sharp_setting *)value;
+	const struct jh7110_isp_sharp_weight *weight = &setting->weight;
+	const struct jh7110_isp_sharp_strength *strength = &setting->strength;
+	u32 reg_addr = ISP_REG_SHARPEN0;
+	u32 i;
+
+	for (i = 0; i < 4; i++, reg_addr += 4)
+		stf_isp_reg_write(stfcamss, reg_addr,
+				  S_WEIGHT(weight->weight[i]) | S_DELTA(strength->diff[i]));
+
+	for (; i < 15; i++, reg_addr += 4)
+		stf_isp_reg_write(stfcamss, reg_addr, S_WEIGHT(weight->weight[i]));
+
+	reg_addr = ISP_REG_SHARPEN_FS0;
+
+	for (i = 0; i < 3; i++, reg_addr += 4)
+		stf_isp_reg_write(stfcamss, reg_addr,
+				  S_SLOPE(strength->s[i]) | S_FACTOR(strength->f[i]));
+
+	stf_isp_reg_write(stfcamss, ISP_REG_SHARPEN_WN,
+			  WSUM(weight->recip_wei_sum) | NDIRF(setting->ndirf) |
+			  PDIRF(setting->pdirf));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_ycrv(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_ycrv];
+	const struct jh7110_isp_ycrv_setting *setting =
+		(const struct jh7110_isp_ycrv_setting *)value;
+	const struct jh7110_isp_ycrv_curve *curve = &setting->curve;
+	u32 reg_addr = ISP_REG_YCURVE_0;
+	u32 i;
+
+	for (i = 0; i < 64; i++, reg_addr += 4)
+		stf_isp_reg_write(stfcamss, reg_addr, curve->y[i]);
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
+}
+
+int isp_set_ctrl_sc(struct stfcamss *stfcamss, const void *value)
+{
+	const struct stf_isp_module_info *reg_info = &mod_info[imi_sc];
+	const struct jh7110_isp_sc_setting *setting =
+		(const struct jh7110_isp_sc_setting *)value;
+	const struct jh7110_isp_sc_config *crop = &setting->crop_config;
+	const struct jh7110_isp_sc_af_config *af = &setting->af_config;
+	const struct jh7110_isp_sc_awb_config *awb = &setting->awb_config;
+	const struct jh7110_isp_sc_awb_ps *awb_ps = &awb->ps_config;
+	const struct jh7110_isp_sc_awb_ws *awb_ws = &awb->ws_config;
+	const struct jh7110_isp_sc_awb_point *pts = awb->pts;
+	u32 reg_addr0, reg_addr1;
+	u32 i;
+
+	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_1, AXI_ID(1));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_0,
+			  HSTART(crop->h_start) | VSTART(crop->v_start));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_1,
+			  SC_WIDTH(crop->sw_width) | SC_HEIGHT(crop->sw_height) |
+			  AWB_PS_GRB_BA(awb->awb_ps_grb_ba) | SEL_TYPE(awb->sel));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_DEC,
+			  SC_DEC_H_PERIOD(crop->hperiod) | SC_DEC_H_KEEP(crop->hkeep) |
+			  SC_DEC_V_PERIOD(crop->vperiod) | SC_DEC_V_KEEP(crop->vkeep));
+
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_0,
+			  AWB_PS_RL(awb_ps->awb_ps_rl) | AWB_PS_RU(awb_ps->awb_ps_ru) |
+			  AWB_PS_GL(awb_ps->awb_ps_gl) | AWB_PS_GU(awb_ps->awb_ps_gu));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_1,
+			  AWB_PS_BL(awb_ps->awb_ps_bl) | AWB_PS_BU(awb_ps->awb_ps_bu) |
+			  AWB_PS_YL(awb_ps->awb_ps_yl) | AWB_PS_YU(awb_ps->awb_ps_yu));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_2,
+			  AWB_PS_GRL(awb_ps->awb_ps_grl) | AWB_PS_GRU(awb_ps->awb_ps_gru));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_3,
+			  AWB_PS_GBL(awb_ps->awb_ps_gbl) | AWB_PS_GBU(awb_ps->awb_ps_gbu));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_4,
+			  AWB_PS_GRBL(awb_ps->awb_ps_grbl) | AWB_PS_GRBU(awb_ps->awb_ps_grbu));
+
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AF,
+			  AF_ES_HM(af->es_hor_mode) | AF_ES_SM(af->es_sum_mode) |
+			  AF_ES_HE(af->hor_en) | AF_ES_VE(af->ver_en) |
+			  AF_ES_VTHR(af->es_ver_thr) | AF_ES_HTHR(af->es_hor_thr));
+
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_CFG_0,
+			  AWB_WS_RL(awb_ws->awb_ws_rl) | AWB_WS_RU(awb_ws->awb_ws_ru) |
+			  AWB_WS_GRL(awb_ws->awb_ws_grl) | AWB_WS_GRU(awb_ws->awb_ws_gru));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_CFG_1,
+			  AWB_WS_GBL(awb_ws->awb_ws_gbl) | AWB_WS_GBU(awb_ws->awb_ws_gbu) |
+			  AWB_WS_BL(awb_ws->awb_ws_bl) | AWB_WS_BU(awb_ws->awb_ws_bu));
+
+	reg_addr0 = ISP_REG_SC_AWB_WS_CW0_CFG_0;
+	reg_addr1 = ISP_REG_SC_AWB_WS_CW0_CFG_1;
+
+	for (i = 0; i < 13; i++, reg_addr0 += 8, reg_addr1 += 8) {
+		stf_isp_reg_write(stfcamss, reg_addr0,
+				  AWB_WS_CW_W_0(awb->awb_cw[13 * i]) |
+				  AWB_WS_CW_W_1(awb->awb_cw[13 * i + 1]) |
+				  AWB_WS_CW_W_2(awb->awb_cw[13 * i + 2]) |
+				  AWB_WS_CW_W_3(awb->awb_cw[13 * i + 3]) |
+				  AWB_WS_CW_W_4(awb->awb_cw[13 * i + 4]) |
+				  AWB_WS_CW_W_5(awb->awb_cw[13 * i + 5]) |
+				  AWB_WS_CW_W_6(awb->awb_cw[13 * i + 6]) |
+				  AWB_WS_CW_W_7(awb->awb_cw[13 * i + 7]));
+		stf_isp_reg_write(stfcamss, reg_addr1,
+				  AWB_WS_CW_W_0(awb->awb_cw[13 * i + 8]) |
+				  AWB_WS_CW_W_1(awb->awb_cw[13 * i + 9]) |
+				  AWB_WS_CW_W_2(awb->awb_cw[13 * i + 10]) |
+				  AWB_WS_CW_W_3(awb->awb_cw[13 * i + 11]) |
+				  AWB_WS_CW_W_4(awb->awb_cw[13 * i + 12]));
+	}
+
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWV_CFG_0,
+			  AWB_WS_IW_V_0(pts[0].weight) | AWB_WS_IW_V_1(pts[1].weight) |
+			  AWB_WS_IW_V_2(pts[2].weight) | AWB_WS_IW_V_3(pts[3].weight) |
+			  AWB_WS_IW_V_4(pts[4].weight) | AWB_WS_IW_V_5(pts[5].weight) |
+			  AWB_WS_IW_V_6(pts[6].weight) | AWB_WS_IW_V_7(pts[7].weight));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWV_CFG_1,
+			  AWB_WS_IW_V_0(pts[8].weight) | AWB_WS_IW_V_1(pts[9].weight) |
+			  AWB_WS_IW_V_2(pts[10].weight) | AWB_WS_IW_V_3(pts[11].weight) |
+			  AWB_WS_IW_V_4(pts[12].weight) | AWB_WS_IW_V_5(pts[13].weight) |
+			  AWB_WS_IW_V_6(pts[14].weight) | AWB_WS_IW_V_7(pts[15].weight));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_0,
+			  AWB_WS_IW_S_0(2 * (pts[1].weight - pts[0].weight)) |
+			  AWB_WS_IW_S_1(2 * (pts[2].weight - pts[1].weight)) |
+			  AWB_WS_IW_S_2(2 * (pts[3].weight - pts[2].weight)) |
+			  AWB_WS_IW_S_3(2 * (pts[4].weight - pts[3].weight)));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_1,
+			  AWB_WS_IW_S_0(2 * (pts[5].weight - pts[4].weight)) |
+			  AWB_WS_IW_S_1(2 * (pts[6].weight - pts[5].weight)) |
+			  AWB_WS_IW_S_2(2 * (pts[7].weight - pts[6].weight)) |
+			  AWB_WS_IW_S_3(2 * (pts[8].weight - pts[7].weight)));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_2,
+			  AWB_WS_IW_S_0(2 * (pts[9].weight - pts[8].weight)) |
+			  AWB_WS_IW_S_1(2 * (pts[10].weight - pts[9].weight)) |
+			  AWB_WS_IW_S_2(2 * (pts[11].weight - pts[10].weight)) |
+			  AWB_WS_IW_S_3(2 * (pts[12].weight - pts[11].weight)));
+	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_3,
+			  AWB_WS_IW_S_0(2 * (pts[13].weight - pts[12].weight)) |
+			  AWB_WS_IW_S_1(2 * (pts[14].weight - pts[13].weight)) |
+			  AWB_WS_IW_S_2(2 * (pts[15].weight - pts[14].weight)) |
+			  AWB_WS_IW_S_3(2 * (pts[16].weight - pts[15].weight)));
+
+	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
+			    setting->enabled ? 1 << reg_info->en_nbit : 0);
+
+	return 0;
 }
diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
index 76ea943bfe98..80c4571dc522 100644
--- a/drivers/staging/media/starfive/camss/stf-isp.h
+++ b/drivers/staging/media/starfive/camss/stf-isp.h
@@ -124,6 +124,44 @@
 #define SC_HEIGHT(n)				((n) << 8)
 #define SC_WIDTH(n)				((n) << 0)
 
+#define ISP_REG_SC_AF				0x0c0
+#define AF_ES_HTHR(n)				((n) << 16)
+#define AF_ES_VTHR(n)				((n) << 8)
+#define AF_ES_VE(n)				((n) << 3)
+#define AF_ES_HE(n)				((n) << 2)
+#define AF_ES_SM(n)				((n) << 1)
+#define AF_ES_HM(n)				((n) << 0)
+
+#define ISP_REG_SC_AWB_PS_CFG_0			0x0c4
+#define AWB_PS_GU(n)				((n) << 24)
+#define AWB_PS_GL(n)				((n) << 16)
+#define AWB_PS_RU(n)				((n) << 8)
+#define AWB_PS_RL(n)				((n) << 0)
+
+#define ISP_REG_SC_AWB_PS_CFG_1			0x0c8
+#define AWB_PS_YU(n)				((n) << 24)
+#define AWB_PS_YL(n)				((n) << 16)
+#define AWB_PS_BU(n)				((n) << 8)
+#define AWB_PS_BL(n)				((n) << 0)
+
+#define ISP_REG_SC_AWB_PS_CFG_2			0x0cc
+#define AWB_PS_GRU(n)				((n) << 16)
+#define AWB_PS_GRL(n)				((n) << 0)
+
+#define ISP_REG_SC_AWB_PS_CFG_3			0x0d0
+#define AWB_PS_GBU(n)				((n) << 16)
+#define AWB_PS_GBL(n)				((n) << 0)
+
+#define ISP_REG_SC_AWB_PS_CFG_4			0x0d4
+#define AWB_PS_GRBU(n)				((n) << 16)
+#define AWB_PS_GRBL(n)				((n) << 0)
+
+#define ISP_REG_SC_DEC				0x0d8
+#define SC_DEC_V_KEEP(n)			((n) << 24)
+#define SC_DEC_V_PERIOD(n)			((n) << 16)
+#define SC_DEC_H_KEEP(n)			((n) << 8)
+#define SC_DEC_H_PERIOD(n)			((n) << 0)
+
 #define ISP_REG_LCCF_CFG_2			0x0e0
 #define ISP_REG_LCCF_CFG_3			0x0e4
 #define ISP_REG_LCCF_CFG_4			0x0e8
@@ -140,6 +178,8 @@
 #define ISP_REG_OECF_X0_CFG6			0x118
 #define ISP_REG_OECF_X0_CFG7			0x11c
 
+#define ISP_REG_OECF_Y0_CFG0			0x180
+
 #define ISP_REG_OECF_Y3_CFG0			0x1e0
 #define ISP_REG_OECF_Y3_CFG1			0x1e4
 #define ISP_REG_OECF_Y3_CFG2			0x1e8
@@ -204,6 +244,49 @@
 #define OFFSET_B_POINT(x)			((x) << 8)
 #define OFFSET_A_POINT(x)			((x) << 0)
 
+#define ISP_REG_SC_AWB_WS_CW0_CFG_0		0x4d0
+#define ISP_REG_SC_AWB_WS_CW0_CFG_1		0x4d4
+#define AWB_WS_CW_W_7(x)			((x) << 28)
+#define AWB_WS_CW_W_6(x)			((x) << 24)
+#define AWB_WS_CW_W_5(x)			((x) << 20)
+#define AWB_WS_CW_W_4(x)			((x) << 16)
+#define AWB_WS_CW_W_3(x)			((x) << 12)
+#define AWB_WS_CW_W_2(x)			((x) << 8)
+#define AWB_WS_CW_W_1(x)			((x) << 4)
+#define AWB_WS_CW_W_0(x)			((x) << 0)
+
+#define ISP_REG_SC_AWB_WS_IWV_CFG_0		0x538
+#define ISP_REG_SC_AWB_WS_IWV_CFG_1		0x53c
+#define AWB_WS_IW_V_7(x)			((x) << 28)
+#define AWB_WS_IW_V_6(x)			((x) << 24)
+#define AWB_WS_IW_V_5(x)			((x) << 20)
+#define AWB_WS_IW_V_4(x)			((x) << 16)
+#define AWB_WS_IW_V_3(x)			((x) << 12)
+#define AWB_WS_IW_V_2(x)			((x) << 8)
+#define AWB_WS_IW_V_1(x)			((x) << 4)
+#define AWB_WS_IW_V_0(x)			((x) << 0)
+
+#define ISP_REG_SC_AWB_WS_IWS_CFG_0		0x540
+#define ISP_REG_SC_AWB_WS_IWS_CFG_1		0x544
+#define ISP_REG_SC_AWB_WS_IWS_CFG_2		0x548
+#define ISP_REG_SC_AWB_WS_IWS_CFG_3		0x54c
+#define AWB_WS_IW_S_3(x)			((x) << 24)
+#define AWB_WS_IW_S_2(x)			((x) << 16)
+#define AWB_WS_IW_S_1(x)			((x) << 8)
+#define AWB_WS_IW_S_0(x)			((x) << 0)
+
+#define ISP_REG_SC_AWB_WS_CFG_0			0x5d0
+#define AWB_WS_GRU(n)				((n) << 24)
+#define AWB_WS_GRL(n)				((n) << 16)
+#define AWB_WS_RU(n)				((n) << 8)
+#define AWB_WS_RL(n)				((n) << 0)
+
+#define ISP_REG_SC_AWB_WS_CFG_1			0x5d4
+#define AWB_WS_BU(n)				((n) << 24)
+#define AWB_WS_BL(n)				((n) << 16)
+#define AWB_WS_GBU(n)				((n) << 8)
+#define AWB_WS_GBL(n)				((n) << 0)
+
 #define ISP_REG_ISP_CTRL_0			0xa00
 #define ISPC_LINE				BIT(27)
 #define ISPC_SC					BIT(26)
@@ -315,8 +398,19 @@
 #define CURVE_D_L(n)				((n) << 0)
 
 #define ISP_REG_ICAMD_0				0xc40
+#define ISP_REG_ICAMD_1				0xc44
 #define ISP_REG_ICAMD_12			0xc70
+#define ISP_REG_ICAMD_13			0xc74
+#define ISP_REG_ICAMD_14			0xc78
+#define ISP_REG_ICAMD_15			0xc7c
+#define ISP_REG_ICAMD_16			0xc80
+#define ISP_REG_ICAMD_17			0xc84
+#define ISP_REG_ICAMD_18			0xc88
+#define ISP_REG_ICAMD_19			0xc8c
 #define ISP_REG_ICAMD_20			0xc90
+#define ISP_REG_ICAMD_21			0xc94
+#define ISP_REG_ICAMD_22			0xc98
+#define ISP_REG_ICAMD_23			0xc9c
 #define ISP_REG_ICAMD_24			0xca0
 #define ISP_REG_ICAMD_25			0xca4
 #define DNRM_F(n)				((n) << 16)
@@ -439,6 +533,30 @@ enum stf_isp_pad_id {
 	STF_ISP_PAD_MAX
 };
 
+enum stf_isp_modules_index {
+	imi_obc = 0,
+	imi_oecf,
+	imi_lccf,
+	imi_awb,
+	imi_sc,
+	imi_cfa,
+	imi_car,
+	imi_ccm,
+	imi_gmargb,
+	imi_r2y,
+	imi_shrp,
+	imi_sat,
+	imi_dnyuv,
+	imi_ycrv,
+	imi_ctc,
+	imi_dbc,
+};
+
+struct stf_isp_module_info {
+	u32 en_reg;
+	u8 en_nbit;
+};
+
 struct stf_isp_format {
 	u32 code;
 	u8 bpp;
@@ -478,4 +596,21 @@ int stf_isp_params_register(struct stfcamss_video *video,
 			    struct v4l2_device *v4l2_dev,
 			    const char *name);
 
+int isp_set_ctrl_wb(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_car(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_ccm(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_cfa(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_ctc(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_dbc(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_dnyuv(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_gmargb(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_lccf(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_obc(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_oecf(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_r2y(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_sat(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_sharp(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_ycrv(struct stfcamss *stfcamss, const void *value);
+int isp_set_ctrl_sc(struct stfcamss *stfcamss, const void *value);
+
 #endif /* STF_ISP_H */
-- 
2.25.1

[PATCH v5 13/14] staging: media: starfive: Drop read support for video capture devices

[Changhuang Liang]

StarFive video capture devices is not support VB2_READ queue access
method. Drop the implementation of the read hook.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 drivers/staging/media/starfive/camss/stf-video.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/staging/media/starfive/camss/stf-video.c b/drivers/staging/media/starfive/camss/stf-video.c
index 8c6c45d8b7a1..28e6679ee9b9 100644
--- a/drivers/staging/media/starfive/camss/stf-video.c
+++ b/drivers/staging/media/starfive/camss/stf-video.c
@@ -570,7 +570,6 @@ static const struct v4l2_file_operations stf_vid_fops = {
 	.release        = vb2_fop_release,
 	.poll           = vb2_fop_poll,
 	.mmap           = vb2_fop_mmap,
-	.read           = vb2_fop_read,
 };
 
 /* -----------------------------------------------------------------------------
-- 
2.25.1

[PATCH v5 14/14] admin-guide: media: Update documents for StarFive Camera Subsystem

[Changhuang Liang]

Add ISP output parameters and 3A statistisc collection data video
device for documents.

Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
---
 .../admin-guide/media/starfive_camss.rst      | 11 +++++++---
 .../media/starfive_camss_graph.dot            | 22 +++++++++++--------
 2 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
index ca42e9447c47..020f1969e67f 100644
--- a/Documentation/admin-guide/media/starfive_camss.rst
+++ b/Documentation/admin-guide/media/starfive_camss.rst
@@ -58,15 +58,20 @@ The media controller pipeline graph is as follows:
     :alt:   starfive_camss_graph.dot
     :align: center
 
-The driver has 2 video devices:
+The driver has 4 video devices:
 
+- output_params: The meta output device, transmitting the parameters to ISP
+  module.
 - capture_raw: The capture device, capturing image data directly from a sensor.
 - capture_yuv: The capture device, capturing YUV frame data processed by the
-  ISP module
+  ISP module.
+- capture_scd: The meta capture device, capturing 3A statistics collection data
+  processed by the ISP module.
 
 The driver has 3 subdevices:
 
-- stf_isp: is responsible for all the isp operations, outputs YUV frames.
+- stf_isp: is responsible for all the isp operations, outputs YUV frames
+  and 3A statistics collection data.
 - cdns_csi2rx: a CSI-2 bridge supporting up to 4 CSI lanes in input, and 4
   different pixel streams in output.
 - imx219: an image sensor, image data is sent through MIPI CSI-2.
diff --git a/Documentation/admin-guide/media/starfive_camss_graph.dot b/Documentation/admin-guide/media/starfive_camss_graph.dot
index 8eff1f161ac7..7961255d3ad6 100644
--- a/Documentation/admin-guide/media/starfive_camss_graph.dot
+++ b/Documentation/admin-guide/media/starfive_camss_graph.dot
@@ -1,12 +1,16 @@
 digraph board {
 	rankdir=TB
-	n00000001 [label="{{<port0> 0} | stf_isp\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
-	n00000001:port1 -> n00000008 [style=dashed]
-	n00000004 [label="capture_raw\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
-	n00000008 [label="capture_yuv\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
-	n0000000e [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
-	n0000000e:port1 -> n00000001:port0 [style=dashed]
-	n0000000e:port1 -> n00000004 [style=dashed]
-	n00000018 [label="{{} | imx219 6-0010\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
-	n00000018:port0 -> n0000000e:port0 [style=bold]
+	n00000001 [label="{{<port0> 0 | <port1> 1} | stf_isp\n/dev/v4l-subdev0 | {<port2> 2 | <port3> 3}}", shape=Mrecord, style=filled, fillcolor=green]
+	n00000001:port2 -> n0000000e
+	n00000001:port3 -> n00000012 [style=dashed]
+	n00000006 [label="output_params\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+	n00000006 -> n00000001:port1 [style=dashed]
+	n0000000a [label="capture_raw\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
+	n0000000e [label="capture_yuv\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
+	n00000012 [label="capture_scd\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
+	n0000001c [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n/dev/v4l-subdev1 | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
+	n0000001c:port1 -> n00000001:port0 [style=dashed]
+	n0000001c:port1 -> n0000000a [style=dashed]
+	n00000026 [label="{{} | imx219 6-0010\n/dev/v4l-subdev2 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+	n00000026:port0 -> n0000001c:port0 [style=bold]
 }
-- 
2.25.1

Re: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Jacopo Mondi]

Hello Changhuang

On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> Add JH7110 ISP module definitions for user space.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> ---
>  MAINTAINERS                     |   1 +
>  include/uapi/linux/jh7110-isp.h | 739 ++++++++++++++++++++++++++++++++

With the recently merged support for the RaspberryPi PiSP BE we have
introduced include/uapi/linux/media/raspberry.

Would you consider placing this in
include/uapi/linux/media/startfive/ ?

>  2 files changed, 740 insertions(+)
>  create mode 100644 include/uapi/linux/jh7110-isp.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 507f04a80499..890604eb0d64 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -21305,6 +21305,7 @@ S:	Maintained
>  F:	Documentation/admin-guide/media/starfive_camss.rst
>  F:	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
>  F:	drivers/staging/media/starfive/camss
> +F:	include/uapi/linux/jh7110-isp.h
>
>  STARFIVE CRYPTO DRIVER
>  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> diff --git a/include/uapi/linux/jh7110-isp.h b/include/uapi/linux/jh7110-isp.h
> new file mode 100644
> index 000000000000..4939cd63e771
> --- /dev/null
> +++ b/include/uapi/linux/jh7110-isp.h
> @@ -0,0 +1,739 @@
> +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> +/*
> + * jh7110-isp.h
> + *
> + * JH7110 ISP driver - user space header file.
> + *
> + * Copyright © 2023 StarFive Technology Co., Ltd.
> + *
> + * Author: Zejian Su <zejian.su@starfivetech.com>
> + *
> + */
> +
> +#ifndef __JH7110_ISP_H_
> +#define __JH7110_ISP_H_
> +

Do you need to include

#include <linux/types.h>

> +/**

Is this kernel-doc or a single * would do ?

> + * ISP Module Diagram
> + * ------------------
> + *
> + *  Raw  +-----+    +------+    +------+    +----+
> + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> + *       +-----+    +------+    +------+    +----+     |
> + *                                                     |
> + *  +--------------------------------------------------+
> + *  |
> + *  |    +-----+    +-----+    +-----+    +-----+
> + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> + *       +-----+    +-----+    +-----+    +-----+      |
> + *                                                     |
> + *  +--------------------------------------------------+
> + *  |
> + *  |    +-----+    +--------+    +-----+    +------+
> + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> + *       +-----+    +--------+    +-----+    +------+  |
> + *                                                     |
> + *  +--------------------------------------------------+
> + *  |
> + *  |    +-------+    +-------+    +-----+    +----+
> + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> + *       +-------+    +-------+    +-----+    +----+
> + *
> + */
> +
> +/* Auto White Balance */
> +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> +/* Color Artifact Removal */
> +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> +/* Color Correction Matrix */
> +#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
> +/* Color Filter Arrays */
> +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> +/* Crosstalk Correction */
> +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> +/* Defect Bad Pixel Correction */
> +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> +/* Denoise YUV */
> +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
> +/* RGB Gamma */
> +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
> +/* Lens Correction Cosine Fourth */
> +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
> +/* Optical Black Correction */
> +#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
> +/* Opto-Electric Conversion Function */
> +#define JH7110_ISP_MODULE_OECF_SETTING			(1U << 10)
> +/* RGB To YUV */
> +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U << 11)
> +/* Saturation */
> +#define JH7110_ISP_MODULE_SAT_SETTING			(1U << 12)
> +/* Sharpen */
> +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U << 13)
> +/* Y Curve */
> +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U << 14)
> +/* Statistics Collection */
> +#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)
> +
> +/**
> + * struct jh7110_isp_wb_gain - auto white balance gain
> + *
> + * @gain_r: gain value for red component.
> + * @gain_g: gain value for green component.
> + * @gain_b: gain value for blue component.
> + */
> +struct jh7110_isp_wb_gain {
> +	__u16 gain_r;
> +	__u16 gain_g;
> +	__u16 gain_b;
> +};
> +
> +/**
> + * struct jh7110_isp_wb_setting - Configuration used by auto white balance gain
> + *
> + * @enabled: enabled setting flag.
> + * @gains: auto white balance gain setting.
> + */
> +struct jh7110_isp_wb_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_wb_gain gains;
> +};
> +
> +/**
> + * struct jh7110_isp_car_setting - Configuration used by color artifact removal
> + *
> + * @enabled: enabled setting flag.
> + */
> +struct jh7110_isp_car_setting {
> +	__u32 enabled;
> +};
> +
> +/**
> + * struct jh7110_isp_ccm_smlow - Color correction matrix
> + *
> + * @ccm: color transform matrix with size 3 by 3.
> + * @offsets: the offsets of R, G, B after the transform by the ccm.
> + */
> +struct jh7110_isp_ccm_smlow {
> +	__s32 ccm[3][3];
> +	__s32 offsets[3];
> +};
> +
> +/**
> + * struct jh7110_isp_ccm_setting - Configuration used by color correction matrix
> + *
> + * @enabled: enabled setting flag.
> + * @ccm_smlow: Color correction matrix.
> + */
> +struct jh7110_isp_ccm_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_ccm_smlow ccm_smlow;
> +};
> +
> +/**
> + * struct jh7110_isp_cfa_params - demosaic parameters
> + *
> + * @hv_width: detail smooth factor
> + * @cross_cov: Cross covariance weighting.
> + */
> +struct jh7110_isp_cfa_params {
> +	__s32 hv_width;
> +	__s32 cross_cov;
> +};
> +
> +/**
> + * struct jh7110_isp_cfa_params - Configuration used by demosaic module
> + *
> + * @enabled: enabled setting flag.
> + * @settings: demosaic parameters.
> + */
> +struct jh7110_isp_cfa_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_cfa_params settings;
> +};
> +
> +/**
> + * struct jh7110_isp_ctc_params - crosstalk remove parameters
> + *
> + * @saf_mode: smooth area filter mode.
> + * @daf_mode: detail area filter mode.
> + * @max_gt: the threshold for imbalance detection when pixel intensity is close to maximum.

You could easily make this < 80 cols (here and in other places).

I know there are different opinions on how strict on the 80 cols limit
we should be, so up to you.

> + * @min_gt: the threshold for imbalance detection when pixel intensity is close to 0.
> + */
> +struct jh7110_isp_ctc_params {
> +	__u8 saf_mode;
> +	__u8 daf_mode;
> +	__s32 max_gt;
> +	__s32 min_gt;
> +};
> +
> +/**
> + * struct jh7110_isp_ctc_params - Configuration used by crosstalk remove
> + *
> + * @enabled: enabled setting flag.
> + * @settings: corsstalk remove parameters.

crosstalk

> + */
> +struct jh7110_isp_ctc_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_ctc_params settings;
> +};
> +
> +/**
> + * struct jh7110_isp_dbc_params - defect pixels correction parameters
> + *
> + * @bad_gt: bad pixel threshold for the green channel.
> + * @bad_xt: bad pixel threshold for the red and blue channels.
> + */
> +struct jh7110_isp_dbc_params {
> +	__s32 bad_gt;
> +	__s32 bad_xt;
> +};
> +
> +/**
> + * struct jh7110_isp_dbc_params - Configuration used by defect bad pixels correction
> + *
> + * @enabled: enabled setting flag.
> + * @settings: defect pixels correction parameters.
> + */
> +struct jh7110_isp_dbc_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_dbc_params settings;
> +};
> +
> +/**
> + * struct jh7110_isp_dnyuv_params - yuv domain denoise parameters
> + *
> + * @y_sweight: ten coefficients of 7x7 spatial filter for Y channel.
> + * @y_curve: intensity difference (similarity) weight lookup table for Y channel.
> + * @uv_sweight: ten coefficients of 7x7 spatial filter for U and V channel.
> + * @uv_curve: intensity difference (similarity) weight lookup table for U and V channel.
> + */
> +struct jh7110_isp_dnyuv_params {
> +	__u8 y_sweight[10];
> +	__u16 y_curve[7];
> +	__u8 uv_sweight[10];
> +	__u16 uv_curve[7];
> +};
> +
> +/**
> + * struct jh7110_isp_dnyuv_params - Configuration used by yuv domain denoise
> + *
> + * @enabled: enabled setting flag.
> + * @settings: yuv domain denoise parameters.
> + */
> +struct jh7110_isp_dnyuv_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_dnyuv_params settings;
> +};
> +
> +/**
> + * struct jh7110_isp_gmargb_point - RGB Gamma point
> + *
> + * @g_val: RGB gamma value.
> + * @sg_val: RGB gamma slope value.
> + */
> +struct jh7110_isp_gmargb_point {
> +	__u16 g_val;
> +	__u16 sg_val;
> +};
> +
> +/**
> + * struct jh7110_isp_gmargb_setting - Configuration used by RGB gamma
> + *
> + * @enabled: enabled setting flag.
> + * @curve: RGB Gamma point table.
> + */
> +struct jh7110_isp_gmargb_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_gmargb_point curve[15];
> +};
> +
> +/**
> + * struct jh7110_isp_lccf_circle - len circle

lens ?

> + *
> + * @center_x: center X distance from capture window.
> + * @center_y: center Y distance from capture window.
> + * @radius: len circle radius.

here as well

> + */
> +struct jh7110_isp_lccf_circle {
> +	__s16 center_x;
> +	__s16 center_y;
> +	__u8 radius;
> +};
> +
> +/**
> + * struct jh7110_isp_lccf_curve_param - lens correction cosine fourth curve param
> + *
> + * @f1: F1 parameter.
> + * @f2: F2 parameter.
> + */
> +struct jh7110_isp_lccf_curve_param {
> +	__s16 f1;
> +	__s16 f2;
> +};
> +
> +/**
> + * struct jh7110_isp_lccf_setting - Configuration used by lens correction cosine fourth
> + *
> + * @enabled: enabled setting flag.
> + * @circle: len circle.
> + * @r_param: lens correction cosine fourth curve param for Bayer pattern R.
> + * @gr_param: lens correction cosine fourth curve param for Bayer pattern Gr.
> + * @gb_param: lens correction cosine fourth curve param for Bayer pattern Gb.
> + * @b_param: lens correction cosine fourth curve param for Bayer pattern B.
> + */
> +struct jh7110_isp_lccf_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_lccf_circle circle;
> +	struct jh7110_isp_lccf_curve_param r_param;
> +	struct jh7110_isp_lccf_curve_param gr_param;
> +	struct jh7110_isp_lccf_curve_param gb_param;
> +	struct jh7110_isp_lccf_curve_param b_param;
> +};
> +
> +/**
> + * struct jh7110_isp_obc_win_size - optical black correction window size
> + *
> + * @width: window width.
> + * @height: window height.
> + */
> +struct jh7110_isp_obc_win_size {
> +	__u32 width;
> +	__u32 height;
> +};
> +
> +/**
> + * struct jh7110_isp_obc_gain - optical black correction symbol gain
> + *
> + * @tl_gain: gain at point A for symbol.
> + * @tr_gain: gain at point B for symbol.
> + * @bl_gain: gain at point C for symbol.
> + * @br_gain: gain at point D for symbol.
> + */
> +struct jh7110_isp_obc_gain {
> +	__u8 tl_gain;
> +	__u8 tr_gain;
> +	__u8 bl_gain;
> +	__u8 br_gain;
> +};
> +
> +/**
> + * struct jh7110_isp_obc_offset - optical black correction symbol offset
> + *
> + * @tl_offset: offset at point A for symbol.
> + * @tr_offset: offset at point B for symbol.
> + * @bl_offset: offset at point C for symbol.
> + * @br_offset: offset at point D for symbol.
> + */
> +struct jh7110_isp_obc_offset {
> +	__u8 tl_offset;
> +	__u8 tr_offset;
> +	__u8 bl_offset;
> +	__u8 br_offset;
> +};
> +
> +/**
> + * struct jh7110_isp_obc_setting - Configuration used by optical black correction
> + *
> + * @enabled: enabled setting flag.
> + * @win_size: optical black correction window size.
> + * @gain: optical black correction symbol gain.
> + * @offset: optical black correction symbol offset.
> + */
> +struct jh7110_isp_obc_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_obc_win_size win_size;
> +	struct jh7110_isp_obc_gain gain[4];
> +	struct jh7110_isp_obc_offset offset[4];
> +};
> +
> +/**
> + * struct jh7110_isp_oecf_point - oecf curve
> + *
> + * @x: x coordinate.
> + * @y: y coordinate.
> + * @slope: the slope between this point and the next point.
> + */
> +struct jh7110_isp_oecf_point {
> +	__u16 x;
> +	__u16 y;
> +	__s16 slope;
> +};
> +
> +/**
> + * struct jh7110_isp_oecf_setting - Configuration used by opto-electric conversion function
> + *
> + * @enabled: enabled setting flag.
> + * @r_curve: red pixel oecf curve.
> + * @gr_curve: green pixel oecf curve in GR line.
> + * @gb_curve: green pixel oecf curve in GB line.
> + * @b_curve: blue pixel oecf curve.
> + */
> +struct jh7110_isp_oecf_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_oecf_point r_curve[16];
> +	struct jh7110_isp_oecf_point gr_curve[16];
> +	struct jh7110_isp_oecf_point gb_curve[16];
> +	struct jh7110_isp_oecf_point b_curve[16];
> +};
> +
> +/**
> + * struct jh7110_isp_r2y_matrix - RGB to YUV color conversion matrix
> + *
> + * @m: The 3x3 color conversion matrix coefficient.
> + */
> +struct jh7110_isp_r2y_matrix {
> +	__s16 m[9];
> +};
> +
> +/**
> + * struct jh7110_isp_r2y_setting - Configuration used by RGB To YUV
> + *
> + * @enabled: enabled setting flag.
> + * @matrix: RGB to YUV color conversion matrix.
> + */
> +struct jh7110_isp_r2y_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_r2y_matrix matrix;
> +};
> +
> +/**
> + * struct jh7110_isp_sat_curve - Saturation curve
> + *
> + * @yi_min: the minimum input Y value.
> + * @yo_ir: the ratio of Y output range to input range.
> + * @yo_min: the minimum output Y value.
> + * @yo_max: the maximum output Y value.
> + */
> +struct jh7110_isp_sat_curve {
> +	__s16 yi_min;
> +	__s16 yo_ir;
> +	__s16 yo_min;
> +	__s16 yo_max;
> +};
> +
> +/**
> + * struct jh7110_isp_sat_hue_info - Chroma Saturation Hue Factor
> + *
> + * @cos: COS hue factor.
> + * @sin: SIN hue factor.
> + */
> +struct jh7110_isp_sat_hue_info {
> +	__s16 cos;
> +	__s16 sin;
> +};
> +
> +/**
> + * struct jh7110_isp_sat_info - Saturation information
> + *
> + * @gain_cmab: Chroma saturation magnitude amplification base for gain.
> + * @gain_cmmd: Chroma saturation magnitude amplification delta for gain.
> + * @threshold_cmb: Chroma saturation magnitude base threshold.
> + * @threshold_cmd: Chroma saturation magnitude delta threshold.
> + * @offset_u: Chroma saturation U offset.
> + * @offset_v: Chroma saturation V offset.
> + * @cmsf: Chroma saturation magnitude scaling factor.
> + */
> +struct jh7110_isp_sat_info {
> +	__s16 gain_cmab;
> +	__s16 gain_cmmd;
> +	__s16 threshold_cmb;
> +	__s16 threshold_cmd;
> +	__s16 offset_u;
> +	__s16 offset_v;
> +	__s16 cmsf;
> +};
> +
> +/**
> + * struct jh7110_isp_sat_setting - Configuration used by Saturation
> + *
> + * @enabled: enabled setting flag.
> + * @curve: Saturation curve.
> + * @hue_info: Chroma Saturation Hue Factor.
> + * @sat_info: Saturation information.s

informations.

> + */
> +struct jh7110_isp_sat_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_sat_curve curve;
> +	struct jh7110_isp_sat_hue_info hue_info;
> +	struct jh7110_isp_sat_info sat_info;
> +};
> +
> +/**
> + * struct jh7110_isp_sharp_weight - Sharpe weight
> + *
> + * @weight: Sharpen filter weight.
> + * @recip_wei_sum: Sharpen amplification filter weight normalization factor.
> + */
> +struct jh7110_isp_sharp_weight {
> +	__u8 weight[15];
> +	__u32 recip_wei_sum;
> +};
> +
> +/**
> + * struct jh7110_isp_sharp_strength - Sharpen strength
> + *
> + * @diff: Sharpen Edge amplification delta level.
> + * @f: Sharpen Edge amplification factor.
> + * @s: Sharpen Edge amplification factor slope.
> + */
> +struct jh7110_isp_sharp_strength {
> +	__s16 diff[4];
> +	__s16 f[3];
> +	__s32 s[3];
> +};
> +
> +/**
> + * struct jh7110_isp_sharp_setting - Configuration used by Sharpen
> + *
> + * @enabled: enabled setting flag.
> + * @weight: Sharpe weight.
> + * @strength: Sharpen strength.
> + * @pdirf: Positive Factor Multiplier.
> + * @ndirf: Negative Factor Multiplier.
> + */
> +struct jh7110_isp_sharp_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_sharp_weight weight;
> +	struct jh7110_isp_sharp_strength strength;
> +	__s8 pdirf;
> +	__s8 ndirf;
> +};
> +
> +/**
> + * struct jh7110_isp_ycrv_curve - Y Curve parameters table
> + *
> + * @y: Y curve L parameters value.
> + */
> +struct jh7110_isp_ycrv_curve {
> +	__s16 y[64];
> +};
> +
> +/**
> + * struct jh7110_isp_ycrv_setting - Configuration used by Y Curve
> + *
> + * @enabled: enabled setting flag.
> + * @curve: Y Curve parameters table.
> + */
> +struct jh7110_isp_ycrv_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_ycrv_curve curve;

I am a bit failing in seeing the point of embedding the settings in a
dedicated structure when you have a single instance of the
configuration like this and in other instances. Isn't

        struct jh7110_isp_ycrv_setting {
                __u32 enabled;
                __s16 y[64];
        };

easier ? Or do you need a dedicated type for other reasons ?

> +};
> +
> +/**
> + * struct jh7110_isp_sc_config - statistics collection crop configure
> + *
> + * @h_start: Horizontal starting point for frame cropping.
> + * @v_start: Vertical starting point for frame cropping.
> + * @sw_width: Width of statistics collection sub-window.
> + * @sw_height: Height of statistics collection sub-window.
> + * @hperiod: Horizontal period.
> + * @hkeep: Horizontal keep.
> + * @vperiod: Vertical period.
> + * @vkeep: Vertical keep.
> + */
> +struct jh7110_isp_sc_config {
> +	__u16 h_start;
> +	__u16 v_start;
> +	__u8 sw_width;
> +	__u8 sw_height;
> +	__u8 hperiod;
> +	__u8 hkeep;
> +	__u8 vperiod;
> +	__u8 vkeep;
> +};
> +
> +/**
> + * struct jh7110_isp_sc_af_config - statistics collection auto focus configure
> + *
> + * @es_hor_mode: Horizontal mode.
> + * @es_sum_mode: sum mode.

Other fields are documented with a capital letter -> "Sum mode."

> + * @hor_en: Horizontal enable.
> + * @ver_en: Vertical enable.
> + * @es_ver_thr: Vertical threshold.
> + * @es_hor_thr: Horizontal threshold.
> + */
> +struct jh7110_isp_sc_af_config {
> +	__u8 es_hor_mode;
> +	__u8 es_sum_mode;
> +	__u8 hor_en;
> +	__u8 ver_en;
> +	__u8 es_ver_thr;
> +	__u16 es_hor_thr;
> +};
> +
> +/**
> + * struct jh7110_isp_sc_awb_ps - statistics collection auto white balance pixel sum
> + *
> + * @awb_ps_rl: Lower boundary of R value for pixel sum.
> + * @awb_ps_ru: Upper boundary of R value for pixel sum.
> + * @awb_ps_gl: Lower boundary of G value for pixel sum.
> + * @awb_ps_gu: Upper boundary of G value for pixel sum.
> + * @awb_ps_bl: Lower boundary of B value for pixel sum.
> + * @awb_ps_bu: Upper boundary of B value for pixel sum.
> + * @awb_ps_yl: Lower boundary of Y value for pixel sum.
> + * @awb_ps_yu: Upper boundary of Y value for pixel sum.
> + * @awb_ps_grl: Lower boundary of G/R ratio for pixel sum.
> + * @awb_ps_gru: Upper boundary of G/R ratio for pixel sum.
> + * @awb_ps_gbl: Lower boundary of G/B ratio for pixel sum.
> + * @awb_ps_gbu: Upper boundary of G/B ratio for pixel sum.
> + * @awb_ps_grbl: Lower boundary of (Gr/R + b/a * Gb/B) for pixel sum.
> + * @awb_ps_grbu: Upper boundary of (Gr/R + b/a * Gb/B) for pixel sum.
> + */
> +struct jh7110_isp_sc_awb_ps {
> +	__u8 awb_ps_rl;
> +	__u8 awb_ps_ru;
> +	__u8 awb_ps_gl;
> +	__u8 awb_ps_gu;
> +	__u8 awb_ps_bl;
> +	__u8 awb_ps_bu;
> +	__u8 awb_ps_yl;
> +	__u8 awb_ps_yu;
> +	__u16 awb_ps_grl;
> +	__u16 awb_ps_gru;
> +	__u16 awb_ps_gbl;
> +	__u16 awb_ps_gbu;
> +	__u16 awb_ps_grbl;
> +	__u16 awb_ps_grbu;
> +};
> +
> +/**
> + * struct jh7110_isp_sc_awb_ws - statistics collection auto white balance weight sum
> + *
> + * @awb_ws_rl: Lower boundary of R value for weight sum.
> + * @awb_ws_ru: Upper boundary of R value for weight sum.
> + * @awb_ws_grl: Lower boundary of Gr value for weight sum.
> + * @awb_ws_gru: Upper boundary of Gr value for weight sum.
> + * @awb_ws_gbl: Lower boundary of Gb value for weight sum.
> + * @awb_ws_gbu: Upper boundary of Gb value for weight sum.
> + * @awb_ws_bl: Lower boundary of B value for weight sum.
> + * @awb_ws_bu: Upper boundary of B value for weight sum.
> + */
> +struct jh7110_isp_sc_awb_ws {
> +	__u8 awb_ws_rl;
> +	__u8 awb_ws_ru;
> +	__u8 awb_ws_grl;
> +	__u8 awb_ws_gru;
> +	__u8 awb_ws_gbl;
> +	__u8 awb_ws_gbu;
> +	__u8 awb_ws_bl;
> +	__u8 awb_ws_bu;
> +};
> +
> +/**
> + * struct jh7110_isp_sc_awb_point - statistics collection auto white balance point
> + *
> + * @weight: Weighting value at point.
> + */
> +struct jh7110_isp_sc_awb_point {
> +	__u8 weight;
> +};
> +
> +/**
> + * struct jh7110_isp_sc_awb_config - statistics collection auto white balance configure
> + *
> + * @ps_config: statistics collection auto white balance pixel sum.

nit: please be consistent with using capital letters or not in doc.

> + * @awb_ps_grb_ba: auto white balance b/a value.
> + * @sel: input mux for statistics collection auto white balance.
> + * @ws_config: statistics collection auto white balance weight sum.
> + * @awb_cw: Weighting value at 13x13 point.
> + * @pts: statistics collection auto white balance point.
> + */
> +struct jh7110_isp_sc_awb_config {
> +	struct jh7110_isp_sc_awb_ps ps_config;
> +	__u8 awb_ps_grb_ba;
> +	__u8 sel;
> +	struct jh7110_isp_sc_awb_ws ws_config;
> +	__u8 awb_cw[169];
> +	struct jh7110_isp_sc_awb_point pts[17];
> +};
> +
> +/**
> + * struct jh7110_isp_sc_setting - Configuration used by statistics collection
> + *
> + * @enabled: enabled setting flag.
> + * @crop_config: statistics collection crop configure.
> + * @af_config: statistics collection auto focus configure.
> + * @awb_config: statistics collection auto white balance configure.
> + */
> +struct jh7110_isp_sc_setting {
> +	__u32 enabled;
> +	struct jh7110_isp_sc_config crop_config;
> +	struct jh7110_isp_sc_af_config af_config;
> +	struct jh7110_isp_sc_awb_config awb_config;
> +};
> +
> +/**
> + * struct jh7110_isp_params_buffer - StarFive JH7110 ISP Parameters Meta Data
> + *
> + * @enable_setting: enabled setting module (JH7110_ISP_MODULE_* definitions).
> + * @wb_setting: Configuration used by auto white balance gain.
> + * @car_setting: Configuration used by color artifact removal.
> + * @ccm_setting: Configuration used by color correction matrix.
> + * @cfa_setting: Configuration used by demosaic module.
> + * @ctc_setting: Configuration used by crosstalk remove.
> + * @dbc_setting: Configuration used by defect bad pixels correction.
> + * @dnyuv_setting: Configuration used by yuv domain denoise.
> + * @gmargb_setting: Configuration used by RGB gamma.
> + * @lccf_setting: Configuration used by lens correction cosine fourth.
> + * @obc_setting: Configuration used by optical black compensation.
> + * @oecf_setting: Configuration used by opto-electric conversion function.
> + * @r2y_setting: Configuration used by RGB To YUV.
> + * @sat_setting: Configuration used by Saturation.
> + * @sharp_setting: Configuration used by Sharpen.
> + * @ycrv_setting: Configuration used by Y Curve.
> + * @sc_setting: Configuration used by statistics collection.
> + */
> +struct jh7110_isp_params_buffer {
> +	__u32 enable_setting;
> +	struct jh7110_isp_wb_setting wb_setting;
> +	struct jh7110_isp_car_setting car_setting;
> +	struct jh7110_isp_ccm_setting ccm_setting;
> +	struct jh7110_isp_cfa_setting cfa_setting;
> +	struct jh7110_isp_ctc_setting ctc_setting;
> +	struct jh7110_isp_dbc_setting dbc_setting;
> +	struct jh7110_isp_dnyuv_setting dnyuv_setting;
> +	struct jh7110_isp_gmargb_setting gmargb_setting;
> +	struct jh7110_isp_lccf_setting lccf_setting;
> +	struct jh7110_isp_obc_setting obc_setting;
> +	struct jh7110_isp_oecf_setting oecf_setting;
> +	struct jh7110_isp_r2y_setting r2y_setting;
> +	struct jh7110_isp_sat_setting sat_setting;
> +	struct jh7110_isp_sharp_setting sharp_setting;
> +	struct jh7110_isp_ycrv_setting ycrv_setting;
> +	struct jh7110_isp_sc_setting sc_setting;
> +};
> +
> +/**
> + * Statistics Collection Meta Data Flag
> + */
> +#define JH7110_ISP_SC_FLAG_AWB			0x0
> +#define JH7110_ISP_SC_FLAG_AE_AF		0xffff
> +
> +#pragma pack(1)
> +
> +/**
> + * struct jh7110_isp_sc_buffer - StarFive JH7110 ISP Statistics Collection Meta Data
> + *
> + * @y_histogram: Y histogram data for saturation control.
> + * @reserv0: reserve byte.
> + * @bright_sc: bright statistic. If flag is JH7110_ISP_SC_FLAG_AE_AF, This field is

s/bright statistics/brightness statistics/

no capital "T" after ,

> + *             saved auto exposure and auto focus. If flag is JH7110_ISP_SC_FLAG_AWB,
> + *             This field is saved auto white balance.

no capital "T" after ,

I would replace "this field is saved" which doesn't sound great in
English (not a native speaker though) with "this field stores".

> + * @reserv1: reserve byte.
> + * @ae_hist_y: Y histogram for auto exposure.
> + * @reserv2: reserve byte.
> + * @flag: Statistics Collection Meta Data Flag (JH7110_ISP_SC_FLAG_* definitions)
> + */
> +struct jh7110_isp_sc_buffer {
> +	__u32 y_histogram[64];
> +	__u32 reserv0[33];
> +	__u32 bright_sc[4096];
> +	__u32 reserv1[96];
> +	__u32 ae_hist_y[128];
> +	__u32 reserv2[511];
> +	__u16 flag;
> +};
> +
> +#pragma pack()

This structure is packed, is it populated directly from HW registers
with a memcpy or a DMA transfer ? I guess I'll find it out in the next
patches.

Thanks
   j

> +
> +#endif
> --
> 2.25.1
>
>

Re: [PATCH v5 02/14] media: Documentation: Add description for StarFive ISP metadata formats

[Jacopo Mondi]

Hi Changhuang

On Tue, Jul 09, 2024 at 01:38:12AM GMT, Changhuang Liang wrote:
> Add description for V4L2_META_FMT_STF_ISP_PARAMS and
> V4L2_META_FMT_STF_ISP_STAT_3A meta data formats.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>

I get this warnings when compiling documentation

Please fix these in the uapi header

../include/uapi/linux/jh7110-isp.h:17: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
 * ISP Module Diagram
../include/uapi/linux/jh7110-isp.h:152: warning: expecting prototype for struct jh7110_isp_cfa_params. Prototype was for struct jh7110_isp_cfa_setting instead
../include/uapi/linux/jh7110-isp.h:178: warning: expecting prototype for struct jh7110_isp_ctc_params. Prototype was for struct jh7110_isp_ctc_setting instead
../include/uapi/linux/jh7110-isp.h:200: warning: expecting prototype for struct jh7110_isp_dbc_params. Prototype was for struct jh7110_isp_dbc_setting instead
../include/uapi/linux/jh7110-isp.h:226: warning: expecting prototype for struct jh7110_isp_dnyuv_params. Prototype was for struct jh7110_isp_dnyuv_setting instead
../include/uapi/linux/jh7110-isp.h:707: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
 * Statistics Collection Meta Data Flag

and this in the documentation here below:

Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst:73: WARNING: Title underline too short.

JH7110 ISP uAPI data types
======================

Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst: WARNING: document isn't included in any toctree

(You should  add this file to
Documentation/userspace-api/media/v4l/meta-formats.rst)

> ---
>  .../media/v4l/metafmt-starfive-isp.rst        | 75 +++++++++++++++++++
>  MAINTAINERS                                   |  1 +
>  2 files changed, 76 insertions(+)
>  create mode 100644 Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst b/Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
> new file mode 100644
> index 000000000000..ebb4291833d6
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
> @@ -0,0 +1,75 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +.. _v4l2-meta-fmt-stf-isp-params:
> +
> +.. _v4l2-meta-fmt-stf-isp-stat-3a:
> +
> +*****************************************************************************
> +V4L2_META_FMT_STF_ISP_PARAMS ('stfp'), V4L2_META_FMT_STF_ISP_STAT_3A ('stfs')
> +*****************************************************************************
> +
> +.. jh7110_isp_params_buffer
> +
> +Configuration parameters
> +========================
> +
> +The configuration parameters are passed to the "output_params" metadata output
> +video node, using the :c:type:`v4l2_meta_format` interface. They are formatted
> +as described by the :c:type:`jh7110_isp_params_buffer` structure.
> +
> +.. code-block:: c
> +
> +	struct jh7110_isp_params_buffer {
> +		__u32 enable_setting;
> +		struct jh7110_isp_wb_setting wb_setting;
> +		struct jh7110_isp_car_setting car_setting;
> +		struct jh7110_isp_ccm_setting ccm_setting;
> +		struct jh7110_isp_cfa_setting cfa_setting;
> +		struct jh7110_isp_ctc_setting ctc_setting;
> +		struct jh7110_isp_dbc_setting dbc_setting;
> +		struct jh7110_isp_dnyuv_setting dnyuv_setting;
> +		struct jh7110_isp_gmargb_setting gmargb_setting;
> +		struct jh7110_isp_lccf_setting lccf_setting;
> +		struct jh7110_isp_obc_setting obc_setting;
> +		struct jh7110_isp_oecf_setting oecf_setting;
> +		struct jh7110_isp_r2y_setting r2y_setting;
> +		struct jh7110_isp_sat_setting sat_setting;
> +		struct jh7110_isp_sharp_setting sharp_setting;
> +		struct jh7110_isp_ycrv_setting ycrv_setting;
> +		struct jh7110_isp_sc_setting sc_setting;
> +	};
> +
> +.. jh7110_isp_sc_buffer

Is this used ?

> +
> +3A and histogram statistics
> +===========================
> +
> +The ISP device collects different statistics over an input Bayer frame.
> +Those statistics are obtained from the "capture_scd" metadata capture video
> +node, using the :c:type:`v4l2_meta_format` interface. They are formatted as
> +described by the :c:type:`jh7110_isp_sc_buffer` structure.
> +
> +.. code-block:: c
> +
> +	struct jh7110_isp_sc_buffer {
> +		__u32 y_histogram[64];
> +		__u32 reserv0[33];
> +		__u32 bright_sc[4096];
> +		__u32 reserv1[96];
> +		__u32 ae_hist_y[128];
> +		__u32 reserv2[511];
> +		__u16 flag;
> +	};
> +
> +The statistics collected are Auto Exposure, AWB (Auto-white balance), Histogram
> +and AF (Auto-focus). See :c:type:`jh7110_isp_sc_buffer` for details of the
> +statistics.
> +
> +The 3A statistics and configuration parameters described here are usually
> +consumed and produced by dedicated user space libraries that comprise the
> +important tuning tools using software control loop.
> +
> +JH7110 ISP uAPI data types
> +======================
> +
> +.. kernel-doc:: include/uapi/linux/jh7110-isp.h
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 890604eb0d64..8fd613c93e62 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -21304,6 +21304,7 @@ L:	linux-media@vger.kernel.org
>  S:	Maintained
>  F:	Documentation/admin-guide/media/starfive_camss.rst
>  F:	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> +F:	Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
>  F:	drivers/staging/media/starfive/camss
>  F:	include/uapi/linux/jh7110-isp.h
>
> --
> 2.25.1
>
>

Re: [PATCH v5 04/14] staging: media: starfive: Add a params sink pad and a scd source pad for ISP

[Jacopo Mondi]

Hello Changhuang

On Tue, Jul 09, 2024 at 01:38:14AM GMT, Changhuang Liang wrote:
> StarFive ISP can use params sink pad to transmit ISP parameters and use
> scd source pad to capture statistics collection data.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  .../staging/media/starfive/camss/stf-isp.c    | 77 +++++++++++++++++--
>  .../staging/media/starfive/camss/stf-isp.h    |  2 +
>  2 files changed, 71 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.c b/drivers/staging/media/starfive/camss/stf-isp.c
> index 4e6e26736852..0ebffd09842a 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.c
> +++ b/drivers/staging/media/starfive/camss/stf-isp.c
> @@ -21,13 +21,23 @@ static const struct stf_isp_format isp_formats_sink[] = {
>  	{ MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
>  };
>
> +static const struct stf_isp_format isp_formats_sink_params[] = {
> +	{ MEDIA_BUS_FMT_METADATA_FIXED },
> +};
> +
>  static const struct stf_isp_format isp_formats_source[] = {
>  	{ MEDIA_BUS_FMT_YUYV8_1_5X8, 8 },
>  };
>
> +static const struct stf_isp_format isp_formats_source_scd[] = {
> +	{ MEDIA_BUS_FMT_METADATA_FIXED },
> +};
> +
>  static const struct stf_isp_format_table isp_formats_st7110[] = {
>  	{ isp_formats_sink, ARRAY_SIZE(isp_formats_sink) },
> +	{ isp_formats_sink_params, ARRAY_SIZE(isp_formats_sink_params) },
>  	{ isp_formats_source, ARRAY_SIZE(isp_formats_source) },
> +	{ isp_formats_source_scd, ARRAY_SIZE(isp_formats_source_scd) },
>  };
>
>  static const struct stf_isp_format *
> @@ -93,13 +103,19 @@ static void isp_try_format(struct stf_isp_dev *isp_dev,
>
>  	formats = &isp_dev->formats[pad];
>
> -	fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
> -			     STFCAMSS_FRAME_MAX_WIDTH);
> -	fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
> -			      STFCAMSS_FRAME_MAX_HEIGHT);
> -	fmt->height &= ~0x1;
> +	if (pad != STF_ISP_PAD_SRC_SCD && pad != STF_ISP_PAD_SINK_PARAMS) {
> +		fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
> +				     STFCAMSS_FRAME_MAX_WIDTH);
> +		fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
> +				      STFCAMSS_FRAME_MAX_HEIGHT);
> +		fmt->height &= ~0x1;
> +		fmt->colorspace = V4L2_COLORSPACE_SRGB;
> +	} else {
> +		fmt->width = 1;
> +		fmt->height = 1;
> +	}
> +
>  	fmt->field = V4L2_FIELD_NONE;
> -	fmt->colorspace = V4L2_COLORSPACE_SRGB;
>  	fmt->flags = 0;
>
>  	if (!stf_g_fmt_by_mcode(formats, fmt->code))
> @@ -119,7 +135,7 @@ static int isp_enum_mbus_code(struct v4l2_subdev *sd,
>
>  		formats = &isp_dev->formats[code->pad];
>  		code->code = formats->fmts[code->index].code;
> -	} else {
> +	} else if (code->pad == STF_ISP_PAD_SRC) {
>  		struct v4l2_mbus_framefmt *sink_fmt;
>
>  		if (code->index >= ARRAY_SIZE(isp_formats_source))
> @@ -131,6 +147,10 @@ static int isp_enum_mbus_code(struct v4l2_subdev *sd,
>  		code->code = sink_fmt->code;
>  		if (!code->code)
>  			return -EINVAL;
> +	} else {
> +		if (code->index > 0)
> +			return -EINVAL;
> +		code->code = MEDIA_BUS_FMT_METADATA_FIXED;
>  	}
>  	code->flags = 0;
>
> @@ -151,6 +171,9 @@ static int isp_set_format(struct v4l2_subdev *sd,
>  	isp_try_format(isp_dev, state, fmt->pad, &fmt->format);
>  	*format = fmt->format;
>
> +	if (fmt->pad == STF_ISP_PAD_SRC_SCD || fmt->pad == STF_ISP_PAD_SINK_PARAMS)

nit: can easily be made < 80 cols (again, not mandatory in Linux but
enforced in v4l according to
Documentation/driver-api/media/maintainer-entry-profile.rst)

> +		return 0;
> +
>  	isp_dev->current_fmt = stf_g_fmt_by_mcode(&isp_dev->formats[fmt->pad],
>  						  fmt->format.code);
>
> @@ -202,6 +225,9 @@ static int isp_get_selection(struct v4l2_subdev *sd,
>  	struct v4l2_subdev_format fmt = { 0 };
>  	struct v4l2_rect *rect;
>
> +	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad == STF_ISP_PAD_SINK_PARAMS)
> +		return -EINVAL;
> +
>  	switch (sel->target) {
>  	case V4L2_SEL_TGT_CROP_BOUNDS:
>  		if (sel->pad == STF_ISP_PAD_SINK) {
> @@ -239,6 +265,9 @@ static int isp_set_selection(struct v4l2_subdev *sd,
>  	struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>  	struct v4l2_rect *rect;
>
> +	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad == STF_ISP_PAD_SINK_PARAMS)
> +		return -EINVAL;
> +
>  	if (sel->target != V4L2_SEL_TGT_CROP)
>  		return -EINVAL;
>
> @@ -296,8 +325,38 @@ static int isp_init_formats(struct v4l2_subdev *sd,
>  			.height = 1080
>  		}
>  	};
> +	struct v4l2_subdev_format format_params = {
> +		.pad = STF_ISP_PAD_SINK_PARAMS,
> +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +		.format = {
> +			.code = MEDIA_BUS_FMT_METADATA_FIXED,
> +			.width = 1,
> +			.height = 1
> +		}
> +	};
> +	struct v4l2_subdev_format format_scd = {
> +		.pad = STF_ISP_PAD_SRC_SCD,
> +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +		.format = {
> +			.code = MEDIA_BUS_FMT_METADATA_FIXED,
> +			.width = 1,
> +			.height = 1
> +		}
> +	};
> +	int ret;
> +
> +	/* Init for STF_ISP_PAD_SINK and STF_ISP_PAD_SRC pad */

Does this initialize the format on STF_ISP_PAD_SRC for real ?

Anyway, was there already so
Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Thanks
  j

> +	ret = isp_set_format(sd, sd_state, &format);
> +	if (ret < 0)
> +		return ret;
> +
> +	/* Init for STF_ISP_PAD_SINK_PARAMS pad */
> +	ret = isp_set_format(sd, sd_state, &format_params);
> +	if (ret < 0)
> +		return ret;
>
> -	return isp_set_format(sd, sd_state, &format);
> +	/* Init for STF_ISP_PAD_SRC_SCD pad */
> +	return isp_set_format(sd, sd_state, &format_scd);
>  }
>
>  static const struct v4l2_subdev_video_ops isp_video_ops = {
> @@ -338,7 +397,9 @@ int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev)
>  	v4l2_set_subdevdata(sd, isp_dev);
>
>  	pads[STF_ISP_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +	pads[STF_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
>  	pads[STF_ISP_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
> +	pads[STF_ISP_PAD_SRC_SCD].flags = MEDIA_PAD_FL_SOURCE;
>
>  	sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
>  	sd->entity.ops = &isp_media_ops;
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
> index 955cbb048363..bc7e7b0736fa 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.h
> +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> @@ -392,7 +392,9 @@
>  /* pad id for media framework */
>  enum stf_isp_pad_id {
>  	STF_ISP_PAD_SINK = 0,
> +	STF_ISP_PAD_SINK_PARAMS,
>  	STF_ISP_PAD_SRC,
> +	STF_ISP_PAD_SRC_SCD,
>  	STF_ISP_PAD_MAX
>  };
>
> --
> 2.25.1
>
>

Re: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Jacopo Mondi]

Hi Changhuang

On Tue, Jul 09, 2024 at 01:38:18AM GMT, Changhuang Liang wrote:
> Register ISP 3A "capture_scd" video device to receive statistics
> collection data.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  .../staging/media/starfive/camss/stf-buffer.h |   1 +
>  .../staging/media/starfive/camss/stf-camss.c  |  15 ++
>  .../media/starfive/camss/stf-capture.c        |  21 ++-
>  .../media/starfive/camss/stf-isp-hw-ops.c     |  66 ++++++++
>  .../staging/media/starfive/camss/stf-isp.h    |  23 +++
>  .../staging/media/starfive/camss/stf-video.c  | 146 +++++++++++++++++-
>  .../staging/media/starfive/camss/stf-video.h  |   1 +
>  7 files changed, 264 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/staging/media/starfive/camss/stf-buffer.h b/drivers/staging/media/starfive/camss/stf-buffer.h
> index 9d1670fb05ed..727d00617448 100644
> --- a/drivers/staging/media/starfive/camss/stf-buffer.h
> +++ b/drivers/staging/media/starfive/camss/stf-buffer.h
> @@ -23,6 +23,7 @@ enum stf_v_state {
>  struct stfcamss_buffer {
>  	struct vb2_v4l2_buffer vb;
>  	dma_addr_t addr[2];
> +	void *vaddr;
>  	struct list_head queue;
>  };
>
> diff --git a/drivers/staging/media/starfive/camss/stf-camss.c b/drivers/staging/media/starfive/camss/stf-camss.c
> index fecd3e67c7a1..fafa3ce2f6da 100644
> --- a/drivers/staging/media/starfive/camss/stf-camss.c
> +++ b/drivers/staging/media/starfive/camss/stf-camss.c
> @@ -8,6 +8,7 @@
>   *
>   * Author: Jack Zhu <jack.zhu@starfivetech.com>
>   * Author: Changhuang Liang <changhuang.liang@starfivetech.com>
> + * Author: Keith Zhao <keith.zhao@starfivetech.com>
>   *
>   */
>  #include <linux/module.h>
> @@ -126,6 +127,7 @@ static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
>  static int stfcamss_register_devs(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>  	int ret;
>
> @@ -150,8 +152,18 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
>
>  	cap_yuv->video.source_subdev = &isp_dev->subdev;
>
> +	ret = media_create_pad_link(&isp_dev->subdev.entity, STF_ISP_PAD_SRC_SCD,
> +				    &cap_scd->video.vdev.entity, 0, 0);
> +	if (ret)
> +		goto err_rm_links0;

or just 'err_rm_links'

> +
> +	cap_scd->video.source_subdev = &isp_dev->subdev;
> +
>  	return ret;

here you can return 0

>
> +err_rm_links0:
> +	media_entity_remove_links(&isp_dev->subdev.entity);
> +	media_entity_remove_links(&cap_yuv->video.vdev.entity);
>  err_cap_unregister:
>  	stf_capture_unregister(stfcamss);
>  err_isp_unregister:
> @@ -163,10 +175,12 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
>  static void stfcamss_unregister_devs(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>
>  	media_entity_remove_links(&isp_dev->subdev.entity);
>  	media_entity_remove_links(&cap_yuv->video.vdev.entity);
> +	media_entity_remove_links(&cap_scd->video.vdev.entity);
>
>  	stf_isp_unregister(&stfcamss->isp_dev);
>  	stf_capture_unregister(stfcamss);
> @@ -436,5 +450,6 @@ module_platform_driver(stfcamss_driver);
>
>  MODULE_AUTHOR("Jack Zhu <jack.zhu@starfivetech.com>");
>  MODULE_AUTHOR("Changhuang Liang <changhuang.liang@starfivetech.com>");
> +MODULE_AUTHOR("Keith Zhao <keith.zhao@starfivetech.com>");
>  MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
>  MODULE_LICENSE("GPL");
> diff --git a/drivers/staging/media/starfive/camss/stf-capture.c b/drivers/staging/media/starfive/camss/stf-capture.c
> index 75f6ef405e61..328b8c6e351d 100644
> --- a/drivers/staging/media/starfive/camss/stf-capture.c
> +++ b/drivers/staging/media/starfive/camss/stf-capture.c
> @@ -12,6 +12,7 @@
>  static const char * const stf_cap_names[] = {
>  	"capture_raw",
>  	"capture_yuv",
> +	"capture_scd",
>  };
>
>  static const struct stfcamss_format_info stf_wr_fmts[] = {
> @@ -55,6 +56,14 @@ static const struct stfcamss_format_info stf_isp_fmts[] = {
>  	},
>  };
>
> +/* 3A Statistics Collection Data */
> +static const struct stfcamss_format_info stf_isp_scd_fmts[] = {
> +	{
> +		.code = MEDIA_BUS_FMT_METADATA_FIXED,
> +		.pixelformat = V4L2_META_FMT_STF_ISP_STAT_3A,
> +	},
> +};
> +
>  static inline struct stf_capture *to_stf_capture(struct stfcamss_video *video)
>  {
>  	return container_of(video, struct stf_capture, video);
> @@ -84,6 +93,8 @@ static void stf_init_addrs(struct stfcamss_video *video)
>  		stf_set_raw_addr(video->stfcamss, addr0);
>  	else if (cap->type == STF_CAPTURE_YUV)
>  		stf_set_yuv_addr(video->stfcamss, addr0, addr1);
> +	else
> +		stf_set_scd_addr(video->stfcamss, addr0, addr1, TYPE_AWB);
>  }
>
>  static void stf_cap_s_cfg(struct stfcamss_video *video)
> @@ -227,18 +238,24 @@ static void stf_capture_init(struct stfcamss *stfcamss, struct stf_capture *cap)
>  	INIT_LIST_HEAD(&cap->buffers.ready_bufs);
>  	spin_lock_init(&cap->buffers.lock);
>
> -	cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  	cap->video.stfcamss = stfcamss;
>  	cap->video.bpl_alignment = 16 * 8;
>
>  	if (cap->type == STF_CAPTURE_RAW) {
> +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  		cap->video.formats = stf_wr_fmts;
>  		cap->video.nformats = ARRAY_SIZE(stf_wr_fmts);
>  		cap->video.bpl_alignment = 8;
>  	} else if (cap->type == STF_CAPTURE_YUV) {
> +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  		cap->video.formats = stf_isp_fmts;
>  		cap->video.nformats = ARRAY_SIZE(stf_isp_fmts);
>  		cap->video.bpl_alignment = 1;
> +	} else {
> +		cap->video.type = V4L2_BUF_TYPE_META_CAPTURE;
> +		cap->video.formats = stf_isp_scd_fmts;
> +		cap->video.nformats = ARRAY_SIZE(stf_isp_scd_fmts);
> +		cap->video.bpl_alignment = 16 * 8;
>  	}
>  }
>
> @@ -362,9 +379,11 @@ void stf_capture_unregister(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_raw = &stfcamss->captures[STF_CAPTURE_RAW];
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>
>  	stf_capture_unregister_one(cap_raw);
>  	stf_capture_unregister_one(cap_yuv);
> +	stf_capture_unregister_one(cap_scd);
>  }
>
>  int stf_capture_register(struct stfcamss *stfcamss,
> diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> index 6b3966ca18bf..3b18d09f2cc6 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> +++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> @@ -451,11 +451,57 @@ void stf_set_yuv_addr(struct stfcamss *stfcamss,
>  	stf_isp_reg_write(stfcamss, ISP_REG_UV_PLANE_START_ADDR, uv_addr);
>  }
>
> +static enum stf_isp_type_scd stf_isp_get_scd_type(struct stfcamss *stfcamss)
> +{
> +	int val;
> +
> +	val = stf_isp_reg_read(stfcamss, ISP_REG_SC_CFG_1);
> +	return (enum stf_isp_type_scd)(val & ISP_SC_SEL_MASK) >> 30;
> +}

So far used by a single caller, could be inlined

> +
> +void stf_set_scd_addr(struct stfcamss *stfcamss,
> +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> +		      enum stf_isp_type_scd type_scd)
> +{
> +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> +			    SEL_TYPE(type_scd));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> +}
> +
> +static void stf_isp_fill_yhist(struct stfcamss *stfcamss, void *vaddr)
> +{
> +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
> +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> +	u32 i;
> +
> +	for (i = 0; i < 64; i++, reg_addr += 4)
> +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss, reg_addr);

If you have a contigous memory space to read, could memcpy_fromio() help
instead of going through 64 reads ?

> +}
> +
> +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> +			      enum stf_isp_type_scd *type_scd)
> +{
> +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
> +
> +	*type_scd = stf_isp_get_scd_type(stfcamss);
> +	if (*type_scd == TYPE_AWB) {
> +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> +		*type_scd = TYPE_OECF;
> +	} else {
> +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> +		*type_scd = TYPE_AWB;

Is this correct ? Why are you overwriting the value read from HW that
indicates AE/AF stats with AWB ones ?

> +	}
> +}
> +
>  irqreturn_t stf_line_irq_handler(int irq, void *priv)
>  {
>  	struct stfcamss *stfcamss = priv;
>  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stfcamss_buffer *change_buf;
> +	enum stf_isp_type_scd type_scd;
> +	u32 value;
>  	u32 status;
>
>  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
> @@ -467,6 +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
>  					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
>  							 change_buf->addr[1]);
>  			}
> +
> +			value = stf_isp_reg_read(stfcamss, ISP_REG_CSI_MODULE_CFG);
> +			if (value & CSI_SC_EN) {
> +				change_buf = stf_change_buffer(&cap_scd->buffers);
> +				if (change_buf) {
> +					stf_isp_fill_flag(stfcamss, change_buf->vaddr,
> +							  &type_scd);
> +					stf_set_scd_addr(stfcamss, change_buf->addr[0],
> +							 change_buf->addr[1], type_scd);

Sorry if I'm un-familiar with the HW but this seems to be the
line-interrupt. Are you swapping buffers every line or it's just that
you have a single line irq for the stats ?

> +				}
> +			}
>  		}
>
>  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS,
> @@ -485,6 +542,7 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
>  {
>  	struct stfcamss *stfcamss = priv;
>  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stfcamss_buffer *ready_buf;
>  	u32 status;
>
> @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
>  				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
>  		}
>
> +		if (status & ISPC_SC) {
> +			ready_buf = stf_buf_done(&cap_scd->buffers);
> +			if (ready_buf) {
> +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> +				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +			}
> +		}
> +
>  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
>  				  (status & ~ISPC_INT_ALL_MASK) |
>  				  ISPC_ISP | ISPC_CSI | ISPC_SC);
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
> index fcda0502e3b0..0af7b367e57a 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.h
> +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> @@ -10,6 +10,7 @@
>  #ifndef STF_ISP_H
>  #define STF_ISP_H
>
> +#include <linux/jh7110-isp.h>
>  #include <media/v4l2-subdev.h>
>
>  #include "stf-video.h"
> @@ -107,6 +108,12 @@
>  #define Y_COOR(y)				((y) << 16)
>  #define X_COOR(x)				((x) << 0)
>
> +#define ISP_REG_SCD_CFG_0			0x098
> +
> +#define ISP_REG_SC_CFG_1			0x0bc
> +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> +#define SEL_TYPE(n)				((n) << 30)
> +
>  #define ISP_REG_LCCF_CFG_2			0x0e0
>  #define ISP_REG_LCCF_CFG_3			0x0e4
>  #define ISP_REG_LCCF_CFG_4			0x0e8
> @@ -305,6 +312,10 @@
>  #define DNRM_F(n)				((n) << 16)
>  #define CCM_M_DAT(n)				((n) << 0)
>
> +#define ISP_REG_YHIST_CFG_4			0xcd8
> +
> +#define ISP_REG_YHIST_ACC_0			0xd00
> +
>  #define ISP_REG_GAMMA_VAL0			0xe00
>  #define ISP_REG_GAMMA_VAL1			0xe04
>  #define ISP_REG_GAMMA_VAL2			0xe08
> @@ -389,6 +400,15 @@
>  #define IMAGE_MAX_WIDTH				1920
>  #define IMAGE_MAX_HEIGH				1080
>
> +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))

Should this be in the uAPI header as it is useful to userspace as well ?

you could:

struct jh7110_isp_sc_buffer {
	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
	__u32 reserv0[33];
	__u32 bright_sc[4096];
	__u32 reserv1[96];
	__u32 ae_hist_y[128];
	__u32 reserv2[511];
	__u16 flag;
};

ofc if the size is made part of the uAPI you need a more proper name
such as JH7110_ISP_YHIST_SIZE

> +
> +enum stf_isp_type_scd {
> +	TYPE_DEC = 0,
> +	TYPE_OBC,
> +	TYPE_OECF,
> +	TYPE_AWB,
> +};
> +
>  /* pad id for media framework */
>  enum stf_isp_pad_id {
>  	STF_ISP_PAD_SINK = 0,
> @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev *isp_dev);
>
>  void stf_set_yuv_addr(struct stfcamss *stfcamss,
>  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> +void stf_set_scd_addr(struct stfcamss *stfcamss,
> +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> +		      enum stf_isp_type_scd type_scd);
>
>  #endif /* STF_ISP_H */
> diff --git a/drivers/staging/media/starfive/camss/stf-video.c b/drivers/staging/media/starfive/camss/stf-video.c
> index 989b5e82bae9..2203605ec9c7 100644
> --- a/drivers/staging/media/starfive/camss/stf-video.c
> +++ b/drivers/staging/media/starfive/camss/stf-video.c
> @@ -125,6 +125,14 @@ static int stf_video_init_format(struct stfcamss_video *video)
>  	return 0;
>  }
>
> +static int stf_video_scd_init_format(struct stfcamss_video *video)

Make it void if it can't fail (see below)

> +{
> +	video->active_fmt.fmt.meta.dataformat = video->formats[0].pixelformat;
> +	video->active_fmt.fmt.meta.buffersize = sizeof(struct jh7110_isp_sc_buffer);
> +
> +	return 0;
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * Video queue operations
>   */
> @@ -330,6 +338,78 @@ static const struct vb2_ops stf_video_vb2_q_ops = {
>  	.stop_streaming  = video_stop_streaming,
>  };
>
> +static int video_scd_queue_setup(struct vb2_queue *q,
> +				 unsigned int *num_buffers,
> +				 unsigned int *num_planes,
> +				 unsigned int sizes[],
> +				 struct device *alloc_devs[])
> +{
> +	if (*num_planes)
> +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL : 0;
> +
> +	*num_planes = 1;
> +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> +
> +	return 0;
> +}
> +
> +static int video_scd_buf_init(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> +	dma_addr_t *paddr;
> +
> +	paddr = vb2_plane_cookie(vb, 0);
> +	buffer->addr[0] = *paddr;
> +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;

Interesting, I don't see many users of vb2_plane_cookie() in mainline
and I'm not sure what this gives you as you use it to program the
following registers:

	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);


> +	buffer->vaddr = vb2_plane_vaddr(vb, 0);
> +
> +	return 0;
> +}
> +
> +static int video_scd_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +
> +	if (sizeof(struct jh7110_isp_sc_buffer) > vb2_plane_size(vb, 0))
> +		return -EINVAL;
> +
> +	vb2_set_plane_payload(vb, 0, sizeof(struct jh7110_isp_sc_buffer));
> +
> +	vbuf->field = V4L2_FIELD_NONE;

is this necessary ?

> +
> +	return 0;
> +}
> +
> +static int video_scd_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> +
> +	video->ops->start_streaming(video);
> +
> +	return 0;
> +}
> +
> +static void video_scd_stop_streaming(struct vb2_queue *q)
> +{
> +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> +
> +	video->ops->stop_streaming(video);
> +
> +	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
> +}
> +
> +static const struct vb2_ops stf_video_scd_vb2_q_ops = {
> +	.queue_setup     = video_scd_queue_setup,
> +	.wait_prepare    = vb2_ops_wait_prepare,
> +	.wait_finish     = vb2_ops_wait_finish,
> +	.buf_init        = video_scd_buf_init,
> +	.buf_prepare     = video_scd_buf_prepare,
> +	.buf_queue       = video_buf_queue,
> +	.start_streaming = video_scd_start_streaming,
> +	.stop_streaming  = video_scd_stop_streaming,
> +};
> +
>  /* -----------------------------------------------------------------------------
>   * V4L2 ioctls
>   */
> @@ -448,6 +528,37 @@ static const struct v4l2_ioctl_ops stf_vid_ioctl_ops = {
>  	.vidioc_streamoff               = vb2_ioctl_streamoff,
>  };
>
> +static int video_scd_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct stfcamss_video *video = video_drvdata(file);
> +	struct v4l2_meta_format *meta = &f->fmt.meta;
> +
> +	if (f->type != video->type)
> +		return -EINVAL;
> +
> +	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
> +	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops stf_vid_scd_ioctl_ops = {
> +	.vidioc_querycap                = video_querycap,
> +	.vidioc_enum_fmt_meta_cap       = video_enum_fmt,
> +	.vidioc_g_fmt_meta_cap          = video_scd_g_fmt,
> +	.vidioc_s_fmt_meta_cap          = video_scd_g_fmt,
> +	.vidioc_try_fmt_meta_cap        = video_scd_g_fmt,
> +	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> +	.vidioc_querybuf                = vb2_ioctl_querybuf,
> +	.vidioc_qbuf                    = vb2_ioctl_qbuf,
> +	.vidioc_expbuf                  = vb2_ioctl_expbuf,
> +	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> +	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> +	.vidioc_streamon                = vb2_ioctl_streamon,
> +	.vidioc_streamoff               = vb2_ioctl_streamoff,
> +};
> +
>  /* -----------------------------------------------------------------------------
>   * V4L2 file operations
>   */
> @@ -473,6 +584,9 @@ static int stf_link_validate(struct media_link *link)
>  	struct stfcamss_video *video = video_get_drvdata(vdev);
>  	int ret;
>
> +	if (video->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		return 0;
> +
>  	ret = stf_video_check_format(video);
>
>  	return ret;
> @@ -506,7 +620,11 @@ int stf_video_register(struct stfcamss_video *video,
>  	q = &video->vb2_q;
>  	q->drv_priv = video;
>  	q->mem_ops = &vb2_dma_contig_memops;
> -	q->ops = &stf_video_vb2_q_ops;
> +
> +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> +		q->ops = &stf_video_vb2_q_ops;
> +	else
> +		q->ops = &stf_video_scd_vb2_q_ops;
>  	q->type = video->type;
>  	q->io_modes = VB2_DMABUF | VB2_MMAP;
>  	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> @@ -529,16 +647,28 @@ int stf_video_register(struct stfcamss_video *video,
>  		goto err_mutex_destroy;
>  	}
>
> -	ret = stf_video_init_format(video);
> -	if (ret < 0) {
> -		dev_err(video->stfcamss->dev,
> -			"Failed to init format: %d\n", ret);
> -		goto err_media_cleanup;
> +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
> +		ret = stf_video_init_format(video);

I don't think this can fail

> +		if (ret < 0) {
> +			dev_err(video->stfcamss->dev,
> +				"Failed to init format: %d\n", ret);
> +			goto err_media_cleanup;
> +		}
> +		vdev->ioctl_ops = &stf_vid_ioctl_ops;
> +		vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
> +	} else {
> +		ret = stf_video_scd_init_format(video);

This can't fail as noted above

> +		if (ret < 0) {
> +			dev_err(video->stfcamss->dev,
> +				"Failed to init format: %d\n", ret);
> +			goto err_media_cleanup;
> +		}
> +		vdev->ioctl_ops = &stf_vid_scd_ioctl_ops;
> +		vdev->device_caps = V4L2_CAP_META_CAPTURE;
>  	}
>
>  	vdev->fops = &stf_vid_fops;
> -	vdev->ioctl_ops = &stf_vid_ioctl_ops;
> -	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
> +	vdev->device_caps |= V4L2_CAP_STREAMING;
>  	vdev->entity.ops = &stf_media_ops;
>  	vdev->vfl_dir = VFL_DIR_RX;
>  	vdev->release = stf_video_release;
> diff --git a/drivers/staging/media/starfive/camss/stf-video.h b/drivers/staging/media/starfive/camss/stf-video.h
> index 59799b65cbe5..53a1cf4e59b7 100644
> --- a/drivers/staging/media/starfive/camss/stf-video.h
> +++ b/drivers/staging/media/starfive/camss/stf-video.h
> @@ -37,6 +37,7 @@ enum stf_v_line_id {
>  enum stf_capture_type {
>  	STF_CAPTURE_RAW = 0,
>  	STF_CAPTURE_YUV,
> +	STF_CAPTURE_SCD,
>  	STF_CAPTURE_NUM,
>  };
>
> --
> 2.25.1
>
>

Re: [PATCH v5 10/14] staging: media: starfive: Add V4L2_CAP_IO_MC capability

[Jacopo Mondi]

Hi Changhuang

On Tue, Jul 09, 2024 at 01:38:20AM GMT, Changhuang Liang wrote:
> Add V4L2_CAP_IO_MC capabality for video device. User space can enumerate
> formats by mbus code.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  drivers/staging/media/starfive/camss/stf-video.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/starfive/camss/stf-video.c b/drivers/staging/media/starfive/camss/stf-video.c
> index 2203605ec9c7..8c6c45d8b7a1 100644
> --- a/drivers/staging/media/starfive/camss/stf-video.c
> +++ b/drivers/staging/media/starfive/camss/stf-video.c
> @@ -668,7 +668,7 @@ int stf_video_register(struct stfcamss_video *video,
>  	}
>
>  	vdev->fops = &stf_vid_fops;
> -	vdev->device_caps |= V4L2_CAP_STREAMING;
> +	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;

Does this set the flag for the stat node as well ? Not that it hurts
but filtering on mbus code where only METADATA_FIXED is allowed
doesn't make much sense ?

Same for the parameters node

>  	vdev->entity.ops = &stf_media_ops;
>  	vdev->vfl_dir = VFL_DIR_RX;
>  	vdev->release = stf_video_release;
> --
> 2.25.1
>
>

回复: [PATCH v5 04/14] staging: media: starfive: Add a params sink pad and a scd source pad for ISP

[Changhuang Liang]

Hi, Jacopo

Thanks for your comments.

[...]

> > @@ -151,6 +171,9 @@ static int isp_set_format(struct v4l2_subdev *sd,
> >  	isp_try_format(isp_dev, state, fmt->pad, &fmt->format);
> >  	*format = fmt->format;
> >
> > +	if (fmt->pad == STF_ISP_PAD_SRC_SCD || fmt->pad ==
> > +STF_ISP_PAD_SINK_PARAMS)
> 
> nit: can easily be made < 80 cols (again, not mandatory in Linux but enforced
> in v4l according to
> Documentation/driver-api/media/maintainer-entry-profile.rst)
> 

I will add --max-line-length=80 to check patchs again.

> > +		return 0;
> > +
> >  	isp_dev->current_fmt =
> stf_g_fmt_by_mcode(&isp_dev->formats[fmt->pad],
> >  						  fmt->format.code);
> >
> > @@ -202,6 +225,9 @@ static int isp_get_selection(struct v4l2_subdev *sd,
> >  	struct v4l2_subdev_format fmt = { 0 };
> >  	struct v4l2_rect *rect;
> >
> > +	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad ==
> STF_ISP_PAD_SINK_PARAMS)
> > +		return -EINVAL;
> > +
> >  	switch (sel->target) {
> >  	case V4L2_SEL_TGT_CROP_BOUNDS:
> >  		if (sel->pad == STF_ISP_PAD_SINK) { @@ -239,6 +265,9 @@ static
> int
> > isp_set_selection(struct v4l2_subdev *sd,
> >  	struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
> >  	struct v4l2_rect *rect;
> >
> > +	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad ==
> STF_ISP_PAD_SINK_PARAMS)
> > +		return -EINVAL;
> > +
> >  	if (sel->target != V4L2_SEL_TGT_CROP)
> >  		return -EINVAL;
> >
> > @@ -296,8 +325,38 @@ static int isp_init_formats(struct v4l2_subdev *sd,
> >  			.height = 1080
> >  		}
> >  	};
> > +	struct v4l2_subdev_format format_params = {
> > +		.pad = STF_ISP_PAD_SINK_PARAMS,
> > +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > +		.format = {
> > +			.code = MEDIA_BUS_FMT_METADATA_FIXED,
> > +			.width = 1,
> > +			.height = 1
> > +		}
> > +	};
> > +	struct v4l2_subdev_format format_scd = {
> > +		.pad = STF_ISP_PAD_SRC_SCD,
> > +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > +		.format = {
> > +			.code = MEDIA_BUS_FMT_METADATA_FIXED,
> > +			.width = 1,
> > +			.height = 1
> > +		}
> > +	};
> > +	int ret;
> > +
> > +	/* Init for STF_ISP_PAD_SINK and STF_ISP_PAD_SRC pad */
> 
> Does this initialize the format on STF_ISP_PAD_SRC for real ?
> 

Yes, it will initialize the PAD_SINK and PAND_SRC formats by isp_set_selection.

> Anyway, was there already so
> Reviewed-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> Thanks
>   j
> 

Re: [PATCH v5 11/14] staging: media: starfive: Add ISP params video device

[Jacopo Mondi]

Hi Changhuang

   + Hans for one question on the vb2 queue mem_ops to use.

On Tue, Jul 09, 2024 at 01:38:21AM GMT, Changhuang Liang wrote:
> Add ISP params video device to write ISP parameters for 3A.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  drivers/staging/media/starfive/camss/Makefile |   2 +
>  .../staging/media/starfive/camss/stf-camss.c  |  23 +-
>  .../staging/media/starfive/camss/stf-camss.h  |   3 +
>  .../media/starfive/camss/stf-isp-params.c     | 240 ++++++++++++++++++
>  .../staging/media/starfive/camss/stf-isp.h    |   4 +
>  .../staging/media/starfive/camss/stf-output.c |  83 ++++++
>  .../staging/media/starfive/camss/stf-output.h |  22 ++
>  7 files changed, 376 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/staging/media/starfive/camss/stf-isp-params.c
>  create mode 100644 drivers/staging/media/starfive/camss/stf-output.c
>  create mode 100644 drivers/staging/media/starfive/camss/stf-output.h
>
> diff --git a/drivers/staging/media/starfive/camss/Makefile b/drivers/staging/media/starfive/camss/Makefile
> index 411b45f3fb52..077165cbba7a 100644
> --- a/drivers/staging/media/starfive/camss/Makefile
> +++ b/drivers/staging/media/starfive/camss/Makefile
> @@ -9,6 +9,8 @@ starfive-camss-objs += \
>  		stf-capture.o \
>  		stf-isp.o \
>  		stf-isp-hw-ops.o \
> +		stf-isp-params.o \
> +		stf-output.o \

so now you have

stf-capture.c and stf-output.c

that define the entry point for the isp driver to register video nodes
and initialize the type-specific operations.

They call into:

stf-video.c (and now) into stf-params.c

to initialize the vb2 queue and register the video devices.

stf-video.c handles RAW, YUV and STAT nodes
stf-params.c handles PARAMS

I'm not asking you to change this, but most drivers have a single file
for -params, -stats and -video where they handle both the vb2 queue,
the video device registration.

Seeing however that stf-output is only 83 lines I wonder if it
shouldn't be collapsed into stf-isp-params.c

You're now a v5 already, this is an invasive change, but I guess it's
something that could be post-poned to when the driver will be
de-staged

>  		stf-video.o
>
>  obj-$(CONFIG_VIDEO_STARFIVE_CAMSS) += starfive-camss.o
> diff --git a/drivers/staging/media/starfive/camss/stf-camss.c b/drivers/staging/media/starfive/camss/stf-camss.c
> index fafa3ce2f6da..4b2288c3199c 100644
> --- a/drivers/staging/media/starfive/camss/stf-camss.c
> +++ b/drivers/staging/media/starfive/camss/stf-camss.c
> @@ -128,6 +128,7 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
>  	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> +	struct stf_output *output = &stfcamss->output;
>  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>  	int ret;
>
> @@ -138,13 +139,20 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
>  		return ret;
>  	}
>
> -	ret = stf_capture_register(stfcamss, &stfcamss->v4l2_dev);
> +	ret = stf_output_register(stfcamss, &stfcamss->v4l2_dev);
>  	if (ret < 0) {
>  		dev_err(stfcamss->dev,
>  			"failed to register capture: %d\n", ret);
>  		goto err_isp_unregister;
>  	}
>
> +	ret = stf_capture_register(stfcamss, &stfcamss->v4l2_dev);
> +	if (ret < 0) {
> +		dev_err(stfcamss->dev,
> +			"failed to register capture: %d\n", ret);
> +		goto err_out_unregister;
> +	}
> +
>  	ret = media_create_pad_link(&isp_dev->subdev.entity, STF_ISP_PAD_SRC,
>  				    &cap_yuv->video.vdev.entity, 0, 0);
>  	if (ret)
> @@ -159,13 +167,23 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
>
>  	cap_scd->video.source_subdev = &isp_dev->subdev;
>
> +	ret = media_create_pad_link(&output->video.vdev.entity, 0,
> +				    &isp_dev->subdev.entity, STF_ISP_PAD_SINK_PARAMS,
> +				    0);
> +	if (ret)
> +		goto err_rm_links1;
> +
>  	return ret;
>
> +err_rm_links1:

or err_rm_link_scd

> +	media_entity_remove_links(&cap_scd->video.vdev.entity);
>  err_rm_links0:
>  	media_entity_remove_links(&isp_dev->subdev.entity);
>  	media_entity_remove_links(&cap_yuv->video.vdev.entity);
>  err_cap_unregister:
>  	stf_capture_unregister(stfcamss);
> +err_out_unregister:
> +	stf_output_unregister(stfcamss);
>  err_isp_unregister:
>  	stf_isp_unregister(&stfcamss->isp_dev);
>
> @@ -176,14 +194,17 @@ static void stfcamss_unregister_devs(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
>  	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> +	struct stf_output *output = &stfcamss->output;
>  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>
> +	media_entity_remove_links(&output->video.vdev.entity);
>  	media_entity_remove_links(&isp_dev->subdev.entity);
>  	media_entity_remove_links(&cap_yuv->video.vdev.entity);
>  	media_entity_remove_links(&cap_scd->video.vdev.entity);
>
>  	stf_isp_unregister(&stfcamss->isp_dev);
>  	stf_capture_unregister(stfcamss);
> +	stf_output_unregister(stfcamss);
>  }
>
>  static int stfcamss_subdev_notifier_bound(struct v4l2_async_notifier *async,
> diff --git a/drivers/staging/media/starfive/camss/stf-camss.h b/drivers/staging/media/starfive/camss/stf-camss.h
> index ae49c7031ab7..3f84f1a1e997 100644
> --- a/drivers/staging/media/starfive/camss/stf-camss.h
> +++ b/drivers/staging/media/starfive/camss/stf-camss.h
> @@ -21,6 +21,7 @@
>  #include "stf-buffer.h"
>  #include "stf-isp.h"
>  #include "stf-capture.h"
> +#include "stf-output.h"
>
>  enum stf_port_num {
>  	STF_PORT_DVP = 0,
> @@ -55,6 +56,7 @@ struct stfcamss {
>  	struct device *dev;
>  	struct stf_isp_dev isp_dev;
>  	struct stf_capture captures[STF_CAPTURE_NUM];
> +	struct stf_output output;
>  	struct v4l2_async_notifier notifier;
>  	void __iomem *syscon_base;
>  	void __iomem *isp_base;
> @@ -132,4 +134,5 @@ static inline void stf_syscon_reg_clear_bit(struct stfcamss *stfcamss,
>  	value = ioread32(stfcamss->syscon_base + reg);
>  	iowrite32(value & ~bit_mask, stfcamss->syscon_base + reg);
>  }
> +

not necessary

>  #endif /* STF_CAMSS_H */
> diff --git a/drivers/staging/media/starfive/camss/stf-isp-params.c b/drivers/staging/media/starfive/camss/stf-isp-params.c
> new file mode 100644
> index 000000000000..e015857815f0
> --- /dev/null
> +++ b/drivers/staging/media/starfive/camss/stf-isp-params.c
> @@ -0,0 +1,240 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * stf-isp-params.c
> + *
> + * StarFive Camera Subsystem - V4L2 device node
> + *
> + * Copyright (C) 2021-2023 StarFive Technology Co., Ltd.
> + */
> +
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "stf-camss.h"
> +#include "stf-video.h"
> +
> +static inline struct stfcamss_buffer *
> +to_stfcamss_buffer(struct vb2_v4l2_buffer *vbuf)
> +{
> +	return container_of(vbuf, struct stfcamss_buffer, vb);
> +}
> +
> +static int stf_isp_params_queue_setup(struct vb2_queue *q,
> +				      unsigned int *num_buffers,
> +				      unsigned int *num_planes,
> +				      unsigned int sizes[],
> +				      struct device *alloc_devs[])
> +{
> +	if (*num_planes)
> +		return sizes[0] < sizeof(struct jh7110_isp_params_buffer) ? -EINVAL : 0;
> +
> +	*num_planes = 1;
> +	sizes[0] = sizeof(struct jh7110_isp_params_buffer);
> +
> +	return 0;
> +}
> +
> +static int stf_isp_params_buf_init(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> +	dma_addr_t *paddr;
> +
> +	paddr = vb2_plane_cookie(vb, 0);

same question as for the stat

> +	buffer->addr[0] = *paddr;
> +	buffer->vaddr = vb2_plane_vaddr(vb, 0);
> +
> +	return 0;
> +}

You could as well (unless something's related to cookies I don't
understand) get the vaddr() of the buffer at the time it gets used and
drop _buf_init() completely

> +
> +static int stf_isp_params_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +
> +	if (sizeof(struct jh7110_isp_params_buffer) > vb2_plane_size(vb, 0))
> +		return -EINVAL;
> +
> +	vb2_set_plane_payload(vb, 0, sizeof(struct jh7110_isp_params_buffer));

This shouldn't be done by the kernel on OUTPUT queues, but you should
make sure that the payload set by userspace has the correct size
(which is sizeof(struct jh7110_isp_params_buffer)). I think you can
also drop the check for vb2_plane_size() as the buffer size is set at
queue_setup().

> +
> +	vbuf->field = V4L2_FIELD_NONE;

Also this is not necessary I guess
> +
> +	return 0;
> +}
> +
> +static void stf_isp_params_buf_queue(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct stfcamss_video *video = vb2_get_drv_priv(vb->vb2_queue);
> +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> +
> +	video->ops->queue_buffer(video, buffer);

You can really do that here and drop the -output.c file

> +}
> +
> +static void stf_isp_params_stop_streaming(struct vb2_queue *q)
> +{
> +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> +
> +	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
> +}
> +
> +static const struct vb2_ops stf_isp_params_vb2_q_ops = {
> +	.queue_setup     = stf_isp_params_queue_setup,
> +	.wait_prepare    = vb2_ops_wait_prepare,
> +	.wait_finish     = vb2_ops_wait_finish,
> +	.buf_init        = stf_isp_params_buf_init,
> +	.buf_prepare     = stf_isp_params_buf_prepare,
> +	.buf_queue       = stf_isp_params_buf_queue,
> +	.stop_streaming  = stf_isp_params_stop_streaming,
> +};
> +
> +static int stf_isp_params_init_format(struct stfcamss_video *video)
> +{
> +	video->active_fmt.fmt.meta.dataformat = V4L2_META_FMT_STF_ISP_PARAMS;
> +	video->active_fmt.fmt.meta.buffersize = sizeof(struct jh7110_isp_params_buffer);

There's not need to store this in the active_fmt. params.c can use
V4L2_META_FMT_STF_ISP_PARAMS and sizeof(struct
jh7110_isp_params_buffer) directly in enum_fmt and g_fmt.

> +
> +	return 0;
> +}
> +
> +static int stf_isp_params_querycap(struct file *file, void *fh,
> +				   struct v4l2_capability *cap)
> +{
> +	strscpy(cap->driver, "starfive-camss", sizeof(cap->driver));
> +	strscpy(cap->card, "Starfive Camera Subsystem", sizeof(cap->card));
> +
> +	return 0;
> +}
> +
> +static int stf_isp_params_enum_fmt(struct file *file, void *priv,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	struct stfcamss_video *video = video_drvdata(file);
> +
> +	if (f->index > 0 || f->type != video->type)
> +		return -EINVAL;
> +
> +	f->pixelformat = video->active_fmt.fmt.meta.dataformat;
> +	return 0;
> +}
> +
> +static int stf_isp_params_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct stfcamss_video *video = video_drvdata(file);
> +	struct v4l2_meta_format *meta = &f->fmt.meta;
> +
> +	if (f->type != video->type)
> +		return -EINVAL;
> +
> +	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
> +	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops stf_isp_params_ioctl_ops = {
> +	.vidioc_querycap                = stf_isp_params_querycap,
> +	.vidioc_enum_fmt_meta_out	= stf_isp_params_enum_fmt,
> +	.vidioc_g_fmt_meta_out          = stf_isp_params_g_fmt,
> +	.vidioc_s_fmt_meta_out          = stf_isp_params_g_fmt,
> +	.vidioc_try_fmt_meta_out        = stf_isp_params_g_fmt,
> +	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> +	.vidioc_querybuf                = vb2_ioctl_querybuf,
> +	.vidioc_qbuf                    = vb2_ioctl_qbuf,
> +	.vidioc_expbuf                  = vb2_ioctl_expbuf,
> +	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> +	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> +	.vidioc_streamon                = vb2_ioctl_streamon,
> +	.vidioc_streamoff               = vb2_ioctl_streamoff,
> +};
> +
> +static const struct v4l2_file_operations stf_isp_params_fops = {
> +	.owner          = THIS_MODULE,
> +	.unlocked_ioctl = video_ioctl2,
> +	.open           = v4l2_fh_open,
> +	.release        = vb2_fop_release,
> +	.poll           = vb2_fop_poll,
> +	.mmap           = vb2_fop_mmap,
> +};
> +
> +static void stf_isp_params_release(struct video_device *vdev)
> +{
> +	struct stfcamss_video *video = video_get_drvdata(vdev);
> +
> +	media_entity_cleanup(&vdev->entity);
> +
> +	mutex_destroy(&video->q_lock);
> +	mutex_destroy(&video->lock);
> +}
> +
> +int stf_isp_params_register(struct stfcamss_video *video,
> +			    struct v4l2_device *v4l2_dev,
> +			    const char *name)
> +{
> +	struct video_device *vdev = &video->vdev;
> +	struct vb2_queue *q;
> +	struct media_pad *pad = &video->pad;
> +	int ret;
> +
> +	mutex_init(&video->q_lock);
> +	mutex_init(&video->lock);

are two mutexes required for the vb2 queue and the video node ? I see,
in example, rkisp1-params.c uses a single one

> +
> +	q = &video->vb2_q;
> +	q->drv_priv = video;
> +	q->mem_ops = &vb2_dma_contig_memops;

Now, I might be wrong, but unless you need to allocate memory from a
DMA-capable area, you shouldn't need to use vb2_dma_contig_memops.

Looking at the next patches you apply configuration parameters to the
ISP by inspecting the user supplied parameters one by one, not by
transfering the whole parameters buffer to a memory area. Hans what do
you think ?

> +	q->ops = &stf_isp_params_vb2_q_ops;
> +	q->type = video->type;
> +	q->io_modes = VB2_DMABUF | VB2_MMAP;
> +	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> +	q->buf_struct_size = sizeof(struct stfcamss_buffer);
> +	q->dev = video->stfcamss->dev;
> +	q->lock = &video->q_lock;
> +	q->min_queued_buffers = STFCAMSS_MIN_BUFFERS;
> +	ret = vb2_queue_init(q);
> +	if (ret < 0) {
> +		dev_err(video->stfcamss->dev,
> +			"Failed to init vb2 queue: %d\n", ret);
> +		goto err_mutex_destroy;
> +	}
> +
> +	pad->flags = MEDIA_PAD_FL_SOURCE;
> +	ret = media_entity_pads_init(&vdev->entity, 1, pad);
> +	if (ret < 0) {
> +		dev_err(video->stfcamss->dev,
> +			"Failed to init video entity: %d\n", ret);
> +		goto err_mutex_destroy;
> +	}
> +
> +	ret = stf_isp_params_init_format(video);
> +	if (ret < 0) {
> +		dev_err(video->stfcamss->dev,
> +			"Failed to init format: %d\n", ret);
> +		goto err_media_cleanup;
> +	}
> +	vdev->ioctl_ops = &stf_isp_params_ioctl_ops;
> +	vdev->device_caps = V4L2_CAP_META_OUTPUT;
> +	vdev->fops = &stf_isp_params_fops;
> +	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;

Same as per the stat nodes, there's a single format, IO_MC doesn't
give you anything ?

> +	vdev->vfl_dir = VFL_DIR_TX;
> +	vdev->release = stf_isp_params_release;
> +	vdev->v4l2_dev = v4l2_dev;
> +	vdev->queue = &video->vb2_q;
> +	vdev->lock = &video->lock;
> +	strscpy(vdev->name, name, sizeof(vdev->name));
> +
> +	video_set_drvdata(vdev, video);
> +
> +	ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> +	if (ret < 0) {
> +		dev_err(video->stfcamss->dev,
> +			"Failed to register video device: %d\n", ret);
> +		goto err_media_cleanup;
> +	}
> +
> +	return 0;
> +
> +err_media_cleanup:
> +	media_entity_cleanup(&vdev->entity);
> +err_mutex_destroy:
> +	mutex_destroy(&video->lock);
> +	mutex_destroy(&video->q_lock);
> +	return ret;
> +}
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
> index eca3ba1ade75..76ea943bfe98 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.h
> +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> @@ -474,4 +474,8 @@ void stf_set_scd_addr(struct stfcamss *stfcamss,
>  		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
>  		      enum stf_isp_type_scd type_scd);
>
> +int stf_isp_params_register(struct stfcamss_video *video,
> +			    struct v4l2_device *v4l2_dev,
> +			    const char *name);
> +
>  #endif /* STF_ISP_H */
> diff --git a/drivers/staging/media/starfive/camss/stf-output.c b/drivers/staging/media/starfive/camss/stf-output.c
> new file mode 100644
> index 000000000000..8eaf4979cafa
> --- /dev/null
> +++ b/drivers/staging/media/starfive/camss/stf-output.c
> @@ -0,0 +1,83 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * StarFive Camera Subsystem - output device
> + *
> + * Copyright (C) 2023 StarFive Technology Co., Ltd.
> + */
> +
> +#include "stf-camss.h"
> +
> +static inline struct stf_output *to_stf_output(struct stfcamss_video *video)
> +{
> +	return container_of(video, struct stf_output, video);
> +}
> +
> +static int stf_output_queue_buffer(struct stfcamss_video *video,
> +				   struct stfcamss_buffer *buf)
> +{
> +	struct stf_output *output = to_stf_output(video);
> +	struct stf_v_buf *v_bufs = &output->buffers;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&v_bufs->lock, flags);
> +	stf_buf_add_ready(v_bufs, buf);
> +	spin_unlock_irqrestore(&v_bufs->lock, flags);
> +
> +	return 0;
> +}
> +
> +static int stf_output_flush_buffers(struct stfcamss_video *video,
> +				    enum vb2_buffer_state state)
> +{
> +	struct stf_output *output = to_stf_output(video);
> +	struct stf_v_buf *v_bufs = &output->buffers;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&v_bufs->lock, flags);
> +	stf_buf_flush(v_bufs, state);
> +	spin_unlock_irqrestore(&v_bufs->lock, flags);
> +
> +	return 0;
> +}
> +
> +static const struct stfcamss_video_ops stf_output_ops = {
> +	.queue_buffer = stf_output_queue_buffer,
> +	.flush_buffers = stf_output_flush_buffers,
> +};
> +
> +static void stf_output_init(struct stfcamss *stfcamss, struct stf_output *out)
> +{
> +	out->buffers.state = STF_OUTPUT_OFF;
> +	out->buffers.buf[0] = NULL;
> +	out->buffers.buf[1] = NULL;
> +	out->buffers.active_buf = 0;
> +	INIT_LIST_HEAD(&out->buffers.pending_bufs);
> +	INIT_LIST_HEAD(&out->buffers.ready_bufs);
> +	spin_lock_init(&out->buffers.lock);
> +
> +	out->video.stfcamss = stfcamss;
> +	out->video.type = V4L2_BUF_TYPE_META_OUTPUT;
> +}
> +
> +void stf_output_unregister(struct stfcamss *stfcamss)
> +{
> +	struct stf_output *output = &stfcamss->output;
> +
> +	if (!video_is_registered(&output->video.vdev))
> +		return;
> +
> +	media_entity_cleanup(&output->video.vdev.entity);
> +	vb2_video_unregister_device(&output->video.vdev);
> +}
> +
> +int stf_output_register(struct stfcamss *stfcamss,
> +			struct v4l2_device *v4l2_dev)
> +{
> +	struct stf_output *output = &stfcamss->output;
> +
> +	output->video.ops = &stf_output_ops;
> +	stf_output_init(stfcamss, output);
> +	stf_isp_params_register(&output->video, v4l2_dev, "output_params");
> +
> +	return 0;
> +}
> diff --git a/drivers/staging/media/starfive/camss/stf-output.h b/drivers/staging/media/starfive/camss/stf-output.h
> new file mode 100644
> index 000000000000..d3591a0b609b
> --- /dev/null
> +++ b/drivers/staging/media/starfive/camss/stf-output.h
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Starfive Camera Subsystem driver
> + *
> + * Copyright (C) 2023 StarFive Technology Co., Ltd.
> + */
> +
> +#ifndef STF_OUTPUT_H
> +#define STF_OUTPUT_H
> +
> +#include "stf-video.h"
> +
> +struct stf_output {
> +	struct stfcamss_video video;
> +	struct stf_v_buf buffers;
> +};
> +
> +int stf_output_register(struct stfcamss *stfcamss,
> +			struct v4l2_device *v4l2_dev);
> +void stf_output_unregister(struct stfcamss *stfcamss);
> +
> +#endif
> --
> 2.25.1
>
>

回复: [PATCH v5 10/14] staging: media: starfive: Add V4L2_CAP_IO_MC capability

[Changhuang Liang]

Hi Jacopo

Thanks for your comments.

> 
> Hi Changhuang
> 
> On Tue, Jul 09, 2024 at 01:38:20AM GMT, Changhuang Liang wrote:
> > Add V4L2_CAP_IO_MC capabality for video device. User space can
> > enumerate formats by mbus code.
> >
> > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > ---
> >  drivers/staging/media/starfive/camss/stf-video.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/staging/media/starfive/camss/stf-video.c
> > b/drivers/staging/media/starfive/camss/stf-video.c
> > index 2203605ec9c7..8c6c45d8b7a1 100644
> > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > @@ -668,7 +668,7 @@ int stf_video_register(struct stfcamss_video *video,
> >  	}
> >
> >  	vdev->fops = &stf_vid_fops;
> > -	vdev->device_caps |= V4L2_CAP_STREAMING;
> > +	vdev->device_caps |= V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> 
> Does this set the flag for the stat node as well ? Not that it hurts but filtering
> on mbus code where only METADATA_FIXED is allowed doesn't make much
> sense ?
> 
> Same for the parameters node
> 

Yes, I will drop this flag from stat node and parameters node.

> >  	vdev->entity.ops = &stf_media_ops;
> >  	vdev->vfl_dir = VFL_DIR_RX;
> >  	vdev->release = stf_video_release;
> > --

Regards,
Changhuang

Re: [PATCH v5 12/14] staging: media: starfive: Add ISP parameters hardware configure

[Jacopo Mondi]

Hi Changhuang

On Tue, Jul 09, 2024 at 01:38:22AM GMT, Changhuang Liang wrote:
> Add ISP parameters hardware configure.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  .../staging/media/starfive/camss/stf-camss.h  |   8 +
>  .../media/starfive/camss/stf-isp-hw-ops.c     | 573 ++++++++++++++++++
>  .../staging/media/starfive/camss/stf-isp.h    | 135 +++++
>  3 files changed, 716 insertions(+)
>
> diff --git a/drivers/staging/media/starfive/camss/stf-camss.h b/drivers/staging/media/starfive/camss/stf-camss.h
> index 3f84f1a1e997..328318d61c6b 100644
> --- a/drivers/staging/media/starfive/camss/stf-camss.h
> +++ b/drivers/staging/media/starfive/camss/stf-camss.h
> @@ -106,6 +106,14 @@ static inline void stf_isp_reg_set(struct stfcamss *stfcamss, u32 reg, u32 mask)
>  		  stfcamss->isp_base + reg);
>  }
>
> +static inline void stf_isp_reg_fill_zero(struct stfcamss *stfcamss, u32 reg, u32 size)
> +{
> +	u32 i;
> +
> +	for (i = 0; i < size; i++, reg += 4)
> +		iowrite32(0, stfcamss->isp_base + reg);
> +}
> +

Why in the header file ? Also could memset_io() replace this ?

>  static inline u32 stf_syscon_reg_read(struct stfcamss *stfcamss, u32 reg)
>  {
>  	return ioread32(stfcamss->syscon_base + reg);
> diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> index 0bc5e36f952e..e54368910906 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> +++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> @@ -10,6 +10,25 @@
>
>  #include "stf-camss.h"
>
> +static const struct stf_isp_module_info mod_info[] = {
> +	{ ISP_REG_CSI_MODULE_CFG, 2 },
> +	{ ISP_REG_CSI_MODULE_CFG, 4 },
> +	{ ISP_REG_CSI_MODULE_CFG, 6 },
> +	{ ISP_REG_CSI_MODULE_CFG, 7 },
> +	{ ISP_REG_CSI_MODULE_CFG, 17 },
> +	{ ISP_REG_ISP_CTRL_1, 1 },
> +	{ ISP_REG_ISP_CTRL_1, 2 },
> +	{ ISP_REG_ISP_CTRL_1, 3 },
> +	{ ISP_REG_ISP_CTRL_1, 4 },
> +	{ ISP_REG_ISP_CTRL_1, 5 },
> +	{ ISP_REG_ISP_CTRL_1, 7 },
> +	{ ISP_REG_ISP_CTRL_1, 8 },
> +	{ ISP_REG_ISP_CTRL_1, 17 },
> +	{ ISP_REG_ISP_CTRL_1, 19 },
> +	{ ISP_REG_ISP_CTRL_1, 21 },
> +	{ ISP_REG_ISP_CTRL_1, 22 },
> +};
> +
>  static void stf_isp_config_obc(struct stfcamss *stfcamss)
>  {
>  	u32 reg_val, reg_add;
> @@ -517,6 +536,59 @@ static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
>  	}
>  }
>
> +static void stf_isp_set_params(struct stfcamss *stfcamss, void *vaddr)
> +{

Shouldn't all these operations be in the -params.c file ?

> +	struct jh7110_isp_params_buffer *params = (struct jh7110_isp_params_buffer *)vaddr;
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_WB_SETTING)
> +		isp_set_ctrl_wb(stfcamss, &params->wb_setting);

Also this function is exported in stf-isp.h but are only called
from within this file ? Same for all the other ones I guess ?

> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_CAR_SETTING)
> +		isp_set_ctrl_car(stfcamss, &params->car_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_CCM_SETTING)
> +		isp_set_ctrl_ccm(stfcamss, &params->ccm_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_CFA_SETTING)
> +		isp_set_ctrl_cfa(stfcamss, &params->cfa_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_CTC_SETTING)
> +		isp_set_ctrl_ctc(stfcamss, &params->ctc_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_DBC_SETTING)
> +		isp_set_ctrl_dbc(stfcamss, &params->dbc_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_DNYUV_SETTING)
> +		isp_set_ctrl_dnyuv(stfcamss, &params->dnyuv_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_GMARGB_SETTING)
> +		isp_set_ctrl_gmargb(stfcamss, &params->gmargb_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_LCCF_SETTING)
> +		isp_set_ctrl_lccf(stfcamss, &params->lccf_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_OBC_SETTING)
> +		isp_set_ctrl_obc(stfcamss, &params->obc_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_OECF_SETTING)
> +		isp_set_ctrl_oecf(stfcamss, &params->oecf_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_R2Y_SETTING)
> +		isp_set_ctrl_r2y(stfcamss, &params->r2y_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_SAT_SETTING)
> +		isp_set_ctrl_sat(stfcamss, &params->sat_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_SHARP_SETTING)
> +		isp_set_ctrl_sharp(stfcamss, &params->sharp_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_YCRV_SETTING)
> +		isp_set_ctrl_ycrv(stfcamss, &params->ycrv_setting);
> +
> +	if (params->enable_setting & JH7110_ISP_MODULE_SC_SETTING)
> +		isp_set_ctrl_sc(stfcamss, &params->sc_setting);
> +}
> +
>  irqreturn_t stf_line_irq_handler(int irq, void *priv)
>  {
>  	struct stfcamss *stfcamss = priv;
> @@ -566,11 +638,20 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
>  	struct stfcamss *stfcamss = priv;
>  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
>  	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> +	struct stf_output *output = &stfcamss->output;
>  	struct stfcamss_buffer *ready_buf;
>  	u32 status;
>
>  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
>  	if (status & ISPC_ISP) {
> +		ready_buf = stf_buf_get_ready(&output->buffers);
> +		if (ready_buf) {
> +			stf_isp_set_params(stfcamss, ready_buf->vaddr);
> +			ready_buf->vb.vb2_buf.timestamp = ktime_get_ns();
> +			ready_buf->vb.sequence = output->buffers.sequence++;
> +			vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +		}
> +
>  		if (status & ISPC_ENUO) {
>  			ready_buf = stf_buf_done(&cap->buffers);
>  			if (ready_buf)
> @@ -591,4 +672,496 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
>  	}
>
>  	return IRQ_HANDLED;
> +};
> +
> +int isp_set_ctrl_wb(struct stfcamss *stfcamss, const void *value)

This function is generic, there's no need to pass a const
void * here and cast it below, you can use const struct jh7110_isp_wb_setting *
as the second argument type.

> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_awb];
> +	const struct jh7110_isp_wb_setting *setting =
> +		(const struct jh7110_isp_wb_setting *)value;
> +	const struct jh7110_isp_wb_gain *gains = &setting->gains;
> +
> +	stf_isp_reg_fill_zero(stfcamss, ISP_REG_AWB_X0_CFG_0, 16);
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S0_CFG_0,
> +			  AWB_S_SYMBOL_H(gains->gain_r) | AWB_S_SYMBOL_L(gains->gain_r));
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S0_CFG_1,
> +			  AWB_S_SYMBOL_H(gains->gain_r) | AWB_S_SYMBOL_L(gains->gain_r));
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S1_CFG_0,
> +			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S1_CFG_1,
> +			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S2_CFG_0,
> +			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S2_CFG_1,
> +			  AWB_S_SYMBOL_H(gains->gain_g) | AWB_S_SYMBOL_L(gains->gain_g));
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S3_CFG_0,
> +			  AWB_S_SYMBOL_H(gains->gain_b) | AWB_S_SYMBOL_L(gains->gain_b));
> +	stf_isp_reg_write(stfcamss, ISP_REG_AWB_S3_CFG_1,
> +			  AWB_S_SYMBOL_H(gains->gain_b) | AWB_S_SYMBOL_L(gains->gain_b));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_car(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_car];
> +	const struct jh7110_isp_car_setting *setting =
> +		(const struct jh7110_isp_car_setting *)value;
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_ccm(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_ccm];
> +	const struct jh7110_isp_ccm_setting *setting =
> +		(const struct jh7110_isp_ccm_setting *)value;
> +	const struct jh7110_isp_ccm_smlow *ccm = &setting->ccm_smlow;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_0, DNRM_F(6));
> +	stf_isp_reg_fill_zero(stfcamss, ISP_REG_ICAMD_1, 11);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_12, ccm->ccm[0][0]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_13, ccm->ccm[0][1]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_14, ccm->ccm[0][2]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_15, ccm->ccm[1][0]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_16, ccm->ccm[1][1]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_17, ccm->ccm[1][2]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_18, ccm->ccm[2][0]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_19, ccm->ccm[2][1]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_20, ccm->ccm[2][2]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_21, ccm->offsets[0]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_22, ccm->offsets[1]);
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICAMD_23, ccm->offsets[2]);
> +	stf_isp_reg_fill_zero(stfcamss, ISP_REG_ICAMD_24, 2);
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_cfa(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_cfa];
> +	const struct jh7110_isp_cfa_setting *setting =
> +		(const struct jh7110_isp_cfa_setting *)value;
> +	const struct jh7110_isp_cfa_params *cfa = &setting->settings;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICFAM,
> +			  HV_W(cfa->hv_width) | CROSS_COV(cfa->cross_cov));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_ctc(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_ctc];
> +	const struct jh7110_isp_ctc_setting *setting =
> +		(const struct jh7110_isp_ctc_setting *)value;
> +	const struct jh7110_isp_ctc_params *ctc = &setting->settings;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_ICTC,
> +			  MINGT(ctc->min_gt) | MAXGT(ctc->max_gt) |
> +			  GF_MODE(ctc->saf_mode | ctc->daf_mode));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_dbc(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_dbc];
> +	const struct jh7110_isp_dbc_setting *setting =
> +		(const struct jh7110_isp_dbc_setting *)value;
> +	const struct jh7110_isp_dbc_params *dbc = &setting->settings;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_IDBC,
> +			  BADXT(dbc->bad_xt) | BADGT(dbc->bad_gt));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_dnyuv(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_dnyuv];
> +	const struct jh7110_isp_dnyuv_setting *setting =
> +		(const struct jh7110_isp_dnyuv_setting *)value;
> +	const struct jh7110_isp_dnyuv_params *cfg = &setting->settings;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR0,
> +			  YUVSW0(cfg->y_sweight[0]) | YUVSW1(cfg->y_sweight[1]) |
> +			  YUVSW2(cfg->y_sweight[2]) | YUVSW3(cfg->y_sweight[3]) |
> +			  YUVSW4(cfg->y_sweight[4]) | YUVSW5(cfg->y_sweight[5]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YSWR1,
> +			  YUVSW0(cfg->y_sweight[6]) | YUVSW1(cfg->y_sweight[7]) |
> +			  YUVSW2(cfg->y_sweight[8]) | YUVSW3(cfg->y_sweight[9]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR0,
> +			  YUVSW0(cfg->uv_sweight[0]) | YUVSW1(cfg->uv_sweight[1]) |
> +			  YUVSW2(cfg->uv_sweight[2]) | YUVSW3(cfg->uv_sweight[3]) |
> +			  YUVSW4(cfg->uv_sweight[4]) | YUVSW5(cfg->uv_sweight[5]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CSWR1,
> +			  YUVSW0(cfg->uv_sweight[6]) | YUVSW1(cfg->uv_sweight[7]) |
> +			  YUVSW2(cfg->uv_sweight[8]) | YUVSW3(cfg->uv_sweight[9]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR0,
> +			  CURVE_D_L(cfg->y_curve[0]) | CURVE_D_H(cfg->y_curve[1]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR1,
> +			  CURVE_D_L(cfg->y_curve[2]) | CURVE_D_H(cfg->y_curve[3]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_YDR2,
> +			  CURVE_D_L(cfg->y_curve[4]) | CURVE_D_H(cfg->y_curve[5]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR0,
> +			  CURVE_D_L(cfg->uv_curve[0]) | CURVE_D_H(cfg->uv_curve[1]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR1,
> +			  CURVE_D_L(cfg->uv_curve[2]) | CURVE_D_H(cfg->uv_curve[3]));
> +	stf_isp_reg_write(stfcamss, ISP_REG_DNYUV_CDR2,
> +			  CURVE_D_L(cfg->uv_curve[4]) | CURVE_D_H(cfg->uv_curve[5]));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_gmargb(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_gmargb];
> +	const struct jh7110_isp_gmargb_setting *setting =
> +		(const struct jh7110_isp_gmargb_setting *)value;
> +	const struct jh7110_isp_gmargb_point *curve = setting->curve;
> +	u32 reg_addr = ISP_REG_GAMMA_VAL0;
> +	u32 i;
> +
> +	for (i = 0; i < 15; i++, reg_addr += 4)
> +		stf_isp_reg_write(stfcamss, reg_addr,
> +				  GAMMA_S_VAL(curve[i].sg_val) | GAMMA_VAL(curve[i].g_val));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_lccf(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_lccf];
> +	const struct jh7110_isp_lccf_setting *setting =
> +		(const struct jh7110_isp_lccf_setting *)value;
> +	const struct jh7110_isp_lccf_circle *circle = &setting->circle;
> +	const struct jh7110_isp_lccf_curve_param *r_param = &setting->r_param;
> +	const struct jh7110_isp_lccf_curve_param *gr_param = &setting->gr_param;
> +	const struct jh7110_isp_lccf_curve_param *gb_param = &setting->gb_param;
> +	const struct jh7110_isp_lccf_curve_param *b_param = &setting->b_param;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_0,
> +			  Y_DISTANCE(circle->center_y) | X_DISTANCE(circle->center_x));
> +	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_1,
> +			  LCCF_MAX_DIS(circle->radius));
> +	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_2,
> +			  LCCF_F1_PAR(r_param->f1) | LCCF_F2_PAR(r_param->f2));
> +	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_3,
> +			  LCCF_F1_PAR(gr_param->f1) | LCCF_F2_PAR(gr_param->f2));
> +	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_4,
> +			  LCCF_F1_PAR(gb_param->f1) | LCCF_F2_PAR(gb_param->f2));
> +	stf_isp_reg_write(stfcamss, ISP_REG_LCCF_CFG_5,
> +			  LCCF_F1_PAR(b_param->f1) | LCCF_F2_PAR(b_param->f2));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_obc(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_obc];
> +	const struct jh7110_isp_obc_setting *setting =
> +		(const struct jh7110_isp_obc_setting *)value;
> +	const struct jh7110_isp_obc_win_size *win_size = &setting->win_size;
> +	const struct jh7110_isp_obc_gain *gain = setting->gain;
> +	const struct jh7110_isp_obc_offset *offset = setting->offset;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBC_CFG,
> +			  OBC_W_W(win_size->width) | OBC_W_H(win_size->height));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_0,
> +			  GAIN_A_POINT(gain[0].tl_gain) | GAIN_B_POINT(gain[0].tr_gain) |
> +			  GAIN_C_POINT(gain[0].bl_gain) | GAIN_D_POINT(gain[0].br_gain));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_1,
> +			  GAIN_A_POINT(gain[1].tl_gain) | GAIN_B_POINT(gain[1].tr_gain) |
> +			  GAIN_C_POINT(gain[1].bl_gain) | GAIN_D_POINT(gain[1].br_gain));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_2,
> +			  GAIN_A_POINT(gain[2].tl_gain) | GAIN_B_POINT(gain[2].tr_gain) |
> +			  GAIN_C_POINT(gain[2].bl_gain) | GAIN_D_POINT(gain[2].br_gain));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCG_CFG_3,
> +			  GAIN_A_POINT(gain[3].tl_gain) | GAIN_B_POINT(gain[3].tr_gain) |
> +			  GAIN_C_POINT(gain[3].bl_gain) | GAIN_D_POINT(gain[3].br_gain));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_0,
> +			  OFFSET_A_POINT(offset[0].tl_offset) |
> +			  OFFSET_B_POINT(offset[0].tr_offset) |
> +			  OFFSET_C_POINT(offset[0].bl_offset) |
> +			  OFFSET_D_POINT(offset[0].br_offset));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_1,
> +			  OFFSET_A_POINT(offset[1].tl_offset) |
> +			  OFFSET_B_POINT(offset[1].tr_offset) |
> +			  OFFSET_C_POINT(offset[1].bl_offset) |
> +			  OFFSET_D_POINT(offset[1].br_offset));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_2,
> +			  OFFSET_A_POINT(offset[2].tl_offset) |
> +			  OFFSET_B_POINT(offset[2].tr_offset) |
> +			  OFFSET_C_POINT(offset[2].bl_offset) |
> +			  OFFSET_D_POINT(offset[2].br_offset));
> +	stf_isp_reg_write(stfcamss, ISP_REG_OBCO_CFG_3,
> +			  OFFSET_A_POINT(offset[3].tl_offset) |
> +			  OFFSET_B_POINT(offset[3].tr_offset) |
> +			  OFFSET_C_POINT(offset[3].bl_offset) |
> +			  OFFSET_D_POINT(offset[3].br_offset));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_oecf(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_oecf];
> +	const struct jh7110_isp_oecf_setting *setting =
> +		(const struct jh7110_isp_oecf_setting *)value;
> +	const struct jh7110_isp_oecf_point *oecf = setting->r_curve;
> +	u32 reg_x_addr = ISP_REG_OECF_X0_CFG0;
> +	u32 reg_y_addr = ISP_REG_OECF_Y0_CFG0;
> +	u32 reg_s_addr = ISP_REG_OECF_S0_CFG0;
> +	u32 i;
> +
> +	for (i = 0; i < 64; i += 2, reg_x_addr += 4, reg_y_addr += 4, reg_s_addr += 4) {
> +		stf_isp_reg_write(stfcamss, reg_x_addr,
> +				  OCEF_PAR_L(oecf[i].x) | OCEF_PAR_H(oecf[i + 1].x));
> +		stf_isp_reg_write(stfcamss, reg_y_addr,
> +				  OCEF_PAR_L(oecf[i].y) | OCEF_PAR_H(oecf[i + 1].y));
> +		stf_isp_reg_write(stfcamss, reg_s_addr,
> +				  OCEF_PAR_L(oecf[i].slope) | OCEF_PAR_H(oecf[i + 1].slope));
> +	}
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_r2y(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_r2y];
> +	const struct jh7110_isp_r2y_setting *setting =
> +		(const struct jh7110_isp_r2y_setting *)value;
> +	const struct jh7110_isp_r2y_matrix *matrix = &setting->matrix;
> +	u32 reg_addr = ISP_REG_R2Y_0;
> +	u32 i;
> +
> +	for (i = 0; i < 9; i++, reg_addr += 4)
> +		stf_isp_reg_write(stfcamss, reg_addr, matrix->m[i]);
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_sat(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_sat];
> +	const struct jh7110_isp_sat_setting *setting =
> +		(const struct jh7110_isp_sat_setting *)value;
> +	const struct jh7110_isp_sat_info *sat = &setting->sat_info;
> +	const struct jh7110_isp_sat_hue_info *hue = &setting->hue_info;
> +	const struct jh7110_isp_sat_curve *curve = &setting->curve;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_CS_GAIN,
> +			  CMAB(sat->gain_cmab) | CMAD(sat->gain_cmmd));
> +	stf_isp_reg_write(stfcamss, ISP_REG_CS_THRESHOLD,
> +			  CMB(sat->threshold_cmb) | CMD(sat->threshold_cmd));
> +	stf_isp_reg_write(stfcamss, ISP_REG_CS_OFFSET,
> +			  UOFF(sat->offset_u) | VOFF(sat->offset_v));
> +	stf_isp_reg_write(stfcamss, ISP_REG_CS_SCALE, sat->cmsf);
> +	stf_isp_reg_write(stfcamss, ISP_REG_CS_HUE_F,
> +			  COS(hue->cos) | SIN(hue->sin));
> +	stf_isp_reg_write(stfcamss, ISP_REG_YADJ0,
> +			  YIMIN(curve->yi_min) | YOIR(curve->yo_ir));
> +	stf_isp_reg_write(stfcamss, ISP_REG_YADJ1,
> +			  YOMIN(curve->yo_min) | YOMAX(curve->yo_max));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_sharp(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_shrp];
> +	const struct jh7110_isp_sharp_setting *setting =
> +		(const struct jh7110_isp_sharp_setting *)value;
> +	const struct jh7110_isp_sharp_weight *weight = &setting->weight;
> +	const struct jh7110_isp_sharp_strength *strength = &setting->strength;
> +	u32 reg_addr = ISP_REG_SHARPEN0;
> +	u32 i;
> +
> +	for (i = 0; i < 4; i++, reg_addr += 4)
> +		stf_isp_reg_write(stfcamss, reg_addr,
> +				  S_WEIGHT(weight->weight[i]) | S_DELTA(strength->diff[i]));
> +
> +	for (; i < 15; i++, reg_addr += 4)
> +		stf_isp_reg_write(stfcamss, reg_addr, S_WEIGHT(weight->weight[i]));
> +
> +	reg_addr = ISP_REG_SHARPEN_FS0;
> +
> +	for (i = 0; i < 3; i++, reg_addr += 4)
> +		stf_isp_reg_write(stfcamss, reg_addr,
> +				  S_SLOPE(strength->s[i]) | S_FACTOR(strength->f[i]));
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_SHARPEN_WN,
> +			  WSUM(weight->recip_wei_sum) | NDIRF(setting->ndirf) |
> +			  PDIRF(setting->pdirf));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_ycrv(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_ycrv];
> +	const struct jh7110_isp_ycrv_setting *setting =
> +		(const struct jh7110_isp_ycrv_setting *)value;
> +	const struct jh7110_isp_ycrv_curve *curve = &setting->curve;
> +	u32 reg_addr = ISP_REG_YCURVE_0;
> +	u32 i;
> +
> +	for (i = 0; i < 64; i++, reg_addr += 4)
> +		stf_isp_reg_write(stfcamss, reg_addr, curve->y[i]);
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
> +}
> +
> +int isp_set_ctrl_sc(struct stfcamss *stfcamss, const void *value)
> +{
> +	const struct stf_isp_module_info *reg_info = &mod_info[imi_sc];
> +	const struct jh7110_isp_sc_setting *setting =
> +		(const struct jh7110_isp_sc_setting *)value;
> +	const struct jh7110_isp_sc_config *crop = &setting->crop_config;
> +	const struct jh7110_isp_sc_af_config *af = &setting->af_config;
> +	const struct jh7110_isp_sc_awb_config *awb = &setting->awb_config;
> +	const struct jh7110_isp_sc_awb_ps *awb_ps = &awb->ps_config;
> +	const struct jh7110_isp_sc_awb_ws *awb_ws = &awb->ws_config;
> +	const struct jh7110_isp_sc_awb_point *pts = awb->pts;
> +	u32 reg_addr0, reg_addr1;
> +	u32 i;
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_1, AXI_ID(1));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_0,
> +			  HSTART(crop->h_start) | VSTART(crop->v_start));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_1,
> +			  SC_WIDTH(crop->sw_width) | SC_HEIGHT(crop->sw_height) |
> +			  AWB_PS_GRB_BA(awb->awb_ps_grb_ba) | SEL_TYPE(awb->sel));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_DEC,
> +			  SC_DEC_H_PERIOD(crop->hperiod) | SC_DEC_H_KEEP(crop->hkeep) |
> +			  SC_DEC_V_PERIOD(crop->vperiod) | SC_DEC_V_KEEP(crop->vkeep));
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_0,
> +			  AWB_PS_RL(awb_ps->awb_ps_rl) | AWB_PS_RU(awb_ps->awb_ps_ru) |
> +			  AWB_PS_GL(awb_ps->awb_ps_gl) | AWB_PS_GU(awb_ps->awb_ps_gu));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_1,
> +			  AWB_PS_BL(awb_ps->awb_ps_bl) | AWB_PS_BU(awb_ps->awb_ps_bu) |
> +			  AWB_PS_YL(awb_ps->awb_ps_yl) | AWB_PS_YU(awb_ps->awb_ps_yu));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_2,
> +			  AWB_PS_GRL(awb_ps->awb_ps_grl) | AWB_PS_GRU(awb_ps->awb_ps_gru));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_3,
> +			  AWB_PS_GBL(awb_ps->awb_ps_gbl) | AWB_PS_GBU(awb_ps->awb_ps_gbu));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_PS_CFG_4,
> +			  AWB_PS_GRBL(awb_ps->awb_ps_grbl) | AWB_PS_GRBU(awb_ps->awb_ps_grbu));
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AF,
> +			  AF_ES_HM(af->es_hor_mode) | AF_ES_SM(af->es_sum_mode) |
> +			  AF_ES_HE(af->hor_en) | AF_ES_VE(af->ver_en) |
> +			  AF_ES_VTHR(af->es_ver_thr) | AF_ES_HTHR(af->es_hor_thr));
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_CFG_0,
> +			  AWB_WS_RL(awb_ws->awb_ws_rl) | AWB_WS_RU(awb_ws->awb_ws_ru) |
> +			  AWB_WS_GRL(awb_ws->awb_ws_grl) | AWB_WS_GRU(awb_ws->awb_ws_gru));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_CFG_1,
> +			  AWB_WS_GBL(awb_ws->awb_ws_gbl) | AWB_WS_GBU(awb_ws->awb_ws_gbu) |
> +			  AWB_WS_BL(awb_ws->awb_ws_bl) | AWB_WS_BU(awb_ws->awb_ws_bu));
> +
> +	reg_addr0 = ISP_REG_SC_AWB_WS_CW0_CFG_0;
> +	reg_addr1 = ISP_REG_SC_AWB_WS_CW0_CFG_1;
> +
> +	for (i = 0; i < 13; i++, reg_addr0 += 8, reg_addr1 += 8) {
> +		stf_isp_reg_write(stfcamss, reg_addr0,
> +				  AWB_WS_CW_W_0(awb->awb_cw[13 * i]) |
> +				  AWB_WS_CW_W_1(awb->awb_cw[13 * i + 1]) |
> +				  AWB_WS_CW_W_2(awb->awb_cw[13 * i + 2]) |
> +				  AWB_WS_CW_W_3(awb->awb_cw[13 * i + 3]) |
> +				  AWB_WS_CW_W_4(awb->awb_cw[13 * i + 4]) |
> +				  AWB_WS_CW_W_5(awb->awb_cw[13 * i + 5]) |
> +				  AWB_WS_CW_W_6(awb->awb_cw[13 * i + 6]) |
> +				  AWB_WS_CW_W_7(awb->awb_cw[13 * i + 7]));
> +		stf_isp_reg_write(stfcamss, reg_addr1,
> +				  AWB_WS_CW_W_0(awb->awb_cw[13 * i + 8]) |
> +				  AWB_WS_CW_W_1(awb->awb_cw[13 * i + 9]) |
> +				  AWB_WS_CW_W_2(awb->awb_cw[13 * i + 10]) |
> +				  AWB_WS_CW_W_3(awb->awb_cw[13 * i + 11]) |
> +				  AWB_WS_CW_W_4(awb->awb_cw[13 * i + 12]));
> +	}
> +
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWV_CFG_0,
> +			  AWB_WS_IW_V_0(pts[0].weight) | AWB_WS_IW_V_1(pts[1].weight) |
> +			  AWB_WS_IW_V_2(pts[2].weight) | AWB_WS_IW_V_3(pts[3].weight) |
> +			  AWB_WS_IW_V_4(pts[4].weight) | AWB_WS_IW_V_5(pts[5].weight) |
> +			  AWB_WS_IW_V_6(pts[6].weight) | AWB_WS_IW_V_7(pts[7].weight));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWV_CFG_1,
> +			  AWB_WS_IW_V_0(pts[8].weight) | AWB_WS_IW_V_1(pts[9].weight) |
> +			  AWB_WS_IW_V_2(pts[10].weight) | AWB_WS_IW_V_3(pts[11].weight) |
> +			  AWB_WS_IW_V_4(pts[12].weight) | AWB_WS_IW_V_5(pts[13].weight) |
> +			  AWB_WS_IW_V_6(pts[14].weight) | AWB_WS_IW_V_7(pts[15].weight));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_0,
> +			  AWB_WS_IW_S_0(2 * (pts[1].weight - pts[0].weight)) |
> +			  AWB_WS_IW_S_1(2 * (pts[2].weight - pts[1].weight)) |
> +			  AWB_WS_IW_S_2(2 * (pts[3].weight - pts[2].weight)) |
> +			  AWB_WS_IW_S_3(2 * (pts[4].weight - pts[3].weight)));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_1,
> +			  AWB_WS_IW_S_0(2 * (pts[5].weight - pts[4].weight)) |
> +			  AWB_WS_IW_S_1(2 * (pts[6].weight - pts[5].weight)) |
> +			  AWB_WS_IW_S_2(2 * (pts[7].weight - pts[6].weight)) |
> +			  AWB_WS_IW_S_3(2 * (pts[8].weight - pts[7].weight)));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_2,
> +			  AWB_WS_IW_S_0(2 * (pts[9].weight - pts[8].weight)) |
> +			  AWB_WS_IW_S_1(2 * (pts[10].weight - pts[9].weight)) |
> +			  AWB_WS_IW_S_2(2 * (pts[11].weight - pts[10].weight)) |
> +			  AWB_WS_IW_S_3(2 * (pts[12].weight - pts[11].weight)));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_AWB_WS_IWS_CFG_3,
> +			  AWB_WS_IW_S_0(2 * (pts[13].weight - pts[12].weight)) |
> +			  AWB_WS_IW_S_1(2 * (pts[14].weight - pts[13].weight)) |
> +			  AWB_WS_IW_S_2(2 * (pts[15].weight - pts[14].weight)) |
> +			  AWB_WS_IW_S_3(2 * (pts[16].weight - pts[15].weight)));
> +
> +	stf_isp_reg_set_bit(stfcamss, reg_info->en_reg, 1 << reg_info->en_nbit,
> +			    setting->enabled ? 1 << reg_info->en_nbit : 0);
> +
> +	return 0;
>  }
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
> index 76ea943bfe98..80c4571dc522 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.h
> +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> @@ -124,6 +124,44 @@
>  #define SC_HEIGHT(n)				((n) << 8)
>  #define SC_WIDTH(n)				((n) << 0)
>
> +#define ISP_REG_SC_AF				0x0c0
> +#define AF_ES_HTHR(n)				((n) << 16)
> +#define AF_ES_VTHR(n)				((n) << 8)
> +#define AF_ES_VE(n)				((n) << 3)
> +#define AF_ES_HE(n)				((n) << 2)
> +#define AF_ES_SM(n)				((n) << 1)
> +#define AF_ES_HM(n)				((n) << 0)
> +
> +#define ISP_REG_SC_AWB_PS_CFG_0			0x0c4
> +#define AWB_PS_GU(n)				((n) << 24)
> +#define AWB_PS_GL(n)				((n) << 16)
> +#define AWB_PS_RU(n)				((n) << 8)
> +#define AWB_PS_RL(n)				((n) << 0)
> +
> +#define ISP_REG_SC_AWB_PS_CFG_1			0x0c8
> +#define AWB_PS_YU(n)				((n) << 24)
> +#define AWB_PS_YL(n)				((n) << 16)
> +#define AWB_PS_BU(n)				((n) << 8)
> +#define AWB_PS_BL(n)				((n) << 0)
> +
> +#define ISP_REG_SC_AWB_PS_CFG_2			0x0cc
> +#define AWB_PS_GRU(n)				((n) << 16)
> +#define AWB_PS_GRL(n)				((n) << 0)
> +
> +#define ISP_REG_SC_AWB_PS_CFG_3			0x0d0
> +#define AWB_PS_GBU(n)				((n) << 16)
> +#define AWB_PS_GBL(n)				((n) << 0)
> +
> +#define ISP_REG_SC_AWB_PS_CFG_4			0x0d4
> +#define AWB_PS_GRBU(n)				((n) << 16)
> +#define AWB_PS_GRBL(n)				((n) << 0)
> +
> +#define ISP_REG_SC_DEC				0x0d8
> +#define SC_DEC_V_KEEP(n)			((n) << 24)
> +#define SC_DEC_V_PERIOD(n)			((n) << 16)
> +#define SC_DEC_H_KEEP(n)			((n) << 8)
> +#define SC_DEC_H_PERIOD(n)			((n) << 0)
> +
>  #define ISP_REG_LCCF_CFG_2			0x0e0
>  #define ISP_REG_LCCF_CFG_3			0x0e4
>  #define ISP_REG_LCCF_CFG_4			0x0e8
> @@ -140,6 +178,8 @@
>  #define ISP_REG_OECF_X0_CFG6			0x118
>  #define ISP_REG_OECF_X0_CFG7			0x11c
>
> +#define ISP_REG_OECF_Y0_CFG0			0x180
> +
>  #define ISP_REG_OECF_Y3_CFG0			0x1e0
>  #define ISP_REG_OECF_Y3_CFG1			0x1e4
>  #define ISP_REG_OECF_Y3_CFG2			0x1e8
> @@ -204,6 +244,49 @@
>  #define OFFSET_B_POINT(x)			((x) << 8)
>  #define OFFSET_A_POINT(x)			((x) << 0)
>
> +#define ISP_REG_SC_AWB_WS_CW0_CFG_0		0x4d0
> +#define ISP_REG_SC_AWB_WS_CW0_CFG_1		0x4d4
> +#define AWB_WS_CW_W_7(x)			((x) << 28)
> +#define AWB_WS_CW_W_6(x)			((x) << 24)
> +#define AWB_WS_CW_W_5(x)			((x) << 20)
> +#define AWB_WS_CW_W_4(x)			((x) << 16)
> +#define AWB_WS_CW_W_3(x)			((x) << 12)
> +#define AWB_WS_CW_W_2(x)			((x) << 8)
> +#define AWB_WS_CW_W_1(x)			((x) << 4)
> +#define AWB_WS_CW_W_0(x)			((x) << 0)
> +
> +#define ISP_REG_SC_AWB_WS_IWV_CFG_0		0x538
> +#define ISP_REG_SC_AWB_WS_IWV_CFG_1		0x53c
> +#define AWB_WS_IW_V_7(x)			((x) << 28)
> +#define AWB_WS_IW_V_6(x)			((x) << 24)
> +#define AWB_WS_IW_V_5(x)			((x) << 20)
> +#define AWB_WS_IW_V_4(x)			((x) << 16)
> +#define AWB_WS_IW_V_3(x)			((x) << 12)
> +#define AWB_WS_IW_V_2(x)			((x) << 8)
> +#define AWB_WS_IW_V_1(x)			((x) << 4)
> +#define AWB_WS_IW_V_0(x)			((x) << 0)
> +
> +#define ISP_REG_SC_AWB_WS_IWS_CFG_0		0x540
> +#define ISP_REG_SC_AWB_WS_IWS_CFG_1		0x544
> +#define ISP_REG_SC_AWB_WS_IWS_CFG_2		0x548
> +#define ISP_REG_SC_AWB_WS_IWS_CFG_3		0x54c
> +#define AWB_WS_IW_S_3(x)			((x) << 24)
> +#define AWB_WS_IW_S_2(x)			((x) << 16)
> +#define AWB_WS_IW_S_1(x)			((x) << 8)
> +#define AWB_WS_IW_S_0(x)			((x) << 0)
> +
> +#define ISP_REG_SC_AWB_WS_CFG_0			0x5d0
> +#define AWB_WS_GRU(n)				((n) << 24)
> +#define AWB_WS_GRL(n)				((n) << 16)
> +#define AWB_WS_RU(n)				((n) << 8)
> +#define AWB_WS_RL(n)				((n) << 0)
> +
> +#define ISP_REG_SC_AWB_WS_CFG_1			0x5d4
> +#define AWB_WS_BU(n)				((n) << 24)
> +#define AWB_WS_BL(n)				((n) << 16)
> +#define AWB_WS_GBU(n)				((n) << 8)
> +#define AWB_WS_GBL(n)				((n) << 0)
> +
>  #define ISP_REG_ISP_CTRL_0			0xa00
>  #define ISPC_LINE				BIT(27)
>  #define ISPC_SC					BIT(26)
> @@ -315,8 +398,19 @@
>  #define CURVE_D_L(n)				((n) << 0)
>
>  #define ISP_REG_ICAMD_0				0xc40
> +#define ISP_REG_ICAMD_1				0xc44
>  #define ISP_REG_ICAMD_12			0xc70
> +#define ISP_REG_ICAMD_13			0xc74
> +#define ISP_REG_ICAMD_14			0xc78
> +#define ISP_REG_ICAMD_15			0xc7c
> +#define ISP_REG_ICAMD_16			0xc80
> +#define ISP_REG_ICAMD_17			0xc84
> +#define ISP_REG_ICAMD_18			0xc88
> +#define ISP_REG_ICAMD_19			0xc8c
>  #define ISP_REG_ICAMD_20			0xc90
> +#define ISP_REG_ICAMD_21			0xc94
> +#define ISP_REG_ICAMD_22			0xc98
> +#define ISP_REG_ICAMD_23			0xc9c
>  #define ISP_REG_ICAMD_24			0xca0
>  #define ISP_REG_ICAMD_25			0xca4
>  #define DNRM_F(n)				((n) << 16)
> @@ -439,6 +533,30 @@ enum stf_isp_pad_id {
>  	STF_ISP_PAD_MAX
>  };
>
> +enum stf_isp_modules_index {
> +	imi_obc = 0,
> +	imi_oecf,
> +	imi_lccf,
> +	imi_awb,
> +	imi_sc,
> +	imi_cfa,
> +	imi_car,
> +	imi_ccm,
> +	imi_gmargb,
> +	imi_r2y,
> +	imi_shrp,
> +	imi_sat,
> +	imi_dnyuv,
> +	imi_ycrv,
> +	imi_ctc,
> +	imi_dbc,
> +};
> +
> +struct stf_isp_module_info {
> +	u32 en_reg;
> +	u8 en_nbit;
> +};
> +
>  struct stf_isp_format {
>  	u32 code;
>  	u8 bpp;
> @@ -478,4 +596,21 @@ int stf_isp_params_register(struct stfcamss_video *video,
>  			    struct v4l2_device *v4l2_dev,
>  			    const char *name);
>
> +int isp_set_ctrl_wb(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_car(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_ccm(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_cfa(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_ctc(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_dbc(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_dnyuv(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_gmargb(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_lccf(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_obc(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_oecf(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_r2y(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_sat(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_sharp(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_ycrv(struct stfcamss *stfcamss, const void *value);
> +int isp_set_ctrl_sc(struct stfcamss *stfcamss, const void *value);
> +
>  #endif /* STF_ISP_H */
> --
> 2.25.1
>
>

Re: [PATCH v5 00/14] Add ISP 3A for StarFive

[Jacopo Mondi]

Hi Changhuang

I've passed the series through the media-staging CI
https://gitlab.freedesktop.org/linux-media/users/jmondi/-/pipelines/1220920
Could you check the errors reported there ?


On Tue, Jul 09, 2024 at 01:38:10AM GMT, Changhuang Liang wrote:
> This series implements the ISP 3A function to the Camera Subsystem on StarFive
> JH7110 SoC. The series has been tested on the VisionFive 2 board.
>
> This series is based on top of the master branch of media_stage repository,
> which is tested with a v4l2-compliance compiled from the git repo
> (git://linuxtv.org/v4l-utils.git).
>
> changes since v4:
> - Rebase on top of the master branch of media_stage repository.
> - Insert a new PATCH 13
> - PATCH 1:
> 1. Fix spelling error:
> 	- corsstakl -> crosstalk
> 	- balck -> black
> 	- magbitude -> magnitude
> 	- FALG -> FLAG
> 2. Exchange of the values of JH7110_ISP_SC_FLAG_AE_AF and JH7110_ISP_SC_FLAG_AWB
>
> - PATCH 8:
> 1. Synchronous spelling error: FALG -> FLAG
> 2. Add:
>
> if (*num_planes)
> 	return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL : 0;
>
> - PATCH 11:
> 1. Drop read hook.
> 2. Add:
>
> if (*num_planes)
> 	return sizes[0] < sizeof(struct jh7110_isp_params_buffer) ? -EINVAL : 0;
>
> - PATCH 12:
> 1. Add:
>
> ready_buf->vb.vb2_buf.timestamp = ktime_get_ns();
> ready_buf->vb.sequence = output->buffers.sequence++;
>
> to fix "v4l2-compliance -s" Failed.
>
> v4: https://lore.kernel.org/all/20240402100011.13480-1-changhuang.liang@starfivetech.com/
>
> changes since v3:
> - Rebased on top of the master branch of media_stage repository and patch [1][2].
> - Replaced "stf_get_isp_scd_type" with "stf_isp_get_scd_type" in PATCH 8.
>
> v3: https://lore.kernel.org/all/20240205090424.40302-1-changhuang.liang@starfivetech.com/
>
> changes since v2:
> - Rebased on top of the master branch of media_stage repository.
> - Added helper function "media_entity_remove_links" in PATCH 8 and PATCH 11 to cleanup links.
>
> v2: https://lore.kernel.org/all/20240111084120.16685-1-changhuang.liang@starfivetech.com/
>
> changes since v1:
> - Rebased on top of the master branch of media_stage repository.
> - Documentated ISP submodule.
> - Added new ISP parameters format and document it.
> - Added a meta output video device and link to ISP subdev.
> - Replaced control with meta output video buffer for write ISP parameters.
> - Separated buffer operation which can use for capture and output device.
> - Droped ISP frame sync event.
>
> v1: https://lore.kernel.org/all/20231214065027.28564-1-changhuang.liang@starfivetech.com/
>
> The following are the media graph for the device and the v4l2-compliance
> output.
>
> ==========================================================================================
>
> [the media graph]:
>
> digraph board {
> 	rankdir=TB
> 	n00000001 [label="{{<port0> 0 | <port1> 1} | stf_isp\n/dev/v4l-subdev0 | {<port2> 2 | <port3> 3}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000001:port2 -> n0000000e
> 	n00000001:port3 -> n00000012 [style=dashed]
> 	n00000006 [label="output_params\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> 	n00000006 -> n00000001:port1 [style=dashed]
> 	n0000000a [label="capture_raw\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> 	n0000000e [label="capture_yuv\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
> 	n00000012 [label="capture_scd\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
> 	n0000001c [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n/dev/v4l-subdev1 | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n0000001c:port1 -> n00000001:port0 [style=dashed]
> 	n0000001c:port1 -> n0000000a [style=dashed]
> 	n00000026 [label="{{} | imx219 6-0010\n/dev/v4l-subdev2 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000026:port0 -> n0000001c:port0 [style=bold]
> }
>
> [v4l2-compliance test]:
>
> # ./v4l2-compliance -s -d /dev/video0
> v4l2-compliance 1.27.0-5220, 64 bits, 64-bit time_t
> v4l2-compliance SHA: 8387e3673837 2024-07-01 11:09:32
>
> Compliance test for starfive-camss device /dev/video0:
>
> Driver Info:
>         Driver name      : starfive-camss
>         Card type        : Starfive Camera Subsystem
>         Bus info         : platform:19840000.isp
>         Driver version   : 6.10.0
>         Capabilities     : 0xac200000
>                 Metadata Output
>                 I/O MC
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps      : 0x2c200000
>                 Metadata Output
>                 I/O MC
>                 Streaming
>                 Extended Pix Format
> Media Driver Info:
>         Driver name      : starfive-camss
>         Model            : Starfive Camera Subsystem
>         Serial           :
>         Bus info         : platform:19840000.isp
>         Media version    : 6.10.0
>         Hardware revision: 0x00000000 (0)
>         Driver version   : 6.10.0
> Interface Info:
>         ID               : 0x03000008
>         Type             : V4L Video
> Entity Info:
>         ID               : 0x00000006 (6)
>         Name             : output_params
>         Function         : V4L2 I/O
>         Pad 0x01000007   : 0: Source
>           Link 0x0200001a: to remote pad 0x1000003 of entity 'stf_isp' (Image Signal Processor): Data
>
> Required ioctls:
>         test MC information (see 'Media Driver Info' above): OK
>         test VIDIOC_QUERYCAP: OK
>         test invalid ioctls: OK
>
> Allow for multiple opens:
>         test second /dev/video0 open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
>         test for unlimited opens: OK
>
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 1 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls (Output 0):
>         test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>         test VIDIOC_QUERYCTRL: OK (Not Supported)
>         test VIDIOC_G/S_CTRL: OK (Not Supported)
>         test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>         test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>         test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>         Standard Controls: 0 Private Controls: 0
>
> Format ioctls (Output 0):
>         test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>         test VIDIOC_G/S_PARM: OK (Not Supported)
>         test VIDIOC_G_FBUF: OK (Not Supported)
>         test VIDIOC_G_FMT: OK
>         test VIDIOC_TRY_FMT: OK
>         test VIDIOC_S_FMT: OK
>         test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>         test Cropping: OK (Not Supported)
>         test Composing: OK (Not Supported)
>         test Scaling: OK (Not Supported)
>
> Codec ioctls (Output 0):
>         test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>         test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>         test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls (Output 0):
>         test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>         test CREATE_BUFS maximum buffers: OK
>         test VIDIOC_REMOVE_BUFS: OK
>         test VIDIOC_EXPBUF: OK
>         test Requests: OK (Not Supported)
>
> Test output 0:
>
> Streaming ioctls:
>         test read/write: OK (Not Supported)
>         test blocking wait: OK
>         test MMAP (no poll): OK
>         test MMAP (select): OK
>         test MMAP (epoll): OK
>         test USERPTR (no poll): OK (Not Supported)
>         test USERPTR (select): OK (Not Supported)
>         test DMABUF: Cannot test, specify --expbuf-device
>
> Total for starfive-camss device /dev/video0: 55, Succeeded: 55, Failed: 0, Warnings: 0
>
> # ./v4l2-compliance -s -d /dev/video3
> v4l2-compliance 1.27.0-5220, 64 bits, 64-bit time_t
> v4l2-compliance SHA: 8387e3673837 2024-07-01 11:09:32
>
> Compliance test for starfive-camss device /dev/video3:
>
> Driver Info:
>         Driver name      : starfive-camss
>         Card type        : Starfive Camera Subsystem
>         Bus info         : platform:19840000.isp
>         Driver version   : 6.10.0
>         Capabilities     : 0xa4a00000
>                 Metadata Capture
>                 I/O MC
>                 Streaming
>                 Extended Pix Format
>                 Device Capabilities
>         Device Caps      : 0x24a00000
>                 Metadata Capture
>                 I/O MC
>                 Streaming
>                 Extended Pix Format
> Media Driver Info:
>         Driver name      : starfive-camss
>         Model            : Starfive Camera Subsystem
>         Serial           :
>         Bus info         : platform:19840000.isp
>         Media version    : 6.10.0
>         Hardware revision: 0x00000000 (0)
>         Driver version   : 6.10.0
> Interface Info:
>         ID               : 0x03000014
>         Type             : V4L Video
> Entity Info:
>         ID               : 0x00000012 (18)
>         Name             : capture_scd
>         Function         : V4L2 I/O
>         Pad 0x01000013   : 0: Sink
>           Link 0x02000018: from remote pad 0x1000005 of entity 'stf_isp' (Image Signal Processor): Data
>
> Required ioctls:
>         test MC information (see 'Media Driver Info' above): OK
>         test VIDIOC_QUERYCAP: OK
>         test invalid ioctls: OK
>
> Allow for multiple opens:
>         test second /dev/video3 open: OK
>         test VIDIOC_QUERYCAP: OK
>         test VIDIOC_G/S_PRIORITY: OK
>         test for unlimited opens: OK
>
> Debug ioctls:
>         test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>         test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
>         test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>         test VIDIOC_ENUMAUDIO: OK (Not Supported)
>         test VIDIOC_G/S/ENUMINPUT: OK
>         test VIDIOC_G/S_AUDIO: OK (Not Supported)
>         Inputs: 1 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
>         test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>         test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>         test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>         test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>         test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>         Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
>         test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>         test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>         test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>         test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls (Input 0):
>         test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>         test VIDIOC_QUERYCTRL: OK (Not Supported)
>         test VIDIOC_G/S_CTRL: OK (Not Supported)
>         test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>         test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>         test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>         Standard Controls: 0 Private Controls: 0
>
> Format ioctls (Input 0):
>         test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>         test VIDIOC_G/S_PARM: OK (Not Supported)
>         test VIDIOC_G_FBUF: OK (Not Supported)
>         test VIDIOC_G_FMT: OK
>         test VIDIOC_TRY_FMT: OK
>         test VIDIOC_S_FMT: OK
>         test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>         test Cropping: OK (Not Supported)
>         test Composing: OK (Not Supported)
>         test Scaling: OK (Not Supported)
>
> Codec ioctls (Input 0):
>         test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>         test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>         test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls (Input 0):
>         test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>         test CREATE_BUFS maximum buffers: OK
>         test VIDIOC_REMOVE_BUFS: OK
>         test VIDIOC_EXPBUF: OK
>         test Requests: OK (Not Supported)
>
> Test input 0:
>
> Streaming ioctls:
>         test read/write: OK (Not Supported)
>         test blocking wait: OK
>         test MMAP (no poll): OK
>         test MMAP (select): OK
>         test MMAP (epoll): OK
>         test USERPTR (no poll): OK (Not Supported)
>         test USERPTR (select): OK (Not Supported)
>         test DMABUF: Cannot test, specify --expbuf-device
>
> Total for starfive-camss device /dev/video3: 55, Succeeded: 55, Failed: 0, Warnings: 0
>
> Changhuang Liang (14):
>   media: starfive: Add JH7110 ISP module definitions
>   media: Documentation: Add description for StarFive ISP metadata
>     formats
>   media: videodev2.h, v4l2-ioctl: Add StarFive ISP meta buffer format
>   staging: media: starfive: Add a params sink pad and a scd source pad
>     for ISP
>   staging: media: starfive: Separate buffer from ISP hardware operation
>   staging: media: starfive: Separate buffer be a common file
>   staging: media: starfive: Separate ISP hardware from capture device
>   staging: media: starfive: Add for StarFive ISP 3A SC
>   staging: media: starfive: Update ISP initialise config for 3A
>   staging: media: starfive: Add V4L2_CAP_IO_MC capability
>   staging: media: starfive: Add ISP params video device
>   staging: media: starfive: Add ISP parameters hardware configure
>   staging: media: starfive: Drop read support for video capture devices
>   admin-guide: media: Update documents for StarFive Camera Subsystem
>
>  .../admin-guide/media/starfive_camss.rst      |  11 +-
>  .../media/starfive_camss_graph.dot            |  22 +-
>  .../media/v4l/metafmt-starfive-isp.rst        |  75 ++
>  MAINTAINERS                                   |   2 +
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  drivers/staging/media/starfive/camss/Makefile |   3 +
>  .../staging/media/starfive/camss/stf-buffer.c | 166 ++++
>  .../staging/media/starfive/camss/stf-buffer.h |  53 ++
>  .../staging/media/starfive/camss/stf-camss.c  |  38 +-
>  .../staging/media/starfive/camss/stf-camss.h  |  12 +
>  .../media/starfive/camss/stf-capture.c        | 248 +-----
>  .../media/starfive/camss/stf-capture.h        |  22 -
>  .../media/starfive/camss/stf-isp-hw-ops.c     | 722 +++++++++++++++++
>  .../media/starfive/camss/stf-isp-params.c     | 240 ++++++
>  .../staging/media/starfive/camss/stf-isp.c    |  77 +-
>  .../staging/media/starfive/camss/stf-isp.h    | 188 +++++
>  .../staging/media/starfive/camss/stf-output.c |  83 ++
>  .../staging/media/starfive/camss/stf-output.h |  22 +
>  .../staging/media/starfive/camss/stf-video.c  | 147 +++-
>  .../staging/media/starfive/camss/stf-video.h  |  11 +-
>  include/uapi/linux/jh7110-isp.h               | 739 ++++++++++++++++++
>  include/uapi/linux/videodev2.h                |   4 +
>  22 files changed, 2608 insertions(+), 279 deletions(-)
>  create mode 100644 Documentation/userspace-api/media/v4l/metafmt-starfive-isp.rst
>  create mode 100644 drivers/staging/media/starfive/camss/stf-buffer.c
>  create mode 100644 drivers/staging/media/starfive/camss/stf-buffer.h
>  create mode 100644 drivers/staging/media/starfive/camss/stf-isp-params.c
>  create mode 100644 drivers/staging/media/starfive/camss/stf-output.c
>  create mode 100644 drivers/staging/media/starfive/camss/stf-output.h
>  create mode 100644 include/uapi/linux/jh7110-isp.h
>
> --
> 2.25.1
>

回复: [PATCH v5 00/14] Add ISP 3A for StarFive

[Changhuang Liang]

Hi, Jacopo

Thanks for your comments.

> 
> Hi Changhuang
> 
> I've passed the series through the media-staging CI
> https://gitlab.freedesktop.org/linux-media/users/jmondi/-/pipelines/122092
> 0
> Could you check the errors reported there ?
> 

Sorry, my bad, I pulled the branch too early, forget the rebase the newest, there is a conflict in PATCH 3:

<<<<<<< HEAD
/* Vendor specific - used for RaspberryPi PiSP */
#define V4L2_META_FMT_RPI_BE_CFG	v4l2_fourcc('R', 'P', 'B', 'C') /* PiSP BE configuration */
=======
/* Vendor specific - used for StarFive JH7110 ISP camera sub-system */
#define V4L2_META_FMT_STF_ISP_PARAMS	v4l2_fourcc('S', 'T', 'F', 'P') /* StarFive ISP 3A Parameters */
#define V4L2_META_FMT_STF_ISP_STAT_3A	v4l2_fourcc('S', 'T', 'F', 'S') /* StarFive ISP 3A Statistics */
>>>>>>> 0ef1c3eb85f5 (media: videodev2.h, v4l2-ioctl: Add StarFive ISP meta buffer format)

So you didn't apply the PATCH 3.

Regards,
Changhuang

回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Changhuang Liang]

Hi Jacopo

Thanks for your comments.

> Hi Changhuang
> 
> On Tue, Jul 09, 2024 at 01:38:18AM GMT, Changhuang Liang wrote:
> > Register ISP 3A "capture_scd" video device to receive statistics
> > collection data.
> >
> > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > ---
> >  .../staging/media/starfive/camss/stf-buffer.h |   1 +
> >  .../staging/media/starfive/camss/stf-camss.c  |  15 ++
> >  .../media/starfive/camss/stf-capture.c        |  21 ++-
> >  .../media/starfive/camss/stf-isp-hw-ops.c     |  66 ++++++++
> >  .../staging/media/starfive/camss/stf-isp.h    |  23 +++
> >  .../staging/media/starfive/camss/stf-video.c  | 146
> +++++++++++++++++-
> >  .../staging/media/starfive/camss/stf-video.h  |   1 +
> >  7 files changed, 264 insertions(+), 9 deletions(-)
> >
> > diff --git a/drivers/staging/media/starfive/camss/stf-buffer.h
> > b/drivers/staging/media/starfive/camss/stf-buffer.h
> > index 9d1670fb05ed..727d00617448 100644
> > --- a/drivers/staging/media/starfive/camss/stf-buffer.h
> > +++ b/drivers/staging/media/starfive/camss/stf-buffer.h
> > @@ -23,6 +23,7 @@ enum stf_v_state {
> >  struct stfcamss_buffer {
> >  	struct vb2_v4l2_buffer vb;
> >  	dma_addr_t addr[2];
> > +	void *vaddr;
> >  	struct list_head queue;
> >  };
> >
> > diff --git a/drivers/staging/media/starfive/camss/stf-camss.c
> > b/drivers/staging/media/starfive/camss/stf-camss.c
> > index fecd3e67c7a1..fafa3ce2f6da 100644
> > --- a/drivers/staging/media/starfive/camss/stf-camss.c
> > +++ b/drivers/staging/media/starfive/camss/stf-camss.c
> > @@ -8,6 +8,7 @@
> >   *
> >   * Author: Jack Zhu <jack.zhu@starfivetech.com>
> >   * Author: Changhuang Liang <changhuang.liang@starfivetech.com>
> > + * Author: Keith Zhao <keith.zhao@starfivetech.com>
> >   *
> >   */
> >  #include <linux/module.h>
> > @@ -126,6 +127,7 @@ static int stfcamss_of_parse_ports(struct stfcamss
> > *stfcamss)  static int stfcamss_register_devs(struct stfcamss
> > *stfcamss)  {
> >  	struct stf_capture *cap_yuv =
> &stfcamss->captures[STF_CAPTURE_YUV];
> > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> >  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> >  	int ret;
> >
> > @@ -150,8 +152,18 @@ static int stfcamss_register_devs(struct stfcamss
> > *stfcamss)
> >
> >  	cap_yuv->video.source_subdev = &isp_dev->subdev;
> >
> > +	ret = media_create_pad_link(&isp_dev->subdev.entity,
> STF_ISP_PAD_SRC_SCD,
> > +				    &cap_scd->video.vdev.entity, 0, 0);
> > +	if (ret)
> > +		goto err_rm_links0;
> 
> or just 'err_rm_links'
> 

Agreed.

> > +
> > +	cap_scd->video.source_subdev = &isp_dev->subdev;
> > +
> >  	return ret;
> 
> here you can return 0
> 

Okay.

> >
> > +err_rm_links0:
> > +	media_entity_remove_links(&isp_dev->subdev.entity);
> > +	media_entity_remove_links(&cap_yuv->video.vdev.entity);
> >  err_cap_unregister:
> >  	stf_capture_unregister(stfcamss);
> >  err_isp_unregister:
> > @@ -163,10 +175,12 @@ static int stfcamss_register_devs(struct
> > stfcamss *stfcamss)  static void stfcamss_unregister_devs(struct
> > stfcamss *stfcamss)  {
> >  	struct stf_capture *cap_yuv =
> &stfcamss->captures[STF_CAPTURE_YUV];
> > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> >  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> >
> >  	media_entity_remove_links(&isp_dev->subdev.entity);
> >  	media_entity_remove_links(&cap_yuv->video.vdev.entity);
> > +	media_entity_remove_links(&cap_scd->video.vdev.entity);
> >
> >  	stf_isp_unregister(&stfcamss->isp_dev);
> >  	stf_capture_unregister(stfcamss);
> > @@ -436,5 +450,6 @@ module_platform_driver(stfcamss_driver);
> >
> >  MODULE_AUTHOR("Jack Zhu <jack.zhu@starfivetech.com>");
> > MODULE_AUTHOR("Changhuang Liang
> <changhuang.liang@starfivetech.com>");
> > +MODULE_AUTHOR("Keith Zhao <keith.zhao@starfivetech.com>");
> >  MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
> > MODULE_LICENSE("GPL"); diff --git
> > a/drivers/staging/media/starfive/camss/stf-capture.c
> > b/drivers/staging/media/starfive/camss/stf-capture.c
> > index 75f6ef405e61..328b8c6e351d 100644
> > --- a/drivers/staging/media/starfive/camss/stf-capture.c
> > +++ b/drivers/staging/media/starfive/camss/stf-capture.c
> > @@ -12,6 +12,7 @@
> >  static const char * const stf_cap_names[] = {
> >  	"capture_raw",
> >  	"capture_yuv",
> > +	"capture_scd",
> >  };
> >
> >  static const struct stfcamss_format_info stf_wr_fmts[] = { @@ -55,6
> > +56,14 @@ static const struct stfcamss_format_info stf_isp_fmts[] = {
> >  	},
> >  };
> >
> > +/* 3A Statistics Collection Data */
> > +static const struct stfcamss_format_info stf_isp_scd_fmts[] = {
> > +	{
> > +		.code = MEDIA_BUS_FMT_METADATA_FIXED,
> > +		.pixelformat = V4L2_META_FMT_STF_ISP_STAT_3A,
> > +	},
> > +};
> > +
> >  static inline struct stf_capture *to_stf_capture(struct
> > stfcamss_video *video)  {
> >  	return container_of(video, struct stf_capture, video); @@ -84,6
> > +93,8 @@ static void stf_init_addrs(struct stfcamss_video *video)
> >  		stf_set_raw_addr(video->stfcamss, addr0);
> >  	else if (cap->type == STF_CAPTURE_YUV)
> >  		stf_set_yuv_addr(video->stfcamss, addr0, addr1);
> > +	else
> > +		stf_set_scd_addr(video->stfcamss, addr0, addr1, TYPE_AWB);
> >  }
> >
> >  static void stf_cap_s_cfg(struct stfcamss_video *video) @@ -227,18
> > +238,24 @@ static void stf_capture_init(struct stfcamss *stfcamss, struct
> stf_capture *cap)
> >  	INIT_LIST_HEAD(&cap->buffers.ready_bufs);
> >  	spin_lock_init(&cap->buffers.lock);
> >
> > -	cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> >  	cap->video.stfcamss = stfcamss;
> >  	cap->video.bpl_alignment = 16 * 8;
> >
> >  	if (cap->type == STF_CAPTURE_RAW) {
> > +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> >  		cap->video.formats = stf_wr_fmts;
> >  		cap->video.nformats = ARRAY_SIZE(stf_wr_fmts);
> >  		cap->video.bpl_alignment = 8;
> >  	} else if (cap->type == STF_CAPTURE_YUV) {
> > +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> >  		cap->video.formats = stf_isp_fmts;
> >  		cap->video.nformats = ARRAY_SIZE(stf_isp_fmts);
> >  		cap->video.bpl_alignment = 1;
> > +	} else {
> > +		cap->video.type = V4L2_BUF_TYPE_META_CAPTURE;
> > +		cap->video.formats = stf_isp_scd_fmts;
> > +		cap->video.nformats = ARRAY_SIZE(stf_isp_scd_fmts);
> > +		cap->video.bpl_alignment = 16 * 8;
> >  	}
> >  }
> >
> > @@ -362,9 +379,11 @@ void stf_capture_unregister(struct stfcamss
> > *stfcamss)  {
> >  	struct stf_capture *cap_raw =
> &stfcamss->captures[STF_CAPTURE_RAW];
> >  	struct stf_capture *cap_yuv =
> &stfcamss->captures[STF_CAPTURE_YUV];
> > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> >
> >  	stf_capture_unregister_one(cap_raw);
> >  	stf_capture_unregister_one(cap_yuv);
> > +	stf_capture_unregister_one(cap_scd);
> >  }
> >
> >  int stf_capture_register(struct stfcamss *stfcamss, diff --git
> > a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > index 6b3966ca18bf..3b18d09f2cc6 100644
> > --- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > +++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > @@ -451,11 +451,57 @@ void stf_set_yuv_addr(struct stfcamss *stfcamss,
> >  	stf_isp_reg_write(stfcamss, ISP_REG_UV_PLANE_START_ADDR,
> uv_addr);
> > }
> >
> > +static enum stf_isp_type_scd stf_isp_get_scd_type(struct stfcamss
> > +*stfcamss) {
> > +	int val;
> > +
> > +	val = stf_isp_reg_read(stfcamss, ISP_REG_SC_CFG_1);
> > +	return (enum stf_isp_type_scd)(val & ISP_SC_SEL_MASK) >> 30; }
> 
> So far used by a single caller, could be inlined
> 

Okay.

> > +
> > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > +		      enum stf_isp_type_scd type_scd) {
> > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> > +			    SEL_TYPE(type_scd));
> > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); }
> > +
> > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss, void
> > +*vaddr) {
> > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
> > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > +	u32 i;
> > +
> > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss, reg_addr);
> 
> If you have a contigous memory space to read, could memcpy_fromio() help
> instead of going through 64 reads ?
> 

I will try this function.

> > +}
> > +
> > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> > +			      enum stf_isp_type_scd *type_scd) {
> > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer
> > +*)vaddr;
> > +
> > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > +	if (*type_scd == TYPE_AWB) {
> > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > +		*type_scd = TYPE_OECF;
> > +	} else {
> > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > +		*type_scd = TYPE_AWB;
> 
> Is this correct ? Why are you overwriting the value read from HW that
> indicates AE/AF stats with AWB ones ?

The AWB frame and AE/AF frames will alternate, so the current frame indicates the AE/AF,
then set AWB type just for next AWB frame.

> 
> > +	}
> > +}
> > +
> >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> >  	struct stfcamss *stfcamss = priv;
> >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> >  	struct stfcamss_buffer *change_buf;
> > +	enum stf_isp_type_scd type_scd;
> > +	u32 value;
> >  	u32 status;
> >
> >  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0); @@ -467,6
> > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
> >  					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
> >  							 change_buf->addr[1]);
> >  			}
> > +
> > +			value = stf_isp_reg_read(stfcamss,
> ISP_REG_CSI_MODULE_CFG);
> > +			if (value & CSI_SC_EN) {
> > +				change_buf = stf_change_buffer(&cap_scd->buffers);
> > +				if (change_buf) {
> > +					stf_isp_fill_flag(stfcamss, change_buf->vaddr,
> > +							  &type_scd);
> > +					stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > +							 change_buf->addr[1], type_scd);
> 
> Sorry if I'm un-familiar with the HW but this seems to be the line-interrupt.
> Are you swapping buffers every line or it's just that you have a single line irq
> for the stats ?
> 

Every frame triggers a line-interrupt, and we will swap buffers in it.

> > +				}
> > +			}
> >  		}
> >
> >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@ -485,6 +542,7
> @@
> > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> >  	struct stfcamss *stfcamss = priv;
> >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> >  	struct stfcamss_buffer *ready_buf;
> >  	u32 status;
> >
> > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
> >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> VB2_BUF_STATE_DONE);
> >  		}
> >
> > +		if (status & ISPC_SC) {
> > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > +			if (ready_buf) {
> > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> VB2_BUF_STATE_DONE);
> > +			}
> > +		}
> > +
> >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> >  				  (status & ~ISPC_INT_ALL_MASK) |
> >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > a/drivers/staging/media/starfive/camss/stf-isp.h
> > b/drivers/staging/media/starfive/camss/stf-isp.h
> > index fcda0502e3b0..0af7b367e57a 100644
> > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > @@ -10,6 +10,7 @@
> >  #ifndef STF_ISP_H
> >  #define STF_ISP_H
> >
> > +#include <linux/jh7110-isp.h>
> >  #include <media/v4l2-subdev.h>
> >
> >  #include "stf-video.h"
> > @@ -107,6 +108,12 @@
> >  #define Y_COOR(y)				((y) << 16)
> >  #define X_COOR(x)				((x) << 0)
> >
> > +#define ISP_REG_SCD_CFG_0			0x098
> > +
> > +#define ISP_REG_SC_CFG_1			0x0bc
> > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > +#define SEL_TYPE(n)				((n) << 30)
> > +
> >  #define ISP_REG_LCCF_CFG_2			0x0e0
> >  #define ISP_REG_LCCF_CFG_3			0x0e4
> >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > @@ -305,6 +312,10 @@
> >  #define DNRM_F(n)				((n) << 16)
> >  #define CCM_M_DAT(n)				((n) << 0)
> >
> > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > +
> > +#define ISP_REG_YHIST_ACC_0			0xd00
> > +
> >  #define ISP_REG_GAMMA_VAL0			0xe00
> >  #define ISP_REG_GAMMA_VAL1			0xe04
> >  #define ISP_REG_GAMMA_VAL2			0xe08
> > @@ -389,6 +400,15 @@
> >  #define IMAGE_MAX_WIDTH				1920
> >  #define IMAGE_MAX_HEIGH				1080
> >
> > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> 
> Should this be in the uAPI header as it is useful to userspace as well ?
> 
> you could:
> 
> struct jh7110_isp_sc_buffer {
> 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> 	__u32 reserv0[33];
> 	__u32 bright_sc[4096];
> 	__u32 reserv1[96];
> 	__u32 ae_hist_y[128];
> 	__u32 reserv2[511];
> 	__u16 flag;
> };
> 
> ofc if the size is made part of the uAPI you need a more proper name such as
> JH7110_ISP_YHIST_SIZE
> 

OK, I will try this.

> > +
> > +enum stf_isp_type_scd {
> > +	TYPE_DEC = 0,
> > +	TYPE_OBC,
> > +	TYPE_OECF,
> > +	TYPE_AWB,
> > +};
> > +
> >  /* pad id for media framework */
> >  enum stf_isp_pad_id {
> >  	STF_ISP_PAD_SINK = 0,
> > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev
> > *isp_dev);
> >
> >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > +		      enum stf_isp_type_scd type_scd);
> >
> >  #endif /* STF_ISP_H */
> > diff --git a/drivers/staging/media/starfive/camss/stf-video.c
> > b/drivers/staging/media/starfive/camss/stf-video.c
> > index 989b5e82bae9..2203605ec9c7 100644
> > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > @@ -125,6 +125,14 @@ static int stf_video_init_format(struct
> stfcamss_video *video)
> >  	return 0;
> >  }
> >
> > +static int stf_video_scd_init_format(struct stfcamss_video *video)
> 
> Make it void if it can't fail (see below)
> 

OK.

> > +{
> > +	video->active_fmt.fmt.meta.dataformat =
> video->formats[0].pixelformat;
> > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > +jh7110_isp_sc_buffer);
> > +
> > +	return 0;
> > +}
> > +
> >  /* -----------------------------------------------------------------------------
> >   * Video queue operations
> >   */
> > @@ -330,6 +338,78 @@ static const struct vb2_ops stf_video_vb2_q_ops =
> {
> >  	.stop_streaming  = video_stop_streaming,  };
> >
> > +static int video_scd_queue_setup(struct vb2_queue *q,
> > +				 unsigned int *num_buffers,
> > +				 unsigned int *num_planes,
> > +				 unsigned int sizes[],
> > +				 struct device *alloc_devs[])
> > +{
> > +	if (*num_planes)
> > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL :
> > +0;
> > +
> > +	*num_planes = 1;
> > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > +
> > +	return 0;
> > +}
> > +
> > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > +	dma_addr_t *paddr;
> > +
> > +	paddr = vb2_plane_cookie(vb, 0);
> > +	buffer->addr[0] = *paddr;
> > +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> 
> Interesting, I don't see many users of vb2_plane_cookie() in mainline and I'm
> not sure what this gives you as you use it to program the following registers:
> 
> 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> 

We set the value for ISP hardware, then ISP will transfer the statistics to the buffer.
when the stf_isp_irq_handler interrupt is triggered, indicates that the buffer fill is 
complete.

> 
> > +	buffer->vaddr = vb2_plane_vaddr(vb, 0);
> > +
> > +	return 0;
> > +}
> > +
> > +static int video_scd_buf_prepare(struct vb2_buffer *vb) {
> > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > +
> > +	if (sizeof(struct jh7110_isp_sc_buffer) > vb2_plane_size(vb, 0))
> > +		return -EINVAL;
> > +
> > +	vb2_set_plane_payload(vb, 0, sizeof(struct jh7110_isp_sc_buffer));
> > +
> > +	vbuf->field = V4L2_FIELD_NONE;
> 
> is this necessary ?
> 

Will drop it.

> > +
> > +	return 0;
> > +}
> > +
> > +static int video_scd_start_streaming(struct vb2_queue *q, unsigned
> > +int count) {
> > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > +
> > +	video->ops->start_streaming(video);
> > +
> > +	return 0;
> > +}
> > +
> > +static void video_scd_stop_streaming(struct vb2_queue *q) {
> > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > +
> > +	video->ops->stop_streaming(video);
> > +
> > +	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR); }
> > +
> > +static const struct vb2_ops stf_video_scd_vb2_q_ops = {
> > +	.queue_setup     = video_scd_queue_setup,
> > +	.wait_prepare    = vb2_ops_wait_prepare,
> > +	.wait_finish     = vb2_ops_wait_finish,
> > +	.buf_init        = video_scd_buf_init,
> > +	.buf_prepare     = video_scd_buf_prepare,
> > +	.buf_queue       = video_buf_queue,
> > +	.start_streaming = video_scd_start_streaming,
> > +	.stop_streaming  = video_scd_stop_streaming, };
> > +
> >  /* -----------------------------------------------------------------------------
> >   * V4L2 ioctls
> >   */
> > @@ -448,6 +528,37 @@ static const struct v4l2_ioctl_ops stf_vid_ioctl_ops
> = {
> >  	.vidioc_streamoff               = vb2_ioctl_streamoff,
> >  };
> >
> > +static int video_scd_g_fmt(struct file *file, void *fh, struct
> > +v4l2_format *f) {
> > +	struct stfcamss_video *video = video_drvdata(file);
> > +	struct v4l2_meta_format *meta = &f->fmt.meta;
> > +
> > +	if (f->type != video->type)
> > +		return -EINVAL;
> > +
> > +	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
> > +	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_ioctl_ops stf_vid_scd_ioctl_ops = {
> > +	.vidioc_querycap                = video_querycap,
> > +	.vidioc_enum_fmt_meta_cap       = video_enum_fmt,
> > +	.vidioc_g_fmt_meta_cap          = video_scd_g_fmt,
> > +	.vidioc_s_fmt_meta_cap          = video_scd_g_fmt,
> > +	.vidioc_try_fmt_meta_cap        = video_scd_g_fmt,
> > +	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> > +	.vidioc_querybuf                = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf                    = vb2_ioctl_qbuf,
> > +	.vidioc_expbuf                  = vb2_ioctl_expbuf,
> > +	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> > +	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> > +	.vidioc_streamon                = vb2_ioctl_streamon,
> > +	.vidioc_streamoff               = vb2_ioctl_streamoff,
> > +};
> > +
> >  /* -----------------------------------------------------------------------------
> >   * V4L2 file operations
> >   */
> > @@ -473,6 +584,9 @@ static int stf_link_validate(struct media_link *link)
> >  	struct stfcamss_video *video = video_get_drvdata(vdev);
> >  	int ret;
> >
> > +	if (video->type == V4L2_BUF_TYPE_META_CAPTURE)
> > +		return 0;
> > +
> >  	ret = stf_video_check_format(video);
> >
> >  	return ret;
> > @@ -506,7 +620,11 @@ int stf_video_register(struct stfcamss_video
> *video,
> >  	q = &video->vb2_q;
> >  	q->drv_priv = video;
> >  	q->mem_ops = &vb2_dma_contig_memops;
> > -	q->ops = &stf_video_vb2_q_ops;
> > +
> > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> > +		q->ops = &stf_video_vb2_q_ops;
> > +	else
> > +		q->ops = &stf_video_scd_vb2_q_ops;
> >  	q->type = video->type;
> >  	q->io_modes = VB2_DMABUF | VB2_MMAP;
> >  	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > @@ -529,16 +647,28 @@ int stf_video_register(struct stfcamss_video
> *video,
> >  		goto err_mutex_destroy;
> >  	}
> >
> > -	ret = stf_video_init_format(video);
> > -	if (ret < 0) {
> > -		dev_err(video->stfcamss->dev,
> > -			"Failed to init format: %d\n", ret);
> > -		goto err_media_cleanup;
> > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
> > +		ret = stf_video_init_format(video);
> 
> I don't think this can fail
> 

This already exists, and I probably will not change it here.

> > +		if (ret < 0) {
> > +			dev_err(video->stfcamss->dev,
> > +				"Failed to init format: %d\n", ret);
> > +			goto err_media_cleanup;
> > +		}
> > +		vdev->ioctl_ops = &stf_vid_ioctl_ops;
> > +		vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
> > +	} else {
> > +		ret = stf_video_scd_init_format(video);
> 
> This can't fail as noted above
> 

I will change this return void.

> > +		if (ret < 0) {
> > +			dev_err(video->stfcamss->dev,
> > +				"Failed to init format: %d\n", ret);
> > +			goto err_media_cleanup;
> > +		}
> > +		vdev->ioctl_ops = &stf_vid_scd_ioctl_ops;
> > +		vdev->device_caps = V4L2_CAP_META_CAPTURE;
> >  	}
> >
> >  	vdev->fops = &stf_vid_fops;
> > -	vdev->ioctl_ops = &stf_vid_ioctl_ops;
> > -	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE |
> V4L2_CAP_STREAMING;
> > +	vdev->device_caps |= V4L2_CAP_STREAMING;
> >  	vdev->entity.ops = &stf_media_ops;
> >  	vdev->vfl_dir = VFL_DIR_RX;
> >  	vdev->release = stf_video_release;
> > diff --git a/drivers/staging/media/starfive/camss/stf-video.h
> > b/drivers/staging/media/starfive/camss/stf-video.h
> > index 59799b65cbe5..53a1cf4e59b7 100644
> > --- a/drivers/staging/media/starfive/camss/stf-video.h
> > +++ b/drivers/staging/media/starfive/camss/stf-video.h
> > @@ -37,6 +37,7 @@ enum stf_v_line_id {  enum stf_capture_type {
> >  	STF_CAPTURE_RAW = 0,
> >  	STF_CAPTURE_YUV,
> > +	STF_CAPTURE_SCD,
> >  	STF_CAPTURE_NUM,
> >  };
> >
> > --
> > 2.25.1
> >
> >

Regards,
Changhuang

Re: 回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Jacopo Mondi]

Hi Changhuang

On Thu, Jul 11, 2024 at 06:48:21AM GMT, Changhuang Liang wrote:
> Hi Jacopo
>
> Thanks for your comments.
>
> > Hi Changhuang
> >
> > On Tue, Jul 09, 2024 at 01:38:18AM GMT, Changhuang Liang wrote:
> > > Register ISP 3A "capture_scd" video device to receive statistics
> > > collection data.
> > >
> > > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > > ---
> > >  .../staging/media/starfive/camss/stf-buffer.h |   1 +
> > >  .../staging/media/starfive/camss/stf-camss.c  |  15 ++
> > >  .../media/starfive/camss/stf-capture.c        |  21 ++-
> > >  .../media/starfive/camss/stf-isp-hw-ops.c     |  66 ++++++++
> > >  .../staging/media/starfive/camss/stf-isp.h    |  23 +++
> > >  .../staging/media/starfive/camss/stf-video.c  | 146
> > +++++++++++++++++-
> > >  .../staging/media/starfive/camss/stf-video.h  |   1 +
> > >  7 files changed, 264 insertions(+), 9 deletions(-)
> > >
> > > diff --git a/drivers/staging/media/starfive/camss/stf-buffer.h
> > > b/drivers/staging/media/starfive/camss/stf-buffer.h
> > > index 9d1670fb05ed..727d00617448 100644
> > > --- a/drivers/staging/media/starfive/camss/stf-buffer.h
> > > +++ b/drivers/staging/media/starfive/camss/stf-buffer.h
> > > @@ -23,6 +23,7 @@ enum stf_v_state {
> > >  struct stfcamss_buffer {
> > >  	struct vb2_v4l2_buffer vb;
> > >  	dma_addr_t addr[2];
> > > +	void *vaddr;
> > >  	struct list_head queue;
> > >  };
> > >
> > > diff --git a/drivers/staging/media/starfive/camss/stf-camss.c
> > > b/drivers/staging/media/starfive/camss/stf-camss.c
> > > index fecd3e67c7a1..fafa3ce2f6da 100644
> > > --- a/drivers/staging/media/starfive/camss/stf-camss.c
> > > +++ b/drivers/staging/media/starfive/camss/stf-camss.c
> > > @@ -8,6 +8,7 @@
> > >   *
> > >   * Author: Jack Zhu <jack.zhu@starfivetech.com>
> > >   * Author: Changhuang Liang <changhuang.liang@starfivetech.com>
> > > + * Author: Keith Zhao <keith.zhao@starfivetech.com>
> > >   *
> > >   */
> > >  #include <linux/module.h>
> > > @@ -126,6 +127,7 @@ static int stfcamss_of_parse_ports(struct stfcamss
> > > *stfcamss)  static int stfcamss_register_devs(struct stfcamss
> > > *stfcamss)  {
> > >  	struct stf_capture *cap_yuv =
> > &stfcamss->captures[STF_CAPTURE_YUV];
> > > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> > >  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> > >  	int ret;
> > >
> > > @@ -150,8 +152,18 @@ static int stfcamss_register_devs(struct stfcamss
> > > *stfcamss)
> > >
> > >  	cap_yuv->video.source_subdev = &isp_dev->subdev;
> > >
> > > +	ret = media_create_pad_link(&isp_dev->subdev.entity,
> > STF_ISP_PAD_SRC_SCD,
> > > +				    &cap_scd->video.vdev.entity, 0, 0);
> > > +	if (ret)
> > > +		goto err_rm_links0;
> >
> > or just 'err_rm_links'
> >
>
> Agreed.
>
> > > +
> > > +	cap_scd->video.source_subdev = &isp_dev->subdev;
> > > +
> > >  	return ret;
> >
> > here you can return 0
> >
>
> Okay.
>
> > >
> > > +err_rm_links0:
> > > +	media_entity_remove_links(&isp_dev->subdev.entity);
> > > +	media_entity_remove_links(&cap_yuv->video.vdev.entity);
> > >  err_cap_unregister:
> > >  	stf_capture_unregister(stfcamss);
> > >  err_isp_unregister:
> > > @@ -163,10 +175,12 @@ static int stfcamss_register_devs(struct
> > > stfcamss *stfcamss)  static void stfcamss_unregister_devs(struct
> > > stfcamss *stfcamss)  {
> > >  	struct stf_capture *cap_yuv =
> > &stfcamss->captures[STF_CAPTURE_YUV];
> > > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> > >  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
> > >
> > >  	media_entity_remove_links(&isp_dev->subdev.entity);
> > >  	media_entity_remove_links(&cap_yuv->video.vdev.entity);
> > > +	media_entity_remove_links(&cap_scd->video.vdev.entity);
> > >
> > >  	stf_isp_unregister(&stfcamss->isp_dev);
> > >  	stf_capture_unregister(stfcamss);
> > > @@ -436,5 +450,6 @@ module_platform_driver(stfcamss_driver);
> > >
> > >  MODULE_AUTHOR("Jack Zhu <jack.zhu@starfivetech.com>");
> > > MODULE_AUTHOR("Changhuang Liang
> > <changhuang.liang@starfivetech.com>");
> > > +MODULE_AUTHOR("Keith Zhao <keith.zhao@starfivetech.com>");
> > >  MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
> > > MODULE_LICENSE("GPL"); diff --git
> > > a/drivers/staging/media/starfive/camss/stf-capture.c
> > > b/drivers/staging/media/starfive/camss/stf-capture.c
> > > index 75f6ef405e61..328b8c6e351d 100644
> > > --- a/drivers/staging/media/starfive/camss/stf-capture.c
> > > +++ b/drivers/staging/media/starfive/camss/stf-capture.c
> > > @@ -12,6 +12,7 @@
> > >  static const char * const stf_cap_names[] = {
> > >  	"capture_raw",
> > >  	"capture_yuv",
> > > +	"capture_scd",
> > >  };
> > >
> > >  static const struct stfcamss_format_info stf_wr_fmts[] = { @@ -55,6
> > > +56,14 @@ static const struct stfcamss_format_info stf_isp_fmts[] = {
> > >  	},
> > >  };
> > >
> > > +/* 3A Statistics Collection Data */
> > > +static const struct stfcamss_format_info stf_isp_scd_fmts[] = {
> > > +	{
> > > +		.code = MEDIA_BUS_FMT_METADATA_FIXED,
> > > +		.pixelformat = V4L2_META_FMT_STF_ISP_STAT_3A,
> > > +	},
> > > +};
> > > +
> > >  static inline struct stf_capture *to_stf_capture(struct
> > > stfcamss_video *video)  {
> > >  	return container_of(video, struct stf_capture, video); @@ -84,6
> > > +93,8 @@ static void stf_init_addrs(struct stfcamss_video *video)
> > >  		stf_set_raw_addr(video->stfcamss, addr0);
> > >  	else if (cap->type == STF_CAPTURE_YUV)
> > >  		stf_set_yuv_addr(video->stfcamss, addr0, addr1);
> > > +	else
> > > +		stf_set_scd_addr(video->stfcamss, addr0, addr1, TYPE_AWB);
> > >  }
> > >
> > >  static void stf_cap_s_cfg(struct stfcamss_video *video) @@ -227,18
> > > +238,24 @@ static void stf_capture_init(struct stfcamss *stfcamss, struct
> > stf_capture *cap)
> > >  	INIT_LIST_HEAD(&cap->buffers.ready_bufs);
> > >  	spin_lock_init(&cap->buffers.lock);
> > >
> > > -	cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > >  	cap->video.stfcamss = stfcamss;
> > >  	cap->video.bpl_alignment = 16 * 8;
> > >
> > >  	if (cap->type == STF_CAPTURE_RAW) {
> > > +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > >  		cap->video.formats = stf_wr_fmts;
> > >  		cap->video.nformats = ARRAY_SIZE(stf_wr_fmts);
> > >  		cap->video.bpl_alignment = 8;
> > >  	} else if (cap->type == STF_CAPTURE_YUV) {
> > > +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > >  		cap->video.formats = stf_isp_fmts;
> > >  		cap->video.nformats = ARRAY_SIZE(stf_isp_fmts);
> > >  		cap->video.bpl_alignment = 1;
> > > +	} else {
> > > +		cap->video.type = V4L2_BUF_TYPE_META_CAPTURE;
> > > +		cap->video.formats = stf_isp_scd_fmts;
> > > +		cap->video.nformats = ARRAY_SIZE(stf_isp_scd_fmts);
> > > +		cap->video.bpl_alignment = 16 * 8;
> > >  	}
> > >  }
> > >
> > > @@ -362,9 +379,11 @@ void stf_capture_unregister(struct stfcamss
> > > *stfcamss)  {
> > >  	struct stf_capture *cap_raw =
> > &stfcamss->captures[STF_CAPTURE_RAW];
> > >  	struct stf_capture *cap_yuv =
> > &stfcamss->captures[STF_CAPTURE_YUV];
> > > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> > >
> > >  	stf_capture_unregister_one(cap_raw);
> > >  	stf_capture_unregister_one(cap_yuv);
> > > +	stf_capture_unregister_one(cap_scd);
> > >  }
> > >
> > >  int stf_capture_register(struct stfcamss *stfcamss, diff --git
> > > a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > > b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > > index 6b3966ca18bf..3b18d09f2cc6 100644
> > > --- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > > +++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > > @@ -451,11 +451,57 @@ void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > >  	stf_isp_reg_write(stfcamss, ISP_REG_UV_PLANE_START_ADDR,
> > uv_addr);
> > > }
> > >
> > > +static enum stf_isp_type_scd stf_isp_get_scd_type(struct stfcamss
> > > +*stfcamss) {
> > > +	int val;
> > > +
> > > +	val = stf_isp_reg_read(stfcamss, ISP_REG_SC_CFG_1);
> > > +	return (enum stf_isp_type_scd)(val & ISP_SC_SEL_MASK) >> 30; }
> >
> > So far used by a single caller, could be inlined
> >
>
> Okay.
>
> > > +
> > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > +		      enum stf_isp_type_scd type_scd) {
> > > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> > > +			    SEL_TYPE(type_scd));
> > > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); }
> > > +
> > > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss, void
> > > +*vaddr) {
> > > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
> > > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > > +	u32 i;
> > > +
> > > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss, reg_addr);
> >
> > If you have a contigous memory space to read, could memcpy_fromio() help
> > instead of going through 64 reads ?
> >
>
> I will try this function.
>
> > > +}
> > > +
> > > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> > > +			      enum stf_isp_type_scd *type_scd) {
> > > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer
> > > +*)vaddr;
> > > +
> > > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > > +	if (*type_scd == TYPE_AWB) {
> > > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > > +		*type_scd = TYPE_OECF;
> > > +	} else {
> > > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > > +		*type_scd = TYPE_AWB;
> >
> > Is this correct ? Why are you overwriting the value read from HW that
> > indicates AE/AF stats with AWB ones ?
>
> The AWB frame and AE/AF frames will alternate, so the current frame indicates the AE/AF,
> then set AWB type just for next AWB frame.
>

Ah! Shouldn't it be userspace configuring which type of statistics it
wants to receive instead of the driver alternating the two ?

> >
> > > +	}
> > > +}
> > > +
> > >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> > >  	struct stfcamss *stfcamss = priv;
> > >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> > >  	struct stfcamss_buffer *change_buf;
> > > +	enum stf_isp_type_scd type_scd;
> > > +	u32 value;
> > >  	u32 status;
> > >
> > >  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0); @@ -467,6
> > > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
> > >  					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
> > >  							 change_buf->addr[1]);
> > >  			}
> > > +
> > > +			value = stf_isp_reg_read(stfcamss,
> > ISP_REG_CSI_MODULE_CFG);
> > > +			if (value & CSI_SC_EN) {
> > > +				change_buf = stf_change_buffer(&cap_scd->buffers);
> > > +				if (change_buf) {
> > > +					stf_isp_fill_flag(stfcamss, change_buf->vaddr,
> > > +							  &type_scd);
> > > +					stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > > +							 change_buf->addr[1], type_scd);
> >
> > Sorry if I'm un-familiar with the HW but this seems to be the line-interrupt.
> > Are you swapping buffers every line or it's just that you have a single line irq
> > for the stats ?
> >
>
> Every frame triggers a line-interrupt, and we will swap buffers in it.
>

ah, frames completion triggers a line-interrupt ?

> > > +				}
> > > +			}
> > >  		}
> > >
> > >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@ -485,6 +542,7
> > @@
> > > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> > >  	struct stfcamss *stfcamss = priv;
> > >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > > +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> > >  	struct stfcamss_buffer *ready_buf;
> > >  	u32 status;
> > >
> > > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
> > >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > VB2_BUF_STATE_DONE);
> > >  		}
> > >
> > > +		if (status & ISPC_SC) {
> > > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > > +			if (ready_buf) {
> > > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > VB2_BUF_STATE_DONE);
> > > +			}
> > > +		}
> > > +
> > >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> > >  				  (status & ~ISPC_INT_ALL_MASK) |
> > >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > > a/drivers/staging/media/starfive/camss/stf-isp.h
> > > b/drivers/staging/media/starfive/camss/stf-isp.h
> > > index fcda0502e3b0..0af7b367e57a 100644
> > > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > > @@ -10,6 +10,7 @@
> > >  #ifndef STF_ISP_H
> > >  #define STF_ISP_H
> > >
> > > +#include <linux/jh7110-isp.h>
> > >  #include <media/v4l2-subdev.h>
> > >
> > >  #include "stf-video.h"
> > > @@ -107,6 +108,12 @@
> > >  #define Y_COOR(y)				((y) << 16)
> > >  #define X_COOR(x)				((x) << 0)
> > >
> > > +#define ISP_REG_SCD_CFG_0			0x098
> > > +
> > > +#define ISP_REG_SC_CFG_1			0x0bc
> > > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > > +#define SEL_TYPE(n)				((n) << 30)
> > > +
> > >  #define ISP_REG_LCCF_CFG_2			0x0e0
> > >  #define ISP_REG_LCCF_CFG_3			0x0e4
> > >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > > @@ -305,6 +312,10 @@
> > >  #define DNRM_F(n)				((n) << 16)
> > >  #define CCM_M_DAT(n)				((n) << 0)
> > >
> > > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > > +
> > > +#define ISP_REG_YHIST_ACC_0			0xd00
> > > +
> > >  #define ISP_REG_GAMMA_VAL0			0xe00
> > >  #define ISP_REG_GAMMA_VAL1			0xe04
> > >  #define ISP_REG_GAMMA_VAL2			0xe08
> > > @@ -389,6 +400,15 @@
> > >  #define IMAGE_MAX_WIDTH				1920
> > >  #define IMAGE_MAX_HEIGH				1080
> > >
> > > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> >
> > Should this be in the uAPI header as it is useful to userspace as well ?
> >
> > you could:
> >
> > struct jh7110_isp_sc_buffer {
> > 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> > 	__u32 reserv0[33];
> > 	__u32 bright_sc[4096];
> > 	__u32 reserv1[96];
> > 	__u32 ae_hist_y[128];
> > 	__u32 reserv2[511];
> > 	__u16 flag;
> > };
> >
> > ofc if the size is made part of the uAPI you need a more proper name such as
> > JH7110_ISP_YHIST_SIZE
> >
>
> OK, I will try this.
>
> > > +
> > > +enum stf_isp_type_scd {
> > > +	TYPE_DEC = 0,
> > > +	TYPE_OBC,
> > > +	TYPE_OECF,
> > > +	TYPE_AWB,
> > > +};
> > > +
> > >  /* pad id for media framework */
> > >  enum stf_isp_pad_id {
> > >  	STF_ISP_PAD_SINK = 0,
> > > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev
> > > *isp_dev);
> > >
> > >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > +		      enum stf_isp_type_scd type_scd);
> > >
> > >  #endif /* STF_ISP_H */
> > > diff --git a/drivers/staging/media/starfive/camss/stf-video.c
> > > b/drivers/staging/media/starfive/camss/stf-video.c
> > > index 989b5e82bae9..2203605ec9c7 100644
> > > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > > @@ -125,6 +125,14 @@ static int stf_video_init_format(struct
> > stfcamss_video *video)
> > >  	return 0;
> > >  }
> > >
> > > +static int stf_video_scd_init_format(struct stfcamss_video *video)
> >
> > Make it void if it can't fail (see below)
> >
>
> OK.
>
> > > +{
> > > +	video->active_fmt.fmt.meta.dataformat =
> > video->formats[0].pixelformat;
> > > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > > +jh7110_isp_sc_buffer);
> > > +
> > > +	return 0;
> > > +}
> > > +
> > >  /* -----------------------------------------------------------------------------
> > >   * Video queue operations
> > >   */
> > > @@ -330,6 +338,78 @@ static const struct vb2_ops stf_video_vb2_q_ops =
> > {
> > >  	.stop_streaming  = video_stop_streaming,  };
> > >
> > > +static int video_scd_queue_setup(struct vb2_queue *q,
> > > +				 unsigned int *num_buffers,
> > > +				 unsigned int *num_planes,
> > > +				 unsigned int sizes[],
> > > +				 struct device *alloc_devs[])
> > > +{
> > > +	if (*num_planes)
> > > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL :
> > > +0;
> > > +
> > > +	*num_planes = 1;
> > > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > > +	dma_addr_t *paddr;
> > > +
> > > +	paddr = vb2_plane_cookie(vb, 0);
> > > +	buffer->addr[0] = *paddr;
> > > +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> >
> > Interesting, I don't see many users of vb2_plane_cookie() in mainline and I'm
> > not sure what this gives you as you use it to program the following registers:
> >
> > 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> >
>
> We set the value for ISP hardware, then ISP will transfer the statistics to the buffer.
> when the stf_isp_irq_handler interrupt is triggered, indicates that the buffer fill is
> complete.
>

So I take this as

	paddr = vb2_plane_cookie(vb, 0);
	buffer->addr[0] = *paddr;
	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;

        stf_set_scd_addr(stfcamss, change_buf->addr[0],
                         change_buf->addr[1], type_scd);

Makes the ISP transfer data directly to the memory areas in addr[0]
and addr[1] (which explains why struct jh7110_isp_sc_buffer is packed,
as it has to match the HW registers layout)

If this is the case, why are you then manually copying the histograms
and the flags to vaddr ?

                ready_buf = stf_buf_done(&cap_scd->buffers);
                if (ready_buf) {
                        stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
                        vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
                }

                change_buf = stf_change_buffer(&cap_scd->buffers);
                if (change_buf) {
                        stf_isp_fill_flag(stfcamss, change_buf->vaddr,
                                          &type_scd);

If I read vb2_dc_alloc_coherent() right 'cookie' == 'vaddr'

static int vb2_dc_alloc_coherent(struct vb2_dc_buf *buf)
{
	struct vb2_queue *q = buf->vb->vb2_queue;

	buf->cookie = dma_alloc_attrs(buf->dev,
				      buf->size,
				      &buf->dma_addr,
				      GFP_KERNEL | q->gfp_flags,
				      buf->attrs);
	if (!buf->cookie)
		return -ENOMEM;

	if (q->dma_attrs & DMA_ATTR_NO_KERNEL_MAPPING)
		return 0;

	buf->vaddr = buf->cookie;
	return 0;
}

Could you verify what you get by printing out 'paddr' and 'vaddr' in
video_scd_buf_init() ?


> >
> > > +	buffer->vaddr = vb2_plane_vaddr(vb, 0);
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int video_scd_buf_prepare(struct vb2_buffer *vb) {
> > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > +
> > > +	if (sizeof(struct jh7110_isp_sc_buffer) > vb2_plane_size(vb, 0))
> > > +		return -EINVAL;
> > > +
> > > +	vb2_set_plane_payload(vb, 0, sizeof(struct jh7110_isp_sc_buffer));
> > > +
> > > +	vbuf->field = V4L2_FIELD_NONE;
> >
> > is this necessary ?
> >
>
> Will drop it.
>
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int video_scd_start_streaming(struct vb2_queue *q, unsigned
> > > +int count) {
> > > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > > +
> > > +	video->ops->start_streaming(video);
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static void video_scd_stop_streaming(struct vb2_queue *q) {
> > > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > > +
> > > +	video->ops->stop_streaming(video);
> > > +
> > > +	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR); }
> > > +
> > > +static const struct vb2_ops stf_video_scd_vb2_q_ops = {
> > > +	.queue_setup     = video_scd_queue_setup,
> > > +	.wait_prepare    = vb2_ops_wait_prepare,
> > > +	.wait_finish     = vb2_ops_wait_finish,
> > > +	.buf_init        = video_scd_buf_init,
> > > +	.buf_prepare     = video_scd_buf_prepare,
> > > +	.buf_queue       = video_buf_queue,
> > > +	.start_streaming = video_scd_start_streaming,
> > > +	.stop_streaming  = video_scd_stop_streaming, };
> > > +
> > >  /* -----------------------------------------------------------------------------
> > >   * V4L2 ioctls
> > >   */
> > > @@ -448,6 +528,37 @@ static const struct v4l2_ioctl_ops stf_vid_ioctl_ops
> > = {
> > >  	.vidioc_streamoff               = vb2_ioctl_streamoff,
> > >  };
> > >
> > > +static int video_scd_g_fmt(struct file *file, void *fh, struct
> > > +v4l2_format *f) {
> > > +	struct stfcamss_video *video = video_drvdata(file);
> > > +	struct v4l2_meta_format *meta = &f->fmt.meta;
> > > +
> > > +	if (f->type != video->type)
> > > +		return -EINVAL;
> > > +
> > > +	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
> > > +	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static const struct v4l2_ioctl_ops stf_vid_scd_ioctl_ops = {
> > > +	.vidioc_querycap                = video_querycap,
> > > +	.vidioc_enum_fmt_meta_cap       = video_enum_fmt,
> > > +	.vidioc_g_fmt_meta_cap          = video_scd_g_fmt,
> > > +	.vidioc_s_fmt_meta_cap          = video_scd_g_fmt,
> > > +	.vidioc_try_fmt_meta_cap        = video_scd_g_fmt,
> > > +	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> > > +	.vidioc_querybuf                = vb2_ioctl_querybuf,
> > > +	.vidioc_qbuf                    = vb2_ioctl_qbuf,
> > > +	.vidioc_expbuf                  = vb2_ioctl_expbuf,
> > > +	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> > > +	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
> > > +	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> > > +	.vidioc_streamon                = vb2_ioctl_streamon,
> > > +	.vidioc_streamoff               = vb2_ioctl_streamoff,
> > > +};
> > > +
> > >  /* -----------------------------------------------------------------------------
> > >   * V4L2 file operations
> > >   */
> > > @@ -473,6 +584,9 @@ static int stf_link_validate(struct media_link *link)
> > >  	struct stfcamss_video *video = video_get_drvdata(vdev);
> > >  	int ret;
> > >
> > > +	if (video->type == V4L2_BUF_TYPE_META_CAPTURE)
> > > +		return 0;
> > > +
> > >  	ret = stf_video_check_format(video);
> > >
> > >  	return ret;
> > > @@ -506,7 +620,11 @@ int stf_video_register(struct stfcamss_video
> > *video,
> > >  	q = &video->vb2_q;
> > >  	q->drv_priv = video;
> > >  	q->mem_ops = &vb2_dma_contig_memops;
> > > -	q->ops = &stf_video_vb2_q_ops;
> > > +
> > > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> > > +		q->ops = &stf_video_vb2_q_ops;
> > > +	else
> > > +		q->ops = &stf_video_scd_vb2_q_ops;
> > >  	q->type = video->type;
> > >  	q->io_modes = VB2_DMABUF | VB2_MMAP;
> > >  	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > > @@ -529,16 +647,28 @@ int stf_video_register(struct stfcamss_video
> > *video,
> > >  		goto err_mutex_destroy;
> > >  	}
> > >
> > > -	ret = stf_video_init_format(video);
> > > -	if (ret < 0) {
> > > -		dev_err(video->stfcamss->dev,
> > > -			"Failed to init format: %d\n", ret);
> > > -		goto err_media_cleanup;
> > > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
> > > +		ret = stf_video_init_format(video);
> >
> > I don't think this can fail
> >
>
> This already exists, and I probably will not change it here.
>

No problem! Maybe something for later when de-staging the driver.

Thanks
  j

> > > +		if (ret < 0) {
> > > +			dev_err(video->stfcamss->dev,
> > > +				"Failed to init format: %d\n", ret);
> > > +			goto err_media_cleanup;
> > > +		}
> > > +		vdev->ioctl_ops = &stf_vid_ioctl_ops;
> > > +		vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
> > > +	} else {
> > > +		ret = stf_video_scd_init_format(video);
> >
> > This can't fail as noted above
> >
>
> I will change this return void.
>
> > > +		if (ret < 0) {
> > > +			dev_err(video->stfcamss->dev,
> > > +				"Failed to init format: %d\n", ret);
> > > +			goto err_media_cleanup;
> > > +		}
> > > +		vdev->ioctl_ops = &stf_vid_scd_ioctl_ops;
> > > +		vdev->device_caps = V4L2_CAP_META_CAPTURE;
> > >  	}
> > >
> > >  	vdev->fops = &stf_vid_fops;
> > > -	vdev->ioctl_ops = &stf_vid_ioctl_ops;
> > > -	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE |
> > V4L2_CAP_STREAMING;
> > > +	vdev->device_caps |= V4L2_CAP_STREAMING;
> > >  	vdev->entity.ops = &stf_media_ops;
> > >  	vdev->vfl_dir = VFL_DIR_RX;
> > >  	vdev->release = stf_video_release;
> > > diff --git a/drivers/staging/media/starfive/camss/stf-video.h
> > > b/drivers/staging/media/starfive/camss/stf-video.h
> > > index 59799b65cbe5..53a1cf4e59b7 100644
> > > --- a/drivers/staging/media/starfive/camss/stf-video.h
> > > +++ b/drivers/staging/media/starfive/camss/stf-video.h
> > > @@ -37,6 +37,7 @@ enum stf_v_line_id {  enum stf_capture_type {
> > >  	STF_CAPTURE_RAW = 0,
> > >  	STF_CAPTURE_YUV,
> > > +	STF_CAPTURE_SCD,
> > >  	STF_CAPTURE_NUM,
> > >  };
> > >
> > > --
> > > 2.25.1
> > >
> > >
>
> Regards,
> Changhuang
>

回复: 回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Changhuang Liang]

Hi, Jacopo

[...]
> > > > +
> > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > +		      enum stf_isp_type_scd type_scd) {
> > > > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> > > > +			    SEL_TYPE(type_scd));
> > > > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); }
> > > > +
> > > > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss, void
> > > > +*vaddr) {
> > > > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer
> *)vaddr;
> > > > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > > > +	u32 i;
> > > > +
> > > > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > > > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss, reg_addr);
> > >
> > > If you have a contigous memory space to read, could memcpy_fromio()
> > > help instead of going through 64 reads ?
> > >
> >
> > I will try this function.
> >
> > > > +}
> > > > +
> > > > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> > > > +			      enum stf_isp_type_scd *type_scd) {
> > > > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer
> > > > +*)vaddr;
> > > > +
> > > > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > > > +	if (*type_scd == TYPE_AWB) {
> > > > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > > > +		*type_scd = TYPE_OECF;
> > > > +	} else {
> > > > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > > > +		*type_scd = TYPE_AWB;
> > >
> > > Is this correct ? Why are you overwriting the value read from HW
> > > that indicates AE/AF stats with AWB ones ?
> >
> > The AWB frame and AE/AF frames will alternate, so the current frame
> > indicates the AE/AF, then set AWB type just for next AWB frame.
> >
> 
> Ah! Shouldn't it be userspace configuring which type of statistics it wants to
> receive instead of the driver alternating the two ?
> 

No, this is determined by hardware, cannot be configured by userspace.

> > >
> > > > +	}
> > > > +}
> > > > +
> > > >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> > > >  	struct stfcamss *stfcamss = priv;
> > > >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > > > +	struct stf_capture *cap_scd =
> > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > >  	struct stfcamss_buffer *change_buf;
> > > > +	enum stf_isp_type_scd type_scd;
> > > > +	u32 value;
> > > >  	u32 status;
> > > >
> > > >  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0); @@
> > > > -467,6
> > > > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
> > > >  					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
> > > >  							 change_buf->addr[1]);
> > > >  			}
> > > > +
> > > > +			value = stf_isp_reg_read(stfcamss,
> > > ISP_REG_CSI_MODULE_CFG);
> > > > +			if (value & CSI_SC_EN) {
> > > > +				change_buf = stf_change_buffer(&cap_scd->buffers);
> > > > +				if (change_buf) {
> > > > +					stf_isp_fill_flag(stfcamss, change_buf->vaddr,
> > > > +							  &type_scd);
> > > > +					stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > > > +							 change_buf->addr[1], type_scd);
> > >
> > > Sorry if I'm un-familiar with the HW but this seems to be the line-interrupt.
> > > Are you swapping buffers every line or it's just that you have a
> > > single line irq for the stats ?
> > >
> >
> > Every frame triggers a line-interrupt, and we will swap buffers in it.
> >
> 
> ah, frames completion triggers a line-interrupt ?
> 

Every frame will trigger line-interrupt and stf_isp_irq_handler.
We use line-interrupt changing buffer, the stf_isp_irq_handler will indicate that
image transfer to DDR is complete.


> > > > +				}
> > > > +			}
> > > >  		}
> > > >
> > > >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@ -485,6
> +542,7
> > > @@
> > > > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> > > >  	struct stfcamss *stfcamss = priv;
> > > >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > > > +	struct stf_capture *cap_scd =
> > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > >  	struct stfcamss_buffer *ready_buf;
> > > >  	u32 status;
> > > >
> > > > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
> > > >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > VB2_BUF_STATE_DONE);
> > > >  		}
> > > >
> > > > +		if (status & ISPC_SC) {
> > > > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > > > +			if (ready_buf) {
> > > > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > > > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > VB2_BUF_STATE_DONE);
> > > > +			}
> > > > +		}
> > > > +
> > > >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> > > >  				  (status & ~ISPC_INT_ALL_MASK) |
> > > >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > > > a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > index fcda0502e3b0..0af7b367e57a 100644
> > > > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > @@ -10,6 +10,7 @@
> > > >  #ifndef STF_ISP_H
> > > >  #define STF_ISP_H
> > > >
> > > > +#include <linux/jh7110-isp.h>
> > > >  #include <media/v4l2-subdev.h>
> > > >
> > > >  #include "stf-video.h"
> > > > @@ -107,6 +108,12 @@
> > > >  #define Y_COOR(y)				((y) << 16)
> > > >  #define X_COOR(x)				((x) << 0)
> > > >
> > > > +#define ISP_REG_SCD_CFG_0			0x098
> > > > +
> > > > +#define ISP_REG_SC_CFG_1			0x0bc
> > > > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > > > +#define SEL_TYPE(n)				((n) << 30)
> > > > +
> > > >  #define ISP_REG_LCCF_CFG_2			0x0e0
> > > >  #define ISP_REG_LCCF_CFG_3			0x0e4
> > > >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > > > @@ -305,6 +312,10 @@
> > > >  #define DNRM_F(n)				((n) << 16)
> > > >  #define CCM_M_DAT(n)				((n) << 0)
> > > >
> > > > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > > > +
> > > > +#define ISP_REG_YHIST_ACC_0			0xd00
> > > > +
> > > >  #define ISP_REG_GAMMA_VAL0			0xe00
> > > >  #define ISP_REG_GAMMA_VAL1			0xe04
> > > >  #define ISP_REG_GAMMA_VAL2			0xe08
> > > > @@ -389,6 +400,15 @@
> > > >  #define IMAGE_MAX_WIDTH				1920
> > > >  #define IMAGE_MAX_HEIGH				1080
> > > >
> > > > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> > >
> > > Should this be in the uAPI header as it is useful to userspace as well ?
> > >
> > > you could:
> > >
> > > struct jh7110_isp_sc_buffer {
> > > 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> > > 	__u32 reserv0[33];
> > > 	__u32 bright_sc[4096];
> > > 	__u32 reserv1[96];
> > > 	__u32 ae_hist_y[128];
> > > 	__u32 reserv2[511];
> > > 	__u16 flag;
> > > };
> > >
> > > ofc if the size is made part of the uAPI you need a more proper name
> > > such as JH7110_ISP_YHIST_SIZE
> > >
> >
> > OK, I will try this.
> >
> > > > +
> > > > +enum stf_isp_type_scd {
> > > > +	TYPE_DEC = 0,
> > > > +	TYPE_OBC,
> > > > +	TYPE_OECF,
> > > > +	TYPE_AWB,
> > > > +};
> > > > +
> > > >  /* pad id for media framework */
> > > >  enum stf_isp_pad_id {
> > > >  	STF_ISP_PAD_SINK = 0,
> > > > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev
> > > > *isp_dev);
> > > >
> > > >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > > >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > +		      enum stf_isp_type_scd type_scd);
> > > >
> > > >  #endif /* STF_ISP_H */
> > > > diff --git a/drivers/staging/media/starfive/camss/stf-video.c
> > > > b/drivers/staging/media/starfive/camss/stf-video.c
> > > > index 989b5e82bae9..2203605ec9c7 100644
> > > > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > > > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > > > @@ -125,6 +125,14 @@ static int stf_video_init_format(struct
> > > stfcamss_video *video)
> > > >  	return 0;
> > > >  }
> > > >
> > > > +static int stf_video_scd_init_format(struct stfcamss_video
> > > > +*video)
> > >
> > > Make it void if it can't fail (see below)
> > >
> >
> > OK.
> >
> > > > +{
> > > > +	video->active_fmt.fmt.meta.dataformat =
> > > video->formats[0].pixelformat;
> > > > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > > > +jh7110_isp_sc_buffer);
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > >  /* -----------------------------------------------------------------------------
> > > >   * Video queue operations
> > > >   */
> > > > @@ -330,6 +338,78 @@ static const struct vb2_ops
> > > > stf_video_vb2_q_ops =
> > > {
> > > >  	.stop_streaming  = video_stop_streaming,  };
> > > >
> > > > +static int video_scd_queue_setup(struct vb2_queue *q,
> > > > +				 unsigned int *num_buffers,
> > > > +				 unsigned int *num_planes,
> > > > +				 unsigned int sizes[],
> > > > +				 struct device *alloc_devs[]) {
> > > > +	if (*num_planes)
> > > > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL :
> > > > +0;
> > > > +
> > > > +	*num_planes = 1;
> > > > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > > > +	dma_addr_t *paddr;
> > > > +
> > > > +	paddr = vb2_plane_cookie(vb, 0);
> > > > +	buffer->addr[0] = *paddr;
> > > > +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > >
> > > Interesting, I don't see many users of vb2_plane_cookie() in
> > > mainline and I'm not sure what this gives you as you use it to program the
> following registers:
> > >
> > > 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> > >
> >
> > We set the value for ISP hardware, then ISP will transfer the statistics to the
> buffer.
> > when the stf_isp_irq_handler interrupt is triggered, indicates that
> > the buffer fill is complete.
> >
> 
> So I take this as
> 
> 	paddr = vb2_plane_cookie(vb, 0);
> 	buffer->addr[0] = *paddr;
> 	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> 
>         stf_set_scd_addr(stfcamss, change_buf->addr[0],
>                          change_buf->addr[1], type_scd);
> 
> Makes the ISP transfer data directly to the memory areas in addr[0] and addr[1]
> (which explains why struct jh7110_isp_sc_buffer is packed, as it has to match
> the HW registers layout)
> 
> If this is the case, why are you then manually copying the histograms and the
> flags to vaddr ?
> 

Yes, your are right.
But actually there is a problem with our ISP RTL.
We set this yhist_addr to ISP, but it actually not work.
	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
or I will drop this line in next version.

So, in this structure
struct jh7110_isp_sc_buffer {
	__u32 y_histogram[64];
	__u32 reserv0[33];
	__u32 bright_sc[4096];
	__u32 reserv1[96];
	__u32 ae_hist_y[128];
	__u32 reserv2[511];
	__u16 flag;
};

Only 

	__u32 reserv0[33];
	__u32 bright_sc[4096];
	__u32 reserv1[96];
	__u32 ae_hist_y[128];
	__u32 reserv2[511];

transfer by ISP hardware.

I need to fill 
	__u32 y_histogram[64];
	__u16 flag;

by vaddr.

Regards,
Changhuang

>                 ready_buf = stf_buf_done(&cap_scd->buffers);
>                 if (ready_buf) {
>                         stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
>                         vb2_buffer_done(&ready_buf->vb.vb2_buf,
> VB2_BUF_STATE_DONE);
>                 }
> 
>                 change_buf = stf_change_buffer(&cap_scd->buffers);
>                 if (change_buf) {
>                         stf_isp_fill_flag(stfcamss, change_buf->vaddr,
>                                           &type_scd);
> 
> If I read vb2_dc_alloc_coherent() right 'cookie' == 'vaddr'
> 
> static int vb2_dc_alloc_coherent(struct vb2_dc_buf *buf) {
> 	struct vb2_queue *q = buf->vb->vb2_queue;
> 
> 	buf->cookie = dma_alloc_attrs(buf->dev,
> 				      buf->size,
> 				      &buf->dma_addr,
> 				      GFP_KERNEL | q->gfp_flags,
> 				      buf->attrs);
> 	if (!buf->cookie)
> 		return -ENOMEM;
> 
> 	if (q->dma_attrs & DMA_ATTR_NO_KERNEL_MAPPING)
> 		return 0;
> 
> 	buf->vaddr = buf->cookie;
> 	return 0;
> }
> 
> Could you verify what you get by printing out 'paddr' and 'vaddr' in
> video_scd_buf_init() ?
> 
> 
> > >
> > > > +	buffer->vaddr = vb2_plane_vaddr(vb, 0);
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static int video_scd_buf_prepare(struct vb2_buffer *vb) {
> > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > +
> > > > +	if (sizeof(struct jh7110_isp_sc_buffer) > vb2_plane_size(vb, 0))
> > > > +		return -EINVAL;
> > > > +
> > > > +	vb2_set_plane_payload(vb, 0, sizeof(struct
> > > > +jh7110_isp_sc_buffer));
> > > > +
> > > > +	vbuf->field = V4L2_FIELD_NONE;
> > >
> > > is this necessary ?
> > >
> >
> > Will drop it.
> >
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static int video_scd_start_streaming(struct vb2_queue *q,
> > > > +unsigned int count) {
> > > > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > > > +
> > > > +	video->ops->start_streaming(video);
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static void video_scd_stop_streaming(struct vb2_queue *q) {
> > > > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > > > +
> > > > +	video->ops->stop_streaming(video);
> > > > +
> > > > +	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR); }
> > > > +
> > > > +static const struct vb2_ops stf_video_scd_vb2_q_ops = {
> > > > +	.queue_setup     = video_scd_queue_setup,
> > > > +	.wait_prepare    = vb2_ops_wait_prepare,
> > > > +	.wait_finish     = vb2_ops_wait_finish,
> > > > +	.buf_init        = video_scd_buf_init,
> > > > +	.buf_prepare     = video_scd_buf_prepare,
> > > > +	.buf_queue       = video_buf_queue,
> > > > +	.start_streaming = video_scd_start_streaming,
> > > > +	.stop_streaming  = video_scd_stop_streaming, };
> > > > +
> > > >  /* -----------------------------------------------------------------------------
> > > >   * V4L2 ioctls
> > > >   */
> > > > @@ -448,6 +528,37 @@ static const struct v4l2_ioctl_ops
> > > > stf_vid_ioctl_ops
> > > = {
> > > >  	.vidioc_streamoff               = vb2_ioctl_streamoff,
> > > >  };
> > > >
> > > > +static int video_scd_g_fmt(struct file *file, void *fh, struct
> > > > +v4l2_format *f) {
> > > > +	struct stfcamss_video *video = video_drvdata(file);
> > > > +	struct v4l2_meta_format *meta = &f->fmt.meta;
> > > > +
> > > > +	if (f->type != video->type)
> > > > +		return -EINVAL;
> > > > +
> > > > +	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
> > > > +	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static const struct v4l2_ioctl_ops stf_vid_scd_ioctl_ops = {
> > > > +	.vidioc_querycap                = video_querycap,
> > > > +	.vidioc_enum_fmt_meta_cap       = video_enum_fmt,
> > > > +	.vidioc_g_fmt_meta_cap          = video_scd_g_fmt,
> > > > +	.vidioc_s_fmt_meta_cap          = video_scd_g_fmt,
> > > > +	.vidioc_try_fmt_meta_cap        = video_scd_g_fmt,
> > > > +	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> > > > +	.vidioc_querybuf                = vb2_ioctl_querybuf,
> > > > +	.vidioc_qbuf                    = vb2_ioctl_qbuf,
> > > > +	.vidioc_expbuf                  = vb2_ioctl_expbuf,
> > > > +	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> > > > +	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
> > > > +	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> > > > +	.vidioc_streamon                = vb2_ioctl_streamon,
> > > > +	.vidioc_streamoff               = vb2_ioctl_streamoff,
> > > > +};
> > > > +
> > > >  /* -----------------------------------------------------------------------------
> > > >   * V4L2 file operations
> > > >   */
> > > > @@ -473,6 +584,9 @@ static int stf_link_validate(struct media_link *link)
> > > >  	struct stfcamss_video *video = video_get_drvdata(vdev);
> > > >  	int ret;
> > > >
> > > > +	if (video->type == V4L2_BUF_TYPE_META_CAPTURE)
> > > > +		return 0;
> > > > +
> > > >  	ret = stf_video_check_format(video);
> > > >
> > > >  	return ret;
> > > > @@ -506,7 +620,11 @@ int stf_video_register(struct stfcamss_video
> > > *video,
> > > >  	q = &video->vb2_q;
> > > >  	q->drv_priv = video;
> > > >  	q->mem_ops = &vb2_dma_contig_memops;
> > > > -	q->ops = &stf_video_vb2_q_ops;
> > > > +
> > > > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> > > > +		q->ops = &stf_video_vb2_q_ops;
> > > > +	else
> > > > +		q->ops = &stf_video_scd_vb2_q_ops;
> > > >  	q->type = video->type;
> > > >  	q->io_modes = VB2_DMABUF | VB2_MMAP;
> > > >  	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > > > @@ -529,16 +647,28 @@ int stf_video_register(struct stfcamss_video
> > > *video,
> > > >  		goto err_mutex_destroy;
> > > >  	}
> > > >
> > > > -	ret = stf_video_init_format(video);
> > > > -	if (ret < 0) {
> > > > -		dev_err(video->stfcamss->dev,
> > > > -			"Failed to init format: %d\n", ret);
> > > > -		goto err_media_cleanup;
> > > > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
> > > > +		ret = stf_video_init_format(video);
> > >
> > > I don't think this can fail
> > >
> >
> > This already exists, and I probably will not change it here.
> >
> 
> No problem! Maybe something for later when de-staging the driver.
> 
> Thanks
>   j
> > >

Re: 回复: 回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Jacopo Mondi]

Hi Changhuang

On Fri, Jul 12, 2024 at 08:36:21AM GMT, Changhuang Liang wrote:
> Hi, Jacopo
>
> [...]
> > > > > +
> > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > +		      enum stf_isp_type_scd type_scd) {
> > > > > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> > > > > +			    SEL_TYPE(type_scd));
> > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); }
> > > > > +
> > > > > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss, void
> > > > > +*vaddr) {
> > > > > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer
> > *)vaddr;
> > > > > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > > > > +	u32 i;
> > > > > +
> > > > > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > > > > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss, reg_addr);
> > > >
> > > > If you have a contigous memory space to read, could memcpy_fromio()
> > > > help instead of going through 64 reads ?
> > > >
> > >
> > > I will try this function.
> > >
> > > > > +}
> > > > > +
> > > > > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> > > > > +			      enum stf_isp_type_scd *type_scd) {
> > > > > +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer
> > > > > +*)vaddr;
> > > > > +
> > > > > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > > > > +	if (*type_scd == TYPE_AWB) {
> > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > > > > +		*type_scd = TYPE_OECF;
> > > > > +	} else {
> > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > > > > +		*type_scd = TYPE_AWB;
> > > >
> > > > Is this correct ? Why are you overwriting the value read from HW
> > > > that indicates AE/AF stats with AWB ones ?
> > >
> > > The AWB frame and AE/AF frames will alternate, so the current frame
> > > indicates the AE/AF, then set AWB type just for next AWB frame.
> > >
> >
> > Ah! Shouldn't it be userspace configuring which type of statistics it wants to
> > receive instead of the driver alternating the two ?
> >
>
> No, this is determined by hardware, cannot be configured by userspace.
>


So this
	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
			    SEL_TYPE(type_scd));

doesn't actually select which stats type you get from the HW

> > > >
> > > > > +	}
> > > > > +}
> > > > > +
> > > > >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> > > > >  	struct stfcamss *stfcamss = priv;
> > > > >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > +	struct stf_capture *cap_scd =
> > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > >  	struct stfcamss_buffer *change_buf;
> > > > > +	enum stf_isp_type_scd type_scd;
> > > > > +	u32 value;
> > > > >  	u32 status;
> > > > >
> > > > >  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0); @@
> > > > > -467,6
> > > > > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
> > > > >  					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
> > > > >  							 change_buf->addr[1]);
> > > > >  			}
> > > > > +
> > > > > +			value = stf_isp_reg_read(stfcamss,
> > > > ISP_REG_CSI_MODULE_CFG);
> > > > > +			if (value & CSI_SC_EN) {
> > > > > +				change_buf = stf_change_buffer(&cap_scd->buffers);
> > > > > +				if (change_buf) {
> > > > > +					stf_isp_fill_flag(stfcamss, change_buf->vaddr,
> > > > > +							  &type_scd);
> > > > > +					stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > > > > +							 change_buf->addr[1], type_scd);
> > > >
> > > > Sorry if I'm un-familiar with the HW but this seems to be the line-interrupt.
> > > > Are you swapping buffers every line or it's just that you have a
> > > > single line irq for the stats ?
> > > >
> > >
> > > Every frame triggers a line-interrupt, and we will swap buffers in it.
> > >
> >
> > ah, frames completion triggers a line-interrupt ?
> >
>
> Every frame will trigger line-interrupt and stf_isp_irq_handler.
> We use line-interrupt changing buffer, the stf_isp_irq_handler will indicate that
> image transfer to DDR is complete.
>
>
> > > > > +				}
> > > > > +			}
> > > > >  		}
> > > > >
> > > > >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@ -485,6
> > +542,7
> > > > @@
> > > > > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> > > > >  	struct stfcamss *stfcamss = priv;
> > > > >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > +	struct stf_capture *cap_scd =
> > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > >  	struct stfcamss_buffer *ready_buf;
> > > > >  	u32 status;
> > > > >
> > > > > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
> > > > >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > VB2_BUF_STATE_DONE);
> > > > >  		}
> > > > >
> > > > > +		if (status & ISPC_SC) {
> > > > > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > > > > +			if (ready_buf) {
> > > > > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > > > > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > VB2_BUF_STATE_DONE);
> > > > > +			}
> > > > > +		}
> > > > > +
> > > > >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> > > > >  				  (status & ~ISPC_INT_ALL_MASK) |
> > > > >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > > > > a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > index fcda0502e3b0..0af7b367e57a 100644
> > > > > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > @@ -10,6 +10,7 @@
> > > > >  #ifndef STF_ISP_H
> > > > >  #define STF_ISP_H
> > > > >
> > > > > +#include <linux/jh7110-isp.h>
> > > > >  #include <media/v4l2-subdev.h>
> > > > >
> > > > >  #include "stf-video.h"
> > > > > @@ -107,6 +108,12 @@
> > > > >  #define Y_COOR(y)				((y) << 16)
> > > > >  #define X_COOR(x)				((x) << 0)
> > > > >
> > > > > +#define ISP_REG_SCD_CFG_0			0x098
> > > > > +
> > > > > +#define ISP_REG_SC_CFG_1			0x0bc
> > > > > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > > > > +#define SEL_TYPE(n)				((n) << 30)
> > > > > +
> > > > >  #define ISP_REG_LCCF_CFG_2			0x0e0
> > > > >  #define ISP_REG_LCCF_CFG_3			0x0e4
> > > > >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > > > > @@ -305,6 +312,10 @@
> > > > >  #define DNRM_F(n)				((n) << 16)
> > > > >  #define CCM_M_DAT(n)				((n) << 0)
> > > > >
> > > > > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > > > > +
> > > > > +#define ISP_REG_YHIST_ACC_0			0xd00
> > > > > +
> > > > >  #define ISP_REG_GAMMA_VAL0			0xe00
> > > > >  #define ISP_REG_GAMMA_VAL1			0xe04
> > > > >  #define ISP_REG_GAMMA_VAL2			0xe08
> > > > > @@ -389,6 +400,15 @@
> > > > >  #define IMAGE_MAX_WIDTH				1920
> > > > >  #define IMAGE_MAX_HEIGH				1080
> > > > >
> > > > > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> > > >
> > > > Should this be in the uAPI header as it is useful to userspace as well ?
> > > >
> > > > you could:
> > > >
> > > > struct jh7110_isp_sc_buffer {
> > > > 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> > > > 	__u32 reserv0[33];
> > > > 	__u32 bright_sc[4096];
> > > > 	__u32 reserv1[96];
> > > > 	__u32 ae_hist_y[128];
> > > > 	__u32 reserv2[511];
> > > > 	__u16 flag;
> > > > };
> > > >
> > > > ofc if the size is made part of the uAPI you need a more proper name
> > > > such as JH7110_ISP_YHIST_SIZE
> > > >
> > >
> > > OK, I will try this.
> > >
> > > > > +
> > > > > +enum stf_isp_type_scd {
> > > > > +	TYPE_DEC = 0,
> > > > > +	TYPE_OBC,
> > > > > +	TYPE_OECF,
> > > > > +	TYPE_AWB,
> > > > > +};
> > > > > +
> > > > >  /* pad id for media framework */
> > > > >  enum stf_isp_pad_id {
> > > > >  	STF_ISP_PAD_SINK = 0,
> > > > > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev
> > > > > *isp_dev);
> > > > >
> > > > >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > > > >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > +		      enum stf_isp_type_scd type_scd);
> > > > >
> > > > >  #endif /* STF_ISP_H */
> > > > > diff --git a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > index 989b5e82bae9..2203605ec9c7 100644
> > > > > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > @@ -125,6 +125,14 @@ static int stf_video_init_format(struct
> > > > stfcamss_video *video)
> > > > >  	return 0;
> > > > >  }
> > > > >
> > > > > +static int stf_video_scd_init_format(struct stfcamss_video
> > > > > +*video)
> > > >
> > > > Make it void if it can't fail (see below)
> > > >
> > >
> > > OK.
> > >
> > > > > +{
> > > > > +	video->active_fmt.fmt.meta.dataformat =
> > > > video->formats[0].pixelformat;
> > > > > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > > > > +jh7110_isp_sc_buffer);
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > >  /* -----------------------------------------------------------------------------
> > > > >   * Video queue operations
> > > > >   */
> > > > > @@ -330,6 +338,78 @@ static const struct vb2_ops
> > > > > stf_video_vb2_q_ops =
> > > > {
> > > > >  	.stop_streaming  = video_stop_streaming,  };
> > > > >
> > > > > +static int video_scd_queue_setup(struct vb2_queue *q,
> > > > > +				 unsigned int *num_buffers,
> > > > > +				 unsigned int *num_planes,
> > > > > +				 unsigned int sizes[],
> > > > > +				 struct device *alloc_devs[]) {
> > > > > +	if (*num_planes)
> > > > > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL :
> > > > > +0;
> > > > > +
> > > > > +	*num_planes = 1;
> > > > > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > > > > +	dma_addr_t *paddr;
> > > > > +
> > > > > +	paddr = vb2_plane_cookie(vb, 0);
> > > > > +	buffer->addr[0] = *paddr;
> > > > > +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > > >
> > > > Interesting, I don't see many users of vb2_plane_cookie() in
> > > > mainline and I'm not sure what this gives you as you use it to program the
> > following registers:
> > > >
> > > > 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> > > >
> > >
> > > We set the value for ISP hardware, then ISP will transfer the statistics to the
> > buffer.
> > > when the stf_isp_irq_handler interrupt is triggered, indicates that
> > > the buffer fill is complete.
> > >
> >
> > So I take this as
> >
> > 	paddr = vb2_plane_cookie(vb, 0);
> > 	buffer->addr[0] = *paddr;
> > 	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> >
> >         stf_set_scd_addr(stfcamss, change_buf->addr[0],
> >                          change_buf->addr[1], type_scd);
> >
> > Makes the ISP transfer data directly to the memory areas in addr[0] and addr[1]
> > (which explains why struct jh7110_isp_sc_buffer is packed, as it has to match
> > the HW registers layout)
> >
> > If this is the case, why are you then manually copying the histograms and the
> > flags to vaddr ?
> >
>
> Yes, your are right.
> But actually there is a problem with our ISP RTL.
> We set this yhist_addr to ISP, but it actually not work.
> 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> or I will drop this line in next version.
>
> So, in this structure
> struct jh7110_isp_sc_buffer {
> 	__u32 y_histogram[64];
> 	__u32 reserv0[33];
> 	__u32 bright_sc[4096];
> 	__u32 reserv1[96];
> 	__u32 ae_hist_y[128];
> 	__u32 reserv2[511];
> 	__u16 flag;
> };
>
> Only
>
> 	__u32 reserv0[33];
> 	__u32 bright_sc[4096];
> 	__u32 reserv1[96];
> 	__u32 ae_hist_y[128];
> 	__u32 reserv2[511];
>
> transfer by ISP hardware.
>
> I need to fill
> 	__u32 y_histogram[64];
> 	__u16 flag;
>
> by vaddr.

I see.

Apart from the fact you can drop paddr and vb2_plane_cookie() and use
vaddr for all (if I'm not mistaken), could you please record the above
rationale for manually filling y_histogram and flag by hand in a
comment to avoid future readers being confused by this as I was ?

Thank you
   j

>
> Regards,
> Changhuang
>
> >                 ready_buf = stf_buf_done(&cap_scd->buffers);
> >                 if (ready_buf) {
> >                         stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> >                         vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > VB2_BUF_STATE_DONE);
> >                 }
> >
> >                 change_buf = stf_change_buffer(&cap_scd->buffers);
> >                 if (change_buf) {
> >                         stf_isp_fill_flag(stfcamss, change_buf->vaddr,
> >                                           &type_scd);
> >
> > If I read vb2_dc_alloc_coherent() right 'cookie' == 'vaddr'
> >
> > static int vb2_dc_alloc_coherent(struct vb2_dc_buf *buf) {
> > 	struct vb2_queue *q = buf->vb->vb2_queue;
> >
> > 	buf->cookie = dma_alloc_attrs(buf->dev,
> > 				      buf->size,
> > 				      &buf->dma_addr,
> > 				      GFP_KERNEL | q->gfp_flags,
> > 				      buf->attrs);
> > 	if (!buf->cookie)
> > 		return -ENOMEM;
> >
> > 	if (q->dma_attrs & DMA_ATTR_NO_KERNEL_MAPPING)
> > 		return 0;
> >
> > 	buf->vaddr = buf->cookie;
> > 	return 0;
> > }
> >
> > Could you verify what you get by printing out 'paddr' and 'vaddr' in
> > video_scd_buf_init() ?
> >
> >
> > > >
> > > > > +	buffer->vaddr = vb2_plane_vaddr(vb, 0);
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > > +static int video_scd_buf_prepare(struct vb2_buffer *vb) {
> > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > +
> > > > > +	if (sizeof(struct jh7110_isp_sc_buffer) > vb2_plane_size(vb, 0))
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	vb2_set_plane_payload(vb, 0, sizeof(struct
> > > > > +jh7110_isp_sc_buffer));
> > > > > +
> > > > > +	vbuf->field = V4L2_FIELD_NONE;
> > > >
> > > > is this necessary ?
> > > >
> > >
> > > Will drop it.
> > >
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > > +static int video_scd_start_streaming(struct vb2_queue *q,
> > > > > +unsigned int count) {
> > > > > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > > > > +
> > > > > +	video->ops->start_streaming(video);
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > > +static void video_scd_stop_streaming(struct vb2_queue *q) {
> > > > > +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> > > > > +
> > > > > +	video->ops->stop_streaming(video);
> > > > > +
> > > > > +	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR); }
> > > > > +
> > > > > +static const struct vb2_ops stf_video_scd_vb2_q_ops = {
> > > > > +	.queue_setup     = video_scd_queue_setup,
> > > > > +	.wait_prepare    = vb2_ops_wait_prepare,
> > > > > +	.wait_finish     = vb2_ops_wait_finish,
> > > > > +	.buf_init        = video_scd_buf_init,
> > > > > +	.buf_prepare     = video_scd_buf_prepare,
> > > > > +	.buf_queue       = video_buf_queue,
> > > > > +	.start_streaming = video_scd_start_streaming,
> > > > > +	.stop_streaming  = video_scd_stop_streaming, };
> > > > > +
> > > > >  /* -----------------------------------------------------------------------------
> > > > >   * V4L2 ioctls
> > > > >   */
> > > > > @@ -448,6 +528,37 @@ static const struct v4l2_ioctl_ops
> > > > > stf_vid_ioctl_ops
> > > > = {
> > > > >  	.vidioc_streamoff               = vb2_ioctl_streamoff,
> > > > >  };
> > > > >
> > > > > +static int video_scd_g_fmt(struct file *file, void *fh, struct
> > > > > +v4l2_format *f) {
> > > > > +	struct stfcamss_video *video = video_drvdata(file);
> > > > > +	struct v4l2_meta_format *meta = &f->fmt.meta;
> > > > > +
> > > > > +	if (f->type != video->type)
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
> > > > > +	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
> > > > > +
> > > > > +	return 0;
> > > > > +}
> > > > > +
> > > > > +static const struct v4l2_ioctl_ops stf_vid_scd_ioctl_ops = {
> > > > > +	.vidioc_querycap                = video_querycap,
> > > > > +	.vidioc_enum_fmt_meta_cap       = video_enum_fmt,
> > > > > +	.vidioc_g_fmt_meta_cap          = video_scd_g_fmt,
> > > > > +	.vidioc_s_fmt_meta_cap          = video_scd_g_fmt,
> > > > > +	.vidioc_try_fmt_meta_cap        = video_scd_g_fmt,
> > > > > +	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> > > > > +	.vidioc_querybuf                = vb2_ioctl_querybuf,
> > > > > +	.vidioc_qbuf                    = vb2_ioctl_qbuf,
> > > > > +	.vidioc_expbuf                  = vb2_ioctl_expbuf,
> > > > > +	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> > > > > +	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
> > > > > +	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> > > > > +	.vidioc_streamon                = vb2_ioctl_streamon,
> > > > > +	.vidioc_streamoff               = vb2_ioctl_streamoff,
> > > > > +};
> > > > > +
> > > > >  /* -----------------------------------------------------------------------------
> > > > >   * V4L2 file operations
> > > > >   */
> > > > > @@ -473,6 +584,9 @@ static int stf_link_validate(struct media_link *link)
> > > > >  	struct stfcamss_video *video = video_get_drvdata(vdev);
> > > > >  	int ret;
> > > > >
> > > > > +	if (video->type == V4L2_BUF_TYPE_META_CAPTURE)
> > > > > +		return 0;
> > > > > +
> > > > >  	ret = stf_video_check_format(video);
> > > > >
> > > > >  	return ret;
> > > > > @@ -506,7 +620,11 @@ int stf_video_register(struct stfcamss_video
> > > > *video,
> > > > >  	q = &video->vb2_q;
> > > > >  	q->drv_priv = video;
> > > > >  	q->mem_ops = &vb2_dma_contig_memops;
> > > > > -	q->ops = &stf_video_vb2_q_ops;
> > > > > +
> > > > > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> > > > > +		q->ops = &stf_video_vb2_q_ops;
> > > > > +	else
> > > > > +		q->ops = &stf_video_scd_vb2_q_ops;
> > > > >  	q->type = video->type;
> > > > >  	q->io_modes = VB2_DMABUF | VB2_MMAP;
> > > > >  	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > > > > @@ -529,16 +647,28 @@ int stf_video_register(struct stfcamss_video
> > > > *video,
> > > > >  		goto err_mutex_destroy;
> > > > >  	}
> > > > >
> > > > > -	ret = stf_video_init_format(video);
> > > > > -	if (ret < 0) {
> > > > > -		dev_err(video->stfcamss->dev,
> > > > > -			"Failed to init format: %d\n", ret);
> > > > > -		goto err_media_cleanup;
> > > > > +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
> > > > > +		ret = stf_video_init_format(video);
> > > >
> > > > I don't think this can fail
> > > >
> > >
> > > This already exists, and I probably will not change it here.
> > >
> >
> > No problem! Maybe something for later when de-staging the driver.
> >
> > Thanks
> >   j
> > > >

回复: 回复: 回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Changhuang Liang]

Hi, Jacopo

> 
> Hi Changhuang
> 
> On Fri, Jul 12, 2024 at 08:36:21AM GMT, Changhuang Liang wrote:
> > Hi, Jacopo
> >
> > [...]
> > > > > > +
> > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > +		      enum stf_isp_type_scd type_scd) {
> > > > > > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1,
> ISP_SC_SEL_MASK,
> > > > > > +			    SEL_TYPE(type_scd));
> > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4,
> > > > > > +yhist_addr); }
> > > > > > +
> > > > > > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss,
> > > > > > +void
> > > > > > +*vaddr) {
> > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > +jh7110_isp_sc_buffer
> > > *)vaddr;
> > > > > > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > > > > > +	u32 i;
> > > > > > +
> > > > > > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > > > > > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss,
> reg_addr);
> > > > >
> > > > > If you have a contigous memory space to read, could
> > > > > memcpy_fromio() help instead of going through 64 reads ?
> > > > >
> > > >
> > > > I will try this function.
> > > >
> > > > > > +}
> > > > > > +
> > > > > > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> > > > > > +			      enum stf_isp_type_scd *type_scd) {
> > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > +jh7110_isp_sc_buffer *)vaddr;
> > > > > > +
> > > > > > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > > > > > +	if (*type_scd == TYPE_AWB) {
> > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > > > > > +		*type_scd = TYPE_OECF;
> > > > > > +	} else {
> > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > > > > > +		*type_scd = TYPE_AWB;
> > > > >
> > > > > Is this correct ? Why are you overwriting the value read from HW
> > > > > that indicates AE/AF stats with AWB ones ?
> > > >
> > > > The AWB frame and AE/AF frames will alternate, so the current
> > > > frame indicates the AE/AF, then set AWB type just for next AWB frame.
> > > >
> > >
> > > Ah! Shouldn't it be userspace configuring which type of statistics
> > > it wants to receive instead of the driver alternating the two ?
> > >
> >
> > No, this is determined by hardware, cannot be configured by userspace.
> >
> 
> 
> So this
> 	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> 			    SEL_TYPE(type_scd));
> 
> doesn't actually select which stats type you get from the HW
> 

You can understand it that way. But it still needs to be written to work with the hardware.

> > > > >
> > > > > > +	}
> > > > > > +}
> > > > > > +
> > > > > >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > >  	struct stf_capture *cap =
> > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > +	struct stf_capture *cap_scd =
> > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > >  	struct stfcamss_buffer *change_buf;
> > > > > > +	enum stf_isp_type_scd type_scd;
> > > > > > +	u32 value;
> > > > > >  	u32 status;
> > > > > >
> > > > > >  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
> @@
> > > > > > -467,6
> > > > > > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void
> > > > > > +*priv)
> > > > > >  					stf_set_yuv_addr(stfcamss,
> change_buf->addr[0],
> > > > > >  							 change_buf->addr[1]);
> > > > > >  			}
> > > > > > +
> > > > > > +			value = stf_isp_reg_read(stfcamss,
> > > > > ISP_REG_CSI_MODULE_CFG);
> > > > > > +			if (value & CSI_SC_EN) {
> > > > > > +				change_buf =
> stf_change_buffer(&cap_scd->buffers);
> > > > > > +				if (change_buf) {
> > > > > > +					stf_isp_fill_flag(stfcamss,
> change_buf->vaddr,
> > > > > > +							  &type_scd);
> > > > > > +					stf_set_scd_addr(stfcamss,
> change_buf->addr[0],
> > > > > > +							 change_buf->addr[1], type_scd);
> > > > >
> > > > > Sorry if I'm un-familiar with the HW but this seems to be the
> line-interrupt.
> > > > > Are you swapping buffers every line or it's just that you have a
> > > > > single line irq for the stats ?
> > > > >
> > > >
> > > > Every frame triggers a line-interrupt, and we will swap buffers in it.
> > > >
> > >
> > > ah, frames completion triggers a line-interrupt ?
> > >
> >
> > Every frame will trigger line-interrupt and stf_isp_irq_handler.
> > We use line-interrupt changing buffer, the stf_isp_irq_handler will
> > indicate that image transfer to DDR is complete.
> >
> >
> > > > > > +				}
> > > > > > +			}
> > > > > >  		}
> > > > > >
> > > > > >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@
> -485,6
> > > +542,7
> > > > > @@
> > > > > > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > >  	struct stf_capture *cap =
> > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > +	struct stf_capture *cap_scd =
> > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > >  	struct stfcamss_buffer *ready_buf;
> > > > > >  	u32 status;
> > > > > >
> > > > > > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void
> *priv)
> > > > > >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > VB2_BUF_STATE_DONE);
> > > > > >  		}
> > > > > >
> > > > > > +		if (status & ISPC_SC) {
> > > > > > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > > > > > +			if (ready_buf) {
> > > > > > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > > > > > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > VB2_BUF_STATE_DONE);
> > > > > > +			}
> > > > > > +		}
> > > > > > +
> > > > > >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> > > > > >  				  (status & ~ISPC_INT_ALL_MASK) |
> > > > > >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > > > > > a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > index fcda0502e3b0..0af7b367e57a 100644
> > > > > > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > @@ -10,6 +10,7 @@
> > > > > >  #ifndef STF_ISP_H
> > > > > >  #define STF_ISP_H
> > > > > >
> > > > > > +#include <linux/jh7110-isp.h>
> > > > > >  #include <media/v4l2-subdev.h>
> > > > > >
> > > > > >  #include "stf-video.h"
> > > > > > @@ -107,6 +108,12 @@
> > > > > >  #define Y_COOR(y)				((y) << 16)
> > > > > >  #define X_COOR(x)				((x) << 0)
> > > > > >
> > > > > > +#define ISP_REG_SCD_CFG_0			0x098
> > > > > > +
> > > > > > +#define ISP_REG_SC_CFG_1			0x0bc
> > > > > > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > > > > > +#define SEL_TYPE(n)				((n) << 30)
> > > > > > +
> > > > > >  #define ISP_REG_LCCF_CFG_2			0x0e0
> > > > > >  #define ISP_REG_LCCF_CFG_3			0x0e4
> > > > > >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > > > > > @@ -305,6 +312,10 @@
> > > > > >  #define DNRM_F(n)				((n) << 16)
> > > > > >  #define CCM_M_DAT(n)				((n) << 0)
> > > > > >
> > > > > > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > > > > > +
> > > > > > +#define ISP_REG_YHIST_ACC_0			0xd00
> > > > > > +
> > > > > >  #define ISP_REG_GAMMA_VAL0			0xe00
> > > > > >  #define ISP_REG_GAMMA_VAL1			0xe04
> > > > > >  #define ISP_REG_GAMMA_VAL2			0xe08
> > > > > > @@ -389,6 +400,15 @@
> > > > > >  #define IMAGE_MAX_WIDTH				1920
> > > > > >  #define IMAGE_MAX_HEIGH				1080
> > > > > >
> > > > > > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> > > > >
> > > > > Should this be in the uAPI header as it is useful to userspace as well ?
> > > > >
> > > > > you could:
> > > > >
> > > > > struct jh7110_isp_sc_buffer {
> > > > > 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> > > > > 	__u32 reserv0[33];
> > > > > 	__u32 bright_sc[4096];
> > > > > 	__u32 reserv1[96];
> > > > > 	__u32 ae_hist_y[128];
> > > > > 	__u32 reserv2[511];
> > > > > 	__u16 flag;
> > > > > };
> > > > >
> > > > > ofc if the size is made part of the uAPI you need a more proper
> > > > > name such as JH7110_ISP_YHIST_SIZE
> > > > >
> > > >
> > > > OK, I will try this.
> > > >
> > > > > > +
> > > > > > +enum stf_isp_type_scd {
> > > > > > +	TYPE_DEC = 0,
> > > > > > +	TYPE_OBC,
> > > > > > +	TYPE_OECF,
> > > > > > +	TYPE_AWB,
> > > > > > +};
> > > > > > +
> > > > > >  /* pad id for media framework */  enum stf_isp_pad_id {
> > > > > >  	STF_ISP_PAD_SINK = 0,
> > > > > > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev
> > > > > > *isp_dev);
> > > > > >
> > > > > >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > > > > >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > +		      enum stf_isp_type_scd type_scd);
> > > > > >
> > > > > >  #endif /* STF_ISP_H */
> > > > > > diff --git a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > index 989b5e82bae9..2203605ec9c7 100644
> > > > > > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > @@ -125,6 +125,14 @@ static int stf_video_init_format(struct
> > > > > stfcamss_video *video)
> > > > > >  	return 0;
> > > > > >  }
> > > > > >
> > > > > > +static int stf_video_scd_init_format(struct stfcamss_video
> > > > > > +*video)
> > > > >
> > > > > Make it void if it can't fail (see below)
> > > > >
> > > >
> > > > OK.
> > > >
> > > > > > +{
> > > > > > +	video->active_fmt.fmt.meta.dataformat =
> > > > > video->formats[0].pixelformat;
> > > > > > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > > > > > +jh7110_isp_sc_buffer);
> > > > > > +
> > > > > > +	return 0;
> > > > > > +}
> > > > > > +
> > > > > >  /* -----------------------------------------------------------------------------
> > > > > >   * Video queue operations
> > > > > >   */
> > > > > > @@ -330,6 +338,78 @@ static const struct vb2_ops
> > > > > > stf_video_vb2_q_ops =
> > > > > {
> > > > > >  	.stop_streaming  = video_stop_streaming,  };
> > > > > >
> > > > > > +static int video_scd_queue_setup(struct vb2_queue *q,
> > > > > > +				 unsigned int *num_buffers,
> > > > > > +				 unsigned int *num_planes,
> > > > > > +				 unsigned int sizes[],
> > > > > > +				 struct device *alloc_devs[]) {
> > > > > > +	if (*num_planes)
> > > > > > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ?
> -EINVAL :
> > > > > > +0;
> > > > > > +
> > > > > > +	*num_planes = 1;
> > > > > > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > > > > > +
> > > > > > +	return 0;
> > > > > > +}
> > > > > > +
> > > > > > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > > > > > +	dma_addr_t *paddr;
> > > > > > +
> > > > > > +	paddr = vb2_plane_cookie(vb, 0);
> > > > > > +	buffer->addr[0] = *paddr;
> > > > > > +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > > > >
> > > > > Interesting, I don't see many users of vb2_plane_cookie() in
> > > > > mainline and I'm not sure what this gives you as you use it to
> > > > > program the
> > > following registers:
> > > > >
> > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> > > > >
> > > >
> > > > We set the value for ISP hardware, then ISP will transfer the
> > > > statistics to the
> > > buffer.
> > > > when the stf_isp_irq_handler interrupt is triggered, indicates
> > > > that the buffer fill is complete.
> > > >
> > >
> > > So I take this as
> > >
> > > 	paddr = vb2_plane_cookie(vb, 0);
> > > 	buffer->addr[0] = *paddr;
> > > 	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > >
> > >         stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > >                          change_buf->addr[1], type_scd);
> > >
> > > Makes the ISP transfer data directly to the memory areas in addr[0]
> > > and addr[1] (which explains why struct jh7110_isp_sc_buffer is
> > > packed, as it has to match the HW registers layout)
> > >
> > > If this is the case, why are you then manually copying the
> > > histograms and the flags to vaddr ?
> > >
> >
> > Yes, your are right.
> > But actually there is a problem with our ISP RTL.
> > We set this yhist_addr to ISP, but it actually not work.
> > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); or I
> > will drop this line in next version.
> >
> > So, in this structure
> > struct jh7110_isp_sc_buffer {
> > 	__u32 y_histogram[64];
> > 	__u32 reserv0[33];
> > 	__u32 bright_sc[4096];
> > 	__u32 reserv1[96];
> > 	__u32 ae_hist_y[128];
> > 	__u32 reserv2[511];
> > 	__u16 flag;
> > };
> >
> > Only
> >
> > 	__u32 reserv0[33];
> > 	__u32 bright_sc[4096];
> > 	__u32 reserv1[96];
> > 	__u32 ae_hist_y[128];
> > 	__u32 reserv2[511];
> >
> > transfer by ISP hardware.
> >
> > I need to fill
> > 	__u32 y_histogram[64];
> > 	__u16 flag;
> >
> > by vaddr.
> 
> I see.
> 
> Apart from the fact you can drop paddr and vb2_plane_cookie() and use vaddr
> for all (if I'm not mistaken), could you please record the above rationale for
> manually filling y_histogram and flag by hand in a comment to avoid future
> readers being confused by this as I was ?
> 

Still need to keep the paddr and vb2_plane_cookie() for 
	__u32 reserv0[33];
	__u32 bright_sc[4096];
	__u32 reserv1[96];
	__u32 ae_hist_y[128];
 	__u32 reserv2[511];

Because this part is filled by the hardware.

I will add more information for struct jh7110_isp_sc_buffer

Regards,
Changhuang

回复: [PATCH v5 12/14] staging: media: starfive: Add ISP parameters hardware configure

[Changhuang Liang]

Hi Jacopo

Thanks for your comments.

> Hi Changhuang
> 
> On Tue, Jul 09, 2024 at 01:38:22AM GMT, Changhuang Liang wrote:
> > Add ISP parameters hardware configure.
> >
> > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > ---
> >  .../staging/media/starfive/camss/stf-camss.h  |   8 +
> >  .../media/starfive/camss/stf-isp-hw-ops.c     | 573
> ++++++++++++++++++
> >  .../staging/media/starfive/camss/stf-isp.h    | 135 +++++
> >  3 files changed, 716 insertions(+)
> >
> > diff --git a/drivers/staging/media/starfive/camss/stf-camss.h
> > b/drivers/staging/media/starfive/camss/stf-camss.h
> > index 3f84f1a1e997..328318d61c6b 100644
> > --- a/drivers/staging/media/starfive/camss/stf-camss.h
> > +++ b/drivers/staging/media/starfive/camss/stf-camss.h
> > @@ -106,6 +106,14 @@ static inline void stf_isp_reg_set(struct stfcamss
> *stfcamss, u32 reg, u32 mask)
> >  		  stfcamss->isp_base + reg);
> >  }
> >
> > +static inline void stf_isp_reg_fill_zero(struct stfcamss *stfcamss,
> > +u32 reg, u32 size) {
> > +	u32 i;
> > +
> > +	for (i = 0; i < size; i++, reg += 4)
> > +		iowrite32(0, stfcamss->isp_base + reg); }
> > +
> 
> Why in the header file ? Also could memset_io() replace this ?
> 

OK, I will try this.

> >  static inline u32 stf_syscon_reg_read(struct stfcamss *stfcamss, u32
> > reg)  {
> >  	return ioread32(stfcamss->syscon_base + reg); diff --git
> > a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > index 0bc5e36f952e..e54368910906 100644
> > --- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > +++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> > @@ -10,6 +10,25 @@
> >
> >  #include "stf-camss.h"
> >
> > +static const struct stf_isp_module_info mod_info[] = {
> > +	{ ISP_REG_CSI_MODULE_CFG, 2 },
> > +	{ ISP_REG_CSI_MODULE_CFG, 4 },
> > +	{ ISP_REG_CSI_MODULE_CFG, 6 },
> > +	{ ISP_REG_CSI_MODULE_CFG, 7 },
> > +	{ ISP_REG_CSI_MODULE_CFG, 17 },
> > +	{ ISP_REG_ISP_CTRL_1, 1 },
> > +	{ ISP_REG_ISP_CTRL_1, 2 },
> > +	{ ISP_REG_ISP_CTRL_1, 3 },
> > +	{ ISP_REG_ISP_CTRL_1, 4 },
> > +	{ ISP_REG_ISP_CTRL_1, 5 },
> > +	{ ISP_REG_ISP_CTRL_1, 7 },
> > +	{ ISP_REG_ISP_CTRL_1, 8 },
> > +	{ ISP_REG_ISP_CTRL_1, 17 },
> > +	{ ISP_REG_ISP_CTRL_1, 19 },
> > +	{ ISP_REG_ISP_CTRL_1, 21 },
> > +	{ ISP_REG_ISP_CTRL_1, 22 },
> > +};
> > +
> >  static void stf_isp_config_obc(struct stfcamss *stfcamss)  {
> >  	u32 reg_val, reg_add;
> > @@ -517,6 +536,59 @@ static void stf_isp_fill_flag(struct stfcamss
> *stfcamss, void *vaddr,
> >  	}
> >  }
> >
> > +static void stf_isp_set_params(struct stfcamss *stfcamss, void
> > +*vaddr) {
> 
> Shouldn't all these operations be in the -params.c file ?
> 
> > +	struct jh7110_isp_params_buffer *params = (struct
> > +jh7110_isp_params_buffer *)vaddr;
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_WB_SETTING)
> > +		isp_set_ctrl_wb(stfcamss, &params->wb_setting);
> 
> Also this function is exported in stf-isp.h but are only called from within this
> file ? Same for all the other ones I guess ?
> 

Yes, but we have a file stf-isp-hw-ops.c which store the ISP hardware operations.
Should these operations move into -params.c?

> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_CAR_SETTING)
> > +		isp_set_ctrl_car(stfcamss, &params->car_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_CCM_SETTING)
> > +		isp_set_ctrl_ccm(stfcamss, &params->ccm_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_CFA_SETTING)
> > +		isp_set_ctrl_cfa(stfcamss, &params->cfa_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_CTC_SETTING)
> > +		isp_set_ctrl_ctc(stfcamss, &params->ctc_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_DBC_SETTING)
> > +		isp_set_ctrl_dbc(stfcamss, &params->dbc_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_DNYUV_SETTING)
> > +		isp_set_ctrl_dnyuv(stfcamss, &params->dnyuv_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_GMARGB_SETTING)
> > +		isp_set_ctrl_gmargb(stfcamss, &params->gmargb_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_LCCF_SETTING)
> > +		isp_set_ctrl_lccf(stfcamss, &params->lccf_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_OBC_SETTING)
> > +		isp_set_ctrl_obc(stfcamss, &params->obc_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_OECF_SETTING)
> > +		isp_set_ctrl_oecf(stfcamss, &params->oecf_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_R2Y_SETTING)
> > +		isp_set_ctrl_r2y(stfcamss, &params->r2y_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_SAT_SETTING)
> > +		isp_set_ctrl_sat(stfcamss, &params->sat_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_SHARP_SETTING)
> > +		isp_set_ctrl_sharp(stfcamss, &params->sharp_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_YCRV_SETTING)
> > +		isp_set_ctrl_ycrv(stfcamss, &params->ycrv_setting);
> > +
> > +	if (params->enable_setting & JH7110_ISP_MODULE_SC_SETTING)
> > +		isp_set_ctrl_sc(stfcamss, &params->sc_setting); }
> > +
> >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> >  	struct stfcamss *stfcamss = priv;
> > @@ -566,11 +638,20 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
> >  	struct stfcamss *stfcamss = priv;
> >  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> >  	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
> > +	struct stf_output *output = &stfcamss->output;
> >  	struct stfcamss_buffer *ready_buf;
> >  	u32 status;
> >
> >  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
> >  	if (status & ISPC_ISP) {
> > +		ready_buf = stf_buf_get_ready(&output->buffers);
> > +		if (ready_buf) {
> > +			stf_isp_set_params(stfcamss, ready_buf->vaddr);
> > +			ready_buf->vb.vb2_buf.timestamp = ktime_get_ns();
> > +			ready_buf->vb.sequence = output->buffers.sequence++;
> > +			vb2_buffer_done(&ready_buf->vb.vb2_buf,
> VB2_BUF_STATE_DONE);
> > +		}
> > +
> >  		if (status & ISPC_ENUO) {
> >  			ready_buf = stf_buf_done(&cap->buffers);
> >  			if (ready_buf)
> > @@ -591,4 +672,496 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
> >  	}
> >
> >  	return IRQ_HANDLED;
> > +};
> > +
> > +int isp_set_ctrl_wb(struct stfcamss *stfcamss, const void *value)
> 
> This function is generic, there's no need to pass a const void * here and cast it
> below, you can use const struct jh7110_isp_wb_setting * as the second
> argument type.
> 

OK

Regards,
Changhuang

Re: 回复: 回复: 回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Jacopo Mondi]

Hi Changhuang

On Fri, Jul 12, 2024 at 01:00:03PM GMT, Changhuang Liang wrote:
> Hi, Jacopo
>
> >
> > Hi Changhuang
> >
> > On Fri, Jul 12, 2024 at 08:36:21AM GMT, Changhuang Liang wrote:
> > > Hi, Jacopo
> > >
> > > [...]
> > > > > > > +
> > > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > > +		      enum stf_isp_type_scd type_scd) {
> > > > > > > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1,
> > ISP_SC_SEL_MASK,
> > > > > > > +			    SEL_TYPE(type_scd));
> > > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4,
> > > > > > > +yhist_addr); }
> > > > > > > +
> > > > > > > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss,
> > > > > > > +void
> > > > > > > +*vaddr) {
> > > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > > +jh7110_isp_sc_buffer
> > > > *)vaddr;
> > > > > > > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > > > > > > +	u32 i;
> > > > > > > +
> > > > > > > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > > > > > > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss,
> > reg_addr);
> > > > > >
> > > > > > If you have a contigous memory space to read, could
> > > > > > memcpy_fromio() help instead of going through 64 reads ?
> > > > > >
> > > > >
> > > > > I will try this function.
> > > > >
> > > > > > > +}
> > > > > > > +
> > > > > > > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> > > > > > > +			      enum stf_isp_type_scd *type_scd) {
> > > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > > +jh7110_isp_sc_buffer *)vaddr;
> > > > > > > +
> > > > > > > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > > > > > > +	if (*type_scd == TYPE_AWB) {
> > > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > > > > > > +		*type_scd = TYPE_OECF;
> > > > > > > +	} else {
> > > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > > > > > > +		*type_scd = TYPE_AWB;
> > > > > >
> > > > > > Is this correct ? Why are you overwriting the value read from HW
> > > > > > that indicates AE/AF stats with AWB ones ?
> > > > >
> > > > > The AWB frame and AE/AF frames will alternate, so the current
> > > > > frame indicates the AE/AF, then set AWB type just for next AWB frame.
> > > > >
> > > >
> > > > Ah! Shouldn't it be userspace configuring which type of statistics
> > > > it wants to receive instead of the driver alternating the two ?
> > > >
> > >
> > > No, this is determined by hardware, cannot be configured by userspace.
> > >
> >
> >
> > So this
> > 	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> > 			    SEL_TYPE(type_scd));
> >
> > doesn't actually select which stats type you get from the HW
> >
>
> You can understand it that way. But it still needs to be written to work with the hardware.
>

ack, maybe a comment here as well would help

> > > > > >
> > > > > > > +	}
> > > > > > > +}
> > > > > > > +
> > > > > > >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> > > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > > >  	struct stf_capture *cap =
> > > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > > +	struct stf_capture *cap_scd =
> > > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > > >  	struct stfcamss_buffer *change_buf;
> > > > > > > +	enum stf_isp_type_scd type_scd;
> > > > > > > +	u32 value;
> > > > > > >  	u32 status;
> > > > > > >
> > > > > > >  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
> > @@
> > > > > > > -467,6
> > > > > > > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void
> > > > > > > +*priv)
> > > > > > >  					stf_set_yuv_addr(stfcamss,
> > change_buf->addr[0],
> > > > > > >  							 change_buf->addr[1]);
> > > > > > >  			}
> > > > > > > +
> > > > > > > +			value = stf_isp_reg_read(stfcamss,
> > > > > > ISP_REG_CSI_MODULE_CFG);
> > > > > > > +			if (value & CSI_SC_EN) {
> > > > > > > +				change_buf =
> > stf_change_buffer(&cap_scd->buffers);
> > > > > > > +				if (change_buf) {
> > > > > > > +					stf_isp_fill_flag(stfcamss,
> > change_buf->vaddr,
> > > > > > > +							  &type_scd);
> > > > > > > +					stf_set_scd_addr(stfcamss,
> > change_buf->addr[0],
> > > > > > > +							 change_buf->addr[1], type_scd);
> > > > > >
> > > > > > Sorry if I'm un-familiar with the HW but this seems to be the
> > line-interrupt.
> > > > > > Are you swapping buffers every line or it's just that you have a
> > > > > > single line irq for the stats ?
> > > > > >
> > > > >
> > > > > Every frame triggers a line-interrupt, and we will swap buffers in it.
> > > > >
> > > >
> > > > ah, frames completion triggers a line-interrupt ?
> > > >
> > >
> > > Every frame will trigger line-interrupt and stf_isp_irq_handler.
> > > We use line-interrupt changing buffer, the stf_isp_irq_handler will
> > > indicate that image transfer to DDR is complete.
> > >
> > >
> > > > > > > +				}
> > > > > > > +			}
> > > > > > >  		}
> > > > > > >
> > > > > > >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@
> > -485,6
> > > > +542,7
> > > > > > @@
> > > > > > > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> > > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > > >  	struct stf_capture *cap =
> > > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > > +	struct stf_capture *cap_scd =
> > > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > > >  	struct stfcamss_buffer *ready_buf;
> > > > > > >  	u32 status;
> > > > > > >
> > > > > > > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void
> > *priv)
> > > > > > >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > > VB2_BUF_STATE_DONE);
> > > > > > >  		}
> > > > > > >
> > > > > > > +		if (status & ISPC_SC) {
> > > > > > > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > > > > > > +			if (ready_buf) {
> > > > > > > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > > > > > > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > > VB2_BUF_STATE_DONE);
> > > > > > > +			}
> > > > > > > +		}
> > > > > > > +
> > > > > > >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> > > > > > >  				  (status & ~ISPC_INT_ALL_MASK) |
> > > > > > >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > > > > > > a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > index fcda0502e3b0..0af7b367e57a 100644
> > > > > > > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > @@ -10,6 +10,7 @@
> > > > > > >  #ifndef STF_ISP_H
> > > > > > >  #define STF_ISP_H
> > > > > > >
> > > > > > > +#include <linux/jh7110-isp.h>
> > > > > > >  #include <media/v4l2-subdev.h>
> > > > > > >
> > > > > > >  #include "stf-video.h"
> > > > > > > @@ -107,6 +108,12 @@
> > > > > > >  #define Y_COOR(y)				((y) << 16)
> > > > > > >  #define X_COOR(x)				((x) << 0)
> > > > > > >
> > > > > > > +#define ISP_REG_SCD_CFG_0			0x098
> > > > > > > +
> > > > > > > +#define ISP_REG_SC_CFG_1			0x0bc
> > > > > > > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > > > > > > +#define SEL_TYPE(n)				((n) << 30)
> > > > > > > +
> > > > > > >  #define ISP_REG_LCCF_CFG_2			0x0e0
> > > > > > >  #define ISP_REG_LCCF_CFG_3			0x0e4
> > > > > > >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > > > > > > @@ -305,6 +312,10 @@
> > > > > > >  #define DNRM_F(n)				((n) << 16)
> > > > > > >  #define CCM_M_DAT(n)				((n) << 0)
> > > > > > >
> > > > > > > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > > > > > > +
> > > > > > > +#define ISP_REG_YHIST_ACC_0			0xd00
> > > > > > > +
> > > > > > >  #define ISP_REG_GAMMA_VAL0			0xe00
> > > > > > >  #define ISP_REG_GAMMA_VAL1			0xe04
> > > > > > >  #define ISP_REG_GAMMA_VAL2			0xe08
> > > > > > > @@ -389,6 +400,15 @@
> > > > > > >  #define IMAGE_MAX_WIDTH				1920
> > > > > > >  #define IMAGE_MAX_HEIGH				1080
> > > > > > >
> > > > > > > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> > > > > >
> > > > > > Should this be in the uAPI header as it is useful to userspace as well ?
> > > > > >
> > > > > > you could:
> > > > > >
> > > > > > struct jh7110_isp_sc_buffer {
> > > > > > 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> > > > > > 	__u32 reserv0[33];
> > > > > > 	__u32 bright_sc[4096];
> > > > > > 	__u32 reserv1[96];
> > > > > > 	__u32 ae_hist_y[128];
> > > > > > 	__u32 reserv2[511];
> > > > > > 	__u16 flag;
> > > > > > };
> > > > > >
> > > > > > ofc if the size is made part of the uAPI you need a more proper
> > > > > > name such as JH7110_ISP_YHIST_SIZE
> > > > > >
> > > > >
> > > > > OK, I will try this.
> > > > >
> > > > > > > +
> > > > > > > +enum stf_isp_type_scd {
> > > > > > > +	TYPE_DEC = 0,
> > > > > > > +	TYPE_OBC,
> > > > > > > +	TYPE_OECF,
> > > > > > > +	TYPE_AWB,
> > > > > > > +};
> > > > > > > +
> > > > > > >  /* pad id for media framework */  enum stf_isp_pad_id {
> > > > > > >  	STF_ISP_PAD_SINK = 0,
> > > > > > > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev
> > > > > > > *isp_dev);
> > > > > > >
> > > > > > >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > > > > > >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > > +		      enum stf_isp_type_scd type_scd);
> > > > > > >
> > > > > > >  #endif /* STF_ISP_H */
> > > > > > > diff --git a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > index 989b5e82bae9..2203605ec9c7 100644
> > > > > > > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > @@ -125,6 +125,14 @@ static int stf_video_init_format(struct
> > > > > > stfcamss_video *video)
> > > > > > >  	return 0;
> > > > > > >  }
> > > > > > >
> > > > > > > +static int stf_video_scd_init_format(struct stfcamss_video
> > > > > > > +*video)
> > > > > >
> > > > > > Make it void if it can't fail (see below)
> > > > > >
> > > > >
> > > > > OK.
> > > > >
> > > > > > > +{
> > > > > > > +	video->active_fmt.fmt.meta.dataformat =
> > > > > > video->formats[0].pixelformat;
> > > > > > > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > > > > > > +jh7110_isp_sc_buffer);
> > > > > > > +
> > > > > > > +	return 0;
> > > > > > > +}
> > > > > > > +
> > > > > > >  /* -----------------------------------------------------------------------------
> > > > > > >   * Video queue operations
> > > > > > >   */
> > > > > > > @@ -330,6 +338,78 @@ static const struct vb2_ops
> > > > > > > stf_video_vb2_q_ops =
> > > > > > {
> > > > > > >  	.stop_streaming  = video_stop_streaming,  };
> > > > > > >
> > > > > > > +static int video_scd_queue_setup(struct vb2_queue *q,
> > > > > > > +				 unsigned int *num_buffers,
> > > > > > > +				 unsigned int *num_planes,
> > > > > > > +				 unsigned int sizes[],
> > > > > > > +				 struct device *alloc_devs[]) {
> > > > > > > +	if (*num_planes)
> > > > > > > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ?
> > -EINVAL :
> > > > > > > +0;
> > > > > > > +
> > > > > > > +	*num_planes = 1;
> > > > > > > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > > > > > > +
> > > > > > > +	return 0;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > > > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > > > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > > > > > > +	dma_addr_t *paddr;
> > > > > > > +
> > > > > > > +	paddr = vb2_plane_cookie(vb, 0);
> > > > > > > +	buffer->addr[0] = *paddr;
> > > > > > > +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > > > > >
> > > > > > Interesting, I don't see many users of vb2_plane_cookie() in
> > > > > > mainline and I'm not sure what this gives you as you use it to
> > > > > > program the
> > > > following registers:
> > > > > >
> > > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> > > > > >
> > > > >
> > > > > We set the value for ISP hardware, then ISP will transfer the
> > > > > statistics to the
> > > > buffer.
> > > > > when the stf_isp_irq_handler interrupt is triggered, indicates
> > > > > that the buffer fill is complete.
> > > > >
> > > >
> > > > So I take this as
> > > >
> > > > 	paddr = vb2_plane_cookie(vb, 0);
> > > > 	buffer->addr[0] = *paddr;
> > > > 	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > > >
> > > >         stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > > >                          change_buf->addr[1], type_scd);
> > > >
> > > > Makes the ISP transfer data directly to the memory areas in addr[0]
> > > > and addr[1] (which explains why struct jh7110_isp_sc_buffer is
> > > > packed, as it has to match the HW registers layout)
> > > >
> > > > If this is the case, why are you then manually copying the
> > > > histograms and the flags to vaddr ?
> > > >
> > >
> > > Yes, your are right.
> > > But actually there is a problem with our ISP RTL.
> > > We set this yhist_addr to ISP, but it actually not work.
> > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); or I
> > > will drop this line in next version.
> > >
> > > So, in this structure
> > > struct jh7110_isp_sc_buffer {
> > > 	__u32 y_histogram[64];
> > > 	__u32 reserv0[33];
> > > 	__u32 bright_sc[4096];
> > > 	__u32 reserv1[96];
> > > 	__u32 ae_hist_y[128];
> > > 	__u32 reserv2[511];
> > > 	__u16 flag;
> > > };
> > >
> > > Only
> > >
> > > 	__u32 reserv0[33];
> > > 	__u32 bright_sc[4096];
> > > 	__u32 reserv1[96];
> > > 	__u32 ae_hist_y[128];
> > > 	__u32 reserv2[511];
> > >
> > > transfer by ISP hardware.
> > >
> > > I need to fill
> > > 	__u32 y_histogram[64];
> > > 	__u16 flag;
> > >
> > > by vaddr.
> >
> > I see.
> >
> > Apart from the fact you can drop paddr and vb2_plane_cookie() and use vaddr
> > for all (if I'm not mistaken), could you please record the above rationale for
> > manually filling y_histogram and flag by hand in a comment to avoid future
> > readers being confused by this as I was ?
> >
>
> Still need to keep the paddr and vb2_plane_cookie() for

If you were using the dma API to issue DMA transfers then I would understand
you would have to use the dma_handles as set by dma_alloc_attrs(), but
in the vb2 world buf->vaddr = buf->cookie.

Anyway, I haven't run this part of the code, if you could verify the
addresses of paddr and vaddr are different, then I'll be happy to shut
up :)


> 	__u32 reserv0[33];
> 	__u32 bright_sc[4096];
> 	__u32 reserv1[96];
> 	__u32 ae_hist_y[128];
>  	__u32 reserv2[511];
>
> Because this part is filled by the hardware.
>
> I will add more information for struct jh7110_isp_sc_buffer
>

Thanks, a comment in buf_init() to explain that part of the struct is
filled by hw and part has to be manually filled would be enough. From
an userspace API point of view 'struct jh7110_isp_sc_buffer' will just
match the HW layout, regardless of how it is filled up.

> Regards,
> Changhuang
>
>

回复: 回复: 回复: 回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Changhuang Liang]

Hi, Jacopo

> Hi Changhuang
> 
> On Fri, Jul 12, 2024 at 01:00:03PM GMT, Changhuang Liang wrote:
> > Hi, Jacopo
> >
> > >
> > > Hi Changhuang
> > >
> > > On Fri, Jul 12, 2024 at 08:36:21AM GMT, Changhuang Liang wrote:
> > > > Hi, Jacopo
> > > >
> > > > [...]
> > > > > > > > +
> > > > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > > > +		      enum stf_isp_type_scd type_scd) {
> > > > > > > > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1,
> > > ISP_SC_SEL_MASK,
> > > > > > > > +			    SEL_TYPE(type_scd));
> > > > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4,
> > > > > > > > +yhist_addr); }
> > > > > > > > +
> > > > > > > > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss,
> > > > > > > > +void
> > > > > > > > +*vaddr) {
> > > > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > > > +jh7110_isp_sc_buffer
> > > > > *)vaddr;
> > > > > > > > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > > > > > > > +	u32 i;
> > > > > > > > +
> > > > > > > > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > > > > > > > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss,
> > > reg_addr);
> > > > > > >
> > > > > > > If you have a contigous memory space to read, could
> > > > > > > memcpy_fromio() help instead of going through 64 reads ?
> > > > > > >
> > > > > >
> > > > > > I will try this function.
> > > > > >
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void
> *vaddr,
> > > > > > > > +			      enum stf_isp_type_scd *type_scd) {
> > > > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > > > +jh7110_isp_sc_buffer *)vaddr;
> > > > > > > > +
> > > > > > > > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > > > > > > > +	if (*type_scd == TYPE_AWB) {
> > > > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > > > > > > > +		*type_scd = TYPE_OECF;
> > > > > > > > +	} else {
> > > > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > > > > > > > +		*type_scd = TYPE_AWB;
> > > > > > >
> > > > > > > Is this correct ? Why are you overwriting the value read
> > > > > > > from HW that indicates AE/AF stats with AWB ones ?
> > > > > >
> > > > > > The AWB frame and AE/AF frames will alternate, so the current
> > > > > > frame indicates the AE/AF, then set AWB type just for next AWB
> frame.
> > > > > >
> > > > >
> > > > > Ah! Shouldn't it be userspace configuring which type of
> > > > > statistics it wants to receive instead of the driver alternating the two ?
> > > > >
> > > >
> > > > No, this is determined by hardware, cannot be configured by userspace.
> > > >
> > >
> > >
> > > So this
> > > 	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> > > 			    SEL_TYPE(type_scd));
> > >
> > > doesn't actually select which stats type you get from the HW
> > >
> >
> > You can understand it that way. But it still needs to be written to work with
> the hardware.
> >
> 
> ack, maybe a comment here as well would help
> 

Agreed.

> > > > > > >
> > > > > > > > +	}
> > > > > > > > +}
> > > > > > > > +
> > > > > > > >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> > > > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > > > >  	struct stf_capture *cap =
> > > > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > > > +	struct stf_capture *cap_scd =
> > > > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > > > >  	struct stfcamss_buffer *change_buf;
> > > > > > > > +	enum stf_isp_type_scd type_scd;
> > > > > > > > +	u32 value;
> > > > > > > >  	u32 status;
> > > > > > > >
> > > > > > > >  	status = stf_isp_reg_read(stfcamss,
> ISP_REG_ISP_CTRL_0);
> > > @@
> > > > > > > > -467,6
> > > > > > > > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void
> > > > > > > > +*priv)
> > > > > > > >  					stf_set_yuv_addr(stfcamss,
> > > change_buf->addr[0],
> > > > > > > >  							 change_buf->addr[1]);
> > > > > > > >  			}
> > > > > > > > +
> > > > > > > > +			value = stf_isp_reg_read(stfcamss,
> > > > > > > ISP_REG_CSI_MODULE_CFG);
> > > > > > > > +			if (value & CSI_SC_EN) {
> > > > > > > > +				change_buf =
> > > stf_change_buffer(&cap_scd->buffers);
> > > > > > > > +				if (change_buf) {
> > > > > > > > +					stf_isp_fill_flag(stfcamss,
> > > change_buf->vaddr,
> > > > > > > > +							  &type_scd);
> > > > > > > > +					stf_set_scd_addr(stfcamss,
> > > change_buf->addr[0],
> > > > > > > > +							 change_buf->addr[1], type_scd);
> > > > > > >
> > > > > > > Sorry if I'm un-familiar with the HW but this seems to be
> > > > > > > the
> > > line-interrupt.
> > > > > > > Are you swapping buffers every line or it's just that you
> > > > > > > have a single line irq for the stats ?
> > > > > > >
> > > > > >
> > > > > > Every frame triggers a line-interrupt, and we will swap buffers in it.
> > > > > >
> > > > >
> > > > > ah, frames completion triggers a line-interrupt ?
> > > > >
> > > >
> > > > Every frame will trigger line-interrupt and stf_isp_irq_handler.
> > > > We use line-interrupt changing buffer, the stf_isp_irq_handler
> > > > will indicate that image transfer to DDR is complete.
> > > >
> > > >
> > > > > > > > +				}
> > > > > > > > +			}
> > > > > > > >  		}
> > > > > > > >
> > > > > > > >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@
> > > -485,6
> > > > > +542,7
> > > > > > > @@
> > > > > > > > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> > > > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > > > >  	struct stf_capture *cap =
> > > > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > > > +	struct stf_capture *cap_scd =
> > > > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > > > >  	struct stfcamss_buffer *ready_buf;
> > > > > > > >  	u32 status;
> > > > > > > >
> > > > > > > > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int
> > > > > > > > irq, void
> > > *priv)
> > > > > > > >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > > > VB2_BUF_STATE_DONE);
> > > > > > > >  		}
> > > > > > > >
> > > > > > > > +		if (status & ISPC_SC) {
> > > > > > > > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > > > > > > > +			if (ready_buf) {
> > > > > > > > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > > > > > > > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > > > VB2_BUF_STATE_DONE);
> > > > > > > > +			}
> > > > > > > > +		}
> > > > > > > > +
> > > > > > > >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> > > > > > > >  				  (status & ~ISPC_INT_ALL_MASK) |
> > > > > > > >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > > > > > > > a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > index fcda0502e3b0..0af7b367e57a 100644
> > > > > > > > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > @@ -10,6 +10,7 @@
> > > > > > > >  #ifndef STF_ISP_H
> > > > > > > >  #define STF_ISP_H
> > > > > > > >
> > > > > > > > +#include <linux/jh7110-isp.h>
> > > > > > > >  #include <media/v4l2-subdev.h>
> > > > > > > >
> > > > > > > >  #include "stf-video.h"
> > > > > > > > @@ -107,6 +108,12 @@
> > > > > > > >  #define Y_COOR(y)				((y) << 16)
> > > > > > > >  #define X_COOR(x)				((x) << 0)
> > > > > > > >
> > > > > > > > +#define ISP_REG_SCD_CFG_0			0x098
> > > > > > > > +
> > > > > > > > +#define ISP_REG_SC_CFG_1			0x0bc
> > > > > > > > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > > > > > > > +#define SEL_TYPE(n)				((n) << 30)
> > > > > > > > +
> > > > > > > >  #define ISP_REG_LCCF_CFG_2			0x0e0
> > > > > > > >  #define ISP_REG_LCCF_CFG_3			0x0e4
> > > > > > > >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > > > > > > > @@ -305,6 +312,10 @@
> > > > > > > >  #define DNRM_F(n)				((n) << 16)
> > > > > > > >  #define CCM_M_DAT(n)				((n) << 0)
> > > > > > > >
> > > > > > > > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > > > > > > > +
> > > > > > > > +#define ISP_REG_YHIST_ACC_0			0xd00
> > > > > > > > +
> > > > > > > >  #define ISP_REG_GAMMA_VAL0			0xe00
> > > > > > > >  #define ISP_REG_GAMMA_VAL1			0xe04
> > > > > > > >  #define ISP_REG_GAMMA_VAL2			0xe08
> > > > > > > > @@ -389,6 +400,15 @@
> > > > > > > >  #define IMAGE_MAX_WIDTH				1920
> > > > > > > >  #define IMAGE_MAX_HEIGH				1080
> > > > > > > >
> > > > > > > > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> > > > > > >
> > > > > > > Should this be in the uAPI header as it is useful to userspace as
> well ?
> > > > > > >
> > > > > > > you could:
> > > > > > >
> > > > > > > struct jh7110_isp_sc_buffer {
> > > > > > > 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> > > > > > > 	__u32 reserv0[33];
> > > > > > > 	__u32 bright_sc[4096];
> > > > > > > 	__u32 reserv1[96];
> > > > > > > 	__u32 ae_hist_y[128];
> > > > > > > 	__u32 reserv2[511];
> > > > > > > 	__u16 flag;
> > > > > > > };
> > > > > > >
> > > > > > > ofc if the size is made part of the uAPI you need a more
> > > > > > > proper name such as JH7110_ISP_YHIST_SIZE
> > > > > > >
> > > > > >
> > > > > > OK, I will try this.
> > > > > >
> > > > > > > > +
> > > > > > > > +enum stf_isp_type_scd {
> > > > > > > > +	TYPE_DEC = 0,
> > > > > > > > +	TYPE_OBC,
> > > > > > > > +	TYPE_OECF,
> > > > > > > > +	TYPE_AWB,
> > > > > > > > +};
> > > > > > > > +
> > > > > > > >  /* pad id for media framework */  enum stf_isp_pad_id {
> > > > > > > >  	STF_ISP_PAD_SINK = 0,
> > > > > > > > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct
> > > > > > > > stf_isp_dev *isp_dev);
> > > > > > > >
> > > > > > > >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > > > > > > >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > > > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > > > +		      enum stf_isp_type_scd type_scd);
> > > > > > > >
> > > > > > > >  #endif /* STF_ISP_H */
> > > > > > > > diff --git
> > > > > > > > a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > index 989b5e82bae9..2203605ec9c7 100644
> > > > > > > > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > @@ -125,6 +125,14 @@ static int
> > > > > > > > stf_video_init_format(struct
> > > > > > > stfcamss_video *video)
> > > > > > > >  	return 0;
> > > > > > > >  }
> > > > > > > >
> > > > > > > > +static int stf_video_scd_init_format(struct
> > > > > > > > +stfcamss_video
> > > > > > > > +*video)
> > > > > > >
> > > > > > > Make it void if it can't fail (see below)
> > > > > > >
> > > > > >
> > > > > > OK.
> > > > > >
> > > > > > > > +{
> > > > > > > > +	video->active_fmt.fmt.meta.dataformat =
> > > > > > > video->formats[0].pixelformat;
> > > > > > > > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > > > > > > > +jh7110_isp_sc_buffer);
> > > > > > > > +
> > > > > > > > +	return 0;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > >  /*
> -----------------------------------------------------------------------------
> > > > > > > >   * Video queue operations
> > > > > > > >   */
> > > > > > > > @@ -330,6 +338,78 @@ static const struct vb2_ops
> > > > > > > > stf_video_vb2_q_ops =
> > > > > > > {
> > > > > > > >  	.stop_streaming  = video_stop_streaming,  };
> > > > > > > >
> > > > > > > > +static int video_scd_queue_setup(struct vb2_queue *q,
> > > > > > > > +				 unsigned int *num_buffers,
> > > > > > > > +				 unsigned int *num_planes,
> > > > > > > > +				 unsigned int sizes[],
> > > > > > > > +				 struct device *alloc_devs[]) {
> > > > > > > > +	if (*num_planes)
> > > > > > > > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ?
> > > -EINVAL :
> > > > > > > > +0;
> > > > > > > > +
> > > > > > > > +	*num_planes = 1;
> > > > > > > > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > > > > > > > +
> > > > > > > > +	return 0;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > > > > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > > > > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > > > > > > > +	dma_addr_t *paddr;
> > > > > > > > +
> > > > > > > > +	paddr = vb2_plane_cookie(vb, 0);
> > > > > > > > +	buffer->addr[0] = *paddr;
> > > > > > > > +	buffer->addr[1] = buffer->addr[0] +
> > > > > > > > +ISP_YHIST_BUFFER_SIZE;
> > > > > > >
> > > > > > > Interesting, I don't see many users of vb2_plane_cookie() in
> > > > > > > mainline and I'm not sure what this gives you as you use it
> > > > > > > to program the
> > > > > following registers:
> > > > > > >
> > > > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4,
> > > > > > > yhist_addr);
> > > > > > >
> > > > > >
> > > > > > We set the value for ISP hardware, then ISP will transfer the
> > > > > > statistics to the
> > > > > buffer.
> > > > > > when the stf_isp_irq_handler interrupt is triggered, indicates
> > > > > > that the buffer fill is complete.
> > > > > >
> > > > >
> > > > > So I take this as
> > > > >
> > > > > 	paddr = vb2_plane_cookie(vb, 0);
> > > > > 	buffer->addr[0] = *paddr;
> > > > > 	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > > > >
> > > > >         stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > > > >                          change_buf->addr[1], type_scd);
> > > > >
> > > > > Makes the ISP transfer data directly to the memory areas in
> > > > > addr[0] and addr[1] (which explains why struct
> > > > > jh7110_isp_sc_buffer is packed, as it has to match the HW
> > > > > registers layout)
> > > > >
> > > > > If this is the case, why are you then manually copying the
> > > > > histograms and the flags to vaddr ?
> > > > >
> > > >
> > > > Yes, your are right.
> > > > But actually there is a problem with our ISP RTL.
> > > > We set this yhist_addr to ISP, but it actually not work.
> > > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); or
> > > > I will drop this line in next version.
> > > >
> > > > So, in this structure
> > > > struct jh7110_isp_sc_buffer {
> > > > 	__u32 y_histogram[64];
> > > > 	__u32 reserv0[33];
> > > > 	__u32 bright_sc[4096];
> > > > 	__u32 reserv1[96];
> > > > 	__u32 ae_hist_y[128];
> > > > 	__u32 reserv2[511];
> > > > 	__u16 flag;
> > > > };
> > > >
> > > > Only
> > > >
> > > > 	__u32 reserv0[33];
> > > > 	__u32 bright_sc[4096];
> > > > 	__u32 reserv1[96];
> > > > 	__u32 ae_hist_y[128];
> > > > 	__u32 reserv2[511];
> > > >
> > > > transfer by ISP hardware.
> > > >
> > > > I need to fill
> > > > 	__u32 y_histogram[64];
> > > > 	__u16 flag;
> > > >
> > > > by vaddr.
> > >
> > > I see.
> > >
> > > Apart from the fact you can drop paddr and vb2_plane_cookie() and
> > > use vaddr for all (if I'm not mistaken), could you please record the
> > > above rationale for manually filling y_histogram and flag by hand in
> > > a comment to avoid future readers being confused by this as I was ?
> > >
> >
> > Still need to keep the paddr and vb2_plane_cookie() for
> 
> If you were using the dma API to issue DMA transfers then I would understand
> you would have to use the dma_handles as set by dma_alloc_attrs(), but in
> the vb2 world buf->vaddr = buf->cookie.
> 

Yes, but vb2_plane_cookie() actually called the vb2_dc_cookie,

const struct vb2_mem_ops vb2_dma_contig_memops = {
	.alloc		= vb2_dc_alloc,
	.put		= vb2_dc_put,
	.get_dmabuf	= vb2_dc_get_dmabuf,
	.cookie		= vb2_dc_cookie,
	.vaddr		= vb2_dc_vaddr,
	.mmap		= vb2_dc_mmap,
	.get_userptr	= vb2_dc_get_userptr,
	.put_userptr	= vb2_dc_put_userptr,
	.prepare	= vb2_dc_prepare,
	.finish		= vb2_dc_finish,
	.map_dmabuf	= vb2_dc_map_dmabuf,
	.unmap_dmabuf	= vb2_dc_unmap_dmabuf,
	.attach_dmabuf	= vb2_dc_attach_dmabuf,
	.detach_dmabuf	= vb2_dc_detach_dmabuf,
	.num_users	= vb2_dc_num_users,
};

static void *vb2_dc_cookie(struct vb2_buffer *vb, void *buf_priv)
{
	struct vb2_dc_buf *buf = buf_priv;

	return &buf->dma_addr;
}

It not return the buf->vaddr = buf->cookie.

Regards, 
Changhuang

> Anyway, I haven't run this part of the code, if you could verify the addresses of
> paddr and vaddr are different, then I'll be happy to shut up :)
> 
> 
> > 	__u32 reserv0[33];
> > 	__u32 bright_sc[4096];
> > 	__u32 reserv1[96];
> > 	__u32 ae_hist_y[128];
> >  	__u32 reserv2[511];
> >
> > Because this part is filled by the hardware.
> >
> > I will add more information for struct jh7110_isp_sc_buffer
> >
> 
> Thanks, a comment in buf_init() to explain that part of the struct is filled by
> hw and part has to be manually filled would be enough. From an userspace
> API point of view 'struct jh7110_isp_sc_buffer' will just match the HW layout,
> regardless of how it is filled up.
> 
> > Regards,
> > Changhuang
> >
> >

Re: 回复: 回复: 回复: 回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Jacopo Mondi]

Hi again

On Fri, Jul 12, 2024 at 01:37:58PM GMT, Changhuang Liang wrote:
> Hi, Jacopo
>
> > Hi Changhuang
> >
> > On Fri, Jul 12, 2024 at 01:00:03PM GMT, Changhuang Liang wrote:
> > > Hi, Jacopo
> > >
> > > >
> > > > Hi Changhuang
> > > >
> > > > On Fri, Jul 12, 2024 at 08:36:21AM GMT, Changhuang Liang wrote:
> > > > > Hi, Jacopo
> > > > >
> > > > > [...]
> > > > > > > > > +
> > > > > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > > > > +		      enum stf_isp_type_scd type_scd) {
> > > > > > > > > +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1,
> > > > ISP_SC_SEL_MASK,
> > > > > > > > > +			    SEL_TYPE(type_scd));
> > > > > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > > > > > +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4,
> > > > > > > > > +yhist_addr); }
> > > > > > > > > +
> > > > > > > > > +static void stf_isp_fill_yhist(struct stfcamss *stfcamss,
> > > > > > > > > +void
> > > > > > > > > +*vaddr) {
> > > > > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > > > > +jh7110_isp_sc_buffer
> > > > > > *)vaddr;
> > > > > > > > > +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> > > > > > > > > +	u32 i;
> > > > > > > > > +
> > > > > > > > > +	for (i = 0; i < 64; i++, reg_addr += 4)
> > > > > > > > > +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss,
> > > > reg_addr);
> > > > > > > >
> > > > > > > > If you have a contigous memory space to read, could
> > > > > > > > memcpy_fromio() help instead of going through 64 reads ?
> > > > > > > >
> > > > > > >
> > > > > > > I will try this function.
> > > > > > >
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void
> > *vaddr,
> > > > > > > > > +			      enum stf_isp_type_scd *type_scd) {
> > > > > > > > > +	struct jh7110_isp_sc_buffer *sc = (struct
> > > > > > > > > +jh7110_isp_sc_buffer *)vaddr;
> > > > > > > > > +
> > > > > > > > > +	*type_scd = stf_isp_get_scd_type(stfcamss);
> > > > > > > > > +	if (*type_scd == TYPE_AWB) {
> > > > > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> > > > > > > > > +		*type_scd = TYPE_OECF;
> > > > > > > > > +	} else {
> > > > > > > > > +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> > > > > > > > > +		*type_scd = TYPE_AWB;
> > > > > > > >
> > > > > > > > Is this correct ? Why are you overwriting the value read
> > > > > > > > from HW that indicates AE/AF stats with AWB ones ?
> > > > > > >
> > > > > > > The AWB frame and AE/AF frames will alternate, so the current
> > > > > > > frame indicates the AE/AF, then set AWB type just for next AWB
> > frame.
> > > > > > >
> > > > > >
> > > > > > Ah! Shouldn't it be userspace configuring which type of
> > > > > > statistics it wants to receive instead of the driver alternating the two ?
> > > > > >
> > > > >
> > > > > No, this is determined by hardware, cannot be configured by userspace.
> > > > >
> > > >
> > > >
> > > > So this
> > > > 	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> > > > 			    SEL_TYPE(type_scd));
> > > >
> > > > doesn't actually select which stats type you get from the HW
> > > >
> > >
> > > You can understand it that way. But it still needs to be written to work with
> > the hardware.
> > >
> >
> > ack, maybe a comment here as well would help
> >
>
> Agreed.
>
> > > > > > > >
> > > > > > > > > +	}
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > >  irqreturn_t stf_line_irq_handler(int irq, void *priv)  {
> > > > > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > > > > >  	struct stf_capture *cap =
> > > > > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > > > > +	struct stf_capture *cap_scd =
> > > > > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > > > > >  	struct stfcamss_buffer *change_buf;
> > > > > > > > > +	enum stf_isp_type_scd type_scd;
> > > > > > > > > +	u32 value;
> > > > > > > > >  	u32 status;
> > > > > > > > >
> > > > > > > > >  	status = stf_isp_reg_read(stfcamss,
> > ISP_REG_ISP_CTRL_0);
> > > > @@
> > > > > > > > > -467,6
> > > > > > > > > +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void
> > > > > > > > > +*priv)
> > > > > > > > >  					stf_set_yuv_addr(stfcamss,
> > > > change_buf->addr[0],
> > > > > > > > >  							 change_buf->addr[1]);
> > > > > > > > >  			}
> > > > > > > > > +
> > > > > > > > > +			value = stf_isp_reg_read(stfcamss,
> > > > > > > > ISP_REG_CSI_MODULE_CFG);
> > > > > > > > > +			if (value & CSI_SC_EN) {
> > > > > > > > > +				change_buf =
> > > > stf_change_buffer(&cap_scd->buffers);
> > > > > > > > > +				if (change_buf) {
> > > > > > > > > +					stf_isp_fill_flag(stfcamss,
> > > > change_buf->vaddr,
> > > > > > > > > +							  &type_scd);
> > > > > > > > > +					stf_set_scd_addr(stfcamss,
> > > > change_buf->addr[0],
> > > > > > > > > +							 change_buf->addr[1], type_scd);
> > > > > > > >
> > > > > > > > Sorry if I'm un-familiar with the HW but this seems to be
> > > > > > > > the
> > > > line-interrupt.
> > > > > > > > Are you swapping buffers every line or it's just that you
> > > > > > > > have a single line irq for the stats ?
> > > > > > > >
> > > > > > >
> > > > > > > Every frame triggers a line-interrupt, and we will swap buffers in it.
> > > > > > >
> > > > > >
> > > > > > ah, frames completion triggers a line-interrupt ?
> > > > > >
> > > > >
> > > > > Every frame will trigger line-interrupt and stf_isp_irq_handler.
> > > > > We use line-interrupt changing buffer, the stf_isp_irq_handler
> > > > > will indicate that image transfer to DDR is complete.
> > > > >
> > > > >
> > > > > > > > > +				}
> > > > > > > > > +			}
> > > > > > > > >  		}
> > > > > > > > >
> > > > > > > > >  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS, @@
> > > > -485,6
> > > > > > +542,7
> > > > > > > > @@
> > > > > > > > > irqreturn_t stf_isp_irq_handler(int irq, void *priv)  {
> > > > > > > > >  	struct stfcamss *stfcamss = priv;
> > > > > > > > >  	struct stf_capture *cap =
> > > > > > > > > &stfcamss->captures[STF_CAPTURE_YUV];
> > > > > > > > > +	struct stf_capture *cap_scd =
> > > > > > > > > +&stfcamss->captures[STF_CAPTURE_SCD];
> > > > > > > > >  	struct stfcamss_buffer *ready_buf;
> > > > > > > > >  	u32 status;
> > > > > > > > >
> > > > > > > > > @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int
> > > > > > > > > irq, void
> > > > *priv)
> > > > > > > > >  				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > > > > VB2_BUF_STATE_DONE);
> > > > > > > > >  		}
> > > > > > > > >
> > > > > > > > > +		if (status & ISPC_SC) {
> > > > > > > > > +			ready_buf = stf_buf_done(&cap_scd->buffers);
> > > > > > > > > +			if (ready_buf) {
> > > > > > > > > +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> > > > > > > > > +				vb2_buffer_done(&ready_buf->vb.vb2_buf,
> > > > > > > > VB2_BUF_STATE_DONE);
> > > > > > > > > +			}
> > > > > > > > > +		}
> > > > > > > > > +
> > > > > > > > >  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
> > > > > > > > >  				  (status & ~ISPC_INT_ALL_MASK) |
> > > > > > > > >  				  ISPC_ISP | ISPC_CSI | ISPC_SC); diff --git
> > > > > > > > > a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > > b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > > index fcda0502e3b0..0af7b367e57a 100644
> > > > > > > > > --- a/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > > +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> > > > > > > > > @@ -10,6 +10,7 @@
> > > > > > > > >  #ifndef STF_ISP_H
> > > > > > > > >  #define STF_ISP_H
> > > > > > > > >
> > > > > > > > > +#include <linux/jh7110-isp.h>
> > > > > > > > >  #include <media/v4l2-subdev.h>
> > > > > > > > >
> > > > > > > > >  #include "stf-video.h"
> > > > > > > > > @@ -107,6 +108,12 @@
> > > > > > > > >  #define Y_COOR(y)				((y) << 16)
> > > > > > > > >  #define X_COOR(x)				((x) << 0)
> > > > > > > > >
> > > > > > > > > +#define ISP_REG_SCD_CFG_0			0x098
> > > > > > > > > +
> > > > > > > > > +#define ISP_REG_SC_CFG_1			0x0bc
> > > > > > > > > +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> > > > > > > > > +#define SEL_TYPE(n)				((n) << 30)
> > > > > > > > > +
> > > > > > > > >  #define ISP_REG_LCCF_CFG_2			0x0e0
> > > > > > > > >  #define ISP_REG_LCCF_CFG_3			0x0e4
> > > > > > > > >  #define ISP_REG_LCCF_CFG_4			0x0e8
> > > > > > > > > @@ -305,6 +312,10 @@
> > > > > > > > >  #define DNRM_F(n)				((n) << 16)
> > > > > > > > >  #define CCM_M_DAT(n)				((n) << 0)
> > > > > > > > >
> > > > > > > > > +#define ISP_REG_YHIST_CFG_4			0xcd8
> > > > > > > > > +
> > > > > > > > > +#define ISP_REG_YHIST_ACC_0			0xd00
> > > > > > > > > +
> > > > > > > > >  #define ISP_REG_GAMMA_VAL0			0xe00
> > > > > > > > >  #define ISP_REG_GAMMA_VAL1			0xe04
> > > > > > > > >  #define ISP_REG_GAMMA_VAL2			0xe08
> > > > > > > > > @@ -389,6 +400,15 @@
> > > > > > > > >  #define IMAGE_MAX_WIDTH				1920
> > > > > > > > >  #define IMAGE_MAX_HEIGH				1080
> > > > > > > > >
> > > > > > > > > +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> > > > > > > >
> > > > > > > > Should this be in the uAPI header as it is useful to userspace as
> > well ?
> > > > > > > >
> > > > > > > > you could:
> > > > > > > >
> > > > > > > > struct jh7110_isp_sc_buffer {
> > > > > > > > 	__u8 y_histogram[ISP_YHIST_BUFFER_SIZE];
> > > > > > > > 	__u32 reserv0[33];
> > > > > > > > 	__u32 bright_sc[4096];
> > > > > > > > 	__u32 reserv1[96];
> > > > > > > > 	__u32 ae_hist_y[128];
> > > > > > > > 	__u32 reserv2[511];
> > > > > > > > 	__u16 flag;
> > > > > > > > };
> > > > > > > >
> > > > > > > > ofc if the size is made part of the uAPI you need a more
> > > > > > > > proper name such as JH7110_ISP_YHIST_SIZE
> > > > > > > >
> > > > > > >
> > > > > > > OK, I will try this.
> > > > > > >
> > > > > > > > > +
> > > > > > > > > +enum stf_isp_type_scd {
> > > > > > > > > +	TYPE_DEC = 0,
> > > > > > > > > +	TYPE_OBC,
> > > > > > > > > +	TYPE_OECF,
> > > > > > > > > +	TYPE_AWB,
> > > > > > > > > +};
> > > > > > > > > +
> > > > > > > > >  /* pad id for media framework */  enum stf_isp_pad_id {
> > > > > > > > >  	STF_ISP_PAD_SINK = 0,
> > > > > > > > > @@ -429,5 +449,8 @@ int stf_isp_unregister(struct
> > > > > > > > > stf_isp_dev *isp_dev);
> > > > > > > > >
> > > > > > > > >  void stf_set_yuv_addr(struct stfcamss *stfcamss,
> > > > > > > > >  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> > > > > > > > > +void stf_set_scd_addr(struct stfcamss *stfcamss,
> > > > > > > > > +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> > > > > > > > > +		      enum stf_isp_type_scd type_scd);
> > > > > > > > >
> > > > > > > > >  #endif /* STF_ISP_H */
> > > > > > > > > diff --git
> > > > > > > > > a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > > b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > > index 989b5e82bae9..2203605ec9c7 100644
> > > > > > > > > --- a/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > > +++ b/drivers/staging/media/starfive/camss/stf-video.c
> > > > > > > > > @@ -125,6 +125,14 @@ static int
> > > > > > > > > stf_video_init_format(struct
> > > > > > > > stfcamss_video *video)
> > > > > > > > >  	return 0;
> > > > > > > > >  }
> > > > > > > > >
> > > > > > > > > +static int stf_video_scd_init_format(struct
> > > > > > > > > +stfcamss_video
> > > > > > > > > +*video)
> > > > > > > >
> > > > > > > > Make it void if it can't fail (see below)
> > > > > > > >
> > > > > > >
> > > > > > > OK.
> > > > > > >
> > > > > > > > > +{
> > > > > > > > > +	video->active_fmt.fmt.meta.dataformat =
> > > > > > > > video->formats[0].pixelformat;
> > > > > > > > > +	video->active_fmt.fmt.meta.buffersize = sizeof(struct
> > > > > > > > > +jh7110_isp_sc_buffer);
> > > > > > > > > +
> > > > > > > > > +	return 0;
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > >  /*
> > -----------------------------------------------------------------------------
> > > > > > > > >   * Video queue operations
> > > > > > > > >   */
> > > > > > > > > @@ -330,6 +338,78 @@ static const struct vb2_ops
> > > > > > > > > stf_video_vb2_q_ops =
> > > > > > > > {
> > > > > > > > >  	.stop_streaming  = video_stop_streaming,  };
> > > > > > > > >
> > > > > > > > > +static int video_scd_queue_setup(struct vb2_queue *q,
> > > > > > > > > +				 unsigned int *num_buffers,
> > > > > > > > > +				 unsigned int *num_planes,
> > > > > > > > > +				 unsigned int sizes[],
> > > > > > > > > +				 struct device *alloc_devs[]) {
> > > > > > > > > +	if (*num_planes)
> > > > > > > > > +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ?
> > > > -EINVAL :
> > > > > > > > > +0;
> > > > > > > > > +
> > > > > > > > > +	*num_planes = 1;
> > > > > > > > > +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> > > > > > > > > +
> > > > > > > > > +	return 0;
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +static int video_scd_buf_init(struct vb2_buffer *vb) {
> > > > > > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > > > > > +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> > > > > > > > > +	dma_addr_t *paddr;
> > > > > > > > > +
> > > > > > > > > +	paddr = vb2_plane_cookie(vb, 0);
> > > > > > > > > +	buffer->addr[0] = *paddr;
> > > > > > > > > +	buffer->addr[1] = buffer->addr[0] +
> > > > > > > > > +ISP_YHIST_BUFFER_SIZE;
> > > > > > > >
> > > > > > > > Interesting, I don't see many users of vb2_plane_cookie() in
> > > > > > > > mainline and I'm not sure what this gives you as you use it
> > > > > > > > to program the
> > > > > > following registers:
> > > > > > > >
> > > > > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> > > > > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4,
> > > > > > > > yhist_addr);
> > > > > > > >
> > > > > > >
> > > > > > > We set the value for ISP hardware, then ISP will transfer the
> > > > > > > statistics to the
> > > > > > buffer.
> > > > > > > when the stf_isp_irq_handler interrupt is triggered, indicates
> > > > > > > that the buffer fill is complete.
> > > > > > >
> > > > > >
> > > > > > So I take this as
> > > > > >
> > > > > > 	paddr = vb2_plane_cookie(vb, 0);
> > > > > > 	buffer->addr[0] = *paddr;
> > > > > > 	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> > > > > >
> > > > > >         stf_set_scd_addr(stfcamss, change_buf->addr[0],
> > > > > >                          change_buf->addr[1], type_scd);
> > > > > >
> > > > > > Makes the ISP transfer data directly to the memory areas in
> > > > > > addr[0] and addr[1] (which explains why struct
> > > > > > jh7110_isp_sc_buffer is packed, as it has to match the HW
> > > > > > registers layout)
> > > > > >
> > > > > > If this is the case, why are you then manually copying the
> > > > > > histograms and the flags to vaddr ?
> > > > > >
> > > > >
> > > > > Yes, your are right.
> > > > > But actually there is a problem with our ISP RTL.
> > > > > We set this yhist_addr to ISP, but it actually not work.
> > > > > 	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr); or
> > > > > I will drop this line in next version.
> > > > >
> > > > > So, in this structure
> > > > > struct jh7110_isp_sc_buffer {
> > > > > 	__u32 y_histogram[64];
> > > > > 	__u32 reserv0[33];
> > > > > 	__u32 bright_sc[4096];
> > > > > 	__u32 reserv1[96];
> > > > > 	__u32 ae_hist_y[128];
> > > > > 	__u32 reserv2[511];
> > > > > 	__u16 flag;
> > > > > };
> > > > >
> > > > > Only
> > > > >
> > > > > 	__u32 reserv0[33];
> > > > > 	__u32 bright_sc[4096];
> > > > > 	__u32 reserv1[96];
> > > > > 	__u32 ae_hist_y[128];
> > > > > 	__u32 reserv2[511];
> > > > >
> > > > > transfer by ISP hardware.
> > > > >
> > > > > I need to fill
> > > > > 	__u32 y_histogram[64];
> > > > > 	__u16 flag;
> > > > >
> > > > > by vaddr.
> > > >
> > > > I see.
> > > >
> > > > Apart from the fact you can drop paddr and vb2_plane_cookie() and
> > > > use vaddr for all (if I'm not mistaken), could you please record the
> > > > above rationale for manually filling y_histogram and flag by hand in
> > > > a comment to avoid future readers being confused by this as I was ?
> > > >
> > >
> > > Still need to keep the paddr and vb2_plane_cookie() for
> >
> > If you were using the dma API to issue DMA transfers then I would understand
> > you would have to use the dma_handles as set by dma_alloc_attrs(), but in
> > the vb2 world buf->vaddr = buf->cookie.
> >
>
> Yes, but vb2_plane_cookie() actually called the vb2_dc_cookie,
>
> const struct vb2_mem_ops vb2_dma_contig_memops = {
> 	.alloc		= vb2_dc_alloc,
> 	.put		= vb2_dc_put,
> 	.get_dmabuf	= vb2_dc_get_dmabuf,
> 	.cookie		= vb2_dc_cookie,
> 	.vaddr		= vb2_dc_vaddr,
> 	.mmap		= vb2_dc_mmap,
> 	.get_userptr	= vb2_dc_get_userptr,
> 	.put_userptr	= vb2_dc_put_userptr,
> 	.prepare	= vb2_dc_prepare,
> 	.finish		= vb2_dc_finish,
> 	.map_dmabuf	= vb2_dc_map_dmabuf,
> 	.unmap_dmabuf	= vb2_dc_unmap_dmabuf,
> 	.attach_dmabuf	= vb2_dc_attach_dmabuf,
> 	.detach_dmabuf	= vb2_dc_detach_dmabuf,
> 	.num_users	= vb2_dc_num_users,
> };
>
> static void *vb2_dc_cookie(struct vb2_buffer *vb, void *buf_priv)
> {
> 	struct vb2_dc_buf *buf = buf_priv;
>
> 	return &buf->dma_addr;
> }
>
> It not return the buf->vaddr = buf->cookie.

right right

I was only aware of vb2_dma_contig_plane_dma_addr() to get the
dma_addr handle, and I just realized it internally gets the plane
cookie:

static inline dma_addr_t
vb2_dma_contig_plane_dma_addr(struct vb2_buffer *vb, unsigned int plane_no)
{
	dma_addr_t *addr = vb2_plane_cookie(vb, plane_no);

	return *addr;
}

>
> Regards,
> Changhuang
>
> > Anyway, I haven't run this part of the code, if you could verify the addresses of
> > paddr and vaddr are different, then I'll be happy to shut up :)

Now I'm happy to shut up then :)

Thanks for explaining
   j

> >
> >
> > > 	__u32 reserv0[33];
> > > 	__u32 bright_sc[4096];
> > > 	__u32 reserv1[96];
> > > 	__u32 ae_hist_y[128];
> > >  	__u32 reserv2[511];
> > >
> > > Because this part is filled by the hardware.
> > >
> > > I will add more information for struct jh7110_isp_sc_buffer
> > >
> >
> > Thanks, a comment in buf_init() to explain that part of the struct is filled by
> > hw and part has to be manually filled would be enough. From an userspace
> > API point of view 'struct jh7110_isp_sc_buffer' will just match the HW layout,
> > regardless of how it is filled up.
> >
> > > Regards,
> > > Changhuang
> > >
> > >

Re: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Jacopo Mondi]

Hi Changhuang

Sorry, I missed this: the subject

    staging: media: starfive: Add for StarFive ISP 3A SC

Seems to be missing a word.

    staging: media: starfive: Add stats for StarFive ISP 3A

maybe ?

On Tue, Jul 09, 2024 at 01:38:18AM GMT, Changhuang Liang wrote:
> Register ISP 3A "capture_scd" video device to receive statistics
> collection data.
>
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  .../staging/media/starfive/camss/stf-buffer.h |   1 +
>  .../staging/media/starfive/camss/stf-camss.c  |  15 ++
>  .../media/starfive/camss/stf-capture.c        |  21 ++-
>  .../media/starfive/camss/stf-isp-hw-ops.c     |  66 ++++++++
>  .../staging/media/starfive/camss/stf-isp.h    |  23 +++
>  .../staging/media/starfive/camss/stf-video.c  | 146 +++++++++++++++++-
>  .../staging/media/starfive/camss/stf-video.h  |   1 +
>  7 files changed, 264 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/staging/media/starfive/camss/stf-buffer.h b/drivers/staging/media/starfive/camss/stf-buffer.h
> index 9d1670fb05ed..727d00617448 100644
> --- a/drivers/staging/media/starfive/camss/stf-buffer.h
> +++ b/drivers/staging/media/starfive/camss/stf-buffer.h
> @@ -23,6 +23,7 @@ enum stf_v_state {
>  struct stfcamss_buffer {
>  	struct vb2_v4l2_buffer vb;
>  	dma_addr_t addr[2];
> +	void *vaddr;
>  	struct list_head queue;
>  };
>
> diff --git a/drivers/staging/media/starfive/camss/stf-camss.c b/drivers/staging/media/starfive/camss/stf-camss.c
> index fecd3e67c7a1..fafa3ce2f6da 100644
> --- a/drivers/staging/media/starfive/camss/stf-camss.c
> +++ b/drivers/staging/media/starfive/camss/stf-camss.c
> @@ -8,6 +8,7 @@
>   *
>   * Author: Jack Zhu <jack.zhu@starfivetech.com>
>   * Author: Changhuang Liang <changhuang.liang@starfivetech.com>
> + * Author: Keith Zhao <keith.zhao@starfivetech.com>
>   *
>   */
>  #include <linux/module.h>
> @@ -126,6 +127,7 @@ static int stfcamss_of_parse_ports(struct stfcamss *stfcamss)
>  static int stfcamss_register_devs(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>  	int ret;
>
> @@ -150,8 +152,18 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
>
>  	cap_yuv->video.source_subdev = &isp_dev->subdev;
>
> +	ret = media_create_pad_link(&isp_dev->subdev.entity, STF_ISP_PAD_SRC_SCD,
> +				    &cap_scd->video.vdev.entity, 0, 0);
> +	if (ret)
> +		goto err_rm_links0;
> +
> +	cap_scd->video.source_subdev = &isp_dev->subdev;
> +
>  	return ret;
>
> +err_rm_links0:
> +	media_entity_remove_links(&isp_dev->subdev.entity);
> +	media_entity_remove_links(&cap_yuv->video.vdev.entity);
>  err_cap_unregister:
>  	stf_capture_unregister(stfcamss);
>  err_isp_unregister:
> @@ -163,10 +175,12 @@ static int stfcamss_register_devs(struct stfcamss *stfcamss)
>  static void stfcamss_unregister_devs(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stf_isp_dev *isp_dev = &stfcamss->isp_dev;
>
>  	media_entity_remove_links(&isp_dev->subdev.entity);
>  	media_entity_remove_links(&cap_yuv->video.vdev.entity);
> +	media_entity_remove_links(&cap_scd->video.vdev.entity);
>
>  	stf_isp_unregister(&stfcamss->isp_dev);
>  	stf_capture_unregister(stfcamss);
> @@ -436,5 +450,6 @@ module_platform_driver(stfcamss_driver);
>
>  MODULE_AUTHOR("Jack Zhu <jack.zhu@starfivetech.com>");
>  MODULE_AUTHOR("Changhuang Liang <changhuang.liang@starfivetech.com>");
> +MODULE_AUTHOR("Keith Zhao <keith.zhao@starfivetech.com>");
>  MODULE_DESCRIPTION("StarFive Camera Subsystem driver");
>  MODULE_LICENSE("GPL");
> diff --git a/drivers/staging/media/starfive/camss/stf-capture.c b/drivers/staging/media/starfive/camss/stf-capture.c
> index 75f6ef405e61..328b8c6e351d 100644
> --- a/drivers/staging/media/starfive/camss/stf-capture.c
> +++ b/drivers/staging/media/starfive/camss/stf-capture.c
> @@ -12,6 +12,7 @@
>  static const char * const stf_cap_names[] = {
>  	"capture_raw",
>  	"capture_yuv",
> +	"capture_scd",
>  };
>
>  static const struct stfcamss_format_info stf_wr_fmts[] = {
> @@ -55,6 +56,14 @@ static const struct stfcamss_format_info stf_isp_fmts[] = {
>  	},
>  };
>
> +/* 3A Statistics Collection Data */
> +static const struct stfcamss_format_info stf_isp_scd_fmts[] = {
> +	{
> +		.code = MEDIA_BUS_FMT_METADATA_FIXED,
> +		.pixelformat = V4L2_META_FMT_STF_ISP_STAT_3A,
> +	},
> +};
> +
>  static inline struct stf_capture *to_stf_capture(struct stfcamss_video *video)
>  {
>  	return container_of(video, struct stf_capture, video);
> @@ -84,6 +93,8 @@ static void stf_init_addrs(struct stfcamss_video *video)
>  		stf_set_raw_addr(video->stfcamss, addr0);
>  	else if (cap->type == STF_CAPTURE_YUV)
>  		stf_set_yuv_addr(video->stfcamss, addr0, addr1);
> +	else
> +		stf_set_scd_addr(video->stfcamss, addr0, addr1, TYPE_AWB);
>  }
>
>  static void stf_cap_s_cfg(struct stfcamss_video *video)
> @@ -227,18 +238,24 @@ static void stf_capture_init(struct stfcamss *stfcamss, struct stf_capture *cap)
>  	INIT_LIST_HEAD(&cap->buffers.ready_bufs);
>  	spin_lock_init(&cap->buffers.lock);
>
> -	cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  	cap->video.stfcamss = stfcamss;
>  	cap->video.bpl_alignment = 16 * 8;
>
>  	if (cap->type == STF_CAPTURE_RAW) {
> +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  		cap->video.formats = stf_wr_fmts;
>  		cap->video.nformats = ARRAY_SIZE(stf_wr_fmts);
>  		cap->video.bpl_alignment = 8;
>  	} else if (cap->type == STF_CAPTURE_YUV) {
> +		cap->video.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  		cap->video.formats = stf_isp_fmts;
>  		cap->video.nformats = ARRAY_SIZE(stf_isp_fmts);
>  		cap->video.bpl_alignment = 1;
> +	} else {
> +		cap->video.type = V4L2_BUF_TYPE_META_CAPTURE;
> +		cap->video.formats = stf_isp_scd_fmts;
> +		cap->video.nformats = ARRAY_SIZE(stf_isp_scd_fmts);
> +		cap->video.bpl_alignment = 16 * 8;
>  	}
>  }
>
> @@ -362,9 +379,11 @@ void stf_capture_unregister(struct stfcamss *stfcamss)
>  {
>  	struct stf_capture *cap_raw = &stfcamss->captures[STF_CAPTURE_RAW];
>  	struct stf_capture *cap_yuv = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>
>  	stf_capture_unregister_one(cap_raw);
>  	stf_capture_unregister_one(cap_yuv);
> +	stf_capture_unregister_one(cap_scd);
>  }
>
>  int stf_capture_register(struct stfcamss *stfcamss,
> diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> index 6b3966ca18bf..3b18d09f2cc6 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> +++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> @@ -451,11 +451,57 @@ void stf_set_yuv_addr(struct stfcamss *stfcamss,
>  	stf_isp_reg_write(stfcamss, ISP_REG_UV_PLANE_START_ADDR, uv_addr);
>  }
>
> +static enum stf_isp_type_scd stf_isp_get_scd_type(struct stfcamss *stfcamss)
> +{
> +	int val;
> +
> +	val = stf_isp_reg_read(stfcamss, ISP_REG_SC_CFG_1);
> +	return (enum stf_isp_type_scd)(val & ISP_SC_SEL_MASK) >> 30;
> +}
> +
> +void stf_set_scd_addr(struct stfcamss *stfcamss,
> +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> +		      enum stf_isp_type_scd type_scd)
> +{
> +	stf_isp_reg_set_bit(stfcamss, ISP_REG_SC_CFG_1, ISP_SC_SEL_MASK,
> +			    SEL_TYPE(type_scd));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_0, scd_addr);
> +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_4, yhist_addr);
> +}
> +
> +static void stf_isp_fill_yhist(struct stfcamss *stfcamss, void *vaddr)
> +{
> +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
> +	u32 reg_addr = ISP_REG_YHIST_ACC_0;
> +	u32 i;
> +
> +	for (i = 0; i < 64; i++, reg_addr += 4)
> +		sc->y_histogram[i] = stf_isp_reg_read(stfcamss, reg_addr);
> +}
> +
> +static void stf_isp_fill_flag(struct stfcamss *stfcamss, void *vaddr,
> +			      enum stf_isp_type_scd *type_scd)
> +{
> +	struct jh7110_isp_sc_buffer *sc = (struct jh7110_isp_sc_buffer *)vaddr;
> +
> +	*type_scd = stf_isp_get_scd_type(stfcamss);
> +	if (*type_scd == TYPE_AWB) {
> +		sc->flag = JH7110_ISP_SC_FLAG_AWB;
> +		*type_scd = TYPE_OECF;
> +	} else {
> +		sc->flag = JH7110_ISP_SC_FLAG_AE_AF;
> +		*type_scd = TYPE_AWB;
> +	}
> +}
> +
>  irqreturn_t stf_line_irq_handler(int irq, void *priv)
>  {
>  	struct stfcamss *stfcamss = priv;
>  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stfcamss_buffer *change_buf;
> +	enum stf_isp_type_scd type_scd;
> +	u32 value;
>  	u32 status;
>
>  	status = stf_isp_reg_read(stfcamss, ISP_REG_ISP_CTRL_0);
> @@ -467,6 +513,17 @@ irqreturn_t stf_line_irq_handler(int irq, void *priv)
>  					stf_set_yuv_addr(stfcamss, change_buf->addr[0],
>  							 change_buf->addr[1]);
>  			}
> +
> +			value = stf_isp_reg_read(stfcamss, ISP_REG_CSI_MODULE_CFG);
> +			if (value & CSI_SC_EN) {
> +				change_buf = stf_change_buffer(&cap_scd->buffers);
> +				if (change_buf) {
> +					stf_isp_fill_flag(stfcamss, change_buf->vaddr,
> +							  &type_scd);
> +					stf_set_scd_addr(stfcamss, change_buf->addr[0],
> +							 change_buf->addr[1], type_scd);
> +				}
> +			}
>  		}
>
>  		stf_isp_reg_set_bit(stfcamss, ISP_REG_CSIINTS,
> @@ -485,6 +542,7 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
>  {
>  	struct stfcamss *stfcamss = priv;
>  	struct stf_capture *cap = &stfcamss->captures[STF_CAPTURE_YUV];
> +	struct stf_capture *cap_scd = &stfcamss->captures[STF_CAPTURE_SCD];
>  	struct stfcamss_buffer *ready_buf;
>  	u32 status;
>
> @@ -496,6 +554,14 @@ irqreturn_t stf_isp_irq_handler(int irq, void *priv)
>  				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
>  		}
>
> +		if (status & ISPC_SC) {
> +			ready_buf = stf_buf_done(&cap_scd->buffers);
> +			if (ready_buf) {
> +				stf_isp_fill_yhist(stfcamss, ready_buf->vaddr);
> +				vb2_buffer_done(&ready_buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +			}
> +		}
> +
>  		stf_isp_reg_write(stfcamss, ISP_REG_ISP_CTRL_0,
>  				  (status & ~ISPC_INT_ALL_MASK) |
>  				  ISPC_ISP | ISPC_CSI | ISPC_SC);
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
> index fcda0502e3b0..0af7b367e57a 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.h
> +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> @@ -10,6 +10,7 @@
>  #ifndef STF_ISP_H
>  #define STF_ISP_H
>
> +#include <linux/jh7110-isp.h>
>  #include <media/v4l2-subdev.h>
>
>  #include "stf-video.h"
> @@ -107,6 +108,12 @@
>  #define Y_COOR(y)				((y) << 16)
>  #define X_COOR(x)				((x) << 0)
>
> +#define ISP_REG_SCD_CFG_0			0x098
> +
> +#define ISP_REG_SC_CFG_1			0x0bc
> +#define ISP_SC_SEL_MASK				GENMASK(31, 30)
> +#define SEL_TYPE(n)				((n) << 30)
> +
>  #define ISP_REG_LCCF_CFG_2			0x0e0
>  #define ISP_REG_LCCF_CFG_3			0x0e4
>  #define ISP_REG_LCCF_CFG_4			0x0e8
> @@ -305,6 +312,10 @@
>  #define DNRM_F(n)				((n) << 16)
>  #define CCM_M_DAT(n)				((n) << 0)
>
> +#define ISP_REG_YHIST_CFG_4			0xcd8
> +
> +#define ISP_REG_YHIST_ACC_0			0xd00
> +
>  #define ISP_REG_GAMMA_VAL0			0xe00
>  #define ISP_REG_GAMMA_VAL1			0xe04
>  #define ISP_REG_GAMMA_VAL2			0xe08
> @@ -389,6 +400,15 @@
>  #define IMAGE_MAX_WIDTH				1920
>  #define IMAGE_MAX_HEIGH				1080
>
> +#define ISP_YHIST_BUFFER_SIZE			(64 * sizeof(__u32))
> +
> +enum stf_isp_type_scd {
> +	TYPE_DEC = 0,
> +	TYPE_OBC,
> +	TYPE_OECF,
> +	TYPE_AWB,
> +};
> +
>  /* pad id for media framework */
>  enum stf_isp_pad_id {
>  	STF_ISP_PAD_SINK = 0,
> @@ -429,5 +449,8 @@ int stf_isp_unregister(struct stf_isp_dev *isp_dev);
>
>  void stf_set_yuv_addr(struct stfcamss *stfcamss,
>  		      dma_addr_t y_addr, dma_addr_t uv_addr);
> +void stf_set_scd_addr(struct stfcamss *stfcamss,
> +		      dma_addr_t yhist_addr, dma_addr_t scd_addr,
> +		      enum stf_isp_type_scd type_scd);
>
>  #endif /* STF_ISP_H */
> diff --git a/drivers/staging/media/starfive/camss/stf-video.c b/drivers/staging/media/starfive/camss/stf-video.c
> index 989b5e82bae9..2203605ec9c7 100644
> --- a/drivers/staging/media/starfive/camss/stf-video.c
> +++ b/drivers/staging/media/starfive/camss/stf-video.c
> @@ -125,6 +125,14 @@ static int stf_video_init_format(struct stfcamss_video *video)
>  	return 0;
>  }
>
> +static int stf_video_scd_init_format(struct stfcamss_video *video)
> +{
> +	video->active_fmt.fmt.meta.dataformat = video->formats[0].pixelformat;
> +	video->active_fmt.fmt.meta.buffersize = sizeof(struct jh7110_isp_sc_buffer);
> +
> +	return 0;
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * Video queue operations
>   */
> @@ -330,6 +338,78 @@ static const struct vb2_ops stf_video_vb2_q_ops = {
>  	.stop_streaming  = video_stop_streaming,
>  };
>
> +static int video_scd_queue_setup(struct vb2_queue *q,
> +				 unsigned int *num_buffers,
> +				 unsigned int *num_planes,
> +				 unsigned int sizes[],
> +				 struct device *alloc_devs[])
> +{
> +	if (*num_planes)
> +		return sizes[0] < sizeof(struct jh7110_isp_sc_buffer) ? -EINVAL : 0;
> +
> +	*num_planes = 1;
> +	sizes[0] = sizeof(struct jh7110_isp_sc_buffer);
> +
> +	return 0;
> +}
> +
> +static int video_scd_buf_init(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct stfcamss_buffer *buffer = to_stfcamss_buffer(vbuf);
> +	dma_addr_t *paddr;
> +
> +	paddr = vb2_plane_cookie(vb, 0);
> +	buffer->addr[0] = *paddr;
> +	buffer->addr[1] = buffer->addr[0] + ISP_YHIST_BUFFER_SIZE;
> +	buffer->vaddr = vb2_plane_vaddr(vb, 0);
> +
> +	return 0;
> +}
> +
> +static int video_scd_buf_prepare(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +
> +	if (sizeof(struct jh7110_isp_sc_buffer) > vb2_plane_size(vb, 0))
> +		return -EINVAL;
> +
> +	vb2_set_plane_payload(vb, 0, sizeof(struct jh7110_isp_sc_buffer));
> +
> +	vbuf->field = V4L2_FIELD_NONE;
> +
> +	return 0;
> +}
> +
> +static int video_scd_start_streaming(struct vb2_queue *q, unsigned int count)
> +{
> +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> +
> +	video->ops->start_streaming(video);
> +
> +	return 0;
> +}
> +
> +static void video_scd_stop_streaming(struct vb2_queue *q)
> +{
> +	struct stfcamss_video *video = vb2_get_drv_priv(q);
> +
> +	video->ops->stop_streaming(video);
> +
> +	video->ops->flush_buffers(video, VB2_BUF_STATE_ERROR);
> +}
> +
> +static const struct vb2_ops stf_video_scd_vb2_q_ops = {
> +	.queue_setup     = video_scd_queue_setup,
> +	.wait_prepare    = vb2_ops_wait_prepare,
> +	.wait_finish     = vb2_ops_wait_finish,
> +	.buf_init        = video_scd_buf_init,
> +	.buf_prepare     = video_scd_buf_prepare,
> +	.buf_queue       = video_buf_queue,
> +	.start_streaming = video_scd_start_streaming,
> +	.stop_streaming  = video_scd_stop_streaming,
> +};
> +
>  /* -----------------------------------------------------------------------------
>   * V4L2 ioctls
>   */
> @@ -448,6 +528,37 @@ static const struct v4l2_ioctl_ops stf_vid_ioctl_ops = {
>  	.vidioc_streamoff               = vb2_ioctl_streamoff,
>  };
>
> +static int video_scd_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
> +{
> +	struct stfcamss_video *video = video_drvdata(file);
> +	struct v4l2_meta_format *meta = &f->fmt.meta;
> +
> +	if (f->type != video->type)
> +		return -EINVAL;
> +
> +	meta->dataformat = video->active_fmt.fmt.meta.dataformat;
> +	meta->buffersize = video->active_fmt.fmt.meta.buffersize;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops stf_vid_scd_ioctl_ops = {
> +	.vidioc_querycap                = video_querycap,
> +	.vidioc_enum_fmt_meta_cap       = video_enum_fmt,
> +	.vidioc_g_fmt_meta_cap          = video_scd_g_fmt,
> +	.vidioc_s_fmt_meta_cap          = video_scd_g_fmt,
> +	.vidioc_try_fmt_meta_cap        = video_scd_g_fmt,
> +	.vidioc_reqbufs                 = vb2_ioctl_reqbufs,
> +	.vidioc_querybuf                = vb2_ioctl_querybuf,
> +	.vidioc_qbuf                    = vb2_ioctl_qbuf,
> +	.vidioc_expbuf                  = vb2_ioctl_expbuf,
> +	.vidioc_dqbuf                   = vb2_ioctl_dqbuf,
> +	.vidioc_create_bufs             = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf             = vb2_ioctl_prepare_buf,
> +	.vidioc_streamon                = vb2_ioctl_streamon,
> +	.vidioc_streamoff               = vb2_ioctl_streamoff,
> +};
> +
>  /* -----------------------------------------------------------------------------
>   * V4L2 file operations
>   */
> @@ -473,6 +584,9 @@ static int stf_link_validate(struct media_link *link)
>  	struct stfcamss_video *video = video_get_drvdata(vdev);
>  	int ret;
>
> +	if (video->type == V4L2_BUF_TYPE_META_CAPTURE)
> +		return 0;
> +
>  	ret = stf_video_check_format(video);
>
>  	return ret;
> @@ -506,7 +620,11 @@ int stf_video_register(struct stfcamss_video *video,
>  	q = &video->vb2_q;
>  	q->drv_priv = video;
>  	q->mem_ops = &vb2_dma_contig_memops;
> -	q->ops = &stf_video_vb2_q_ops;
> +
> +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> +		q->ops = &stf_video_vb2_q_ops;
> +	else
> +		q->ops = &stf_video_scd_vb2_q_ops;
>  	q->type = video->type;
>  	q->io_modes = VB2_DMABUF | VB2_MMAP;
>  	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> @@ -529,16 +647,28 @@ int stf_video_register(struct stfcamss_video *video,
>  		goto err_mutex_destroy;
>  	}
>
> -	ret = stf_video_init_format(video);
> -	if (ret < 0) {
> -		dev_err(video->stfcamss->dev,
> -			"Failed to init format: %d\n", ret);
> -		goto err_media_cleanup;
> +	if (video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
> +		ret = stf_video_init_format(video);
> +		if (ret < 0) {
> +			dev_err(video->stfcamss->dev,
> +				"Failed to init format: %d\n", ret);
> +			goto err_media_cleanup;
> +		}
> +		vdev->ioctl_ops = &stf_vid_ioctl_ops;
> +		vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE;
> +	} else {
> +		ret = stf_video_scd_init_format(video);
> +		if (ret < 0) {
> +			dev_err(video->stfcamss->dev,
> +				"Failed to init format: %d\n", ret);
> +			goto err_media_cleanup;
> +		}
> +		vdev->ioctl_ops = &stf_vid_scd_ioctl_ops;
> +		vdev->device_caps = V4L2_CAP_META_CAPTURE;
>  	}
>
>  	vdev->fops = &stf_vid_fops;
> -	vdev->ioctl_ops = &stf_vid_ioctl_ops;
> -	vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
> +	vdev->device_caps |= V4L2_CAP_STREAMING;
>  	vdev->entity.ops = &stf_media_ops;
>  	vdev->vfl_dir = VFL_DIR_RX;
>  	vdev->release = stf_video_release;
> diff --git a/drivers/staging/media/starfive/camss/stf-video.h b/drivers/staging/media/starfive/camss/stf-video.h
> index 59799b65cbe5..53a1cf4e59b7 100644
> --- a/drivers/staging/media/starfive/camss/stf-video.h
> +++ b/drivers/staging/media/starfive/camss/stf-video.h
> @@ -37,6 +37,7 @@ enum stf_v_line_id {
>  enum stf_capture_type {
>  	STF_CAPTURE_RAW = 0,
>  	STF_CAPTURE_YUV,
> +	STF_CAPTURE_SCD,
>  	STF_CAPTURE_NUM,
>  };
>
> --
> 2.25.1
>
>

回复: [PATCH v5 08/14] staging: media: starfive: Add for StarFive ISP 3A SC

[Changhuang Liang]

Hi Jacopo

> 
> Hi Changhuang
> 
> Sorry, I missed this: the subject
> 
>     staging: media: starfive: Add for StarFive ISP 3A SC
> 
> Seems to be missing a word.
> 
>     staging: media: starfive: Add stats for StarFive ISP 3A
> 
> maybe ?
> 

Yes, thank you for your reminder.

Regards,
Changhuang

Re: [PATCH v5 11/14] staging: media: starfive: Add ISP params video device

[Hans Verkuil]

On 10/07/2024 15:07, Jacopo Mondi wrote:
> Hi Changhuang
> 
>    + Hans for one question on the vb2 queue mem_ops to use.
> 
> On Tue, Jul 09, 2024 at 01:38:21AM GMT, Changhuang Liang wrote:
>> Add ISP params video device to write ISP parameters for 3A.
>>
>> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
>> ---
>>  drivers/staging/media/starfive/camss/Makefile |   2 +
>>  .../staging/media/starfive/camss/stf-camss.c  |  23 +-
>>  .../staging/media/starfive/camss/stf-camss.h  |   3 +
>>  .../media/starfive/camss/stf-isp-params.c     | 240 ++++++++++++++++++
>>  .../staging/media/starfive/camss/stf-isp.h    |   4 +
>>  .../staging/media/starfive/camss/stf-output.c |  83 ++++++
>>  .../staging/media/starfive/camss/stf-output.h |  22 ++
>>  7 files changed, 376 insertions(+), 1 deletion(-)
>>  create mode 100644 drivers/staging/media/starfive/camss/stf-isp-params.c
>>  create mode 100644 drivers/staging/media/starfive/camss/stf-output.c
>>  create mode 100644 drivers/staging/media/starfive/camss/stf-output.h
>>

<snip>

>> +int stf_isp_params_register(struct stfcamss_video *video,
>> +			    struct v4l2_device *v4l2_dev,
>> +			    const char *name)
>> +{
>> +	struct video_device *vdev = &video->vdev;
>> +	struct vb2_queue *q;
>> +	struct media_pad *pad = &video->pad;
>> +	int ret;
>> +
>> +	mutex_init(&video->q_lock);
>> +	mutex_init(&video->lock);
> 
> are two mutexes required for the vb2 queue and the video node ? I see,
> in example, rkisp1-params.c uses a single one
> 
>> +
>> +	q = &video->vb2_q;
>> +	q->drv_priv = video;
>> +	q->mem_ops = &vb2_dma_contig_memops;
> 
> Now, I might be wrong, but unless you need to allocate memory from a
> DMA-capable area, you shouldn't need to use vb2_dma_contig_memops.
> 
> Looking at the next patches you apply configuration parameters to the
> ISP by inspecting the user supplied parameters one by one, not by
> transfering the whole parameters buffer to a memory area. Hans what do
> you think ?

Yes, if the data is not DMAed to the hardware, then use vb2_vmalloc_memops.

Regards,

	Hans

回复: [PATCH v5 02/14] media: Documentation: Add description for StarFive ISP metadata formats

[Changhuang Liang]

Hi, Jacopo

> 
> Hi Changhuang
> 
> On Tue, Jul 09, 2024 at 01:38:12AM GMT, Changhuang Liang wrote:
> > Add description for V4L2_META_FMT_STF_ISP_PARAMS and
> > V4L2_META_FMT_STF_ISP_STAT_3A meta data formats.
> >
> > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> 
> I get this warnings when compiling documentation
> 

Can you share the steps about compiling documentation?

Regards,
Changhuang

Re: 回复: [PATCH v5 02/14] media: Documentation: Add description for StarFive ISP metadata formats

[Jacopo Mondi]

Hi Changhuang

On Tue, Jul 16, 2024 at 12:31:31PM GMT, Changhuang Liang wrote:
> Hi, Jacopo
>
> >
> > Hi Changhuang
> >
> > On Tue, Jul 09, 2024 at 01:38:12AM GMT, Changhuang Liang wrote:
> > > Add description for V4L2_META_FMT_STF_ISP_PARAMS and
> > > V4L2_META_FMT_STF_ISP_STAT_3A meta data formats.
> > >
> > > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> >
> > I get this warnings when compiling documentation
> >
>
> Can you share the steps about compiling documentation?
>

From `make help`

-------------------------------------------------------------------------------
Documentation targets:
 Linux kernel internal documentation in different formats from ReST:
  htmldocs        - HTML
  texinfodocs     - Texinfo
  infodocs        - Info
  latexdocs       - LaTeX
  pdfdocs         - PDF
  epubdocs        - EPUB
  xmldocs         - XML
  linkcheckdocs   - check for broken external links
                    (will connect to external hosts)
  refcheckdocs    - check for references to non-existing files under
                    Documentation
  cleandocs       - clean all generated files

  make SPHINXDIRS="s1 s2" [target] Generate only docs of folder s1, s2
  valid values for SPHINXDIRS are: PCI RCU accel accounting admin-guide arch block bpf cdrom core-api cpu-freq crypto dev-tools devicetree doc-guide driver-api fault-injection fb filesystems firmware-guide fpga gpu hid hwmon i2c iio infiniband input isdn kbuild kernel-hacking leds livepatch locking maintainer mhi misc-devices mm netlabel networking pcmcia peci power powerpc process riscv rust scheduler scsi security sound spi staging target tee timers tools trace translations usb userspace-api virt w1 watchdog wmi
-------------------------------------------------------------------------------

To compile, in example, the userspace documentation in html:

`make $your-usual-args-here SPHINXDIRS="userspace-api" htmldocs`


> Regards,
> Changhuang

回复: 回复: [PATCH v5 02/14] media: Documentation: Add description for StarFive ISP metadata formats

[Changhuang Liang]

Hi, Jacopo

> 
> Hi Changhuang
> 
> On Tue, Jul 16, 2024 at 12:31:31PM GMT, Changhuang Liang wrote:
> > Hi, Jacopo
> >
> > >
> > > Hi Changhuang
> > >
> > > On Tue, Jul 09, 2024 at 01:38:12AM GMT, Changhuang Liang wrote:
> > > > Add description for V4L2_META_FMT_STF_ISP_PARAMS and
> > > > V4L2_META_FMT_STF_ISP_STAT_3A meta data formats.
> > > >
> > > > Signed-off-by: Changhuang Liang
> > > > <changhuang.liang@starfivetech.com>
> > >
> > > I get this warnings when compiling documentation
> > >
> >
> > Can you share the steps about compiling documentation?
> >
> 
> From `make help`
> 
> -------------------------------------------------------------------------------
> Documentation targets:
>  Linux kernel internal documentation in different formats from ReST:
>   htmldocs        - HTML
>   texinfodocs     - Texinfo
>   infodocs        - Info
>   latexdocs       - LaTeX
>   pdfdocs         - PDF
>   epubdocs        - EPUB
>   xmldocs         - XML
>   linkcheckdocs   - check for broken external links
>                     (will connect to external hosts)
>   refcheckdocs    - check for references to non-existing files under
>                     Documentation
>   cleandocs       - clean all generated files
> 
>   make SPHINXDIRS="s1 s2" [target] Generate only docs of folder s1, s2
>   valid values for SPHINXDIRS are: PCI RCU accel accounting admin-guide
> arch block bpf cdrom core-api cpu-freq crypto dev-tools devicetree doc-guide
> driver-api fault-injection fb filesystems firmware-guide fpga gpu hid hwmon
> i2c iio infiniband input isdn kbuild kernel-hacking leds livepatch locking
> maintainer mhi misc-devices mm netlabel networking pcmcia peci power
> powerpc process riscv rust scheduler scsi security sound spi staging target tee
> timers tools trace translations usb userspace-api virt w1 watchdog wmi
> -------------------------------------------------------------------------------
> 
> To compile, in example, the userspace documentation in html:
> 
> `make $your-usual-args-here SPHINXDIRS="userspace-api" htmldocs`
> 

What should $your-usual-args-here fill? Can you share your specific commands?

This is the result of my side run here:

# make SPHINXDIRS="userspace-api" htmldocs
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/rt_link.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/rt_link.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/netdev.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/netdev.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ovs_vport.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ovs_vport.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/handshake.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/handshake.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/fou.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/fou.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/nftables.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/nftables.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ovs_datapath.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ovs_datapath.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/nfsd.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/nfsd.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/dpll.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/dpll.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/nlctrl.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/nlctrl.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ethtool.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ethtool.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/mptcp_pm.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/mptcp_pm.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/rt_route.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/rt_route.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/team.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/team.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/devlink.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/devlink.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/rt_addr.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/rt_addr.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ovs_flow.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ovs_flow.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/tc.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/tc.rst
/home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/index.rst -x
make: *** empty variable name.  Stop.
make: *** [Makefile:122: htmldocs] Error 2

Is there any special environment configuration present?

Regards,
Changhuang

Re: 回复: 回复: [PATCH v5 02/14] media: Documentation: Add description for StarFive ISP metadata formats

[Jacopo Mondi]

Hi Changhuang

On Wed, Jul 17, 2024 at 02:37:00AM GMT, Changhuang Liang wrote:
> Hi, Jacopo
>
> >
> > Hi Changhuang
> >
> > On Tue, Jul 16, 2024 at 12:31:31PM GMT, Changhuang Liang wrote:
> > > Hi, Jacopo
> > >
> > > >
> > > > Hi Changhuang
> > > >
> > > > On Tue, Jul 09, 2024 at 01:38:12AM GMT, Changhuang Liang wrote:
> > > > > Add description for V4L2_META_FMT_STF_ISP_PARAMS and
> > > > > V4L2_META_FMT_STF_ISP_STAT_3A meta data formats.
> > > > >
> > > > > Signed-off-by: Changhuang Liang
> > > > > <changhuang.liang@starfivetech.com>
> > > >
> > > > I get this warnings when compiling documentation
> > > >
> > >
> > > Can you share the steps about compiling documentation?
> > >
> >
> > From `make help`
> >
> > -------------------------------------------------------------------------------
> > Documentation targets:
> >  Linux kernel internal documentation in different formats from ReST:
> >   htmldocs        - HTML
> >   texinfodocs     - Texinfo
> >   infodocs        - Info
> >   latexdocs       - LaTeX
> >   pdfdocs         - PDF
> >   epubdocs        - EPUB
> >   xmldocs         - XML
> >   linkcheckdocs   - check for broken external links
> >                     (will connect to external hosts)
> >   refcheckdocs    - check for references to non-existing files under
> >                     Documentation
> >   cleandocs       - clean all generated files
> >
> >   make SPHINXDIRS="s1 s2" [target] Generate only docs of folder s1, s2
> >   valid values for SPHINXDIRS are: PCI RCU accel accounting admin-guide
> > arch block bpf cdrom core-api cpu-freq crypto dev-tools devicetree doc-guide
> > driver-api fault-injection fb filesystems firmware-guide fpga gpu hid hwmon
> > i2c iio infiniband input isdn kbuild kernel-hacking leds livepatch locking
> > maintainer mhi misc-devices mm netlabel networking pcmcia peci power
> > powerpc process riscv rust scheduler scsi security sound spi staging target tee
> > timers tools trace translations usb userspace-api virt w1 watchdog wmi
> > -------------------------------------------------------------------------------
> >
> > To compile, in example, the userspace documentation in html:
> >
> > `make $your-usual-args-here SPHINXDIRS="userspace-api" htmldocs`
> >
>
> What should $your-usual-args-here fill? Can you share your specific commands?

make -j24 ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- O=./linux-build SPHINXDIRS="userspace-api" htmldocs

Make sure to adapt it to your environment

>
> This is the result of my side run here:
>
> # make SPHINXDIRS="userspace-api" htmldocs
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/rt_link.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/rt_link.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/netdev.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/netdev.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ovs_vport.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ovs_vport.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/handshake.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/handshake.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/fou.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/fou.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/nftables.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/nftables.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ovs_datapath.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ovs_datapath.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/nfsd.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/nfsd.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/dpll.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/dpll.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/nlctrl.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/nlctrl.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ethtool.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ethtool.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/mptcp_pm.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/mptcp_pm.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/rt_route.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/rt_route.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/team.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/team.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/devlink.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/devlink.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/rt_addr.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/rt_addr.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/ovs_flow.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/ovs_flow.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -i /home/huang/workspace/upstream/linux/Documentation/netlink/specs/tc.yaml -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/tc.rst
> /home/huang/workspace/upstream/linux/tools/net/ynl/ynl-gen-rst.py -o /home/huang/workspace/upstream/linux/Documentation/networking/netlink_spec/index.rst -x
> make: *** empty variable name.  Stop.
> make: *** [Makefile:122: htmldocs] Error 2
>
> Is there any special environment configuration present?
>
> Regards,
> Changhuang

Re: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Laurent Pinchart]

On Wed, Jul 10, 2024 at 11:11:57AM +0200, Jacopo Mondi wrote:
> On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> > Add JH7110 ISP module definitions for user space.
> >
> > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> > ---
> >  MAINTAINERS                     |   1 +
> >  include/uapi/linux/jh7110-isp.h | 739 ++++++++++++++++++++++++++++++++
> 
> With the recently merged support for the RaspberryPi PiSP BE we have
> introduced include/uapi/linux/media/raspberry.
> 
> Would you consider placing this in
> include/uapi/linux/media/startfive/ ?

That sounds like a good idea.

> >  2 files changed, 740 insertions(+)
> >  create mode 100644 include/uapi/linux/jh7110-isp.h
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 507f04a80499..890604eb0d64 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -21305,6 +21305,7 @@ S:	Maintained
> >  F:	Documentation/admin-guide/media/starfive_camss.rst
> >  F:	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> >  F:	drivers/staging/media/starfive/camss
> > +F:	include/uapi/linux/jh7110-isp.h
> >
> >  STARFIVE CRYPTO DRIVER
> >  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> > diff --git a/include/uapi/linux/jh7110-isp.h b/include/uapi/linux/jh7110-isp.h
> > new file mode 100644
> > index 000000000000..4939cd63e771
> > --- /dev/null
> > +++ b/include/uapi/linux/jh7110-isp.h
> > @@ -0,0 +1,739 @@
> > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> > +/*
> > + * jh7110-isp.h
> > + *
> > + * JH7110 ISP driver - user space header file.
> > + *
> > + * Copyright © 2023 StarFive Technology Co., Ltd.
> > + *
> > + * Author: Zejian Su <zejian.su@starfivetech.com>
> > + *
> > + */
> > +
> > +#ifndef __JH7110_ISP_H_
> > +#define __JH7110_ISP_H_
> > +
> 
> Do you need to include
> 
> #include <linux/types.h>
> 
> > +/**
> 
> Is this kernel-doc or a single * would do ?
> 
> > + * ISP Module Diagram
> > + * ------------------
> > + *
> > + *  Raw  +-----+    +------+    +------+    +----+
> > + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> > + *       +-----+    +------+    +------+    +----+     |
> > + *                                                     |
> > + *  +--------------------------------------------------+
> > + *  |
> > + *  |    +-----+    +-----+    +-----+    +-----+
> > + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> > + *       +-----+    +-----+    +-----+    +-----+      |
> > + *                                                     |
> > + *  +--------------------------------------------------+
> > + *  |
> > + *  |    +-----+    +--------+    +-----+    +------+
> > + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> > + *       +-----+    +--------+    +-----+    +------+  |
> > + *                                                     |
> > + *  +--------------------------------------------------+
> > + *  |
> > + *  |    +-------+    +-------+    +-----+    +----+
> > + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> > + *       +-------+    +-------+    +-----+    +----+
> > + *

The diagram is useful, thank you. A glossary would also be nice, maybe
as

 * - OBC: Optical Black Correction
 * - OECF: Opto-Electric Conversion Function
 * ...

I think that would be easier to read than the comments above each macro
below. Up to you.

> > + */
> > +
> > +/* Auto White Balance */
> > +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> > +/* Color Artifact Removal */
> > +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> > +/* Color Correction Matrix */
> > +#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
> > +/* Color Filter Arrays */
> > +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> > +/* Crosstalk Correction */
> > +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> > +/* Defect Bad Pixel Correction */
> > +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> > +/* Denoise YUV */
> > +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
> > +/* RGB Gamma */
> > +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
> > +/* Lens Correction Cosine Fourth */
> > +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
> > +/* Optical Black Correction */
> > +#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
> > +/* Opto-Electric Conversion Function */
> > +#define JH7110_ISP_MODULE_OECF_SETTING			(1U << 10)
> > +/* RGB To YUV */
> > +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U << 11)
> > +/* Saturation */
> > +#define JH7110_ISP_MODULE_SAT_SETTING			(1U << 12)
> > +/* Sharpen */
> > +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U << 13)
> > +/* Y Curve */
> > +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U << 14)
> > +/* Statistics Collection */
> > +#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)

Unless there's a specific reason to keep the current order, maybe you
could sort those macros in the same order as in the module diagram ?

> > +
> > +/**
> > + * struct jh7110_isp_wb_gain - auto white balance gain
> > + *
> > + * @gain_r: gain value for red component.
> > + * @gain_g: gain value for green component.
> > + * @gain_b: gain value for blue component.

I suppose the gains are expressed as fixed-point integers. This needs
more details, what are the limits, and how many bits of integer and
fractional parts are there ?

Same comment for all the other values below.

> > + */
> > +struct jh7110_isp_wb_gain {
> > +	__u16 gain_r;
> > +	__u16 gain_g;
> > +	__u16 gain_b;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_wb_setting - Configuration used by auto white balance gain
> > + *
> > + * @enabled: enabled setting flag.
> > + * @gains: auto white balance gain setting.
> > + */
> > +struct jh7110_isp_wb_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_wb_gain gains;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_car_setting - Configuration used by color artifact removal
> > + *
> > + * @enabled: enabled setting flag.
> > + */
> > +struct jh7110_isp_car_setting {
> > +	__u32 enabled;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_ccm_smlow - Color correction matrix
> > + *
> > + * @ccm: color transform matrix with size 3 by 3.
> > + * @offsets: the offsets of R, G, B after the transform by the ccm.
> > + */
> > +struct jh7110_isp_ccm_smlow {
> > +	__s32 ccm[3][3];
> > +	__s32 offsets[3];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_ccm_setting - Configuration used by color correction matrix
> > + *
> > + * @enabled: enabled setting flag.
> > + * @ccm_smlow: Color correction matrix.
> > + */
> > +struct jh7110_isp_ccm_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_ccm_smlow ccm_smlow;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_cfa_params - demosaic parameters
> > + *
> > + * @hv_width: detail smooth factor
> > + * @cross_cov: Cross covariance weighting.

This documentation doesn't tell how to use those paraemeters. This
comment applies to many other parameters below. There are three main
options to improve that:

- Expanding the documentation in this header file to clearly explain how
  to compute the parameters values.

- Providing an open userspace implementation of ISP algorithms that
  showcase how to calculate the values.

- Providing detailed hardware documentation for the ISP. This is usually
  not favoured by ISP vendors, and we are not pushing for this, but I
  wanted to mention it for completeness.

If you would prefer the second option, any open-source camera framework
would be acceptable, but in practice the only real option is likely
libcamera.

This does not mean that you have to open-source all your ISP control
algorithms. Only the code needed to explain how ISP parameters are
applied to the image and are computed is needed. Other parts, such as
for instance AI-based computation of white balance gains, or complex AGC
calculations, don't need to be open-source.

The explain this requirement different and perhaps more clearly, the
goal is to make sure that developers who have access to only open-source
code (ISP kernel driver, this header, any open-source userspace code,
...) will have enough information to configure and control the ISP.

> > + */
> > +struct jh7110_isp_cfa_params {
> > +	__s32 hv_width;
> > +	__s32 cross_cov;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_cfa_params - Configuration used by demosaic module
> > + *
> > + * @enabled: enabled setting flag.
> > + * @settings: demosaic parameters.
> > + */
> > +struct jh7110_isp_cfa_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_cfa_params settings;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_ctc_params - crosstalk remove parameters
> > + *
> > + * @saf_mode: smooth area filter mode.
> > + * @daf_mode: detail area filter mode.
> > + * @max_gt: the threshold for imbalance detection when pixel intensity is close to maximum.
> 
> You could easily make this < 80 cols (here and in other places).
> 
> I know there are different opinions on how strict on the 80 cols limit
> we should be, so up to you.
> 
> > + * @min_gt: the threshold for imbalance detection when pixel intensity is close to 0.
> > + */
> > +struct jh7110_isp_ctc_params {
> > +	__u8 saf_mode;
> > +	__u8 daf_mode;
> > +	__s32 max_gt;
> > +	__s32 min_gt;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_ctc_params - Configuration used by crosstalk remove
> > + *
> > + * @enabled: enabled setting flag.
> > + * @settings: corsstalk remove parameters.
> 
> crosstalk
> 
> > + */
> > +struct jh7110_isp_ctc_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_ctc_params settings;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_dbc_params - defect pixels correction parameters
> > + *
> > + * @bad_gt: bad pixel threshold for the green channel.
> > + * @bad_xt: bad pixel threshold for the red and blue channels.
> > + */
> > +struct jh7110_isp_dbc_params {
> > +	__s32 bad_gt;
> > +	__s32 bad_xt;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_dbc_params - Configuration used by defect bad pixels correction
> > + *
> > + * @enabled: enabled setting flag.
> > + * @settings: defect pixels correction parameters.
> > + */
> > +struct jh7110_isp_dbc_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_dbc_params settings;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_dnyuv_params - yuv domain denoise parameters
> > + *
> > + * @y_sweight: ten coefficients of 7x7 spatial filter for Y channel.
> > + * @y_curve: intensity difference (similarity) weight lookup table for Y channel.
> > + * @uv_sweight: ten coefficients of 7x7 spatial filter for U and V channel.
> > + * @uv_curve: intensity difference (similarity) weight lookup table for U and V channel.
> > + */
> > +struct jh7110_isp_dnyuv_params {
> > +	__u8 y_sweight[10];
> > +	__u16 y_curve[7];
> > +	__u8 uv_sweight[10];
> > +	__u16 uv_curve[7];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_dnyuv_params - Configuration used by yuv domain denoise
> > + *
> > + * @enabled: enabled setting flag.
> > + * @settings: yuv domain denoise parameters.
> > + */
> > +struct jh7110_isp_dnyuv_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_dnyuv_params settings;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_gmargb_point - RGB Gamma point
> > + *
> > + * @g_val: RGB gamma value.
> > + * @sg_val: RGB gamma slope value.
> > + */
> > +struct jh7110_isp_gmargb_point {
> > +	__u16 g_val;
> > +	__u16 sg_val;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_gmargb_setting - Configuration used by RGB gamma
> > + *
> > + * @enabled: enabled setting flag.
> > + * @curve: RGB Gamma point table.
> > + */
> > +struct jh7110_isp_gmargb_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_gmargb_point curve[15];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_lccf_circle - len circle
> 
> lens ?
> 
> > + *
> > + * @center_x: center X distance from capture window.
> > + * @center_y: center Y distance from capture window.
> > + * @radius: len circle radius.
> 
> here as well
> 
> > + */
> > +struct jh7110_isp_lccf_circle {
> > +	__s16 center_x;
> > +	__s16 center_y;
> > +	__u8 radius;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_lccf_curve_param - lens correction cosine fourth curve param
> > + *
> > + * @f1: F1 parameter.
> > + * @f2: F2 parameter.
> > + */
> > +struct jh7110_isp_lccf_curve_param {
> > +	__s16 f1;
> > +	__s16 f2;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_lccf_setting - Configuration used by lens correction cosine fourth
> > + *
> > + * @enabled: enabled setting flag.
> > + * @circle: len circle.
> > + * @r_param: lens correction cosine fourth curve param for Bayer pattern R.
> > + * @gr_param: lens correction cosine fourth curve param for Bayer pattern Gr.
> > + * @gb_param: lens correction cosine fourth curve param for Bayer pattern Gb.
> > + * @b_param: lens correction cosine fourth curve param for Bayer pattern B.
> > + */
> > +struct jh7110_isp_lccf_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_lccf_circle circle;
> > +	struct jh7110_isp_lccf_curve_param r_param;
> > +	struct jh7110_isp_lccf_curve_param gr_param;
> > +	struct jh7110_isp_lccf_curve_param gb_param;
> > +	struct jh7110_isp_lccf_curve_param b_param;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_obc_win_size - optical black correction window size
> > + *
> > + * @width: window width.
> > + * @height: window height.
> > + */
> > +struct jh7110_isp_obc_win_size {
> > +	__u32 width;
> > +	__u32 height;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_obc_gain - optical black correction symbol gain
> > + *
> > + * @tl_gain: gain at point A for symbol.
> > + * @tr_gain: gain at point B for symbol.
> > + * @bl_gain: gain at point C for symbol.
> > + * @br_gain: gain at point D for symbol.
> > + */
> > +struct jh7110_isp_obc_gain {
> > +	__u8 tl_gain;
> > +	__u8 tr_gain;
> > +	__u8 bl_gain;
> > +	__u8 br_gain;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_obc_offset - optical black correction symbol offset
> > + *
> > + * @tl_offset: offset at point A for symbol.
> > + * @tr_offset: offset at point B for symbol.
> > + * @bl_offset: offset at point C for symbol.
> > + * @br_offset: offset at point D for symbol.
> > + */
> > +struct jh7110_isp_obc_offset {
> > +	__u8 tl_offset;
> > +	__u8 tr_offset;
> > +	__u8 bl_offset;
> > +	__u8 br_offset;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_obc_setting - Configuration used by optical black correction
> > + *
> > + * @enabled: enabled setting flag.
> > + * @win_size: optical black correction window size.
> > + * @gain: optical black correction symbol gain.
> > + * @offset: optical black correction symbol offset.
> > + */
> > +struct jh7110_isp_obc_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_obc_win_size win_size;
> > +	struct jh7110_isp_obc_gain gain[4];
> > +	struct jh7110_isp_obc_offset offset[4];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_oecf_point - oecf curve
> > + *
> > + * @x: x coordinate.
> > + * @y: y coordinate.
> > + * @slope: the slope between this point and the next point.
> > + */
> > +struct jh7110_isp_oecf_point {
> > +	__u16 x;
> > +	__u16 y;
> > +	__s16 slope;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_oecf_setting - Configuration used by opto-electric conversion function
> > + *
> > + * @enabled: enabled setting flag.
> > + * @r_curve: red pixel oecf curve.
> > + * @gr_curve: green pixel oecf curve in GR line.
> > + * @gb_curve: green pixel oecf curve in GB line.
> > + * @b_curve: blue pixel oecf curve.
> > + */
> > +struct jh7110_isp_oecf_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_oecf_point r_curve[16];
> > +	struct jh7110_isp_oecf_point gr_curve[16];
> > +	struct jh7110_isp_oecf_point gb_curve[16];
> > +	struct jh7110_isp_oecf_point b_curve[16];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_r2y_matrix - RGB to YUV color conversion matrix
> > + *
> > + * @m: The 3x3 color conversion matrix coefficient.
> > + */
> > +struct jh7110_isp_r2y_matrix {
> > +	__s16 m[9];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_r2y_setting - Configuration used by RGB To YUV
> > + *
> > + * @enabled: enabled setting flag.
> > + * @matrix: RGB to YUV color conversion matrix.
> > + */
> > +struct jh7110_isp_r2y_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_r2y_matrix matrix;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sat_curve - Saturation curve
> > + *
> > + * @yi_min: the minimum input Y value.
> > + * @yo_ir: the ratio of Y output range to input range.
> > + * @yo_min: the minimum output Y value.
> > + * @yo_max: the maximum output Y value.
> > + */
> > +struct jh7110_isp_sat_curve {
> > +	__s16 yi_min;
> > +	__s16 yo_ir;
> > +	__s16 yo_min;
> > +	__s16 yo_max;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sat_hue_info - Chroma Saturation Hue Factor
> > + *
> > + * @cos: COS hue factor.
> > + * @sin: SIN hue factor.
> > + */
> > +struct jh7110_isp_sat_hue_info {
> > +	__s16 cos;
> > +	__s16 sin;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sat_info - Saturation information
> > + *
> > + * @gain_cmab: Chroma saturation magnitude amplification base for gain.
> > + * @gain_cmmd: Chroma saturation magnitude amplification delta for gain.
> > + * @threshold_cmb: Chroma saturation magnitude base threshold.
> > + * @threshold_cmd: Chroma saturation magnitude delta threshold.
> > + * @offset_u: Chroma saturation U offset.
> > + * @offset_v: Chroma saturation V offset.
> > + * @cmsf: Chroma saturation magnitude scaling factor.
> > + */
> > +struct jh7110_isp_sat_info {
> > +	__s16 gain_cmab;
> > +	__s16 gain_cmmd;
> > +	__s16 threshold_cmb;
> > +	__s16 threshold_cmd;
> > +	__s16 offset_u;
> > +	__s16 offset_v;
> > +	__s16 cmsf;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sat_setting - Configuration used by Saturation
> > + *
> > + * @enabled: enabled setting flag.
> > + * @curve: Saturation curve.
> > + * @hue_info: Chroma Saturation Hue Factor.
> > + * @sat_info: Saturation information.s
> 
> informations.
> 
> > + */
> > +struct jh7110_isp_sat_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_sat_curve curve;
> > +	struct jh7110_isp_sat_hue_info hue_info;
> > +	struct jh7110_isp_sat_info sat_info;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sharp_weight - Sharpe weight
> > + *
> > + * @weight: Sharpen filter weight.
> > + * @recip_wei_sum: Sharpen amplification filter weight normalization factor.
> > + */
> > +struct jh7110_isp_sharp_weight {
> > +	__u8 weight[15];
> > +	__u32 recip_wei_sum;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sharp_strength - Sharpen strength
> > + *
> > + * @diff: Sharpen Edge amplification delta level.
> > + * @f: Sharpen Edge amplification factor.
> > + * @s: Sharpen Edge amplification factor slope.
> > + */
> > +struct jh7110_isp_sharp_strength {
> > +	__s16 diff[4];
> > +	__s16 f[3];
> > +	__s32 s[3];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sharp_setting - Configuration used by Sharpen
> > + *
> > + * @enabled: enabled setting flag.
> > + * @weight: Sharpe weight.
> > + * @strength: Sharpen strength.
> > + * @pdirf: Positive Factor Multiplier.
> > + * @ndirf: Negative Factor Multiplier.
> > + */
> > +struct jh7110_isp_sharp_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_sharp_weight weight;
> > +	struct jh7110_isp_sharp_strength strength;
> > +	__s8 pdirf;
> > +	__s8 ndirf;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_ycrv_curve - Y Curve parameters table
> > + *
> > + * @y: Y curve L parameters value.
> > + */
> > +struct jh7110_isp_ycrv_curve {
> > +	__s16 y[64];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_ycrv_setting - Configuration used by Y Curve
> > + *
> > + * @enabled: enabled setting flag.
> > + * @curve: Y Curve parameters table.
> > + */
> > +struct jh7110_isp_ycrv_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_ycrv_curve curve;
> 
> I am a bit failing in seeing the point of embedding the settings in a
> dedicated structure when you have a single instance of the
> configuration like this and in other instances. Isn't
> 
>         struct jh7110_isp_ycrv_setting {
>                 __u32 enabled;
>                 __s16 y[64];
>         };
> 
> easier ? Or do you need a dedicated type for other reasons ?
> 
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sc_config - statistics collection crop configure
> > + *
> > + * @h_start: Horizontal starting point for frame cropping.
> > + * @v_start: Vertical starting point for frame cropping.
> > + * @sw_width: Width of statistics collection sub-window.
> > + * @sw_height: Height of statistics collection sub-window.
> > + * @hperiod: Horizontal period.
> > + * @hkeep: Horizontal keep.
> > + * @vperiod: Vertical period.
> > + * @vkeep: Vertical keep.
> > + */
> > +struct jh7110_isp_sc_config {
> > +	__u16 h_start;
> > +	__u16 v_start;
> > +	__u8 sw_width;
> > +	__u8 sw_height;
> > +	__u8 hperiod;
> > +	__u8 hkeep;
> > +	__u8 vperiod;
> > +	__u8 vkeep;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sc_af_config - statistics collection auto focus configure
> > + *
> > + * @es_hor_mode: Horizontal mode.
> > + * @es_sum_mode: sum mode.
> 
> Other fields are documented with a capital letter -> "Sum mode."
> 
> > + * @hor_en: Horizontal enable.
> > + * @ver_en: Vertical enable.
> > + * @es_ver_thr: Vertical threshold.
> > + * @es_hor_thr: Horizontal threshold.
> > + */
> > +struct jh7110_isp_sc_af_config {
> > +	__u8 es_hor_mode;
> > +	__u8 es_sum_mode;
> > +	__u8 hor_en;
> > +	__u8 ver_en;
> > +	__u8 es_ver_thr;
> > +	__u16 es_hor_thr;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sc_awb_ps - statistics collection auto white balance pixel sum
> > + *
> > + * @awb_ps_rl: Lower boundary of R value for pixel sum.
> > + * @awb_ps_ru: Upper boundary of R value for pixel sum.
> > + * @awb_ps_gl: Lower boundary of G value for pixel sum.
> > + * @awb_ps_gu: Upper boundary of G value for pixel sum.
> > + * @awb_ps_bl: Lower boundary of B value for pixel sum.
> > + * @awb_ps_bu: Upper boundary of B value for pixel sum.
> > + * @awb_ps_yl: Lower boundary of Y value for pixel sum.
> > + * @awb_ps_yu: Upper boundary of Y value for pixel sum.
> > + * @awb_ps_grl: Lower boundary of G/R ratio for pixel sum.
> > + * @awb_ps_gru: Upper boundary of G/R ratio for pixel sum.
> > + * @awb_ps_gbl: Lower boundary of G/B ratio for pixel sum.
> > + * @awb_ps_gbu: Upper boundary of G/B ratio for pixel sum.
> > + * @awb_ps_grbl: Lower boundary of (Gr/R + b/a * Gb/B) for pixel sum.
> > + * @awb_ps_grbu: Upper boundary of (Gr/R + b/a * Gb/B) for pixel sum.
> > + */
> > +struct jh7110_isp_sc_awb_ps {
> > +	__u8 awb_ps_rl;
> > +	__u8 awb_ps_ru;
> > +	__u8 awb_ps_gl;
> > +	__u8 awb_ps_gu;
> > +	__u8 awb_ps_bl;
> > +	__u8 awb_ps_bu;
> > +	__u8 awb_ps_yl;
> > +	__u8 awb_ps_yu;
> > +	__u16 awb_ps_grl;
> > +	__u16 awb_ps_gru;
> > +	__u16 awb_ps_gbl;
> > +	__u16 awb_ps_gbu;
> > +	__u16 awb_ps_grbl;
> > +	__u16 awb_ps_grbu;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sc_awb_ws - statistics collection auto white balance weight sum
> > + *
> > + * @awb_ws_rl: Lower boundary of R value for weight sum.
> > + * @awb_ws_ru: Upper boundary of R value for weight sum.
> > + * @awb_ws_grl: Lower boundary of Gr value for weight sum.
> > + * @awb_ws_gru: Upper boundary of Gr value for weight sum.
> > + * @awb_ws_gbl: Lower boundary of Gb value for weight sum.
> > + * @awb_ws_gbu: Upper boundary of Gb value for weight sum.
> > + * @awb_ws_bl: Lower boundary of B value for weight sum.
> > + * @awb_ws_bu: Upper boundary of B value for weight sum.
> > + */
> > +struct jh7110_isp_sc_awb_ws {
> > +	__u8 awb_ws_rl;
> > +	__u8 awb_ws_ru;
> > +	__u8 awb_ws_grl;
> > +	__u8 awb_ws_gru;
> > +	__u8 awb_ws_gbl;
> > +	__u8 awb_ws_gbu;
> > +	__u8 awb_ws_bl;
> > +	__u8 awb_ws_bu;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sc_awb_point - statistics collection auto white balance point
> > + *
> > + * @weight: Weighting value at point.
> > + */
> > +struct jh7110_isp_sc_awb_point {
> > +	__u8 weight;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sc_awb_config - statistics collection auto white balance configure
> > + *
> > + * @ps_config: statistics collection auto white balance pixel sum.
> 
> nit: please be consistent with using capital letters or not in doc.
> 
> > + * @awb_ps_grb_ba: auto white balance b/a value.
> > + * @sel: input mux for statistics collection auto white balance.
> > + * @ws_config: statistics collection auto white balance weight sum.
> > + * @awb_cw: Weighting value at 13x13 point.
> > + * @pts: statistics collection auto white balance point.
> > + */
> > +struct jh7110_isp_sc_awb_config {
> > +	struct jh7110_isp_sc_awb_ps ps_config;
> > +	__u8 awb_ps_grb_ba;
> > +	__u8 sel;
> > +	struct jh7110_isp_sc_awb_ws ws_config;
> > +	__u8 awb_cw[169];
> > +	struct jh7110_isp_sc_awb_point pts[17];
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_sc_setting - Configuration used by statistics collection
> > + *
> > + * @enabled: enabled setting flag.
> > + * @crop_config: statistics collection crop configure.
> > + * @af_config: statistics collection auto focus configure.
> > + * @awb_config: statistics collection auto white balance configure.
> > + */
> > +struct jh7110_isp_sc_setting {
> > +	__u32 enabled;
> > +	struct jh7110_isp_sc_config crop_config;
> > +	struct jh7110_isp_sc_af_config af_config;
> > +	struct jh7110_isp_sc_awb_config awb_config;
> > +};
> > +
> > +/**
> > + * struct jh7110_isp_params_buffer - StarFive JH7110 ISP Parameters Meta Data
> > + *
> > + * @enable_setting: enabled setting module (JH7110_ISP_MODULE_* definitions).
> > + * @wb_setting: Configuration used by auto white balance gain.
> > + * @car_setting: Configuration used by color artifact removal.
> > + * @ccm_setting: Configuration used by color correction matrix.
> > + * @cfa_setting: Configuration used by demosaic module.
> > + * @ctc_setting: Configuration used by crosstalk remove.
> > + * @dbc_setting: Configuration used by defect bad pixels correction.
> > + * @dnyuv_setting: Configuration used by yuv domain denoise.
> > + * @gmargb_setting: Configuration used by RGB gamma.
> > + * @lccf_setting: Configuration used by lens correction cosine fourth.
> > + * @obc_setting: Configuration used by optical black compensation.
> > + * @oecf_setting: Configuration used by opto-electric conversion function.
> > + * @r2y_setting: Configuration used by RGB To YUV.
> > + * @sat_setting: Configuration used by Saturation.
> > + * @sharp_setting: Configuration used by Sharpen.
> > + * @ycrv_setting: Configuration used by Y Curve.
> > + * @sc_setting: Configuration used by statistics collection.
> > + */
> > +struct jh7110_isp_params_buffer {
> > +	__u32 enable_setting;
> > +	struct jh7110_isp_wb_setting wb_setting;
> > +	struct jh7110_isp_car_setting car_setting;
> > +	struct jh7110_isp_ccm_setting ccm_setting;
> > +	struct jh7110_isp_cfa_setting cfa_setting;
> > +	struct jh7110_isp_ctc_setting ctc_setting;
> > +	struct jh7110_isp_dbc_setting dbc_setting;
> > +	struct jh7110_isp_dnyuv_setting dnyuv_setting;
> > +	struct jh7110_isp_gmargb_setting gmargb_setting;
> > +	struct jh7110_isp_lccf_setting lccf_setting;
> > +	struct jh7110_isp_obc_setting obc_setting;
> > +	struct jh7110_isp_oecf_setting oecf_setting;
> > +	struct jh7110_isp_r2y_setting r2y_setting;
> > +	struct jh7110_isp_sat_setting sat_setting;
> > +	struct jh7110_isp_sharp_setting sharp_setting;
> > +	struct jh7110_isp_ycrv_setting ycrv_setting;
> > +	struct jh7110_isp_sc_setting sc_setting;
> > +};
> > +
> > +/**
> > + * Statistics Collection Meta Data Flag
> > + */
> > +#define JH7110_ISP_SC_FLAG_AWB			0x0
> > +#define JH7110_ISP_SC_FLAG_AE_AF		0xffff
> > +
> > +#pragma pack(1)
> > +
> > +/**
> > + * struct jh7110_isp_sc_buffer - StarFive JH7110 ISP Statistics Collection Meta Data
> > + *
> > + * @y_histogram: Y histogram data for saturation control.
> > + * @reserv0: reserve byte.
> > + * @bright_sc: bright statistic. If flag is JH7110_ISP_SC_FLAG_AE_AF, This field is
> 
> s/bright statistics/brightness statistics/
> 
> no capital "T" after ,
> 
> > + *             saved auto exposure and auto focus. If flag is JH7110_ISP_SC_FLAG_AWB,
> > + *             This field is saved auto white balance.
> 
> no capital "T" after ,
> 
> I would replace "this field is saved" which doesn't sound great in
> English (not a native speaker though) with "this field stores".
> 
> > + * @reserv1: reserve byte.
> > + * @ae_hist_y: Y histogram for auto exposure.
> > + * @reserv2: reserve byte.
> > + * @flag: Statistics Collection Meta Data Flag (JH7110_ISP_SC_FLAG_* definitions)
> > + */
> > +struct jh7110_isp_sc_buffer {
> > +	__u32 y_histogram[64];
> > +	__u32 reserv0[33];
> > +	__u32 bright_sc[4096];
> > +	__u32 reserv1[96];
> > +	__u32 ae_hist_y[128];
> > +	__u32 reserv2[511];
> > +	__u16 flag;
> > +};
> > +
> > +#pragma pack()
> 
> This structure is packed, is it populated directly from HW registers
> with a memcpy or a DMA transfer ? I guess I'll find it out in the next
> patches.
> 
> > +
> > +#endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Laurent Pinchart]

On Mon, Jul 22, 2024 at 05:27:26PM +0300, Laurent Pinchart wrote:
> On Wed, Jul 10, 2024 at 11:11:57AM +0200, Jacopo Mondi wrote:
> > On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> > > Add JH7110 ISP module definitions for user space.
> > >
> > > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > > Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> > > ---
> > >  MAINTAINERS                     |   1 +
> > >  include/uapi/linux/jh7110-isp.h | 739 ++++++++++++++++++++++++++++++++
> > 
> > With the recently merged support for the RaspberryPi PiSP BE we have
> > introduced include/uapi/linux/media/raspberry.
> > 
> > Would you consider placing this in
> > include/uapi/linux/media/startfive/ ?
> 
> That sounds like a good idea.
> 
> > >  2 files changed, 740 insertions(+)
> > >  create mode 100644 include/uapi/linux/jh7110-isp.h
> > >
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index 507f04a80499..890604eb0d64 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -21305,6 +21305,7 @@ S:	Maintained
> > >  F:	Documentation/admin-guide/media/starfive_camss.rst
> > >  F:	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> > >  F:	drivers/staging/media/starfive/camss
> > > +F:	include/uapi/linux/jh7110-isp.h
> > >
> > >  STARFIVE CRYPTO DRIVER
> > >  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> > > diff --git a/include/uapi/linux/jh7110-isp.h b/include/uapi/linux/jh7110-isp.h
> > > new file mode 100644
> > > index 000000000000..4939cd63e771
> > > --- /dev/null
> > > +++ b/include/uapi/linux/jh7110-isp.h
> > > @@ -0,0 +1,739 @@
> > > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> > > +/*
> > > + * jh7110-isp.h
> > > + *
> > > + * JH7110 ISP driver - user space header file.
> > > + *
> > > + * Copyright © 2023 StarFive Technology Co., Ltd.
> > > + *
> > > + * Author: Zejian Su <zejian.su@starfivetech.com>
> > > + *
> > > + */
> > > +
> > > +#ifndef __JH7110_ISP_H_
> > > +#define __JH7110_ISP_H_
> > > +
> > 
> > Do you need to include
> > 
> > #include <linux/types.h>
> > 
> > > +/**
> > 
> > Is this kernel-doc or a single * would do ?
> > 
> > > + * ISP Module Diagram
> > > + * ------------------
> > > + *
> > > + *  Raw  +-----+    +------+    +------+    +----+
> > > + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> > > + *       +-----+    +------+    +------+    +----+     |
> > > + *                                                     |
> > > + *  +--------------------------------------------------+
> > > + *  |
> > > + *  |    +-----+    +-----+    +-----+    +-----+
> > > + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> > > + *       +-----+    +-----+    +-----+    +-----+      |
> > > + *                                                     |
> > > + *  +--------------------------------------------------+
> > > + *  |
> > > + *  |    +-----+    +--------+    +-----+    +------+
> > > + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> > > + *       +-----+    +--------+    +-----+    +------+  |
> > > + *                                                     |
> > > + *  +--------------------------------------------------+
> > > + *  |
> > > + *  |    +-------+    +-------+    +-----+    +----+
> > > + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> > > + *       +-------+    +-------+    +-----+    +----+
> > > + *
> 
> The diagram is useful, thank you. A glossary would also be nice, maybe
> as
> 
>  * - OBC: Optical Black Correction
>  * - OECF: Opto-Electric Conversion Function
>  * ...
> 
> I think that would be easier to read than the comments above each macro
> below. Up to you.
> 
> > > + */
> > > +
> > > +/* Auto White Balance */
> > > +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> > > +/* Color Artifact Removal */
> > > +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> > > +/* Color Correction Matrix */
> > > +#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
> > > +/* Color Filter Arrays */
> > > +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> > > +/* Crosstalk Correction */
> > > +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> > > +/* Defect Bad Pixel Correction */
> > > +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> > > +/* Denoise YUV */
> > > +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
> > > +/* RGB Gamma */
> > > +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
> > > +/* Lens Correction Cosine Fourth */
> > > +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
> > > +/* Optical Black Correction */
> > > +#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
> > > +/* Opto-Electric Conversion Function */
> > > +#define JH7110_ISP_MODULE_OECF_SETTING			(1U << 10)
> > > +/* RGB To YUV */
> > > +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U << 11)
> > > +/* Saturation */
> > > +#define JH7110_ISP_MODULE_SAT_SETTING			(1U << 12)
> > > +/* Sharpen */
> > > +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U << 13)
> > > +/* Y Curve */
> > > +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U << 14)
> > > +/* Statistics Collection */
> > > +#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)
> 
> Unless there's a specific reason to keep the current order, maybe you
> could sort those macros in the same order as in the module diagram ?
> 
> > > +
> > > +/**
> > > + * struct jh7110_isp_wb_gain - auto white balance gain
> > > + *
> > > + * @gain_r: gain value for red component.
> > > + * @gain_g: gain value for green component.
> > > + * @gain_b: gain value for blue component.
> 
> I suppose the gains are expressed as fixed-point integers. This needs
> more details, what are the limits, and how many bits of integer and
> fractional parts are there ?
> 
> Same comment for all the other values below.
> 
> > > + */
> > > +struct jh7110_isp_wb_gain {
> > > +	__u16 gain_r;
> > > +	__u16 gain_g;
> > > +	__u16 gain_b;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_wb_setting - Configuration used by auto white balance gain
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @gains: auto white balance gain setting.
> > > + */
> > > +struct jh7110_isp_wb_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_wb_gain gains;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_car_setting - Configuration used by color artifact removal
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + */
> > > +struct jh7110_isp_car_setting {
> > > +	__u32 enabled;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ccm_smlow - Color correction matrix
> > > + *
> > > + * @ccm: color transform matrix with size 3 by 3.
> > > + * @offsets: the offsets of R, G, B after the transform by the ccm.
> > > + */
> > > +struct jh7110_isp_ccm_smlow {
> > > +	__s32 ccm[3][3];
> > > +	__s32 offsets[3];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ccm_setting - Configuration used by color correction matrix
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @ccm_smlow: Color correction matrix.
> > > + */
> > > +struct jh7110_isp_ccm_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_ccm_smlow ccm_smlow;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_cfa_params - demosaic parameters
> > > + *
> > > + * @hv_width: detail smooth factor
> > > + * @cross_cov: Cross covariance weighting.
> 
> This documentation doesn't tell how to use those paraemeters. This
> comment applies to many other parameters below. There are three main
> options to improve that:
> 
> - Expanding the documentation in this header file to clearly explain how
>   to compute the parameters values.

Here's an example of the level of details that would be expected:
https://lore.kernel.org/linux-media/20240709132906.3198927-19-dan.scally@ideasonboard.com/T/#m1bc3fe5b5cea831ece6452b13225d291bc4619db

> 
> - Providing an open userspace implementation of ISP algorithms that
>   showcase how to calculate the values.
> 
> - Providing detailed hardware documentation for the ISP. This is usually
>   not favoured by ISP vendors, and we are not pushing for this, but I
>   wanted to mention it for completeness.
> 
> If you would prefer the second option, any open-source camera framework
> would be acceptable, but in practice the only real option is likely
> libcamera.
> 
> This does not mean that you have to open-source all your ISP control
> algorithms. Only the code needed to explain how ISP parameters are
> applied to the image and are computed is needed. Other parts, such as
> for instance AI-based computation of white balance gains, or complex AGC
> calculations, don't need to be open-source.
> 
> The explain this requirement different and perhaps more clearly, the
> goal is to make sure that developers who have access to only open-source
> code (ISP kernel driver, this header, any open-source userspace code,
> ...) will have enough information to configure and control the ISP.
> 
> > > + */
> > > +struct jh7110_isp_cfa_params {
> > > +	__s32 hv_width;
> > > +	__s32 cross_cov;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_cfa_params - Configuration used by demosaic module
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @settings: demosaic parameters.
> > > + */
> > > +struct jh7110_isp_cfa_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_cfa_params settings;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ctc_params - crosstalk remove parameters
> > > + *
> > > + * @saf_mode: smooth area filter mode.
> > > + * @daf_mode: detail area filter mode.
> > > + * @max_gt: the threshold for imbalance detection when pixel intensity is close to maximum.
> > 
> > You could easily make this < 80 cols (here and in other places).
> > 
> > I know there are different opinions on how strict on the 80 cols limit
> > we should be, so up to you.
> > 
> > > + * @min_gt: the threshold for imbalance detection when pixel intensity is close to 0.
> > > + */
> > > +struct jh7110_isp_ctc_params {
> > > +	__u8 saf_mode;
> > > +	__u8 daf_mode;
> > > +	__s32 max_gt;
> > > +	__s32 min_gt;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ctc_params - Configuration used by crosstalk remove
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @settings: corsstalk remove parameters.
> > 
> > crosstalk
> > 
> > > + */
> > > +struct jh7110_isp_ctc_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_ctc_params settings;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_dbc_params - defect pixels correction parameters
> > > + *
> > > + * @bad_gt: bad pixel threshold for the green channel.
> > > + * @bad_xt: bad pixel threshold for the red and blue channels.
> > > + */
> > > +struct jh7110_isp_dbc_params {
> > > +	__s32 bad_gt;
> > > +	__s32 bad_xt;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_dbc_params - Configuration used by defect bad pixels correction
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @settings: defect pixels correction parameters.
> > > + */
> > > +struct jh7110_isp_dbc_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_dbc_params settings;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_dnyuv_params - yuv domain denoise parameters
> > > + *
> > > + * @y_sweight: ten coefficients of 7x7 spatial filter for Y channel.
> > > + * @y_curve: intensity difference (similarity) weight lookup table for Y channel.
> > > + * @uv_sweight: ten coefficients of 7x7 spatial filter for U and V channel.
> > > + * @uv_curve: intensity difference (similarity) weight lookup table for U and V channel.
> > > + */
> > > +struct jh7110_isp_dnyuv_params {
> > > +	__u8 y_sweight[10];
> > > +	__u16 y_curve[7];
> > > +	__u8 uv_sweight[10];
> > > +	__u16 uv_curve[7];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_dnyuv_params - Configuration used by yuv domain denoise
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @settings: yuv domain denoise parameters.
> > > + */
> > > +struct jh7110_isp_dnyuv_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_dnyuv_params settings;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_gmargb_point - RGB Gamma point
> > > + *
> > > + * @g_val: RGB gamma value.
> > > + * @sg_val: RGB gamma slope value.
> > > + */
> > > +struct jh7110_isp_gmargb_point {
> > > +	__u16 g_val;
> > > +	__u16 sg_val;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_gmargb_setting - Configuration used by RGB gamma
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @curve: RGB Gamma point table.
> > > + */
> > > +struct jh7110_isp_gmargb_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_gmargb_point curve[15];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_lccf_circle - len circle
> > 
> > lens ?
> > 
> > > + *
> > > + * @center_x: center X distance from capture window.
> > > + * @center_y: center Y distance from capture window.
> > > + * @radius: len circle radius.
> > 
> > here as well
> > 
> > > + */
> > > +struct jh7110_isp_lccf_circle {
> > > +	__s16 center_x;
> > > +	__s16 center_y;
> > > +	__u8 radius;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_lccf_curve_param - lens correction cosine fourth curve param
> > > + *
> > > + * @f1: F1 parameter.
> > > + * @f2: F2 parameter.
> > > + */
> > > +struct jh7110_isp_lccf_curve_param {
> > > +	__s16 f1;
> > > +	__s16 f2;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_lccf_setting - Configuration used by lens correction cosine fourth
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @circle: len circle.
> > > + * @r_param: lens correction cosine fourth curve param for Bayer pattern R.
> > > + * @gr_param: lens correction cosine fourth curve param for Bayer pattern Gr.
> > > + * @gb_param: lens correction cosine fourth curve param for Bayer pattern Gb.
> > > + * @b_param: lens correction cosine fourth curve param for Bayer pattern B.
> > > + */
> > > +struct jh7110_isp_lccf_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_lccf_circle circle;
> > > +	struct jh7110_isp_lccf_curve_param r_param;
> > > +	struct jh7110_isp_lccf_curve_param gr_param;
> > > +	struct jh7110_isp_lccf_curve_param gb_param;
> > > +	struct jh7110_isp_lccf_curve_param b_param;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_obc_win_size - optical black correction window size
> > > + *
> > > + * @width: window width.
> > > + * @height: window height.
> > > + */
> > > +struct jh7110_isp_obc_win_size {
> > > +	__u32 width;
> > > +	__u32 height;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_obc_gain - optical black correction symbol gain
> > > + *
> > > + * @tl_gain: gain at point A for symbol.
> > > + * @tr_gain: gain at point B for symbol.
> > > + * @bl_gain: gain at point C for symbol.
> > > + * @br_gain: gain at point D for symbol.
> > > + */
> > > +struct jh7110_isp_obc_gain {
> > > +	__u8 tl_gain;
> > > +	__u8 tr_gain;
> > > +	__u8 bl_gain;
> > > +	__u8 br_gain;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_obc_offset - optical black correction symbol offset
> > > + *
> > > + * @tl_offset: offset at point A for symbol.
> > > + * @tr_offset: offset at point B for symbol.
> > > + * @bl_offset: offset at point C for symbol.
> > > + * @br_offset: offset at point D for symbol.
> > > + */
> > > +struct jh7110_isp_obc_offset {
> > > +	__u8 tl_offset;
> > > +	__u8 tr_offset;
> > > +	__u8 bl_offset;
> > > +	__u8 br_offset;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_obc_setting - Configuration used by optical black correction
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @win_size: optical black correction window size.
> > > + * @gain: optical black correction symbol gain.
> > > + * @offset: optical black correction symbol offset.
> > > + */
> > > +struct jh7110_isp_obc_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_obc_win_size win_size;
> > > +	struct jh7110_isp_obc_gain gain[4];
> > > +	struct jh7110_isp_obc_offset offset[4];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_oecf_point - oecf curve
> > > + *
> > > + * @x: x coordinate.
> > > + * @y: y coordinate.
> > > + * @slope: the slope between this point and the next point.
> > > + */
> > > +struct jh7110_isp_oecf_point {
> > > +	__u16 x;
> > > +	__u16 y;
> > > +	__s16 slope;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_oecf_setting - Configuration used by opto-electric conversion function
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @r_curve: red pixel oecf curve.
> > > + * @gr_curve: green pixel oecf curve in GR line.
> > > + * @gb_curve: green pixel oecf curve in GB line.
> > > + * @b_curve: blue pixel oecf curve.
> > > + */
> > > +struct jh7110_isp_oecf_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_oecf_point r_curve[16];
> > > +	struct jh7110_isp_oecf_point gr_curve[16];
> > > +	struct jh7110_isp_oecf_point gb_curve[16];
> > > +	struct jh7110_isp_oecf_point b_curve[16];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_r2y_matrix - RGB to YUV color conversion matrix
> > > + *
> > > + * @m: The 3x3 color conversion matrix coefficient.
> > > + */
> > > +struct jh7110_isp_r2y_matrix {
> > > +	__s16 m[9];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_r2y_setting - Configuration used by RGB To YUV
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @matrix: RGB to YUV color conversion matrix.
> > > + */
> > > +struct jh7110_isp_r2y_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_r2y_matrix matrix;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sat_curve - Saturation curve
> > > + *
> > > + * @yi_min: the minimum input Y value.
> > > + * @yo_ir: the ratio of Y output range to input range.
> > > + * @yo_min: the minimum output Y value.
> > > + * @yo_max: the maximum output Y value.
> > > + */
> > > +struct jh7110_isp_sat_curve {
> > > +	__s16 yi_min;
> > > +	__s16 yo_ir;
> > > +	__s16 yo_min;
> > > +	__s16 yo_max;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sat_hue_info - Chroma Saturation Hue Factor
> > > + *
> > > + * @cos: COS hue factor.
> > > + * @sin: SIN hue factor.
> > > + */
> > > +struct jh7110_isp_sat_hue_info {
> > > +	__s16 cos;
> > > +	__s16 sin;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sat_info - Saturation information
> > > + *
> > > + * @gain_cmab: Chroma saturation magnitude amplification base for gain.
> > > + * @gain_cmmd: Chroma saturation magnitude amplification delta for gain.
> > > + * @threshold_cmb: Chroma saturation magnitude base threshold.
> > > + * @threshold_cmd: Chroma saturation magnitude delta threshold.
> > > + * @offset_u: Chroma saturation U offset.
> > > + * @offset_v: Chroma saturation V offset.
> > > + * @cmsf: Chroma saturation magnitude scaling factor.
> > > + */
> > > +struct jh7110_isp_sat_info {
> > > +	__s16 gain_cmab;
> > > +	__s16 gain_cmmd;
> > > +	__s16 threshold_cmb;
> > > +	__s16 threshold_cmd;
> > > +	__s16 offset_u;
> > > +	__s16 offset_v;
> > > +	__s16 cmsf;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sat_setting - Configuration used by Saturation
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @curve: Saturation curve.
> > > + * @hue_info: Chroma Saturation Hue Factor.
> > > + * @sat_info: Saturation information.s
> > 
> > informations.
> > 
> > > + */
> > > +struct jh7110_isp_sat_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_sat_curve curve;
> > > +	struct jh7110_isp_sat_hue_info hue_info;
> > > +	struct jh7110_isp_sat_info sat_info;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sharp_weight - Sharpe weight
> > > + *
> > > + * @weight: Sharpen filter weight.
> > > + * @recip_wei_sum: Sharpen amplification filter weight normalization factor.
> > > + */
> > > +struct jh7110_isp_sharp_weight {
> > > +	__u8 weight[15];
> > > +	__u32 recip_wei_sum;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sharp_strength - Sharpen strength
> > > + *
> > > + * @diff: Sharpen Edge amplification delta level.
> > > + * @f: Sharpen Edge amplification factor.
> > > + * @s: Sharpen Edge amplification factor slope.
> > > + */
> > > +struct jh7110_isp_sharp_strength {
> > > +	__s16 diff[4];
> > > +	__s16 f[3];
> > > +	__s32 s[3];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sharp_setting - Configuration used by Sharpen
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @weight: Sharpe weight.
> > > + * @strength: Sharpen strength.
> > > + * @pdirf: Positive Factor Multiplier.
> > > + * @ndirf: Negative Factor Multiplier.
> > > + */
> > > +struct jh7110_isp_sharp_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_sharp_weight weight;
> > > +	struct jh7110_isp_sharp_strength strength;
> > > +	__s8 pdirf;
> > > +	__s8 ndirf;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ycrv_curve - Y Curve parameters table
> > > + *
> > > + * @y: Y curve L parameters value.
> > > + */
> > > +struct jh7110_isp_ycrv_curve {
> > > +	__s16 y[64];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ycrv_setting - Configuration used by Y Curve
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @curve: Y Curve parameters table.
> > > + */
> > > +struct jh7110_isp_ycrv_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_ycrv_curve curve;
> > 
> > I am a bit failing in seeing the point of embedding the settings in a
> > dedicated structure when you have a single instance of the
> > configuration like this and in other instances. Isn't
> > 
> >         struct jh7110_isp_ycrv_setting {
> >                 __u32 enabled;
> >                 __s16 y[64];
> >         };
> > 
> > easier ? Or do you need a dedicated type for other reasons ?
> > 
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_config - statistics collection crop configure
> > > + *
> > > + * @h_start: Horizontal starting point for frame cropping.
> > > + * @v_start: Vertical starting point for frame cropping.
> > > + * @sw_width: Width of statistics collection sub-window.
> > > + * @sw_height: Height of statistics collection sub-window.
> > > + * @hperiod: Horizontal period.
> > > + * @hkeep: Horizontal keep.
> > > + * @vperiod: Vertical period.
> > > + * @vkeep: Vertical keep.
> > > + */
> > > +struct jh7110_isp_sc_config {
> > > +	__u16 h_start;
> > > +	__u16 v_start;
> > > +	__u8 sw_width;
> > > +	__u8 sw_height;
> > > +	__u8 hperiod;
> > > +	__u8 hkeep;
> > > +	__u8 vperiod;
> > > +	__u8 vkeep;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_af_config - statistics collection auto focus configure
> > > + *
> > > + * @es_hor_mode: Horizontal mode.
> > > + * @es_sum_mode: sum mode.
> > 
> > Other fields are documented with a capital letter -> "Sum mode."
> > 
> > > + * @hor_en: Horizontal enable.
> > > + * @ver_en: Vertical enable.
> > > + * @es_ver_thr: Vertical threshold.
> > > + * @es_hor_thr: Horizontal threshold.
> > > + */
> > > +struct jh7110_isp_sc_af_config {
> > > +	__u8 es_hor_mode;
> > > +	__u8 es_sum_mode;
> > > +	__u8 hor_en;
> > > +	__u8 ver_en;
> > > +	__u8 es_ver_thr;
> > > +	__u16 es_hor_thr;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_awb_ps - statistics collection auto white balance pixel sum
> > > + *
> > > + * @awb_ps_rl: Lower boundary of R value for pixel sum.
> > > + * @awb_ps_ru: Upper boundary of R value for pixel sum.
> > > + * @awb_ps_gl: Lower boundary of G value for pixel sum.
> > > + * @awb_ps_gu: Upper boundary of G value for pixel sum.
> > > + * @awb_ps_bl: Lower boundary of B value for pixel sum.
> > > + * @awb_ps_bu: Upper boundary of B value for pixel sum.
> > > + * @awb_ps_yl: Lower boundary of Y value for pixel sum.
> > > + * @awb_ps_yu: Upper boundary of Y value for pixel sum.
> > > + * @awb_ps_grl: Lower boundary of G/R ratio for pixel sum.
> > > + * @awb_ps_gru: Upper boundary of G/R ratio for pixel sum.
> > > + * @awb_ps_gbl: Lower boundary of G/B ratio for pixel sum.
> > > + * @awb_ps_gbu: Upper boundary of G/B ratio for pixel sum.
> > > + * @awb_ps_grbl: Lower boundary of (Gr/R + b/a * Gb/B) for pixel sum.
> > > + * @awb_ps_grbu: Upper boundary of (Gr/R + b/a * Gb/B) for pixel sum.
> > > + */
> > > +struct jh7110_isp_sc_awb_ps {
> > > +	__u8 awb_ps_rl;
> > > +	__u8 awb_ps_ru;
> > > +	__u8 awb_ps_gl;
> > > +	__u8 awb_ps_gu;
> > > +	__u8 awb_ps_bl;
> > > +	__u8 awb_ps_bu;
> > > +	__u8 awb_ps_yl;
> > > +	__u8 awb_ps_yu;
> > > +	__u16 awb_ps_grl;
> > > +	__u16 awb_ps_gru;
> > > +	__u16 awb_ps_gbl;
> > > +	__u16 awb_ps_gbu;
> > > +	__u16 awb_ps_grbl;
> > > +	__u16 awb_ps_grbu;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_awb_ws - statistics collection auto white balance weight sum
> > > + *
> > > + * @awb_ws_rl: Lower boundary of R value for weight sum.
> > > + * @awb_ws_ru: Upper boundary of R value for weight sum.
> > > + * @awb_ws_grl: Lower boundary of Gr value for weight sum.
> > > + * @awb_ws_gru: Upper boundary of Gr value for weight sum.
> > > + * @awb_ws_gbl: Lower boundary of Gb value for weight sum.
> > > + * @awb_ws_gbu: Upper boundary of Gb value for weight sum.
> > > + * @awb_ws_bl: Lower boundary of B value for weight sum.
> > > + * @awb_ws_bu: Upper boundary of B value for weight sum.
> > > + */
> > > +struct jh7110_isp_sc_awb_ws {
> > > +	__u8 awb_ws_rl;
> > > +	__u8 awb_ws_ru;
> > > +	__u8 awb_ws_grl;
> > > +	__u8 awb_ws_gru;
> > > +	__u8 awb_ws_gbl;
> > > +	__u8 awb_ws_gbu;
> > > +	__u8 awb_ws_bl;
> > > +	__u8 awb_ws_bu;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_awb_point - statistics collection auto white balance point
> > > + *
> > > + * @weight: Weighting value at point.
> > > + */
> > > +struct jh7110_isp_sc_awb_point {
> > > +	__u8 weight;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_awb_config - statistics collection auto white balance configure
> > > + *
> > > + * @ps_config: statistics collection auto white balance pixel sum.
> > 
> > nit: please be consistent with using capital letters or not in doc.
> > 
> > > + * @awb_ps_grb_ba: auto white balance b/a value.
> > > + * @sel: input mux for statistics collection auto white balance.
> > > + * @ws_config: statistics collection auto white balance weight sum.
> > > + * @awb_cw: Weighting value at 13x13 point.
> > > + * @pts: statistics collection auto white balance point.
> > > + */
> > > +struct jh7110_isp_sc_awb_config {
> > > +	struct jh7110_isp_sc_awb_ps ps_config;
> > > +	__u8 awb_ps_grb_ba;
> > > +	__u8 sel;
> > > +	struct jh7110_isp_sc_awb_ws ws_config;
> > > +	__u8 awb_cw[169];
> > > +	struct jh7110_isp_sc_awb_point pts[17];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_setting - Configuration used by statistics collection
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @crop_config: statistics collection crop configure.
> > > + * @af_config: statistics collection auto focus configure.
> > > + * @awb_config: statistics collection auto white balance configure.
> > > + */
> > > +struct jh7110_isp_sc_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_sc_config crop_config;
> > > +	struct jh7110_isp_sc_af_config af_config;
> > > +	struct jh7110_isp_sc_awb_config awb_config;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_params_buffer - StarFive JH7110 ISP Parameters Meta Data
> > > + *
> > > + * @enable_setting: enabled setting module (JH7110_ISP_MODULE_* definitions).
> > > + * @wb_setting: Configuration used by auto white balance gain.
> > > + * @car_setting: Configuration used by color artifact removal.
> > > + * @ccm_setting: Configuration used by color correction matrix.
> > > + * @cfa_setting: Configuration used by demosaic module.
> > > + * @ctc_setting: Configuration used by crosstalk remove.
> > > + * @dbc_setting: Configuration used by defect bad pixels correction.
> > > + * @dnyuv_setting: Configuration used by yuv domain denoise.
> > > + * @gmargb_setting: Configuration used by RGB gamma.
> > > + * @lccf_setting: Configuration used by lens correction cosine fourth.
> > > + * @obc_setting: Configuration used by optical black compensation.
> > > + * @oecf_setting: Configuration used by opto-electric conversion function.
> > > + * @r2y_setting: Configuration used by RGB To YUV.
> > > + * @sat_setting: Configuration used by Saturation.
> > > + * @sharp_setting: Configuration used by Sharpen.
> > > + * @ycrv_setting: Configuration used by Y Curve.
> > > + * @sc_setting: Configuration used by statistics collection.
> > > + */
> > > +struct jh7110_isp_params_buffer {
> > > +	__u32 enable_setting;
> > > +	struct jh7110_isp_wb_setting wb_setting;
> > > +	struct jh7110_isp_car_setting car_setting;
> > > +	struct jh7110_isp_ccm_setting ccm_setting;
> > > +	struct jh7110_isp_cfa_setting cfa_setting;
> > > +	struct jh7110_isp_ctc_setting ctc_setting;
> > > +	struct jh7110_isp_dbc_setting dbc_setting;
> > > +	struct jh7110_isp_dnyuv_setting dnyuv_setting;
> > > +	struct jh7110_isp_gmargb_setting gmargb_setting;
> > > +	struct jh7110_isp_lccf_setting lccf_setting;
> > > +	struct jh7110_isp_obc_setting obc_setting;
> > > +	struct jh7110_isp_oecf_setting oecf_setting;
> > > +	struct jh7110_isp_r2y_setting r2y_setting;
> > > +	struct jh7110_isp_sat_setting sat_setting;
> > > +	struct jh7110_isp_sharp_setting sharp_setting;
> > > +	struct jh7110_isp_ycrv_setting ycrv_setting;
> > > +	struct jh7110_isp_sc_setting sc_setting;
> > > +};
> > > +
> > > +/**
> > > + * Statistics Collection Meta Data Flag
> > > + */
> > > +#define JH7110_ISP_SC_FLAG_AWB			0x0
> > > +#define JH7110_ISP_SC_FLAG_AE_AF		0xffff
> > > +
> > > +#pragma pack(1)
> > > +
> > > +/**
> > > + * struct jh7110_isp_sc_buffer - StarFive JH7110 ISP Statistics Collection Meta Data
> > > + *
> > > + * @y_histogram: Y histogram data for saturation control.
> > > + * @reserv0: reserve byte.
> > > + * @bright_sc: bright statistic. If flag is JH7110_ISP_SC_FLAG_AE_AF, This field is
> > 
> > s/bright statistics/brightness statistics/
> > 
> > no capital "T" after ,
> > 
> > > + *             saved auto exposure and auto focus. If flag is JH7110_ISP_SC_FLAG_AWB,
> > > + *             This field is saved auto white balance.
> > 
> > no capital "T" after ,
> > 
> > I would replace "this field is saved" which doesn't sound great in
> > English (not a native speaker though) with "this field stores".
> > 
> > > + * @reserv1: reserve byte.
> > > + * @ae_hist_y: Y histogram for auto exposure.
> > > + * @reserv2: reserve byte.
> > > + * @flag: Statistics Collection Meta Data Flag (JH7110_ISP_SC_FLAG_* definitions)
> > > + */

The statistics also require more documentation. See
https://lore.kernel.org/linux-media/20240709132906.3198927-19-dan.scally@ideasonboard.com/T/#m107e94a23f6deccddf2afdfba9c9cf5bf63522a9
for an example of the level of details that is expected.

> > > +struct jh7110_isp_sc_buffer {
> > > +	__u32 y_histogram[64];
> > > +	__u32 reserv0[33];
> > > +	__u32 bright_sc[4096];
> > > +	__u32 reserv1[96];
> > > +	__u32 ae_hist_y[128];
> > > +	__u32 reserv2[511];
> > > +	__u16 flag;
> > > +};
> > > +
> > > +#pragma pack()
> > 
> > This structure is packed, is it populated directly from HW registers
> > with a memcpy or a DMA transfer ? I guess I'll find it out in the next
> > patches.
> > 
> > > +
> > > +#endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 03/14] media: videodev2.h, v4l2-ioctl: Add StarFive ISP meta buffer format

[Laurent Pinchart]

Hello Changhuang,

Thank you for the patch.

On Tue, Jul 09, 2024 at 01:38:13AM -0700, Changhuang Liang wrote:
> Add the StarFive ISP specific metadata format
> V4L2_META_FMT_STF_ISP_PARAMS & V4L2_META_FMT_STF_ISP_STAT_3A for 3A.
> 
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>

This patch looks fine. Once the documentation issues are addressed in
patch 01/14,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
>  include/uapi/linux/videodev2.h       | 4 ++++
>  2 files changed, 6 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 4c76d17b4629..8770bfb31c5c 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1456,6 +1456,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
>  	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
>  	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
> +	case V4L2_META_FMT_STF_ISP_PARAMS:	descr = "StarFive ISP 3A Parameters"; break;
> +	case V4L2_META_FMT_STF_ISP_STAT_3A:	descr = "StarFive ISP 3A Statistics"; break;
>  	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
>  	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
>  	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index fe6b67e83751..cfcbfe9bf973 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -841,6 +841,10 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>  #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>  
> +/* Vendor specific - used for StarFive JH7110 ISP camera sub-system */
> +#define V4L2_META_FMT_STF_ISP_PARAMS	v4l2_fourcc('S', 'T', 'F', 'P') /* StarFive ISP 3A Parameters */
> +#define V4L2_META_FMT_STF_ISP_STAT_3A	v4l2_fourcc('S', 'T', 'F', 'S') /* StarFive ISP 3A Statistics */
> +
>  #ifdef __KERNEL__
>  /*
>   * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 04/14] staging: media: starfive: Add a params sink pad and a scd source pad for ISP

[Laurent Pinchart]

Hello Changhuang,

Thank you for the patch.

On Tue, Jul 09, 2024 at 01:38:14AM -0700, Changhuang Liang wrote:
> StarFive ISP can use params sink pad to transmit ISP parameters and use
> scd source pad to capture statistics collection data.
> 
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  .../staging/media/starfive/camss/stf-isp.c    | 77 +++++++++++++++++--
>  .../staging/media/starfive/camss/stf-isp.h    |  2 +
>  2 files changed, 71 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.c b/drivers/staging/media/starfive/camss/stf-isp.c
> index 4e6e26736852..0ebffd09842a 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.c
> +++ b/drivers/staging/media/starfive/camss/stf-isp.c
> @@ -21,13 +21,23 @@ static const struct stf_isp_format isp_formats_sink[] = {
>  	{ MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
>  };
>  
> +static const struct stf_isp_format isp_formats_sink_params[] = {
> +	{ MEDIA_BUS_FMT_METADATA_FIXED },
> +};
> +
>  static const struct stf_isp_format isp_formats_source[] = {
>  	{ MEDIA_BUS_FMT_YUYV8_1_5X8, 8 },
>  };
>  
> +static const struct stf_isp_format isp_formats_source_scd[] = {
> +	{ MEDIA_BUS_FMT_METADATA_FIXED },
> +};
> +
>  static const struct stf_isp_format_table isp_formats_st7110[] = {
>  	{ isp_formats_sink, ARRAY_SIZE(isp_formats_sink) },
> +	{ isp_formats_sink_params, ARRAY_SIZE(isp_formats_sink_params) },
>  	{ isp_formats_source, ARRAY_SIZE(isp_formats_source) },
> +	{ isp_formats_source_scd, ARRAY_SIZE(isp_formats_source_scd) },
>  };
>  
>  static const struct stf_isp_format *
> @@ -93,13 +103,19 @@ static void isp_try_format(struct stf_isp_dev *isp_dev,
>  
>  	formats = &isp_dev->formats[pad];
>  
> -	fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
> -			     STFCAMSS_FRAME_MAX_WIDTH);
> -	fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
> -			      STFCAMSS_FRAME_MAX_HEIGHT);
> -	fmt->height &= ~0x1;
> +	if (pad != STF_ISP_PAD_SRC_SCD && pad != STF_ISP_PAD_SINK_PARAMS) {
> +		fmt->width = clamp_t(u32, fmt->width, STFCAMSS_FRAME_MIN_WIDTH,
> +				     STFCAMSS_FRAME_MAX_WIDTH);
> +		fmt->height = clamp_t(u32, fmt->height, STFCAMSS_FRAME_MIN_HEIGHT,
> +				      STFCAMSS_FRAME_MAX_HEIGHT);
> +		fmt->height &= ~0x1;
> +		fmt->colorspace = V4L2_COLORSPACE_SRGB;
> +	} else {
> +		fmt->width = 1;
> +		fmt->height = 1;

You should set fmt->colorspace here too. I think you can set it to 0.

> +	}
> +
>  	fmt->field = V4L2_FIELD_NONE;
> -	fmt->colorspace = V4L2_COLORSPACE_SRGB;
>  	fmt->flags = 0;
>  
>  	if (!stf_g_fmt_by_mcode(formats, fmt->code))
> @@ -119,7 +135,7 @@ static int isp_enum_mbus_code(struct v4l2_subdev *sd,
>  
>  		formats = &isp_dev->formats[code->pad];
>  		code->code = formats->fmts[code->index].code;
> -	} else {
> +	} else if (code->pad == STF_ISP_PAD_SRC) {
>  		struct v4l2_mbus_framefmt *sink_fmt;
>  
>  		if (code->index >= ARRAY_SIZE(isp_formats_source))
> @@ -131,6 +147,10 @@ static int isp_enum_mbus_code(struct v4l2_subdev *sd,
>  		code->code = sink_fmt->code;
>  		if (!code->code)
>  			return -EINVAL;
> +	} else {
> +		if (code->index > 0)
> +			return -EINVAL;
> +		code->code = MEDIA_BUS_FMT_METADATA_FIXED;
>  	}
>  	code->flags = 0;
>  
> @@ -151,6 +171,9 @@ static int isp_set_format(struct v4l2_subdev *sd,
>  	isp_try_format(isp_dev, state, fmt->pad, &fmt->format);
>  	*format = fmt->format;
>  
> +	if (fmt->pad == STF_ISP_PAD_SRC_SCD || fmt->pad == STF_ISP_PAD_SINK_PARAMS)
> +		return 0;
> +
>  	isp_dev->current_fmt = stf_g_fmt_by_mcode(&isp_dev->formats[fmt->pad],
>  						  fmt->format.code);
>  
> @@ -202,6 +225,9 @@ static int isp_get_selection(struct v4l2_subdev *sd,
>  	struct v4l2_subdev_format fmt = { 0 };
>  	struct v4l2_rect *rect;
>  
> +	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad == STF_ISP_PAD_SINK_PARAMS)
> +		return -EINVAL;
> +
>  	switch (sel->target) {
>  	case V4L2_SEL_TGT_CROP_BOUNDS:
>  		if (sel->pad == STF_ISP_PAD_SINK) {
> @@ -239,6 +265,9 @@ static int isp_set_selection(struct v4l2_subdev *sd,
>  	struct stf_isp_dev *isp_dev = v4l2_get_subdevdata(sd);
>  	struct v4l2_rect *rect;
>  
> +	if (sel->pad == STF_ISP_PAD_SRC_SCD || sel->pad == STF_ISP_PAD_SINK_PARAMS)
> +		return -EINVAL;
> +
>  	if (sel->target != V4L2_SEL_TGT_CROP)
>  		return -EINVAL;
>  
> @@ -296,8 +325,38 @@ static int isp_init_formats(struct v4l2_subdev *sd,

While at it, you could rename the function to isp_init_state().

>  			.height = 1080
>  		}
>  	};
> +	struct v4l2_subdev_format format_params = {
> +		.pad = STF_ISP_PAD_SINK_PARAMS,
> +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,

That's not quite right, as the .init_state() handler is used to
initialized both the TRY and ACTIVE states. As the "which" field is
currently ignored through the driver the code should behave correctly
(as far as I can tell), but you may want at some point to initialize the
formats and selection rectangles directly in this function instead of
calling isp_set_format().

> +		.format = {
> +			.code = MEDIA_BUS_FMT_METADATA_FIXED,
> +			.width = 1,
> +			.height = 1

According to
https://docs.kernel.org/userspace-api/media/v4l/subdev-formats.html#metadata-formats,
width and height should be set to 0 for MEDIA_BUS_FMT_METADATA_FIXED.

> +		}
> +	};
> +	struct v4l2_subdev_format format_scd = {
> +		.pad = STF_ISP_PAD_SRC_SCD,
> +		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
> +		.format = {
> +			.code = MEDIA_BUS_FMT_METADATA_FIXED,
> +			.width = 1,
> +			.height = 1
> +		}
> +	};
> +	int ret;
> +
> +	/* Init for STF_ISP_PAD_SINK and STF_ISP_PAD_SRC pad */
> +	ret = isp_set_format(sd, sd_state, &format);
> +	if (ret < 0)
> +		return ret;
> +
> +	/* Init for STF_ISP_PAD_SINK_PARAMS pad */
> +	ret = isp_set_format(sd, sd_state, &format_params);
> +	if (ret < 0)
> +		return ret;
>  
> -	return isp_set_format(sd, sd_state, &format);
> +	/* Init for STF_ISP_PAD_SRC_SCD pad */
> +	return isp_set_format(sd, sd_state, &format_scd);
>  }
>  
>  static const struct v4l2_subdev_video_ops isp_video_ops = {
> @@ -338,7 +397,9 @@ int stf_isp_register(struct stf_isp_dev *isp_dev, struct v4l2_device *v4l2_dev)
>  	v4l2_set_subdevdata(sd, isp_dev);
>  
>  	pads[STF_ISP_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +	pads[STF_ISP_PAD_SINK_PARAMS].flags = MEDIA_PAD_FL_SINK;
>  	pads[STF_ISP_PAD_SRC].flags = MEDIA_PAD_FL_SOURCE;
> +	pads[STF_ISP_PAD_SRC_SCD].flags = MEDIA_PAD_FL_SOURCE;
>  
>  	sd->entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
>  	sd->entity.ops = &isp_media_ops;
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
> index 955cbb048363..bc7e7b0736fa 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.h
> +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> @@ -392,7 +392,9 @@
>  /* pad id for media framework */
>  enum stf_isp_pad_id {
>  	STF_ISP_PAD_SINK = 0,
> +	STF_ISP_PAD_SINK_PARAMS,
>  	STF_ISP_PAD_SRC,
> +	STF_ISP_PAD_SRC_SCD,
>  	STF_ISP_PAD_MAX
>  };
>  

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 09/14] staging: media: starfive: Update ISP initialise config for 3A

[Laurent Pinchart]

Hi Changhuang,

Thank you for the patch.

On Tue, Jul 09, 2024 at 01:38:19AM -0700, Changhuang Liang wrote:
> Upadte ISP initialise for 3A statistics collection data.
> 
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  .../media/starfive/camss/stf-isp-hw-ops.c     | 23 +++++++++++++++++++
>  .../staging/media/starfive/camss/stf-isp.h    | 21 +++++++++++++++++
>  2 files changed, 44 insertions(+)
> 
> diff --git a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> index 3b18d09f2cc6..0bc5e36f952e 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> +++ b/drivers/staging/media/starfive/camss/stf-isp-hw-ops.c
> @@ -300,6 +300,25 @@ static void stf_isp_config_sat(struct stfcamss *stfcamss)
>  	stf_isp_reg_write(stfcamss, ISP_REG_YADJ1, YOMAX(0x3ff) | YOMIN(0x1));
>  }
>  
> +static void stf_isp_config_sc(struct stfcamss *stfcamss)
> +{
> +	stf_isp_reg_write(stfcamss, ISP_REG_SCD_CFG_1, AXI_ID(0));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_0, HSTART(0) | VSTART(0xc));
> +	stf_isp_reg_write(stfcamss, ISP_REG_SC_CFG_1,
> +			  SC_WIDTH(0x1d) | SC_HEIGHT(0x15) |
> +			  AWB_PS_GRB_BA(0x10) | SEL_TYPE(0x3));

There's lots of magic values here and below. A comment would be good.

> +}
> +
> +static void stf_isp_config_yhist(struct stfcamss *stfcamss)
> +{
> +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_0, 0);
> +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_1,
> +			  YH_WIDTH(0x77f) | YH_HEIGHT(0x437));

Does this hardcode the size of the histogram window to 1920x1080 ? It
should be made configurable from userspace. Maybe that's handled in
further patches in this series.

> +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_2,
> +			  YH_DEC_ETW(2) | YH_DEC_ETH(1));
> +	stf_isp_reg_write(stfcamss, ISP_REG_YHIST_CFG_3, 0);
> +}
> +
>  int stf_isp_reset(struct stf_isp_dev *isp_dev)
>  {
>  	stf_isp_reg_set_bit(isp_dev->stfcamss, ISP_REG_ISP_CTRL_0,
> @@ -332,7 +351,11 @@ void stf_isp_init_cfg(struct stf_isp_dev *isp_dev)
>  	stf_isp_config_sharpen(isp_dev->stfcamss);
>  	stf_isp_config_dnyuv(isp_dev->stfcamss);
>  	stf_isp_config_sat(isp_dev->stfcamss);
> +	stf_isp_config_sc(isp_dev->stfcamss);
> +	stf_isp_config_yhist(isp_dev->stfcamss);
>  
> +	stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_DUMP_CFG_1,
> +			  DUMP_BURST_LEN(3) | DUMP_SD(0xb80));
>  	stf_isp_reg_write(isp_dev->stfcamss, ISP_REG_CSI_MODULE_CFG,
>  			  CSI_DUMP_EN | CSI_SC_EN | CSI_AWB_EN |
>  			  CSI_LCCF_EN | CSI_OECF_EN | CSI_OBC_EN | CSI_DEC_EN);
> diff --git a/drivers/staging/media/starfive/camss/stf-isp.h b/drivers/staging/media/starfive/camss/stf-isp.h
> index 0af7b367e57a..eca3ba1ade75 100644
> --- a/drivers/staging/media/starfive/camss/stf-isp.h
> +++ b/drivers/staging/media/starfive/camss/stf-isp.h
> @@ -110,9 +110,19 @@
>  
>  #define ISP_REG_SCD_CFG_0			0x098
>  
> +#define ISP_REG_SCD_CFG_1			0x09c
> +#define AXI_ID(n)				((n) << 24)
> +
> +#define ISP_REG_SC_CFG_0			0x0b8
> +#define VSTART(n)				((n) << 16)
> +#define HSTART(n)				((n) << 0)
> +
>  #define ISP_REG_SC_CFG_1			0x0bc
>  #define ISP_SC_SEL_MASK				GENMASK(31, 30)
>  #define SEL_TYPE(n)				((n) << 30)
> +#define AWB_PS_GRB_BA(n)			((n) << 16)
> +#define SC_HEIGHT(n)				((n) << 8)
> +#define SC_WIDTH(n)				((n) << 0)
>  
>  #define ISP_REG_LCCF_CFG_2			0x0e0
>  #define ISP_REG_LCCF_CFG_3			0x0e4
> @@ -312,6 +322,17 @@
>  #define DNRM_F(n)				((n) << 16)
>  #define CCM_M_DAT(n)				((n) << 0)
>  
> +#define ISP_REG_YHIST_CFG_0			0xcc8
> +
> +#define ISP_REG_YHIST_CFG_1			0xccc
> +#define YH_HEIGHT(n)				((n) << 16)
> +#define YH_WIDTH(n)				((n) << 0)
> +
> +#define ISP_REG_YHIST_CFG_2			0xcd0
> +#define YH_DEC_ETH(n)				((n) << 16)
> +#define YH_DEC_ETW(n)				((n) << 0)
> +
> +#define ISP_REG_YHIST_CFG_3			0xcd4
>  #define ISP_REG_YHIST_CFG_4			0xcd8
>  
>  #define ISP_REG_YHIST_ACC_0			0xd00
> -- 
> 2.25.1
> 

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 13/14] staging: media: starfive: Drop read support for video capture devices

[Laurent Pinchart]

Hi Changhuang,

Thank you for the patch.

On Tue, Jul 09, 2024 at 01:38:23AM -0700, Changhuang Liang wrote:
> StarFive video capture devices is not support VB2_READ queue access
> method. Drop the implementation of the read hook.
> 
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/staging/media/starfive/camss/stf-video.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/drivers/staging/media/starfive/camss/stf-video.c b/drivers/staging/media/starfive/camss/stf-video.c
> index 8c6c45d8b7a1..28e6679ee9b9 100644
> --- a/drivers/staging/media/starfive/camss/stf-video.c
> +++ b/drivers/staging/media/starfive/camss/stf-video.c
> @@ -570,7 +570,6 @@ static const struct v4l2_file_operations stf_vid_fops = {
>  	.release        = vb2_fop_release,
>  	.poll           = vb2_fop_poll,
>  	.mmap           = vb2_fop_mmap,
> -	.read           = vb2_fop_read,
>  };
>  
>  /* -----------------------------------------------------------------------------

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 14/14] admin-guide: media: Update documents for StarFive Camera Subsystem

[Laurent Pinchart]

Hi Changhuang,

Thank you for the patch.

On Tue, Jul 09, 2024 at 01:38:24AM -0700, Changhuang Liang wrote:
> Add ISP output parameters and 3A statistisc collection data video
> device for documents.
> 
> Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> ---
>  .../admin-guide/media/starfive_camss.rst      | 11 +++++++---
>  .../media/starfive_camss_graph.dot            | 22 +++++++++++--------
>  2 files changed, 21 insertions(+), 12 deletions(-)
> 
> diff --git a/Documentation/admin-guide/media/starfive_camss.rst b/Documentation/admin-guide/media/starfive_camss.rst
> index ca42e9447c47..020f1969e67f 100644
> --- a/Documentation/admin-guide/media/starfive_camss.rst
> +++ b/Documentation/admin-guide/media/starfive_camss.rst
> @@ -58,15 +58,20 @@ The media controller pipeline graph is as follows:
>      :alt:   starfive_camss_graph.dot
>      :align: center
>  
> -The driver has 2 video devices:
> +The driver has 4 video devices:
>  
> +- output_params: The meta output device, transmitting the parameters to ISP
> +  module.
>  - capture_raw: The capture device, capturing image data directly from a sensor.
>  - capture_yuv: The capture device, capturing YUV frame data processed by the
> -  ISP module
> +  ISP module.
> +- capture_scd: The meta capture device, capturing 3A statistics collection data

Renaming "scd" to "stats" (here and through the whole series) would make
the code and documentation easier to understand for third parties, as
it's not immediately clear what "scd" means. I won't insist too much if
this would cause lots of issues.

> +  processed by the ISP module.
>  
>  The driver has 3 subdevices:
>  
> -- stf_isp: is responsible for all the isp operations, outputs YUV frames.
> +- stf_isp: is responsible for all the isp operations, outputs YUV frames
> +  and 3A statistics collection data.
>  - cdns_csi2rx: a CSI-2 bridge supporting up to 4 CSI lanes in input, and 4
>    different pixel streams in output.
>  - imx219: an image sensor, image data is sent through MIPI CSI-2.
> diff --git a/Documentation/admin-guide/media/starfive_camss_graph.dot b/Documentation/admin-guide/media/starfive_camss_graph.dot
> index 8eff1f161ac7..7961255d3ad6 100644
> --- a/Documentation/admin-guide/media/starfive_camss_graph.dot
> +++ b/Documentation/admin-guide/media/starfive_camss_graph.dot
> @@ -1,12 +1,16 @@
>  digraph board {
>  	rankdir=TB
> -	n00000001 [label="{{<port0> 0} | stf_isp\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> -	n00000001:port1 -> n00000008 [style=dashed]
> -	n00000004 [label="capture_raw\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> -	n00000008 [label="capture_yuv\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> -	n0000000e [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
> -	n0000000e:port1 -> n00000001:port0 [style=dashed]
> -	n0000000e:port1 -> n00000004 [style=dashed]
> -	n00000018 [label="{{} | imx219 6-0010\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> -	n00000018:port0 -> n0000000e:port0 [style=bold]
> +	n00000001 [label="{{<port0> 0 | <port1> 1} | stf_isp\n/dev/v4l-subdev0 | {<port2> 2 | <port3> 3}}", shape=Mrecord, style=filled, fillcolor=green]
> +	n00000001:port2 -> n0000000e
> +	n00000001:port3 -> n00000012 [style=dashed]
> +	n00000006 [label="output_params\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
> +	n00000006 -> n00000001:port1 [style=dashed]
> +	n0000000a [label="capture_raw\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> +	n0000000e [label="capture_yuv\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
> +	n00000012 [label="capture_scd\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
> +	n0000001c [label="{{<port0> 0} | cdns_csi2rx.19800000.csi-bridge\n/dev/v4l-subdev1 | {<port1> 1 | <port2> 2 | <port3> 3 | <port4> 4}}", shape=Mrecord, style=filled, fillcolor=green]
> +	n0000001c:port1 -> n00000001:port0 [style=dashed]
> +	n0000001c:port1 -> n0000000a [style=dashed]
> +	n00000026 [label="{{} | imx219 6-0010\n/dev/v4l-subdev2 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> +	n00000026:port0 -> n0000001c:port0 [style=bold]
>  }

-- 
Regards,

Laurent Pinchart

回复: [PATCH v5 14/14] admin-guide: media: Update documents for StarFive Camera Subsystem

[Changhuang Liang]

Hi, Laurent

Thanks for your comments.

> Hi Changhuang,
> 
> Thank you for the patch.
> 
> On Tue, Jul 09, 2024 at 01:38:24AM -0700, Changhuang Liang wrote:
> > Add ISP output parameters and 3A statistisc collection data video
> > device for documents.
> >
> > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > ---
> >  .../admin-guide/media/starfive_camss.rst      | 11 +++++++---
> >  .../media/starfive_camss_graph.dot            | 22
> +++++++++++--------
> >  2 files changed, 21 insertions(+), 12 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/media/starfive_camss.rst
> > b/Documentation/admin-guide/media/starfive_camss.rst
> > index ca42e9447c47..020f1969e67f 100644
> > --- a/Documentation/admin-guide/media/starfive_camss.rst
> > +++ b/Documentation/admin-guide/media/starfive_camss.rst
> > @@ -58,15 +58,20 @@ The media controller pipeline graph is as follows:
> >      :alt:   starfive_camss_graph.dot
> >      :align: center
> >
> > -The driver has 2 video devices:
> > +The driver has 4 video devices:
> >
> > +- output_params: The meta output device, transmitting the parameters
> > +to ISP
> > +  module.
> >  - capture_raw: The capture device, capturing image data directly from a
> sensor.
> >  - capture_yuv: The capture device, capturing YUV frame data processed
> > by the
> > -  ISP module
> > +  ISP module.
> > +- capture_scd: The meta capture device, capturing 3A statistics
> > +collection data
> 
> Renaming "scd" to "stats" (here and through the whole series) would make
> the code and documentation easier to understand for third parties, as it's not
> immediately clear what "scd" means. I won't insist too much if this would
> cause lots of issues.
> 

Agreed, I will change this. 

Regards,
Changhuang

回复: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Changhuang Liang]

Hi, Laurent

Thanks for your comments.

> 
> On Wed, Jul 10, 2024 at 11:11:57AM +0200, Jacopo Mondi wrote:
> > On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> > > Add JH7110 ISP module definitions for user space.
> > >
> > > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > > Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> > > ---
> > >  MAINTAINERS                     |   1 +
> > >  include/uapi/linux/jh7110-isp.h | 739
> > > ++++++++++++++++++++++++++++++++
> >
> > With the recently merged support for the RaspberryPi PiSP BE we have
> > introduced include/uapi/linux/media/raspberry.
> >
> > Would you consider placing this in
> > include/uapi/linux/media/startfive/ ?
> 
> That sounds like a good idea.
> 
> > >  2 files changed, 740 insertions(+)
> > >  create mode 100644 include/uapi/linux/jh7110-isp.h
> > >
> > > diff --git a/MAINTAINERS b/MAINTAINERS index
> > > 507f04a80499..890604eb0d64 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -21305,6 +21305,7 @@ S:	Maintained
> > >  F:	Documentation/admin-guide/media/starfive_camss.rst
> > >  F:
> 	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> > >  F:	drivers/staging/media/starfive/camss
> > > +F:	include/uapi/linux/jh7110-isp.h
> > >
> > >  STARFIVE CRYPTO DRIVER
> > >  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> > > diff --git a/include/uapi/linux/jh7110-isp.h
> > > b/include/uapi/linux/jh7110-isp.h new file mode 100644 index
> > > 000000000000..4939cd63e771
> > > --- /dev/null
> > > +++ b/include/uapi/linux/jh7110-isp.h
> > > @@ -0,0 +1,739 @@
> > > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR
> > > +BSD-3-Clause) */
> > > +/*
> > > + * jh7110-isp.h
> > > + *
> > > + * JH7110 ISP driver - user space header file.
> > > + *
> > > + * Copyright © 2023 StarFive Technology Co., Ltd.
> > > + *
> > > + * Author: Zejian Su <zejian.su@starfivetech.com>
> > > + *
> > > + */
> > > +
> > > +#ifndef __JH7110_ISP_H_
> > > +#define __JH7110_ISP_H_
> > > +
> >
> > Do you need to include
> >
> > #include <linux/types.h>
> >
> > > +/**
> >
> > Is this kernel-doc or a single * would do ?
> >
> > > + * ISP Module Diagram
> > > + * ------------------
> > > + *
> > > + *  Raw  +-----+    +------+    +------+    +----+
> > > + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> > > + *       +-----+    +------+    +------+    +----+     |
> > > + *                                                     |
> > > + *  +--------------------------------------------------+
> > > + *  |
> > > + *  |    +-----+    +-----+    +-----+    +-----+
> > > + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> > > + *       +-----+    +-----+    +-----+    +-----+      |
> > > + *                                                     |
> > > + *  +--------------------------------------------------+
> > > + *  |
> > > + *  |    +-----+    +--------+    +-----+    +------+
> > > + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> > > + *       +-----+    +--------+    +-----+    +------+  |
> > > + *                                                     |
> > > + *  +--------------------------------------------------+
> > > + *  |
> > > + *  |    +-------+    +-------+    +-----+    +----+
> > > + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> > > + *       +-------+    +-------+    +-----+    +----+
> > > + *
> 
> The diagram is useful, thank you. A glossary would also be nice, maybe as
> 
>  * - OBC: Optical Black Correction
>  * - OECF: Opto-Electric Conversion Function
>  * ...
> 
> I think that would be easier to read than the comments above each macro
> below. Up to you.
> 
> > > + */
> > > +
> > > +/* Auto White Balance */
> > > +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> > > +/* Color Artifact Removal */
> > > +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> > > +/* Color Correction Matrix */
> > > +#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
> > > +/* Color Filter Arrays */
> > > +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> > > +/* Crosstalk Correction */
> > > +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> > > +/* Defect Bad Pixel Correction */
> > > +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> > > +/* Denoise YUV */
> > > +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
> > > +/* RGB Gamma */
> > > +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
> > > +/* Lens Correction Cosine Fourth */
> > > +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
> > > +/* Optical Black Correction */
> > > +#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
> > > +/* Opto-Electric Conversion Function */
> > > +#define JH7110_ISP_MODULE_OECF_SETTING			(1U << 10)
> > > +/* RGB To YUV */
> > > +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U << 11)
> > > +/* Saturation */
> > > +#define JH7110_ISP_MODULE_SAT_SETTING			(1U << 12)
> > > +/* Sharpen */
> > > +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U << 13)
> > > +/* Y Curve */
> > > +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U << 14)
> > > +/* Statistics Collection */
> > > +#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)
> 
> Unless there's a specific reason to keep the current order, maybe you could
> sort those macros in the same order as in the module diagram ?
> 
> > > +
> > > +/**
> > > + * struct jh7110_isp_wb_gain - auto white balance gain
> > > + *
> > > + * @gain_r: gain value for red component.
> > > + * @gain_g: gain value for green component.
> > > + * @gain_b: gain value for blue component.
> 
> I suppose the gains are expressed as fixed-point integers. This needs more
> details, what are the limits, and how many bits of integer and fractional parts
> are there ?
> 
> Same comment for all the other values below.
> 
> > > + */
> > > +struct jh7110_isp_wb_gain {
> > > +	__u16 gain_r;
> > > +	__u16 gain_g;
> > > +	__u16 gain_b;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_wb_setting - Configuration used by auto white
> > > +balance gain
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @gains: auto white balance gain setting.
> > > + */
> > > +struct jh7110_isp_wb_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_wb_gain gains;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_car_setting - Configuration used by color
> > > +artifact removal
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + */
> > > +struct jh7110_isp_car_setting {
> > > +	__u32 enabled;
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ccm_smlow - Color correction matrix
> > > + *
> > > + * @ccm: color transform matrix with size 3 by 3.
> > > + * @offsets: the offsets of R, G, B after the transform by the ccm.
> > > + */
> > > +struct jh7110_isp_ccm_smlow {
> > > +	__s32 ccm[3][3];
> > > +	__s32 offsets[3];
> > > +};
> > > +
> > > +/**
> > > + * struct jh7110_isp_ccm_setting - Configuration used by color
> > > +correction matrix
> > > + *
> > > + * @enabled: enabled setting flag.
> > > + * @ccm_smlow: Color correction matrix.
> > > + */
> > > +struct jh7110_isp_ccm_setting {
> > > +	__u32 enabled;
> > > +	struct jh7110_isp_ccm_smlow ccm_smlow; };
> > > +
> > > +/**
> > > + * struct jh7110_isp_cfa_params - demosaic parameters
> > > + *
> > > + * @hv_width: detail smooth factor
> > > + * @cross_cov: Cross covariance weighting.
> 
> This documentation doesn't tell how to use those paraemeters. This comment
> applies to many other parameters below. There are three main options to
> improve that:
> 
> - Expanding the documentation in this header file to clearly explain how
>   to compute the parameters values.
> 
> - Providing an open userspace implementation of ISP algorithms that
>   showcase how to calculate the values.
> 
> - Providing detailed hardware documentation for the ISP. This is usually
>   not favoured by ISP vendors, and we are not pushing for this, but I
>   wanted to mention it for completeness.
> 

We prefer the first option. It will take a lot of time for us to supplement the documentation.

> If you would prefer the second option, any open-source camera framework
> would be acceptable, but in practice the only real option is likely libcamera.
> 
> This does not mean that you have to open-source all your ISP control
> algorithms. Only the code needed to explain how ISP parameters are applied
> to the image and are computed is needed. Other parts, such as for instance
> AI-based computation of white balance gains, or complex AGC calculations,
> don't need to be open-source.
> 
> The explain this requirement different and perhaps more clearly, the goal is to
> make sure that developers who have access to only open-source code (ISP
> kernel driver, this header, any open-source userspace code,
> ...) will have enough information to configure and control the ISP.
> 

Regards
Changhuang

Re: 回复: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Laurent Pinchart]

Hello Changhuang,

On Wed, Jul 24, 2024 at 02:20:17AM +0000, Changhuang Liang wrote:
> > On Wed, Jul 10, 2024 at 11:11:57AM +0200, Jacopo Mondi wrote:
> > > On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> > > > Add JH7110 ISP module definitions for user space.
> > > >
> > > > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > > > Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> > > > ---
> > > >  MAINTAINERS                     |   1 +
> > > >  include/uapi/linux/jh7110-isp.h | 739 ++++++++++++++++++++++++++++++++
> > >
> > > With the recently merged support for the RaspberryPi PiSP BE we have
> > > introduced include/uapi/linux/media/raspberry.
> > >
> > > Would you consider placing this in
> > > include/uapi/linux/media/startfive/ ?
> > 
> > That sounds like a good idea.
> > 
> > > >  2 files changed, 740 insertions(+)
> > > >  create mode 100644 include/uapi/linux/jh7110-isp.h
> > > >
> > > > diff --git a/MAINTAINERS b/MAINTAINERS index
> > > > 507f04a80499..890604eb0d64 100644
> > > > --- a/MAINTAINERS
> > > > +++ b/MAINTAINERS
> > > > @@ -21305,6 +21305,7 @@ S:	Maintained
> > > >  F:	Documentation/admin-guide/media/starfive_camss.rst
> > > >  F:
> > 	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> > > >  F:	drivers/staging/media/starfive/camss
> > > > +F:	include/uapi/linux/jh7110-isp.h
> > > >
> > > >  STARFIVE CRYPTO DRIVER
> > > >  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> > > > diff --git a/include/uapi/linux/jh7110-isp.h
> > > > b/include/uapi/linux/jh7110-isp.h new file mode 100644 index
> > > > 000000000000..4939cd63e771
> > > > --- /dev/null
> > > > +++ b/include/uapi/linux/jh7110-isp.h
> > > > @@ -0,0 +1,739 @@
> > > > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR
> > > > +BSD-3-Clause) */
> > > > +/*
> > > > + * jh7110-isp.h
> > > > + *
> > > > + * JH7110 ISP driver - user space header file.
> > > > + *
> > > > + * Copyright © 2023 StarFive Technology Co., Ltd.
> > > > + *
> > > > + * Author: Zejian Su <zejian.su@starfivetech.com>
> > > > + *
> > > > + */
> > > > +
> > > > +#ifndef __JH7110_ISP_H_
> > > > +#define __JH7110_ISP_H_
> > > > +
> > >
> > > Do you need to include
> > >
> > > #include <linux/types.h>
> > >
> > > > +/**
> > >
> > > Is this kernel-doc or a single * would do ?
> > >
> > > > + * ISP Module Diagram
> > > > + * ------------------
> > > > + *
> > > > + *  Raw  +-----+    +------+    +------+    +----+
> > > > + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> > > > + *       +-----+    +------+    +------+    +----+     |
> > > > + *                                                     |
> > > > + *  +--------------------------------------------------+
> > > > + *  |
> > > > + *  |    +-----+    +-----+    +-----+    +-----+
> > > > + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> > > > + *       +-----+    +-----+    +-----+    +-----+      |
> > > > + *                                                     |
> > > > + *  +--------------------------------------------------+
> > > > + *  |
> > > > + *  |    +-----+    +--------+    +-----+    +------+
> > > > + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> > > > + *       +-----+    +--------+    +-----+    +------+  |
> > > > + *                                                     |
> > > > + *  +--------------------------------------------------+
> > > > + *  |
> > > > + *  |    +-------+    +-------+    +-----+    +----+
> > > > + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> > > > + *       +-------+    +-------+    +-----+    +----+
> > > > + *
> > 
> > The diagram is useful, thank you. A glossary would also be nice, maybe as
> > 
> >  * - OBC: Optical Black Correction
> >  * - OECF: Opto-Electric Conversion Function
> >  * ...
> > 
> > I think that would be easier to read than the comments above each macro
> > below. Up to you.
> > 
> > > > + */
> > > > +
> > > > +/* Auto White Balance */
> > > > +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> > > > +/* Color Artifact Removal */
> > > > +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> > > > +/* Color Correction Matrix */
> > > > +#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
> > > > +/* Color Filter Arrays */
> > > > +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> > > > +/* Crosstalk Correction */
> > > > +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> > > > +/* Defect Bad Pixel Correction */
> > > > +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> > > > +/* Denoise YUV */
> > > > +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
> > > > +/* RGB Gamma */
> > > > +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
> > > > +/* Lens Correction Cosine Fourth */
> > > > +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
> > > > +/* Optical Black Correction */
> > > > +#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
> > > > +/* Opto-Electric Conversion Function */
> > > > +#define JH7110_ISP_MODULE_OECF_SETTING			(1U << 10)
> > > > +/* RGB To YUV */
> > > > +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U << 11)
> > > > +/* Saturation */
> > > > +#define JH7110_ISP_MODULE_SAT_SETTING			(1U << 12)
> > > > +/* Sharpen */
> > > > +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U << 13)
> > > > +/* Y Curve */
> > > > +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U << 14)
> > > > +/* Statistics Collection */
> > > > +#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)
> > 
> > Unless there's a specific reason to keep the current order, maybe you could
> > sort those macros in the same order as in the module diagram ?
> > 
> > > > +
> > > > +/**
> > > > + * struct jh7110_isp_wb_gain - auto white balance gain
> > > > + *
> > > > + * @gain_r: gain value for red component.
> > > > + * @gain_g: gain value for green component.
> > > > + * @gain_b: gain value for blue component.
> > 
> > I suppose the gains are expressed as fixed-point integers. This needs more
> > details, what are the limits, and how many bits of integer and fractional parts
> > are there ?
> > 
> > Same comment for all the other values below.
> > 
> > > > + */
> > > > +struct jh7110_isp_wb_gain {
> > > > +	__u16 gain_r;
> > > > +	__u16 gain_g;
> > > > +	__u16 gain_b;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct jh7110_isp_wb_setting - Configuration used by auto white
> > > > +balance gain
> > > > + *
> > > > + * @enabled: enabled setting flag.
> > > > + * @gains: auto white balance gain setting.
> > > > + */
> > > > +struct jh7110_isp_wb_setting {
> > > > +	__u32 enabled;
> > > > +	struct jh7110_isp_wb_gain gains;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct jh7110_isp_car_setting - Configuration used by color
> > > > +artifact removal
> > > > + *
> > > > + * @enabled: enabled setting flag.
> > > > + */
> > > > +struct jh7110_isp_car_setting {
> > > > +	__u32 enabled;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct jh7110_isp_ccm_smlow - Color correction matrix
> > > > + *
> > > > + * @ccm: color transform matrix with size 3 by 3.
> > > > + * @offsets: the offsets of R, G, B after the transform by the ccm.
> > > > + */
> > > > +struct jh7110_isp_ccm_smlow {
> > > > +	__s32 ccm[3][3];
> > > > +	__s32 offsets[3];
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct jh7110_isp_ccm_setting - Configuration used by color
> > > > +correction matrix
> > > > + *
> > > > + * @enabled: enabled setting flag.
> > > > + * @ccm_smlow: Color correction matrix.
> > > > + */
> > > > +struct jh7110_isp_ccm_setting {
> > > > +	__u32 enabled;
> > > > +	struct jh7110_isp_ccm_smlow ccm_smlow; };
> > > > +
> > > > +/**
> > > > + * struct jh7110_isp_cfa_params - demosaic parameters
> > > > + *
> > > > + * @hv_width: detail smooth factor
> > > > + * @cross_cov: Cross covariance weighting.
> > 
> > This documentation doesn't tell how to use those paraemeters. This comment
> > applies to many other parameters below. There are three main options to
> > improve that:
> > 
> > - Expanding the documentation in this header file to clearly explain how
> >   to compute the parameters values.
> > 
> > - Providing an open userspace implementation of ISP algorithms that
> >   showcase how to calculate the values.
> > 
> > - Providing detailed hardware documentation for the ISP. This is usually
> >   not favoured by ISP vendors, and we are not pushing for this, but I
> >   wanted to mention it for completeness.
> > 
> 
> We prefer the first option. It will take a lot of time for us to
> supplement the documentation.

That is fine. Very broadly speaking, the level of documentation we are
aiming for should be enough for a third party developer to implement
support for the ISP control algorithms in libcamera. Please let me know
if you would like to discuss this in more details, to make sure there's
no misunderstanding in the expectations.

> > If you would prefer the second option, any open-source camera framework
> > would be acceptable, but in practice the only real option is likely libcamera.
> > 
> > This does not mean that you have to open-source all your ISP control
> > algorithms. Only the code needed to explain how ISP parameters are applied
> > to the image and are computed is needed. Other parts, such as for instance
> > AI-based computation of white balance gains, or complex AGC calculations,
> > don't need to be open-source.
> > 
> > The explain this requirement different and perhaps more clearly, the goal is to
> > make sure that developers who have access to only open-source code (ISP
> > kernel driver, this header, any open-source userspace code,
> > ...) will have enough information to configure and control the ISP.

-- 
Regards,

Laurent Pinchart

回复: 回复: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Changhuang Liang]

Hi, Laurent

> 
> Hello Changhuang,
> 
> On Wed, Jul 24, 2024 at 02:20:17AM +0000, Changhuang Liang wrote:
> > > On Wed, Jul 10, 2024 at 11:11:57AM +0200, Jacopo Mondi wrote:
> > > > On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> > > > > Add JH7110 ISP module definitions for user space.
> > > > >
> > > > > Signed-off-by: Changhuang Liang
> > > > > <changhuang.liang@starfivetech.com>
> > > > > Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> > > > > ---
> > > > >  MAINTAINERS                     |   1 +
> > > > >  include/uapi/linux/jh7110-isp.h | 739
> > > > > ++++++++++++++++++++++++++++++++
> > > >
> > > > With the recently merged support for the RaspberryPi PiSP BE we
> > > > have introduced include/uapi/linux/media/raspberry.
> > > >
> > > > Would you consider placing this in
> > > > include/uapi/linux/media/startfive/ ?
> > >
> > > That sounds like a good idea.
> > >
> > > > >  2 files changed, 740 insertions(+)  create mode 100644
> > > > > include/uapi/linux/jh7110-isp.h
> > > > >
> > > > > diff --git a/MAINTAINERS b/MAINTAINERS index
> > > > > 507f04a80499..890604eb0d64 100644
> > > > > --- a/MAINTAINERS
> > > > > +++ b/MAINTAINERS
> > > > > @@ -21305,6 +21305,7 @@ S:	Maintained
> > > > >  F:	Documentation/admin-guide/media/starfive_camss.rst
> > > > >  F:
> > > 	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> > > > >  F:	drivers/staging/media/starfive/camss
> > > > > +F:	include/uapi/linux/jh7110-isp.h
> > > > >
> > > > >  STARFIVE CRYPTO DRIVER
> > > > >  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> > > > > diff --git a/include/uapi/linux/jh7110-isp.h
> > > > > b/include/uapi/linux/jh7110-isp.h new file mode 100644 index
> > > > > 000000000000..4939cd63e771
> > > > > --- /dev/null
> > > > > +++ b/include/uapi/linux/jh7110-isp.h
> > > > > @@ -0,0 +1,739 @@
> > > > > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note)
> > > > > +OR
> > > > > +BSD-3-Clause) */
> > > > > +/*
> > > > > + * jh7110-isp.h
> > > > > + *
> > > > > + * JH7110 ISP driver - user space header file.
> > > > > + *
> > > > > + * Copyright © 2023 StarFive Technology Co., Ltd.
> > > > > + *
> > > > > + * Author: Zejian Su <zejian.su@starfivetech.com>
> > > > > + *
> > > > > + */
> > > > > +
> > > > > +#ifndef __JH7110_ISP_H_
> > > > > +#define __JH7110_ISP_H_
> > > > > +
> > > >
> > > > Do you need to include
> > > >
> > > > #include <linux/types.h>
> > > >
> > > > > +/**
> > > >
> > > > Is this kernel-doc or a single * would do ?
> > > >
> > > > > + * ISP Module Diagram
> > > > > + * ------------------
> > > > > + *
> > > > > + *  Raw  +-----+    +------+    +------+    +----+
> > > > > + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> > > > > + *       +-----+    +------+    +------+    +----+     |
> > > > > + *                                                     |
> > > > > + *  +--------------------------------------------------+
> > > > > + *  |
> > > > > + *  |    +-----+    +-----+    +-----+    +-----+
> > > > > + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> > > > > + *       +-----+    +-----+    +-----+    +-----+      |
> > > > > + *                                                     |
> > > > > + *  +--------------------------------------------------+
> > > > > + *  |
> > > > > + *  |    +-----+    +--------+    +-----+    +------+
> > > > > + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> > > > > + *       +-----+    +--------+    +-----+    +------+  |
> > > > > + *                                                     |
> > > > > + *  +--------------------------------------------------+
> > > > > + *  |
> > > > > + *  |    +-------+    +-------+    +-----+    +----+
> > > > > + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> > > > > + *       +-------+    +-------+    +-----+    +----+
> > > > > + *
> > >
> > > The diagram is useful, thank you. A glossary would also be nice,
> > > maybe as
> > >
> > >  * - OBC: Optical Black Correction
> > >  * - OECF: Opto-Electric Conversion Function
> > >  * ...
> > >
> > > I think that would be easier to read than the comments above each
> > > macro below. Up to you.
> > >
> > > > > + */
> > > > > +
> > > > > +/* Auto White Balance */
> > > > > +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> > > > > +/* Color Artifact Removal */
> > > > > +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> > > > > +/* Color Correction Matrix */
> > > > > +#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
> > > > > +/* Color Filter Arrays */
> > > > > +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> > > > > +/* Crosstalk Correction */
> > > > > +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> > > > > +/* Defect Bad Pixel Correction */
> > > > > +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> > > > > +/* Denoise YUV */
> > > > > +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
> > > > > +/* RGB Gamma */
> > > > > +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
> > > > > +/* Lens Correction Cosine Fourth */
> > > > > +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
> > > > > +/* Optical Black Correction */
> > > > > +#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
> > > > > +/* Opto-Electric Conversion Function */
> > > > > +#define JH7110_ISP_MODULE_OECF_SETTING			(1U <<
> 10)
> > > > > +/* RGB To YUV */
> > > > > +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U <<
> 11)
> > > > > +/* Saturation */
> > > > > +#define JH7110_ISP_MODULE_SAT_SETTING			(1U <<
> 12)
> > > > > +/* Sharpen */
> > > > > +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U <<
> 13)
> > > > > +/* Y Curve */
> > > > > +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U <<
> 14)
> > > > > +/* Statistics Collection */
> > > > > +#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)
> > >
> > > Unless there's a specific reason to keep the current order, maybe
> > > you could sort those macros in the same order as in the module diagram ?
> > >
> > > > > +
> > > > > +/**
> > > > > + * struct jh7110_isp_wb_gain - auto white balance gain
> > > > > + *
> > > > > + * @gain_r: gain value for red component.
> > > > > + * @gain_g: gain value for green component.
> > > > > + * @gain_b: gain value for blue component.
> > >
> > > I suppose the gains are expressed as fixed-point integers. This
> > > needs more details, what are the limits, and how many bits of
> > > integer and fractional parts are there ?
> > >
> > > Same comment for all the other values below.
> > >
> > > > > + */
> > > > > +struct jh7110_isp_wb_gain {
> > > > > +	__u16 gain_r;
> > > > > +	__u16 gain_g;
> > > > > +	__u16 gain_b;
> > > > > +};
> > > > > +
> > > > > +/**
> > > > > + * struct jh7110_isp_wb_setting - Configuration used by auto
> > > > > +white balance gain
> > > > > + *
> > > > > + * @enabled: enabled setting flag.
> > > > > + * @gains: auto white balance gain setting.
> > > > > + */
> > > > > +struct jh7110_isp_wb_setting {
> > > > > +	__u32 enabled;
> > > > > +	struct jh7110_isp_wb_gain gains; };
> > > > > +
> > > > > +/**
> > > > > + * struct jh7110_isp_car_setting - Configuration used by color
> > > > > +artifact removal
> > > > > + *
> > > > > + * @enabled: enabled setting flag.
> > > > > + */
> > > > > +struct jh7110_isp_car_setting {
> > > > > +	__u32 enabled;
> > > > > +};
> > > > > +
> > > > > +/**
> > > > > + * struct jh7110_isp_ccm_smlow - Color correction matrix
> > > > > + *
> > > > > + * @ccm: color transform matrix with size 3 by 3.
> > > > > + * @offsets: the offsets of R, G, B after the transform by the ccm.
> > > > > + */
> > > > > +struct jh7110_isp_ccm_smlow {
> > > > > +	__s32 ccm[3][3];
> > > > > +	__s32 offsets[3];
> > > > > +};
> > > > > +
> > > > > +/**
> > > > > + * struct jh7110_isp_ccm_setting - Configuration used by color
> > > > > +correction matrix
> > > > > + *
> > > > > + * @enabled: enabled setting flag.
> > > > > + * @ccm_smlow: Color correction matrix.
> > > > > + */
> > > > > +struct jh7110_isp_ccm_setting {
> > > > > +	__u32 enabled;
> > > > > +	struct jh7110_isp_ccm_smlow ccm_smlow; };
> > > > > +
> > > > > +/**
> > > > > + * struct jh7110_isp_cfa_params - demosaic parameters
> > > > > + *
> > > > > + * @hv_width: detail smooth factor
> > > > > + * @cross_cov: Cross covariance weighting.
> > >
> > > This documentation doesn't tell how to use those paraemeters. This
> > > comment applies to many other parameters below. There are three main
> > > options to improve that:
> > >
> > > - Expanding the documentation in this header file to clearly explain how
> > >   to compute the parameters values.
> > >
> > > - Providing an open userspace implementation of ISP algorithms that
> > >   showcase how to calculate the values.
> > >
> > > - Providing detailed hardware documentation for the ISP. This is usually
> > >   not favoured by ISP vendors, and we are not pushing for this, but I
> > >   wanted to mention it for completeness.
> > >
> >
> > We prefer the first option. It will take a lot of time for us to
> > supplement the documentation.
> 
> That is fine. Very broadly speaking, the level of documentation we are aiming
> for should be enough for a third party developer to implement support for the
> ISP control algorithms in libcamera. Please let me know if you would like to
> discuss this in more details, to make sure there's no misunderstanding in the
> expectations.
> 

After the discussion at our company, our company is not going to open source the ISP algorithm section in the short term.
That's why I selected the first option at the first time.

As you said, more ISP information is needed, however, due to the suspension of ISP and the dissolution of the team, we 
actually lack ISP information ourselves. 

So now we can only get more information about the ISP from the source code.

Therefore, we can only provide the libcamera repository that is not open source ISP algorithm.
https://github.com/changhuangliang/libcamera/tree/starfive_isp_3a
(At the beginning, I was responsible for implementing the drivers part, and Zejian was responsible
for implementing the libcamera, but after zejian left the company, I took over the libcamera part, 
but I am not familiar with this part)

We won't open source in the short term, But we'll probably consider open source later when ISP 
has no commercial value.

So I am confused whether to continue the next version.

> > > If you would prefer the second option, any open-source camera
> > > framework would be acceptable, but in practice the only real option is
> likely libcamera.
> > >
> > > This does not mean that you have to open-source all your ISP control
> > > algorithms. Only the code needed to explain how ISP parameters are
> > > applied to the image and are computed is needed. Other parts, such
> > > as for instance AI-based computation of white balance gains, or
> > > complex AGC calculations, don't need to be open-source.
> > >
> > > The explain this requirement different and perhaps more clearly, the
> > > goal is to make sure that developers who have access to only
> > > open-source code (ISP kernel driver, this header, any open-source
> > > userspace code,
> > > ...) will have enough information to configure and control the ISP.
> 
> --

Best Regards
Changhuang

Re: 回复: 回复: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Laurent Pinchart]

Hello Changhuang,

On Sat, Sep 14, 2024 at 11:53:31AM +0000, Changhuang Liang wrote:
> > On Wed, Jul 24, 2024 at 02:20:17AM +0000, Changhuang Liang wrote:
> > > > On Wed, Jul 10, 2024 at 11:11:57AM +0200, Jacopo Mondi wrote:
> > > > > On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> > > > > > Add JH7110 ISP module definitions for user space.
> > > > > >
> > > > > > Signed-off-by: Changhuang Liang <changhuang.liang@starfivetech.com>
> > > > > > Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> > > > > > ---
> > > > > >  MAINTAINERS                     |   1 +
> > > > > >  include/uapi/linux/jh7110-isp.h | 739 ++++++++++++++++++++++++++++++++
> > > > >
> > > > > With the recently merged support for the RaspberryPi PiSP BE we
> > > > > have introduced include/uapi/linux/media/raspberry.
> > > > >
> > > > > Would you consider placing this in
> > > > > include/uapi/linux/media/startfive/ ?
> > > >
> > > > That sounds like a good idea.
> > > >
> > > > > >  2 files changed, 740 insertions(+)  create mode 100644
> > > > > > include/uapi/linux/jh7110-isp.h
> > > > > >
> > > > > > diff --git a/MAINTAINERS b/MAINTAINERS index
> > > > > > 507f04a80499..890604eb0d64 100644
> > > > > > --- a/MAINTAINERS
> > > > > > +++ b/MAINTAINERS
> > > > > > @@ -21305,6 +21305,7 @@ S:	Maintained
> > > > > >  F:	Documentation/admin-guide/media/starfive_camss.rst
> > > > > >  F:	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> > > > > >  F:	drivers/staging/media/starfive/camss
> > > > > > +F:	include/uapi/linux/jh7110-isp.h
> > > > > >
> > > > > >  STARFIVE CRYPTO DRIVER
> > > > > >  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> > > > > > diff --git a/include/uapi/linux/jh7110-isp.h b/include/uapi/linux/jh7110-isp.h
> > > > > > new file mode 100644 index 000000000000..4939cd63e771
> > > > > > --- /dev/null
> > > > > > +++ b/include/uapi/linux/jh7110-isp.h
> > > > > > @@ -0,0 +1,739 @@
> > > > > > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> > > > > > +/*
> > > > > > + * jh7110-isp.h
> > > > > > + *
> > > > > > + * JH7110 ISP driver - user space header file.
> > > > > > + *
> > > > > > + * Copyright © 2023 StarFive Technology Co., Ltd.
> > > > > > + *
> > > > > > + * Author: Zejian Su <zejian.su@starfivetech.com>
> > > > > > + *
> > > > > > + */
> > > > > > +
> > > > > > +#ifndef __JH7110_ISP_H_
> > > > > > +#define __JH7110_ISP_H_
> > > > > > +
> > > > >
> > > > > Do you need to include
> > > > >
> > > > > #include <linux/types.h>
> > > > >
> > > > > > +/**
> > > > >
> > > > > Is this kernel-doc or a single * would do ?
> > > > >
> > > > > > + * ISP Module Diagram
> > > > > > + * ------------------
> > > > > > + *
> > > > > > + *  Raw  +-----+    +------+    +------+    +----+
> > > > > > + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> > > > > > + *       +-----+    +------+    +------+    +----+     |
> > > > > > + *                                                     |
> > > > > > + *  +--------------------------------------------------+
> > > > > > + *  |
> > > > > > + *  |    +-----+    +-----+    +-----+    +-----+
> > > > > > + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> > > > > > + *       +-----+    +-----+    +-----+    +-----+      |
> > > > > > + *                                                     |
> > > > > > + *  +--------------------------------------------------+
> > > > > > + *  |
> > > > > > + *  |    +-----+    +--------+    +-----+    +------+
> > > > > > + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> > > > > > + *       +-----+    +--------+    +-----+    +------+  |
> > > > > > + *                                                     |
> > > > > > + *  +--------------------------------------------------+
> > > > > > + *  |
> > > > > > + *  |    +-------+    +-------+    +-----+    +----+
> > > > > > + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> > > > > > + *       +-------+    +-------+    +-----+    +----+
> > > > > > + *
> > > >
> > > > The diagram is useful, thank you. A glossary would also be nice,
> > > > maybe as
> > > >
> > > >  * - OBC: Optical Black Correction
> > > >  * - OECF: Opto-Electric Conversion Function
> > > >  * ...
> > > >
> > > > I think that would be easier to read than the comments above each
> > > > macro below. Up to you.
> > > >
> > > > > > + */
> > > > > > +
> > > > > > +/* Auto White Balance */
> > > > > > +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> > > > > > +/* Color Artifact Removal */
> > > > > > +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> > > > > > +/* Color Correction Matrix */
> > > > > > +#define JH7110_ISP_MODULE_CCM_SETTING			(1U << 2)
> > > > > > +/* Color Filter Arrays */
> > > > > > +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> > > > > > +/* Crosstalk Correction */
> > > > > > +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> > > > > > +/* Defect Bad Pixel Correction */
> > > > > > +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> > > > > > +/* Denoise YUV */
> > > > > > +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U << 6)
> > > > > > +/* RGB Gamma */
> > > > > > +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U << 7)
> > > > > > +/* Lens Correction Cosine Fourth */
> > > > > > +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U << 8)
> > > > > > +/* Optical Black Correction */
> > > > > > +#define JH7110_ISP_MODULE_OBC_SETTING			(1U << 9)
> > > > > > +/* Opto-Electric Conversion Function */
> > > > > > +#define JH7110_ISP_MODULE_OECF_SETTING			(1U << 10)
> > > > > > +/* RGB To YUV */
> > > > > > +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U << 11)
> > > > > > +/* Saturation */
> > > > > > +#define JH7110_ISP_MODULE_SAT_SETTING			(1U << 12)
> > > > > > +/* Sharpen */
> > > > > > +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U << 13)
> > > > > > +/* Y Curve */
> > > > > > +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U << 14)
> > > > > > +/* Statistics Collection */
> > > > > > +#define JH7110_ISP_MODULE_SC_SETTING			(1U << 15)
> > > >
> > > > Unless there's a specific reason to keep the current order, maybe
> > > > you could sort those macros in the same order as in the module diagram ?
> > > >
> > > > > > +
> > > > > > +/**
> > > > > > + * struct jh7110_isp_wb_gain - auto white balance gain
> > > > > > + *
> > > > > > + * @gain_r: gain value for red component.
> > > > > > + * @gain_g: gain value for green component.
> > > > > > + * @gain_b: gain value for blue component.
> > > >
> > > > I suppose the gains are expressed as fixed-point integers. This
> > > > needs more details, what are the limits, and how many bits of
> > > > integer and fractional parts are there ?
> > > >
> > > > Same comment for all the other values below.
> > > >
> > > > > > + */
> > > > > > +struct jh7110_isp_wb_gain {
> > > > > > +	__u16 gain_r;
> > > > > > +	__u16 gain_g;
> > > > > > +	__u16 gain_b;
> > > > > > +};
> > > > > > +
> > > > > > +/**
> > > > > > + * struct jh7110_isp_wb_setting - Configuration used by auto
> > > > > > +white balance gain
> > > > > > + *
> > > > > > + * @enabled: enabled setting flag.
> > > > > > + * @gains: auto white balance gain setting.
> > > > > > + */
> > > > > > +struct jh7110_isp_wb_setting {
> > > > > > +	__u32 enabled;
> > > > > > +	struct jh7110_isp_wb_gain gains; };
> > > > > > +
> > > > > > +/**
> > > > > > + * struct jh7110_isp_car_setting - Configuration used by color
> > > > > > +artifact removal
> > > > > > + *
> > > > > > + * @enabled: enabled setting flag.
> > > > > > + */
> > > > > > +struct jh7110_isp_car_setting {
> > > > > > +	__u32 enabled;
> > > > > > +};
> > > > > > +
> > > > > > +/**
> > > > > > + * struct jh7110_isp_ccm_smlow - Color correction matrix
> > > > > > + *
> > > > > > + * @ccm: color transform matrix with size 3 by 3.
> > > > > > + * @offsets: the offsets of R, G, B after the transform by the ccm.
> > > > > > + */
> > > > > > +struct jh7110_isp_ccm_smlow {
> > > > > > +	__s32 ccm[3][3];
> > > > > > +	__s32 offsets[3];
> > > > > > +};
> > > > > > +
> > > > > > +/**
> > > > > > + * struct jh7110_isp_ccm_setting - Configuration used by color
> > > > > > +correction matrix
> > > > > > + *
> > > > > > + * @enabled: enabled setting flag.
> > > > > > + * @ccm_smlow: Color correction matrix.
> > > > > > + */
> > > > > > +struct jh7110_isp_ccm_setting {
> > > > > > +	__u32 enabled;
> > > > > > +	struct jh7110_isp_ccm_smlow ccm_smlow; };
> > > > > > +
> > > > > > +/**
> > > > > > + * struct jh7110_isp_cfa_params - demosaic parameters
> > > > > > + *
> > > > > > + * @hv_width: detail smooth factor
> > > > > > + * @cross_cov: Cross covariance weighting.
> > > >
> > > > This documentation doesn't tell how to use those paraemeters. This
> > > > comment applies to many other parameters below. There are three main
> > > > options to improve that:
> > > >
> > > > - Expanding the documentation in this header file to clearly explain how
> > > >   to compute the parameters values.
> > > >
> > > > - Providing an open userspace implementation of ISP algorithms that
> > > >   showcase how to calculate the values.
> > > >
> > > > - Providing detailed hardware documentation for the ISP. This is usually
> > > >   not favoured by ISP vendors, and we are not pushing for this, but I
> > > >   wanted to mention it for completeness.
> > >
> > > We prefer the first option. It will take a lot of time for us to
> > > supplement the documentation.
> > 
> > That is fine. Very broadly speaking, the level of documentation we are aiming
> > for should be enough for a third party developer to implement support for the
> > ISP control algorithms in libcamera. Please let me know if you would like to
> > discuss this in more details, to make sure there's no misunderstanding in the
> > expectations.
> 
> After the discussion at our company, our company is not going to open source the ISP algorithm section in the short term.
> That's why I selected the first option at the first time.
> 
> As you said, more ISP information is needed, however, due to the suspension of ISP and the dissolution of the team, we 
> actually lack ISP information ourselves. 
> 
> So now we can only get more information about the ISP from the source code.
> 
> Therefore, we can only provide the libcamera repository that is not open source ISP algorithm.
> https://github.com/changhuangliang/libcamera/tree/starfive_isp_3a
> (At the beginning, I was responsible for implementing the drivers part, and Zejian was responsible
> for implementing the libcamera, but after zejian left the company, I took over the libcamera part, 
> but I am not familiar with this part)
> 
> We won't open source in the short term, But we'll probably consider open source later when ISP 
> has no commercial value.
> 
> So I am confused whether to continue the next version.

A kernel driver must be usable by open and closed userspace alike. With
neither of documentation of the ISP or a reference open-source userspace
implementation being available, this is not possible. Therefore, I'm
afraid we won't be able to accept ISP support in the mainline kernel.

> > > > If you would prefer the second option, any open-source camera
> > > > framework would be acceptable, but in practice the only real option is
> > > > likely libcamera.
> > > >
> > > > This does not mean that you have to open-source all your ISP control
> > > > algorithms. Only the code needed to explain how ISP parameters are
> > > > applied to the image and are computed is needed. Other parts, such
> > > > as for instance AI-based computation of white balance gains, or
> > > > complex AGC calculations, don't need to be open-source.
> > > >
> > > > The explain this requirement different and perhaps more clearly, the
> > > > goal is to make sure that developers who have access to only
> > > > open-source code (ISP kernel driver, this header, any open-source
> > > > userspace code,
> > > > ...) will have enough information to configure and control the ISP.

-- 
Regards,

Laurent Pinchart

回复: 回复: 回复: [PATCH v5 01/14] media: starfive: Add JH7110 ISP module definitions

[Changhuang Liang]

Hi, Laurent

> Hello Changhuang,
> 
> On Sat, Sep 14, 2024 at 11:53:31AM +0000, Changhuang Liang wrote:
> > > On Wed, Jul 24, 2024 at 02:20:17AM +0000, Changhuang Liang wrote:
> > > > > On Wed, Jul 10, 2024 at 11:11:57AM +0200, Jacopo Mondi wrote:
> > > > > > On Tue, Jul 09, 2024 at 01:38:11AM GMT, Changhuang Liang wrote:
> > > > > > > Add JH7110 ISP module definitions for user space.
> > > > > > >
> > > > > > > Signed-off-by: Changhuang Liang
> > > > > > > <changhuang.liang@starfivetech.com>
> > > > > > > Signed-off-by: Zejian Su <zejian.su@starfivetech.com>
> > > > > > > ---
> > > > > > >  MAINTAINERS                     |   1 +
> > > > > > >  include/uapi/linux/jh7110-isp.h | 739
> > > > > > > ++++++++++++++++++++++++++++++++
> > > > > >
> > > > > > With the recently merged support for the RaspberryPi PiSP BE
> > > > > > we have introduced include/uapi/linux/media/raspberry.
> > > > > >
> > > > > > Would you consider placing this in
> > > > > > include/uapi/linux/media/startfive/ ?
> > > > >
> > > > > That sounds like a good idea.
> > > > >
> > > > > > >  2 files changed, 740 insertions(+)  create mode 100644
> > > > > > > include/uapi/linux/jh7110-isp.h
> > > > > > >
> > > > > > > diff --git a/MAINTAINERS b/MAINTAINERS index
> > > > > > > 507f04a80499..890604eb0d64 100644
> > > > > > > --- a/MAINTAINERS
> > > > > > > +++ b/MAINTAINERS
> > > > > > > @@ -21305,6 +21305,7 @@ S:	Maintained
> > > > > > >  F:	Documentation/admin-guide/media/starfive_camss.rst
> > > > > > >  F:
> 	Documentation/devicetree/bindings/media/starfive,jh7110-camss.yaml
> > > > > > >  F:	drivers/staging/media/starfive/camss
> > > > > > > +F:	include/uapi/linux/jh7110-isp.h
> > > > > > >
> > > > > > >  STARFIVE CRYPTO DRIVER
> > > > > > >  M:	Jia Jie Ho <jiajie.ho@starfivetech.com>
> > > > > > > diff --git a/include/uapi/linux/jh7110-isp.h
> > > > > > > b/include/uapi/linux/jh7110-isp.h new file mode 100644 index
> > > > > > > 000000000000..4939cd63e771
> > > > > > > --- /dev/null
> > > > > > > +++ b/include/uapi/linux/jh7110-isp.h
> > > > > > > @@ -0,0 +1,739 @@
> > > > > > > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH
> > > > > > > +Linux-syscall-note) OR BSD-3-Clause) */
> > > > > > > +/*
> > > > > > > + * jh7110-isp.h
> > > > > > > + *
> > > > > > > + * JH7110 ISP driver - user space header file.
> > > > > > > + *
> > > > > > > + * Copyright © 2023 StarFive Technology Co., Ltd.
> > > > > > > + *
> > > > > > > + * Author: Zejian Su <zejian.su@starfivetech.com>
> > > > > > > + *
> > > > > > > + */
> > > > > > > +
> > > > > > > +#ifndef __JH7110_ISP_H_
> > > > > > > +#define __JH7110_ISP_H_
> > > > > > > +
> > > > > >
> > > > > > Do you need to include
> > > > > >
> > > > > > #include <linux/types.h>
> > > > > >
> > > > > > > +/**
> > > > > >
> > > > > > Is this kernel-doc or a single * would do ?
> > > > > >
> > > > > > > + * ISP Module Diagram
> > > > > > > + * ------------------
> > > > > > > + *
> > > > > > > + *  Raw  +-----+    +------+    +------+    +----+
> > > > > > > + *  ---->| OBC |--->| OECF |--->| LCCF |--->| WB |-----+
> > > > > > > + *       +-----+    +------+    +------+    +----+     |
> > > > > > > + *
> |
> > > > > > > + *  +--------------------------------------------------+
> > > > > > > + *  |
> > > > > > > + *  |    +-----+    +-----+    +-----+    +-----+
> > > > > > > + *  +--->| DBC |--->| CTC |--->| CFA |--->| CAR |------+
> > > > > > > + *       +-----+    +-----+    +-----+    +-----+      |
> > > > > > > + *
> |
> > > > > > > + *  +--------------------------------------------------+
> > > > > > > + *  |
> > > > > > > + *  |    +-----+    +--------+    +-----+    +------+
> > > > > > > + *  +--->| CCM |--->| GMARGB |--->| R2Y |--->| YCRV |--+
> > > > > > > + *       +-----+    +--------+    +-----+    +------+  |
> > > > > > > + *
> |
> > > > > > > + *  +--------------------------------------------------+
> > > > > > > + *  |
> > > > > > > + *  |    +-------+    +-------+    +-----+    +----+
> > > > > > > + *  +--->| SHARP |--->| DNYUV |--->| SAT |--->| SC |
> > > > > > > + *       +-------+    +-------+    +-----+    +----+
> > > > > > > + *
> > > > >
> > > > > The diagram is useful, thank you. A glossary would also be nice,
> > > > > maybe as
> > > > >
> > > > >  * - OBC: Optical Black Correction
> > > > >  * - OECF: Opto-Electric Conversion Function
> > > > >  * ...
> > > > >
> > > > > I think that would be easier to read than the comments above
> > > > > each macro below. Up to you.
> > > > >
> > > > > > > + */
> > > > > > > +
> > > > > > > +/* Auto White Balance */
> > > > > > > +#define JH7110_ISP_MODULE_WB_SETTING			(1U << 0)
> > > > > > > +/* Color Artifact Removal */
> > > > > > > +#define JH7110_ISP_MODULE_CAR_SETTING			(1U << 1)
> > > > > > > +/* Color Correction Matrix */
> > > > > > > +#define JH7110_ISP_MODULE_CCM_SETTING			(1U
> << 2)
> > > > > > > +/* Color Filter Arrays */
> > > > > > > +#define JH7110_ISP_MODULE_CFA_SETTING			(1U << 3)
> > > > > > > +/* Crosstalk Correction */
> > > > > > > +#define JH7110_ISP_MODULE_CTC_SETTING			(1U << 4)
> > > > > > > +/* Defect Bad Pixel Correction */
> > > > > > > +#define JH7110_ISP_MODULE_DBC_SETTING			(1U << 5)
> > > > > > > +/* Denoise YUV */
> > > > > > > +#define JH7110_ISP_MODULE_DNYUV_SETTING			(1U
> << 6)
> > > > > > > +/* RGB Gamma */
> > > > > > > +#define JH7110_ISP_MODULE_GMARGB_SETTING		(1U
> << 7)
> > > > > > > +/* Lens Correction Cosine Fourth */
> > > > > > > +#define JH7110_ISP_MODULE_LCCF_SETTING			(1U
> << 8)
> > > > > > > +/* Optical Black Correction */
> > > > > > > +#define JH7110_ISP_MODULE_OBC_SETTING			(1U
> << 9)
> > > > > > > +/* Opto-Electric Conversion Function */
> > > > > > > +#define JH7110_ISP_MODULE_OECF_SETTING			(1U
> << 10)
> > > > > > > +/* RGB To YUV */
> > > > > > > +#define JH7110_ISP_MODULE_R2Y_SETTING			(1U <<
> 11)
> > > > > > > +/* Saturation */
> > > > > > > +#define JH7110_ISP_MODULE_SAT_SETTING			(1U <<
> 12)
> > > > > > > +/* Sharpen */
> > > > > > > +#define JH7110_ISP_MODULE_SHARP_SETTING			(1U
> << 13)
> > > > > > > +/* Y Curve */
> > > > > > > +#define JH7110_ISP_MODULE_YCRV_SETTING			(1U
> << 14)
> > > > > > > +/* Statistics Collection */
> > > > > > > +#define JH7110_ISP_MODULE_SC_SETTING			(1U <<
> 15)
> > > > >
> > > > > Unless there's a specific reason to keep the current order,
> > > > > maybe you could sort those macros in the same order as in the module
> diagram ?
> > > > >
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * struct jh7110_isp_wb_gain - auto white balance gain
> > > > > > > + *
> > > > > > > + * @gain_r: gain value for red component.
> > > > > > > + * @gain_g: gain value for green component.
> > > > > > > + * @gain_b: gain value for blue component.
> > > > >
> > > > > I suppose the gains are expressed as fixed-point integers. This
> > > > > needs more details, what are the limits, and how many bits of
> > > > > integer and fractional parts are there ?
> > > > >
> > > > > Same comment for all the other values below.
> > > > >
> > > > > > > + */
> > > > > > > +struct jh7110_isp_wb_gain {
> > > > > > > +	__u16 gain_r;
> > > > > > > +	__u16 gain_g;
> > > > > > > +	__u16 gain_b;
> > > > > > > +};
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * struct jh7110_isp_wb_setting - Configuration used by
> > > > > > > +auto white balance gain
> > > > > > > + *
> > > > > > > + * @enabled: enabled setting flag.
> > > > > > > + * @gains: auto white balance gain setting.
> > > > > > > + */
> > > > > > > +struct jh7110_isp_wb_setting {
> > > > > > > +	__u32 enabled;
> > > > > > > +	struct jh7110_isp_wb_gain gains; };
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * struct jh7110_isp_car_setting - Configuration used by
> > > > > > > +color artifact removal
> > > > > > > + *
> > > > > > > + * @enabled: enabled setting flag.
> > > > > > > + */
> > > > > > > +struct jh7110_isp_car_setting {
> > > > > > > +	__u32 enabled;
> > > > > > > +};
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * struct jh7110_isp_ccm_smlow - Color correction matrix
> > > > > > > + *
> > > > > > > + * @ccm: color transform matrix with size 3 by 3.
> > > > > > > + * @offsets: the offsets of R, G, B after the transform by the ccm.
> > > > > > > + */
> > > > > > > +struct jh7110_isp_ccm_smlow {
> > > > > > > +	__s32 ccm[3][3];
> > > > > > > +	__s32 offsets[3];
> > > > > > > +};
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * struct jh7110_isp_ccm_setting - Configuration used by
> > > > > > > +color correction matrix
> > > > > > > + *
> > > > > > > + * @enabled: enabled setting flag.
> > > > > > > + * @ccm_smlow: Color correction matrix.
> > > > > > > + */
> > > > > > > +struct jh7110_isp_ccm_setting {
> > > > > > > +	__u32 enabled;
> > > > > > > +	struct jh7110_isp_ccm_smlow ccm_smlow; };
> > > > > > > +
> > > > > > > +/**
> > > > > > > + * struct jh7110_isp_cfa_params - demosaic parameters
> > > > > > > + *
> > > > > > > + * @hv_width: detail smooth factor
> > > > > > > + * @cross_cov: Cross covariance weighting.
> > > > >
> > > > > This documentation doesn't tell how to use those paraemeters.
> > > > > This comment applies to many other parameters below. There are
> > > > > three main options to improve that:
> > > > >
> > > > > - Expanding the documentation in this header file to clearly explain
> how
> > > > >   to compute the parameters values.
> > > > >
> > > > > - Providing an open userspace implementation of ISP algorithms that
> > > > >   showcase how to calculate the values.
> > > > >
> > > > > - Providing detailed hardware documentation for the ISP. This is usually
> > > > >   not favoured by ISP vendors, and we are not pushing for this, but I
> > > > >   wanted to mention it for completeness.
> > > >
> > > > We prefer the first option. It will take a lot of time for us to
> > > > supplement the documentation.
> > >
> > > That is fine. Very broadly speaking, the level of documentation we
> > > are aiming for should be enough for a third party developer to
> > > implement support for the ISP control algorithms in libcamera.
> > > Please let me know if you would like to discuss this in more
> > > details, to make sure there's no misunderstanding in the expectations.
> >
> > After the discussion at our company, our company is not going to open
> source the ISP algorithm section in the short term.
> > That's why I selected the first option at the first time.
> >
> > As you said, more ISP information is needed, however, due to the
> > suspension of ISP and the dissolution of the team, we actually lack ISP
> information ourselves.
> >
> > So now we can only get more information about the ISP from the source
> code.
> >
> > Therefore, we can only provide the libcamera repository that is not open
> source ISP algorithm.
> > https://github.com/changhuangliang/libcamera/tree/starfive_isp_3a
> > (At the beginning, I was responsible for implementing the drivers
> > part, and Zejian was responsible for implementing the libcamera, but
> > after zejian left the company, I took over the libcamera part, but I
> > am not familiar with this part)
> >
> > We won't open source in the short term, But we'll probably consider
> > open source later when ISP has no commercial value.
> >
> > So I am confused whether to continue the next version.
> 
> A kernel driver must be usable by open and closed userspace alike. With
> neither of documentation of the ISP or a reference open-source userspace
> implementation being available, this is not possible. Therefore, I'm afraid we
> won't be able to accept ISP support in the mainline kernel.
> 

Understand, so I will pause the series first. Once it can be open-source, I will 
Work with the new series.

Thank you all in review during this period

> > > > > If you would prefer the second option, any open-source camera
> > > > > framework would be acceptable, but in practice the only real
> > > > > option is likely libcamera.
> > > > >
> > > > > This does not mean that you have to open-source all your ISP
> > > > > control algorithms. Only the code needed to explain how ISP
> > > > > parameters are applied to the image and are computed is needed.
> > > > > Other parts, such as for instance AI-based computation of white
> > > > > balance gains, or complex AGC calculations, don't need to be
> open-source.
> > > > >
> > > > > The explain this requirement different and perhaps more clearly,
> > > > > the goal is to make sure that developers who have access to only
> > > > > open-source code (ISP kernel driver, this header, any
> > > > > open-source userspace code,
> > > > > ...) will have enough information to configure and control the ISP.
> 
> --

Best Regards
Changhuang