[PATCH 0/8] media: rkisp1: Implement support for extensible parameters

linux-media.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* [PATCH 0/8] media: rkisp1: Implement support for extensible parameters
@ 2024-06-05 16:54 Jacopo Mondi
  2024-06-05 16:54 ` [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format Jacopo Mondi
                   ` (7 more replies)
  0 siblings, 8 replies; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

The VeriSilicon ISP8000 IP, supported through the rkisp1 driver in the Linux
kernel, is integrated in several SoCs from different vendors. Different
revisions of the IP differ in the number of supported features and in the
register space location assigned to specific ISP blocks.

The current configuration parameters format, defined in
include/uapi/linux/rkisp1-config.h is realized by a C structure (struct
rkisp1_params_cfg) which wraps other structures that allows to configure
specific ISP blocks. The layout of the parameters buffer is part of the Linux
kernel uAPI and can hardly be extended or modified to adapt it to different
revisions of the same IP.

This series proposes the introduction of a new parameters format for the RkISP1
(without dropping support for the existing one) which is designed with the goals
of being:

1) versioned: can be changed without breaking existing applications
2) extensible: new blocks and parameters can be added without breaking the uABI

To do so, a new 'struct rkisp1_ext_params_cfg' type is introduced. It wraps an
header and a data buffer, which userspace fills with configuration blocks
for each ISP block it intends to configure. The parameters buffer is thus of
different effective sizes, depending on the number of blocks userspace intends
to configure.

The kernel driver parses the data block and decides, based on the versioning
number and the platform it operates on, how to handle each block.

The parameters format is very similar to the parameters format implemented
in the in-review Mali C55 ISP driver [1]

I considered sending this out as RFC, but the implementation has been tested
with a libcamera version ported to use the newly introduced format [1] and works
as expected, so I considered it good enough to send a proper patch series

A kernel branch for the interested ones is available at [3] with an associated
CI pipeline [4]

[1] https://lore.kernel.org/linux-media/20240529152858.183799-15-dan.scally@ideasonboard.com/
[2] https://gitlab.freedesktop.org/camera/libcamera/-/commits/imx8mp/extensible-format?ref_type=heads
[3] https://gitlab.freedesktop.org/linux-media/users/jmondi/-/commits/imx8mp/media-state/extensible-format
[4] https://gitlab.freedesktop.org/linux-media/users/jmondi/-/pipelines/1194073

Jacopo Mondi (8):
  uapi: rkisp1-config: Add extensible parameters format
  uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS
  media: rkisp1: Remove cached format info
  media: rkisp1: Add support for ext format
  media: rkisp1: Implement extensible params support
  media: rkisp1: Propagate pre/post-config errors
  media: rkisp1: Add struct rkisp1_params_buffer
  media: rkisp1: Copy and validate parameters buffer

 Documentation/admin-guide/media/rkisp1.rst    |  11 +-
 .../media/v4l/metafmt-rkisp1.rst              |  62 +-
 .../platform/rockchip/rkisp1/rkisp1-common.h  |  29 +-
 .../platform/rockchip/rkisp1/rkisp1-isp.c     |  14 +-
 .../platform/rockchip/rkisp1/rkisp1-params.c  | 795 ++++++++++++++++--
 drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
 include/uapi/linux/rkisp1-config.h            | 482 +++++++++++
 include/uapi/linux/videodev2.h                |   1 +
 8 files changed, 1314 insertions(+), 81 deletions(-)


base-commit: 1aea3d1d4a21e3e7895663b848ffae79ee82e065
--
2.45.1


^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 10:02   ` Dan Scally
  2024-06-05 16:54 ` [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS Jacopo Mondi
                   ` (6 subsequent siblings)
  7 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

Add to the rkisp1-config.h header data types and documentation of
the extensible parameters format.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 include/uapi/linux/rkisp1-config.h | 482 +++++++++++++++++++++++++++++
 1 file changed, 482 insertions(+)

diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h
index 6eeaf8bf2362..9c93e536f270 100644
--- a/include/uapi/linux/rkisp1-config.h
+++ b/include/uapi/linux/rkisp1-config.h
@@ -996,4 +996,486 @@ struct rkisp1_stat_buffer {
 	struct rkisp1_cif_isp_stat params;
 };
 
+/*---------- PART3: Extensible Configuration Parameters  ------------*/
+
+/**
+ * enum rkisp1_ext_params_block_type - RkISP1 extensible params block type
+ *
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS: Black level subtraction
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC: Defect pixel cluster correction
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG: Sensor de-gamma
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS: Auto white balance gains
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT: ISP filtering
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM: Bayer de-mosaic
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK: Cross-talk correction
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC: Gamma out correction
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF: De-noise pre-filter
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT: De-noise pre-filter strength
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC: Color processing
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_IE: Image effects
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC: Lens shading correction
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS: Auto white balance statistics
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS: Histogram statistics
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS: Auto exposure statistics
+ * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS: Auto-focus statistics
+ */
+enum rkisp1_ext_params_block_type {
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_IE,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS,
+	RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL,
+};
+
+/**
+ * enum rkisp1_ext_params_block_state - RkISP1 extensible parameter block enable
+ *					state flags
+ *
+ * @RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE: Do not change the block power state
+ * @RKISP1_EXT_PARAMS_BLOCK_DISABLE: Disable the HW block
+ * @RKISP1_EXT_PARAMS_BLOCK_ENABLE: Enable the HW block
+ */
+enum rkisp1_ext_params_block_state {
+	RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE,
+	RKISP1_EXT_PARAMS_BLOCK_DISABLE,
+	RKISP1_EXT_PARAMS_BLOCK_ENABLE,
+};
+
+/**
+ * struct rkisp1_ext_params_block_header - RkISP1 extensible parameter block
+ *					   header
+ *
+ * This structure represents the common part of all the ISP configuration
+ * blocks. Each parameters block shall embed an instance of this structure type
+ * as its first member, followed by the block-specific configuration data. The
+ * driver inspects this common header to discern the block type and its size and
+ * properly handle the block content by casting it to the correct block-specific
+ * type.
+ *
+ * The @type field is one of the values enumerated by
+ * :c:type:`rkisp1_ext_params_block_type` and specifies how the data should be
+ * interpreted by the driver. The @size field specifies the size of the
+ * parameters block and is used by the driver for validation purposes.
+ *
+ * The @state field specifies if the ISP block power state should be changed,
+ * and, if it has to, if it has to be enabled to disabled. The possible
+ * states are enumerated by :c:type:`rkisp1_ext_params_block_state`.
+ * When userspace needs to configure and enable an ISP block it shall fully
+ * populate the block configuration and the @state flag shall be set to
+ * RKISP1_EXT_PARAMS_BLOCK_ENABLE. When userspace simply wants to disable the
+ * ISP block the @state flag shall be set to RKISP1_EXT_PARAMS_BLOCK_DISABLE. If
+ * a new configuration of an ISP block should be applied but the power state
+ * doesn't need to be changed, userspace shall fully populate the ISP block
+ * configuration and the @state flag shall be set to
+ * RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE.
+ *
+ * Userspace is responsible for correctly populating the parameters block header
+ * fields (@type, @state and @size) and correctly populate the block-specific
+ * parameters.
+ *
+ * For example:
+ *
+ * .. code-block:: c
+ *
+ *	void populate_bls(struct rkisp1_ext_params_block_header *block) {
+ *		struct rkisp1_ext_params_bls_config *bls =
+ *			(struct rkisp1_ext_params_bls_config *)block;
+ *
+ *		block->header.type = RKISP1_EXT_PARAMS_BLOCK_ID_BLS;
+ *		block->header.state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;
+ *		block->header.size = sizeof(struct rkisp1_ext_params_bls_config);
+ *
+ *		bls->bls_config.enable_auto = 0;
+ *		bls->bls_config.fixed_val.r = blackLevelRed_;
+ *		bls->bls_config.fixed_val.gr = blackLevelGreenR_;
+ *		bls->bls_config.fixed_val.gb = blackLevelGreenB_;
+ *		bls->bls_config.fixed_val.b = blackLevelBlue_;
+ *	}
+ *
+ * @type: The parameters block type, see
+ *	  :c:type:`rkisp1_ext_params_block_type`
+ * @state: The block enable state flag, see
+ *	   :c:type:`rkisp1_ext_params_block_state`
+ * @size: Size (in bytes) of the parameters block, including this header
+ */
+struct rkisp1_ext_params_block_header {
+	__u32 type;
+	__u32 state;
+	__u64 size;
+};
+
+/**
+ * struct rkisp1_ext_params_bls_config - RkISP1 extensible params BLS config
+ *
+ * RkISP1 extensible parameters Black Level Subtraction configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @bls_config: Black Level Subtraction configuration, see
+ *		:c:type:`rkisp1_cif_isp_bls_config`
+ */
+struct rkisp1_ext_params_bls_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_bls_config bls_config;
+};
+
+/**
+ * struct rkisp1_ext_params_dpcc_config - RkISP1 extensible params DPCC config
+ *
+ * RkISP1 extensible parameters Defective Pixel Cluster Correction configuration
+ * block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @dpcc_config: Defective Pixel Cluster Correction configuration, see
+ *		 :c:type:`rkisp1_cif_isp_dpcc_config`
+ */
+struct rkisp1_ext_params_dpcc_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_dpcc_config dpcc_config;
+};
+
+/**
+ * struct rkisp1_ext_params_sdg_config - RkISP1 extensible params SDG config
+ *
+ * RkISP1 extensible parameters Sensor Degamma configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @sdg_config: Sensor Degamma configuration, see
+ *		:c:type:`rkisp1_cif_isp_sdg_config`
+ */
+struct rkisp1_ext_params_sdg_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_sdg_config sdg_config;
+};
+
+/**
+ * struct rkisp1_ext_params_lsc_config - RkISP1 extensible params LSC config
+ *
+ * RkISP1 extensible parameters Lens Shading Correction configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @lsc_config: Lens Shading Correction configuration, see
+ *		:c:type:`rkisp1_cif_isp_lsc_config`
+ */
+struct rkisp1_ext_params_lsc_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_lsc_config lsc_config;
+};
+
+/**
+ * struct rkisp1_ext_params_awb_gain_config - RkISP1 extensible params AWB
+ *					      gain config
+ *
+ * RkISP1 extensible parameters Auto-White Balance Gains configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @awb_config: Auto-White Balance Gains configuration, see
+ *		:c:type:`rkisp1_cif_isp_awb_gain_config`
+ */
+struct rkisp1_ext_params_awb_gain_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_awb_gain_config awb_config;
+};
+
+/**
+ * struct rkisp1_ext_params_flt_config - RkISP1 extensible params FLT config
+ *
+ * RkISP1 extensible parameters Filter configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @flt_config: Filter configuration, see
+ *		:c:type:`rkisp1_cif_isp_flt_config`
+ */
+struct rkisp1_ext_params_flt_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_flt_config flt_config;
+};
+
+/**
+ * struct rkisp1_ext_params_bdm_config - RkISP1 extensible params BDM config
+ *
+ * RkISP1 extensible parameters Demosaicing configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @bdm_config: Demosaicing configuration, see
+ *		:c:type:`rkisp1_cif_isp_bdm_config`
+ */
+struct rkisp1_ext_params_bdm_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_bdm_config bdm_config;
+};
+
+/**
+ * struct rkisp1_ext_params_ctk_config - RkISP1 extensible params CTK config
+ *
+ * RkISP1 extensible parameters Cross-Talk configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @ctk_config: Cross-Talk configuration, see
+ *		:c:type:`rkisp1_cif_isp_ctk_config`
+ */
+struct rkisp1_ext_params_ctk_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_ctk_config ctk_config;
+};
+
+/**
+ * struct rkisp1_ext_params_goc_config - RkISP1 extensible params GOC config
+ *
+ * RkISP1 extensible parameters Gamma-Out configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @goc_config: Gamma-Out configuration, see
+ *		:c:type:`rkisp1_cif_isp_goc_config`
+ */
+struct rkisp1_ext_params_goc_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_goc_config goc_config;
+};
+
+/**
+ * struct rkisp1_ext_params_dpf_config - RkISP1 extensible params DPF config
+ *
+ * RkISP1 extensible parameters De-noise Pre-Filter configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @dpf_config: De-noise Pre-Filter configuration, see
+ *		:c:type:`rkisp1_cif_isp_dpf_config`
+ */
+struct rkisp1_ext_params_dpf_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_dpf_config dpf_config;
+};
+
+/**
+ * struct rkisp1_ext_params_dpf_strength_config - RkISP1 extensible params DPF
+ *						  strength config
+ *
+ * RkISP1 extensible parameters De-noise Pre-Filter strength configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @dpf_strength_config: De-noise Pre-Filter strength configuration, see
+ *			 :c:type:`rkisp1_cif_isp_dpf_strength_config`
+ */
+struct rkisp1_ext_params_dpf_strength_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_dpf_strength_config dpf_strength_config;
+};
+
+/**
+ * struct rkisp1_ext_params_cproc_config - RkISP1 extensible params CPROC config
+ *
+ * RkISP1 extensible parameters Color Processing configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @cproc_config: Color processing configuration, see
+ *		  :c:type:`rkisp1_cif_isp_cproc_config`
+ */
+struct rkisp1_ext_params_cproc_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_cproc_config cproc_config;
+};
+
+/**
+ * struct rkisp1_ext_params_ie_config - RkISP1 extensible params IE config
+ *
+ * RkISP1 extensible parameters Image Effect configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @ie_config: Image Effect configuration, see
+ *	       :c:type:`rkisp1_cif_isp_ie_config`
+ */
+struct rkisp1_ext_params_ie_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_ie_config ie_config;
+};
+
+/**
+ * struct rkisp1_ext_params_awb_meas_config - RkISP1 extensible params AWB
+ *					      Meas config
+ *
+ * RkISP1 extensible parameters Auto-White Balance Measurement configuration
+ * block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @awb_meas_config: Auto-White Balance measure configuration, see
+ *		     :c:type:`rkisp1_cif_isp_awb_meas_config`
+ */
+struct rkisp1_ext_params_awb_meas_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_awb_meas_config awb_meas_config;
+};
+
+/**
+ * struct rkisp1_ext_params_hst_config - RkISP1 extensible params Histogram config
+ *
+ * RkISP1 extensible parameters Histogram statistics configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @hst_config: Histogram statistics configuration, see
+ *		:c:type:`rkisp1_cif_isp_hst_config`
+ */
+struct rkisp1_ext_params_hst_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_hst_config hst_config;
+};
+
+/**
+ * struct rkisp1_ext_params_aec_config - RkISP1 extensible params AEC config
+ *
+ * RkISP1 extensible parameters Auto-Exposure statistics configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @aec_config: Auto-Exposure statistics configuration, see
+ *		:c:type:`rkisp1_cif_isp_aec_config`
+ */
+struct rkisp1_ext_params_aec_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_aec_config aec_config;
+};
+
+/**
+ * struct rkisp1_ext_params_afc_config - RkISP1 extensible params AFC config
+ *
+ * RkISP1 extensible parameters Auto-Focus statistics configuration block
+ *
+ * @header: The RkISP1 extensible parameters header, see
+ *	    :c:type:`rkisp1_ext_params_block_header`
+ * @afc_config: Auto-Focus statistics configuration, see
+ *		:c:type:`rkisp1_cif_isp_afc_config`
+ */
+struct rkisp1_ext_params_afc_config {
+	struct rkisp1_ext_params_block_header header;
+	struct rkisp1_cif_isp_afc_config afc_config;
+};
+
+#define RKISP1_EXT_PARAMS_MAX_SIZE					\
+	(sizeof(struct rkisp1_ext_params_bls_config)			+\
+	sizeof(struct rkisp1_ext_params_dpcc_config)			+\
+	sizeof(struct rkisp1_ext_params_sdg_config)			+\
+	sizeof(struct rkisp1_ext_params_lsc_config)			+\
+	sizeof(struct rkisp1_ext_params_awb_gain_config)		+\
+	sizeof(struct rkisp1_ext_params_flt_config)			+\
+	sizeof(struct rkisp1_ext_params_bdm_config)			+\
+	sizeof(struct rkisp1_ext_params_ctk_config)			+\
+	sizeof(struct rkisp1_ext_params_goc_config)			+\
+	sizeof(struct rkisp1_ext_params_dpf_config)			+\
+	sizeof(struct rkisp1_ext_params_dpf_strength_config)		+\
+	sizeof(struct rkisp1_ext_params_cproc_config)			+\
+	sizeof(struct rkisp1_ext_params_ie_config)			+\
+	sizeof(struct rkisp1_ext_params_awb_meas_config)		+\
+	sizeof(struct rkisp1_ext_params_hst_config)			+\
+	sizeof(struct rkisp1_ext_params_aec_config)			+\
+	sizeof(struct rkisp1_ext_params_afc_config))
+
+/**
+ * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version
+ *
+ * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters
+ */
+enum rksip1_ext_param_buffer_version {
+	RKISP1_EXT_PARAM_BUFFER_V1 = 1,
+};
+
+/**
+ * struct rkisp1_ext_params_cfg - RkISP1 extensible parameters configuration
+ *
+ * This struct contains the configuration parameters of the RkISP1 ISP
+ * algorithms, serialized by userspace into a data buffer. Each configuration
+ * parameter block is represented by a block-specific structure which contains a
+ * :c:type:`rkisp1_ext_params_block_header` entry as first member. Userspace
+ * populates the @data buffer with configuration parameters for the blocks that
+ * it intends to configure. As a consequence, the data buffer effective size
+ * changes according to the number of ISP blocks that userspace intends to
+ * configure and is set by userspace in the @total_size field.
+ *
+ * The parameters buffer is versioned by the @version field to allow modifying
+ * and extending its definition. Userspace shall populate the @version field to
+ * inform the driver about the version it intends to use. The driver will parse
+ * and handle the @data buffer according to the data layout specific to the
+ * indicated version and return an error if the desired version is not
+ * supported.
+ *
+ * For each ISP block that userspace wants to configure, a block-specific
+ * structure is appended to the @data buffer, one after the other without gaps
+ * in between nor overlaps. Userspace shall populate the @total_size field with
+ * the effective size, in bytes, of the @data buffer.
+ *
+ * The expected memory layout of the parameters buffer is::
+ *
+ *	+-------------------- struct rkisp1_ext_params_cfg -------------------+
+ *	| version = RKISP_EXT_PARAMS_BUFFER_V1;                               |
+ *	| total_size = sizeof(struct rkisp1_ext_params_bls_config)            |
+ *	|              sizeof(struct rkisp1_ext_params_dpcc_config);          |
+ *	| +------------------------- data  ---------------------------------+ |
+ *	| | +------------- struct rkisp1_ext_params_bls_config -----------+ | |
+ *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
+ *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS;                | | | |
+ *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
+ *	| | | | size = sizeof(struct rkisp1_ext_params_bls_config);     | | | |
+ *	| | | +---------------------------------------------------------+ | | |
+ *	| | | +---------- struct rkisp1_cif_isp_bls_config -------------+ | | |
+ *	| | | | enable_auto = 0;                                        | | | |
+ *	| | | | fixed_val.r = 256;                                      | | | |
+ *	| | | | fixed_val.gr = 256;                                     | | | |
+ *	| | | | fixed_val.gb = 256;                                     | | | |
+ *	| | | | fixed_val.b = 256;                                      | | | |
+ *	| | | +---------------------------------------------------------+ | | |
+ *	| | +------------ struct rkisp1_ext_params_dpcc_config -----------+ | |
+ *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
+ *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC;               | | | |
+ *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
+ *	| | | | size = sizeof(struct rkisp1_ext_params_dpcc_config);    | | | |
+ *	| | | +---------------------------------------------------------+ | | |
+ *	| | | +---------- struct rkisp1_cif_isp_dpcc_config ------------+ | | |
+ *	| | | | mode = RKISP1_CIF_ISP_DPCC_MODE_STAGE1_ENABLE;          | | | |
+ *	| | | | output_mode =                                           | | | |
+ *	| | | |   RKISP1_CIF_ISP_DPCC_OUTPUT_MODE_STAGE1_INCL_G_CENTER; | | | |
+ *	| | | | set_use = ... ;                                         | | | |
+ *	| | | | ...  = ... ;                                            | | | |
+ *	| | | +---------------------------------------------------------+ | | |
+ *	| | +-------------------------------------------------------------+ | |
+ *	| +-----------------------------------------------------------------+ |
+ *	+---------------------------------------------------------------------+
+ *
+ * @version: The RkISP1 extensible parameters buffer version, see
+ *	     :c:type:`rksip1_ext_param_buffer_version`
+ * @total_size: The RkISP1 configuration data effective size, excluding this
+ *		header
+ * @data: The RkISP1 extensible configuration data blocks
+ */
+struct rkisp1_ext_params_cfg {
+	__u32 version;
+	__u64 total_size;
+	__u8 data[RKISP1_EXT_PARAMS_MAX_SIZE];
+};
+
 #endif /* _UAPI_RKISP1_CONFIG_H */
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format
  2024-06-05 16:54 ` [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format Jacopo Mondi
@ 2024-06-12 10:02   ` Dan Scally
  2024-06-12 12:56     ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Dan Scally @ 2024-06-12 10:02 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo - thanks for the patchset

On 05/06/2024 17:54, Jacopo Mondi wrote:
> Add to the rkisp1-config.h header data types and documentation of
> the extensible parameters format.
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>   include/uapi/linux/rkisp1-config.h | 482 +++++++++++++++++++++++++++++
>   1 file changed, 482 insertions(+)
>
> diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h
> index 6eeaf8bf2362..9c93e536f270 100644
> --- a/include/uapi/linux/rkisp1-config.h
> +++ b/include/uapi/linux/rkisp1-config.h
> @@ -996,4 +996,486 @@ struct rkisp1_stat_buffer {
>   	struct rkisp1_cif_isp_stat params;
>   };
>   
> +/*---------- PART3: Extensible Configuration Parameters  ------------*/
> +
> +/**
> + * enum rkisp1_ext_params_block_type - RkISP1 extensible params block type
> + *
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS: Black level subtraction
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC: Defect pixel cluster correction
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG: Sensor de-gamma
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS: Auto white balance gains
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT: ISP filtering
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM: Bayer de-mosaic
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK: Cross-talk correction
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC: Gamma out correction
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF: De-noise pre-filter
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT: De-noise pre-filter strength
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC: Color processing
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_IE: Image effects
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC: Lens shading correction
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS: Auto white balance statistics
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS: Histogram statistics
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS: Auto exposure statistics
> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS: Auto-focus statistics
> + */
> +enum rkisp1_ext_params_block_type {
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_IE,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS,
> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL,
> +};


Laurent suggested referencing the enum value in the comments for each block in his review of the C55 
series, and I think that that's a good idea - can we do that?


Otherwise, looks good to me:

Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>

> +
> +/**
> + * enum rkisp1_ext_params_block_state - RkISP1 extensible parameter block enable
> + *					state flags
> + *
> + * @RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE: Do not change the block power state
> + * @RKISP1_EXT_PARAMS_BLOCK_DISABLE: Disable the HW block
> + * @RKISP1_EXT_PARAMS_BLOCK_ENABLE: Enable the HW block
> + */
> +enum rkisp1_ext_params_block_state {
> +	RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE,
> +	RKISP1_EXT_PARAMS_BLOCK_DISABLE,
> +	RKISP1_EXT_PARAMS_BLOCK_ENABLE,
> +};
> +
> +/**
> + * struct rkisp1_ext_params_block_header - RkISP1 extensible parameter block
> + *					   header
> + *
> + * This structure represents the common part of all the ISP configuration
> + * blocks. Each parameters block shall embed an instance of this structure type
> + * as its first member, followed by the block-specific configuration data. The
> + * driver inspects this common header to discern the block type and its size and
> + * properly handle the block content by casting it to the correct block-specific
> + * type.
> + *
> + * The @type field is one of the values enumerated by
> + * :c:type:`rkisp1_ext_params_block_type` and specifies how the data should be
> + * interpreted by the driver. The @size field specifies the size of the
> + * parameters block and is used by the driver for validation purposes.
> + *
> + * The @state field specifies if the ISP block power state should be changed,
> + * and, if it has to, if it has to be enabled to disabled. The possible
> + * states are enumerated by :c:type:`rkisp1_ext_params_block_state`.
> + * When userspace needs to configure and enable an ISP block it shall fully
> + * populate the block configuration and the @state flag shall be set to
> + * RKISP1_EXT_PARAMS_BLOCK_ENABLE. When userspace simply wants to disable the
> + * ISP block the @state flag shall be set to RKISP1_EXT_PARAMS_BLOCK_DISABLE. If
> + * a new configuration of an ISP block should be applied but the power state
> + * doesn't need to be changed, userspace shall fully populate the ISP block
> + * configuration and the @state flag shall be set to
> + * RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE.
> + *
> + * Userspace is responsible for correctly populating the parameters block header
> + * fields (@type, @state and @size) and correctly populate the block-specific
> + * parameters.
> + *
> + * For example:
> + *
> + * .. code-block:: c
> + *
> + *	void populate_bls(struct rkisp1_ext_params_block_header *block) {
> + *		struct rkisp1_ext_params_bls_config *bls =
> + *			(struct rkisp1_ext_params_bls_config *)block;
> + *
> + *		block->header.type = RKISP1_EXT_PARAMS_BLOCK_ID_BLS;
> + *		block->header.state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;
> + *		block->header.size = sizeof(struct rkisp1_ext_params_bls_config);
> + *
> + *		bls->bls_config.enable_auto = 0;
> + *		bls->bls_config.fixed_val.r = blackLevelRed_;
> + *		bls->bls_config.fixed_val.gr = blackLevelGreenR_;
> + *		bls->bls_config.fixed_val.gb = blackLevelGreenB_;
> + *		bls->bls_config.fixed_val.b = blackLevelBlue_;
> + *	}
> + *
> + * @type: The parameters block type, see
> + *	  :c:type:`rkisp1_ext_params_block_type`
> + * @state: The block enable state flag, see
> + *	   :c:type:`rkisp1_ext_params_block_state`
> + * @size: Size (in bytes) of the parameters block, including this header
> + */
> +struct rkisp1_ext_params_block_header {
> +	__u32 type;
> +	__u32 state;
> +	__u64 size;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_bls_config - RkISP1 extensible params BLS config
> + *
> + * RkISP1 extensible parameters Black Level Subtraction configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @bls_config: Black Level Subtraction configuration, see
> + *		:c:type:`rkisp1_cif_isp_bls_config`
> + */
> +struct rkisp1_ext_params_bls_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_bls_config bls_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_dpcc_config - RkISP1 extensible params DPCC config
> + *
> + * RkISP1 extensible parameters Defective Pixel Cluster Correction configuration
> + * block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @dpcc_config: Defective Pixel Cluster Correction configuration, see
> + *		 :c:type:`rkisp1_cif_isp_dpcc_config`
> + */
> +struct rkisp1_ext_params_dpcc_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_dpcc_config dpcc_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_sdg_config - RkISP1 extensible params SDG config
> + *
> + * RkISP1 extensible parameters Sensor Degamma configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @sdg_config: Sensor Degamma configuration, see
> + *		:c:type:`rkisp1_cif_isp_sdg_config`
> + */
> +struct rkisp1_ext_params_sdg_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_sdg_config sdg_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_lsc_config - RkISP1 extensible params LSC config
> + *
> + * RkISP1 extensible parameters Lens Shading Correction configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @lsc_config: Lens Shading Correction configuration, see
> + *		:c:type:`rkisp1_cif_isp_lsc_config`
> + */
> +struct rkisp1_ext_params_lsc_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_lsc_config lsc_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_awb_gain_config - RkISP1 extensible params AWB
> + *					      gain config
> + *
> + * RkISP1 extensible parameters Auto-White Balance Gains configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @awb_config: Auto-White Balance Gains configuration, see
> + *		:c:type:`rkisp1_cif_isp_awb_gain_config`
> + */
> +struct rkisp1_ext_params_awb_gain_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_awb_gain_config awb_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_flt_config - RkISP1 extensible params FLT config
> + *
> + * RkISP1 extensible parameters Filter configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @flt_config: Filter configuration, see
> + *		:c:type:`rkisp1_cif_isp_flt_config`
> + */
> +struct rkisp1_ext_params_flt_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_flt_config flt_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_bdm_config - RkISP1 extensible params BDM config
> + *
> + * RkISP1 extensible parameters Demosaicing configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @bdm_config: Demosaicing configuration, see
> + *		:c:type:`rkisp1_cif_isp_bdm_config`
> + */
> +struct rkisp1_ext_params_bdm_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_bdm_config bdm_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_ctk_config - RkISP1 extensible params CTK config
> + *
> + * RkISP1 extensible parameters Cross-Talk configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @ctk_config: Cross-Talk configuration, see
> + *		:c:type:`rkisp1_cif_isp_ctk_config`
> + */
> +struct rkisp1_ext_params_ctk_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_ctk_config ctk_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_goc_config - RkISP1 extensible params GOC config
> + *
> + * RkISP1 extensible parameters Gamma-Out configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @goc_config: Gamma-Out configuration, see
> + *		:c:type:`rkisp1_cif_isp_goc_config`
> + */
> +struct rkisp1_ext_params_goc_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_goc_config goc_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_dpf_config - RkISP1 extensible params DPF config
> + *
> + * RkISP1 extensible parameters De-noise Pre-Filter configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @dpf_config: De-noise Pre-Filter configuration, see
> + *		:c:type:`rkisp1_cif_isp_dpf_config`
> + */
> +struct rkisp1_ext_params_dpf_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_dpf_config dpf_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_dpf_strength_config - RkISP1 extensible params DPF
> + *						  strength config
> + *
> + * RkISP1 extensible parameters De-noise Pre-Filter strength configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @dpf_strength_config: De-noise Pre-Filter strength configuration, see
> + *			 :c:type:`rkisp1_cif_isp_dpf_strength_config`
> + */
> +struct rkisp1_ext_params_dpf_strength_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_dpf_strength_config dpf_strength_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_cproc_config - RkISP1 extensible params CPROC config
> + *
> + * RkISP1 extensible parameters Color Processing configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @cproc_config: Color processing configuration, see
> + *		  :c:type:`rkisp1_cif_isp_cproc_config`
> + */
> +struct rkisp1_ext_params_cproc_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_cproc_config cproc_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_ie_config - RkISP1 extensible params IE config
> + *
> + * RkISP1 extensible parameters Image Effect configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @ie_config: Image Effect configuration, see
> + *	       :c:type:`rkisp1_cif_isp_ie_config`
> + */
> +struct rkisp1_ext_params_ie_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_ie_config ie_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_awb_meas_config - RkISP1 extensible params AWB
> + *					      Meas config
> + *
> + * RkISP1 extensible parameters Auto-White Balance Measurement configuration
> + * block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @awb_meas_config: Auto-White Balance measure configuration, see
> + *		     :c:type:`rkisp1_cif_isp_awb_meas_config`
> + */
> +struct rkisp1_ext_params_awb_meas_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_awb_meas_config awb_meas_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_hst_config - RkISP1 extensible params Histogram config
> + *
> + * RkISP1 extensible parameters Histogram statistics configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @hst_config: Histogram statistics configuration, see
> + *		:c:type:`rkisp1_cif_isp_hst_config`
> + */
> +struct rkisp1_ext_params_hst_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_hst_config hst_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_aec_config - RkISP1 extensible params AEC config
> + *
> + * RkISP1 extensible parameters Auto-Exposure statistics configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @aec_config: Auto-Exposure statistics configuration, see
> + *		:c:type:`rkisp1_cif_isp_aec_config`
> + */
> +struct rkisp1_ext_params_aec_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_aec_config aec_config;
> +};
> +
> +/**
> + * struct rkisp1_ext_params_afc_config - RkISP1 extensible params AFC config
> + *
> + * RkISP1 extensible parameters Auto-Focus statistics configuration block
> + *
> + * @header: The RkISP1 extensible parameters header, see
> + *	    :c:type:`rkisp1_ext_params_block_header`
> + * @afc_config: Auto-Focus statistics configuration, see
> + *		:c:type:`rkisp1_cif_isp_afc_config`
> + */
> +struct rkisp1_ext_params_afc_config {
> +	struct rkisp1_ext_params_block_header header;
> +	struct rkisp1_cif_isp_afc_config afc_config;
> +};
> +
> +#define RKISP1_EXT_PARAMS_MAX_SIZE					\
> +	(sizeof(struct rkisp1_ext_params_bls_config)			+\
> +	sizeof(struct rkisp1_ext_params_dpcc_config)			+\
> +	sizeof(struct rkisp1_ext_params_sdg_config)			+\
> +	sizeof(struct rkisp1_ext_params_lsc_config)			+\
> +	sizeof(struct rkisp1_ext_params_awb_gain_config)		+\
> +	sizeof(struct rkisp1_ext_params_flt_config)			+\
> +	sizeof(struct rkisp1_ext_params_bdm_config)			+\
> +	sizeof(struct rkisp1_ext_params_ctk_config)			+\
> +	sizeof(struct rkisp1_ext_params_goc_config)			+\
> +	sizeof(struct rkisp1_ext_params_dpf_config)			+\
> +	sizeof(struct rkisp1_ext_params_dpf_strength_config)		+\
> +	sizeof(struct rkisp1_ext_params_cproc_config)			+\
> +	sizeof(struct rkisp1_ext_params_ie_config)			+\
> +	sizeof(struct rkisp1_ext_params_awb_meas_config)		+\
> +	sizeof(struct rkisp1_ext_params_hst_config)			+\
> +	sizeof(struct rkisp1_ext_params_aec_config)			+\
> +	sizeof(struct rkisp1_ext_params_afc_config))
> +
> +/**
> + * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version
> + *
> + * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters
> + */
> +enum rksip1_ext_param_buffer_version {
> +	RKISP1_EXT_PARAM_BUFFER_V1 = 1,
> +};
> +
> +/**
> + * struct rkisp1_ext_params_cfg - RkISP1 extensible parameters configuration
> + *
> + * This struct contains the configuration parameters of the RkISP1 ISP
> + * algorithms, serialized by userspace into a data buffer. Each configuration
> + * parameter block is represented by a block-specific structure which contains a
> + * :c:type:`rkisp1_ext_params_block_header` entry as first member. Userspace
> + * populates the @data buffer with configuration parameters for the blocks that
> + * it intends to configure. As a consequence, the data buffer effective size
> + * changes according to the number of ISP blocks that userspace intends to
> + * configure and is set by userspace in the @total_size field.
> + *
> + * The parameters buffer is versioned by the @version field to allow modifying
> + * and extending its definition. Userspace shall populate the @version field to
> + * inform the driver about the version it intends to use. The driver will parse
> + * and handle the @data buffer according to the data layout specific to the
> + * indicated version and return an error if the desired version is not
> + * supported.
> + *
> + * For each ISP block that userspace wants to configure, a block-specific
> + * structure is appended to the @data buffer, one after the other without gaps
> + * in between nor overlaps. Userspace shall populate the @total_size field with
> + * the effective size, in bytes, of the @data buffer.
> + *
> + * The expected memory layout of the parameters buffer is::
> + *
> + *	+-------------------- struct rkisp1_ext_params_cfg -------------------+
> + *	| version = RKISP_EXT_PARAMS_BUFFER_V1;                               |
> + *	| total_size = sizeof(struct rkisp1_ext_params_bls_config)            |
> + *	|              sizeof(struct rkisp1_ext_params_dpcc_config);          |
> + *	| +------------------------- data  ---------------------------------+ |
> + *	| | +------------- struct rkisp1_ext_params_bls_config -----------+ | |
> + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS;                | | | |
> + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> + *	| | | | size = sizeof(struct rkisp1_ext_params_bls_config);     | | | |
> + *	| | | +---------------------------------------------------------+ | | |
> + *	| | | +---------- struct rkisp1_cif_isp_bls_config -------------+ | | |
> + *	| | | | enable_auto = 0;                                        | | | |
> + *	| | | | fixed_val.r = 256;                                      | | | |
> + *	| | | | fixed_val.gr = 256;                                     | | | |
> + *	| | | | fixed_val.gb = 256;                                     | | | |
> + *	| | | | fixed_val.b = 256;                                      | | | |
> + *	| | | +---------------------------------------------------------+ | | |
> + *	| | +------------ struct rkisp1_ext_params_dpcc_config -----------+ | |
> + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC;               | | | |
> + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> + *	| | | | size = sizeof(struct rkisp1_ext_params_dpcc_config);    | | | |
> + *	| | | +---------------------------------------------------------+ | | |
> + *	| | | +---------- struct rkisp1_cif_isp_dpcc_config ------------+ | | |
> + *	| | | | mode = RKISP1_CIF_ISP_DPCC_MODE_STAGE1_ENABLE;          | | | |
> + *	| | | | output_mode =                                           | | | |
> + *	| | | |   RKISP1_CIF_ISP_DPCC_OUTPUT_MODE_STAGE1_INCL_G_CENTER; | | | |
> + *	| | | | set_use = ... ;                                         | | | |
> + *	| | | | ...  = ... ;                                            | | | |
> + *	| | | +---------------------------------------------------------+ | | |
> + *	| | +-------------------------------------------------------------+ | |
> + *	| +-----------------------------------------------------------------+ |
> + *	+---------------------------------------------------------------------+
> + *
> + * @version: The RkISP1 extensible parameters buffer version, see
> + *	     :c:type:`rksip1_ext_param_buffer_version`
> + * @total_size: The RkISP1 configuration data effective size, excluding this
> + *		header
> + * @data: The RkISP1 extensible configuration data blocks
> + */
> +struct rkisp1_ext_params_cfg {
> +	__u32 version;
> +	__u64 total_size;
> +	__u8 data[RKISP1_EXT_PARAMS_MAX_SIZE];
> +};
> +
>   #endif /* _UAPI_RKISP1_CONFIG_H */

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format
  2024-06-12 10:02   ` Dan Scally
@ 2024-06-12 12:56     ` Laurent Pinchart
  2024-06-12 13:49       ` Dan Scally
  2024-06-19 12:44       ` Jacopo Mondi
  0 siblings, 2 replies; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 12:56 UTC (permalink / raw)
  To: Dan Scally
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 12, 2024 at 11:02:58AM +0100, Daniel Scally wrote:
> Hi Jacopo - thanks for the patchset
> 
> On 05/06/2024 17:54, Jacopo Mondi wrote:
> > Add to the rkisp1-config.h header data types and documentation of
> > the extensible parameters format.
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >   include/uapi/linux/rkisp1-config.h | 482 +++++++++++++++++++++++++++++
> >   1 file changed, 482 insertions(+)
> >
> > diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h
> > index 6eeaf8bf2362..9c93e536f270 100644
> > --- a/include/uapi/linux/rkisp1-config.h
> > +++ b/include/uapi/linux/rkisp1-config.h
> > @@ -996,4 +996,486 @@ struct rkisp1_stat_buffer {
> >   	struct rkisp1_cif_isp_stat params;
> >   };
> >   
> > +/*---------- PART3: Extensible Configuration Parameters  ------------*/
> > +
> > +/**
> > + * enum rkisp1_ext_params_block_type - RkISP1 extensible params block type
> > + *
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS: Black level subtraction
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC: Defect pixel cluster correction
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG: Sensor de-gamma
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS: Auto white balance gains
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT: ISP filtering
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM: Bayer de-mosaic
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK: Cross-talk correction
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC: Gamma out correction
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF: De-noise pre-filter
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT: De-noise pre-filter strength
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC: Color processing
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_IE: Image effects
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC: Lens shading correction
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS: Auto white balance statistics
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS: Histogram statistics
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS: Auto exposure statistics
> > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS: Auto-focus statistics
> > + */
> > +enum rkisp1_ext_params_block_type {
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_IE,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS,
> > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL,
> > +};
> 
> Laurent suggested referencing the enum value in the comments for each
> block in his review of the C55 series, and I think that that's a good
> idea - can we do that?

I would like that too.

> Otherwise, looks good to me:
> 
> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> 
> > +
> > +/**
> > + * enum rkisp1_ext_params_block_state - RkISP1 extensible parameter block enable
> > + *					state flags
> > + *
> > + * @RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE: Do not change the block power state

I wouldn't call that "power" state, I don't think it's about power.

> > + * @RKISP1_EXT_PARAMS_BLOCK_DISABLE: Disable the HW block
> > + * @RKISP1_EXT_PARAMS_BLOCK_ENABLE: Enable the HW block
> > + */
> > +enum rkisp1_ext_params_block_state {
> > +	RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE,
> > +	RKISP1_EXT_PARAMS_BLOCK_DISABLE,
> > +	RKISP1_EXT_PARAMS_BLOCK_ENABLE,
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_block_header - RkISP1 extensible parameter block
> > + *					   header
> > + *
> > + * This structure represents the common part of all the ISP configuration
> > + * blocks. Each parameters block shall embed an instance of this structure type
> > + * as its first member, followed by the block-specific configuration data. The
> > + * driver inspects this common header to discern the block type and its size and
> > + * properly handle the block content by casting it to the correct block-specific
> > + * type.
> > + *
> > + * The @type field is one of the values enumerated by
> > + * :c:type:`rkisp1_ext_params_block_type` and specifies how the data should be
> > + * interpreted by the driver. The @size field specifies the size of the
> > + * parameters block and is used by the driver for validation purposes.
> > + *
> > + * The @state field specifies if the ISP block power state should be changed,
> > + * and, if it has to, if it has to be enabled to disabled. The possible
> > + * states are enumerated by :c:type:`rkisp1_ext_params_block_state`.
> > + * When userspace needs to configure and enable an ISP block it shall fully
> > + * populate the block configuration and the @state flag shall be set to
> > + * RKISP1_EXT_PARAMS_BLOCK_ENABLE. When userspace simply wants to disable the
> > + * ISP block the @state flag shall be set to RKISP1_EXT_PARAMS_BLOCK_DISABLE.

You should document here what happens to the rest of the parameters in
the block in that case. I think they should be ignored by the driver,
and possibly set to 0 by userspace.

Another option when disabling a block would be to include the header
only, with the size field set to the header size, and not include the
rest of the block contents. It's a bit pointless to include data that
the kernel won't use. While that would be my preference from an API
point of view (at least until someone points out to use cases that would
benefit from a different option), I haven't checked what it would imply
from a userspace and kernelspace implementation point of view. I could
agree to keeping the data part of the block in for disabled blocks if
omitting it would result in implementation issues.

> If
> > + * a new configuration of an ISP block should be applied but the power state
> > + * doesn't need to be changed, userspace shall fully populate the ISP block
> > + * configuration and the @state flag shall be set to
> > + * RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE.

I don't think we need RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE for that. The
state can just be set to @RKISP1_EXT_PARAMS_BLOCK_ENABLE.

> > + *
> > + * Userspace is responsible for correctly populating the parameters block header
> > + * fields (@type, @state and @size) and correctly populate the block-specific
> > + * parameters.
> > + *
> > + * For example:
> > + *
> > + * .. code-block:: c
> > + *
> > + *	void populate_bls(struct rkisp1_ext_params_block_header *block) {
> > + *		struct rkisp1_ext_params_bls_config *bls =
> > + *			(struct rkisp1_ext_params_bls_config *)block;
> > + *
> > + *		block->header.type = RKISP1_EXT_PARAMS_BLOCK_ID_BLS;
> > + *		block->header.state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;
> > + *		block->header.size = sizeof(struct rkisp1_ext_params_bls_config);
> > + *
> > + *		bls->bls_config.enable_auto = 0;
> > + *		bls->bls_config.fixed_val.r = blackLevelRed_;
> > + *		bls->bls_config.fixed_val.gr = blackLevelGreenR_;
> > + *		bls->bls_config.fixed_val.gb = blackLevelGreenB_;
> > + *		bls->bls_config.fixed_val.b = blackLevelBlue_;
> > + *	}
> > + *
> > + * @type: The parameters block type, see
> > + *	  :c:type:`rkisp1_ext_params_block_type`
> > + * @state: The block enable state flag, see
> > + *	   :c:type:`rkisp1_ext_params_block_state`
> > + * @size: Size (in bytes) of the parameters block, including this header
> > + */
> > +struct rkisp1_ext_params_block_header {
> > +	__u32 type;
> > +	__u32 state;
> > +	__u64 size;

I think a 32-bit size would be move than enough. The header should
however be 64-bit aligned.

> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_bls_config - RkISP1 extensible params BLS config
> > + *
> > + * RkISP1 extensible parameters Black Level Subtraction configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @bls_config: Black Level Subtraction configuration, see
> > + *		:c:type:`rkisp1_cif_isp_bls_config`
> > + */
> > +struct rkisp1_ext_params_bls_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_bls_config bls_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_dpcc_config - RkISP1 extensible params DPCC config
> > + *
> > + * RkISP1 extensible parameters Defective Pixel Cluster Correction configuration
> > + * block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @dpcc_config: Defective Pixel Cluster Correction configuration, see
> > + *		 :c:type:`rkisp1_cif_isp_dpcc_config`
> > + */
> > +struct rkisp1_ext_params_dpcc_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_dpcc_config dpcc_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_sdg_config - RkISP1 extensible params SDG config
> > + *
> > + * RkISP1 extensible parameters Sensor Degamma configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @sdg_config: Sensor Degamma configuration, see
> > + *		:c:type:`rkisp1_cif_isp_sdg_config`
> > + */
> > +struct rkisp1_ext_params_sdg_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_sdg_config sdg_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_lsc_config - RkISP1 extensible params LSC config
> > + *
> > + * RkISP1 extensible parameters Lens Shading Correction configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @lsc_config: Lens Shading Correction configuration, see
> > + *		:c:type:`rkisp1_cif_isp_lsc_config`
> > + */
> > +struct rkisp1_ext_params_lsc_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_lsc_config lsc_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_awb_gain_config - RkISP1 extensible params AWB
> > + *					      gain config
> > + *
> > + * RkISP1 extensible parameters Auto-White Balance Gains configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @awb_config: Auto-White Balance Gains configuration, see
> > + *		:c:type:`rkisp1_cif_isp_awb_gain_config`
> > + */
> > +struct rkisp1_ext_params_awb_gain_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_awb_gain_config awb_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_flt_config - RkISP1 extensible params FLT config
> > + *
> > + * RkISP1 extensible parameters Filter configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @flt_config: Filter configuration, see
> > + *		:c:type:`rkisp1_cif_isp_flt_config`
> > + */
> > +struct rkisp1_ext_params_flt_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_flt_config flt_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_bdm_config - RkISP1 extensible params BDM config
> > + *
> > + * RkISP1 extensible parameters Demosaicing configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @bdm_config: Demosaicing configuration, see
> > + *		:c:type:`rkisp1_cif_isp_bdm_config`
> > + */
> > +struct rkisp1_ext_params_bdm_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_bdm_config bdm_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_ctk_config - RkISP1 extensible params CTK config
> > + *
> > + * RkISP1 extensible parameters Cross-Talk configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @ctk_config: Cross-Talk configuration, see
> > + *		:c:type:`rkisp1_cif_isp_ctk_config`
> > + */
> > +struct rkisp1_ext_params_ctk_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_ctk_config ctk_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_goc_config - RkISP1 extensible params GOC config
> > + *
> > + * RkISP1 extensible parameters Gamma-Out configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @goc_config: Gamma-Out configuration, see
> > + *		:c:type:`rkisp1_cif_isp_goc_config`
> > + */
> > +struct rkisp1_ext_params_goc_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_goc_config goc_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_dpf_config - RkISP1 extensible params DPF config
> > + *
> > + * RkISP1 extensible parameters De-noise Pre-Filter configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @dpf_config: De-noise Pre-Filter configuration, see
> > + *		:c:type:`rkisp1_cif_isp_dpf_config`
> > + */
> > +struct rkisp1_ext_params_dpf_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_dpf_config dpf_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_dpf_strength_config - RkISP1 extensible params DPF
> > + *						  strength config
> > + *
> > + * RkISP1 extensible parameters De-noise Pre-Filter strength configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @dpf_strength_config: De-noise Pre-Filter strength configuration, see
> > + *			 :c:type:`rkisp1_cif_isp_dpf_strength_config`
> > + */
> > +struct rkisp1_ext_params_dpf_strength_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_dpf_strength_config dpf_strength_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_cproc_config - RkISP1 extensible params CPROC config
> > + *
> > + * RkISP1 extensible parameters Color Processing configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @cproc_config: Color processing configuration, see
> > + *		  :c:type:`rkisp1_cif_isp_cproc_config`
> > + */
> > +struct rkisp1_ext_params_cproc_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_cproc_config cproc_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_ie_config - RkISP1 extensible params IE config
> > + *
> > + * RkISP1 extensible parameters Image Effect configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @ie_config: Image Effect configuration, see
> > + *	       :c:type:`rkisp1_cif_isp_ie_config`
> > + */
> > +struct rkisp1_ext_params_ie_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_ie_config ie_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_awb_meas_config - RkISP1 extensible params AWB
> > + *					      Meas config
> > + *
> > + * RkISP1 extensible parameters Auto-White Balance Measurement configuration
> > + * block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @awb_meas_config: Auto-White Balance measure configuration, see
> > + *		     :c:type:`rkisp1_cif_isp_awb_meas_config`
> > + */
> > +struct rkisp1_ext_params_awb_meas_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_awb_meas_config awb_meas_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_hst_config - RkISP1 extensible params Histogram config
> > + *
> > + * RkISP1 extensible parameters Histogram statistics configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @hst_config: Histogram statistics configuration, see
> > + *		:c:type:`rkisp1_cif_isp_hst_config`
> > + */
> > +struct rkisp1_ext_params_hst_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_hst_config hst_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_aec_config - RkISP1 extensible params AEC config
> > + *
> > + * RkISP1 extensible parameters Auto-Exposure statistics configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @aec_config: Auto-Exposure statistics configuration, see
> > + *		:c:type:`rkisp1_cif_isp_aec_config`
> > + */
> > +struct rkisp1_ext_params_aec_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_aec_config aec_config;
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_afc_config - RkISP1 extensible params AFC config
> > + *
> > + * RkISP1 extensible parameters Auto-Focus statistics configuration block
> > + *
> > + * @header: The RkISP1 extensible parameters header, see
> > + *	    :c:type:`rkisp1_ext_params_block_header`
> > + * @afc_config: Auto-Focus statistics configuration, see
> > + *		:c:type:`rkisp1_cif_isp_afc_config`
> > + */
> > +struct rkisp1_ext_params_afc_config {
> > +	struct rkisp1_ext_params_block_header header;
> > +	struct rkisp1_cif_isp_afc_config afc_config;
> > +};
> > +
> > +#define RKISP1_EXT_PARAMS_MAX_SIZE					\
> > +	(sizeof(struct rkisp1_ext_params_bls_config)			+\
> > +	sizeof(struct rkisp1_ext_params_dpcc_config)			+\
> > +	sizeof(struct rkisp1_ext_params_sdg_config)			+\
> > +	sizeof(struct rkisp1_ext_params_lsc_config)			+\
> > +	sizeof(struct rkisp1_ext_params_awb_gain_config)		+\
> > +	sizeof(struct rkisp1_ext_params_flt_config)			+\
> > +	sizeof(struct rkisp1_ext_params_bdm_config)			+\
> > +	sizeof(struct rkisp1_ext_params_ctk_config)			+\
> > +	sizeof(struct rkisp1_ext_params_goc_config)			+\
> > +	sizeof(struct rkisp1_ext_params_dpf_config)			+\
> > +	sizeof(struct rkisp1_ext_params_dpf_strength_config)		+\
> > +	sizeof(struct rkisp1_ext_params_cproc_config)			+\
> > +	sizeof(struct rkisp1_ext_params_ie_config)			+\
> > +	sizeof(struct rkisp1_ext_params_awb_meas_config)		+\
> > +	sizeof(struct rkisp1_ext_params_hst_config)			+\
> > +	sizeof(struct rkisp1_ext_params_aec_config)			+\
> > +	sizeof(struct rkisp1_ext_params_afc_config))
> > +
> > +/**
> > + * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version
> > + *
> > + * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters
> > + */
> > +enum rksip1_ext_param_buffer_version {
> > +	RKISP1_EXT_PARAM_BUFFER_V1 = 1,
> > +};
> > +
> > +/**
> > + * struct rkisp1_ext_params_cfg - RkISP1 extensible parameters configuration
> > + *
> > + * This struct contains the configuration parameters of the RkISP1 ISP
> > + * algorithms, serialized by userspace into a data buffer. Each configuration
> > + * parameter block is represented by a block-specific structure which contains a
> > + * :c:type:`rkisp1_ext_params_block_header` entry as first member. Userspace
> > + * populates the @data buffer with configuration parameters for the blocks that
> > + * it intends to configure. As a consequence, the data buffer effective size
> > + * changes according to the number of ISP blocks that userspace intends to
> > + * configure and is set by userspace in the @total_size field.
> > + *
> > + * The parameters buffer is versioned by the @version field to allow modifying
> > + * and extending its definition. Userspace shall populate the @version field to
> > + * inform the driver about the version it intends to use. The driver will parse
> > + * and handle the @data buffer according to the data layout specific to the
> > + * indicated version and return an error if the desired version is not
> > + * supported.
> > + *
> > + * For each ISP block that userspace wants to configure, a block-specific
> > + * structure is appended to the @data buffer, one after the other without gaps

I think we should align all the blocks to a 64 bits boundary. Otherwise
we'll have unaligned access issues, as well as layout differences
between 32-bitand 64-bit userspace.

> > + * in between nor overlaps. Userspace shall populate the @total_size field with
> > + * the effective size, in bytes, of the @data buffer.
> > + *
> > + * The expected memory layout of the parameters buffer is::
> > + *
> > + *	+-------------------- struct rkisp1_ext_params_cfg -------------------+
> > + *	| version = RKISP_EXT_PARAMS_BUFFER_V1;                               |
> > + *	| total_size = sizeof(struct rkisp1_ext_params_bls_config)            |
> > + *	|              sizeof(struct rkisp1_ext_params_dpcc_config);          |
> > + *	| +------------------------- data  ---------------------------------+ |
> > + *	| | +------------- struct rkisp1_ext_params_bls_config -----------+ | |
> > + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> > + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS;                | | | |
> > + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> > + *	| | | | size = sizeof(struct rkisp1_ext_params_bls_config);     | | | |
> > + *	| | | +---------------------------------------------------------+ | | |
> > + *	| | | +---------- struct rkisp1_cif_isp_bls_config -------------+ | | |
> > + *	| | | | enable_auto = 0;                                        | | | |
> > + *	| | | | fixed_val.r = 256;                                      | | | |
> > + *	| | | | fixed_val.gr = 256;                                     | | | |
> > + *	| | | | fixed_val.gb = 256;                                     | | | |
> > + *	| | | | fixed_val.b = 256;                                      | | | |
> > + *	| | | +---------------------------------------------------------+ | | |
> > + *	| | +------------ struct rkisp1_ext_params_dpcc_config -----------+ | |
> > + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> > + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC;               | | | |
> > + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> > + *	| | | | size = sizeof(struct rkisp1_ext_params_dpcc_config);    | | | |
> > + *	| | | +---------------------------------------------------------+ | | |
> > + *	| | | +---------- struct rkisp1_cif_isp_dpcc_config ------------+ | | |
> > + *	| | | | mode = RKISP1_CIF_ISP_DPCC_MODE_STAGE1_ENABLE;          | | | |
> > + *	| | | | output_mode =                                           | | | |
> > + *	| | | |   RKISP1_CIF_ISP_DPCC_OUTPUT_MODE_STAGE1_INCL_G_CENTER; | | | |
> > + *	| | | | set_use = ... ;                                         | | | |
> > + *	| | | | ...  = ... ;                                            | | | |
> > + *	| | | +---------------------------------------------------------+ | | |
> > + *	| | +-------------------------------------------------------------+ | |
> > + *	| +-----------------------------------------------------------------+ |
> > + *	+---------------------------------------------------------------------+
> > + *
> > + * @version: The RkISP1 extensible parameters buffer version, see
> > + *	     :c:type:`rksip1_ext_param_buffer_version`
> > + * @total_size: The RkISP1 configuration data effective size, excluding this
> > + *		header
> > + * @data: The RkISP1 extensible configuration data blocks
> > + */
> > +struct rkisp1_ext_params_cfg {
> > +	__u32 version;

On a 64-bit system there will be a 32-bit hole here, while on a 32-bit
system there won't. This means that a 32-bit userspace won't run on a
64-bit kernel. You can add a a __u32 reserved field to fix that.

> > +	__u64 total_size;
> > +	__u8 data[RKISP1_EXT_PARAMS_MAX_SIZE];
> > +};
> > +
> >   #endif /* _UAPI_RKISP1_CONFIG_H */

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format
  2024-06-12 12:56     ` Laurent Pinchart
@ 2024-06-12 13:49       ` Dan Scally
  2024-06-12 14:56         ` Laurent Pinchart
  2024-06-19 12:44       ` Jacopo Mondi
  1 sibling, 1 reply; 38+ messages in thread
From: Dan Scally @ 2024-06-12 13:49 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner


On 12/06/2024 13:56, Laurent Pinchart wrote:
> On Wed, Jun 12, 2024 at 11:02:58AM +0100, Daniel Scally wrote:
>> Hi Jacopo - thanks for the patchset
>>
>> On 05/06/2024 17:54, Jacopo Mondi wrote:
>>> Add to the rkisp1-config.h header data types and documentation of
>>> the extensible parameters format.
>>>
>>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>>> ---
>>>    include/uapi/linux/rkisp1-config.h | 482 +++++++++++++++++++++++++++++
>>>    1 file changed, 482 insertions(+)
>>>
>>> diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h
>>> index 6eeaf8bf2362..9c93e536f270 100644
>>> --- a/include/uapi/linux/rkisp1-config.h
>>> +++ b/include/uapi/linux/rkisp1-config.h
>>> @@ -996,4 +996,486 @@ struct rkisp1_stat_buffer {
>>>    	struct rkisp1_cif_isp_stat params;
>>>    };
>>>    
>>> +/*---------- PART3: Extensible Configuration Parameters  ------------*/
>>> +
>>> +/**
>>> + * enum rkisp1_ext_params_block_type - RkISP1 extensible params block type
>>> + *
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS: Black level subtraction
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC: Defect pixel cluster correction
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG: Sensor de-gamma
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS: Auto white balance gains
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT: ISP filtering
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM: Bayer de-mosaic
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK: Cross-talk correction
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC: Gamma out correction
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF: De-noise pre-filter
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT: De-noise pre-filter strength
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC: Color processing
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_IE: Image effects
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC: Lens shading correction
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS: Auto white balance statistics
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS: Histogram statistics
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS: Auto exposure statistics
>>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS: Auto-focus statistics
>>> + */
>>> +enum rkisp1_ext_params_block_type {
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_IE,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS,
>>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL,
>>> +};
>> Laurent suggested referencing the enum value in the comments for each
>> block in his review of the C55 series, and I think that that's a good
>> idea - can we do that?
> I would like that too.
>
>> Otherwise, looks good to me:
>>
>> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
>>
>>> +
>>> +/**
>>> + * enum rkisp1_ext_params_block_state - RkISP1 extensible parameter block enable
>>> + *					state flags
>>> + *
>>> + * @RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE: Do not change the block power state
> I wouldn't call that "power" state, I don't think it's about power.
>
>>> + * @RKISP1_EXT_PARAMS_BLOCK_DISABLE: Disable the HW block
>>> + * @RKISP1_EXT_PARAMS_BLOCK_ENABLE: Enable the HW block
>>> + */
>>> +enum rkisp1_ext_params_block_state {
>>> +	RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE,
>>> +	RKISP1_EXT_PARAMS_BLOCK_DISABLE,
>>> +	RKISP1_EXT_PARAMS_BLOCK_ENABLE,
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_block_header - RkISP1 extensible parameter block
>>> + *					   header
>>> + *
>>> + * This structure represents the common part of all the ISP configuration
>>> + * blocks. Each parameters block shall embed an instance of this structure type
>>> + * as its first member, followed by the block-specific configuration data. The
>>> + * driver inspects this common header to discern the block type and its size and
>>> + * properly handle the block content by casting it to the correct block-specific
>>> + * type.
>>> + *
>>> + * The @type field is one of the values enumerated by
>>> + * :c:type:`rkisp1_ext_params_block_type` and specifies how the data should be
>>> + * interpreted by the driver. The @size field specifies the size of the
>>> + * parameters block and is used by the driver for validation purposes.
>>> + *
>>> + * The @state field specifies if the ISP block power state should be changed,
>>> + * and, if it has to, if it has to be enabled to disabled. The possible
>>> + * states are enumerated by :c:type:`rkisp1_ext_params_block_state`.
>>> + * When userspace needs to configure and enable an ISP block it shall fully
>>> + * populate the block configuration and the @state flag shall be set to
>>> + * RKISP1_EXT_PARAMS_BLOCK_ENABLE. When userspace simply wants to disable the
>>> + * ISP block the @state flag shall be set to RKISP1_EXT_PARAMS_BLOCK_DISABLE.
> You should document here what happens to the rest of the parameters in
> the block in that case. I think they should be ignored by the driver,
> and possibly set to 0 by userspace.
>
> Another option when disabling a block would be to include the header
> only, with the size field set to the header size, and not include the
> rest of the block contents. It's a bit pointless to include data that
> the kernel won't use. While that would be my preference from an API
> point of view (at least until someone points out to use cases that would
> benefit from a different option), I haven't checked what it would imply
> from a userspace and kernelspace implementation point of view. I could
> agree to keeping the data part of the block in for disabled blocks if
> omitting it would result in implementation issues.


I forgot about this suggestion - for what it's worth I agree that it's a good one, I'm going to 
follow it for the C55

>
>> If
>>> + * a new configuration of an ISP block should be applied but the power state
>>> + * doesn't need to be changed, userspace shall fully populate the ISP block
>>> + * configuration and the @state flag shall be set to
>>> + * RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE.
> I don't think we need RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE for that. The
> state can just be set to @RKISP1_EXT_PARAMS_BLOCK_ENABLE.


I did think about that, but it avoids reading the register to check if it's already enabled or not / 
writing enable unconditionally...is the saved overhead not worth it?

>
>>> + *
>>> + * Userspace is responsible for correctly populating the parameters block header
>>> + * fields (@type, @state and @size) and correctly populate the block-specific
>>> + * parameters.
>>> + *
>>> + * For example:
>>> + *
>>> + * .. code-block:: c
>>> + *
>>> + *	void populate_bls(struct rkisp1_ext_params_block_header *block) {
>>> + *		struct rkisp1_ext_params_bls_config *bls =
>>> + *			(struct rkisp1_ext_params_bls_config *)block;
>>> + *
>>> + *		block->header.type = RKISP1_EXT_PARAMS_BLOCK_ID_BLS;
>>> + *		block->header.state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;
>>> + *		block->header.size = sizeof(struct rkisp1_ext_params_bls_config);
>>> + *
>>> + *		bls->bls_config.enable_auto = 0;
>>> + *		bls->bls_config.fixed_val.r = blackLevelRed_;
>>> + *		bls->bls_config.fixed_val.gr = blackLevelGreenR_;
>>> + *		bls->bls_config.fixed_val.gb = blackLevelGreenB_;
>>> + *		bls->bls_config.fixed_val.b = blackLevelBlue_;
>>> + *	}
>>> + *
>>> + * @type: The parameters block type, see
>>> + *	  :c:type:`rkisp1_ext_params_block_type`
>>> + * @state: The block enable state flag, see
>>> + *	   :c:type:`rkisp1_ext_params_block_state`
>>> + * @size: Size (in bytes) of the parameters block, including this header
>>> + */
>>> +struct rkisp1_ext_params_block_header {
>>> +	__u32 type;
>>> +	__u32 state;
>>> +	__u64 size;
> I think a 32-bit size would be move than enough. The header should
> however be 64-bit aligned.
>
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_bls_config - RkISP1 extensible params BLS config
>>> + *
>>> + * RkISP1 extensible parameters Black Level Subtraction configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @bls_config: Black Level Subtraction configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_bls_config`
>>> + */
>>> +struct rkisp1_ext_params_bls_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_bls_config bls_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_dpcc_config - RkISP1 extensible params DPCC config
>>> + *
>>> + * RkISP1 extensible parameters Defective Pixel Cluster Correction configuration
>>> + * block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @dpcc_config: Defective Pixel Cluster Correction configuration, see
>>> + *		 :c:type:`rkisp1_cif_isp_dpcc_config`
>>> + */
>>> +struct rkisp1_ext_params_dpcc_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_dpcc_config dpcc_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_sdg_config - RkISP1 extensible params SDG config
>>> + *
>>> + * RkISP1 extensible parameters Sensor Degamma configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @sdg_config: Sensor Degamma configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_sdg_config`
>>> + */
>>> +struct rkisp1_ext_params_sdg_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_sdg_config sdg_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_lsc_config - RkISP1 extensible params LSC config
>>> + *
>>> + * RkISP1 extensible parameters Lens Shading Correction configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @lsc_config: Lens Shading Correction configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_lsc_config`
>>> + */
>>> +struct rkisp1_ext_params_lsc_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_lsc_config lsc_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_awb_gain_config - RkISP1 extensible params AWB
>>> + *					      gain config
>>> + *
>>> + * RkISP1 extensible parameters Auto-White Balance Gains configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @awb_config: Auto-White Balance Gains configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_awb_gain_config`
>>> + */
>>> +struct rkisp1_ext_params_awb_gain_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_awb_gain_config awb_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_flt_config - RkISP1 extensible params FLT config
>>> + *
>>> + * RkISP1 extensible parameters Filter configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @flt_config: Filter configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_flt_config`
>>> + */
>>> +struct rkisp1_ext_params_flt_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_flt_config flt_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_bdm_config - RkISP1 extensible params BDM config
>>> + *
>>> + * RkISP1 extensible parameters Demosaicing configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @bdm_config: Demosaicing configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_bdm_config`
>>> + */
>>> +struct rkisp1_ext_params_bdm_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_bdm_config bdm_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_ctk_config - RkISP1 extensible params CTK config
>>> + *
>>> + * RkISP1 extensible parameters Cross-Talk configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @ctk_config: Cross-Talk configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_ctk_config`
>>> + */
>>> +struct rkisp1_ext_params_ctk_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_ctk_config ctk_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_goc_config - RkISP1 extensible params GOC config
>>> + *
>>> + * RkISP1 extensible parameters Gamma-Out configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @goc_config: Gamma-Out configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_goc_config`
>>> + */
>>> +struct rkisp1_ext_params_goc_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_goc_config goc_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_dpf_config - RkISP1 extensible params DPF config
>>> + *
>>> + * RkISP1 extensible parameters De-noise Pre-Filter configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @dpf_config: De-noise Pre-Filter configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_dpf_config`
>>> + */
>>> +struct rkisp1_ext_params_dpf_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_dpf_config dpf_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_dpf_strength_config - RkISP1 extensible params DPF
>>> + *						  strength config
>>> + *
>>> + * RkISP1 extensible parameters De-noise Pre-Filter strength configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @dpf_strength_config: De-noise Pre-Filter strength configuration, see
>>> + *			 :c:type:`rkisp1_cif_isp_dpf_strength_config`
>>> + */
>>> +struct rkisp1_ext_params_dpf_strength_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_dpf_strength_config dpf_strength_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_cproc_config - RkISP1 extensible params CPROC config
>>> + *
>>> + * RkISP1 extensible parameters Color Processing configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @cproc_config: Color processing configuration, see
>>> + *		  :c:type:`rkisp1_cif_isp_cproc_config`
>>> + */
>>> +struct rkisp1_ext_params_cproc_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_cproc_config cproc_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_ie_config - RkISP1 extensible params IE config
>>> + *
>>> + * RkISP1 extensible parameters Image Effect configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @ie_config: Image Effect configuration, see
>>> + *	       :c:type:`rkisp1_cif_isp_ie_config`
>>> + */
>>> +struct rkisp1_ext_params_ie_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_ie_config ie_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_awb_meas_config - RkISP1 extensible params AWB
>>> + *					      Meas config
>>> + *
>>> + * RkISP1 extensible parameters Auto-White Balance Measurement configuration
>>> + * block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @awb_meas_config: Auto-White Balance measure configuration, see
>>> + *		     :c:type:`rkisp1_cif_isp_awb_meas_config`
>>> + */
>>> +struct rkisp1_ext_params_awb_meas_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_awb_meas_config awb_meas_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_hst_config - RkISP1 extensible params Histogram config
>>> + *
>>> + * RkISP1 extensible parameters Histogram statistics configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @hst_config: Histogram statistics configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_hst_config`
>>> + */
>>> +struct rkisp1_ext_params_hst_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_hst_config hst_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_aec_config - RkISP1 extensible params AEC config
>>> + *
>>> + * RkISP1 extensible parameters Auto-Exposure statistics configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @aec_config: Auto-Exposure statistics configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_aec_config`
>>> + */
>>> +struct rkisp1_ext_params_aec_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_aec_config aec_config;
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_afc_config - RkISP1 extensible params AFC config
>>> + *
>>> + * RkISP1 extensible parameters Auto-Focus statistics configuration block
>>> + *
>>> + * @header: The RkISP1 extensible parameters header, see
>>> + *	    :c:type:`rkisp1_ext_params_block_header`
>>> + * @afc_config: Auto-Focus statistics configuration, see
>>> + *		:c:type:`rkisp1_cif_isp_afc_config`
>>> + */
>>> +struct rkisp1_ext_params_afc_config {
>>> +	struct rkisp1_ext_params_block_header header;
>>> +	struct rkisp1_cif_isp_afc_config afc_config;
>>> +};
>>> +
>>> +#define RKISP1_EXT_PARAMS_MAX_SIZE					\
>>> +	(sizeof(struct rkisp1_ext_params_bls_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_dpcc_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_sdg_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_lsc_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_awb_gain_config)		+\
>>> +	sizeof(struct rkisp1_ext_params_flt_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_bdm_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_ctk_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_goc_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_dpf_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_dpf_strength_config)		+\
>>> +	sizeof(struct rkisp1_ext_params_cproc_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_ie_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_awb_meas_config)		+\
>>> +	sizeof(struct rkisp1_ext_params_hst_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_aec_config)			+\
>>> +	sizeof(struct rkisp1_ext_params_afc_config))
>>> +
>>> +/**
>>> + * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version
>>> + *
>>> + * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters
>>> + */
>>> +enum rksip1_ext_param_buffer_version {
>>> +	RKISP1_EXT_PARAM_BUFFER_V1 = 1,
>>> +};
>>> +
>>> +/**
>>> + * struct rkisp1_ext_params_cfg - RkISP1 extensible parameters configuration
>>> + *
>>> + * This struct contains the configuration parameters of the RkISP1 ISP
>>> + * algorithms, serialized by userspace into a data buffer. Each configuration
>>> + * parameter block is represented by a block-specific structure which contains a
>>> + * :c:type:`rkisp1_ext_params_block_header` entry as first member. Userspace
>>> + * populates the @data buffer with configuration parameters for the blocks that
>>> + * it intends to configure. As a consequence, the data buffer effective size
>>> + * changes according to the number of ISP blocks that userspace intends to
>>> + * configure and is set by userspace in the @total_size field.
>>> + *
>>> + * The parameters buffer is versioned by the @version field to allow modifying
>>> + * and extending its definition. Userspace shall populate the @version field to
>>> + * inform the driver about the version it intends to use. The driver will parse
>>> + * and handle the @data buffer according to the data layout specific to the
>>> + * indicated version and return an error if the desired version is not
>>> + * supported.
>>> + *
>>> + * For each ISP block that userspace wants to configure, a block-specific
>>> + * structure is appended to the @data buffer, one after the other without gaps
> I think we should align all the blocks to a 64 bits boundary. Otherwise
> we'll have unaligned access issues, as well as layout differences
> between 32-bitand 64-bit userspace.
>
>>> + * in between nor overlaps. Userspace shall populate the @total_size field with
>>> + * the effective size, in bytes, of the @data buffer.
>>> + *
>>> + * The expected memory layout of the parameters buffer is::
>>> + *
>>> + *	+-------------------- struct rkisp1_ext_params_cfg -------------------+
>>> + *	| version = RKISP_EXT_PARAMS_BUFFER_V1;                               |
>>> + *	| total_size = sizeof(struct rkisp1_ext_params_bls_config)            |
>>> + *	|              sizeof(struct rkisp1_ext_params_dpcc_config);          |
>>> + *	| +------------------------- data  ---------------------------------+ |
>>> + *	| | +------------- struct rkisp1_ext_params_bls_config -----------+ | |
>>> + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
>>> + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS;                | | | |
>>> + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
>>> + *	| | | | size = sizeof(struct rkisp1_ext_params_bls_config);     | | | |
>>> + *	| | | +---------------------------------------------------------+ | | |
>>> + *	| | | +---------- struct rkisp1_cif_isp_bls_config -------------+ | | |
>>> + *	| | | | enable_auto = 0;                                        | | | |
>>> + *	| | | | fixed_val.r = 256;                                      | | | |
>>> + *	| | | | fixed_val.gr = 256;                                     | | | |
>>> + *	| | | | fixed_val.gb = 256;                                     | | | |
>>> + *	| | | | fixed_val.b = 256;                                      | | | |
>>> + *	| | | +---------------------------------------------------------+ | | |
>>> + *	| | +------------ struct rkisp1_ext_params_dpcc_config -----------+ | |
>>> + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
>>> + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC;               | | | |
>>> + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
>>> + *	| | | | size = sizeof(struct rkisp1_ext_params_dpcc_config);    | | | |
>>> + *	| | | +---------------------------------------------------------+ | | |
>>> + *	| | | +---------- struct rkisp1_cif_isp_dpcc_config ------------+ | | |
>>> + *	| | | | mode = RKISP1_CIF_ISP_DPCC_MODE_STAGE1_ENABLE;          | | | |
>>> + *	| | | | output_mode =                                           | | | |
>>> + *	| | | |   RKISP1_CIF_ISP_DPCC_OUTPUT_MODE_STAGE1_INCL_G_CENTER; | | | |
>>> + *	| | | | set_use = ... ;                                         | | | |
>>> + *	| | | | ...  = ... ;                                            | | | |
>>> + *	| | | +---------------------------------------------------------+ | | |
>>> + *	| | +-------------------------------------------------------------+ | |
>>> + *	| +-----------------------------------------------------------------+ |
>>> + *	+---------------------------------------------------------------------+
>>> + *
>>> + * @version: The RkISP1 extensible parameters buffer version, see
>>> + *	     :c:type:`rksip1_ext_param_buffer_version`
>>> + * @total_size: The RkISP1 configuration data effective size, excluding this
>>> + *		header
>>> + * @data: The RkISP1 extensible configuration data blocks
>>> + */
>>> +struct rkisp1_ext_params_cfg {
>>> +	__u32 version;
> On a 64-bit system there will be a 32-bit hole here, while on a 32-bit
> system there won't. This means that a 32-bit userspace won't run on a
> 64-bit kernel. You can add a a __u32 reserved field to fix that.
>
>>> +	__u64 total_size;
>>> +	__u8 data[RKISP1_EXT_PARAMS_MAX_SIZE];
>>> +};
>>> +
>>>    #endif /* _UAPI_RKISP1_CONFIG_H */

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format
  2024-06-12 13:49       ` Dan Scally
@ 2024-06-12 14:56         ` Laurent Pinchart
  0 siblings, 0 replies; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 14:56 UTC (permalink / raw)
  To: Dan Scally
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 12, 2024 at 02:49:42PM +0100, Daniel Scally wrote:
> On 12/06/2024 13:56, Laurent Pinchart wrote:
> > On Wed, Jun 12, 2024 at 11:02:58AM +0100, Daniel Scally wrote:
> >> On 05/06/2024 17:54, Jacopo Mondi wrote:
> >>> Add to the rkisp1-config.h header data types and documentation of
> >>> the extensible parameters format.
> >>>
> >>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> >>> ---
> >>>    include/uapi/linux/rkisp1-config.h | 482 +++++++++++++++++++++++++++++
> >>>    1 file changed, 482 insertions(+)
> >>>
> >>> diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h
> >>> index 6eeaf8bf2362..9c93e536f270 100644
> >>> --- a/include/uapi/linux/rkisp1-config.h
> >>> +++ b/include/uapi/linux/rkisp1-config.h
> >>> @@ -996,4 +996,486 @@ struct rkisp1_stat_buffer {
> >>>    	struct rkisp1_cif_isp_stat params;
> >>>    };
> >>>    
> >>> +/*---------- PART3: Extensible Configuration Parameters  ------------*/
> >>> +
> >>> +/**
> >>> + * enum rkisp1_ext_params_block_type - RkISP1 extensible params block type
> >>> + *
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS: Black level subtraction
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC: Defect pixel cluster correction
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG: Sensor de-gamma
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS: Auto white balance gains
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT: ISP filtering
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM: Bayer de-mosaic
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK: Cross-talk correction
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC: Gamma out correction
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF: De-noise pre-filter
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT: De-noise pre-filter strength
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC: Color processing
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_IE: Image effects
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC: Lens shading correction
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS: Auto white balance statistics
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS: Histogram statistics
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS: Auto exposure statistics
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS: Auto-focus statistics
> >>> + */
> >>> +enum rkisp1_ext_params_block_type {
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_IE,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL,
> >>> +};
> >>
> >> Laurent suggested referencing the enum value in the comments for each
> >> block in his review of the C55 series, and I think that that's a good
> >> idea - can we do that?
> >
> > I would like that too.
> >
> >> Otherwise, looks good to me:
> >>
> >> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> >>
> >>> +
> >>> +/**
> >>> + * enum rkisp1_ext_params_block_state - RkISP1 extensible parameter block enable
> >>> + *					state flags
> >>> + *
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE: Do not change the block power state
> >
> > I wouldn't call that "power" state, I don't think it's about power.
> >
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_DISABLE: Disable the HW block
> >>> + * @RKISP1_EXT_PARAMS_BLOCK_ENABLE: Enable the HW block
> >>> + */
> >>> +enum rkisp1_ext_params_block_state {
> >>> +	RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_DISABLE,
> >>> +	RKISP1_EXT_PARAMS_BLOCK_ENABLE,
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_block_header - RkISP1 extensible parameter block
> >>> + *					   header
> >>> + *
> >>> + * This structure represents the common part of all the ISP configuration
> >>> + * blocks. Each parameters block shall embed an instance of this structure type
> >>> + * as its first member, followed by the block-specific configuration data. The
> >>> + * driver inspects this common header to discern the block type and its size and
> >>> + * properly handle the block content by casting it to the correct block-specific
> >>> + * type.
> >>> + *
> >>> + * The @type field is one of the values enumerated by
> >>> + * :c:type:`rkisp1_ext_params_block_type` and specifies how the data should be
> >>> + * interpreted by the driver. The @size field specifies the size of the
> >>> + * parameters block and is used by the driver for validation purposes.
> >>> + *
> >>> + * The @state field specifies if the ISP block power state should be changed,
> >>> + * and, if it has to, if it has to be enabled to disabled. The possible
> >>> + * states are enumerated by :c:type:`rkisp1_ext_params_block_state`.
> >>> + * When userspace needs to configure and enable an ISP block it shall fully
> >>> + * populate the block configuration and the @state flag shall be set to
> >>> + * RKISP1_EXT_PARAMS_BLOCK_ENABLE. When userspace simply wants to disable the
> >>> + * ISP block the @state flag shall be set to RKISP1_EXT_PARAMS_BLOCK_DISABLE.
> >
> > You should document here what happens to the rest of the parameters in
> > the block in that case. I think they should be ignored by the driver,
> > and possibly set to 0 by userspace.
> >
> > Another option when disabling a block would be to include the header
> > only, with the size field set to the header size, and not include the
> > rest of the block contents. It's a bit pointless to include data that
> > the kernel won't use. While that would be my preference from an API
> > point of view (at least until someone points out to use cases that would
> > benefit from a different option), I haven't checked what it would imply
> > from a userspace and kernelspace implementation point of view. I could
> > agree to keeping the data part of the block in for disabled blocks if
> > omitting it would result in implementation issues.
> 
> I forgot about this suggestion - for what it's worth I agree that it's
> a good one, I'm going to follow it for the C55

The 64-bit alignment issue applies to the C55 too :-)

> >>> If
> >>> + * a new configuration of an ISP block should be applied but the power state
> >>> + * doesn't need to be changed, userspace shall fully populate the ISP block
> >>> + * configuration and the @state flag shall be set to
> >>> + * RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE.
> >
> > I don't think we need RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE for that. The
> > state can just be set to @RKISP1_EXT_PARAMS_BLOCK_ENABLE.
> 
> I did think about that, but it avoids reading the register to check if
> it's already enabled or not / writing enable unconditionally...is the
> saved overhead not worth it?

Generally speaking, I wouldn't fine-tune the UAPI based on the internal
implementation of the driver. If the driver would benefit from knowing,
without accessing the hardware, if a block is enabled or not, it could
cache that information instead of relying on userspace to provide it. If
we keep RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE, I would expect the driver to
not write hardware registers for a block that is currently disabled if
it receives a NO_CHANGE parameters block. The driver would thus need to
cache that information (or read it back from the hardware) already.

> >>> + *
> >>> + * Userspace is responsible for correctly populating the parameters block header
> >>> + * fields (@type, @state and @size) and correctly populate the block-specific
> >>> + * parameters.
> >>> + *
> >>> + * For example:
> >>> + *
> >>> + * .. code-block:: c
> >>> + *
> >>> + *	void populate_bls(struct rkisp1_ext_params_block_header *block) {
> >>> + *		struct rkisp1_ext_params_bls_config *bls =
> >>> + *			(struct rkisp1_ext_params_bls_config *)block;
> >>> + *
> >>> + *		block->header.type = RKISP1_EXT_PARAMS_BLOCK_ID_BLS;
> >>> + *		block->header.state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;
> >>> + *		block->header.size = sizeof(struct rkisp1_ext_params_bls_config);
> >>> + *
> >>> + *		bls->bls_config.enable_auto = 0;
> >>> + *		bls->bls_config.fixed_val.r = blackLevelRed_;
> >>> + *		bls->bls_config.fixed_val.gr = blackLevelGreenR_;
> >>> + *		bls->bls_config.fixed_val.gb = blackLevelGreenB_;
> >>> + *		bls->bls_config.fixed_val.b = blackLevelBlue_;
> >>> + *	}
> >>> + *
> >>> + * @type: The parameters block type, see
> >>> + *	  :c:type:`rkisp1_ext_params_block_type`
> >>> + * @state: The block enable state flag, see
> >>> + *	   :c:type:`rkisp1_ext_params_block_state`
> >>> + * @size: Size (in bytes) of the parameters block, including this header
> >>> + */
> >>> +struct rkisp1_ext_params_block_header {
> >>> +	__u32 type;
> >>> +	__u32 state;
> >>> +	__u64 size;
> >
> > I think a 32-bit size would be move than enough. The header should
> > however be 64-bit aligned.
> >
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_bls_config - RkISP1 extensible params BLS config
> >>> + *
> >>> + * RkISP1 extensible parameters Black Level Subtraction configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @bls_config: Black Level Subtraction configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_bls_config`
> >>> + */
> >>> +struct rkisp1_ext_params_bls_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_bls_config bls_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_dpcc_config - RkISP1 extensible params DPCC config
> >>> + *
> >>> + * RkISP1 extensible parameters Defective Pixel Cluster Correction configuration
> >>> + * block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @dpcc_config: Defective Pixel Cluster Correction configuration, see
> >>> + *		 :c:type:`rkisp1_cif_isp_dpcc_config`
> >>> + */
> >>> +struct rkisp1_ext_params_dpcc_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_dpcc_config dpcc_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_sdg_config - RkISP1 extensible params SDG config
> >>> + *
> >>> + * RkISP1 extensible parameters Sensor Degamma configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @sdg_config: Sensor Degamma configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_sdg_config`
> >>> + */
> >>> +struct rkisp1_ext_params_sdg_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_sdg_config sdg_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_lsc_config - RkISP1 extensible params LSC config
> >>> + *
> >>> + * RkISP1 extensible parameters Lens Shading Correction configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @lsc_config: Lens Shading Correction configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_lsc_config`
> >>> + */
> >>> +struct rkisp1_ext_params_lsc_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_lsc_config lsc_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_awb_gain_config - RkISP1 extensible params AWB
> >>> + *					      gain config
> >>> + *
> >>> + * RkISP1 extensible parameters Auto-White Balance Gains configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @awb_config: Auto-White Balance Gains configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_awb_gain_config`
> >>> + */
> >>> +struct rkisp1_ext_params_awb_gain_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_awb_gain_config awb_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_flt_config - RkISP1 extensible params FLT config
> >>> + *
> >>> + * RkISP1 extensible parameters Filter configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @flt_config: Filter configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_flt_config`
> >>> + */
> >>> +struct rkisp1_ext_params_flt_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_flt_config flt_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_bdm_config - RkISP1 extensible params BDM config
> >>> + *
> >>> + * RkISP1 extensible parameters Demosaicing configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @bdm_config: Demosaicing configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_bdm_config`
> >>> + */
> >>> +struct rkisp1_ext_params_bdm_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_bdm_config bdm_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_ctk_config - RkISP1 extensible params CTK config
> >>> + *
> >>> + * RkISP1 extensible parameters Cross-Talk configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @ctk_config: Cross-Talk configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_ctk_config`
> >>> + */
> >>> +struct rkisp1_ext_params_ctk_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_ctk_config ctk_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_goc_config - RkISP1 extensible params GOC config
> >>> + *
> >>> + * RkISP1 extensible parameters Gamma-Out configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @goc_config: Gamma-Out configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_goc_config`
> >>> + */
> >>> +struct rkisp1_ext_params_goc_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_goc_config goc_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_dpf_config - RkISP1 extensible params DPF config
> >>> + *
> >>> + * RkISP1 extensible parameters De-noise Pre-Filter configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @dpf_config: De-noise Pre-Filter configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_dpf_config`
> >>> + */
> >>> +struct rkisp1_ext_params_dpf_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_dpf_config dpf_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_dpf_strength_config - RkISP1 extensible params DPF
> >>> + *						  strength config
> >>> + *
> >>> + * RkISP1 extensible parameters De-noise Pre-Filter strength configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @dpf_strength_config: De-noise Pre-Filter strength configuration, see
> >>> + *			 :c:type:`rkisp1_cif_isp_dpf_strength_config`
> >>> + */
> >>> +struct rkisp1_ext_params_dpf_strength_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_dpf_strength_config dpf_strength_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_cproc_config - RkISP1 extensible params CPROC config
> >>> + *
> >>> + * RkISP1 extensible parameters Color Processing configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @cproc_config: Color processing configuration, see
> >>> + *		  :c:type:`rkisp1_cif_isp_cproc_config`
> >>> + */
> >>> +struct rkisp1_ext_params_cproc_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_cproc_config cproc_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_ie_config - RkISP1 extensible params IE config
> >>> + *
> >>> + * RkISP1 extensible parameters Image Effect configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @ie_config: Image Effect configuration, see
> >>> + *	       :c:type:`rkisp1_cif_isp_ie_config`
> >>> + */
> >>> +struct rkisp1_ext_params_ie_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_ie_config ie_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_awb_meas_config - RkISP1 extensible params AWB
> >>> + *					      Meas config
> >>> + *
> >>> + * RkISP1 extensible parameters Auto-White Balance Measurement configuration
> >>> + * block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @awb_meas_config: Auto-White Balance measure configuration, see
> >>> + *		     :c:type:`rkisp1_cif_isp_awb_meas_config`
> >>> + */
> >>> +struct rkisp1_ext_params_awb_meas_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_awb_meas_config awb_meas_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_hst_config - RkISP1 extensible params Histogram config
> >>> + *
> >>> + * RkISP1 extensible parameters Histogram statistics configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @hst_config: Histogram statistics configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_hst_config`
> >>> + */
> >>> +struct rkisp1_ext_params_hst_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_hst_config hst_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_aec_config - RkISP1 extensible params AEC config
> >>> + *
> >>> + * RkISP1 extensible parameters Auto-Exposure statistics configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @aec_config: Auto-Exposure statistics configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_aec_config`
> >>> + */
> >>> +struct rkisp1_ext_params_aec_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_aec_config aec_config;
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_afc_config - RkISP1 extensible params AFC config
> >>> + *
> >>> + * RkISP1 extensible parameters Auto-Focus statistics configuration block
> >>> + *
> >>> + * @header: The RkISP1 extensible parameters header, see
> >>> + *	    :c:type:`rkisp1_ext_params_block_header`
> >>> + * @afc_config: Auto-Focus statistics configuration, see
> >>> + *		:c:type:`rkisp1_cif_isp_afc_config`
> >>> + */
> >>> +struct rkisp1_ext_params_afc_config {
> >>> +	struct rkisp1_ext_params_block_header header;
> >>> +	struct rkisp1_cif_isp_afc_config afc_config;
> >>> +};
> >>> +
> >>> +#define RKISP1_EXT_PARAMS_MAX_SIZE					\
> >>> +	(sizeof(struct rkisp1_ext_params_bls_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_dpcc_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_sdg_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_lsc_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_awb_gain_config)		+\
> >>> +	sizeof(struct rkisp1_ext_params_flt_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_bdm_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_ctk_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_goc_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_dpf_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_dpf_strength_config)		+\
> >>> +	sizeof(struct rkisp1_ext_params_cproc_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_ie_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_awb_meas_config)		+\
> >>> +	sizeof(struct rkisp1_ext_params_hst_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_aec_config)			+\
> >>> +	sizeof(struct rkisp1_ext_params_afc_config))
> >>> +
> >>> +/**
> >>> + * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version
> >>> + *
> >>> + * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters
> >>> + */
> >>> +enum rksip1_ext_param_buffer_version {
> >>> +	RKISP1_EXT_PARAM_BUFFER_V1 = 1,
> >>> +};
> >>> +
> >>> +/**
> >>> + * struct rkisp1_ext_params_cfg - RkISP1 extensible parameters configuration
> >>> + *
> >>> + * This struct contains the configuration parameters of the RkISP1 ISP
> >>> + * algorithms, serialized by userspace into a data buffer. Each configuration
> >>> + * parameter block is represented by a block-specific structure which contains a
> >>> + * :c:type:`rkisp1_ext_params_block_header` entry as first member. Userspace
> >>> + * populates the @data buffer with configuration parameters for the blocks that
> >>> + * it intends to configure. As a consequence, the data buffer effective size
> >>> + * changes according to the number of ISP blocks that userspace intends to
> >>> + * configure and is set by userspace in the @total_size field.
> >>> + *
> >>> + * The parameters buffer is versioned by the @version field to allow modifying
> >>> + * and extending its definition. Userspace shall populate the @version field to
> >>> + * inform the driver about the version it intends to use. The driver will parse
> >>> + * and handle the @data buffer according to the data layout specific to the
> >>> + * indicated version and return an error if the desired version is not
> >>> + * supported.
> >>> + *
> >>> + * For each ISP block that userspace wants to configure, a block-specific
> >>> + * structure is appended to the @data buffer, one after the other without gaps
> >
> > I think we should align all the blocks to a 64 bits boundary. Otherwise
> > we'll have unaligned access issues, as well as layout differences
> > between 32-bitand 64-bit userspace.
> >
> >>> + * in between nor overlaps. Userspace shall populate the @total_size field with
> >>> + * the effective size, in bytes, of the @data buffer.
> >>> + *
> >>> + * The expected memory layout of the parameters buffer is::
> >>> + *
> >>> + *	+-------------------- struct rkisp1_ext_params_cfg -------------------+
> >>> + *	| version = RKISP_EXT_PARAMS_BUFFER_V1;                               |
> >>> + *	| total_size = sizeof(struct rkisp1_ext_params_bls_config)            |
> >>> + *	|              sizeof(struct rkisp1_ext_params_dpcc_config);          |
> >>> + *	| +------------------------- data  ---------------------------------+ |
> >>> + *	| | +------------- struct rkisp1_ext_params_bls_config -----------+ | |
> >>> + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> >>> + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS;                | | | |
> >>> + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> >>> + *	| | | | size = sizeof(struct rkisp1_ext_params_bls_config);     | | | |
> >>> + *	| | | +---------------------------------------------------------+ | | |
> >>> + *	| | | +---------- struct rkisp1_cif_isp_bls_config -------------+ | | |
> >>> + *	| | | | enable_auto = 0;                                        | | | |
> >>> + *	| | | | fixed_val.r = 256;                                      | | | |
> >>> + *	| | | | fixed_val.gr = 256;                                     | | | |
> >>> + *	| | | | fixed_val.gb = 256;                                     | | | |
> >>> + *	| | | | fixed_val.b = 256;                                      | | | |
> >>> + *	| | | +---------------------------------------------------------+ | | |
> >>> + *	| | +------------ struct rkisp1_ext_params_dpcc_config -----------+ | |
> >>> + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> >>> + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC;               | | | |
> >>> + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> >>> + *	| | | | size = sizeof(struct rkisp1_ext_params_dpcc_config);    | | | |
> >>> + *	| | | +---------------------------------------------------------+ | | |
> >>> + *	| | | +---------- struct rkisp1_cif_isp_dpcc_config ------------+ | | |
> >>> + *	| | | | mode = RKISP1_CIF_ISP_DPCC_MODE_STAGE1_ENABLE;          | | | |
> >>> + *	| | | | output_mode =                                           | | | |
> >>> + *	| | | |   RKISP1_CIF_ISP_DPCC_OUTPUT_MODE_STAGE1_INCL_G_CENTER; | | | |
> >>> + *	| | | | set_use = ... ;                                         | | | |
> >>> + *	| | | | ...  = ... ;                                            | | | |
> >>> + *	| | | +---------------------------------------------------------+ | | |
> >>> + *	| | +-------------------------------------------------------------+ | |
> >>> + *	| +-----------------------------------------------------------------+ |
> >>> + *	+---------------------------------------------------------------------+
> >>> + *
> >>> + * @version: The RkISP1 extensible parameters buffer version, see
> >>> + *	     :c:type:`rksip1_ext_param_buffer_version`
> >>> + * @total_size: The RkISP1 configuration data effective size, excluding this
> >>> + *		header
> >>> + * @data: The RkISP1 extensible configuration data blocks
> >>> + */
> >>> +struct rkisp1_ext_params_cfg {
> >>> +	__u32 version;
> >
> > On a 64-bit system there will be a 32-bit hole here, while on a 32-bit
> > system there won't. This means that a 32-bit userspace won't run on a
> > 64-bit kernel. You can add a a __u32 reserved field to fix that.
> >
> >>> +	__u64 total_size;
> >>> +	__u8 data[RKISP1_EXT_PARAMS_MAX_SIZE];
> >>> +};
> >>> +
> >>>    #endif /* _UAPI_RKISP1_CONFIG_H */

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format
  2024-06-12 12:56     ` Laurent Pinchart
  2024-06-12 13:49       ` Dan Scally
@ 2024-06-19 12:44       ` Jacopo Mondi
  2024-06-19 15:49         ` Laurent Pinchart
  1 sibling, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-19 12:44 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Dan Scally, Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi Laurent

On Wed, Jun 12, 2024 at 03:56:00PM GMT, Laurent Pinchart wrote:
> On Wed, Jun 12, 2024 at 11:02:58AM +0100, Daniel Scally wrote:
> > Hi Jacopo - thanks for the patchset
> >
> > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > Add to the rkisp1-config.h header data types and documentation of
> > > the extensible parameters format.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >   include/uapi/linux/rkisp1-config.h | 482 +++++++++++++++++++++++++++++
> > >   1 file changed, 482 insertions(+)
> > >
> > > diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h
> > > index 6eeaf8bf2362..9c93e536f270 100644
> > > --- a/include/uapi/linux/rkisp1-config.h
> > > +++ b/include/uapi/linux/rkisp1-config.h
> > > @@ -996,4 +996,486 @@ struct rkisp1_stat_buffer {
> > >   	struct rkisp1_cif_isp_stat params;
> > >   };
> > >
> > > +/*---------- PART3: Extensible Configuration Parameters  ------------*/
> > > +
> > > +/**
> > > + * enum rkisp1_ext_params_block_type - RkISP1 extensible params block type
> > > + *
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS: Black level subtraction
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC: Defect pixel cluster correction
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG: Sensor de-gamma
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS: Auto white balance gains
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT: ISP filtering
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM: Bayer de-mosaic
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK: Cross-talk correction
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC: Gamma out correction
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF: De-noise pre-filter
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT: De-noise pre-filter strength
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC: Color processing
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_IE: Image effects
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC: Lens shading correction
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS: Auto white balance statistics
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS: Histogram statistics
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS: Auto exposure statistics
> > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS: Auto-focus statistics
> > > + */
> > > +enum rkisp1_ext_params_block_type {
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_IE,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS,
> > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL,
> > > +};
> >
> > Laurent suggested referencing the enum value in the comments for each
> > block in his review of the C55 series, and I think that that's a good
> > idea - can we do that?
>
> I would like that too.
>
> > Otherwise, looks good to me:
> >
> > Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> >
> > > +
> > > +/**
> > > + * enum rkisp1_ext_params_block_state - RkISP1 extensible parameter block enable
> > > + *					state flags
> > > + *
> > > + * @RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE: Do not change the block power state
>
> I wouldn't call that "power" state, I don't think it's about power.

How would you call it ? It's about enabling the block, should I use
'enable' ?

>
> > > + * @RKISP1_EXT_PARAMS_BLOCK_DISABLE: Disable the HW block
> > > + * @RKISP1_EXT_PARAMS_BLOCK_ENABLE: Enable the HW block
> > > + */
> > > +enum rkisp1_ext_params_block_state {
> > > +	RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE,
> > > +	RKISP1_EXT_PARAMS_BLOCK_DISABLE,
> > > +	RKISP1_EXT_PARAMS_BLOCK_ENABLE,
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_block_header - RkISP1 extensible parameter block
> > > + *					   header
> > > + *
> > > + * This structure represents the common part of all the ISP configuration
> > > + * blocks. Each parameters block shall embed an instance of this structure type
> > > + * as its first member, followed by the block-specific configuration data. The
> > > + * driver inspects this common header to discern the block type and its size and
> > > + * properly handle the block content by casting it to the correct block-specific
> > > + * type.
> > > + *
> > > + * The @type field is one of the values enumerated by
> > > + * :c:type:`rkisp1_ext_params_block_type` and specifies how the data should be
> > > + * interpreted by the driver. The @size field specifies the size of the
> > > + * parameters block and is used by the driver for validation purposes.
> > > + *
> > > + * The @state field specifies if the ISP block power state should be changed,
> > > + * and, if it has to, if it has to be enabled to disabled. The possible
> > > + * states are enumerated by :c:type:`rkisp1_ext_params_block_state`.
> > > + * When userspace needs to configure and enable an ISP block it shall fully
> > > + * populate the block configuration and the @state flag shall be set to
> > > + * RKISP1_EXT_PARAMS_BLOCK_ENABLE. When userspace simply wants to disable the
> > > + * ISP block the @state flag shall be set to RKISP1_EXT_PARAMS_BLOCK_DISABLE.
>
> You should document here what happens to the rest of the parameters in
> the block in that case. I think they should be ignored by the driver,
> and possibly set to 0 by userspace.

I'll add "they get ignored"

>
> Another option when disabling a block would be to include the header
> only, with the size field set to the header size, and not include the
> rest of the block contents. It's a bit pointless to include data that
> the kernel won't use. While that would be my preference from an API
> point of view (at least until someone points out to use cases that would
> benefit from a different option), I haven't checked what it would imply
> from a userspace and kernelspace implementation point of view. I could
> agree to keeping the data part of the block in for disabled blocks if
> omitting it would result in implementation issues.
>

I'm not sure it will add issues, but it would make it more cumbersome
for userspace maybe

> > If
> > > + * a new configuration of an ISP block should be applied but the power state
> > > + * doesn't need to be changed, userspace shall fully populate the ISP block
> > > + * configuration and the @state flag shall be set to
> > > + * RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE.
>
> I don't think we need RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE for that. The
> state can just be set to @RKISP1_EXT_PARAMS_BLOCK_ENABLE.
>

The existing code does

        rkisp1_isp_isr_xxx_config()
        {

                if (cfg_update & MODULE_XXX)
                        rkisp1_xxx_config()

                if (en_update & MODULE_XXX)
                        if (enable & MODULE_XXX)
                                set_bit(ENABLE, MODULE_XXX)
                        else
                                clear_bit(ENABLE, MODULE_XXX)
        }

As the enablement maks is separate from the configuration mask,
userspace can in the previous version set (cfg_update |= MODULE_XXX)
without setting the MODULE_XXX bit in en_update.

Now, with the the introduction of this new format, the 'state' field is
always present and I can't express with just two values the "reconfigure
but not enable" case.

Think about LSC, when a new table has to be applied we have to
re-program it without changing the enable bit. I mean, re-asserting
the bit is probably harmless, but isn't it safer avoid doing that ?

> > > + *
> > > + * Userspace is responsible for correctly populating the parameters block header
> > > + * fields (@type, @state and @size) and correctly populate the block-specific
> > > + * parameters.
> > > + *
> > > + * For example:
> > > + *
> > > + * .. code-block:: c
> > > + *
> > > + *	void populate_bls(struct rkisp1_ext_params_block_header *block) {
> > > + *		struct rkisp1_ext_params_bls_config *bls =
> > > + *			(struct rkisp1_ext_params_bls_config *)block;
> > > + *
> > > + *		block->header.type = RKISP1_EXT_PARAMS_BLOCK_ID_BLS;
> > > + *		block->header.state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;
> > > + *		block->header.size = sizeof(struct rkisp1_ext_params_bls_config);
> > > + *
> > > + *		bls->bls_config.enable_auto = 0;
> > > + *		bls->bls_config.fixed_val.r = blackLevelRed_;
> > > + *		bls->bls_config.fixed_val.gr = blackLevelGreenR_;
> > > + *		bls->bls_config.fixed_val.gb = blackLevelGreenB_;
> > > + *		bls->bls_config.fixed_val.b = blackLevelBlue_;
> > > + *	}
> > > + *
> > > + * @type: The parameters block type, see
> > > + *	  :c:type:`rkisp1_ext_params_block_type`
> > > + * @state: The block enable state flag, see
> > > + *	   :c:type:`rkisp1_ext_params_block_state`
> > > + * @size: Size (in bytes) of the parameters block, including this header
> > > + */
> > > +struct rkisp1_ext_params_block_header {
> > > +	__u32 type;
> > > +	__u32 state;
> > > +	__u64 size;
>
> I think a 32-bit size would be move than enough. The header should
> however be 64-bit aligned.

That's why size is 64 bits.

>
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_bls_config - RkISP1 extensible params BLS config
> > > + *
> > > + * RkISP1 extensible parameters Black Level Subtraction configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @bls_config: Black Level Subtraction configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_bls_config`
> > > + */
> > > +struct rkisp1_ext_params_bls_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_bls_config bls_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_dpcc_config - RkISP1 extensible params DPCC config
> > > + *
> > > + * RkISP1 extensible parameters Defective Pixel Cluster Correction configuration
> > > + * block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @dpcc_config: Defective Pixel Cluster Correction configuration, see
> > > + *		 :c:type:`rkisp1_cif_isp_dpcc_config`
> > > + */
> > > +struct rkisp1_ext_params_dpcc_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_dpcc_config dpcc_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_sdg_config - RkISP1 extensible params SDG config
> > > + *
> > > + * RkISP1 extensible parameters Sensor Degamma configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @sdg_config: Sensor Degamma configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_sdg_config`
> > > + */
> > > +struct rkisp1_ext_params_sdg_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_sdg_config sdg_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_lsc_config - RkISP1 extensible params LSC config
> > > + *
> > > + * RkISP1 extensible parameters Lens Shading Correction configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @lsc_config: Lens Shading Correction configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_lsc_config`
> > > + */
> > > +struct rkisp1_ext_params_lsc_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_lsc_config lsc_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_awb_gain_config - RkISP1 extensible params AWB
> > > + *					      gain config
> > > + *
> > > + * RkISP1 extensible parameters Auto-White Balance Gains configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @awb_config: Auto-White Balance Gains configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_awb_gain_config`
> > > + */
> > > +struct rkisp1_ext_params_awb_gain_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_awb_gain_config awb_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_flt_config - RkISP1 extensible params FLT config
> > > + *
> > > + * RkISP1 extensible parameters Filter configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @flt_config: Filter configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_flt_config`
> > > + */
> > > +struct rkisp1_ext_params_flt_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_flt_config flt_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_bdm_config - RkISP1 extensible params BDM config
> > > + *
> > > + * RkISP1 extensible parameters Demosaicing configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @bdm_config: Demosaicing configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_bdm_config`
> > > + */
> > > +struct rkisp1_ext_params_bdm_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_bdm_config bdm_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_ctk_config - RkISP1 extensible params CTK config
> > > + *
> > > + * RkISP1 extensible parameters Cross-Talk configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @ctk_config: Cross-Talk configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_ctk_config`
> > > + */
> > > +struct rkisp1_ext_params_ctk_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_ctk_config ctk_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_goc_config - RkISP1 extensible params GOC config
> > > + *
> > > + * RkISP1 extensible parameters Gamma-Out configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @goc_config: Gamma-Out configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_goc_config`
> > > + */
> > > +struct rkisp1_ext_params_goc_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_goc_config goc_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_dpf_config - RkISP1 extensible params DPF config
> > > + *
> > > + * RkISP1 extensible parameters De-noise Pre-Filter configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @dpf_config: De-noise Pre-Filter configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_dpf_config`
> > > + */
> > > +struct rkisp1_ext_params_dpf_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_dpf_config dpf_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_dpf_strength_config - RkISP1 extensible params DPF
> > > + *						  strength config
> > > + *
> > > + * RkISP1 extensible parameters De-noise Pre-Filter strength configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @dpf_strength_config: De-noise Pre-Filter strength configuration, see
> > > + *			 :c:type:`rkisp1_cif_isp_dpf_strength_config`
> > > + */
> > > +struct rkisp1_ext_params_dpf_strength_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_dpf_strength_config dpf_strength_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_cproc_config - RkISP1 extensible params CPROC config
> > > + *
> > > + * RkISP1 extensible parameters Color Processing configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @cproc_config: Color processing configuration, see
> > > + *		  :c:type:`rkisp1_cif_isp_cproc_config`
> > > + */
> > > +struct rkisp1_ext_params_cproc_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_cproc_config cproc_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_ie_config - RkISP1 extensible params IE config
> > > + *
> > > + * RkISP1 extensible parameters Image Effect configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @ie_config: Image Effect configuration, see
> > > + *	       :c:type:`rkisp1_cif_isp_ie_config`
> > > + */
> > > +struct rkisp1_ext_params_ie_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_ie_config ie_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_awb_meas_config - RkISP1 extensible params AWB
> > > + *					      Meas config
> > > + *
> > > + * RkISP1 extensible parameters Auto-White Balance Measurement configuration
> > > + * block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @awb_meas_config: Auto-White Balance measure configuration, see
> > > + *		     :c:type:`rkisp1_cif_isp_awb_meas_config`
> > > + */
> > > +struct rkisp1_ext_params_awb_meas_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_awb_meas_config awb_meas_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_hst_config - RkISP1 extensible params Histogram config
> > > + *
> > > + * RkISP1 extensible parameters Histogram statistics configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @hst_config: Histogram statistics configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_hst_config`
> > > + */
> > > +struct rkisp1_ext_params_hst_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_hst_config hst_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_aec_config - RkISP1 extensible params AEC config
> > > + *
> > > + * RkISP1 extensible parameters Auto-Exposure statistics configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @aec_config: Auto-Exposure statistics configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_aec_config`
> > > + */
> > > +struct rkisp1_ext_params_aec_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_aec_config aec_config;
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_afc_config - RkISP1 extensible params AFC config
> > > + *
> > > + * RkISP1 extensible parameters Auto-Focus statistics configuration block
> > > + *
> > > + * @header: The RkISP1 extensible parameters header, see
> > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > + * @afc_config: Auto-Focus statistics configuration, see
> > > + *		:c:type:`rkisp1_cif_isp_afc_config`
> > > + */
> > > +struct rkisp1_ext_params_afc_config {
> > > +	struct rkisp1_ext_params_block_header header;
> > > +	struct rkisp1_cif_isp_afc_config afc_config;
> > > +};
> > > +
> > > +#define RKISP1_EXT_PARAMS_MAX_SIZE					\
> > > +	(sizeof(struct rkisp1_ext_params_bls_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_dpcc_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_sdg_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_lsc_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_awb_gain_config)		+\
> > > +	sizeof(struct rkisp1_ext_params_flt_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_bdm_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_ctk_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_goc_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_dpf_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_dpf_strength_config)		+\
> > > +	sizeof(struct rkisp1_ext_params_cproc_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_ie_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_awb_meas_config)		+\
> > > +	sizeof(struct rkisp1_ext_params_hst_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_aec_config)			+\
> > > +	sizeof(struct rkisp1_ext_params_afc_config))
> > > +
> > > +/**
> > > + * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version
> > > + *
> > > + * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters
> > > + */
> > > +enum rksip1_ext_param_buffer_version {
> > > +	RKISP1_EXT_PARAM_BUFFER_V1 = 1,
> > > +};
> > > +
> > > +/**
> > > + * struct rkisp1_ext_params_cfg - RkISP1 extensible parameters configuration
> > > + *
> > > + * This struct contains the configuration parameters of the RkISP1 ISP
> > > + * algorithms, serialized by userspace into a data buffer. Each configuration
> > > + * parameter block is represented by a block-specific structure which contains a
> > > + * :c:type:`rkisp1_ext_params_block_header` entry as first member. Userspace
> > > + * populates the @data buffer with configuration parameters for the blocks that
> > > + * it intends to configure. As a consequence, the data buffer effective size
> > > + * changes according to the number of ISP blocks that userspace intends to
> > > + * configure and is set by userspace in the @total_size field.
> > > + *
> > > + * The parameters buffer is versioned by the @version field to allow modifying
> > > + * and extending its definition. Userspace shall populate the @version field to
> > > + * inform the driver about the version it intends to use. The driver will parse
> > > + * and handle the @data buffer according to the data layout specific to the
> > > + * indicated version and return an error if the desired version is not
> > > + * supported.
> > > + *
> > > + * For each ISP block that userspace wants to configure, a block-specific
> > > + * structure is appended to the @data buffer, one after the other without gaps
>
> I think we should align all the blocks to a 64 bits boundary. Otherwise
> we'll have unaligned access issues, as well as layout differences
> between 32-bitand 64-bit userspace.

Each block header is 64 bits aligned, what follows it's not

Should I add
        __attribute__((aligned(8))

To each block ?

>
> > > + * in between nor overlaps. Userspace shall populate the @total_size field with
> > > + * the effective size, in bytes, of the @data buffer.
> > > + *
> > > + * The expected memory layout of the parameters buffer is::
> > > + *
> > > + *	+-------------------- struct rkisp1_ext_params_cfg -------------------+
> > > + *	| version = RKISP_EXT_PARAMS_BUFFER_V1;                               |
> > > + *	| total_size = sizeof(struct rkisp1_ext_params_bls_config)            |
> > > + *	|              sizeof(struct rkisp1_ext_params_dpcc_config);          |
> > > + *	| +------------------------- data  ---------------------------------+ |
> > > + *	| | +------------- struct rkisp1_ext_params_bls_config -----------+ | |
> > > + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> > > + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS;                | | | |
> > > + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> > > + *	| | | | size = sizeof(struct rkisp1_ext_params_bls_config);     | | | |
> > > + *	| | | +---------------------------------------------------------+ | | |
> > > + *	| | | +---------- struct rkisp1_cif_isp_bls_config -------------+ | | |
> > > + *	| | | | enable_auto = 0;                                        | | | |
> > > + *	| | | | fixed_val.r = 256;                                      | | | |
> > > + *	| | | | fixed_val.gr = 256;                                     | | | |
> > > + *	| | | | fixed_val.gb = 256;                                     | | | |
> > > + *	| | | | fixed_val.b = 256;                                      | | | |
> > > + *	| | | +---------------------------------------------------------+ | | |
> > > + *	| | +------------ struct rkisp1_ext_params_dpcc_config -----------+ | |
> > > + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> > > + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC;               | | | |
> > > + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> > > + *	| | | | size = sizeof(struct rkisp1_ext_params_dpcc_config);    | | | |
> > > + *	| | | +---------------------------------------------------------+ | | |
> > > + *	| | | +---------- struct rkisp1_cif_isp_dpcc_config ------------+ | | |
> > > + *	| | | | mode = RKISP1_CIF_ISP_DPCC_MODE_STAGE1_ENABLE;          | | | |
> > > + *	| | | | output_mode =                                           | | | |
> > > + *	| | | |   RKISP1_CIF_ISP_DPCC_OUTPUT_MODE_STAGE1_INCL_G_CENTER; | | | |
> > > + *	| | | | set_use = ... ;                                         | | | |
> > > + *	| | | | ...  = ... ;                                            | | | |
> > > + *	| | | +---------------------------------------------------------+ | | |
> > > + *	| | +-------------------------------------------------------------+ | |
> > > + *	| +-----------------------------------------------------------------+ |
> > > + *	+---------------------------------------------------------------------+
> > > + *
> > > + * @version: The RkISP1 extensible parameters buffer version, see
> > > + *	     :c:type:`rksip1_ext_param_buffer_version`
> > > + * @total_size: The RkISP1 configuration data effective size, excluding this
> > > + *		header
> > > + * @data: The RkISP1 extensible configuration data blocks
> > > + */
> > > +struct rkisp1_ext_params_cfg {
> > > +	__u32 version;
>
> On a 64-bit system there will be a 32-bit hole here, while on a 32-bit
> system there won't. This means that a 32-bit userspace won't run on a
> 64-bit kernel. You can add a a __u32 reserved field to fix that.

Or make version and __u64 ?

>
> > > +	__u64 total_size;
> > > +	__u8 data[RKISP1_EXT_PARAMS_MAX_SIZE];
> > > +};
> > > +
> > >   #endif /* _UAPI_RKISP1_CONFIG_H */
>
> --
> Regards,
>
> Laurent Pinchart
>

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format
  2024-06-19 12:44       ` Jacopo Mondi
@ 2024-06-19 15:49         ` Laurent Pinchart
  0 siblings, 0 replies; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-19 15:49 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Dan Scally, Linux Media Mailing List, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo,

On Wed, Jun 19, 2024 at 02:44:08PM +0200, Jacopo Mondi wrote:
> On Wed, Jun 12, 2024 at 03:56:00PM GMT, Laurent Pinchart wrote:
> > On Wed, Jun 12, 2024 at 11:02:58AM +0100, Daniel Scally wrote:
> > > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > > Add to the rkisp1-config.h header data types and documentation of
> > > > the extensible parameters format.
> > > >
> > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > ---
> > > >   include/uapi/linux/rkisp1-config.h | 482 +++++++++++++++++++++++++++++
> > > >   1 file changed, 482 insertions(+)
> > > >
> > > > diff --git a/include/uapi/linux/rkisp1-config.h b/include/uapi/linux/rkisp1-config.h
> > > > index 6eeaf8bf2362..9c93e536f270 100644
> > > > --- a/include/uapi/linux/rkisp1-config.h
> > > > +++ b/include/uapi/linux/rkisp1-config.h
> > > > @@ -996,4 +996,486 @@ struct rkisp1_stat_buffer {
> > > >   	struct rkisp1_cif_isp_stat params;
> > > >   };
> > > >
> > > > +/*---------- PART3: Extensible Configuration Parameters  ------------*/
> > > > +
> > > > +/**
> > > > + * enum rkisp1_ext_params_block_type - RkISP1 extensible params block type
> > > > + *
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS: Black level subtraction
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC: Defect pixel cluster correction
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG: Sensor de-gamma
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS: Auto white balance gains
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT: ISP filtering
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM: Bayer de-mosaic
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK: Cross-talk correction
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC: Gamma out correction
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF: De-noise pre-filter
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT: De-noise pre-filter strength
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC: Color processing
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_IE: Image effects
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC: Lens shading correction
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS: Auto white balance statistics
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS: Histogram statistics
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS: Auto exposure statistics
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS: Auto-focus statistics
> > > > + */
> > > > +enum rkisp1_ext_params_block_type {
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_IE,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL,
> > > > +};
> > >
> > > Laurent suggested referencing the enum value in the comments for each
> > > block in his review of the C55 series, and I think that that's a good
> > > idea - can we do that?
> >
> > I would like that too.
> >
> > > Otherwise, looks good to me:
> > >
> > > Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> > >
> > > > +
> > > > +/**
> > > > + * enum rkisp1_ext_params_block_state - RkISP1 extensible parameter block enable
> > > > + *					state flags
> > > > + *
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE: Do not change the block power state
> >
> > I wouldn't call that "power" state, I don't think it's about power.
> 
> How would you call it ? It's about enabling the block, should I use
> 'enable' ?

That would be better I think.

> > > > + * @RKISP1_EXT_PARAMS_BLOCK_DISABLE: Disable the HW block
> > > > + * @RKISP1_EXT_PARAMS_BLOCK_ENABLE: Enable the HW block
> > > > + */
> > > > +enum rkisp1_ext_params_block_state {
> > > > +	RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_DISABLE,
> > > > +	RKISP1_EXT_PARAMS_BLOCK_ENABLE,
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_block_header - RkISP1 extensible parameter block
> > > > + *					   header
> > > > + *
> > > > + * This structure represents the common part of all the ISP configuration
> > > > + * blocks. Each parameters block shall embed an instance of this structure type
> > > > + * as its first member, followed by the block-specific configuration data. The
> > > > + * driver inspects this common header to discern the block type and its size and
> > > > + * properly handle the block content by casting it to the correct block-specific
> > > > + * type.
> > > > + *
> > > > + * The @type field is one of the values enumerated by
> > > > + * :c:type:`rkisp1_ext_params_block_type` and specifies how the data should be
> > > > + * interpreted by the driver. The @size field specifies the size of the
> > > > + * parameters block and is used by the driver for validation purposes.
> > > > + *
> > > > + * The @state field specifies if the ISP block power state should be changed,
> > > > + * and, if it has to, if it has to be enabled to disabled. The possible
> > > > + * states are enumerated by :c:type:`rkisp1_ext_params_block_state`.
> > > > + * When userspace needs to configure and enable an ISP block it shall fully
> > > > + * populate the block configuration and the @state flag shall be set to
> > > > + * RKISP1_EXT_PARAMS_BLOCK_ENABLE. When userspace simply wants to disable the
> > > > + * ISP block the @state flag shall be set to RKISP1_EXT_PARAMS_BLOCK_DISABLE.
> >
> > You should document here what happens to the rest of the parameters in
> > the block in that case. I think they should be ignored by the driver,
> > and possibly set to 0 by userspace.
> 
> I'll add "they get ignored"
> 
> > Another option when disabling a block would be to include the header
> > only, with the size field set to the header size, and not include the
> > rest of the block contents. It's a bit pointless to include data that
> > the kernel won't use. While that would be my preference from an API
> > point of view (at least until someone points out to use cases that would
> > benefit from a different option), I haven't checked what it would imply
> > from a userspace and kernelspace implementation point of view. I could
> > agree to keeping the data part of the block in for disabled blocks if
> > omitting it would result in implementation issues.
> 
> I'm not sure it will add issues, but it would make it more cumbersome
> for userspace maybe

It's the "maybe" that I'd like to figure out :-)

> > > If
> > > > + * a new configuration of an ISP block should be applied but the power state
> > > > + * doesn't need to be changed, userspace shall fully populate the ISP block
> > > > + * configuration and the @state flag shall be set to
> > > > + * RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE.
> >
> > I don't think we need RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE for that. The
> > state can just be set to @RKISP1_EXT_PARAMS_BLOCK_ENABLE.
> 
> The existing code does
> 
>         rkisp1_isp_isr_xxx_config()
>         {
> 
>                 if (cfg_update & MODULE_XXX)
>                         rkisp1_xxx_config()
> 
>                 if (en_update & MODULE_XXX)
>                         if (enable & MODULE_XXX)
>                                 set_bit(ENABLE, MODULE_XXX)
>                         else
>                                 clear_bit(ENABLE, MODULE_XXX)
>         }
> 
> As the enablement maks is separate from the configuration mask,
> userspace can in the previous version set (cfg_update |= MODULE_XXX)
> without setting the MODULE_XXX bit in en_update.
> 
> Now, with the the introduction of this new format, the 'state' field is
> always present and I can't express with just two values the "reconfigure
> but not enable" case.

Are there cases where we would need to program the parameters but leave
the block disabled ?

> Think about LSC, when a new table has to be applied we have to
> re-program it without changing the enable bit. I mean, re-asserting
> the bit is probably harmless, but isn't it safer avoid doing that ?

If you don't want to skip the write to the register than contains the
enable bit, you can cache the bit value in the driver and only write the
register when the value changes.

> > > > + *
> > > > + * Userspace is responsible for correctly populating the parameters block header
> > > > + * fields (@type, @state and @size) and correctly populate the block-specific
> > > > + * parameters.
> > > > + *
> > > > + * For example:
> > > > + *
> > > > + * .. code-block:: c
> > > > + *
> > > > + *	void populate_bls(struct rkisp1_ext_params_block_header *block) {
> > > > + *		struct rkisp1_ext_params_bls_config *bls =
> > > > + *			(struct rkisp1_ext_params_bls_config *)block;
> > > > + *
> > > > + *		block->header.type = RKISP1_EXT_PARAMS_BLOCK_ID_BLS;
> > > > + *		block->header.state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;
> > > > + *		block->header.size = sizeof(struct rkisp1_ext_params_bls_config);
> > > > + *
> > > > + *		bls->bls_config.enable_auto = 0;
> > > > + *		bls->bls_config.fixed_val.r = blackLevelRed_;
> > > > + *		bls->bls_config.fixed_val.gr = blackLevelGreenR_;
> > > > + *		bls->bls_config.fixed_val.gb = blackLevelGreenB_;
> > > > + *		bls->bls_config.fixed_val.b = blackLevelBlue_;
> > > > + *	}
> > > > + *
> > > > + * @type: The parameters block type, see
> > > > + *	  :c:type:`rkisp1_ext_params_block_type`
> > > > + * @state: The block enable state flag, see
> > > > + *	   :c:type:`rkisp1_ext_params_block_state`
> > > > + * @size: Size (in bytes) of the parameters block, including this header
> > > > + */
> > > > +struct rkisp1_ext_params_block_header {
> > > > +	__u32 type;
> > > > +	__u32 state;
> > > > +	__u64 size;
> >
> > I think a 32-bit size would be move than enough. The header should
> > however be 64-bit aligned.
> 
> That's why size is 64 bits.

I'd rather have a 32-bit size and a 32-bit reserved field, or mark the
structure with an alignment attribute.

> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_bls_config - RkISP1 extensible params BLS config
> > > > + *
> > > > + * RkISP1 extensible parameters Black Level Subtraction configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @bls_config: Black Level Subtraction configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_bls_config`
> > > > + */
> > > > +struct rkisp1_ext_params_bls_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_bls_config bls_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_dpcc_config - RkISP1 extensible params DPCC config
> > > > + *
> > > > + * RkISP1 extensible parameters Defective Pixel Cluster Correction configuration
> > > > + * block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @dpcc_config: Defective Pixel Cluster Correction configuration, see
> > > > + *		 :c:type:`rkisp1_cif_isp_dpcc_config`
> > > > + */
> > > > +struct rkisp1_ext_params_dpcc_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_dpcc_config dpcc_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_sdg_config - RkISP1 extensible params SDG config
> > > > + *
> > > > + * RkISP1 extensible parameters Sensor Degamma configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @sdg_config: Sensor Degamma configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_sdg_config`
> > > > + */
> > > > +struct rkisp1_ext_params_sdg_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_sdg_config sdg_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_lsc_config - RkISP1 extensible params LSC config
> > > > + *
> > > > + * RkISP1 extensible parameters Lens Shading Correction configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @lsc_config: Lens Shading Correction configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_lsc_config`
> > > > + */
> > > > +struct rkisp1_ext_params_lsc_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_lsc_config lsc_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_awb_gain_config - RkISP1 extensible params AWB
> > > > + *					      gain config
> > > > + *
> > > > + * RkISP1 extensible parameters Auto-White Balance Gains configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @awb_config: Auto-White Balance Gains configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_awb_gain_config`
> > > > + */
> > > > +struct rkisp1_ext_params_awb_gain_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_awb_gain_config awb_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_flt_config - RkISP1 extensible params FLT config
> > > > + *
> > > > + * RkISP1 extensible parameters Filter configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @flt_config: Filter configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_flt_config`
> > > > + */
> > > > +struct rkisp1_ext_params_flt_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_flt_config flt_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_bdm_config - RkISP1 extensible params BDM config
> > > > + *
> > > > + * RkISP1 extensible parameters Demosaicing configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @bdm_config: Demosaicing configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_bdm_config`
> > > > + */
> > > > +struct rkisp1_ext_params_bdm_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_bdm_config bdm_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_ctk_config - RkISP1 extensible params CTK config
> > > > + *
> > > > + * RkISP1 extensible parameters Cross-Talk configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @ctk_config: Cross-Talk configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_ctk_config`
> > > > + */
> > > > +struct rkisp1_ext_params_ctk_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_ctk_config ctk_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_goc_config - RkISP1 extensible params GOC config
> > > > + *
> > > > + * RkISP1 extensible parameters Gamma-Out configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @goc_config: Gamma-Out configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_goc_config`
> > > > + */
> > > > +struct rkisp1_ext_params_goc_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_goc_config goc_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_dpf_config - RkISP1 extensible params DPF config
> > > > + *
> > > > + * RkISP1 extensible parameters De-noise Pre-Filter configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @dpf_config: De-noise Pre-Filter configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_dpf_config`
> > > > + */
> > > > +struct rkisp1_ext_params_dpf_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_dpf_config dpf_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_dpf_strength_config - RkISP1 extensible params DPF
> > > > + *						  strength config
> > > > + *
> > > > + * RkISP1 extensible parameters De-noise Pre-Filter strength configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @dpf_strength_config: De-noise Pre-Filter strength configuration, see
> > > > + *			 :c:type:`rkisp1_cif_isp_dpf_strength_config`
> > > > + */
> > > > +struct rkisp1_ext_params_dpf_strength_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_dpf_strength_config dpf_strength_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_cproc_config - RkISP1 extensible params CPROC config
> > > > + *
> > > > + * RkISP1 extensible parameters Color Processing configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @cproc_config: Color processing configuration, see
> > > > + *		  :c:type:`rkisp1_cif_isp_cproc_config`
> > > > + */
> > > > +struct rkisp1_ext_params_cproc_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_cproc_config cproc_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_ie_config - RkISP1 extensible params IE config
> > > > + *
> > > > + * RkISP1 extensible parameters Image Effect configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @ie_config: Image Effect configuration, see
> > > > + *	       :c:type:`rkisp1_cif_isp_ie_config`
> > > > + */
> > > > +struct rkisp1_ext_params_ie_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_ie_config ie_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_awb_meas_config - RkISP1 extensible params AWB
> > > > + *					      Meas config
> > > > + *
> > > > + * RkISP1 extensible parameters Auto-White Balance Measurement configuration
> > > > + * block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @awb_meas_config: Auto-White Balance measure configuration, see
> > > > + *		     :c:type:`rkisp1_cif_isp_awb_meas_config`
> > > > + */
> > > > +struct rkisp1_ext_params_awb_meas_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_awb_meas_config awb_meas_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_hst_config - RkISP1 extensible params Histogram config
> > > > + *
> > > > + * RkISP1 extensible parameters Histogram statistics configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @hst_config: Histogram statistics configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_hst_config`
> > > > + */
> > > > +struct rkisp1_ext_params_hst_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_hst_config hst_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_aec_config - RkISP1 extensible params AEC config
> > > > + *
> > > > + * RkISP1 extensible parameters Auto-Exposure statistics configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @aec_config: Auto-Exposure statistics configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_aec_config`
> > > > + */
> > > > +struct rkisp1_ext_params_aec_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_aec_config aec_config;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_afc_config - RkISP1 extensible params AFC config
> > > > + *
> > > > + * RkISP1 extensible parameters Auto-Focus statistics configuration block
> > > > + *
> > > > + * @header: The RkISP1 extensible parameters header, see
> > > > + *	    :c:type:`rkisp1_ext_params_block_header`
> > > > + * @afc_config: Auto-Focus statistics configuration, see
> > > > + *		:c:type:`rkisp1_cif_isp_afc_config`
> > > > + */
> > > > +struct rkisp1_ext_params_afc_config {
> > > > +	struct rkisp1_ext_params_block_header header;
> > > > +	struct rkisp1_cif_isp_afc_config afc_config;
> > > > +};
> > > > +
> > > > +#define RKISP1_EXT_PARAMS_MAX_SIZE					\
> > > > +	(sizeof(struct rkisp1_ext_params_bls_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_dpcc_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_sdg_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_lsc_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_awb_gain_config)		+\
> > > > +	sizeof(struct rkisp1_ext_params_flt_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_bdm_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_ctk_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_goc_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_dpf_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_dpf_strength_config)		+\
> > > > +	sizeof(struct rkisp1_ext_params_cproc_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_ie_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_awb_meas_config)		+\
> > > > +	sizeof(struct rkisp1_ext_params_hst_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_aec_config)			+\
> > > > +	sizeof(struct rkisp1_ext_params_afc_config))
> > > > +
> > > > +/**
> > > > + * enum rksip1_ext_param_buffer_version - RkISP1 extensible parameters version
> > > > + *
> > > > + * @RKISP1_EXT_PARAM_BUFFER_V1: First version of RkISP1 extensible parameters
> > > > + */
> > > > +enum rksip1_ext_param_buffer_version {
> > > > +	RKISP1_EXT_PARAM_BUFFER_V1 = 1,
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct rkisp1_ext_params_cfg - RkISP1 extensible parameters configuration
> > > > + *
> > > > + * This struct contains the configuration parameters of the RkISP1 ISP
> > > > + * algorithms, serialized by userspace into a data buffer. Each configuration
> > > > + * parameter block is represented by a block-specific structure which contains a
> > > > + * :c:type:`rkisp1_ext_params_block_header` entry as first member. Userspace
> > > > + * populates the @data buffer with configuration parameters for the blocks that
> > > > + * it intends to configure. As a consequence, the data buffer effective size
> > > > + * changes according to the number of ISP blocks that userspace intends to
> > > > + * configure and is set by userspace in the @total_size field.
> > > > + *
> > > > + * The parameters buffer is versioned by the @version field to allow modifying
> > > > + * and extending its definition. Userspace shall populate the @version field to
> > > > + * inform the driver about the version it intends to use. The driver will parse
> > > > + * and handle the @data buffer according to the data layout specific to the
> > > > + * indicated version and return an error if the desired version is not
> > > > + * supported.
> > > > + *
> > > > + * For each ISP block that userspace wants to configure, a block-specific
> > > > + * structure is appended to the @data buffer, one after the other without gaps
> >
> > I think we should align all the blocks to a 64 bits boundary. Otherwise
> > we'll have unaligned access issues, as well as layout differences
> > between 32-bitand 64-bit userspace.
> 
> Each block header is 64 bits aligned, what follows it's not
> 
> Should I add
>         __attribute__((aligned(8))
> 
> To each block ?

I think that would be good, just to be safe, as well as explicit.

> > > > + * in between nor overlaps. Userspace shall populate the @total_size field with
> > > > + * the effective size, in bytes, of the @data buffer.
> > > > + *
> > > > + * The expected memory layout of the parameters buffer is::
> > > > + *
> > > > + *	+-------------------- struct rkisp1_ext_params_cfg -------------------+
> > > > + *	| version = RKISP_EXT_PARAMS_BUFFER_V1;                               |
> > > > + *	| total_size = sizeof(struct rkisp1_ext_params_bls_config)            |
> > > > + *	|              sizeof(struct rkisp1_ext_params_dpcc_config);          |
> > > > + *	| +------------------------- data  ---------------------------------+ |
> > > > + *	| | +------------- struct rkisp1_ext_params_bls_config -----------+ | |
> > > > + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> > > > + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS;                | | | |
> > > > + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> > > > + *	| | | | size = sizeof(struct rkisp1_ext_params_bls_config);     | | | |
> > > > + *	| | | +---------------------------------------------------------+ | | |
> > > > + *	| | | +---------- struct rkisp1_cif_isp_bls_config -------------+ | | |
> > > > + *	| | | | enable_auto = 0;                                        | | | |
> > > > + *	| | | | fixed_val.r = 256;                                      | | | |
> > > > + *	| | | | fixed_val.gr = 256;                                     | | | |
> > > > + *	| | | | fixed_val.gb = 256;                                     | | | |
> > > > + *	| | | | fixed_val.b = 256;                                      | | | |
> > > > + *	| | | +---------------------------------------------------------+ | | |
> > > > + *	| | +------------ struct rkisp1_ext_params_dpcc_config -----------+ | |
> > > > + *	| | | +-------- struct rkisp1_ext_params_block_header  ---------+ | | |
> > > > + *	| | | | type = RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC;               | | | |
> > > > + *	| | | | state = RKISP1_EXT_PARAMS_BLOCK_ENABLE;                 | | | |
> > > > + *	| | | | size = sizeof(struct rkisp1_ext_params_dpcc_config);    | | | |
> > > > + *	| | | +---------------------------------------------------------+ | | |
> > > > + *	| | | +---------- struct rkisp1_cif_isp_dpcc_config ------------+ | | |
> > > > + *	| | | | mode = RKISP1_CIF_ISP_DPCC_MODE_STAGE1_ENABLE;          | | | |
> > > > + *	| | | | output_mode =                                           | | | |
> > > > + *	| | | |   RKISP1_CIF_ISP_DPCC_OUTPUT_MODE_STAGE1_INCL_G_CENTER; | | | |
> > > > + *	| | | | set_use = ... ;                                         | | | |
> > > > + *	| | | | ...  = ... ;                                            | | | |
> > > > + *	| | | +---------------------------------------------------------+ | | |
> > > > + *	| | +-------------------------------------------------------------+ | |
> > > > + *	| +-----------------------------------------------------------------+ |
> > > > + *	+---------------------------------------------------------------------+
> > > > + *
> > > > + * @version: The RkISP1 extensible parameters buffer version, see
> > > > + *	     :c:type:`rksip1_ext_param_buffer_version`
> > > > + * @total_size: The RkISP1 configuration data effective size, excluding this
> > > > + *		header
> > > > + * @data: The RkISP1 extensible configuration data blocks
> > > > + */
> > > > +struct rkisp1_ext_params_cfg {
> > > > +	__u32 version;
> >
> > On a 64-bit system there will be a 32-bit hole here, while on a 32-bit
> > system there won't. This means that a 32-bit userspace won't run on a
> > 64-bit kernel. You can add a a __u32 reserved field to fix that.
> 
> Or make version and __u64 ?

I don't think the version needs 64 bits :-)

> > > > +	__u64 total_size;
> > > > +	__u8 data[RKISP1_EXT_PARAMS_MAX_SIZE];
> > > > +};
> > > > +
> > > >   #endif /* _UAPI_RKISP1_CONFIG_H */

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
  2024-06-05 16:54 ` [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 10:00   ` Dan Scally
  2024-06-20  9:31   ` Paul Elder
  2024-06-05 16:54 ` [PATCH 3/8] media: rkisp1: Remove cached format info Jacopo Mondi
                   ` (5 subsequent siblings)
  7 siblings, 2 replies; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

Add a new format definition for the RkISP1 extensible parameters
format and document it.

Document the usage of the new format in the rkisp1 admin guide.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 Documentation/admin-guide/media/rkisp1.rst    | 11 +++-
 .../media/v4l/metafmt-rkisp1.rst              | 62 ++++++++++++++++---
 drivers/media/v4l2-core/v4l2-ioctl.c          |  1 +
 include/uapi/linux/videodev2.h                |  1 +
 4 files changed, 64 insertions(+), 11 deletions(-)

diff --git a/Documentation/admin-guide/media/rkisp1.rst b/Documentation/admin-guide/media/rkisp1.rst
index 6f14d9561fa5..934c25e191df 100644
--- a/Documentation/admin-guide/media/rkisp1.rst
+++ b/Documentation/admin-guide/media/rkisp1.rst
@@ -114,11 +114,18 @@ to be applied to the hardware during a video stream, allowing userspace
 to dynamically modify values such as black level, cross talk corrections
 and others.
 
-The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and
-userspace should set
+The ISP driver supports two different parameters configuration methods, the
+`fixed parameters format` or the `extensible parameters format`.
+
+When using the `fixed parameters` method the buffer format is defined by struct
+:c:type:`rkisp1_params_cfg`, and userspace set
 :ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-rk-isp1-params>` as the
 dataformat.
 
+When using the fixed parameters method the buffer format is defined by struct
+:c:type:`rkisp1_ext_params_cfg`, and userspace set
+:ref:`V4L2_META_FMT_RK_ISP1_EXT_PARAMS <v4l2-meta-fmt-rk-isp1-ext-params>` as
+the dataformat.
 
 Capturing Video Frames Example
 ==============================
diff --git a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
index fa04f00bcd2e..6ff776d071a3 100644
--- a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
+++ b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
@@ -1,28 +1,72 @@
 .. SPDX-License-Identifier: GPL-2.0
 
-.. _v4l2-meta-fmt-rk-isp1-params:
-
 .. _v4l2-meta-fmt-rk-isp1-stat-3a:
 
-*****************************************************************************
-V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s')
-*****************************************************************************
+************************************************************************************************************************
+V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s'), V4L2_META_FMT_RK_ISP1_EXT_PARAMS ('rk1e')
+************************************************************************************************************************
 
+========================
 Configuration parameters
 ========================
 
-The configuration parameters are passed to the
+The configuration of the RkISP1 ISP is performed by userspace by providing
+parameters for the ISP to the driver using the :c:type:`v4l2_meta_format`
+interface.
+
+There are currently two methods that allow to configure the ISP, the `fixed
+parameters` configuration format and the `extensible parameters` configuration
+format.
+
+.. _v4l2-meta-fmt-rk-isp1-params:
+
+Fixed parameters configuration format
+=====================================
+
+When using the fixed configuration format, parameters are passed to the
 :ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
-the :c:type:`v4l2_meta_format` interface. The buffer contains
-a single instance of the C structure :c:type:`rkisp1_params_cfg` defined in
-``rkisp1-config.h``. So the structure can be obtained from the buffer by:
+the `V4L2_META_FMT_RK_ISP1_PARAMS` meta pixel format.
+
+The buffer contains a single instance of the C structure
+:c:type:`rkisp1_params_cfg` defined in ``rkisp1-config.h``. So the structure can
+be obtained from the buffer by:
 
 .. code-block:: c
 
 	struct rkisp1_params_cfg *params = (struct rkisp1_params_cfg*) buffer;
 
+As the members of :c:type:`rkisp1_params_cfg` are defined in the
+``rkisp1-config.h`` header, the structure layout is immutable and cannot be
+extended further. For this reason the fixed configuration format only allows the
+configuration of the ISP blocks supported at the time when the structure had
+been defined in the header file, as introducing new parameters or modifying the
+existing ones would change the buffer layout and cause breakages in existing
+applications.
+
+.. _v4l2-meta-fmt-rk-isp1-ext-params:
+
+Extensible parameters configuration format
+==========================================
+
+When using the extensible configuration format, parameters are passed to the
+:ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
+the `V4L2_META_FMT_RK_ISP1_EXT_PARAMS` meta pixel format.
+
+The buffer contains a single instance of the C structure
+:c:type:`rkisp1_ext_params_cfg` defined in ``rkisp1-config.h``. The
+:c:type:`rkisp1_ext_params_cfg` structure is designed to allow userspace to
+populate the data buffer with only the configuration data for the ISP block it
+intends to configure. The extensible parameters format design allows to define
+new block types and new configuration parameters and defines a versioning scheme
+so that it can be extended and versioned without breaking compatibility with
+existing applications.
+
+For these reasons, this configuration method if preferred over the `fixed
+parameters` format alternative.
+
 .. rkisp1_stat_buffer
 
+===========================
 3A and histogram statistics
 ===========================
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 4c76d17b4629..aefdc1efd24b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
 	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
 	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
+	case V4L2_META_FMT_RK_ISP1_EXT_PARAMS:	descr = "Rockchip ISP1 Ext 3A Params"; break;
 	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
 	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
 	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index fe6b67e83751..7c2a303c6f59 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -840,6 +840,7 @@ struct v4l2_pix_format {
 /* Vendor specific - used for RK_ISP1 camera sub-system */
 #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
 #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
+#define V4L2_META_FMT_RK_ISP1_EXT_PARAMS	v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */
 
 #ifdef __KERNEL__
 /*
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS
  2024-06-05 16:54 ` [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS Jacopo Mondi
@ 2024-06-12 10:00   ` Dan Scally
  2024-06-12 14:35     ` Laurent Pinchart
  2024-06-20  9:31   ` Paul Elder
  1 sibling, 1 reply; 38+ messages in thread
From: Dan Scally @ 2024-06-12 10:00 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo

On 05/06/2024 17:54, Jacopo Mondi wrote:
> Add a new format definition for the RkISP1 extensible parameters
> format and document it.
>
> Document the usage of the new format in the rkisp1 admin guide.
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>   Documentation/admin-guide/media/rkisp1.rst    | 11 +++-
>   .../media/v4l/metafmt-rkisp1.rst              | 62 ++++++++++++++++---
>   drivers/media/v4l2-core/v4l2-ioctl.c          |  1 +
>   include/uapi/linux/videodev2.h                |  1 +
>   4 files changed, 64 insertions(+), 11 deletions(-)
>
> diff --git a/Documentation/admin-guide/media/rkisp1.rst b/Documentation/admin-guide/media/rkisp1.rst
> index 6f14d9561fa5..934c25e191df 100644
> --- a/Documentation/admin-guide/media/rkisp1.rst
> +++ b/Documentation/admin-guide/media/rkisp1.rst
> @@ -114,11 +114,18 @@ to be applied to the hardware during a video stream, allowing userspace
>   to dynamically modify values such as black level, cross talk corrections
>   and others.
>   
> -The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and
> -userspace should set
> +The ISP driver supports two different parameters configuration methods, the
> +`fixed parameters format` or the `extensible parameters format`.
> +
> +When using the `fixed parameters` method the buffer format is defined by struct
> +:c:type:`rkisp1_params_cfg`, and userspace set
>   :ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-rk-isp1-params>` as the
>   dataformat.
>   
> +When using the fixed parameters method the buffer format is defined by struct
> +:c:type:`rkisp1_ext_params_cfg`, and userspace set
s/set/should set
> +:ref:`V4L2_META_FMT_RK_ISP1_EXT_PARAMS <v4l2-meta-fmt-rk-isp1-ext-params>` as
> +the dataformat.
>   
>   Capturing Video Frames Example
>   ==============================
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> index fa04f00bcd2e..6ff776d071a3 100644
> --- a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> +++ b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> @@ -1,28 +1,72 @@
>   .. SPDX-License-Identifier: GPL-2.0
>   
> -.. _v4l2-meta-fmt-rk-isp1-params:
> -
>   .. _v4l2-meta-fmt-rk-isp1-stat-3a:
>   
> -*****************************************************************************
> -V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s')
> -*****************************************************************************
> +************************************************************************************************************************
> +V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s'), V4L2_META_FMT_RK_ISP1_EXT_PARAMS ('rk1e')
> +************************************************************************************************************************
>   
> +========================
>   Configuration parameters
>   ========================
>   
> -The configuration parameters are passed to the
> +The configuration of the RkISP1 ISP is performed by userspace by providing
> +parameters for the ISP to the driver using the :c:type:`v4l2_meta_format`
> +interface.
> +
> +There are currently two methods that allow to configure the ISP, the `fixed
> +parameters` configuration format and the `extensible parameters` configuration
> +format.
> +
> +.. _v4l2-meta-fmt-rk-isp1-params:
> +
> +Fixed parameters configuration format
> +=====================================
> +
> +When using the fixed configuration format, parameters are passed to the
>   :ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
> -the :c:type:`v4l2_meta_format` interface. The buffer contains
> -a single instance of the C structure :c:type:`rkisp1_params_cfg` defined in
> -``rkisp1-config.h``. So the structure can be obtained from the buffer by:
> +the `V4L2_META_FMT_RK_ISP1_PARAMS` meta pixel format.
> +
> +The buffer contains a single instance of the C structure
> +:c:type:`rkisp1_params_cfg` defined in ``rkisp1-config.h``. So the structure can
> +be obtained from the buffer by:
>   
>   .. code-block:: c
>   
>   	struct rkisp1_params_cfg *params = (struct rkisp1_params_cfg*) buffer;
>   
> +As the members of :c:type:`rkisp1_params_cfg` are defined in the
> +``rkisp1-config.h`` header, the structure layout is immutable and cannot be
> +extended further. For this reason the fixed configuration format only allows the
> +configuration of the ISP blocks supported at the time when the structure had
> +been defined in the header file, as introducing new parameters or modifying the
> +existing ones would change the buffer layout and cause breakages in existing
> +applications.


I'm not sure I'd bother with the last sentence; up to you though. Possibly a quick explainer as to 
why there's two separate methods and why the extensible one should be preferred could be in the 
"Configuration parameters" section instead.

> +
> +.. _v4l2-meta-fmt-rk-isp1-ext-params:
> +
> +Extensible parameters configuration format
> +==========================================
> +
> +When using the extensible configuration format, parameters are passed to the
> +:ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
> +the `V4L2_META_FMT_RK_ISP1_EXT_PARAMS` meta pixel format.
> +
> +The buffer contains a single instance of the C structure
> +:c:type:`rkisp1_ext_params_cfg` defined in ``rkisp1-config.h``. The
> +:c:type:`rkisp1_ext_params_cfg` structure is designed to allow userspace to
> +populate the data buffer with only the configuration data for the ISP block it
s/block/block(s)?
> +intends to configure. The extensible parameters format design allows to define
> +new block types and new configuration parameters and defines a versioning scheme
Perhaps "allows developers to define new block types to support new configuration parameters"?
> +so that it can be extended and versioned without breaking compatibility with
> +existing applications.
> +
> +For these reasons, this configuration method if preferred over the `fixed
> +parameters` format alternative.
> +
>   .. rkisp1_stat_buffer
>   
> +===========================
>   3A and histogram statistics
>   ===========================
>   
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 4c76d17b4629..aefdc1efd24b 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>   	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
>   	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
>   	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
> +	case V4L2_META_FMT_RK_ISP1_EXT_PARAMS:	descr = "Rockchip ISP1 Ext 3A Params"; break;

I think spell out "Extensible" personally; we already allow breaking like length limits here.


Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>

>   	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
>   	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
>   	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index fe6b67e83751..7c2a303c6f59 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -840,6 +840,7 @@ struct v4l2_pix_format {
>   /* Vendor specific - used for RK_ISP1 camera sub-system */
>   #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>   #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
> +#define V4L2_META_FMT_RK_ISP1_EXT_PARAMS	v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */
>   
>   #ifdef __KERNEL__
>   /*

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS
  2024-06-12 10:00   ` Dan Scally
@ 2024-06-12 14:35     ` Laurent Pinchart
  2024-06-12 15:09       ` Dan Scally
  0 siblings, 1 reply; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 14:35 UTC (permalink / raw)
  To: Dan Scally
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 12, 2024 at 11:00:37AM +0100, Daniel Scally wrote:
> On 05/06/2024 17:54, Jacopo Mondi wrote:
> > Add a new format definition for the RkISP1 extensible parameters
> > format and document it.
> >
> > Document the usage of the new format in the rkisp1 admin guide.
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >   Documentation/admin-guide/media/rkisp1.rst    | 11 +++-
> >   .../media/v4l/metafmt-rkisp1.rst              | 62 ++++++++++++++++---
> >   drivers/media/v4l2-core/v4l2-ioctl.c          |  1 +
> >   include/uapi/linux/videodev2.h                |  1 +
> >   4 files changed, 64 insertions(+), 11 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/media/rkisp1.rst b/Documentation/admin-guide/media/rkisp1.rst
> > index 6f14d9561fa5..934c25e191df 100644
> > --- a/Documentation/admin-guide/media/rkisp1.rst
> > +++ b/Documentation/admin-guide/media/rkisp1.rst
> > @@ -114,11 +114,18 @@ to be applied to the hardware during a video stream, allowing userspace
> >   to dynamically modify values such as black level, cross talk corrections
> >   and others.
> >   
> > -The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and
> > -userspace should set
> > +The ISP driver supports two different parameters configuration methods, the
> > +`fixed parameters format` or the `extensible parameters format`.
> > +
> > +When using the `fixed parameters` method the buffer format is defined by struct
> > +:c:type:`rkisp1_params_cfg`, and userspace set
> >  :ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-rk-isp1-params>` as the
> >  dataformat.
> >   
> > +When using the fixed parameters method the buffer format is defined by struct
> > +:c:type:`rkisp1_ext_params_cfg`, and userspace set
>
> s/set/should set

Above too.

> > +:ref:`V4L2_META_FMT_RK_ISP1_EXT_PARAMS <v4l2-meta-fmt-rk-isp1-ext-params>` as
> > +the dataformat.
> >   
> >   Capturing Video Frames Example
> >   ==============================
> > diff --git a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> > index fa04f00bcd2e..6ff776d071a3 100644
> > --- a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> > +++ b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> > @@ -1,28 +1,72 @@
> >   .. SPDX-License-Identifier: GPL-2.0
> >   
> > -.. _v4l2-meta-fmt-rk-isp1-params:
> > -
> >   .. _v4l2-meta-fmt-rk-isp1-stat-3a:
> >   
> > -*****************************************************************************
> > -V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s')
> > -*****************************************************************************
> > +************************************************************************************************************************
> > +V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s'), V4L2_META_FMT_RK_ISP1_EXT_PARAMS ('rk1e')
> > +************************************************************************************************************************
> >   
> > +========================
> >  Configuration parameters
> >  ========================
> >   
> > -The configuration parameters are passed to the
> > +The configuration of the RkISP1 ISP is performed by userspace by providing
> > +parameters for the ISP to the driver using the :c:type:`v4l2_meta_format`
> > +interface.
> > +
> > +There are currently two methods that allow to configure the ISP, the `fixed

s/currently //

> > +parameters` configuration format and the `extensible parameters` configuration
> > +format.
> > +
> > +.. _v4l2-meta-fmt-rk-isp1-params:
> > +
> > +Fixed parameters configuration format
> > +=====================================
> > +
> > +When using the fixed configuration format, parameters are passed to the
> > :ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
> > -the :c:type:`v4l2_meta_format` interface. The buffer contains
> > -a single instance of the C structure :c:type:`rkisp1_params_cfg` defined in
> > -``rkisp1-config.h``. So the structure can be obtained from the buffer by:
> > +the `V4L2_META_FMT_RK_ISP1_PARAMS` meta pixel format.

s/meta pixel format/meta format/ I think. Same below.

> > +
> > +The buffer contains a single instance of the C structure
> > +:c:type:`rkisp1_params_cfg` defined in ``rkisp1-config.h``. So the structure can
> > +be obtained from the buffer by:
> >   
> >   .. code-block:: c
> >   
> >   	struct rkisp1_params_cfg *params = (struct rkisp1_params_cfg*) buffer;
> >   
> > +As the members of :c:type:`rkisp1_params_cfg` are defined in the
> > +``rkisp1-config.h`` header, the structure layout is immutable and cannot be
> > +extended further. For this reason the fixed configuration format only allows the
> > +configuration of the ISP blocks supported at the time when the structure had
> > +been defined in the header file, as introducing new parameters or modifying the
> > +existing ones would change the buffer layout and cause breakages in existing
> > +applications.
> 
> I'm not sure I'd bother with the last sentence; up to you though.
> Possibly a quick explainer as to why there's two separate methods and
> why the extensible one should be preferred could be in the
> "Configuration parameters" section instead.

I agree. The last sentence here belongs to a commit message more than to
the in-kernel documentation. I would replace it with "This method
supports a subset of the ISP features only, new applications should use
the extensible parameters method." or something similar.

> > +
> > +.. _v4l2-meta-fmt-rk-isp1-ext-params:
> > +
> > +Extensible parameters configuration format
> > +==========================================
> > +
> > +When using the extensible configuration format, parameters are passed to the
> > +:ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
> > +the `V4L2_META_FMT_RK_ISP1_EXT_PARAMS` meta pixel format.
> > +
> > +The buffer contains a single instance of the C structure
> > +:c:type:`rkisp1_ext_params_cfg` defined in ``rkisp1-config.h``. The
> > +:c:type:`rkisp1_ext_params_cfg` structure is designed to allow userspace to
> > +populate the data buffer with only the configuration data for the ISP block it
>
> s/block/block(s)?
>
> > +intends to configure. The extensible parameters format design allows to define
> > +new block types and new configuration parameters and defines a versioning scheme
>
> Perhaps "allows developers to define new block types to support new configuration parameters"?
>
> > +so that it can be extended and versioned without breaking compatibility with
> > +existing applications.
> > +
> > +For these reasons, this configuration method if preferred over the `fixed
> > +parameters` format alternative.
> > +
> >   .. rkisp1_stat_buffer
> >   
> > +===========================
> >   3A and histogram statistics
> >   ===========================
> >   
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 4c76d17b4629..aefdc1efd24b 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >   	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
> >   	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
> >   	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
> > +	case V4L2_META_FMT_RK_ISP1_EXT_PARAMS:	descr = "Rockchip ISP1 Ext 3A Params"; break;
> 
> I think spell out "Extensible" personally; we already allow breaking like length limits here.

Format descriptions are limited to 32 bytes in the UAPI :-(

> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> 
> >   	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
> >   	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
> >   	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index fe6b67e83751..7c2a303c6f59 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -840,6 +840,7 @@ struct v4l2_pix_format {
> >   /* Vendor specific - used for RK_ISP1 camera sub-system */
> >   #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
> >   #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
> > +#define V4L2_META_FMT_RK_ISP1_EXT_PARAMS	v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */
> >   
> >   #ifdef __KERNEL__
> >   /*

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS
  2024-06-12 14:35     ` Laurent Pinchart
@ 2024-06-12 15:09       ` Dan Scally
  0 siblings, 0 replies; 38+ messages in thread
From: Dan Scally @ 2024-06-12 15:09 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner


On 12/06/2024 15:35, Laurent Pinchart wrote:
> On Wed, Jun 12, 2024 at 11:00:37AM +0100, Daniel Scally wrote:
>> On 05/06/2024 17:54, Jacopo Mondi wrote:
>>> Add a new format definition for the RkISP1 extensible parameters
>>> format and document it.
>>>
>>> Document the usage of the new format in the rkisp1 admin guide.
>>>
>>> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
>>> ---
>>>    Documentation/admin-guide/media/rkisp1.rst    | 11 +++-
>>>    .../media/v4l/metafmt-rkisp1.rst              | 62 ++++++++++++++++---
>>>    drivers/media/v4l2-core/v4l2-ioctl.c          |  1 +
>>>    include/uapi/linux/videodev2.h                |  1 +
>>>    4 files changed, 64 insertions(+), 11 deletions(-)
>>>
>>> diff --git a/Documentation/admin-guide/media/rkisp1.rst b/Documentation/admin-guide/media/rkisp1.rst
>>> index 6f14d9561fa5..934c25e191df 100644
>>> --- a/Documentation/admin-guide/media/rkisp1.rst
>>> +++ b/Documentation/admin-guide/media/rkisp1.rst
>>> @@ -114,11 +114,18 @@ to be applied to the hardware during a video stream, allowing userspace
>>>    to dynamically modify values such as black level, cross talk corrections
>>>    and others.
>>>    
>>> -The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and
>>> -userspace should set
>>> +The ISP driver supports two different parameters configuration methods, the
>>> +`fixed parameters format` or the `extensible parameters format`.
>>> +
>>> +When using the `fixed parameters` method the buffer format is defined by struct
>>> +:c:type:`rkisp1_params_cfg`, and userspace set
>>>   :ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-rk-isp1-params>` as the
>>>   dataformat.
>>>    
>>> +When using the fixed parameters method the buffer format is defined by struct
>>> +:c:type:`rkisp1_ext_params_cfg`, and userspace set
>> s/set/should set
> Above too.
>
>>> +:ref:`V4L2_META_FMT_RK_ISP1_EXT_PARAMS <v4l2-meta-fmt-rk-isp1-ext-params>` as
>>> +the dataformat.
>>>    
>>>    Capturing Video Frames Example
>>>    ==============================
>>> diff --git a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
>>> index fa04f00bcd2e..6ff776d071a3 100644
>>> --- a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
>>> +++ b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
>>> @@ -1,28 +1,72 @@
>>>    .. SPDX-License-Identifier: GPL-2.0
>>>    
>>> -.. _v4l2-meta-fmt-rk-isp1-params:
>>> -
>>>    .. _v4l2-meta-fmt-rk-isp1-stat-3a:
>>>    
>>> -*****************************************************************************
>>> -V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s')
>>> -*****************************************************************************
>>> +************************************************************************************************************************
>>> +V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s'), V4L2_META_FMT_RK_ISP1_EXT_PARAMS ('rk1e')
>>> +************************************************************************************************************************
>>>    
>>> +========================
>>>   Configuration parameters
>>>   ========================
>>>    
>>> -The configuration parameters are passed to the
>>> +The configuration of the RkISP1 ISP is performed by userspace by providing
>>> +parameters for the ISP to the driver using the :c:type:`v4l2_meta_format`
>>> +interface.
>>> +
>>> +There are currently two methods that allow to configure the ISP, the `fixed
> s/currently //
>
>>> +parameters` configuration format and the `extensible parameters` configuration
>>> +format.
>>> +
>>> +.. _v4l2-meta-fmt-rk-isp1-params:
>>> +
>>> +Fixed parameters configuration format
>>> +=====================================
>>> +
>>> +When using the fixed configuration format, parameters are passed to the
>>> :ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
>>> -the :c:type:`v4l2_meta_format` interface. The buffer contains
>>> -a single instance of the C structure :c:type:`rkisp1_params_cfg` defined in
>>> -``rkisp1-config.h``. So the structure can be obtained from the buffer by:
>>> +the `V4L2_META_FMT_RK_ISP1_PARAMS` meta pixel format.
> s/meta pixel format/meta format/ I think. Same below.
>
>>> +
>>> +The buffer contains a single instance of the C structure
>>> +:c:type:`rkisp1_params_cfg` defined in ``rkisp1-config.h``. So the structure can
>>> +be obtained from the buffer by:
>>>    
>>>    .. code-block:: c
>>>    
>>>    	struct rkisp1_params_cfg *params = (struct rkisp1_params_cfg*) buffer;
>>>    
>>> +As the members of :c:type:`rkisp1_params_cfg` are defined in the
>>> +``rkisp1-config.h`` header, the structure layout is immutable and cannot be
>>> +extended further. For this reason the fixed configuration format only allows the
>>> +configuration of the ISP blocks supported at the time when the structure had
>>> +been defined in the header file, as introducing new parameters or modifying the
>>> +existing ones would change the buffer layout and cause breakages in existing
>>> +applications.
>> I'm not sure I'd bother with the last sentence; up to you though.
>> Possibly a quick explainer as to why there's two separate methods and
>> why the extensible one should be preferred could be in the
>> "Configuration parameters" section instead.
> I agree. The last sentence here belongs to a commit message more than to
> the in-kernel documentation. I would replace it with "This method
> supports a subset of the ISP features only, new applications should use
> the extensible parameters method." or something similar.
>
>>> +
>>> +.. _v4l2-meta-fmt-rk-isp1-ext-params:
>>> +
>>> +Extensible parameters configuration format
>>> +==========================================
>>> +
>>> +When using the extensible configuration format, parameters are passed to the
>>> +:ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
>>> +the `V4L2_META_FMT_RK_ISP1_EXT_PARAMS` meta pixel format.
>>> +
>>> +The buffer contains a single instance of the C structure
>>> +:c:type:`rkisp1_ext_params_cfg` defined in ``rkisp1-config.h``. The
>>> +:c:type:`rkisp1_ext_params_cfg` structure is designed to allow userspace to
>>> +populate the data buffer with only the configuration data for the ISP block it
>> s/block/block(s)?
>>
>>> +intends to configure. The extensible parameters format design allows to define
>>> +new block types and new configuration parameters and defines a versioning scheme
>> Perhaps "allows developers to define new block types to support new configuration parameters"?
>>
>>> +so that it can be extended and versioned without breaking compatibility with
>>> +existing applications.
>>> +
>>> +For these reasons, this configuration method if preferred over the `fixed
>>> +parameters` format alternative.
>>> +
>>>    .. rkisp1_stat_buffer
>>>    
>>> +===========================
>>>    3A and histogram statistics
>>>    ===========================
>>>    
>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index 4c76d17b4629..aefdc1efd24b 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>    	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
>>>    	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
>>>    	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
>>> +	case V4L2_META_FMT_RK_ISP1_EXT_PARAMS:	descr = "Rockchip ISP1 Ext 3A Params"; break;
>> I think spell out "Extensible" personally; we already allow breaking like length limits here.
> Format descriptions are limited to 32 bytes in the UAPI :-(

Today I learned!
>
>> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
>>
>>>    	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
>>>    	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
>>>    	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index fe6b67e83751..7c2a303c6f59 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -840,6 +840,7 @@ struct v4l2_pix_format {
>>>    /* Vendor specific - used for RK_ISP1 camera sub-system */
>>>    #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>>>    #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>>> +#define V4L2_META_FMT_RK_ISP1_EXT_PARAMS	v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */
>>>    
>>>    #ifdef __KERNEL__
>>>    /*

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS
  2024-06-05 16:54 ` [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS Jacopo Mondi
  2024-06-12 10:00   ` Dan Scally
@ 2024-06-20  9:31   ` Paul Elder
  1 sibling, 0 replies; 38+ messages in thread
From: Paul Elder @ 2024-06-20  9:31 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Linux Media Mailing List, Laurent Pinchart, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

On Wed, Jun 05, 2024 at 06:54:21PM +0200, Jacopo Mondi wrote:
> Add a new format definition for the RkISP1 extensible parameters
> format and document it.
> 
> Document the usage of the new format in the rkisp1 admin guide.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>  Documentation/admin-guide/media/rkisp1.rst    | 11 +++-
>  .../media/v4l/metafmt-rkisp1.rst              | 62 ++++++++++++++++---
>  drivers/media/v4l2-core/v4l2-ioctl.c          |  1 +
>  include/uapi/linux/videodev2.h                |  1 +
>  4 files changed, 64 insertions(+), 11 deletions(-)
> 
> diff --git a/Documentation/admin-guide/media/rkisp1.rst b/Documentation/admin-guide/media/rkisp1.rst
> index 6f14d9561fa5..934c25e191df 100644
> --- a/Documentation/admin-guide/media/rkisp1.rst
> +++ b/Documentation/admin-guide/media/rkisp1.rst
> @@ -114,11 +114,18 @@ to be applied to the hardware during a video stream, allowing userspace
>  to dynamically modify values such as black level, cross talk corrections
>  and others.
>  
> -The buffer format is defined by struct :c:type:`rkisp1_params_cfg`, and
> -userspace should set
> +The ISP driver supports two different parameters configuration methods, the
> +`fixed parameters format` or the `extensible parameters format`.
> +
> +When using the `fixed parameters` method the buffer format is defined by struct
> +:c:type:`rkisp1_params_cfg`, and userspace set
>  :ref:`V4L2_META_FMT_RK_ISP1_PARAMS <v4l2-meta-fmt-rk-isp1-params>` as the
>  dataformat.
>  
> +When using the fixed parameters method the buffer format is defined by struct

s/fixed/extensible/ ?

> +:c:type:`rkisp1_ext_params_cfg`, and userspace set
> +:ref:`V4L2_META_FMT_RK_ISP1_EXT_PARAMS <v4l2-meta-fmt-rk-isp1-ext-params>` as
> +the dataformat.
>  
>  Capturing Video Frames Example
>  ==============================
> diff --git a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> index fa04f00bcd2e..6ff776d071a3 100644
> --- a/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> +++ b/Documentation/userspace-api/media/v4l/metafmt-rkisp1.rst
> @@ -1,28 +1,72 @@
>  .. SPDX-License-Identifier: GPL-2.0
>  
> -.. _v4l2-meta-fmt-rk-isp1-params:
> -
>  .. _v4l2-meta-fmt-rk-isp1-stat-3a:
>  
> -*****************************************************************************
> -V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s')
> -*****************************************************************************
> +************************************************************************************************************************
> +V4L2_META_FMT_RK_ISP1_PARAMS ('rk1p'), V4L2_META_FMT_RK_ISP1_STAT_3A ('rk1s'), V4L2_META_FMT_RK_ISP1_EXT_PARAMS ('rk1e')
> +************************************************************************************************************************
>  
> +========================
>  Configuration parameters
>  ========================
>  
> -The configuration parameters are passed to the
> +The configuration of the RkISP1 ISP is performed by userspace by providing
> +parameters for the ISP to the driver using the :c:type:`v4l2_meta_format`
> +interface.
> +
> +There are currently two methods that allow to configure the ISP, the `fixed
> +parameters` configuration format and the `extensible parameters` configuration
> +format.
> +
> +.. _v4l2-meta-fmt-rk-isp1-params:
> +
> +Fixed parameters configuration format
> +=====================================
> +
> +When using the fixed configuration format, parameters are passed to the
>  :ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
> -the :c:type:`v4l2_meta_format` interface. The buffer contains
> -a single instance of the C structure :c:type:`rkisp1_params_cfg` defined in
> -``rkisp1-config.h``. So the structure can be obtained from the buffer by:
> +the `V4L2_META_FMT_RK_ISP1_PARAMS` meta pixel format.
> +
> +The buffer contains a single instance of the C structure
> +:c:type:`rkisp1_params_cfg` defined in ``rkisp1-config.h``. So the structure can
> +be obtained from the buffer by:
>  
>  .. code-block:: c
>  
>  	struct rkisp1_params_cfg *params = (struct rkisp1_params_cfg*) buffer;
>  
> +As the members of :c:type:`rkisp1_params_cfg` are defined in the
> +``rkisp1-config.h`` header, the structure layout is immutable and cannot be
> +extended further. For this reason the fixed configuration format only allows the
> +configuration of the ISP blocks supported at the time when the structure had
> +been defined in the header file, as introducing new parameters or modifying the
> +existing ones would change the buffer layout and cause breakages in existing
> +applications.
> +
> +.. _v4l2-meta-fmt-rk-isp1-ext-params:
> +
> +Extensible parameters configuration format
> +==========================================
> +
> +When using the extensible configuration format, parameters are passed to the
> +:ref:`rkisp1_params <rkisp1_params>` metadata output video node, using
> +the `V4L2_META_FMT_RK_ISP1_EXT_PARAMS` meta pixel format.
> +
> +The buffer contains a single instance of the C structure
> +:c:type:`rkisp1_ext_params_cfg` defined in ``rkisp1-config.h``. The
> +:c:type:`rkisp1_ext_params_cfg` structure is designed to allow userspace to
> +populate the data buffer with only the configuration data for the ISP block it
> +intends to configure. The extensible parameters format design allows to define

s/allows to define/allows defining/

> +new block types and new configuration parameters and defines a versioning scheme

s/parameters/parameters,/


Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>

> +so that it can be extended and versioned without breaking compatibility with
> +existing applications.
> +
> +For these reasons, this configuration method if preferred over the `fixed
> +parameters` format alternative.
> +
>  .. rkisp1_stat_buffer
>  
> +===========================
>  3A and histogram statistics
>  ===========================
>  
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 4c76d17b4629..aefdc1efd24b 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1456,6 +1456,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_META_FMT_VIVID:       descr = "Vivid Metadata"; break;
>  	case V4L2_META_FMT_RK_ISP1_PARAMS:	descr = "Rockchip ISP1 3A Parameters"; break;
>  	case V4L2_META_FMT_RK_ISP1_STAT_3A:	descr = "Rockchip ISP1 3A Statistics"; break;
> +	case V4L2_META_FMT_RK_ISP1_EXT_PARAMS:	descr = "Rockchip ISP1 Ext 3A Params"; break;
>  	case V4L2_PIX_FMT_NV12_8L128:	descr = "NV12 (8x128 Linear)"; break;
>  	case V4L2_PIX_FMT_NV12M_8L128:	descr = "NV12M (8x128 Linear)"; break;
>  	case V4L2_PIX_FMT_NV12_10BE_8L128:	descr = "10-bit NV12 (8x128 Linear, BE)"; break;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index fe6b67e83751..7c2a303c6f59 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -840,6 +840,7 @@ struct v4l2_pix_format {
>  /* Vendor specific - used for RK_ISP1 camera sub-system */
>  #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>  #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
> +#define V4L2_META_FMT_RK_ISP1_EXT_PARAMS	v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */
>  
>  #ifdef __KERNEL__
>  /*
> -- 
> 2.45.1
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 3/8] media: rkisp1: Remove cached format info
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
  2024-06-05 16:54 ` [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format Jacopo Mondi
  2024-06-05 16:54 ` [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 10:06   ` Dan Scally
                     ` (2 more replies)
  2024-06-05 16:54 ` [PATCH 4/8] media: rkisp1: Add support for ext format Jacopo Mondi
                   ` (4 subsequent siblings)
  7 siblings, 3 replies; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

The struct rkisp1_params type contains a v4l2_format instance which
is used to store the buffer format and sizes to be used in enum_fmt and
g_fmt operations.

To prepare for supporting multiple meta output formats, to introduce
support for extensible buffer formats, remove the cached format info
and initialize them explicitly in the enum_fmt and g_fmt operations.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../platform/rockchip/rkisp1/rkisp1-common.h  |  2 --
 .../platform/rockchip/rkisp1/rkisp1-params.c  | 28 ++++++-------------
 2 files changed, 9 insertions(+), 21 deletions(-)

diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
index 26573f6ae575..2a715f964f6e 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
@@ -372,7 +372,6 @@ struct rkisp1_params_ops {
  * @ops:		pointer to the variant-specific operations
  * @config_lock:	locks the buffer list 'params'
  * @params:		queue of rkisp1_buffer
- * @vdev_fmt:		v4l2_format of the metadata format
  * @quantization:	the quantization configured on the isp's src pad
  * @raw_type:		the bayer pattern on the isp video sink pad
  */
@@ -383,7 +382,6 @@ struct rkisp1_params {
 
 	spinlock_t config_lock; /* locks the buffers list 'params' */
 	struct list_head params;
-	struct v4l2_format vdev_fmt;
 
 	enum v4l2_quantization quantization;
 	enum v4l2_ycbcr_encoding ycbcr_encoding;
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
index 173d1ea41874..1f449f29b241 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
@@ -1742,12 +1742,11 @@ static int rkisp1_params_enum_fmt_meta_out(struct file *file, void *priv,
 					   struct v4l2_fmtdesc *f)
 {
 	struct video_device *video = video_devdata(file);
-	struct rkisp1_params *params = video_get_drvdata(video);
 
 	if (f->index > 0 || f->type != video->queue->type)
 		return -EINVAL;
 
-	f->pixelformat = params->vdev_fmt.fmt.meta.dataformat;
+	f->pixelformat = V4L2_META_FMT_RK_ISP1_PARAMS;
 
 	return 0;
 }
@@ -1756,15 +1755,14 @@ static int rkisp1_params_g_fmt_meta_out(struct file *file, void *fh,
 					struct v4l2_format *f)
 {
 	struct video_device *video = video_devdata(file);
-	struct rkisp1_params *params = video_get_drvdata(video);
 	struct v4l2_meta_format *meta = &f->fmt.meta;
 
 	if (f->type != video->queue->type)
 		return -EINVAL;
 
 	memset(meta, 0, sizeof(*meta));
-	meta->dataformat = params->vdev_fmt.fmt.meta.dataformat;
-	meta->buffersize = params->vdev_fmt.fmt.meta.buffersize;
+	meta->dataformat = V4L2_META_FMT_RK_ISP1_PARAMS;
+	meta->buffersize = sizeof(struct rkisp1_params_cfg);
 
 	return 0;
 }
@@ -1897,19 +1895,6 @@ static int rkisp1_params_init_vb2_queue(struct vb2_queue *q,
 	return vb2_queue_init(q);
 }
 
-static void rkisp1_init_params(struct rkisp1_params *params)
-{
-	params->vdev_fmt.fmt.meta.dataformat =
-		V4L2_META_FMT_RK_ISP1_PARAMS;
-	params->vdev_fmt.fmt.meta.buffersize =
-		sizeof(struct rkisp1_params_cfg);
-
-	if (params->rkisp1->info->isp_ver == RKISP1_V12)
-		params->ops = &rkisp1_v12_params_ops;
-	else
-		params->ops = &rkisp1_v10_params_ops;
-}
-
 int rkisp1_params_register(struct rkisp1_device *rkisp1)
 {
 	struct rkisp1_params *params = &rkisp1->params;
@@ -1938,7 +1923,12 @@ int rkisp1_params_register(struct rkisp1_device *rkisp1)
 	vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_META_OUTPUT;
 	vdev->vfl_dir = VFL_DIR_TX;
 	rkisp1_params_init_vb2_queue(vdev->queue, params);
-	rkisp1_init_params(params);
+
+	if (params->rkisp1->info->isp_ver == RKISP1_V12)
+		params->ops = &rkisp1_v12_params_ops;
+	else
+		params->ops = &rkisp1_v10_params_ops;
+
 	video_set_drvdata(vdev, params);
 
 	node->pad.flags = MEDIA_PAD_FL_SOURCE;
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/8] media: rkisp1: Remove cached format info
  2024-06-05 16:54 ` [PATCH 3/8] media: rkisp1: Remove cached format info Jacopo Mondi
@ 2024-06-12 10:06   ` Dan Scally
  2024-06-12 14:47   ` Laurent Pinchart
  2024-06-20  9:41   ` Paul Elder
  2 siblings, 0 replies; 38+ messages in thread
From: Dan Scally @ 2024-06-12 10:06 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo

On 05/06/2024 17:54, Jacopo Mondi wrote:
> The struct rkisp1_params type contains a v4l2_format instance which
> is used to store the buffer format and sizes to be used in enum_fmt and
> g_fmt operations.
>
> To prepare for supporting multiple meta output formats, to introduce
> support for extensible buffer formats, remove the cached format info
> and initialize them explicitly in the enum_fmt and g_fmt operations.
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> ---
>   .../platform/rockchip/rkisp1/rkisp1-common.h  |  2 --
>   .../platform/rockchip/rkisp1/rkisp1-params.c  | 28 ++++++-------------
>   2 files changed, 9 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> index 26573f6ae575..2a715f964f6e 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> @@ -372,7 +372,6 @@ struct rkisp1_params_ops {
>    * @ops:		pointer to the variant-specific operations
>    * @config_lock:	locks the buffer list 'params'
>    * @params:		queue of rkisp1_buffer
> - * @vdev_fmt:		v4l2_format of the metadata format
>    * @quantization:	the quantization configured on the isp's src pad
>    * @raw_type:		the bayer pattern on the isp video sink pad
>    */
> @@ -383,7 +382,6 @@ struct rkisp1_params {
>   
>   	spinlock_t config_lock; /* locks the buffers list 'params' */
>   	struct list_head params;
> -	struct v4l2_format vdev_fmt;
>   
>   	enum v4l2_quantization quantization;
>   	enum v4l2_ycbcr_encoding ycbcr_encoding;
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index 173d1ea41874..1f449f29b241 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -1742,12 +1742,11 @@ static int rkisp1_params_enum_fmt_meta_out(struct file *file, void *priv,
>   					   struct v4l2_fmtdesc *f)
>   {
>   	struct video_device *video = video_devdata(file);
> -	struct rkisp1_params *params = video_get_drvdata(video);
>   
>   	if (f->index > 0 || f->type != video->queue->type)
>   		return -EINVAL;
>   
> -	f->pixelformat = params->vdev_fmt.fmt.meta.dataformat;
> +	f->pixelformat = V4L2_META_FMT_RK_ISP1_PARAMS;
>   
>   	return 0;
>   }
> @@ -1756,15 +1755,14 @@ static int rkisp1_params_g_fmt_meta_out(struct file *file, void *fh,
>   					struct v4l2_format *f)
>   {
>   	struct video_device *video = video_devdata(file);
> -	struct rkisp1_params *params = video_get_drvdata(video);
>   	struct v4l2_meta_format *meta = &f->fmt.meta;
>   
>   	if (f->type != video->queue->type)
>   		return -EINVAL;
>   
>   	memset(meta, 0, sizeof(*meta));
> -	meta->dataformat = params->vdev_fmt.fmt.meta.dataformat;
> -	meta->buffersize = params->vdev_fmt.fmt.meta.buffersize;
> +	meta->dataformat = V4L2_META_FMT_RK_ISP1_PARAMS;
> +	meta->buffersize = sizeof(struct rkisp1_params_cfg);
>   
>   	return 0;
>   }
> @@ -1897,19 +1895,6 @@ static int rkisp1_params_init_vb2_queue(struct vb2_queue *q,
>   	return vb2_queue_init(q);
>   }
>   
> -static void rkisp1_init_params(struct rkisp1_params *params)
> -{
> -	params->vdev_fmt.fmt.meta.dataformat =
> -		V4L2_META_FMT_RK_ISP1_PARAMS;
> -	params->vdev_fmt.fmt.meta.buffersize =
> -		sizeof(struct rkisp1_params_cfg);
> -
> -	if (params->rkisp1->info->isp_ver == RKISP1_V12)
> -		params->ops = &rkisp1_v12_params_ops;
> -	else
> -		params->ops = &rkisp1_v10_params_ops;
> -}
> -
>   int rkisp1_params_register(struct rkisp1_device *rkisp1)
>   {
>   	struct rkisp1_params *params = &rkisp1->params;
> @@ -1938,7 +1923,12 @@ int rkisp1_params_register(struct rkisp1_device *rkisp1)
>   	vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_META_OUTPUT;
>   	vdev->vfl_dir = VFL_DIR_TX;
>   	rkisp1_params_init_vb2_queue(vdev->queue, params);
> -	rkisp1_init_params(params);
> +
> +	if (params->rkisp1->info->isp_ver == RKISP1_V12)
> +		params->ops = &rkisp1_v12_params_ops;
> +	else
> +		params->ops = &rkisp1_v10_params_ops;
> +
>   	video_set_drvdata(vdev, params);
>   
>   	node->pad.flags = MEDIA_PAD_FL_SOURCE;

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/8] media: rkisp1: Remove cached format info
  2024-06-05 16:54 ` [PATCH 3/8] media: rkisp1: Remove cached format info Jacopo Mondi
  2024-06-12 10:06   ` Dan Scally
@ 2024-06-12 14:47   ` Laurent Pinchart
  2024-06-20  9:41   ` Paul Elder
  2 siblings, 0 replies; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 14:47 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Linux Media Mailing List, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Daniel Scally, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo,

Thank you for the patch.

On Wed, Jun 05, 2024 at 06:54:22PM +0200, Jacopo Mondi wrote:
> The struct rkisp1_params type contains a v4l2_format instance which
> is used to store the buffer format and sizes to be used in enum_fmt and
> g_fmt operations.
> 
> To prepare for supporting multiple meta output formats, to introduce
> support for extensible buffer formats, remove the cached format info
> and initialize them explicitly in the enum_fmt and g_fmt operations.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  .../platform/rockchip/rkisp1/rkisp1-common.h  |  2 --
>  .../platform/rockchip/rkisp1/rkisp1-params.c  | 28 ++++++-------------
>  2 files changed, 9 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> index 26573f6ae575..2a715f964f6e 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> @@ -372,7 +372,6 @@ struct rkisp1_params_ops {
>   * @ops:		pointer to the variant-specific operations
>   * @config_lock:	locks the buffer list 'params'
>   * @params:		queue of rkisp1_buffer
> - * @vdev_fmt:		v4l2_format of the metadata format
>   * @quantization:	the quantization configured on the isp's src pad
>   * @raw_type:		the bayer pattern on the isp video sink pad
>   */
> @@ -383,7 +382,6 @@ struct rkisp1_params {
>  
>  	spinlock_t config_lock; /* locks the buffers list 'params' */
>  	struct list_head params;
> -	struct v4l2_format vdev_fmt;
>  
>  	enum v4l2_quantization quantization;
>  	enum v4l2_ycbcr_encoding ycbcr_encoding;
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index 173d1ea41874..1f449f29b241 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -1742,12 +1742,11 @@ static int rkisp1_params_enum_fmt_meta_out(struct file *file, void *priv,
>  					   struct v4l2_fmtdesc *f)
>  {
>  	struct video_device *video = video_devdata(file);
> -	struct rkisp1_params *params = video_get_drvdata(video);
>  
>  	if (f->index > 0 || f->type != video->queue->type)
>  		return -EINVAL;
>  
> -	f->pixelformat = params->vdev_fmt.fmt.meta.dataformat;
> +	f->pixelformat = V4L2_META_FMT_RK_ISP1_PARAMS;
>  
>  	return 0;
>  }
> @@ -1756,15 +1755,14 @@ static int rkisp1_params_g_fmt_meta_out(struct file *file, void *fh,
>  					struct v4l2_format *f)
>  {
>  	struct video_device *video = video_devdata(file);
> -	struct rkisp1_params *params = video_get_drvdata(video);
>  	struct v4l2_meta_format *meta = &f->fmt.meta;
>  
>  	if (f->type != video->queue->type)
>  		return -EINVAL;
>  
>  	memset(meta, 0, sizeof(*meta));
> -	meta->dataformat = params->vdev_fmt.fmt.meta.dataformat;
> -	meta->buffersize = params->vdev_fmt.fmt.meta.buffersize;
> +	meta->dataformat = V4L2_META_FMT_RK_ISP1_PARAMS;
> +	meta->buffersize = sizeof(struct rkisp1_params_cfg);
>  
>  	return 0;
>  }
> @@ -1897,19 +1895,6 @@ static int rkisp1_params_init_vb2_queue(struct vb2_queue *q,
>  	return vb2_queue_init(q);
>  }
>  
> -static void rkisp1_init_params(struct rkisp1_params *params)
> -{
> -	params->vdev_fmt.fmt.meta.dataformat =
> -		V4L2_META_FMT_RK_ISP1_PARAMS;
> -	params->vdev_fmt.fmt.meta.buffersize =
> -		sizeof(struct rkisp1_params_cfg);
> -
> -	if (params->rkisp1->info->isp_ver == RKISP1_V12)
> -		params->ops = &rkisp1_v12_params_ops;
> -	else
> -		params->ops = &rkisp1_v10_params_ops;
> -}
> -
>  int rkisp1_params_register(struct rkisp1_device *rkisp1)
>  {
>  	struct rkisp1_params *params = &rkisp1->params;
> @@ -1938,7 +1923,12 @@ int rkisp1_params_register(struct rkisp1_device *rkisp1)
>  	vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_META_OUTPUT;
>  	vdev->vfl_dir = VFL_DIR_TX;
>  	rkisp1_params_init_vb2_queue(vdev->queue, params);
> -	rkisp1_init_params(params);
> +
> +	if (params->rkisp1->info->isp_ver == RKISP1_V12)
> +		params->ops = &rkisp1_v12_params_ops;
> +	else
> +		params->ops = &rkisp1_v10_params_ops;
> +
>  	video_set_drvdata(vdev, params);
>  
>  	node->pad.flags = MEDIA_PAD_FL_SOURCE;

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 3/8] media: rkisp1: Remove cached format info
  2024-06-05 16:54 ` [PATCH 3/8] media: rkisp1: Remove cached format info Jacopo Mondi
  2024-06-12 10:06   ` Dan Scally
  2024-06-12 14:47   ` Laurent Pinchart
@ 2024-06-20  9:41   ` Paul Elder
  2 siblings, 0 replies; 38+ messages in thread
From: Paul Elder @ 2024-06-20  9:41 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Linux Media Mailing List, Laurent Pinchart, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

On Wed, Jun 05, 2024 at 06:54:22PM +0200, Jacopo Mondi wrote:
> The struct rkisp1_params type contains a v4l2_format instance which
> is used to store the buffer format and sizes to be used in enum_fmt and
> g_fmt operations.
> 
> To prepare for supporting multiple meta output formats, to introduce
> support for extensible buffer formats, remove the cached format info
> and initialize them explicitly in the enum_fmt and g_fmt operations.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>

> ---
>  .../platform/rockchip/rkisp1/rkisp1-common.h  |  2 --
>  .../platform/rockchip/rkisp1/rkisp1-params.c  | 28 ++++++-------------
>  2 files changed, 9 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> index 26573f6ae575..2a715f964f6e 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> @@ -372,7 +372,6 @@ struct rkisp1_params_ops {
>   * @ops:		pointer to the variant-specific operations
>   * @config_lock:	locks the buffer list 'params'
>   * @params:		queue of rkisp1_buffer
> - * @vdev_fmt:		v4l2_format of the metadata format
>   * @quantization:	the quantization configured on the isp's src pad
>   * @raw_type:		the bayer pattern on the isp video sink pad
>   */
> @@ -383,7 +382,6 @@ struct rkisp1_params {
>  
>  	spinlock_t config_lock; /* locks the buffers list 'params' */
>  	struct list_head params;
> -	struct v4l2_format vdev_fmt;
>  
>  	enum v4l2_quantization quantization;
>  	enum v4l2_ycbcr_encoding ycbcr_encoding;
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index 173d1ea41874..1f449f29b241 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -1742,12 +1742,11 @@ static int rkisp1_params_enum_fmt_meta_out(struct file *file, void *priv,
>  					   struct v4l2_fmtdesc *f)
>  {
>  	struct video_device *video = video_devdata(file);
> -	struct rkisp1_params *params = video_get_drvdata(video);
>  
>  	if (f->index > 0 || f->type != video->queue->type)
>  		return -EINVAL;
>  
> -	f->pixelformat = params->vdev_fmt.fmt.meta.dataformat;
> +	f->pixelformat = V4L2_META_FMT_RK_ISP1_PARAMS;
>  
>  	return 0;
>  }
> @@ -1756,15 +1755,14 @@ static int rkisp1_params_g_fmt_meta_out(struct file *file, void *fh,
>  					struct v4l2_format *f)
>  {
>  	struct video_device *video = video_devdata(file);
> -	struct rkisp1_params *params = video_get_drvdata(video);
>  	struct v4l2_meta_format *meta = &f->fmt.meta;
>  
>  	if (f->type != video->queue->type)
>  		return -EINVAL;
>  
>  	memset(meta, 0, sizeof(*meta));
> -	meta->dataformat = params->vdev_fmt.fmt.meta.dataformat;
> -	meta->buffersize = params->vdev_fmt.fmt.meta.buffersize;
> +	meta->dataformat = V4L2_META_FMT_RK_ISP1_PARAMS;
> +	meta->buffersize = sizeof(struct rkisp1_params_cfg);
>  
>  	return 0;
>  }
> @@ -1897,19 +1895,6 @@ static int rkisp1_params_init_vb2_queue(struct vb2_queue *q,
>  	return vb2_queue_init(q);
>  }
>  
> -static void rkisp1_init_params(struct rkisp1_params *params)
> -{
> -	params->vdev_fmt.fmt.meta.dataformat =
> -		V4L2_META_FMT_RK_ISP1_PARAMS;
> -	params->vdev_fmt.fmt.meta.buffersize =
> -		sizeof(struct rkisp1_params_cfg);
> -
> -	if (params->rkisp1->info->isp_ver == RKISP1_V12)
> -		params->ops = &rkisp1_v12_params_ops;
> -	else
> -		params->ops = &rkisp1_v10_params_ops;
> -}
> -
>  int rkisp1_params_register(struct rkisp1_device *rkisp1)
>  {
>  	struct rkisp1_params *params = &rkisp1->params;
> @@ -1938,7 +1923,12 @@ int rkisp1_params_register(struct rkisp1_device *rkisp1)
>  	vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_META_OUTPUT;
>  	vdev->vfl_dir = VFL_DIR_TX;
>  	rkisp1_params_init_vb2_queue(vdev->queue, params);
> -	rkisp1_init_params(params);
> +
> +	if (params->rkisp1->info->isp_ver == RKISP1_V12)
> +		params->ops = &rkisp1_v12_params_ops;
> +	else
> +		params->ops = &rkisp1_v10_params_ops;
> +
>  	video_set_drvdata(vdev, params);
>  
>  	node->pad.flags = MEDIA_PAD_FL_SOURCE;
> -- 
> 2.45.1
> 

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 4/8] media: rkisp1: Add support for ext format
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
                   ` (2 preceding siblings ...)
  2024-06-05 16:54 ` [PATCH 3/8] media: rkisp1: Remove cached format info Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 10:46   ` Dan Scally
  2024-06-05 16:54 ` [PATCH 5/8] media: rkisp1: Implement extensible params support Jacopo Mondi
                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

Add support to the rkisp1 driver for the extensible parameters format.

Allow the driver to enumerate the existing and the new format and
implement support for the try_fmt and s_fmt operations.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../platform/rockchip/rkisp1/rkisp1-common.h  |  1 +
 .../platform/rockchip/rkisp1/rkisp1-params.c  | 87 +++++++++++++++++--
 2 files changed, 79 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
index 2a715f964f6e..0bddae8dbdb1 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
@@ -383,6 +383,7 @@ struct rkisp1_params {
 	spinlock_t config_lock; /* locks the buffers list 'params' */
 	struct list_head params;
 
+	struct v4l2_meta_format metafmt;
 	enum v4l2_quantization quantization;
 	enum v4l2_ycbcr_encoding ycbcr_encoding;
 	enum rkisp1_fmt_raw_pat_type raw_type;
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
index 1f449f29b241..6f99c7dad758 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
@@ -33,6 +33,34 @@
 #define RKISP1_ISP_CC_COEFF(n) \
 			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
 
+enum rkisp1_params_formats {
+	RKISP1_PARAMS_FIXED,
+	RKISP1_PARAMS_EXTENSIBLE,
+	RKISP1_PARAMS_NUM_FMT,
+};
+
+static const struct v4l2_meta_format rkisp1_params_formats[] = {
+	[RKISP1_PARAMS_FIXED] = {
+		.dataformat = V4L2_META_FMT_RK_ISP1_PARAMS,
+		.buffersize = sizeof(struct rkisp1_params_cfg),
+	},
+	[RKISP1_PARAMS_EXTENSIBLE] = {
+		.dataformat = V4L2_META_FMT_RK_ISP1_EXT_PARAMS,
+		.buffersize = sizeof(struct rkisp1_ext_params_cfg),
+	},
+};
+
+static const struct v4l2_meta_format *
+rkisp1_params_get_format_info(u32 dataformat)
+{
+	for (unsigned int i = 0; i < RKISP1_PARAMS_NUM_FMT; i++) {
+		if (rkisp1_params_formats[i].dataformat == dataformat)
+			return &rkisp1_params_formats[i];
+	}
+
+	return &rkisp1_params_formats[RKISP1_PARAMS_FIXED];
+}
+
 static inline void
 rkisp1_param_set_bits(struct rkisp1_params *params, u32 reg, u32 bit_mask)
 {
@@ -1742,11 +1770,13 @@ static int rkisp1_params_enum_fmt_meta_out(struct file *file, void *priv,
 					   struct v4l2_fmtdesc *f)
 {
 	struct video_device *video = video_devdata(file);
+	const struct v4l2_meta_format *metafmt;
 
-	if (f->index > 0 || f->type != video->queue->type)
+	if (f->index >= RKISP1_PARAMS_NUM_FMT || f->type != video->queue->type)
 		return -EINVAL;
 
-	f->pixelformat = V4L2_META_FMT_RK_ISP1_PARAMS;
+	metafmt = &rkisp1_params_formats[f->index];
+	f->pixelformat = metafmt->dataformat;
 
 	return 0;
 }
@@ -1755,14 +1785,44 @@ static int rkisp1_params_g_fmt_meta_out(struct file *file, void *fh,
 					struct v4l2_format *f)
 {
 	struct video_device *video = video_devdata(file);
+	struct rkisp1_params *params = video_get_drvdata(video);
 	struct v4l2_meta_format *meta = &f->fmt.meta;
 
 	if (f->type != video->queue->type)
 		return -EINVAL;
 
 	memset(meta, 0, sizeof(*meta));
-	meta->dataformat = V4L2_META_FMT_RK_ISP1_PARAMS;
-	meta->buffersize = sizeof(struct rkisp1_params_cfg);
+	*meta = params->metafmt;
+
+	return 0;
+}
+
+static int rkisp1_params_try_fmt_meta_out(struct file *file, void *fh,
+					  struct v4l2_format *f)
+{
+	struct video_device *video = video_devdata(file);
+	struct v4l2_meta_format *meta = &f->fmt.meta;
+
+	if (f->type != video->queue->type)
+		return -EINVAL;
+
+	*meta = *rkisp1_params_get_format_info(meta->dataformat);
+
+	return 0;
+}
+
+static int rkisp1_params_s_fmt_meta_out(struct file *file, void *fh,
+					struct v4l2_format *f)
+{
+	struct video_device *video = video_devdata(file);
+	struct rkisp1_params *params = video_get_drvdata(video);
+	struct v4l2_meta_format *meta = &f->fmt.meta;
+
+	if (f->type != video->queue->type)
+		return -EINVAL;
+
+	*meta = *rkisp1_params_get_format_info(meta->dataformat);
+	params->metafmt = *meta;
 
 	return 0;
 }
@@ -1792,8 +1852,8 @@ static const struct v4l2_ioctl_ops rkisp1_params_ioctl = {
 	.vidioc_streamoff = vb2_ioctl_streamoff,
 	.vidioc_enum_fmt_meta_out = rkisp1_params_enum_fmt_meta_out,
 	.vidioc_g_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
-	.vidioc_s_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
-	.vidioc_try_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
+	.vidioc_s_fmt_meta_out = rkisp1_params_s_fmt_meta_out,
+	.vidioc_try_fmt_meta_out = rkisp1_params_try_fmt_meta_out,
 	.vidioc_querycap = rkisp1_params_querycap,
 	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
 	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
@@ -1805,13 +1865,16 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
 					 unsigned int sizes[],
 					 struct device *alloc_devs[])
 {
+	struct rkisp1_params *params = vq->drv_priv;
+	size_t buf_size = params->metafmt.buffersize;
+
 	*num_buffers = clamp_t(u32, *num_buffers,
 			       RKISP1_ISP_PARAMS_REQ_BUFS_MIN,
 			       RKISP1_ISP_PARAMS_REQ_BUFS_MAX);
 
 	*num_planes = 1;
 
-	sizes[0] = sizeof(struct rkisp1_params_cfg);
+	sizes[0] = buf_size;
 
 	return 0;
 }
@@ -1831,10 +1894,14 @@ static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
 
 static int rkisp1_params_vb2_buf_prepare(struct vb2_buffer *vb)
 {
-	if (vb2_plane_size(vb, 0) < sizeof(struct rkisp1_params_cfg))
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct rkisp1_params *params = vq->drv_priv;
+	size_t buf_size = params->metafmt.buffersize;
+
+	if (vb2_plane_size(vb, 0) < buf_size)
 		return -EINVAL;
 
-	vb2_set_plane_payload(vb, 0, sizeof(struct rkisp1_params_cfg));
+	vb2_set_plane_payload(vb, 0, buf_size);
 
 	return 0;
 }
@@ -1929,6 +1996,8 @@ int rkisp1_params_register(struct rkisp1_device *rkisp1)
 	else
 		params->ops = &rkisp1_v10_params_ops;
 
+	params->metafmt = rkisp1_params_formats[RKISP1_PARAMS_FIXED];
+
 	video_set_drvdata(vdev, params);
 
 	node->pad.flags = MEDIA_PAD_FL_SOURCE;
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 4/8] media: rkisp1: Add support for ext format
  2024-06-05 16:54 ` [PATCH 4/8] media: rkisp1: Add support for ext format Jacopo Mondi
@ 2024-06-12 10:46   ` Dan Scally
  2024-06-12 14:51     ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Dan Scally @ 2024-06-12 10:46 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo, thanks for the patch

On 05/06/2024 17:54, Jacopo Mondi wrote:
> Add support to the rkisp1 driver for the extensible parameters format.
>
> Allow the driver to enumerate the existing and the new format and
> implement support for the try_fmt and s_fmt operations.
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>


Isn't this too early in the set? I'd expect this to come after the actual support for handling the 
extensible buffers was done.

> ---
>   .../platform/rockchip/rkisp1/rkisp1-common.h  |  1 +
>   .../platform/rockchip/rkisp1/rkisp1-params.c  | 87 +++++++++++++++++--
>   2 files changed, 79 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> index 2a715f964f6e..0bddae8dbdb1 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> @@ -383,6 +383,7 @@ struct rkisp1_params {
>   	spinlock_t config_lock; /* locks the buffers list 'params' */
>   	struct list_head params;
>   
> +	struct v4l2_meta_format metafmt;
>   	enum v4l2_quantization quantization;
>   	enum v4l2_ycbcr_encoding ycbcr_encoding;
>   	enum rkisp1_fmt_raw_pat_type raw_type;
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index 1f449f29b241..6f99c7dad758 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -33,6 +33,34 @@
>   #define RKISP1_ISP_CC_COEFF(n) \
>   			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
>   
> +enum rkisp1_params_formats {
> +	RKISP1_PARAMS_FIXED,
> +	RKISP1_PARAMS_EXTENSIBLE,
> +	RKISP1_PARAMS_NUM_FMT,
> +};
> +
> +static const struct v4l2_meta_format rkisp1_params_formats[] = {
> +	[RKISP1_PARAMS_FIXED] = {
> +		.dataformat = V4L2_META_FMT_RK_ISP1_PARAMS,
> +		.buffersize = sizeof(struct rkisp1_params_cfg),
> +	},
> +	[RKISP1_PARAMS_EXTENSIBLE] = {
> +		.dataformat = V4L2_META_FMT_RK_ISP1_EXT_PARAMS,
> +		.buffersize = sizeof(struct rkisp1_ext_params_cfg),
> +	},
> +};
> +
> +static const struct v4l2_meta_format *
> +rkisp1_params_get_format_info(u32 dataformat)
> +{
> +	for (unsigned int i = 0; i < RKISP1_PARAMS_NUM_FMT; i++) {
> +		if (rkisp1_params_formats[i].dataformat == dataformat)
> +			return &rkisp1_params_formats[i];
> +	}
> +
> +	return &rkisp1_params_formats[RKISP1_PARAMS_FIXED];
> +}
> +
>   static inline void
>   rkisp1_param_set_bits(struct rkisp1_params *params, u32 reg, u32 bit_mask)
>   {
> @@ -1742,11 +1770,13 @@ static int rkisp1_params_enum_fmt_meta_out(struct file *file, void *priv,
>   					   struct v4l2_fmtdesc *f)
>   {
>   	struct video_device *video = video_devdata(file);
> +	const struct v4l2_meta_format *metafmt;
>   
> -	if (f->index > 0 || f->type != video->queue->type)
> +	if (f->index >= RKISP1_PARAMS_NUM_FMT || f->type != video->queue->type)
>   		return -EINVAL;
>   
> -	f->pixelformat = V4L2_META_FMT_RK_ISP1_PARAMS;
> +	metafmt = &rkisp1_params_formats[f->index];
> +	f->pixelformat = metafmt->dataformat;
>   
>   	return 0;
>   }
> @@ -1755,14 +1785,44 @@ static int rkisp1_params_g_fmt_meta_out(struct file *file, void *fh,
>   					struct v4l2_format *f)
>   {
>   	struct video_device *video = video_devdata(file);
> +	struct rkisp1_params *params = video_get_drvdata(video);
>   	struct v4l2_meta_format *meta = &f->fmt.meta;
>   
>   	if (f->type != video->queue->type)
>   		return -EINVAL;
>   
>   	memset(meta, 0, sizeof(*meta));
> -	meta->dataformat = V4L2_META_FMT_RK_ISP1_PARAMS;
> -	meta->buffersize = sizeof(struct rkisp1_params_cfg);
> +	*meta = params->metafmt;
> +
> +	return 0;
> +}
> +
> +static int rkisp1_params_try_fmt_meta_out(struct file *file, void *fh,
> +					  struct v4l2_format *f)
> +{
> +	struct video_device *video = video_devdata(file);
> +	struct v4l2_meta_format *meta = &f->fmt.meta;
> +
> +	if (f->type != video->queue->type)
> +		return -EINVAL;
> +
> +	*meta = *rkisp1_params_get_format_info(meta->dataformat);
> +
> +	return 0;
> +}
> +
> +static int rkisp1_params_s_fmt_meta_out(struct file *file, void *fh,
> +					struct v4l2_format *f)
> +{
> +	struct video_device *video = video_devdata(file);
> +	struct rkisp1_params *params = video_get_drvdata(video);
> +	struct v4l2_meta_format *meta = &f->fmt.meta;
> +
> +	if (f->type != video->queue->type)
> +		return -EINVAL;
> +
> +	*meta = *rkisp1_params_get_format_info(meta->dataformat);
> +	params->metafmt = *meta;
>   
>   	return 0;
>   }
> @@ -1792,8 +1852,8 @@ static const struct v4l2_ioctl_ops rkisp1_params_ioctl = {
>   	.vidioc_streamoff = vb2_ioctl_streamoff,
>   	.vidioc_enum_fmt_meta_out = rkisp1_params_enum_fmt_meta_out,
>   	.vidioc_g_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
> -	.vidioc_s_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
> -	.vidioc_try_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
> +	.vidioc_s_fmt_meta_out = rkisp1_params_s_fmt_meta_out,
> +	.vidioc_try_fmt_meta_out = rkisp1_params_try_fmt_meta_out,
>   	.vidioc_querycap = rkisp1_params_querycap,
>   	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
>   	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> @@ -1805,13 +1865,16 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
>   					 unsigned int sizes[],
>   					 struct device *alloc_devs[])
>   {
> +	struct rkisp1_params *params = vq->drv_priv;
> +	size_t buf_size = params->metafmt.buffersize;
> +
>   	*num_buffers = clamp_t(u32, *num_buffers,
>   			       RKISP1_ISP_PARAMS_REQ_BUFS_MIN,
>   			       RKISP1_ISP_PARAMS_REQ_BUFS_MAX);
>   
>   	*num_planes = 1;
>   
> -	sizes[0] = sizeof(struct rkisp1_params_cfg);
> +	sizes[0] = buf_size;
sizes[0] = params->metafmt.buffersize; would save a variable - up to you.
>   
>   	return 0;
>   }
> @@ -1831,10 +1894,14 @@ static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
>   
>   static int rkisp1_params_vb2_buf_prepare(struct vb2_buffer *vb)
>   {
> -	if (vb2_plane_size(vb, 0) < sizeof(struct rkisp1_params_cfg))
> +	struct vb2_queue *vq = vb->vb2_queue;
> +	struct rkisp1_params *params = vq->drv_priv;
> +	size_t buf_size = params->metafmt.buffersize;
> +
> +	if (vb2_plane_size(vb, 0) < buf_size)
>   		return -EINVAL;
>   
> -	vb2_set_plane_payload(vb, 0, sizeof(struct rkisp1_params_cfg));
> +	vb2_set_plane_payload(vb, 0, buf_size);
>   
>   	return 0;
>   }
> @@ -1929,6 +1996,8 @@ int rkisp1_params_register(struct rkisp1_device *rkisp1)
>   	else
>   		params->ops = &rkisp1_v10_params_ops;
>   
> +	params->metafmt = rkisp1_params_formats[RKISP1_PARAMS_FIXED];
> +
>   	video_set_drvdata(vdev, params);
>   
>   	node->pad.flags = MEDIA_PAD_FL_SOURCE;

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 4/8] media: rkisp1: Add support for ext format
  2024-06-12 10:46   ` Dan Scally
@ 2024-06-12 14:51     ` Laurent Pinchart
  0 siblings, 0 replies; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 14:51 UTC (permalink / raw)
  To: Dan Scally
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 12, 2024 at 11:46:17AM +0100, Daniel Scally wrote:
> Hi Jacopo, thanks for the patch
> 
> On 05/06/2024 17:54, Jacopo Mondi wrote:
> > Add support to the rkisp1 driver for the extensible parameters format.
> >
> > Allow the driver to enumerate the existing and the new format and
> > implement support for the try_fmt and s_fmt operations.
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> 
> Isn't this too early in the set? I'd expect this to come after the
> actual support for handling the extensible buffers was done.

Agreed.

> > ---
> >   .../platform/rockchip/rkisp1/rkisp1-common.h  |  1 +
> >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 87 +++++++++++++++++--
> >   2 files changed, 79 insertions(+), 9 deletions(-)
> >
> > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> > index 2a715f964f6e..0bddae8dbdb1 100644
> > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> > @@ -383,6 +383,7 @@ struct rkisp1_params {
> >   	spinlock_t config_lock; /* locks the buffers list 'params' */
> >   	struct list_head params;
> >   
> > +	struct v4l2_meta_format metafmt;

This could be turned to a static const pointer, pointing to an entry
from rkisp1_params_formats, as none of the fields can be further
modified by userspace once a format is selected. Up to you.

> >   	enum v4l2_quantization quantization;
> >   	enum v4l2_ycbcr_encoding ycbcr_encoding;
> >   	enum rkisp1_fmt_raw_pat_type raw_type;
> > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > index 1f449f29b241..6f99c7dad758 100644
> > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > @@ -33,6 +33,34 @@
> >   #define RKISP1_ISP_CC_COEFF(n) \
> >   			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
> >   
> > +enum rkisp1_params_formats {
> > +	RKISP1_PARAMS_FIXED,
> > +	RKISP1_PARAMS_EXTENSIBLE,
> > +	RKISP1_PARAMS_NUM_FMT,
> > +};
> > +
> > +static const struct v4l2_meta_format rkisp1_params_formats[] = {
> > +	[RKISP1_PARAMS_FIXED] = {
> > +		.dataformat = V4L2_META_FMT_RK_ISP1_PARAMS,
> > +		.buffersize = sizeof(struct rkisp1_params_cfg),
> > +	},
> > +	[RKISP1_PARAMS_EXTENSIBLE] = {
> > +		.dataformat = V4L2_META_FMT_RK_ISP1_EXT_PARAMS,
> > +		.buffersize = sizeof(struct rkisp1_ext_params_cfg),
> > +	},
> > +};
> > +
> > +static const struct v4l2_meta_format *
> > +rkisp1_params_get_format_info(u32 dataformat)
> > +{
> > +	for (unsigned int i = 0; i < RKISP1_PARAMS_NUM_FMT; i++) {

Use ARRAY_SIZE(rkisp1_params_formats) instead of RKISP1_PARAMS_NUM_FMT,
that's safer. Same below, and drop RKISP1_PARAMS_NUM_FMT

> > +		if (rkisp1_params_formats[i].dataformat == dataformat)
> > +			return &rkisp1_params_formats[i];
> > +	}
> > +
> > +	return &rkisp1_params_formats[RKISP1_PARAMS_FIXED];
> > +}
> > +
> >   static inline void
> >   rkisp1_param_set_bits(struct rkisp1_params *params, u32 reg, u32 bit_mask)
> >   {
> > @@ -1742,11 +1770,13 @@ static int rkisp1_params_enum_fmt_meta_out(struct file *file, void *priv,
> >   					   struct v4l2_fmtdesc *f)
> >   {
> >   	struct video_device *video = video_devdata(file);
> > +	const struct v4l2_meta_format *metafmt;
> >   
> > -	if (f->index > 0 || f->type != video->queue->type)
> > +	if (f->index >= RKISP1_PARAMS_NUM_FMT || f->type != video->queue->type)
> >   		return -EINVAL;
> >   
> > -	f->pixelformat = V4L2_META_FMT_RK_ISP1_PARAMS;
> > +	metafmt = &rkisp1_params_formats[f->index];
> > +	f->pixelformat = metafmt->dataformat;

You could drop the metafmt variable:

	f->pixelformat = rkisp1_params_formats[f->index].dataformat;

> >   
> >   	return 0;
> >   }
> > @@ -1755,14 +1785,44 @@ static int rkisp1_params_g_fmt_meta_out(struct file *file, void *fh,
> >   					struct v4l2_format *f)
> >   {
> >   	struct video_device *video = video_devdata(file);
> > +	struct rkisp1_params *params = video_get_drvdata(video);
> >   	struct v4l2_meta_format *meta = &f->fmt.meta;
> >   
> >   	if (f->type != video->queue->type)
> >   		return -EINVAL;
> >   
> >   	memset(meta, 0, sizeof(*meta));
> > -	meta->dataformat = V4L2_META_FMT_RK_ISP1_PARAMS;
> > -	meta->buffersize = sizeof(struct rkisp1_params_cfg);
> > +	*meta = params->metafmt;

You can drop the memset now that you assign the whole structure.

> > +
> > +	return 0;
> > +}
> > +
> > +static int rkisp1_params_try_fmt_meta_out(struct file *file, void *fh,
> > +					  struct v4l2_format *f)
> > +{
> > +	struct video_device *video = video_devdata(file);
> > +	struct v4l2_meta_format *meta = &f->fmt.meta;
> > +
> > +	if (f->type != video->queue->type)
> > +		return -EINVAL;
> > +
> > +	*meta = *rkisp1_params_get_format_info(meta->dataformat);
> > +
> > +	return 0;
> > +}
> > +
> > +static int rkisp1_params_s_fmt_meta_out(struct file *file, void *fh,
> > +					struct v4l2_format *f)
> > +{
> > +	struct video_device *video = video_devdata(file);
> > +	struct rkisp1_params *params = video_get_drvdata(video);
> > +	struct v4l2_meta_format *meta = &f->fmt.meta;
> > +
> > +	if (f->type != video->queue->type)
> > +		return -EINVAL;
> > +
> > +	*meta = *rkisp1_params_get_format_info(meta->dataformat);
> > +	params->metafmt = *meta;
> >   
> >   	return 0;
> >   }
> > @@ -1792,8 +1852,8 @@ static const struct v4l2_ioctl_ops rkisp1_params_ioctl = {
> >   	.vidioc_streamoff = vb2_ioctl_streamoff,
> >   	.vidioc_enum_fmt_meta_out = rkisp1_params_enum_fmt_meta_out,
> >   	.vidioc_g_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
> > -	.vidioc_s_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
> > -	.vidioc_try_fmt_meta_out = rkisp1_params_g_fmt_meta_out,
> > +	.vidioc_s_fmt_meta_out = rkisp1_params_s_fmt_meta_out,
> > +	.vidioc_try_fmt_meta_out = rkisp1_params_try_fmt_meta_out,
> >   	.vidioc_querycap = rkisp1_params_querycap,
> >   	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
> >   	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > @@ -1805,13 +1865,16 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
> >   					 unsigned int sizes[],
> >   					 struct device *alloc_devs[])
> >   {
> > +	struct rkisp1_params *params = vq->drv_priv;
> > +	size_t buf_size = params->metafmt.buffersize;
> > +
> >   	*num_buffers = clamp_t(u32, *num_buffers,
> >   			       RKISP1_ISP_PARAMS_REQ_BUFS_MIN,
> >   			       RKISP1_ISP_PARAMS_REQ_BUFS_MAX);
> >   
> >   	*num_planes = 1;
> >   
> > -	sizes[0] = sizeof(struct rkisp1_params_cfg);
> > +	sizes[0] = buf_size;
>
> sizes[0] = params->metafmt.buffersize; would save a variable - up to you.
>
> >   
> >   	return 0;
> >   }
> > @@ -1831,10 +1894,14 @@ static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
> >   
> >   static int rkisp1_params_vb2_buf_prepare(struct vb2_buffer *vb)
> >   {
> > -	if (vb2_plane_size(vb, 0) < sizeof(struct rkisp1_params_cfg))
> > +	struct vb2_queue *vq = vb->vb2_queue;
> > +	struct rkisp1_params *params = vq->drv_priv;
> > +	size_t buf_size = params->metafmt.buffersize;
> > +
> > +	if (vb2_plane_size(vb, 0) < buf_size)
> >   		return -EINVAL;
> >   
> > -	vb2_set_plane_payload(vb, 0, sizeof(struct rkisp1_params_cfg));
> > +	vb2_set_plane_payload(vb, 0, buf_size);
> >   
> >   	return 0;
> >   }
> > @@ -1929,6 +1996,8 @@ int rkisp1_params_register(struct rkisp1_device *rkisp1)
> >   	else
> >   		params->ops = &rkisp1_v10_params_ops;
> >   
> > +	params->metafmt = rkisp1_params_formats[RKISP1_PARAMS_FIXED];
> > +
> >   	video_set_drvdata(vdev, params);
> >   
> >   	node->pad.flags = MEDIA_PAD_FL_SOURCE;

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 5/8] media: rkisp1: Implement extensible params support
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
                   ` (3 preceding siblings ...)
  2024-06-05 16:54 ` [PATCH 4/8] media: rkisp1: Add support for ext format Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 13:50   ` Dan Scally
  2024-06-05 16:54 ` [PATCH 6/8] media: rkisp1: Propagate pre/post-config errors Jacopo Mondi
                   ` (2 subsequent siblings)
  7 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

Implement support in the rkisp1-params for the extensible configuration
parameters format.

Create a list of handlers for each ISP block that wraps the existing
configuration functions and handles the ISP block enablement state.

Parse the configuration parameters buffer in __rkisp1_ext_params_config
and filter the enable blocks by group, to allow setting the 'other' and
'meas' groups separately from the 'lsc' group to support the
pre/post-configure operations.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../platform/rockchip/rkisp1/rkisp1-params.c  | 565 +++++++++++++++++-
 1 file changed, 544 insertions(+), 21 deletions(-)

diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
index 6f99c7dad758..3d78e643d0b8 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
@@ -33,6 +33,10 @@
 #define RKISP1_ISP_CC_COEFF(n) \
 			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
 
+#define RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS	BIT(0)
+#define RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS	BIT(1)
+#define RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC	BIT(2)
+
 enum rkisp1_params_formats {
 	RKISP1_PARAMS_FIXED,
 	RKISP1_PARAMS_EXTENSIBLE,
@@ -1529,9 +1533,491 @@ static void rkisp1_isp_isr_meas_config(struct rkisp1_params *params,
 	}
 }
 
+/*------------------------------------------------------------------------------
+ * Extensible parameters format handling
+ */
+
+static void rkisp1_ext_params_bls(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_bls_config *bls =
+		(struct rkisp1_ext_params_bls_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
+					RKISP1_CIF_ISP_BLS_ENA);
+		return;
+	}
+
+	rkisp1_bls_config(params, &bls->bls_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
+				      RKISP1_CIF_ISP_BLS_ENA);
+}
+
+static void rkisp1_ext_params_dpcc(struct rkisp1_params *params,
+				   struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_dpcc_config *dpcc =
+		(struct rkisp1_ext_params_dpcc_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
+					RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
+		return;
+	}
+
+	rkisp1_dpcc_config(params, &dpcc->dpcc_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
+				      RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
+}
+
+static void rkisp1_ext_params_sdg(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_sdg_config *sdg =
+		(struct rkisp1_ext_params_sdg_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
+					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
+		return;
+	}
+
+	rkisp1_sdg_config(params, &sdg->sdg_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
+				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
+}
+
+static void rkisp1_ext_params_lsc(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_lsc_config *lsc =
+		(struct rkisp1_ext_params_lsc_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
+					RKISP1_CIF_ISP_LSC_CTRL_ENA);
+		return;
+	}
+
+	rkisp1_lsc_config(params, &lsc->lsc_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
+				      RKISP1_CIF_ISP_LSC_CTRL_ENA);
+}
+
+static void rkisp1_ext_params_awbg(struct rkisp1_params *params,
+				   struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_awb_gain_config *awbg =
+		(struct rkisp1_ext_params_awb_gain_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
+					RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
+		return;
+	}
+
+	params->ops->awb_gain_config(params, &awbg->awb_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
+				      RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
+}
+
+static void rkisp1_ext_params_flt(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_flt_config *flt =
+		(struct rkisp1_ext_params_flt_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_FILT_MODE,
+					RKISP1_CIF_ISP_FLT_ENA);
+		return;
+	}
+
+	rkisp1_flt_config(params, &flt->flt_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_FILT_MODE,
+				      RKISP1_CIF_ISP_FLT_ENA);
+}
+
+static void rkisp1_ext_params_bdm(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_bdm_config *bdm =
+		(struct rkisp1_ext_params_bdm_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
+					RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
+		return;
+	}
+
+	rkisp1_bdm_config(params, &bdm->bdm_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
+				      RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
+}
+
+static void rkisp1_ext_params_ctk(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_ctk_config *ctk =
+		(struct rkisp1_ext_params_ctk_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_ctk_enable(params, false);
+		return;
+	}
+
+	rkisp1_ctk_config(params, &ctk->ctk_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_ctk_enable(params, true);
+}
+
+static void rkisp1_ext_params_goc(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_goc_config *goc =
+		(struct rkisp1_ext_params_goc_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
+					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
+		return;
+	}
+
+	params->ops->goc_config(params, &goc->goc_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
+				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
+}
+
+static void rkisp1_ext_params_dpf(struct rkisp1_params *params,
+				  struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_dpf_config *dpf =
+		(struct rkisp1_ext_params_dpf_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPF_MODE,
+					RKISP1_CIF_ISP_DPF_MODE_EN);
+		return;
+	}
+
+	rkisp1_dpf_config(params, &dpf->dpf_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPF_MODE,
+				      RKISP1_CIF_ISP_DPF_MODE_EN);
+}
+
+static void rkisp1_ext_params_dpfs(struct rkisp1_params *params,
+				   struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_dpf_strength_config *dpfs =
+		(struct rkisp1_ext_params_dpf_strength_config *)hdr;
+
+	rkisp1_dpf_strength_config(params, &dpfs->dpf_strength_config);
+}
+
+static void rkisp1_ext_params_cproc(struct rkisp1_params *params,
+				    struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_cproc_config *cproc =
+		(struct rkisp1_ext_params_cproc_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_C_PROC_CTRL,
+					RKISP1_CIF_C_PROC_CTR_ENABLE);
+		return;
+	}
+
+	rkisp1_cproc_config(params, &cproc->cproc_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_C_PROC_CTRL,
+				      RKISP1_CIF_C_PROC_CTR_ENABLE);
+}
+
+static void rkisp1_ext_params_ie(struct rkisp1_params *params,
+				 struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_ie_config *ie =
+		(struct rkisp1_ext_params_ie_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_ie_enable(params, false);
+		return;
+	}
+
+	rkisp1_ie_config(params, &ie->ie_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_ie_enable(params, true);
+}
+
+static void rkisp1_ext_params_awbm(struct rkisp1_params *params,
+				   struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_awb_meas_config *awbm =
+		(struct rkisp1_ext_params_awb_meas_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
+					     false);
+		return;
+	}
+
+	params->ops->awb_meas_config(params, &awbm->awb_meas_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
+					     true);
+}
+
+static void rkisp1_ext_params_hstm(struct rkisp1_params *params,
+				   struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_hst_config *hst =
+		(struct rkisp1_ext_params_hst_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		params->ops->hst_enable(params, &hst->hst_config, false);
+		return;
+	}
+
+	params->ops->hst_config(params, &hst->hst_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		params->ops->hst_enable(params, &hst->hst_config, true);
+}
+
+static void rkisp1_ext_params_aecm(struct rkisp1_params *params,
+				   struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_aec_config *aec =
+		(struct rkisp1_ext_params_aec_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
+					RKISP1_CIF_ISP_EXP_ENA);
+		return;
+	}
+
+	params->ops->aec_config(params, &aec->aec_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
+				      RKISP1_CIF_ISP_EXP_ENA);
+}
+
+static void rkisp1_ext_params_afcm(struct rkisp1_params *params,
+				   struct rkisp1_ext_params_block_header *hdr)
+{
+	struct rkisp1_ext_params_afc_config *afc =
+		(struct rkisp1_ext_params_afc_config *)hdr;
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
+		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
+					RKISP1_CIF_ISP_AFM_ENA);
+		return;
+	}
+
+	params->ops->afm_config(params, &afc->afc_config);
+
+	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
+		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
+				      RKISP1_CIF_ISP_AFM_ENA);
+}
+
+typedef void (*rkisp1_block_handler)(struct rkisp1_params *params,
+				     struct rkisp1_ext_params_block_header *hdr);
+
+static const struct rkisp1_ext_params_handler {
+	size_t size;
+	rkisp1_block_handler handler;
+	unsigned int group;
+} rkisp1_ext_params_handlers[] = {
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS] = {
+		.size		= sizeof(struct rkisp1_ext_params_bls_config),
+		.handler	= rkisp1_ext_params_bls,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC] = {
+		.size		= sizeof(struct rkisp1_ext_params_dpcc_config),
+		.handler	= rkisp1_ext_params_dpcc,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG] = {
+		.size		= sizeof(struct rkisp1_ext_params_sdg_config),
+		.handler	= rkisp1_ext_params_sdg,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS] = {
+		.size		=
+			sizeof(struct rkisp1_ext_params_awb_gain_config),
+		.handler	= rkisp1_ext_params_awbg,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT] = {
+		.size		= sizeof(struct rkisp1_ext_params_flt_config),
+		.handler	= rkisp1_ext_params_flt,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM] = {
+		.size		= sizeof(struct rkisp1_ext_params_bdm_config),
+		.handler	= rkisp1_ext_params_bdm,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK] = {
+		.size		= sizeof(struct rkisp1_ext_params_ctk_config),
+		.handler	= rkisp1_ext_params_ctk,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC] = {
+		.size		= sizeof(struct rkisp1_ext_params_goc_config),
+		.handler	= rkisp1_ext_params_goc,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF] = {
+		.size		= sizeof(struct rkisp1_ext_params_dpf_config),
+		.handler	= rkisp1_ext_params_dpf,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT] = {
+		.size		=
+			sizeof(struct rkisp1_ext_params_dpf_strength_config),
+		.handler	= rkisp1_ext_params_dpfs,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC] = {
+		.size		= sizeof(struct rkisp1_ext_params_cproc_config),
+		.handler	= rkisp1_ext_params_cproc,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_IE] = {
+		.size		= sizeof(struct rkisp1_ext_params_ie_config),
+		.handler	= rkisp1_ext_params_ie,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC] = {
+		.size		= sizeof(struct rkisp1_ext_params_lsc_config),
+		.handler	= rkisp1_ext_params_lsc,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS] = {
+		.size		=
+			sizeof(struct rkisp1_ext_params_awb_meas_config),
+		.handler	= rkisp1_ext_params_awbm,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS] = {
+		.size		= sizeof(struct rkisp1_ext_params_hst_config),
+		.handler	= rkisp1_ext_params_hstm,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS] = {
+		.size		= sizeof(struct rkisp1_ext_params_aec_config),
+		.handler	= rkisp1_ext_params_aecm,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
+	},
+	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS] = {
+		.size		= sizeof(struct rkisp1_ext_params_afc_config),
+		.handler	= rkisp1_ext_params_afcm,
+		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
+	},
+};
+
+static int __rkisp1_ext_params_config(struct rkisp1_params *params,
+				      struct rkisp1_ext_params_cfg *cfg,
+				      u32 block_group_mask)
+{
+	size_t block_offset = 0;
+
+	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
+		dev_dbg(params->rkisp1->dev,
+			"Invalid parameters buffer size %llu\n",
+			cfg->total_size);
+		return -EINVAL;
+	}
+
+	/* Walk the list of parameter blocks and process them. */
+	while (block_offset < cfg->total_size) {
+		const struct rkisp1_ext_params_handler *block_handler;
+		struct rkisp1_ext_params_block_header *block;
+
+		block = (struct rkisp1_ext_params_block_header *)
+			&cfg->data[block_offset];
+		block_offset += block->size;
+
+		/*
+		 * Validate the block id and make sure the block group is in
+		 * the list of groups to configure.
+		 */
+		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
+			dev_dbg(params->rkisp1->dev,
+				"Invalid parameters block type\n");
+			return -EINVAL;
+		}
+
+		block_handler = &rkisp1_ext_params_handlers[block->type];
+		if (!(block_handler->group & block_group_mask))
+			continue;
+
+		if (block->size != block_handler->size) {
+			dev_dbg(params->rkisp1->dev,
+				"Invalid parameters block size\n");
+			return -EINVAL;
+		}
+
+		block_handler->handler(params, block);
+	}
+
+	return 0;
+}
+
+static int rkisp1_ext_params_config(struct rkisp1_params *params,
+				    struct rkisp1_ext_params_cfg *cfg)
+{
+	return __rkisp1_ext_params_config(params, cfg,
+					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
+					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
+					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
+}
+
+static int
+rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
+				    struct rkisp1_ext_params_cfg *cfg)
+{
+	return __rkisp1_ext_params_config(params, cfg,
+					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
+					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
+}
+
+static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
+					struct rkisp1_ext_params_cfg *cfg)
+{
+	return __rkisp1_ext_params_config(params, cfg,
+					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
+}
+
 static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
 				     struct rkisp1_buffer **buf,
-				     struct rkisp1_params_cfg **cfg)
+				     void **cfg)
 {
 	if (list_empty(&params->params))
 		return false;
@@ -1544,28 +2030,37 @@ static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
 
 static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
 					  struct rkisp1_buffer *buf,
-					  unsigned int frame_sequence)
+					  unsigned int frame_sequence,
+					  enum vb2_buffer_state state)
 {
 	list_del(&buf->queue);
 
 	buf->vb.sequence = frame_sequence;
-	vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+	vb2_buffer_done(&buf->vb.vb2_buf, state);
 }
 
 void rkisp1_params_isr(struct rkisp1_device *rkisp1)
 {
 	struct rkisp1_params *params = &rkisp1->params;
-	struct rkisp1_params_cfg *new_params;
-	struct rkisp1_buffer *cur_buf;
+	struct rkisp1_buffer *buf;
+	int ret = 0;
+	void *cfg;
 
 	spin_lock(&params->config_lock);
 
-	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
+	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
 		goto unlock;
 
-	rkisp1_isp_isr_other_config(params, new_params);
-	rkisp1_isp_isr_lsc_config(params, new_params);
-	rkisp1_isp_isr_meas_config(params, new_params);
+	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
+		rkisp1_isp_isr_other_config(params, cfg);
+		rkisp1_isp_isr_lsc_config(params, cfg);
+		rkisp1_isp_isr_meas_config(params, cfg);
+	} else {
+		ret = rkisp1_ext_params_config(params, cfg);
+	}
+
+	if (ret)
+		goto complete_and_unlock;
 
 	/* update shadow register immediately */
 	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
@@ -1579,8 +2074,11 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
 	 * indicate to userspace on which frame these parameters are being
 	 * applied.
 	 */
-	rkisp1_params_complete_buffer(params, cur_buf,
-				      rkisp1->isp.frame_sequence + 1);
+complete_and_unlock:
+	rkisp1_params_complete_buffer(params, buf,
+				      rkisp1->isp.frame_sequence + 1,
+				      ret ? VB2_BUF_STATE_ERROR
+					  : VB2_BUF_STATE_DONE);
 
 unlock:
 	spin_unlock(&params->config_lock);
@@ -1631,8 +2129,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
 				 enum v4l2_ycbcr_encoding ycbcr_encoding)
 {
 	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
-	struct rkisp1_params_cfg *new_params;
-	struct rkisp1_buffer *cur_buf;
+	struct rkisp1_buffer *buf;
+	int ret = 0;
+	void *cfg;
 
 	params->quantization = quantization;
 	params->ycbcr_encoding = ycbcr_encoding;
@@ -1661,11 +2160,26 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
 
 	/* apply the first buffer if there is one already */
 
-	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
+	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
 		goto unlock;
 
-	rkisp1_isp_isr_other_config(params, new_params);
-	rkisp1_isp_isr_meas_config(params, new_params);
+	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
+		rkisp1_isp_isr_other_config(params, cfg);
+		rkisp1_isp_isr_meas_config(params, cfg);
+	} else {
+		ret = rkisp1_ext_params_other_meas_config(params, cfg);
+	}
+
+	if (ret) {
+		/*
+		 * Complete the buffer in error state immediately. In case of no
+		 * error, the buffer will be completed in
+		 * rkisp1_params_post_configure().
+		 */
+		rkisp1_params_complete_buffer(params, buf, 0,
+					      VB2_BUF_STATE_ERROR);
+		goto unlock;
+	}
 
 	/* update shadow register immediately */
 	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
@@ -1677,8 +2191,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
 
 void rkisp1_params_post_configure(struct rkisp1_params *params)
 {
-	struct rkisp1_params_cfg *new_params;
-	struct rkisp1_buffer *cur_buf;
+	struct rkisp1_buffer *buf;
+	int ret = 0;
+	void *cfg;
 
 	spin_lock_irq(&params->config_lock);
 
@@ -1691,16 +2206,24 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
 	 * unconditionally.
 	 */
 
-	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
+	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
 		goto unlock;
 
-	rkisp1_isp_isr_lsc_config(params, new_params);
+	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
+		rkisp1_isp_isr_lsc_config(params, cfg);
+	else
+		ret = rkisp1_ext_params_lsc_config(params, cfg);
+
+	if (ret)
+		goto complete_and_unlock;
 
 	/* update shadow register immediately */
 	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
 			      RKISP1_CIF_ISP_CTRL_ISP_CFG_UPD);
 
-	rkisp1_params_complete_buffer(params, cur_buf, 0);
+complete_and_unlock:
+	rkisp1_params_complete_buffer(params, buf, 0, ret ? VB2_BUF_STATE_ERROR
+							  : VB2_BUF_STATE_DONE);
 
 unlock:
 	spin_unlock_irq(&params->config_lock);
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 5/8] media: rkisp1: Implement extensible params support
  2024-06-05 16:54 ` [PATCH 5/8] media: rkisp1: Implement extensible params support Jacopo Mondi
@ 2024-06-12 13:50   ` Dan Scally
  2024-06-12 15:42     ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Dan Scally @ 2024-06-12 13:50 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo

On 05/06/2024 17:54, Jacopo Mondi wrote:
> Implement support in the rkisp1-params for the extensible configuration
> parameters format.
>
> Create a list of handlers for each ISP block that wraps the existing
> configuration functions and handles the ISP block enablement state.
>
> Parse the configuration parameters buffer in __rkisp1_ext_params_config
> and filter the enable blocks by group, to allow setting the 'other' and
> 'meas' groups separately from the 'lsc' group to support the
> pre/post-configure operations.
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>   .../platform/rockchip/rkisp1/rkisp1-params.c  | 565 +++++++++++++++++-
>   1 file changed, 544 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index 6f99c7dad758..3d78e643d0b8 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -33,6 +33,10 @@
>   #define RKISP1_ISP_CC_COEFF(n) \
>   			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
>   
> +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS	BIT(0)
> +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS	BIT(1)
> +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC	BIT(2)
> +
>   enum rkisp1_params_formats {
>   	RKISP1_PARAMS_FIXED,
>   	RKISP1_PARAMS_EXTENSIBLE,
> @@ -1529,9 +1533,491 @@ static void rkisp1_isp_isr_meas_config(struct rkisp1_params *params,
>   	}
>   }
>   
> +/*------------------------------------------------------------------------------
> + * Extensible parameters format handling
> + */
> +
> +static void rkisp1_ext_params_bls(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_bls_config *bls =
> +		(struct rkisp1_ext_params_bls_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> +					RKISP1_CIF_ISP_BLS_ENA);
> +		return;
> +	}
> +
> +	rkisp1_bls_config(params, &bls->bls_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> +				      RKISP1_CIF_ISP_BLS_ENA);
> +}


Most of the handlers have all but identical handling for the enable/disable parts; is it worth 
factoring that out perhaps? The register and bits could be added to struct rkisp1_ext_params_handler 
and then a common pre-handler function could be called from __rkisp1_ext_params_config() to 
set/clear the bits.

> +
> +static void rkisp1_ext_params_dpcc(struct rkisp1_params *params,
> +				   struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_dpcc_config *dpcc =
> +		(struct rkisp1_ext_params_dpcc_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> +					RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> +		return;
> +	}
> +
> +	rkisp1_dpcc_config(params, &dpcc->dpcc_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> +				      RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> +}
> +
> +static void rkisp1_ext_params_sdg(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_sdg_config *sdg =
> +		(struct rkisp1_ext_params_sdg_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> +		return;
> +	}
> +
> +	rkisp1_sdg_config(params, &sdg->sdg_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> +}
> +
> +static void rkisp1_ext_params_lsc(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_lsc_config *lsc =
> +		(struct rkisp1_ext_params_lsc_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> +					RKISP1_CIF_ISP_LSC_CTRL_ENA);
> +		return;
> +	}
> +
> +	rkisp1_lsc_config(params, &lsc->lsc_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> +				      RKISP1_CIF_ISP_LSC_CTRL_ENA);
> +}
> +
> +static void rkisp1_ext_params_awbg(struct rkisp1_params *params,
> +				   struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_awb_gain_config *awbg =
> +		(struct rkisp1_ext_params_awb_gain_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> +					RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> +		return;
> +	}
> +
> +	params->ops->awb_gain_config(params, &awbg->awb_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> +				      RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> +}
> +
> +static void rkisp1_ext_params_flt(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_flt_config *flt =
> +		(struct rkisp1_ext_params_flt_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> +					RKISP1_CIF_ISP_FLT_ENA);
> +		return;
> +	}
> +
> +	rkisp1_flt_config(params, &flt->flt_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> +				      RKISP1_CIF_ISP_FLT_ENA);
> +}
> +
> +static void rkisp1_ext_params_bdm(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_bdm_config *bdm =
> +		(struct rkisp1_ext_params_bdm_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> +					RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> +		return;
> +	}
> +
> +	rkisp1_bdm_config(params, &bdm->bdm_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> +				      RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> +}
> +
> +static void rkisp1_ext_params_ctk(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_ctk_config *ctk =
> +		(struct rkisp1_ext_params_ctk_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_ctk_enable(params, false);
> +		return;
> +	}
> +
> +	rkisp1_ctk_config(params, &ctk->ctk_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_ctk_enable(params, true);
> +}
> +
> +static void rkisp1_ext_params_goc(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_goc_config *goc =
> +		(struct rkisp1_ext_params_goc_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> +		return;
> +	}
> +
> +	params->ops->goc_config(params, &goc->goc_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> +}
> +
> +static void rkisp1_ext_params_dpf(struct rkisp1_params *params,
> +				  struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_dpf_config *dpf =
> +		(struct rkisp1_ext_params_dpf_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> +					RKISP1_CIF_ISP_DPF_MODE_EN);
> +		return;
> +	}
> +
> +	rkisp1_dpf_config(params, &dpf->dpf_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> +				      RKISP1_CIF_ISP_DPF_MODE_EN);
> +}
> +
> +static void rkisp1_ext_params_dpfs(struct rkisp1_params *params,
> +				   struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_dpf_strength_config *dpfs =
> +		(struct rkisp1_ext_params_dpf_strength_config *)hdr;
> +
> +	rkisp1_dpf_strength_config(params, &dpfs->dpf_strength_config);
> +}
> +
> +static void rkisp1_ext_params_cproc(struct rkisp1_params *params,
> +				    struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_cproc_config *cproc =
> +		(struct rkisp1_ext_params_cproc_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_C_PROC_CTRL,
> +					RKISP1_CIF_C_PROC_CTR_ENABLE);
> +		return;
> +	}
> +
> +	rkisp1_cproc_config(params, &cproc->cproc_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_C_PROC_CTRL,
> +				      RKISP1_CIF_C_PROC_CTR_ENABLE);
> +}
> +
> +static void rkisp1_ext_params_ie(struct rkisp1_params *params,
> +				 struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_ie_config *ie =
> +		(struct rkisp1_ext_params_ie_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_ie_enable(params, false);
> +		return;
> +	}
> +
> +	rkisp1_ie_config(params, &ie->ie_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_ie_enable(params, true);
> +}
> +
> +static void rkisp1_ext_params_awbm(struct rkisp1_params *params,
> +				   struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_awb_meas_config *awbm =
> +		(struct rkisp1_ext_params_awb_meas_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> +					     false);
> +		return;
> +	}
> +
> +	params->ops->awb_meas_config(params, &awbm->awb_meas_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> +					     true);
> +}
> +
> +static void rkisp1_ext_params_hstm(struct rkisp1_params *params,
> +				   struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_hst_config *hst =
> +		(struct rkisp1_ext_params_hst_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		params->ops->hst_enable(params, &hst->hst_config, false);
> +		return;
> +	}
> +
> +	params->ops->hst_config(params, &hst->hst_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		params->ops->hst_enable(params, &hst->hst_config, true);
> +}
> +
> +static void rkisp1_ext_params_aecm(struct rkisp1_params *params,
> +				   struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_aec_config *aec =
> +		(struct rkisp1_ext_params_aec_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> +					RKISP1_CIF_ISP_EXP_ENA);
> +		return;
> +	}
> +
> +	params->ops->aec_config(params, &aec->aec_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> +				      RKISP1_CIF_ISP_EXP_ENA);
> +}
> +
> +static void rkisp1_ext_params_afcm(struct rkisp1_params *params,
> +				   struct rkisp1_ext_params_block_header *hdr)
> +{
> +	struct rkisp1_ext_params_afc_config *afc =
> +		(struct rkisp1_ext_params_afc_config *)hdr;
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> +					RKISP1_CIF_ISP_AFM_ENA);
> +		return;
> +	}
> +
> +	params->ops->afm_config(params, &afc->afc_config);
> +
> +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> +				      RKISP1_CIF_ISP_AFM_ENA);
> +}
> +
> +typedef void (*rkisp1_block_handler)(struct rkisp1_params *params,
> +				     struct rkisp1_ext_params_block_header *hdr);
> +
> +static const struct rkisp1_ext_params_handler {
> +	size_t size;
> +	rkisp1_block_handler handler;
> +	unsigned int group;
> +} rkisp1_ext_params_handlers[] = {
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS] = {
> +		.size		= sizeof(struct rkisp1_ext_params_bls_config),
> +		.handler	= rkisp1_ext_params_bls,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC] = {
> +		.size		= sizeof(struct rkisp1_ext_params_dpcc_config),
> +		.handler	= rkisp1_ext_params_dpcc,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG] = {
> +		.size		= sizeof(struct rkisp1_ext_params_sdg_config),
> +		.handler	= rkisp1_ext_params_sdg,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS] = {
> +		.size		=
> +			sizeof(struct rkisp1_ext_params_awb_gain_config),
> +		.handler	= rkisp1_ext_params_awbg,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT] = {
> +		.size		= sizeof(struct rkisp1_ext_params_flt_config),
> +		.handler	= rkisp1_ext_params_flt,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM] = {
> +		.size		= sizeof(struct rkisp1_ext_params_bdm_config),
> +		.handler	= rkisp1_ext_params_bdm,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK] = {
> +		.size		= sizeof(struct rkisp1_ext_params_ctk_config),
> +		.handler	= rkisp1_ext_params_ctk,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC] = {
> +		.size		= sizeof(struct rkisp1_ext_params_goc_config),
> +		.handler	= rkisp1_ext_params_goc,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF] = {
> +		.size		= sizeof(struct rkisp1_ext_params_dpf_config),
> +		.handler	= rkisp1_ext_params_dpf,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT] = {
> +		.size		=
> +			sizeof(struct rkisp1_ext_params_dpf_strength_config),
> +		.handler	= rkisp1_ext_params_dpfs,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC] = {
> +		.size		= sizeof(struct rkisp1_ext_params_cproc_config),
> +		.handler	= rkisp1_ext_params_cproc,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_IE] = {
> +		.size		= sizeof(struct rkisp1_ext_params_ie_config),
> +		.handler	= rkisp1_ext_params_ie,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC] = {
> +		.size		= sizeof(struct rkisp1_ext_params_lsc_config),
> +		.handler	= rkisp1_ext_params_lsc,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS] = {
> +		.size		=
> +			sizeof(struct rkisp1_ext_params_awb_meas_config),
> +		.handler	= rkisp1_ext_params_awbm,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS] = {
> +		.size		= sizeof(struct rkisp1_ext_params_hst_config),
> +		.handler	= rkisp1_ext_params_hstm,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS] = {
> +		.size		= sizeof(struct rkisp1_ext_params_aec_config),
> +		.handler	= rkisp1_ext_params_aecm,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> +	},
> +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS] = {
> +		.size		= sizeof(struct rkisp1_ext_params_afc_config),
> +		.handler	= rkisp1_ext_params_afcm,
> +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> +	},
> +};
> +
> +static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> +				      struct rkisp1_ext_params_cfg *cfg,
> +				      u32 block_group_mask)
> +{
> +	size_t block_offset = 0;
> +
> +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> +		dev_dbg(params->rkisp1->dev,
> +			"Invalid parameters buffer size %llu\n",
> +			cfg->total_size);
> +		return -EINVAL;
> +	}
> +
> +	/* Walk the list of parameter blocks and process them. */
> +	while (block_offset < cfg->total_size) {
> +		const struct rkisp1_ext_params_handler *block_handler;
> +		struct rkisp1_ext_params_block_header *block;
> +
> +		block = (struct rkisp1_ext_params_block_header *)
> +			&cfg->data[block_offset];
> +		block_offset += block->size;
> +
> +		/*
> +		 * Validate the block id and make sure the block group is in
> +		 * the list of groups to configure.
> +		 */
> +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> +			dev_dbg(params->rkisp1->dev,
> +				"Invalid parameters block type\n");
> +			return -EINVAL;
> +		}
> +
> +		block_handler = &rkisp1_ext_params_handlers[block->type];
> +		if (!(block_handler->group & block_group_mask))
> +			continue;


So maybe something like


if (block_handler->enable_reg)

         __rkisp1_block_handle_enable_disable(block->state, block_handler->enable_reg, 
block_handler->enable_val);


here to move it out of the handlers.

> +
> +		if (block->size != block_handler->size) {
> +			dev_dbg(params->rkisp1->dev,
> +				"Invalid parameters block size\n");
> +			return -EINVAL;
> +		}
> +
> +		block_handler->handler(params, block);
> +	}
> +
> +	return 0;
> +}
> +
> +static int rkisp1_ext_params_config(struct rkisp1_params *params,
> +				    struct rkisp1_ext_params_cfg *cfg)
> +{
> +	return __rkisp1_ext_params_config(params, cfg,
> +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> +}
> +
> +static int
> +rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> +				    struct rkisp1_ext_params_cfg *cfg)
> +{
> +	return __rkisp1_ext_params_config(params, cfg,
> +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> +}
> +
> +static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> +					struct rkisp1_ext_params_cfg *cfg)
> +{
> +	return __rkisp1_ext_params_config(params, cfg,
> +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> +}
> +
>   static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
>   				     struct rkisp1_buffer **buf,
> -				     struct rkisp1_params_cfg **cfg)
> +				     void **cfg)
>   {
>   	if (list_empty(&params->params))
>   		return false;
> @@ -1544,28 +2030,37 @@ static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
>   
>   static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
>   					  struct rkisp1_buffer *buf,
> -					  unsigned int frame_sequence)
> +					  unsigned int frame_sequence,
> +					  enum vb2_buffer_state state)
>   {
>   	list_del(&buf->queue);
>   
>   	buf->vb.sequence = frame_sequence;
> -	vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +	vb2_buffer_done(&buf->vb.vb2_buf, state);
>   }
>   
>   void rkisp1_params_isr(struct rkisp1_device *rkisp1)
>   {
>   	struct rkisp1_params *params = &rkisp1->params;
> -	struct rkisp1_params_cfg *new_params;
> -	struct rkisp1_buffer *cur_buf;
> +	struct rkisp1_buffer *buf;
Why the rename?
> +	int ret = 0;
> +	void *cfg;
>   
>   	spin_lock(&params->config_lock);
>   
> -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
>   		goto unlock;
>   
> -	rkisp1_isp_isr_other_config(params, new_params);
> -	rkisp1_isp_isr_lsc_config(params, new_params);
> -	rkisp1_isp_isr_meas_config(params, new_params);
> +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> +		rkisp1_isp_isr_other_config(params, cfg);
> +		rkisp1_isp_isr_lsc_config(params, cfg);
> +		rkisp1_isp_isr_meas_config(params, cfg);
> +	} else {
> +		ret = rkisp1_ext_params_config(params, cfg);
> +	}
> +
> +	if (ret)
> +		goto complete_and_unlock;
>   
>   	/* update shadow register immediately */
>   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> @@ -1579,8 +2074,11 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
>   	 * indicate to userspace on which frame these parameters are being
>   	 * applied.
>   	 */
> -	rkisp1_params_complete_buffer(params, cur_buf,
> -				      rkisp1->isp.frame_sequence + 1);
> +complete_and_unlock:
> +	rkisp1_params_complete_buffer(params, buf,
> +				      rkisp1->isp.frame_sequence + 1,
> +				      ret ? VB2_BUF_STATE_ERROR
> +					  : VB2_BUF_STATE_DONE);
>   
>   unlock:
>   	spin_unlock(&params->config_lock);
> @@ -1631,8 +2129,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
>   				 enum v4l2_ycbcr_encoding ycbcr_encoding)
>   {
>   	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
> -	struct rkisp1_params_cfg *new_params;
> -	struct rkisp1_buffer *cur_buf;
> +	struct rkisp1_buffer *buf;
> +	int ret = 0;
> +	void *cfg;
>   
>   	params->quantization = quantization;
>   	params->ycbcr_encoding = ycbcr_encoding;
> @@ -1661,11 +2160,26 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
>   
>   	/* apply the first buffer if there is one already */
>   
> -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
>   		goto unlock;
>   
> -	rkisp1_isp_isr_other_config(params, new_params);
> -	rkisp1_isp_isr_meas_config(params, new_params);
> +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> +		rkisp1_isp_isr_other_config(params, cfg);
> +		rkisp1_isp_isr_meas_config(params, cfg);
> +	} else {
> +		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> +	}
> +
> +	if (ret) {
> +		/*
> +		 * Complete the buffer in error state immediately. In case of no
> +		 * error, the buffer will be completed in
> +		 * rkisp1_params_post_configure().
> +		 */
> +		rkisp1_params_complete_buffer(params, buf, 0,
> +					      VB2_BUF_STATE_ERROR);
> +		goto unlock;
> +	}
>   
>   	/* update shadow register immediately */
>   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> @@ -1677,8 +2191,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
>   
>   void rkisp1_params_post_configure(struct rkisp1_params *params)
>   {
> -	struct rkisp1_params_cfg *new_params;
> -	struct rkisp1_buffer *cur_buf;
> +	struct rkisp1_buffer *buf;
And likewise here?
> +	int ret = 0;
> +	void *cfg;
>   
>   	spin_lock_irq(&params->config_lock);
>   
> @@ -1691,16 +2206,24 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
>   	 * unconditionally.
>   	 */
>   
> -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
>   		goto unlock;
>   
> -	rkisp1_isp_isr_lsc_config(params, new_params);
> +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> +		rkisp1_isp_isr_lsc_config(params, cfg);
> +	else
> +		ret = rkisp1_ext_params_lsc_config(params, cfg);
> +
> +	if (ret)
> +		goto complete_and_unlock;
>   
>   	/* update shadow register immediately */
>   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
>   			      RKISP1_CIF_ISP_CTRL_ISP_CFG_UPD);
>   
> -	rkisp1_params_complete_buffer(params, cur_buf, 0);
> +complete_and_unlock:
> +	rkisp1_params_complete_buffer(params, buf, 0, ret ? VB2_BUF_STATE_ERROR
> +							  : VB2_BUF_STATE_DONE);
>   
>   unlock:
>   	spin_unlock_irq(&params->config_lock);

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 5/8] media: rkisp1: Implement extensible params support
  2024-06-12 13:50   ` Dan Scally
@ 2024-06-12 15:42     ` Laurent Pinchart
  2024-06-19 15:46       ` Jacopo Mondi
  0 siblings, 1 reply; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 15:42 UTC (permalink / raw)
  To: Dan Scally
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 12, 2024 at 02:50:34PM +0100, Daniel Scally wrote:
> On 05/06/2024 17:54, Jacopo Mondi wrote:
> > Implement support in the rkisp1-params for the extensible configuration
> > parameters format.
> >
> > Create a list of handlers for each ISP block that wraps the existing
> > configuration functions and handles the ISP block enablement state.
> >
> > Parse the configuration parameters buffer in __rkisp1_ext_params_config
> > and filter the enable blocks by group, to allow setting the 'other' and
> > 'meas' groups separately from the 'lsc' group to support the
> > pre/post-configure operations.
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 565 +++++++++++++++++-
> >   1 file changed, 544 insertions(+), 21 deletions(-)
> >
> > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > index 6f99c7dad758..3d78e643d0b8 100644
> > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > @@ -33,6 +33,10 @@
> >   #define RKISP1_ISP_CC_COEFF(n) \
> >   			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
> >   
> > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS	BIT(0)
> > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS	BIT(1)
> > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC	BIT(2)

Would "LSC" and "OTHERS" be enough ? The OTHERS and MEAS groups are
always handled together.

> > +
> >   enum rkisp1_params_formats {
> >   	RKISP1_PARAMS_FIXED,
> >   	RKISP1_PARAMS_EXTENSIBLE,
> > @@ -1529,9 +1533,491 @@ static void rkisp1_isp_isr_meas_config(struct rkisp1_params *params,
> >   	}
> >   }
> >   
> > +/*------------------------------------------------------------------------------
> > + * Extensible parameters format handling
> > + */
> > +
> > +static void rkisp1_ext_params_bls(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_bls_config *bls =
> > +		(struct rkisp1_ext_params_bls_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> > +					RKISP1_CIF_ISP_BLS_ENA);
> > +		return;
> > +	}
> > +
> > +	rkisp1_bls_config(params, &bls->bls_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> > +				      RKISP1_CIF_ISP_BLS_ENA);
> > +}
>
> Most of the handlers have all but identical handling for the
> enable/disable parts; is it worth factoring that out perhaps? The
> register and bits could be added to struct rkisp1_ext_params_handler
> and then a common pre-handler function could be called from
> __rkisp1_ext_params_config() to set/clear the bits.

I like the idea of generalizing things :-) The devil is in the details
though, so if it causes annoying issues, we can skip it.

If we drop RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE, we should also cache the
enable state. That should be easy to do in a bitmask indexed by block
type, and will fit nicely in generalized handling of the enable bits.

> > +
> > +static void rkisp1_ext_params_dpcc(struct rkisp1_params *params,
> > +				   struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_dpcc_config *dpcc =
> > +		(struct rkisp1_ext_params_dpcc_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> > +					RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> > +		return;
> > +	}
> > +
> > +	rkisp1_dpcc_config(params, &dpcc->dpcc_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> > +				      RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> > +}
> > +
> > +static void rkisp1_ext_params_sdg(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_sdg_config *sdg =
> > +		(struct rkisp1_ext_params_sdg_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> > +		return;
> > +	}
> > +
> > +	rkisp1_sdg_config(params, &sdg->sdg_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> > +}
> > +
> > +static void rkisp1_ext_params_lsc(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_lsc_config *lsc =
> > +		(struct rkisp1_ext_params_lsc_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> > +					RKISP1_CIF_ISP_LSC_CTRL_ENA);
> > +		return;
> > +	}
> > +
> > +	rkisp1_lsc_config(params, &lsc->lsc_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> > +				      RKISP1_CIF_ISP_LSC_CTRL_ENA);
> > +}
> > +
> > +static void rkisp1_ext_params_awbg(struct rkisp1_params *params,
> > +				   struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_awb_gain_config *awbg =
> > +		(struct rkisp1_ext_params_awb_gain_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > +					RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> > +		return;
> > +	}
> > +
> > +	params->ops->awb_gain_config(params, &awbg->awb_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > +				      RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> > +}
> > +
> > +static void rkisp1_ext_params_flt(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_flt_config *flt =
> > +		(struct rkisp1_ext_params_flt_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> > +					RKISP1_CIF_ISP_FLT_ENA);
> > +		return;
> > +	}
> > +
> > +	rkisp1_flt_config(params, &flt->flt_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> > +				      RKISP1_CIF_ISP_FLT_ENA);
> > +}
> > +
> > +static void rkisp1_ext_params_bdm(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_bdm_config *bdm =
> > +		(struct rkisp1_ext_params_bdm_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> > +					RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> > +		return;
> > +	}
> > +
> > +	rkisp1_bdm_config(params, &bdm->bdm_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> > +				      RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> > +}
> > +
> > +static void rkisp1_ext_params_ctk(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_ctk_config *ctk =
> > +		(struct rkisp1_ext_params_ctk_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_ctk_enable(params, false);
> > +		return;
> > +	}
> > +
> > +	rkisp1_ctk_config(params, &ctk->ctk_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_ctk_enable(params, true);
> > +}
> > +
> > +static void rkisp1_ext_params_goc(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_goc_config *goc =
> > +		(struct rkisp1_ext_params_goc_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> > +		return;
> > +	}
> > +
> > +	params->ops->goc_config(params, &goc->goc_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> > +}
> > +
> > +static void rkisp1_ext_params_dpf(struct rkisp1_params *params,
> > +				  struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_dpf_config *dpf =
> > +		(struct rkisp1_ext_params_dpf_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> > +					RKISP1_CIF_ISP_DPF_MODE_EN);
> > +		return;
> > +	}
> > +
> > +	rkisp1_dpf_config(params, &dpf->dpf_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> > +				      RKISP1_CIF_ISP_DPF_MODE_EN);
> > +}
> > +
> > +static void rkisp1_ext_params_dpfs(struct rkisp1_params *params,
> > +				   struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_dpf_strength_config *dpfs =
> > +		(struct rkisp1_ext_params_dpf_strength_config *)hdr;
> > +
> > +	rkisp1_dpf_strength_config(params, &dpfs->dpf_strength_config);
> > +}
> > +
> > +static void rkisp1_ext_params_cproc(struct rkisp1_params *params,
> > +				    struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_cproc_config *cproc =
> > +		(struct rkisp1_ext_params_cproc_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_C_PROC_CTRL,
> > +					RKISP1_CIF_C_PROC_CTR_ENABLE);
> > +		return;
> > +	}
> > +
> > +	rkisp1_cproc_config(params, &cproc->cproc_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_C_PROC_CTRL,
> > +				      RKISP1_CIF_C_PROC_CTR_ENABLE);
> > +}
> > +
> > +static void rkisp1_ext_params_ie(struct rkisp1_params *params,
> > +				 struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_ie_config *ie =
> > +		(struct rkisp1_ext_params_ie_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_ie_enable(params, false);
> > +		return;
> > +	}
> > +
> > +	rkisp1_ie_config(params, &ie->ie_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_ie_enable(params, true);
> > +}
> > +
> > +static void rkisp1_ext_params_awbm(struct rkisp1_params *params,
> > +				   struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_awb_meas_config *awbm =
> > +		(struct rkisp1_ext_params_awb_meas_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> > +					     false);
> > +		return;
> > +	}
> > +
> > +	params->ops->awb_meas_config(params, &awbm->awb_meas_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> > +					     true);
> > +}
> > +
> > +static void rkisp1_ext_params_hstm(struct rkisp1_params *params,
> > +				   struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_hst_config *hst =
> > +		(struct rkisp1_ext_params_hst_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		params->ops->hst_enable(params, &hst->hst_config, false);
> > +		return;
> > +	}
> > +
> > +	params->ops->hst_config(params, &hst->hst_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		params->ops->hst_enable(params, &hst->hst_config, true);
> > +}
> > +
> > +static void rkisp1_ext_params_aecm(struct rkisp1_params *params,
> > +				   struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_aec_config *aec =
> > +		(struct rkisp1_ext_params_aec_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> > +					RKISP1_CIF_ISP_EXP_ENA);
> > +		return;
> > +	}
> > +
> > +	params->ops->aec_config(params, &aec->aec_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> > +				      RKISP1_CIF_ISP_EXP_ENA);
> > +}
> > +
> > +static void rkisp1_ext_params_afcm(struct rkisp1_params *params,
> > +				   struct rkisp1_ext_params_block_header *hdr)
> > +{
> > +	struct rkisp1_ext_params_afc_config *afc =
> > +		(struct rkisp1_ext_params_afc_config *)hdr;
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> > +					RKISP1_CIF_ISP_AFM_ENA);
> > +		return;
> > +	}
> > +
> > +	params->ops->afm_config(params, &afc->afc_config);
> > +
> > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> > +				      RKISP1_CIF_ISP_AFM_ENA);
> > +}
> > +
> > +typedef void (*rkisp1_block_handler)(struct rkisp1_params *params,
> > +				     struct rkisp1_ext_params_block_header *hdr);
> > +
> > +static const struct rkisp1_ext_params_handler {
> > +	size_t size;
> > +	rkisp1_block_handler handler;
> > +	unsigned int group;

We'll have to extend this structure with a bitmask of the ISP versions
each block supports. That can be done later.

> > +} rkisp1_ext_params_handlers[] = {
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_bls_config),
> > +		.handler	= rkisp1_ext_params_bls,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_dpcc_config),
> > +		.handler	= rkisp1_ext_params_dpcc,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_sdg_config),
> > +		.handler	= rkisp1_ext_params_sdg,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS] = {
> > +		.size		=
> > +			sizeof(struct rkisp1_ext_params_awb_gain_config),
> > +		.handler	= rkisp1_ext_params_awbg,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_flt_config),
> > +		.handler	= rkisp1_ext_params_flt,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_bdm_config),
> > +		.handler	= rkisp1_ext_params_bdm,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_ctk_config),
> > +		.handler	= rkisp1_ext_params_ctk,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_goc_config),
> > +		.handler	= rkisp1_ext_params_goc,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_dpf_config),
> > +		.handler	= rkisp1_ext_params_dpf,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT] = {
> > +		.size		=
> > +			sizeof(struct rkisp1_ext_params_dpf_strength_config),
> > +		.handler	= rkisp1_ext_params_dpfs,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_cproc_config),
> > +		.handler	= rkisp1_ext_params_cproc,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_IE] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_ie_config),
> > +		.handler	= rkisp1_ext_params_ie,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_lsc_config),
> > +		.handler	= rkisp1_ext_params_lsc,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS] = {
> > +		.size		=
> > +			sizeof(struct rkisp1_ext_params_awb_meas_config),
> > +		.handler	= rkisp1_ext_params_awbm,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_hst_config),
> > +		.handler	= rkisp1_ext_params_hstm,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_aec_config),
> > +		.handler	= rkisp1_ext_params_aecm,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > +	},
> > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS] = {
> > +		.size		= sizeof(struct rkisp1_ext_params_afc_config),
> > +		.handler	= rkisp1_ext_params_afcm,
> > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > +	},
> > +};
> > +
> > +static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > +				      struct rkisp1_ext_params_cfg *cfg,
> > +				      u32 block_group_mask)
> > +{
> > +	size_t block_offset = 0;
> > +
> > +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > +		dev_dbg(params->rkisp1->dev,
> > +			"Invalid parameters buffer size %llu\n",
> > +			cfg->total_size);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Walk the list of parameter blocks and process them. */
> > +	while (block_offset < cfg->total_size) {
> > +		const struct rkisp1_ext_params_handler *block_handler;
> > +		struct rkisp1_ext_params_block_header *block;
> > +
> > +		block = (struct rkisp1_ext_params_block_header *)
> > +			&cfg->data[block_offset];
> > +		block_offset += block->size;
> > +
> > +		/*
> > +		 * Validate the block id and make sure the block group is in
> > +		 * the list of groups to configure.
> > +		 */
> > +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {

Use ARRAY_SIZE().

> > +			dev_dbg(params->rkisp1->dev,
> > +				"Invalid parameters block type\n");
> > +			return -EINVAL;
> > +		}
> > +
> > +		block_handler = &rkisp1_ext_params_handlers[block->type];
> > +		if (!(block_handler->group & block_group_mask))
> > +			continue;
> 
> 
> So maybe something like
> 
> 
> if (block_handler->enable_reg)
> 
>          __rkisp1_block_handle_enable_disable(block->state, block_handler->enable_reg, 
> block_handler->enable_val);
> 
> 
> here to move it out of the handlers.
> 
> > +
> > +		if (block->size != block_handler->size) {
> > +			dev_dbg(params->rkisp1->dev,
> > +				"Invalid parameters block size\n");
> > +			return -EINVAL;
> > +		}
> > +
> > +		block_handler->handler(params, block);

It would be nicer to move validation to qbuf time, and applying
parameters at runtime. Maybe I'll find that later in the series :-)

> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > +				    struct rkisp1_ext_params_cfg *cfg)
> > +{
> > +	return __rkisp1_ext_params_config(params, cfg,
> > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > +}
> > +
> > +static int
> > +rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > +				    struct rkisp1_ext_params_cfg *cfg)
> > +{
> > +	return __rkisp1_ext_params_config(params, cfg,
> > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > +}
> > +
> > +static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > +					struct rkisp1_ext_params_cfg *cfg)
> > +{
> > +	return __rkisp1_ext_params_config(params, cfg,
> > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > +}

I'm tempted to inline this in the callers, I think it would be more
readable.

> > +
> >   static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
> >   				     struct rkisp1_buffer **buf,
> > -				     struct rkisp1_params_cfg **cfg)
> > +				     void **cfg)
> >   {
> >   	if (list_empty(&params->params))
> >   		return false;
> > @@ -1544,28 +2030,37 @@ static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
> >   
> >   static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
> >   					  struct rkisp1_buffer *buf,
> > -					  unsigned int frame_sequence)
> > +					  unsigned int frame_sequence,
> > +					  enum vb2_buffer_state state)
> >   {
> >   	list_del(&buf->queue);
> >   
> >   	buf->vb.sequence = frame_sequence;
> > -	vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> > +	vb2_buffer_done(&buf->vb.vb2_buf, state);
> >   }
> >   
> >   void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> >   {
> >   	struct rkisp1_params *params = &rkisp1->params;
> > -	struct rkisp1_params_cfg *new_params;
> > -	struct rkisp1_buffer *cur_buf;
> > +	struct rkisp1_buffer *buf;
>
> Why the rename?
>
> > +	int ret = 0;
> > +	void *cfg;
> >   
> >   	spin_lock(&params->config_lock);
> >   
> > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> >   		goto unlock;
> >   
> > -	rkisp1_isp_isr_other_config(params, new_params);
> > -	rkisp1_isp_isr_lsc_config(params, new_params);
> > -	rkisp1_isp_isr_meas_config(params, new_params);
> > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > +		rkisp1_isp_isr_other_config(params, cfg);
> > +		rkisp1_isp_isr_lsc_config(params, cfg);
> > +		rkisp1_isp_isr_meas_config(params, cfg);
> > +	} else {
> > +		ret = rkisp1_ext_params_config(params, cfg);
> > +	}
> > +
> > +	if (ret)
> > +		goto complete_and_unlock;

As validation should happen at qbuf time, I think it would be nicer to
reorder the patches to avoid introducing error handling here and
removing it later.

> >   
> >   	/* update shadow register immediately */
> >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > @@ -1579,8 +2074,11 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> >   	 * indicate to userspace on which frame these parameters are being
> >   	 * applied.
> >   	 */
> > -	rkisp1_params_complete_buffer(params, cur_buf,
> > -				      rkisp1->isp.frame_sequence + 1);
> > +complete_and_unlock:
> > +	rkisp1_params_complete_buffer(params, buf,
> > +				      rkisp1->isp.frame_sequence + 1,
> > +				      ret ? VB2_BUF_STATE_ERROR
> > +					  : VB2_BUF_STATE_DONE);
> >   
> >   unlock:
> >   	spin_unlock(&params->config_lock);
> > @@ -1631,8 +2129,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> >   				 enum v4l2_ycbcr_encoding ycbcr_encoding)
> >   {
> >   	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
> > -	struct rkisp1_params_cfg *new_params;
> > -	struct rkisp1_buffer *cur_buf;
> > +	struct rkisp1_buffer *buf;
> > +	int ret = 0;
> > +	void *cfg;
> >   
> >   	params->quantization = quantization;
> >   	params->ycbcr_encoding = ycbcr_encoding;
> > @@ -1661,11 +2160,26 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> >   
> >   	/* apply the first buffer if there is one already */
> >   
> > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> >   		goto unlock;
> >   
> > -	rkisp1_isp_isr_other_config(params, new_params);
> > -	rkisp1_isp_isr_meas_config(params, new_params);
> > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > +		rkisp1_isp_isr_other_config(params, cfg);
> > +		rkisp1_isp_isr_meas_config(params, cfg);
> > +	} else {
> > +		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > +	}
> > +
> > +	if (ret) {
> > +		/*
> > +		 * Complete the buffer in error state immediately. In case of no
> > +		 * error, the buffer will be completed in
> > +		 * rkisp1_params_post_configure().
> > +		 */
> > +		rkisp1_params_complete_buffer(params, buf, 0,
> > +					      VB2_BUF_STATE_ERROR);
> > +		goto unlock;
> > +	}
> >   
> >   	/* update shadow register immediately */
> >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > @@ -1677,8 +2191,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> >   
> >   void rkisp1_params_post_configure(struct rkisp1_params *params)
> >   {
> > -	struct rkisp1_params_cfg *new_params;
> > -	struct rkisp1_buffer *cur_buf;
> > +	struct rkisp1_buffer *buf;
>
> And likewise here?
>
> > +	int ret = 0;
> > +	void *cfg;
> >   
> >   	spin_lock_irq(&params->config_lock);
> >   
> > @@ -1691,16 +2206,24 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
> >   	 * unconditionally.
> >   	 */
> >   
> > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> >   		goto unlock;
> >   
> > -	rkisp1_isp_isr_lsc_config(params, new_params);
> > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > +		rkisp1_isp_isr_lsc_config(params, cfg);
> > +	else
> > +		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > +
> > +	if (ret)
> > +		goto complete_and_unlock;
> >   
> >   	/* update shadow register immediately */
> >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> >   			      RKISP1_CIF_ISP_CTRL_ISP_CFG_UPD);
> >   
> > -	rkisp1_params_complete_buffer(params, cur_buf, 0);
> > +complete_and_unlock:
> > +	rkisp1_params_complete_buffer(params, buf, 0, ret ? VB2_BUF_STATE_ERROR
> > +							  : VB2_BUF_STATE_DONE);
> >   
> >   unlock:
> >   	spin_unlock_irq(&params->config_lock);

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 5/8] media: rkisp1: Implement extensible params support
  2024-06-12 15:42     ` Laurent Pinchart
@ 2024-06-19 15:46       ` Jacopo Mondi
  2024-06-19 16:09         ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-19 15:46 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Dan Scally, Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi Laurent

On Wed, Jun 12, 2024 at 06:42:01PM GMT, Laurent Pinchart wrote:
> On Wed, Jun 12, 2024 at 02:50:34PM +0100, Daniel Scally wrote:
> > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > Implement support in the rkisp1-params for the extensible configuration
> > > parameters format.
> > >
> > > Create a list of handlers for each ISP block that wraps the existing
> > > configuration functions and handles the ISP block enablement state.
> > >
> > > Parse the configuration parameters buffer in __rkisp1_ext_params_config
> > > and filter the enable blocks by group, to allow setting the 'other' and
> > > 'meas' groups separately from the 'lsc' group to support the
> > > pre/post-configure operations.
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 565 +++++++++++++++++-
> > >   1 file changed, 544 insertions(+), 21 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > index 6f99c7dad758..3d78e643d0b8 100644
> > > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > @@ -33,6 +33,10 @@
> > >   #define RKISP1_ISP_CC_COEFF(n) \
> > >   			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
> > >
> > > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS	BIT(0)
> > > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS	BIT(1)
> > > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC	BIT(2)
>
> Would "LSC" and "OTHERS" be enough ? The OTHERS and MEAS groups are
> always handled together.
>

The existing code handles the three separately but I actually program
OTHER and MEAS together.

Then we're left with "OTHER" and "LSC". I would like a more
descriptive name for "OTHER"

> > > +
> > >   enum rkisp1_params_formats {
> > >   	RKISP1_PARAMS_FIXED,
> > >   	RKISP1_PARAMS_EXTENSIBLE,
> > > @@ -1529,9 +1533,491 @@ static void rkisp1_isp_isr_meas_config(struct rkisp1_params *params,
> > >   	}
> > >   }
> > >
> > > +/*------------------------------------------------------------------------------
> > > + * Extensible parameters format handling
> > > + */
> > > +
> > > +static void rkisp1_ext_params_bls(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_bls_config *bls =
> > > +		(struct rkisp1_ext_params_bls_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> > > +					RKISP1_CIF_ISP_BLS_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_bls_config(params, &bls->bls_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> > > +				      RKISP1_CIF_ISP_BLS_ENA);
> > > +}
> >
> > Most of the handlers have all but identical handling for the
> > enable/disable parts; is it worth factoring that out perhaps? The
> > register and bits could be added to struct rkisp1_ext_params_handler
> > and then a common pre-handler function could be called from
> > __rkisp1_ext_params_config() to set/clear the bits.
>
> I like the idea of generalizing things :-) The devil is in the details
> though, so if it causes annoying issues, we can skip it.

There are slight differences in how blocks are enabled, most blocks
are enabled setting a bit, other have a dedicated function. I would
keep them in the handlers for clarify (or I should add an .enable and
.disable function, but this seems more cumbersome than what I have
here)

>
> If we drop RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE, we should also cache the
> enable state. That should be easy to do in a bitmask indexed by block
> type, and will fit nicely in generalized handling of the enable bits.
>

Done

> > > +
> > > +static void rkisp1_ext_params_dpcc(struct rkisp1_params *params,
> > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_dpcc_config *dpcc =
> > > +		(struct rkisp1_ext_params_dpcc_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> > > +					RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_dpcc_config(params, &dpcc->dpcc_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> > > +				      RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_sdg(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_sdg_config *sdg =
> > > +		(struct rkisp1_ext_params_sdg_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > > +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_sdg_config(params, &sdg->sdg_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_lsc(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_lsc_config *lsc =
> > > +		(struct rkisp1_ext_params_lsc_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> > > +					RKISP1_CIF_ISP_LSC_CTRL_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_lsc_config(params, &lsc->lsc_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> > > +				      RKISP1_CIF_ISP_LSC_CTRL_ENA);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_awbg(struct rkisp1_params *params,
> > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_awb_gain_config *awbg =
> > > +		(struct rkisp1_ext_params_awb_gain_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > > +					RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	params->ops->awb_gain_config(params, &awbg->awb_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > +				      RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_flt(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_flt_config *flt =
> > > +		(struct rkisp1_ext_params_flt_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> > > +					RKISP1_CIF_ISP_FLT_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_flt_config(params, &flt->flt_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> > > +				      RKISP1_CIF_ISP_FLT_ENA);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_bdm(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_bdm_config *bdm =
> > > +		(struct rkisp1_ext_params_bdm_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> > > +					RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_bdm_config(params, &bdm->bdm_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> > > +				      RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_ctk(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_ctk_config *ctk =
> > > +		(struct rkisp1_ext_params_ctk_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_ctk_enable(params, false);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_ctk_config(params, &ctk->ctk_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_ctk_enable(params, true);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_goc(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_goc_config *goc =
> > > +		(struct rkisp1_ext_params_goc_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > > +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	params->ops->goc_config(params, &goc->goc_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_dpf(struct rkisp1_params *params,
> > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_dpf_config *dpf =
> > > +		(struct rkisp1_ext_params_dpf_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> > > +					RKISP1_CIF_ISP_DPF_MODE_EN);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_dpf_config(params, &dpf->dpf_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> > > +				      RKISP1_CIF_ISP_DPF_MODE_EN);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_dpfs(struct rkisp1_params *params,
> > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_dpf_strength_config *dpfs =
> > > +		(struct rkisp1_ext_params_dpf_strength_config *)hdr;
> > > +
> > > +	rkisp1_dpf_strength_config(params, &dpfs->dpf_strength_config);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_cproc(struct rkisp1_params *params,
> > > +				    struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_cproc_config *cproc =
> > > +		(struct rkisp1_ext_params_cproc_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_C_PROC_CTRL,
> > > +					RKISP1_CIF_C_PROC_CTR_ENABLE);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_cproc_config(params, &cproc->cproc_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_C_PROC_CTRL,
> > > +				      RKISP1_CIF_C_PROC_CTR_ENABLE);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_ie(struct rkisp1_params *params,
> > > +				 struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_ie_config *ie =
> > > +		(struct rkisp1_ext_params_ie_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_ie_enable(params, false);
> > > +		return;
> > > +	}
> > > +
> > > +	rkisp1_ie_config(params, &ie->ie_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_ie_enable(params, true);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_awbm(struct rkisp1_params *params,
> > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_awb_meas_config *awbm =
> > > +		(struct rkisp1_ext_params_awb_meas_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> > > +					     false);
> > > +		return;
> > > +	}
> > > +
> > > +	params->ops->awb_meas_config(params, &awbm->awb_meas_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> > > +					     true);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_hstm(struct rkisp1_params *params,
> > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_hst_config *hst =
> > > +		(struct rkisp1_ext_params_hst_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		params->ops->hst_enable(params, &hst->hst_config, false);
> > > +		return;
> > > +	}
> > > +
> > > +	params->ops->hst_config(params, &hst->hst_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		params->ops->hst_enable(params, &hst->hst_config, true);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_aecm(struct rkisp1_params *params,
> > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_aec_config *aec =
> > > +		(struct rkisp1_ext_params_aec_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> > > +					RKISP1_CIF_ISP_EXP_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	params->ops->aec_config(params, &aec->aec_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> > > +				      RKISP1_CIF_ISP_EXP_ENA);
> > > +}
> > > +
> > > +static void rkisp1_ext_params_afcm(struct rkisp1_params *params,
> > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > +{
> > > +	struct rkisp1_ext_params_afc_config *afc =
> > > +		(struct rkisp1_ext_params_afc_config *)hdr;
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> > > +					RKISP1_CIF_ISP_AFM_ENA);
> > > +		return;
> > > +	}
> > > +
> > > +	params->ops->afm_config(params, &afc->afc_config);
> > > +
> > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> > > +				      RKISP1_CIF_ISP_AFM_ENA);
> > > +}
> > > +
> > > +typedef void (*rkisp1_block_handler)(struct rkisp1_params *params,
> > > +				     struct rkisp1_ext_params_block_header *hdr);
> > > +
> > > +static const struct rkisp1_ext_params_handler {
> > > +	size_t size;
> > > +	rkisp1_block_handler handler;
> > > +	unsigned int group;
>
> We'll have to extend this structure with a bitmask of the ISP versions
> each block supports. That can be done later.
>
> > > +} rkisp1_ext_params_handlers[] = {
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_bls_config),
> > > +		.handler	= rkisp1_ext_params_bls,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_dpcc_config),
> > > +		.handler	= rkisp1_ext_params_dpcc,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_sdg_config),
> > > +		.handler	= rkisp1_ext_params_sdg,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS] = {
> > > +		.size		=
> > > +			sizeof(struct rkisp1_ext_params_awb_gain_config),
> > > +		.handler	= rkisp1_ext_params_awbg,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_flt_config),
> > > +		.handler	= rkisp1_ext_params_flt,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_bdm_config),
> > > +		.handler	= rkisp1_ext_params_bdm,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_ctk_config),
> > > +		.handler	= rkisp1_ext_params_ctk,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_goc_config),
> > > +		.handler	= rkisp1_ext_params_goc,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_dpf_config),
> > > +		.handler	= rkisp1_ext_params_dpf,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT] = {
> > > +		.size		=
> > > +			sizeof(struct rkisp1_ext_params_dpf_strength_config),
> > > +		.handler	= rkisp1_ext_params_dpfs,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_cproc_config),
> > > +		.handler	= rkisp1_ext_params_cproc,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_IE] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_ie_config),
> > > +		.handler	= rkisp1_ext_params_ie,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_lsc_config),
> > > +		.handler	= rkisp1_ext_params_lsc,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS] = {
> > > +		.size		=
> > > +			sizeof(struct rkisp1_ext_params_awb_meas_config),
> > > +		.handler	= rkisp1_ext_params_awbm,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_hst_config),
> > > +		.handler	= rkisp1_ext_params_hstm,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_aec_config),
> > > +		.handler	= rkisp1_ext_params_aecm,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > +	},
> > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS] = {
> > > +		.size		= sizeof(struct rkisp1_ext_params_afc_config),
> > > +		.handler	= rkisp1_ext_params_afcm,
> > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > +	},
> > > +};
> > > +
> > > +static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > +				      struct rkisp1_ext_params_cfg *cfg,
> > > +				      u32 block_group_mask)
> > > +{
> > > +	size_t block_offset = 0;
> > > +
> > > +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > +		dev_dbg(params->rkisp1->dev,
> > > +			"Invalid parameters buffer size %llu\n",
> > > +			cfg->total_size);
> > > +		return -EINVAL;
> > > +	}
> > > +
> > > +	/* Walk the list of parameter blocks and process them. */
> > > +	while (block_offset < cfg->total_size) {
> > > +		const struct rkisp1_ext_params_handler *block_handler;
> > > +		struct rkisp1_ext_params_block_header *block;
> > > +
> > > +		block = (struct rkisp1_ext_params_block_header *)
> > > +			&cfg->data[block_offset];
> > > +		block_offset += block->size;
> > > +
> > > +		/*
> > > +		 * Validate the block id and make sure the block group is in
> > > +		 * the list of groups to configure.
> > > +		 */
> > > +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
>
> Use ARRAY_SIZE().
>

I don't think I can use ARRAY_SIZE() on a enum type

> > > +			dev_dbg(params->rkisp1->dev,
> > > +				"Invalid parameters block type\n");
> > > +			return -EINVAL;
> > > +		}
> > > +
> > > +		block_handler = &rkisp1_ext_params_handlers[block->type];
> > > +		if (!(block_handler->group & block_group_mask))
> > > +			continue;
> >
> >
> > So maybe something like
> >
> >
> > if (block_handler->enable_reg)
> >
> >          __rkisp1_block_handle_enable_disable(block->state, block_handler->enable_reg,
> > block_handler->enable_val);
> >
> >
> > here to move it out of the handlers.
> >
> > > +
> > > +		if (block->size != block_handler->size) {
> > > +			dev_dbg(params->rkisp1->dev,
> > > +				"Invalid parameters block size\n");
> > > +			return -EINVAL;
> > > +		}
> > > +
> > > +		block_handler->handler(params, block);
>
> It would be nicer to move validation to qbuf time, and applying
> parameters at runtime. Maybe I'll find that later in the series :-)
>
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > +				    struct rkisp1_ext_params_cfg *cfg)
> > > +{
> > > +	return __rkisp1_ext_params_config(params, cfg,
> > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > +}
> > > +
> > > +static int
> > > +rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > > +				    struct rkisp1_ext_params_cfg *cfg)
> > > +{
> > > +	return __rkisp1_ext_params_config(params, cfg,
> > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > +}
> > > +
> > > +static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > > +					struct rkisp1_ext_params_cfg *cfg)
> > > +{
> > > +	return __rkisp1_ext_params_config(params, cfg,
> > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > > +}
>
> I'm tempted to inline this in the callers, I think it would be more
> readable.
>
> > > +
> > >   static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
> > >   				     struct rkisp1_buffer **buf,
> > > -				     struct rkisp1_params_cfg **cfg)
> > > +				     void **cfg)
> > >   {
> > >   	if (list_empty(&params->params))
> > >   		return false;
> > > @@ -1544,28 +2030,37 @@ static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
> > >
> > >   static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
> > >   					  struct rkisp1_buffer *buf,
> > > -					  unsigned int frame_sequence)
> > > +					  unsigned int frame_sequence,
> > > +					  enum vb2_buffer_state state)
> > >   {
> > >   	list_del(&buf->queue);
> > >
> > >   	buf->vb.sequence = frame_sequence;
> > > -	vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> > > +	vb2_buffer_done(&buf->vb.vb2_buf, state);
> > >   }
> > >
> > >   void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > >   {
> > >   	struct rkisp1_params *params = &rkisp1->params;
> > > -	struct rkisp1_params_cfg *new_params;
> > > -	struct rkisp1_buffer *cur_buf;
> > > +	struct rkisp1_buffer *buf;
> >
> > Why the rename?
> >

Because "cur" implies there's a "next" or a "prev".
I can drop it though.


> > > +	int ret = 0;
> > > +	void *cfg;
> > >
> > >   	spin_lock(&params->config_lock);
> > >
> > > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> > >   		goto unlock;
> > >
> > > -	rkisp1_isp_isr_other_config(params, new_params);
> > > -	rkisp1_isp_isr_lsc_config(params, new_params);
> > > -	rkisp1_isp_isr_meas_config(params, new_params);
> > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > +		rkisp1_isp_isr_other_config(params, cfg);
> > > +		rkisp1_isp_isr_lsc_config(params, cfg);
> > > +		rkisp1_isp_isr_meas_config(params, cfg);
> > > +	} else {
> > > +		ret = rkisp1_ext_params_config(params, cfg);
> > > +	}
> > > +
> > > +	if (ret)
> > > +		goto complete_and_unlock;
>
> As validation should happen at qbuf time, I think it would be nicer to
> reorder the patches to avoid introducing error handling here and
> removing it later.
>
> > >
> > >   	/* update shadow register immediately */
> > >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > @@ -1579,8 +2074,11 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > >   	 * indicate to userspace on which frame these parameters are being
> > >   	 * applied.
> > >   	 */
> > > -	rkisp1_params_complete_buffer(params, cur_buf,
> > > -				      rkisp1->isp.frame_sequence + 1);
> > > +complete_and_unlock:
> > > +	rkisp1_params_complete_buffer(params, buf,
> > > +				      rkisp1->isp.frame_sequence + 1,
> > > +				      ret ? VB2_BUF_STATE_ERROR
> > > +					  : VB2_BUF_STATE_DONE);
> > >
> > >   unlock:
> > >   	spin_unlock(&params->config_lock);
> > > @@ -1631,8 +2129,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > >   				 enum v4l2_ycbcr_encoding ycbcr_encoding)
> > >   {
> > >   	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
> > > -	struct rkisp1_params_cfg *new_params;
> > > -	struct rkisp1_buffer *cur_buf;
> > > +	struct rkisp1_buffer *buf;
> > > +	int ret = 0;
> > > +	void *cfg;
> > >
> > >   	params->quantization = quantization;
> > >   	params->ycbcr_encoding = ycbcr_encoding;
> > > @@ -1661,11 +2160,26 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > >
> > >   	/* apply the first buffer if there is one already */
> > >
> > > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> > >   		goto unlock;
> > >
> > > -	rkisp1_isp_isr_other_config(params, new_params);
> > > -	rkisp1_isp_isr_meas_config(params, new_params);
> > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > +		rkisp1_isp_isr_other_config(params, cfg);
> > > +		rkisp1_isp_isr_meas_config(params, cfg);
> > > +	} else {
> > > +		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > > +	}
> > > +
> > > +	if (ret) {
> > > +		/*
> > > +		 * Complete the buffer in error state immediately. In case of no
> > > +		 * error, the buffer will be completed in
> > > +		 * rkisp1_params_post_configure().
> > > +		 */
> > > +		rkisp1_params_complete_buffer(params, buf, 0,
> > > +					      VB2_BUF_STATE_ERROR);
> > > +		goto unlock;
> > > +	}
> > >
> > >   	/* update shadow register immediately */
> > >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > @@ -1677,8 +2191,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > >
> > >   void rkisp1_params_post_configure(struct rkisp1_params *params)
> > >   {
> > > -	struct rkisp1_params_cfg *new_params;
> > > -	struct rkisp1_buffer *cur_buf;
> > > +	struct rkisp1_buffer *buf;
> >
> > And likewise here?
> >
> > > +	int ret = 0;
> > > +	void *cfg;
> > >
> > >   	spin_lock_irq(&params->config_lock);
> > >
> > > @@ -1691,16 +2206,24 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
> > >   	 * unconditionally.
> > >   	 */
> > >
> > > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> > >   		goto unlock;
> > >
> > > -	rkisp1_isp_isr_lsc_config(params, new_params);
> > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > +		rkisp1_isp_isr_lsc_config(params, cfg);
> > > +	else
> > > +		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > > +
> > > +	if (ret)
> > > +		goto complete_and_unlock;
> > >
> > >   	/* update shadow register immediately */
> > >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > >   			      RKISP1_CIF_ISP_CTRL_ISP_CFG_UPD);
> > >
> > > -	rkisp1_params_complete_buffer(params, cur_buf, 0);
> > > +complete_and_unlock:
> > > +	rkisp1_params_complete_buffer(params, buf, 0, ret ? VB2_BUF_STATE_ERROR
> > > +							  : VB2_BUF_STATE_DONE);
> > >
> > >   unlock:
> > >   	spin_unlock_irq(&params->config_lock);
>
> --
> Regards,
>
> Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 5/8] media: rkisp1: Implement extensible params support
  2024-06-19 15:46       ` Jacopo Mondi
@ 2024-06-19 16:09         ` Laurent Pinchart
  0 siblings, 0 replies; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-19 16:09 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Dan Scally, Linux Media Mailing List, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 19, 2024 at 05:46:59PM +0200, Jacopo Mondi wrote:
> On Wed, Jun 12, 2024 at 06:42:01PM GMT, Laurent Pinchart wrote:
> > On Wed, Jun 12, 2024 at 02:50:34PM +0100, Daniel Scally wrote:
> > > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > > Implement support in the rkisp1-params for the extensible configuration
> > > > parameters format.
> > > >
> > > > Create a list of handlers for each ISP block that wraps the existing
> > > > configuration functions and handles the ISP block enablement state.
> > > >
> > > > Parse the configuration parameters buffer in __rkisp1_ext_params_config
> > > > and filter the enable blocks by group, to allow setting the 'other' and
> > > > 'meas' groups separately from the 'lsc' group to support the
> > > > pre/post-configure operations.
> > > >
> > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > ---
> > > >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 565 +++++++++++++++++-
> > > >   1 file changed, 544 insertions(+), 21 deletions(-)
> > > >
> > > > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > index 6f99c7dad758..3d78e643d0b8 100644
> > > > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > @@ -33,6 +33,10 @@
> > > >   #define RKISP1_ISP_CC_COEFF(n) \
> > > >   			(RKISP1_CIF_ISP_CC_COEFF_0 + (n) * 4)
> > > >
> > > > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS	BIT(0)
> > > > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS	BIT(1)
> > > > +#define RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC	BIT(2)
> >
> > Would "LSC" and "OTHERS" be enough ? The OTHERS and MEAS groups are
> > always handled together.
> 
> The existing code handles the three separately but I actually program
> OTHER and MEAS together.
> 
> Then we're left with "OTHER" and "LSC". I would like a more
> descriptive name for "OTHER"

Yes "other" doesn't sound great. At the same time, it's really
everythink but LSC :-)

> > > > +
> > > >   enum rkisp1_params_formats {
> > > >   	RKISP1_PARAMS_FIXED,
> > > >   	RKISP1_PARAMS_EXTENSIBLE,
> > > > @@ -1529,9 +1533,491 @@ static void rkisp1_isp_isr_meas_config(struct rkisp1_params *params,
> > > >   	}
> > > >   }
> > > >
> > > > +/*------------------------------------------------------------------------------
> > > > + * Extensible parameters format handling
> > > > + */
> > > > +
> > > > +static void rkisp1_ext_params_bls(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_bls_config *bls =
> > > > +		(struct rkisp1_ext_params_bls_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> > > > +					RKISP1_CIF_ISP_BLS_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_bls_config(params, &bls->bls_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_BLS_CTRL,
> > > > +				      RKISP1_CIF_ISP_BLS_ENA);
> > > > +}
> > >
> > > Most of the handlers have all but identical handling for the
> > > enable/disable parts; is it worth factoring that out perhaps? The
> > > register and bits could be added to struct rkisp1_ext_params_handler
> > > and then a common pre-handler function could be called from
> > > __rkisp1_ext_params_config() to set/clear the bits.
> >
> > I like the idea of generalizing things :-) The devil is in the details
> > though, so if it causes annoying issues, we can skip it.
> 
> There are slight differences in how blocks are enabled, most blocks
> are enabled setting a bit, other have a dedicated function. I would

True, I missed that.

> keep them in the handlers for clarify (or I should add an .enable and
> .disable function, but this seems more cumbersome than what I have
> here)

Hmmmm... That's tempting actually, you wouldn't have to add all the
functions below if you did that. You could use the existing
enable/disable handlers when they exist, and add a single implementation
for the blocks that just set or clear a bit, with the register address
and bit being fields in the rkisp1_ext_params_handler structure.

> > If we drop RKISP1_EXT_PARAMS_BLOCK_NO_CHANGE, we should also cache the
> > enable state. That should be easy to do in a bitmask indexed by block
> > type, and will fit nicely in generalized handling of the enable bits.
> 
> Done
> 
> > > > +
> > > > +static void rkisp1_ext_params_dpcc(struct rkisp1_params *params,
> > > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_dpcc_config *dpcc =
> > > > +		(struct rkisp1_ext_params_dpcc_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> > > > +					RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_dpcc_config(params, &dpcc->dpcc_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPCC_MODE,
> > > > +				      RKISP1_CIF_ISP_DPCC_MODE_DPCC_ENABLE);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_sdg(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_sdg_config *sdg =
> > > > +		(struct rkisp1_ext_params_sdg_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_sdg_config(params, &sdg->sdg_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_IN_ENA);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_lsc(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_lsc_config *lsc =
> > > > +		(struct rkisp1_ext_params_lsc_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> > > > +					RKISP1_CIF_ISP_LSC_CTRL_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_lsc_config(params, &lsc->lsc_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_LSC_CTRL,
> > > > +				      RKISP1_CIF_ISP_LSC_CTRL_ENA);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_awbg(struct rkisp1_params *params,
> > > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_awb_gain_config *awbg =
> > > > +		(struct rkisp1_ext_params_awb_gain_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > +					RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	params->ops->awb_gain_config(params, &awbg->awb_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > +				      RKISP1_CIF_ISP_CTRL_ISP_AWB_ENA);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_flt(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_flt_config *flt =
> > > > +		(struct rkisp1_ext_params_flt_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> > > > +					RKISP1_CIF_ISP_FLT_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_flt_config(params, &flt->flt_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_FILT_MODE,
> > > > +				      RKISP1_CIF_ISP_FLT_ENA);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_bdm(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_bdm_config *bdm =
> > > > +		(struct rkisp1_ext_params_bdm_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> > > > +					RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_bdm_config(params, &bdm->bdm_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DEMOSAIC,
> > > > +				      RKISP1_CIF_ISP_DEMOSAIC_BYPASS);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_ctk(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_ctk_config *ctk =
> > > > +		(struct rkisp1_ext_params_ctk_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_ctk_enable(params, false);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_ctk_config(params, &ctk->ctk_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_ctk_enable(params, true);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_goc(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_goc_config *goc =
> > > > +		(struct rkisp1_ext_params_goc_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > +					RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	params->ops->goc_config(params, &goc->goc_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > +				      RKISP1_CIF_ISP_CTRL_ISP_GAMMA_OUT_ENA);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_dpf(struct rkisp1_params *params,
> > > > +				  struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_dpf_config *dpf =
> > > > +		(struct rkisp1_ext_params_dpf_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> > > > +					RKISP1_CIF_ISP_DPF_MODE_EN);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_dpf_config(params, &dpf->dpf_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_DPF_MODE,
> > > > +				      RKISP1_CIF_ISP_DPF_MODE_EN);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_dpfs(struct rkisp1_params *params,
> > > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_dpf_strength_config *dpfs =
> > > > +		(struct rkisp1_ext_params_dpf_strength_config *)hdr;
> > > > +
> > > > +	rkisp1_dpf_strength_config(params, &dpfs->dpf_strength_config);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_cproc(struct rkisp1_params *params,
> > > > +				    struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_cproc_config *cproc =
> > > > +		(struct rkisp1_ext_params_cproc_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_C_PROC_CTRL,
> > > > +					RKISP1_CIF_C_PROC_CTR_ENABLE);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_cproc_config(params, &cproc->cproc_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_C_PROC_CTRL,
> > > > +				      RKISP1_CIF_C_PROC_CTR_ENABLE);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_ie(struct rkisp1_params *params,
> > > > +				 struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_ie_config *ie =
> > > > +		(struct rkisp1_ext_params_ie_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_ie_enable(params, false);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	rkisp1_ie_config(params, &ie->ie_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_ie_enable(params, true);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_awbm(struct rkisp1_params *params,
> > > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_awb_meas_config *awbm =
> > > > +		(struct rkisp1_ext_params_awb_meas_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> > > > +					     false);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	params->ops->awb_meas_config(params, &awbm->awb_meas_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		params->ops->awb_meas_enable(params, &awbm->awb_meas_config,
> > > > +					     true);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_hstm(struct rkisp1_params *params,
> > > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_hst_config *hst =
> > > > +		(struct rkisp1_ext_params_hst_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		params->ops->hst_enable(params, &hst->hst_config, false);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	params->ops->hst_config(params, &hst->hst_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		params->ops->hst_enable(params, &hst->hst_config, true);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_aecm(struct rkisp1_params *params,
> > > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_aec_config *aec =
> > > > +		(struct rkisp1_ext_params_aec_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> > > > +					RKISP1_CIF_ISP_EXP_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	params->ops->aec_config(params, &aec->aec_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_EXP_CTRL,
> > > > +				      RKISP1_CIF_ISP_EXP_ENA);
> > > > +}
> > > > +
> > > > +static void rkisp1_ext_params_afcm(struct rkisp1_params *params,
> > > > +				   struct rkisp1_ext_params_block_header *hdr)
> > > > +{
> > > > +	struct rkisp1_ext_params_afc_config *afc =
> > > > +		(struct rkisp1_ext_params_afc_config *)hdr;
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_DISABLE) {
> > > > +		rkisp1_param_clear_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> > > > +					RKISP1_CIF_ISP_AFM_ENA);
> > > > +		return;
> > > > +	}
> > > > +
> > > > +	params->ops->afm_config(params, &afc->afc_config);
> > > > +
> > > > +	if (hdr->state == RKISP1_EXT_PARAMS_BLOCK_ENABLE)
> > > > +		rkisp1_param_set_bits(params, RKISP1_CIF_ISP_AFM_CTRL,
> > > > +				      RKISP1_CIF_ISP_AFM_ENA);
> > > > +}
> > > > +
> > > > +typedef void (*rkisp1_block_handler)(struct rkisp1_params *params,
> > > > +				     struct rkisp1_ext_params_block_header *hdr);
> > > > +
> > > > +static const struct rkisp1_ext_params_handler {
> > > > +	size_t size;
> > > > +	rkisp1_block_handler handler;
> > > > +	unsigned int group;
> >
> > We'll have to extend this structure with a bitmask of the ISP versions
> > each block supports. That can be done later.
> >
> > > > +} rkisp1_ext_params_handlers[] = {
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BLS] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_bls_config),
> > > > +		.handler	= rkisp1_ext_params_bls,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPCC] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_dpcc_config),
> > > > +		.handler	= rkisp1_ext_params_dpcc,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_SDG] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_sdg_config),
> > > > +		.handler	= rkisp1_ext_params_sdg,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_GAINS] = {
> > > > +		.size		=
> > > > +			sizeof(struct rkisp1_ext_params_awb_gain_config),
> > > > +		.handler	= rkisp1_ext_params_awbg,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_FLT] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_flt_config),
> > > > +		.handler	= rkisp1_ext_params_flt,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_BDM] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_bdm_config),
> > > > +		.handler	= rkisp1_ext_params_bdm,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CTK] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_ctk_config),
> > > > +		.handler	= rkisp1_ext_params_ctk,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_GOC] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_goc_config),
> > > > +		.handler	= rkisp1_ext_params_goc,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_dpf_config),
> > > > +		.handler	= rkisp1_ext_params_dpf,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_DPF_STRENGHT] = {
> > > > +		.size		=
> > > > +			sizeof(struct rkisp1_ext_params_dpf_strength_config),
> > > > +		.handler	= rkisp1_ext_params_dpfs,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_CPROC] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_cproc_config),
> > > > +		.handler	= rkisp1_ext_params_cproc,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_IE] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_ie_config),
> > > > +		.handler	= rkisp1_ext_params_ie,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_LSC] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_lsc_config),
> > > > +		.handler	= rkisp1_ext_params_lsc,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AWB_MEAS] = {
> > > > +		.size		=
> > > > +			sizeof(struct rkisp1_ext_params_awb_meas_config),
> > > > +		.handler	= rkisp1_ext_params_awbm,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_HST_MEAS] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_hst_config),
> > > > +		.handler	= rkisp1_ext_params_hstm,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AEC_MEAS] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_aec_config),
> > > > +		.handler	= rkisp1_ext_params_aecm,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > > +	},
> > > > +	[RKISP1_EXT_PARAMS_BLOCK_TYPE_AFC_MEAS] = {
> > > > +		.size		= sizeof(struct rkisp1_ext_params_afc_config),
> > > > +		.handler	= rkisp1_ext_params_afcm,
> > > > +		.group		= RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS
> > > > +	},
> > > > +};
> > > > +
> > > > +static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > +				      struct rkisp1_ext_params_cfg *cfg,
> > > > +				      u32 block_group_mask)
> > > > +{
> > > > +	size_t block_offset = 0;
> > > > +
> > > > +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > > +		dev_dbg(params->rkisp1->dev,
> > > > +			"Invalid parameters buffer size %llu\n",
> > > > +			cfg->total_size);
> > > > +		return -EINVAL;
> > > > +	}
> > > > +
> > > > +	/* Walk the list of parameter blocks and process them. */
> > > > +	while (block_offset < cfg->total_size) {
> > > > +		const struct rkisp1_ext_params_handler *block_handler;
> > > > +		struct rkisp1_ext_params_block_header *block;
> > > > +
> > > > +		block = (struct rkisp1_ext_params_block_header *)
> > > > +			&cfg->data[block_offset];
> > > > +		block_offset += block->size;
> > > > +
> > > > +		/*
> > > > +		 * Validate the block id and make sure the block group is in
> > > > +		 * the list of groups to configure.
> > > > +		 */
> > > > +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> >
> > Use ARRAY_SIZE().
> 
> I don't think I can use ARRAY_SIZE() on a enum type

No, but you can on rkisp1_ext_params_handlers, which is the array you
index by block->type just below.

> > > > +			dev_dbg(params->rkisp1->dev,
> > > > +				"Invalid parameters block type\n");
> > > > +			return -EINVAL;
> > > > +		}
> > > > +
> > > > +		block_handler = &rkisp1_ext_params_handlers[block->type];
> > > > +		if (!(block_handler->group & block_group_mask))
> > > > +			continue;
> > >
> > >
> > > So maybe something like
> > >
> > >
> > > if (block_handler->enable_reg)
> > >
> > >          __rkisp1_block_handle_enable_disable(block->state, block_handler->enable_reg,
> > > block_handler->enable_val);
> > >
> > >
> > > here to move it out of the handlers.
> > >
> > > > +
> > > > +		if (block->size != block_handler->size) {
> > > > +			dev_dbg(params->rkisp1->dev,
> > > > +				"Invalid parameters block size\n");
> > > > +			return -EINVAL;
> > > > +		}
> > > > +
> > > > +		block_handler->handler(params, block);
> >
> > It would be nicer to move validation to qbuf time, and applying
> > parameters at runtime. Maybe I'll find that later in the series :-)
> >
> > > > +	}
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > +				    struct rkisp1_ext_params_cfg *cfg)
> > > > +{
> > > > +	return __rkisp1_ext_params_config(params, cfg,
> > > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > +}
> > > > +
> > > > +static int
> > > > +rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > > > +				    struct rkisp1_ext_params_cfg *cfg)
> > > > +{
> > > > +	return __rkisp1_ext_params_config(params, cfg,
> > > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > +}
> > > > +
> > > > +static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > > > +					struct rkisp1_ext_params_cfg *cfg)
> > > > +{
> > > > +	return __rkisp1_ext_params_config(params, cfg,
> > > > +					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > > > +}
> >
> > I'm tempted to inline this in the callers, I think it would be more
> > readable.
> >
> > > > +
> > > >   static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
> > > >   				     struct rkisp1_buffer **buf,
> > > > -				     struct rkisp1_params_cfg **cfg)
> > > > +				     void **cfg)
> > > >   {
> > > >   	if (list_empty(&params->params))
> > > >   		return false;
> > > > @@ -1544,28 +2030,37 @@ static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
> > > >
> > > >   static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
> > > >   					  struct rkisp1_buffer *buf,
> > > > -					  unsigned int frame_sequence)
> > > > +					  unsigned int frame_sequence,
> > > > +					  enum vb2_buffer_state state)
> > > >   {
> > > >   	list_del(&buf->queue);
> > > >
> > > >   	buf->vb.sequence = frame_sequence;
> > > > -	vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> > > > +	vb2_buffer_done(&buf->vb.vb2_buf, state);
> > > >   }
> > > >
> > > >   void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > > >   {
> > > >   	struct rkisp1_params *params = &rkisp1->params;
> > > > -	struct rkisp1_params_cfg *new_params;
> > > > -	struct rkisp1_buffer *cur_buf;
> > > > +	struct rkisp1_buffer *buf;
> > >
> > > Why the rename?
> 
> Because "cur" implies there's a "next" or a "prev".
> I can drop it though.
> 
> > > > +	int ret = 0;
> > > > +	void *cfg;
> > > >
> > > >   	spin_lock(&params->config_lock);
> > > >
> > > > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > > > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> > > >   		goto unlock;
> > > >
> > > > -	rkisp1_isp_isr_other_config(params, new_params);
> > > > -	rkisp1_isp_isr_lsc_config(params, new_params);
> > > > -	rkisp1_isp_isr_meas_config(params, new_params);
> > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > > +		rkisp1_isp_isr_other_config(params, cfg);
> > > > +		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > +		rkisp1_isp_isr_meas_config(params, cfg);
> > > > +	} else {
> > > > +		ret = rkisp1_ext_params_config(params, cfg);
> > > > +	}
> > > > +
> > > > +	if (ret)
> > > > +		goto complete_and_unlock;
> >
> > As validation should happen at qbuf time, I think it would be nicer to
> > reorder the patches to avoid introducing error handling here and
> > removing it later.
> >
> > > >
> > > >   	/* update shadow register immediately */
> > > >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > @@ -1579,8 +2074,11 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > > >   	 * indicate to userspace on which frame these parameters are being
> > > >   	 * applied.
> > > >   	 */
> > > > -	rkisp1_params_complete_buffer(params, cur_buf,
> > > > -				      rkisp1->isp.frame_sequence + 1);
> > > > +complete_and_unlock:
> > > > +	rkisp1_params_complete_buffer(params, buf,
> > > > +				      rkisp1->isp.frame_sequence + 1,
> > > > +				      ret ? VB2_BUF_STATE_ERROR
> > > > +					  : VB2_BUF_STATE_DONE);
> > > >
> > > >   unlock:
> > > >   	spin_unlock(&params->config_lock);
> > > > @@ -1631,8 +2129,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > > >   				 enum v4l2_ycbcr_encoding ycbcr_encoding)
> > > >   {
> > > >   	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
> > > > -	struct rkisp1_params_cfg *new_params;
> > > > -	struct rkisp1_buffer *cur_buf;
> > > > +	struct rkisp1_buffer *buf;
> > > > +	int ret = 0;
> > > > +	void *cfg;
> > > >
> > > >   	params->quantization = quantization;
> > > >   	params->ycbcr_encoding = ycbcr_encoding;
> > > > @@ -1661,11 +2160,26 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > > >
> > > >   	/* apply the first buffer if there is one already */
> > > >
> > > > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > > > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> > > >   		goto unlock;
> > > >
> > > > -	rkisp1_isp_isr_other_config(params, new_params);
> > > > -	rkisp1_isp_isr_meas_config(params, new_params);
> > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > > +		rkisp1_isp_isr_other_config(params, cfg);
> > > > +		rkisp1_isp_isr_meas_config(params, cfg);
> > > > +	} else {
> > > > +		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > > > +	}
> > > > +
> > > > +	if (ret) {
> > > > +		/*
> > > > +		 * Complete the buffer in error state immediately. In case of no
> > > > +		 * error, the buffer will be completed in
> > > > +		 * rkisp1_params_post_configure().
> > > > +		 */
> > > > +		rkisp1_params_complete_buffer(params, buf, 0,
> > > > +					      VB2_BUF_STATE_ERROR);
> > > > +		goto unlock;
> > > > +	}
> > > >
> > > >   	/* update shadow register immediately */
> > > >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > > @@ -1677,8 +2191,9 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > > >
> > > >   void rkisp1_params_post_configure(struct rkisp1_params *params)
> > > >   {
> > > > -	struct rkisp1_params_cfg *new_params;
> > > > -	struct rkisp1_buffer *cur_buf;
> > > > +	struct rkisp1_buffer *buf;
> > >
> > > And likewise here?
> > >
> > > > +	int ret = 0;
> > > > +	void *cfg;
> > > >
> > > >   	spin_lock_irq(&params->config_lock);
> > > >
> > > > @@ -1691,16 +2206,24 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
> > > >   	 * unconditionally.
> > > >   	 */
> > > >
> > > > -	if (!rkisp1_params_get_buffer(params, &cur_buf, &new_params))
> > > > +	if (!rkisp1_params_get_buffer(params, &buf, &cfg))
> > > >   		goto unlock;
> > > >
> > > > -	rkisp1_isp_isr_lsc_config(params, new_params);
> > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > > +		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > +	else
> > > > +		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > > > +
> > > > +	if (ret)
> > > > +		goto complete_and_unlock;
> > > >
> > > >   	/* update shadow register immediately */
> > > >   	rkisp1_param_set_bits(params, RKISP1_CIF_ISP_CTRL,
> > > >   			      RKISP1_CIF_ISP_CTRL_ISP_CFG_UPD);
> > > >
> > > > -	rkisp1_params_complete_buffer(params, cur_buf, 0);
> > > > +complete_and_unlock:
> > > > +	rkisp1_params_complete_buffer(params, buf, 0, ret ? VB2_BUF_STATE_ERROR
> > > > +							  : VB2_BUF_STATE_DONE);
> > > >
> > > >   unlock:
> > > >   	spin_unlock_irq(&params->config_lock);

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 6/8] media: rkisp1: Propagate pre/post-config errors
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
                   ` (4 preceding siblings ...)
  2024-06-05 16:54 ` [PATCH 5/8] media: rkisp1: Implement extensible params support Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 13:35   ` Dan Scally
  2024-06-05 16:54 ` [PATCH 7/8] media: rkisp1: Add struct rkisp1_params_buffer Jacopo Mondi
  2024-06-05 16:54 ` [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer Jacopo Mondi
  7 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

The support for the extensible parameters format introduces the
possibility of failures in handling the parameters buffer.

Errors in parsing the configuration parameters are not propagated
to the rkisp1_config_isp() and the rkisp1_isp_start() functions.

Propagate any possible errors to the callers to report it to userspace.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../media/platform/rockchip/rkisp1/rkisp1-common.h | 10 +++++-----
 .../media/platform/rockchip/rkisp1/rkisp1-isp.c    | 14 +++++++++-----
 .../media/platform/rockchip/rkisp1/rkisp1-params.c | 14 +++++++++-----
 3 files changed, 23 insertions(+), 15 deletions(-)

diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
index 0bddae8dbdb1..f9df5ed96c98 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
@@ -591,10 +591,10 @@ const struct rkisp1_mbus_info *rkisp1_mbus_info_get_by_code(u32 mbus_code);
  * It applies the initial ISP parameters from the first params buffer, but
  * skips LSC as it needs to be configured after the ISP is started.
  */
-void rkisp1_params_pre_configure(struct rkisp1_params *params,
-				 enum rkisp1_fmt_raw_pat_type bayer_pat,
-				 enum v4l2_quantization quantization,
-				 enum v4l2_ycbcr_encoding ycbcr_encoding);
+int rkisp1_params_pre_configure(struct rkisp1_params *params,
+				enum rkisp1_fmt_raw_pat_type bayer_pat,
+				enum v4l2_quantization quantization,
+				enum v4l2_ycbcr_encoding ycbcr_encoding);
 
 /*
  * rkisp1_params_post_configure - Configure the params after stream start
@@ -604,7 +604,7 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
  * This function is called by the ISP entity just after the ISP gets started.
  * It applies the initial ISP LSC parameters from the first params buffer.
  */
-void rkisp1_params_post_configure(struct rkisp1_params *params);
+int rkisp1_params_post_configure(struct rkisp1_params *params);
 
 /* rkisp1_params_disable - disable all parameters.
  *			   This function is called by the isp entity upon stream start
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
index 91301d17d356..05227c6a16fe 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
@@ -310,12 +310,16 @@ static int rkisp1_config_isp(struct rkisp1_isp *isp,
 		rkisp1_params_disable(&rkisp1->params);
 	} else {
 		const struct v4l2_mbus_framefmt *src_frm;
+		int ret;
 
 		src_frm = v4l2_subdev_state_get_format(sd_state,
 						       RKISP1_ISP_PAD_SOURCE_VIDEO);
-		rkisp1_params_pre_configure(&rkisp1->params, sink_fmt->bayer_pat,
-					    src_frm->quantization,
-					    src_frm->ycbcr_enc);
+		ret = rkisp1_params_pre_configure(&rkisp1->params,
+						  sink_fmt->bayer_pat,
+						  src_frm->quantization,
+						  src_frm->ycbcr_enc);
+		if (ret)
+			return ret;
 	}
 
 	isp->sink_fmt = sink_fmt;
@@ -458,9 +462,9 @@ static int rkisp1_isp_start(struct rkisp1_isp *isp,
 	src_info = rkisp1_mbus_info_get_by_code(src_fmt->code);
 
 	if (src_info->pixel_enc != V4L2_PIXEL_ENC_BAYER)
-		rkisp1_params_post_configure(&rkisp1->params);
+		ret = rkisp1_params_post_configure(&rkisp1->params);
 
-	return 0;
+	return ret;
 }
 
 /* ----------------------------------------------------------------------------
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
index 3d78e643d0b8..c081fd490b2b 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
@@ -2123,10 +2123,10 @@ static const struct rkisp1_cif_isp_afc_config rkisp1_afc_params_default_config =
 	14
 };
 
-void rkisp1_params_pre_configure(struct rkisp1_params *params,
-				 enum rkisp1_fmt_raw_pat_type bayer_pat,
-				 enum v4l2_quantization quantization,
-				 enum v4l2_ycbcr_encoding ycbcr_encoding)
+int rkisp1_params_pre_configure(struct rkisp1_params *params,
+				enum rkisp1_fmt_raw_pat_type bayer_pat,
+				enum v4l2_quantization quantization,
+				enum v4l2_ycbcr_encoding ycbcr_encoding)
 {
 	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
 	struct rkisp1_buffer *buf;
@@ -2187,9 +2187,11 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
 
 unlock:
 	spin_unlock_irq(&params->config_lock);
+
+	return ret;
 }
 
-void rkisp1_params_post_configure(struct rkisp1_params *params)
+int rkisp1_params_post_configure(struct rkisp1_params *params)
 {
 	struct rkisp1_buffer *buf;
 	int ret = 0;
@@ -2227,6 +2229,8 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
 
 unlock:
 	spin_unlock_irq(&params->config_lock);
+
+	return ret;
 }
 
 /*
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 6/8] media: rkisp1: Propagate pre/post-config errors
  2024-06-05 16:54 ` [PATCH 6/8] media: rkisp1: Propagate pre/post-config errors Jacopo Mondi
@ 2024-06-12 13:35   ` Dan Scally
  2024-06-12 15:46     ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Dan Scally @ 2024-06-12 13:35 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo - thanks for the patch. I think this probably should come before 5/8 in the series, and 
just hardcode return 0 in rkisp1_params_pre/post_configure() temporarily.

On 05/06/2024 17:54, Jacopo Mondi wrote:
> The support for the extensible parameters format introduces the
> possibility of failures in handling the parameters buffer.
>
> Errors in parsing the configuration parameters are not propagated
> to the rkisp1_config_isp() and the rkisp1_isp_start() functions.
>
> Propagate any possible errors to the callers to report it to userspace.
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>   .../media/platform/rockchip/rkisp1/rkisp1-common.h | 10 +++++-----
>   .../media/platform/rockchip/rkisp1/rkisp1-isp.c    | 14 +++++++++-----
>   .../media/platform/rockchip/rkisp1/rkisp1-params.c | 14 +++++++++-----
>   3 files changed, 23 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> index 0bddae8dbdb1..f9df5ed96c98 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> @@ -591,10 +591,10 @@ const struct rkisp1_mbus_info *rkisp1_mbus_info_get_by_code(u32 mbus_code);
>    * It applies the initial ISP parameters from the first params buffer, but
>    * skips LSC as it needs to be configured after the ISP is started.
>    */
> -void rkisp1_params_pre_configure(struct rkisp1_params *params,
> -				 enum rkisp1_fmt_raw_pat_type bayer_pat,
> -				 enum v4l2_quantization quantization,
> -				 enum v4l2_ycbcr_encoding ycbcr_encoding);
> +int rkisp1_params_pre_configure(struct rkisp1_params *params,
> +				enum rkisp1_fmt_raw_pat_type bayer_pat,
> +				enum v4l2_quantization quantization,
> +				enum v4l2_ycbcr_encoding ycbcr_encoding);
>   
>   /*
>    * rkisp1_params_post_configure - Configure the params after stream start
> @@ -604,7 +604,7 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
>    * This function is called by the ISP entity just after the ISP gets started.
>    * It applies the initial ISP LSC parameters from the first params buffer.
>    */
> -void rkisp1_params_post_configure(struct rkisp1_params *params);
> +int rkisp1_params_post_configure(struct rkisp1_params *params);
>   
>   /* rkisp1_params_disable - disable all parameters.
>    *			   This function is called by the isp entity upon stream start
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> index 91301d17d356..05227c6a16fe 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> @@ -310,12 +310,16 @@ static int rkisp1_config_isp(struct rkisp1_isp *isp,
>   		rkisp1_params_disable(&rkisp1->params);
>   	} else {
>   		const struct v4l2_mbus_framefmt *src_frm;
> +		int ret;
>   
>   		src_frm = v4l2_subdev_state_get_format(sd_state,
>   						       RKISP1_ISP_PAD_SOURCE_VIDEO);
> -		rkisp1_params_pre_configure(&rkisp1->params, sink_fmt->bayer_pat,
> -					    src_frm->quantization,
> -					    src_frm->ycbcr_enc);
> +		ret = rkisp1_params_pre_configure(&rkisp1->params,
> +						  sink_fmt->bayer_pat,
> +						  src_frm->quantization,
> +						  src_frm->ycbcr_enc);
> +		if (ret)
> +			return ret;
>   	}
>   
>   	isp->sink_fmt = sink_fmt;
> @@ -458,9 +462,9 @@ static int rkisp1_isp_start(struct rkisp1_isp *isp,
>   	src_info = rkisp1_mbus_info_get_by_code(src_fmt->code);
>   
>   	if (src_info->pixel_enc != V4L2_PIXEL_ENC_BAYER)
> -		rkisp1_params_post_configure(&rkisp1->params);
> +		ret = rkisp1_params_post_configure(&rkisp1->params);
>   
> -	return 0;
> +	return ret;

I think ret could be returned uninitialised in some circumstances in this function now - if it's not 
the IMX8MP version and the pixel encoding is bayer...or am I missing something?

>   }
>   
>   /* ----------------------------------------------------------------------------
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index 3d78e643d0b8..c081fd490b2b 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -2123,10 +2123,10 @@ static const struct rkisp1_cif_isp_afc_config rkisp1_afc_params_default_config =
>   	14
>   };
>   
> -void rkisp1_params_pre_configure(struct rkisp1_params *params,
> -				 enum rkisp1_fmt_raw_pat_type bayer_pat,
> -				 enum v4l2_quantization quantization,
> -				 enum v4l2_ycbcr_encoding ycbcr_encoding)
> +int rkisp1_params_pre_configure(struct rkisp1_params *params,
> +				enum rkisp1_fmt_raw_pat_type bayer_pat,
> +				enum v4l2_quantization quantization,
> +				enum v4l2_ycbcr_encoding ycbcr_encoding)
>   {
>   	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
>   	struct rkisp1_buffer *buf;
> @@ -2187,9 +2187,11 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
>   
>   unlock:
>   	spin_unlock_irq(&params->config_lock);
> +
> +	return ret;
>   }
>   
> -void rkisp1_params_post_configure(struct rkisp1_params *params)
> +int rkisp1_params_post_configure(struct rkisp1_params *params)
>   {
>   	struct rkisp1_buffer *buf;
>   	int ret = 0;
> @@ -2227,6 +2229,8 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
>   
>   unlock:
>   	spin_unlock_irq(&params->config_lock);
> +
> +	return ret;
>   }
>   
>   /*

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 6/8] media: rkisp1: Propagate pre/post-config errors
  2024-06-12 13:35   ` Dan Scally
@ 2024-06-12 15:46     ` Laurent Pinchart
  0 siblings, 0 replies; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 15:46 UTC (permalink / raw)
  To: Dan Scally
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 12, 2024 at 02:35:39PM +0100, Daniel Scally wrote:
> Hi Jacopo - thanks for the patch. I think this probably should come
> before 5/8 in the series, and just hardcode return 0 in
> rkisp1_params_pre/post_configure() temporarily.

There should be no need to return errors from those two functions.
They're called at runtime to apply parameters. Validation of the
parameters should happen earlier, at qbuf time.

> On 05/06/2024 17:54, Jacopo Mondi wrote:
> > The support for the extensible parameters format introduces the
> > possibility of failures in handling the parameters buffer.
> >
> > Errors in parsing the configuration parameters are not propagated
> > to the rkisp1_config_isp() and the rkisp1_isp_start() functions.
> >
> > Propagate any possible errors to the callers to report it to userspace.
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >   .../media/platform/rockchip/rkisp1/rkisp1-common.h | 10 +++++-----
> >   .../media/platform/rockchip/rkisp1/rkisp1-isp.c    | 14 +++++++++-----
> >   .../media/platform/rockchip/rkisp1/rkisp1-params.c | 14 +++++++++-----
> >   3 files changed, 23 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> > index 0bddae8dbdb1..f9df5ed96c98 100644
> > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> > @@ -591,10 +591,10 @@ const struct rkisp1_mbus_info *rkisp1_mbus_info_get_by_code(u32 mbus_code);
> >    * It applies the initial ISP parameters from the first params buffer, but
> >    * skips LSC as it needs to be configured after the ISP is started.
> >    */
> > -void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > -				 enum rkisp1_fmt_raw_pat_type bayer_pat,
> > -				 enum v4l2_quantization quantization,
> > -				 enum v4l2_ycbcr_encoding ycbcr_encoding);
> > +int rkisp1_params_pre_configure(struct rkisp1_params *params,
> > +				enum rkisp1_fmt_raw_pat_type bayer_pat,
> > +				enum v4l2_quantization quantization,
> > +				enum v4l2_ycbcr_encoding ycbcr_encoding);
> >   
> >   /*
> >    * rkisp1_params_post_configure - Configure the params after stream start
> > @@ -604,7 +604,7 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> >    * This function is called by the ISP entity just after the ISP gets started.
> >    * It applies the initial ISP LSC parameters from the first params buffer.
> >    */
> > -void rkisp1_params_post_configure(struct rkisp1_params *params);
> > +int rkisp1_params_post_configure(struct rkisp1_params *params);
> >   
> >   /* rkisp1_params_disable - disable all parameters.
> >    *			   This function is called by the isp entity upon stream start
> > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> > index 91301d17d356..05227c6a16fe 100644
> > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-isp.c
> > @@ -310,12 +310,16 @@ static int rkisp1_config_isp(struct rkisp1_isp *isp,
> >   		rkisp1_params_disable(&rkisp1->params);
> >   	} else {
> >   		const struct v4l2_mbus_framefmt *src_frm;
> > +		int ret;
> >   
> >   		src_frm = v4l2_subdev_state_get_format(sd_state,
> >   						       RKISP1_ISP_PAD_SOURCE_VIDEO);
> > -		rkisp1_params_pre_configure(&rkisp1->params, sink_fmt->bayer_pat,
> > -					    src_frm->quantization,
> > -					    src_frm->ycbcr_enc);
> > +		ret = rkisp1_params_pre_configure(&rkisp1->params,
> > +						  sink_fmt->bayer_pat,
> > +						  src_frm->quantization,
> > +						  src_frm->ycbcr_enc);
> > +		if (ret)
> > +			return ret;
> >   	}
> >   
> >   	isp->sink_fmt = sink_fmt;
> > @@ -458,9 +462,9 @@ static int rkisp1_isp_start(struct rkisp1_isp *isp,
> >   	src_info = rkisp1_mbus_info_get_by_code(src_fmt->code);
> >   
> >   	if (src_info->pixel_enc != V4L2_PIXEL_ENC_BAYER)
> > -		rkisp1_params_post_configure(&rkisp1->params);
> > +		ret = rkisp1_params_post_configure(&rkisp1->params);
> >   
> > -	return 0;
> > +	return ret;
> 
> I think ret could be returned uninitialised in some circumstances in this function now - if it's not 
> the IMX8MP version and the pixel encoding is bayer...or am I missing something?
> 
> >   }
> >   
> >   /* ----------------------------------------------------------------------------
> > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > index 3d78e643d0b8..c081fd490b2b 100644
> > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > @@ -2123,10 +2123,10 @@ static const struct rkisp1_cif_isp_afc_config rkisp1_afc_params_default_config =
> >   	14
> >   };
> >   
> > -void rkisp1_params_pre_configure(struct rkisp1_params *params,
> > -				 enum rkisp1_fmt_raw_pat_type bayer_pat,
> > -				 enum v4l2_quantization quantization,
> > -				 enum v4l2_ycbcr_encoding ycbcr_encoding)
> > +int rkisp1_params_pre_configure(struct rkisp1_params *params,
> > +				enum rkisp1_fmt_raw_pat_type bayer_pat,
> > +				enum v4l2_quantization quantization,
> > +				enum v4l2_ycbcr_encoding ycbcr_encoding)
> >   {
> >   	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
> >   	struct rkisp1_buffer *buf;
> > @@ -2187,9 +2187,11 @@ void rkisp1_params_pre_configure(struct rkisp1_params *params,
> >   
> >   unlock:
> >   	spin_unlock_irq(&params->config_lock);
> > +
> > +	return ret;
> >   }
> >   
> > -void rkisp1_params_post_configure(struct rkisp1_params *params)
> > +int rkisp1_params_post_configure(struct rkisp1_params *params)
> >   {
> >   	struct rkisp1_buffer *buf;
> >   	int ret = 0;
> > @@ -2227,6 +2229,8 @@ void rkisp1_params_post_configure(struct rkisp1_params *params)
> >   
> >   unlock:
> >   	spin_unlock_irq(&params->config_lock);
> > +
> > +	return ret;
> >   }
> >   
> >   /*

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 7/8] media: rkisp1: Add struct rkisp1_params_buffer
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
                   ` (5 preceding siblings ...)
  2024-06-05 16:54 ` [PATCH 6/8] media: rkisp1: Propagate pre/post-config errors Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 13:52   ` Dan Scally
  2024-06-05 16:54 ` [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer Jacopo Mondi
  7 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

Create the 'struct rkisp1_params_buffer' type that wraps a
vb2_v4l2_buffer and contain a pointer to an optional copy of
the parameters buffer that will be used to copy the user-provided
configuration buffer in the following patches.

Replace usage of 'struct rkisp1_buffer' with 'struct
rkisp1_params_buffer' in rkisp1-params.c to prepare for that.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../platform/rockchip/rkisp1/rkisp1-common.h  | 16 +++++++++++++-
 .../platform/rockchip/rkisp1/rkisp1-params.c  | 21 ++++++++++---------
 2 files changed, 26 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
index f9df5ed96c98..3118f1974246 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
@@ -232,7 +232,7 @@ struct rkisp1_vdev_node {
 
 /*
  * struct rkisp1_buffer - A container for the vb2 buffers used by the video devices:
- *			  params, stats, mainpath, selfpath
+ *			  stats, mainpath, selfpath
  *
  * @vb:		vb2 buffer
  * @queue:	entry of the buffer in the queue
@@ -244,6 +244,20 @@ struct rkisp1_buffer {
 	dma_addr_t buff_addr[VIDEO_MAX_PLANES];
 };
 
+/*
+ * struct rkisp1_params_buffer - A container for the vb2 buffers used by the
+ *				 params video device
+ *
+ * @vb:		vb2 buffer
+ * @queue:	entry of the buffer in the queue
+ * cfg:		scratch buffer used for the extensible parameters format
+ */
+struct rkisp1_params_buffer {
+	struct vb2_v4l2_buffer vb;
+	struct list_head queue;
+	struct rkisp1_ext_params_cfg *cfg;
+};
+
 /*
  * struct rkisp1_dummy_buffer - A buffer to write the next frame to in case
  *				there are no vb2 buffers available.
diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
index c081fd490b2b..4adaf084ce6e 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
@@ -2016,20 +2016,21 @@ static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
 }
 
 static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
-				     struct rkisp1_buffer **buf,
+				     struct rkisp1_params_buffer **buf,
 				     void **cfg)
 {
 	if (list_empty(&params->params))
 		return false;
 
-	*buf = list_first_entry(&params->params, struct rkisp1_buffer, queue);
+	*buf = list_first_entry(&params->params, struct rkisp1_params_buffer,
+				queue);
 	*cfg = vb2_plane_vaddr(&(*buf)->vb.vb2_buf, 0);
 
 	return true;
 }
 
 static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
-					  struct rkisp1_buffer *buf,
+					  struct rkisp1_params_buffer *buf,
 					  unsigned int frame_sequence,
 					  enum vb2_buffer_state state)
 {
@@ -2042,7 +2043,7 @@ static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
 void rkisp1_params_isr(struct rkisp1_device *rkisp1)
 {
 	struct rkisp1_params *params = &rkisp1->params;
-	struct rkisp1_buffer *buf;
+	struct rkisp1_params_buffer *buf;
 	int ret = 0;
 	void *cfg;
 
@@ -2129,7 +2130,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
 				enum v4l2_ycbcr_encoding ycbcr_encoding)
 {
 	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
-	struct rkisp1_buffer *buf;
+	struct rkisp1_params_buffer *buf;
 	int ret = 0;
 	void *cfg;
 
@@ -2193,7 +2194,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
 
 int rkisp1_params_post_configure(struct rkisp1_params *params)
 {
-	struct rkisp1_buffer *buf;
+	struct rkisp1_params_buffer *buf;
 	int ret = 0;
 	void *cfg;
 
@@ -2409,8 +2410,8 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
 static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
 {
 	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
-	struct rkisp1_buffer *params_buf =
-		container_of(vbuf, struct rkisp1_buffer, vb);
+	struct rkisp1_params_buffer *params_buf =
+		container_of(vbuf, struct rkisp1_params_buffer, vb);
 	struct vb2_queue *vq = vb->vb2_queue;
 	struct rkisp1_params *params = vq->drv_priv;
 
@@ -2436,7 +2437,7 @@ static int rkisp1_params_vb2_buf_prepare(struct vb2_buffer *vb)
 static void rkisp1_params_vb2_stop_streaming(struct vb2_queue *vq)
 {
 	struct rkisp1_params *params = vq->drv_priv;
-	struct rkisp1_buffer *buf;
+	struct rkisp1_params_buffer *buf;
 	LIST_HEAD(tmp_list);
 
 	/*
@@ -2482,7 +2483,7 @@ static int rkisp1_params_init_vb2_queue(struct vb2_queue *q,
 	q->drv_priv = params;
 	q->ops = &rkisp1_params_vb2_ops;
 	q->mem_ops = &vb2_vmalloc_memops;
-	q->buf_struct_size = sizeof(struct rkisp1_buffer);
+	q->buf_struct_size = sizeof(struct rkisp1_params_buffer);
 	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
 	q->lock = &node->vlock;
 
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 7/8] media: rkisp1: Add struct rkisp1_params_buffer
  2024-06-05 16:54 ` [PATCH 7/8] media: rkisp1: Add struct rkisp1_params_buffer Jacopo Mondi
@ 2024-06-12 13:52   ` Dan Scally
  0 siblings, 0 replies; 38+ messages in thread
From: Dan Scally @ 2024-06-12 13:52 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo

On 05/06/2024 17:54, Jacopo Mondi wrote:
> Create the 'struct rkisp1_params_buffer' type that wraps a
> vb2_v4l2_buffer and contain a pointer to an optional copy of
> the parameters buffer that will be used to copy the user-provided
> configuration buffer in the following patches.
>
> Replace usage of 'struct rkisp1_buffer' with 'struct
> rkisp1_params_buffer' in rkisp1-params.c to prepare for that.
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>


The contents look fine (one teeny tiny thing below), though I think I'd personally push this before 
5/8 and squash 8/8 into 5/8 - up to you.


Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>

> ---
>   .../platform/rockchip/rkisp1/rkisp1-common.h  | 16 +++++++++++++-
>   .../platform/rockchip/rkisp1/rkisp1-params.c  | 21 ++++++++++---------
>   2 files changed, 26 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> index f9df5ed96c98..3118f1974246 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h
> @@ -232,7 +232,7 @@ struct rkisp1_vdev_node {
>   
>   /*
>    * struct rkisp1_buffer - A container for the vb2 buffers used by the video devices:
> - *			  params, stats, mainpath, selfpath
> + *			  stats, mainpath, selfpath
>    *
>    * @vb:		vb2 buffer
>    * @queue:	entry of the buffer in the queue
> @@ -244,6 +244,20 @@ struct rkisp1_buffer {
>   	dma_addr_t buff_addr[VIDEO_MAX_PLANES];
>   };
>   
> +/*
> + * struct rkisp1_params_buffer - A container for the vb2 buffers used by the
> + *				 params video device
> + *
> + * @vb:		vb2 buffer
> + * @queue:	entry of the buffer in the queue
> + * cfg:		scratch buffer used for the extensible parameters format
s/cfg/@cfg
> + */
> +struct rkisp1_params_buffer {
> +	struct vb2_v4l2_buffer vb;
> +	struct list_head queue;
> +	struct rkisp1_ext_params_cfg *cfg;
> +};
> +
>   /*
>    * struct rkisp1_dummy_buffer - A buffer to write the next frame to in case
>    *				there are no vb2 buffers available.
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index c081fd490b2b..4adaf084ce6e 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -2016,20 +2016,21 @@ static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
>   }
>   
>   static bool rkisp1_params_get_buffer(struct rkisp1_params *params,
> -				     struct rkisp1_buffer **buf,
> +				     struct rkisp1_params_buffer **buf,
>   				     void **cfg)
>   {
>   	if (list_empty(&params->params))
>   		return false;
>   
> -	*buf = list_first_entry(&params->params, struct rkisp1_buffer, queue);
> +	*buf = list_first_entry(&params->params, struct rkisp1_params_buffer,
> +				queue);
>   	*cfg = vb2_plane_vaddr(&(*buf)->vb.vb2_buf, 0);
>   
>   	return true;
>   }
>   
>   static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
> -					  struct rkisp1_buffer *buf,
> +					  struct rkisp1_params_buffer *buf,
>   					  unsigned int frame_sequence,
>   					  enum vb2_buffer_state state)
>   {
> @@ -2042,7 +2043,7 @@ static void rkisp1_params_complete_buffer(struct rkisp1_params *params,
>   void rkisp1_params_isr(struct rkisp1_device *rkisp1)
>   {
>   	struct rkisp1_params *params = &rkisp1->params;
> -	struct rkisp1_buffer *buf;
> +	struct rkisp1_params_buffer *buf;
>   	int ret = 0;
>   	void *cfg;
>   
> @@ -2129,7 +2130,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
>   				enum v4l2_ycbcr_encoding ycbcr_encoding)
>   {
>   	struct rkisp1_cif_isp_hst_config hst = rkisp1_hst_params_default_config;
> -	struct rkisp1_buffer *buf;
> +	struct rkisp1_params_buffer *buf;
>   	int ret = 0;
>   	void *cfg;
>   
> @@ -2193,7 +2194,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
>   
>   int rkisp1_params_post_configure(struct rkisp1_params *params)
>   {
> -	struct rkisp1_buffer *buf;
> +	struct rkisp1_params_buffer *buf;
>   	int ret = 0;
>   	void *cfg;
>   
> @@ -2409,8 +2410,8 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
>   static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
>   {
>   	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> -	struct rkisp1_buffer *params_buf =
> -		container_of(vbuf, struct rkisp1_buffer, vb);
> +	struct rkisp1_params_buffer *params_buf =
> +		container_of(vbuf, struct rkisp1_params_buffer, vb);
>   	struct vb2_queue *vq = vb->vb2_queue;
>   	struct rkisp1_params *params = vq->drv_priv;
>   
> @@ -2436,7 +2437,7 @@ static int rkisp1_params_vb2_buf_prepare(struct vb2_buffer *vb)
>   static void rkisp1_params_vb2_stop_streaming(struct vb2_queue *vq)
>   {
>   	struct rkisp1_params *params = vq->drv_priv;
> -	struct rkisp1_buffer *buf;
> +	struct rkisp1_params_buffer *buf;
>   	LIST_HEAD(tmp_list);
>   
>   	/*
> @@ -2482,7 +2483,7 @@ static int rkisp1_params_init_vb2_queue(struct vb2_queue *q,
>   	q->drv_priv = params;
>   	q->ops = &rkisp1_params_vb2_ops;
>   	q->mem_ops = &vb2_vmalloc_memops;
> -	q->buf_struct_size = sizeof(struct rkisp1_buffer);
> +	q->buf_struct_size = sizeof(struct rkisp1_params_buffer);
>   	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>   	q->lock = &node->vlock;
>   

^ permalink raw reply	[flat|nested] 38+ messages in thread

* [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
                   ` (6 preceding siblings ...)
  2024-06-05 16:54 ` [PATCH 7/8] media: rkisp1: Add struct rkisp1_params_buffer Jacopo Mondi
@ 2024-06-05 16:54 ` Jacopo Mondi
  2024-06-12 14:28   ` Dan Scally
  7 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-05 16:54 UTC (permalink / raw)
  To: Linux Media Mailing List
  Cc: Jacopo Mondi, Laurent Pinchart, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Daniel Scally, Kieran Bingham,
	Umang Jain, Dafna Hirschfeld, Mauro Carvalho Chehab,
	Heiko Stuebner

With the introduction of the extensible parameters format support in the
rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
by parsing the content of a data buffer of variable size provided by
userspace through the V4L2 meta-output interface using the MMAP memory
handling mode.

As the parameters buffer is mapped in the userspace process memory,
applications have access to the buffer content while the driver
parses it.

To prevent potential issues during the parameters buffer parsing and
processing in the driver, implement three vb2_ops to

1) allocate a scratch buffer in the driver private buffer structure
2) validate the buffer content at VIDIOC_QBUF time
3) copy the content of the user provided configuration parameters
   in the driver-private scratch buffer

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
 1 file changed, 124 insertions(+), 30 deletions(-)

diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
index 4adaf084ce6e..003239e14511 100644
--- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
+++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
@@ -5,6 +5,8 @@
  * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
  */
 
+#include <linux/string.h>
+
 #include <media/v4l2-common.h>
 #include <media/v4l2-event.h>
 #include <media/v4l2-ioctl.h>
@@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
 };
 
 static int __rkisp1_ext_params_config(struct rkisp1_params *params,
-				      struct rkisp1_ext_params_cfg *cfg,
+				      struct rkisp1_params_buffer *buffer,
 				      u32 block_group_mask)
 {
+	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;
 	size_t block_offset = 0;
 
-	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
-		dev_dbg(params->rkisp1->dev,
-			"Invalid parameters buffer size %llu\n",
-			cfg->total_size);
-		return -EINVAL;
-	}
+	if (WARN_ON(!cfg))
+		return -ENOMEM;
 
 	/* Walk the list of parameter blocks and process them. */
 	while (block_offset < cfg->total_size) {
@@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
 		block_offset += block->size;
 
 		/*
-		 * Validate the block id and make sure the block group is in
-		 * the list of groups to configure.
+		 * Make sure the block group is in  the list of groups to
+		 * configure.
 		 */
-		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
-			dev_dbg(params->rkisp1->dev,
-				"Invalid parameters block type\n");
-			return -EINVAL;
-		}
-
 		block_handler = &rkisp1_ext_params_handlers[block->type];
 		if (!(block_handler->group & block_group_mask))
 			continue;
 
-		if (block->size != block_handler->size) {
-			dev_dbg(params->rkisp1->dev,
-				"Invalid parameters block size\n");
-			return -EINVAL;
-		}
-
 		block_handler->handler(params, block);
 	}
 
@@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
 }
 
 static int rkisp1_ext_params_config(struct rkisp1_params *params,
-				    struct rkisp1_ext_params_cfg *cfg)
+				    struct rkisp1_params_buffer *buffer)
 {
-	return __rkisp1_ext_params_config(params, cfg,
+	return __rkisp1_ext_params_config(params, buffer,
 					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
 					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
 					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
@@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
 
 static int
 rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
-				    struct rkisp1_ext_params_cfg *cfg)
+				    struct rkisp1_params_buffer *buffer)
 {
-	return __rkisp1_ext_params_config(params, cfg,
+	return __rkisp1_ext_params_config(params, buffer,
 					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
 					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
 }
 
 static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
-					struct rkisp1_ext_params_cfg *cfg)
+					struct rkisp1_params_buffer *buffer)
 {
-	return __rkisp1_ext_params_config(params, cfg,
+	return __rkisp1_ext_params_config(params, buffer,
 					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
 }
 
@@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
 		rkisp1_isp_isr_lsc_config(params, cfg);
 		rkisp1_isp_isr_meas_config(params, cfg);
 	} else {
-		ret = rkisp1_ext_params_config(params, cfg);
+		ret = rkisp1_ext_params_config(params, buf);
 	}
 
 	if (ret)
@@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
 		rkisp1_isp_isr_other_config(params, cfg);
 		rkisp1_isp_isr_meas_config(params, cfg);
 	} else {
-		ret = rkisp1_ext_params_other_meas_config(params, cfg);
+		ret = rkisp1_ext_params_other_meas_config(params, buf);
 	}
 
 	if (ret) {
@@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
 	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
 		rkisp1_isp_isr_lsc_config(params, cfg);
 	else
-		ret = rkisp1_ext_params_lsc_config(params, cfg);
+		ret = rkisp1_ext_params_lsc_config(params, buf);
 
 	if (ret)
 		goto complete_and_unlock;
@@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
 	return 0;
 }
 
+static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct rkisp1_params_buffer *params_buf =
+		container_of(vbuf, struct rkisp1_params_buffer, vb);
+	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
+	size_t buf_size = params->metafmt.buffersize;
+
+	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
+		params_buf->cfg = NULL;
+		return 0;
+	}
+
+	params_buf->cfg = kvmalloc(buf_size, GFP_KERNEL);
+	if (IS_ERR_OR_NULL(params_buf->cfg))
+		return -ENOMEM;
+
+	return 0;
+}
+
+static void rkisp1_params_vb2_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct rkisp1_params_buffer *params_buf =
+		container_of(vbuf, struct rkisp1_params_buffer, vb);
+
+	kvfree(params_buf->cfg);
+}
+
+static int rkisp1_params_validate_ext_params(struct rkisp1_params *params,
+					     struct rkisp1_ext_params_cfg *cfg)
+{
+	size_t block_offset = 0;
+
+	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
+		dev_dbg(params->rkisp1->dev,
+			"Invalid parameters buffer size %llu\n",
+			cfg->total_size);
+		return -EINVAL;
+	}
+
+	/* Walk the list of parameter blocks and validate them. */
+	while (block_offset < cfg->total_size) {
+		const struct rkisp1_ext_params_handler *hdlr;
+		struct rkisp1_ext_params_block_header *block;
+
+		block = (struct rkisp1_ext_params_block_header *)
+			&cfg->data[block_offset];
+		block_offset += block->size;
+
+		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
+			dev_dbg(params->rkisp1->dev,
+				"Invalid parameters block type\n");
+			return -EINVAL;
+		}
+
+		hdlr = &rkisp1_ext_params_handlers[block->type];
+		if (hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS &&
+		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC &&
+		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS) {
+			dev_dbg(params->rkisp1->dev,
+				"Invalid parameters block group\n");
+			return -EINVAL;
+		}
+
+		if (block->size != hdlr->size) {
+			dev_dbg(params->rkisp1->dev,
+				"Invalid parameters block size\n");
+			return -EINVAL;
+		}
+	}
+
+	return 0;
+}
+
+static int rkisp1_params_vb2_buf_out_validate(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct rkisp1_params_buffer *params_buf =
+		container_of(vbuf, struct rkisp1_params_buffer, vb);
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct rkisp1_params *params = vq->drv_priv;
+	struct rkisp1_ext_params_cfg *cfg =
+		vb2_plane_vaddr(&params_buf->vb.vb2_buf, 0);
+	int ret;
+
+	/* Fixed parameters format doesn't require validation. */
+	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
+		return 0;
+
+	ret = rkisp1_params_validate_ext_params(params, cfg);
+	if (ret)
+		return ret;
+
+	/*
+	 * If the parameters buffer is valid, copy it to the internal scratch
+	 * buffer to avoid userspace modifying the buffer content while
+	 * the driver processes it.
+	 */
+	memcpy(params_buf->cfg, cfg, sizeof(*cfg));
+
+	return 0;
+}
+
 static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
 {
 	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
@@ -2455,6 +2546,9 @@ static void rkisp1_params_vb2_stop_streaming(struct vb2_queue *vq)
 
 static const struct vb2_ops rkisp1_params_vb2_ops = {
 	.queue_setup = rkisp1_params_vb2_queue_setup,
+	.buf_init = rkisp1_params_vb2_buf_init,
+	.buf_cleanup = rkisp1_params_vb2_buf_cleanup,
+	.buf_out_validate = rkisp1_params_vb2_buf_out_validate,
 	.wait_prepare = vb2_ops_wait_prepare,
 	.wait_finish = vb2_ops_wait_finish,
 	.buf_queue = rkisp1_params_vb2_buf_queue,
-- 
2.45.1


^ permalink raw reply related	[flat|nested] 38+ messages in thread

* Re: [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-05 16:54 ` [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer Jacopo Mondi
@ 2024-06-12 14:28   ` Dan Scally
  2024-06-12 16:20     ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Dan Scally @ 2024-06-12 14:28 UTC (permalink / raw)
  To: Jacopo Mondi, Linux Media Mailing List
  Cc: Laurent Pinchart, Sakari Ailus, Hans Verkuil, Stefan Klug,
	Paul Elder, Kieran Bingham, Umang Jain, Dafna Hirschfeld,
	Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo. As mentioned in the last patch, I think that this could be squashed into 5/8, but a 
couple of comments below

On 05/06/2024 17:54, Jacopo Mondi wrote:
> With the introduction of the extensible parameters format support in the
> rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
> by parsing the content of a data buffer of variable size provided by
> userspace through the V4L2 meta-output interface using the MMAP memory
> handling mode.
>
> As the parameters buffer is mapped in the userspace process memory,
> applications have access to the buffer content while the driver
> parses it.
>
> To prevent potential issues during the parameters buffer parsing and
> processing in the driver, implement three vb2_ops to
>
> 1) allocate a scratch buffer in the driver private buffer structure
> 2) validate the buffer content at VIDIOC_QBUF time
> 3) copy the content of the user provided configuration parameters
>     in the driver-private scratch buffer
>
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---
>   .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
>   1 file changed, 124 insertions(+), 30 deletions(-)
>
> diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> index 4adaf084ce6e..003239e14511 100644
> --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> @@ -5,6 +5,8 @@
>    * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
>    */
>   
> +#include <linux/string.h>
> +
>   #include <media/v4l2-common.h>
>   #include <media/v4l2-event.h>
>   #include <media/v4l2-ioctl.h>
> @@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
>   };
>   
>   static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> -				      struct rkisp1_ext_params_cfg *cfg,
> +				      struct rkisp1_params_buffer *buffer,
>   				      u32 block_group_mask)
>   {
> +	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;
>   	size_t block_offset = 0;
>   
> -	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> -		dev_dbg(params->rkisp1->dev,
> -			"Invalid parameters buffer size %llu\n",
> -			cfg->total_size);
> -		return -EINVAL;
> -	}
> +	if (WARN_ON(!cfg))
> +		return -ENOMEM;
>   
>   	/* Walk the list of parameter blocks and process them. */
>   	while (block_offset < cfg->total_size) {
> @@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
>   		block_offset += block->size;
>   
>   		/*
> -		 * Validate the block id and make sure the block group is in
> -		 * the list of groups to configure.
> +		 * Make sure the block group is in  the list of groups to
> +		 * configure.
>   		 */
> -		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> -			dev_dbg(params->rkisp1->dev,
> -				"Invalid parameters block type\n");
> -			return -EINVAL;
> -		}
> -
>   		block_handler = &rkisp1_ext_params_handlers[block->type];
>   		if (!(block_handler->group & block_group_mask))
>   			continue;
>   
> -		if (block->size != block_handler->size) {
> -			dev_dbg(params->rkisp1->dev,
> -				"Invalid parameters block size\n");
> -			return -EINVAL;
> -		}
> -
>   		block_handler->handler(params, block);
>   	}
>   
> @@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
>   }
>   
>   static int rkisp1_ext_params_config(struct rkisp1_params *params,
> -				    struct rkisp1_ext_params_cfg *cfg)
> +				    struct rkisp1_params_buffer *buffer)
>   {
> -	return __rkisp1_ext_params_config(params, cfg,
> +	return __rkisp1_ext_params_config(params, buffer,
>   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
>   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
>   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> @@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
>   
>   static int
>   rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> -				    struct rkisp1_ext_params_cfg *cfg)
> +				    struct rkisp1_params_buffer *buffer)
>   {
> -	return __rkisp1_ext_params_config(params, cfg,
> +	return __rkisp1_ext_params_config(params, buffer,
>   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
>   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
>   }
>   
>   static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> -					struct rkisp1_ext_params_cfg *cfg)
> +					struct rkisp1_params_buffer *buffer)
>   {
> -	return __rkisp1_ext_params_config(params, cfg,
> +	return __rkisp1_ext_params_config(params, buffer,
>   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
>   }
>   
> @@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
>   		rkisp1_isp_isr_lsc_config(params, cfg);
>   		rkisp1_isp_isr_meas_config(params, cfg);
>   	} else {
> -		ret = rkisp1_ext_params_config(params, cfg);
> +		ret = rkisp1_ext_params_config(params, buf);
>   	}
>   
>   	if (ret)
> @@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
>   		rkisp1_isp_isr_other_config(params, cfg);
>   		rkisp1_isp_isr_meas_config(params, cfg);
>   	} else {
> -		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> +		ret = rkisp1_ext_params_other_meas_config(params, buf);
>   	}
>   
>   	if (ret) {
> @@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
>   	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
>   		rkisp1_isp_isr_lsc_config(params, cfg);
>   	else
> -		ret = rkisp1_ext_params_lsc_config(params, cfg);
> +		ret = rkisp1_ext_params_lsc_config(params, buf);
>   
>   	if (ret)
>   		goto complete_and_unlock;
> @@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
>   	return 0;
>   }
>   
> +static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct rkisp1_params_buffer *params_buf =
> +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> +	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
> +	size_t buf_size = params->metafmt.buffersize;
> +
> +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> +		params_buf->cfg = NULL;
> +		return 0;
> +	}
> +
> +	params_buf->cfg = kvmalloc(buf_size, GFP_KERNEL);
> +	if (IS_ERR_OR_NULL(params_buf->cfg))
> +		return -ENOMEM;
> +
> +	return 0;
> +}
> +
> +static void rkisp1_params_vb2_buf_cleanup(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct rkisp1_params_buffer *params_buf =
> +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> +
> +	kvfree(params_buf->cfg);
> +}
> +
> +static int rkisp1_params_validate_ext_params(struct rkisp1_params *params,
> +					     struct rkisp1_ext_params_cfg *cfg)
> +{
> +	size_t block_offset = 0;
> +
> +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> +		dev_dbg(params->rkisp1->dev,
> +			"Invalid parameters buffer size %llu\n",
> +			cfg->total_size);
> +		return -EINVAL;
> +	}
> +
> +	/* Walk the list of parameter blocks and validate them. */
> +	while (block_offset < cfg->total_size) {
> +		const struct rkisp1_ext_params_handler *hdlr;
> +		struct rkisp1_ext_params_block_header *block;
> +
> +		block = (struct rkisp1_ext_params_block_header *)
> +			&cfg->data[block_offset];
> +		block_offset += block->size;
> +
> +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> +			dev_dbg(params->rkisp1->dev,
> +				"Invalid parameters block type\n");
> +			return -EINVAL;
> +		}
> +
> +		hdlr = &rkisp1_ext_params_handlers[block->type];
> +		if (hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS &&
> +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC &&
> +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS) {
> +			dev_dbg(params->rkisp1->dev,
> +				"Invalid parameters block group\n");
> +			return -EINVAL;
> +		}
I think this check can probably be dropped; those values are from the kernel driver rather than 
userspace inputs.
> +
> +		if (block->size != hdlr->size) {
> +			dev_dbg(params->rkisp1->dev,
> +				"Invalid parameters block size\n");
> +			return -EINVAL;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int rkisp1_params_vb2_buf_out_validate(struct vb2_buffer *vb)
> +{
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct rkisp1_params_buffer *params_buf =
> +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> +	struct vb2_queue *vq = vb->vb2_queue;
> +	struct rkisp1_params *params = vq->drv_priv;
> +	struct rkisp1_ext_params_cfg *cfg =
> +		vb2_plane_vaddr(&params_buf->vb.vb2_buf, 0);
> +	int ret;
> +
> +	/* Fixed parameters format doesn't require validation. */
> +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> +		return 0;
> +
> +	ret = rkisp1_params_validate_ext_params(params, cfg);
> +	if (ret)
> +		return ret;
> +
> +	/*
> +	 * If the parameters buffer is valid, copy it to the internal scratch
> +	 * buffer to avoid userspace modifying the buffer content while
> +	 * the driver processes it.
> +	 */
> +	memcpy(params_buf->cfg, cfg, sizeof(*cfg));
I think that this part is something that we probably ought to handle in vb2 core if we can, since 
the problem it's fixing isn't specific to the extensible parameters format or even the rkisp1 itself 
(unless I'm missing something). That doesn't have to block this set though, we can change this over 
to a vb2-core implementation when that's done.
> +
> +	return 0;
> +}
> +
>   static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
>   {
>   	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> @@ -2455,6 +2546,9 @@ static void rkisp1_params_vb2_stop_streaming(struct vb2_queue *vq)
>   
>   static const struct vb2_ops rkisp1_params_vb2_ops = {
>   	.queue_setup = rkisp1_params_vb2_queue_setup,
> +	.buf_init = rkisp1_params_vb2_buf_init,
> +	.buf_cleanup = rkisp1_params_vb2_buf_cleanup,
> +	.buf_out_validate = rkisp1_params_vb2_buf_out_validate,
>   	.wait_prepare = vb2_ops_wait_prepare,
>   	.wait_finish = vb2_ops_wait_finish,
>   	.buf_queue = rkisp1_params_vb2_buf_queue,

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-12 14:28   ` Dan Scally
@ 2024-06-12 16:20     ` Laurent Pinchart
  2024-06-19 14:22       ` Jacopo Mondi
  0 siblings, 1 reply; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-12 16:20 UTC (permalink / raw)
  To: Dan Scally
  Cc: Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 12, 2024 at 03:28:05PM +0100, Daniel Scally wrote:
> Hi Jacopo. As mentioned in the last patch, I think that this could be
> squashed into 5/8, but a couple of comments below

I think it should be moved earlier, yes, probably even before the
introduction of extended parameters.

> On 05/06/2024 17:54, Jacopo Mondi wrote:
> > With the introduction of the extensible parameters format support in the
> > rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
> > by parsing the content of a data buffer of variable size provided by
> > userspace through the V4L2 meta-output interface using the MMAP memory
> > handling mode.
> >
> > As the parameters buffer is mapped in the userspace process memory,
> > applications have access to the buffer content while the driver
> > parses it.
> >
> > To prevent potential issues during the parameters buffer parsing and
> > processing in the driver, implement three vb2_ops to
> >
> > 1) allocate a scratch buffer in the driver private buffer structure
> > 2) validate the buffer content at VIDIOC_QBUF time
> > 3) copy the content of the user provided configuration parameters
> >     in the driver-private scratch buffer
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
> >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
> >   1 file changed, 124 insertions(+), 30 deletions(-)
> >
> > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > index 4adaf084ce6e..003239e14511 100644
> > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > @@ -5,6 +5,8 @@
> >    * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
> >    */
> >   
> > +#include <linux/string.h>
> > +
> >   #include <media/v4l2-common.h>
> >   #include <media/v4l2-event.h>
> >   #include <media/v4l2-ioctl.h>
> > @@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
> >   };
> >   
> >   static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > -				      struct rkisp1_ext_params_cfg *cfg,
> > +				      struct rkisp1_params_buffer *buffer,
> >   				      u32 block_group_mask)
> >   {
> > +	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;

Maybe do this in the callers to avoid changing the prototype of this
function and the other functions below.

> >   	size_t block_offset = 0;
> >   
> > -	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > -		dev_dbg(params->rkisp1->dev,
> > -			"Invalid parameters buffer size %llu\n",
> > -			cfg->total_size);
> > -		return -EINVAL;
> > -	}
> > +	if (WARN_ON(!cfg))
> > +		return -ENOMEM;
> >   
> >   	/* Walk the list of parameter blocks and process them. */
> >   	while (block_offset < cfg->total_size) {
> > @@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> >   		block_offset += block->size;
> >   
> >   		/*
> > -		 * Validate the block id and make sure the block group is in
> > -		 * the list of groups to configure.
> > +		 * Make sure the block group is in  the list of groups to
> > +		 * configure.
> >   		 */
> > -		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > -			dev_dbg(params->rkisp1->dev,
> > -				"Invalid parameters block type\n");
> > -			return -EINVAL;
> > -		}
> > -
> >   		block_handler = &rkisp1_ext_params_handlers[block->type];
> >   		if (!(block_handler->group & block_group_mask))
> >   			continue;
> >   
> > -		if (block->size != block_handler->size) {
> > -			dev_dbg(params->rkisp1->dev,
> > -				"Invalid parameters block size\n");
> > -			return -EINVAL;
> > -		}
> > -
> >   		block_handler->handler(params, block);
> >   	}
> >   
> > @@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> >   }
> >   
> >   static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > -				    struct rkisp1_ext_params_cfg *cfg)
> > +				    struct rkisp1_params_buffer *buffer)
> >   {
> > -	return __rkisp1_ext_params_config(params, cfg,
> > +	return __rkisp1_ext_params_config(params, buffer,
> >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > @@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
> >   
> >   static int
> >   rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > -				    struct rkisp1_ext_params_cfg *cfg)
> > +				    struct rkisp1_params_buffer *buffer)
> >   {
> > -	return __rkisp1_ext_params_config(params, cfg,
> > +	return __rkisp1_ext_params_config(params, buffer,
> >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> >   }
> >   
> >   static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > -					struct rkisp1_ext_params_cfg *cfg)
> > +					struct rkisp1_params_buffer *buffer)
> >   {
> > -	return __rkisp1_ext_params_config(params, cfg,
> > +	return __rkisp1_ext_params_config(params, buffer,
> >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> >   }
> >   
> > @@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> >   		rkisp1_isp_isr_lsc_config(params, cfg);
> >   		rkisp1_isp_isr_meas_config(params, cfg);
> >   	} else {
> > -		ret = rkisp1_ext_params_config(params, cfg);
> > +		ret = rkisp1_ext_params_config(params, buf);
> >   	}
> >   
> >   	if (ret)
> > @@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
> >   		rkisp1_isp_isr_other_config(params, cfg);
> >   		rkisp1_isp_isr_meas_config(params, cfg);
> >   	} else {
> > -		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > +		ret = rkisp1_ext_params_other_meas_config(params, buf);
> >   	}
> >   
> >   	if (ret) {
> > @@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
> >   	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> >   		rkisp1_isp_isr_lsc_config(params, cfg);
> >   	else
> > -		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > +		ret = rkisp1_ext_params_lsc_config(params, buf);
> >   
> >   	if (ret)
> >   		goto complete_and_unlock;
> > @@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
> >   	return 0;
> >   }
> >   
> > +static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
> > +{
> > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > +	struct rkisp1_params_buffer *params_buf =
> > +		container_of(vbuf, struct rkisp1_params_buffer, vb);

A to_rkisp1_params_buffer() inline function in the previous patch would
be nice.

> > +	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
> > +	size_t buf_size = params->metafmt.buffersize;
> > +
> > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > +		params_buf->cfg = NULL;
> > +		return 0;
> > +	}

The problem is not restricted to the extensible format, how about doing
the same for the legacy format ?

> > +
> > +	params_buf->cfg = kvmalloc(buf_size, GFP_KERNEL);
> > +	if (IS_ERR_OR_NULL(params_buf->cfg))

Can kvmalloc() return an error pointer ?

> > +		return -ENOMEM;
> > +
> > +	return 0;
> > +}
> > +
> > +static void rkisp1_params_vb2_buf_cleanup(struct vb2_buffer *vb)
> > +{
> > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > +	struct rkisp1_params_buffer *params_buf =
> > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > +
> > +	kvfree(params_buf->cfg);

	params_buf->cfg = NULL;

to be safe.

> > +}
> > +
> > +static int rkisp1_params_validate_ext_params(struct rkisp1_params *params,
> > +					     struct rkisp1_ext_params_cfg *cfg)
> > +{
> > +	size_t block_offset = 0;
> > +
> > +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > +		dev_dbg(params->rkisp1->dev,
> > +			"Invalid parameters buffer size %llu\n",
> > +			cfg->total_size);
> > +		return -EINVAL;
> > +	}
> > +
> > +	/* Walk the list of parameter blocks and validate them. */
> > +	while (block_offset < cfg->total_size) {
> > +		const struct rkisp1_ext_params_handler *hdlr;
> > +		struct rkisp1_ext_params_block_header *block;
> > +
> > +		block = (struct rkisp1_ext_params_block_header *)
> > +			&cfg->data[block_offset];
> > +		block_offset += block->size;

Move this line to the end of the loop to avoid block_offset ever
pointing beyond the end of the buffer.

> > +
> > +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > +			dev_dbg(params->rkisp1->dev,
> > +				"Invalid parameters block type\n");
> > +			return -EINVAL;
> > +		}
> > +
> > +		hdlr = &rkisp1_ext_params_handlers[block->type];
> > +		if (hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS &&
> > +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC &&
> > +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS) {
> > +			dev_dbg(params->rkisp1->dev,
> > +				"Invalid parameters block group\n");
> > +			return -EINVAL;
> > +		}
>
> I think this check can probably be dropped; those values are from the
> kernel driver rather than userspace inputs.
>
> > +
> > +		if (block->size != hdlr->size) {
> > +			dev_dbg(params->rkisp1->dev,
> > +				"Invalid parameters block size\n");
> > +			return -EINVAL;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int rkisp1_params_vb2_buf_out_validate(struct vb2_buffer *vb)
> > +{
> > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > +	struct rkisp1_params_buffer *params_buf =
> > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > +	struct vb2_queue *vq = vb->vb2_queue;
> > +	struct rkisp1_params *params = vq->drv_priv;
> > +	struct rkisp1_ext_params_cfg *cfg =
> > +		vb2_plane_vaddr(&params_buf->vb.vb2_buf, 0);
> > +	int ret;
> > +
> > +	/* Fixed parameters format doesn't require validation. */
> > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > +		return 0;

You need to add a check to rkisp1_params_s_fmt_meta_out() to reject
format changes once the queue is busy, or you'll have bad surprises.

> > +
> > +	ret = rkisp1_params_validate_ext_params(params, cfg);
> > +	if (ret)
> > +		return ret;
> > +
> > +	/*
> > +	 * If the parameters buffer is valid, copy it to the internal scratch
> > +	 * buffer to avoid userspace modifying the buffer content while
> > +	 * the driver processes it.
> > +	 */

You need to swap the validation and memcpy(), otherwise userspace could
modify it after you have validated the contents and before the copy.

> > +	memcpy(params_buf->cfg, cfg, sizeof(*cfg));
>
> I think that this part is something that we probably ought to handle
> in vb2 core if we can, since the problem it's fixing isn't specific to
> the extensible parameters format or even the rkisp1 itself (unless I'm
> missing something). That doesn't have to block this set though, we can
> change this over to a vb2-core implementation when that's done.

I like the idea. I think we can experiment with it in rkisp1, and then
move it to vb2 when adding a second implementation (likely C55 ?).

> > +
> > +	return 0;
> > +}
> > +
> >   static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
> >   {
> >   	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > @@ -2455,6 +2546,9 @@ static void rkisp1_params_vb2_stop_streaming(struct vb2_queue *vq)
> >   
> >   static const struct vb2_ops rkisp1_params_vb2_ops = {
> >   	.queue_setup = rkisp1_params_vb2_queue_setup,
> > +	.buf_init = rkisp1_params_vb2_buf_init,
> > +	.buf_cleanup = rkisp1_params_vb2_buf_cleanup,
> > +	.buf_out_validate = rkisp1_params_vb2_buf_out_validate,
> >   	.wait_prepare = vb2_ops_wait_prepare,
> >   	.wait_finish = vb2_ops_wait_finish,
> >   	.buf_queue = rkisp1_params_vb2_buf_queue,

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-12 16:20     ` Laurent Pinchart
@ 2024-06-19 14:22       ` Jacopo Mondi
  2024-06-19 15:44         ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-19 14:22 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Dan Scally, Jacopo Mondi, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi
On Wed, Jun 12, 2024 at 07:20:48PM GMT, Laurent Pinchart wrote:
> On Wed, Jun 12, 2024 at 03:28:05PM +0100, Daniel Scally wrote:
> > Hi Jacopo. As mentioned in the last patch, I think that this could be
> > squashed into 5/8, but a couple of comments below
>
> I think it should be moved earlier, yes, probably even before the
> introduction of extended parameters.

I'm not sure I agree here

Squashing would create a patch that does too many things. Moving it
before the introduction of the extensible format would not make much
sense, as there would be nothing to validate/copy for the fixed
format.

I'll keep the ordering I have here if not a big deal.
>
> > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > With the introduction of the extensible parameters format support in the
> > > rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
> > > by parsing the content of a data buffer of variable size provided by
> > > userspace through the V4L2 meta-output interface using the MMAP memory
> > > handling mode.
> > >
> > > As the parameters buffer is mapped in the userspace process memory,
> > > applications have access to the buffer content while the driver
> > > parses it.
> > >
> > > To prevent potential issues during the parameters buffer parsing and
> > > processing in the driver, implement three vb2_ops to
> > >
> > > 1) allocate a scratch buffer in the driver private buffer structure
> > > 2) validate the buffer content at VIDIOC_QBUF time
> > > 3) copy the content of the user provided configuration parameters
> > >     in the driver-private scratch buffer
> > >
> > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > ---
> > >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
> > >   1 file changed, 124 insertions(+), 30 deletions(-)
> > >
> > > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > index 4adaf084ce6e..003239e14511 100644
> > > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > @@ -5,6 +5,8 @@
> > >    * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
> > >    */
> > >
> > > +#include <linux/string.h>
> > > +
> > >   #include <media/v4l2-common.h>
> > >   #include <media/v4l2-event.h>
> > >   #include <media/v4l2-ioctl.h>
> > > @@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
> > >   };
> > >
> > >   static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > -				      struct rkisp1_ext_params_cfg *cfg,
> > > +				      struct rkisp1_params_buffer *buffer,
> > >   				      u32 block_group_mask)
> > >   {
> > > +	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;
>
> Maybe do this in the callers to avoid changing the prototype of this
> function and the other functions below.
>
> > >   	size_t block_offset = 0;
> > >
> > > -	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > -		dev_dbg(params->rkisp1->dev,
> > > -			"Invalid parameters buffer size %llu\n",
> > > -			cfg->total_size);
> > > -		return -EINVAL;
> > > -	}
> > > +	if (WARN_ON(!cfg))
> > > +		return -ENOMEM;
> > >
> > >   	/* Walk the list of parameter blocks and process them. */
> > >   	while (block_offset < cfg->total_size) {
> > > @@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > >   		block_offset += block->size;
> > >
> > >   		/*
> > > -		 * Validate the block id and make sure the block group is in
> > > -		 * the list of groups to configure.
> > > +		 * Make sure the block group is in  the list of groups to
> > > +		 * configure.
> > >   		 */
> > > -		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > > -			dev_dbg(params->rkisp1->dev,
> > > -				"Invalid parameters block type\n");
> > > -			return -EINVAL;
> > > -		}
> > > -
> > >   		block_handler = &rkisp1_ext_params_handlers[block->type];
> > >   		if (!(block_handler->group & block_group_mask))
> > >   			continue;
> > >
> > > -		if (block->size != block_handler->size) {
> > > -			dev_dbg(params->rkisp1->dev,
> > > -				"Invalid parameters block size\n");
> > > -			return -EINVAL;
> > > -		}
> > > -
> > >   		block_handler->handler(params, block);
> > >   	}
> > >
> > > @@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > >   }
> > >
> > >   static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > +				    struct rkisp1_params_buffer *buffer)
> > >   {
> > > -	return __rkisp1_ext_params_config(params, cfg,
> > > +	return __rkisp1_ext_params_config(params, buffer,
> > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > @@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > >
> > >   static int
> > >   rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > +				    struct rkisp1_params_buffer *buffer)
> > >   {
> > > -	return __rkisp1_ext_params_config(params, cfg,
> > > +	return __rkisp1_ext_params_config(params, buffer,
> > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > >   }
> > >
> > >   static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > > -					struct rkisp1_ext_params_cfg *cfg)
> > > +					struct rkisp1_params_buffer *buffer)
> > >   {
> > > -	return __rkisp1_ext_params_config(params, cfg,
> > > +	return __rkisp1_ext_params_config(params, buffer,
> > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > >   }
> > >
> > > @@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > >   	} else {
> > > -		ret = rkisp1_ext_params_config(params, cfg);
> > > +		ret = rkisp1_ext_params_config(params, buf);
> > >   	}
> > >
> > >   	if (ret)
> > > @@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
> > >   		rkisp1_isp_isr_other_config(params, cfg);
> > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > >   	} else {
> > > -		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > > +		ret = rkisp1_ext_params_other_meas_config(params, buf);
> > >   	}
> > >
> > >   	if (ret) {
> > > @@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
> > >   	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > >   	else
> > > -		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > > +		ret = rkisp1_ext_params_lsc_config(params, buf);
> > >
> > >   	if (ret)
> > >   		goto complete_and_unlock;
> > > @@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
> > >   	return 0;
> > >   }
> > >
> > > +static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
> > > +{
> > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > +	struct rkisp1_params_buffer *params_buf =
> > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
>
> A to_rkisp1_params_buffer() inline function in the previous patch would
> be nice.
>
> > > +	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
> > > +	size_t buf_size = params->metafmt.buffersize;
> > > +
> > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > +		params_buf->cfg = NULL;
> > > +		return 0;
> > > +	}
>
> The problem is not restricted to the extensible format, how about doing
> the same for the legacy format ?
>

Copying the buffer in ? Not sure I agree. The extensible format allows
userspace to change the memory layout of the buffer, if this happens
while the driver parses it, we'll ooops. With the fixed format the
memory layout is fixed, if userspace changes the content of the buffer
(ie the configuration parameters) after qbuf, well, it's their
problem. Copying (and validation) is costly and if it serves to avoid
an ooops it's more than justified. In the other case I'm not sure.

> > > +
> > > +	params_buf->cfg = kvmalloc(buf_size, GFP_KERNEL);
> > > +	if (IS_ERR_OR_NULL(params_buf->cfg))
>
> Can kvmalloc() return an error pointer ?
>

I think I got a report somewhere about this but I can't find it
anymore :/

> > > +		return -ENOMEM;
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static void rkisp1_params_vb2_buf_cleanup(struct vb2_buffer *vb)
> > > +{
> > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > +	struct rkisp1_params_buffer *params_buf =
> > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > > +
> > > +	kvfree(params_buf->cfg);
>
> 	params_buf->cfg = NULL;
>
> to be safe.
>
> > > +}
> > > +
> > > +static int rkisp1_params_validate_ext_params(struct rkisp1_params *params,
> > > +					     struct rkisp1_ext_params_cfg *cfg)
> > > +{
> > > +	size_t block_offset = 0;
> > > +
> > > +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > +		dev_dbg(params->rkisp1->dev,
> > > +			"Invalid parameters buffer size %llu\n",
> > > +			cfg->total_size);
> > > +		return -EINVAL;
> > > +	}
> > > +
> > > +	/* Walk the list of parameter blocks and validate them. */
> > > +	while (block_offset < cfg->total_size) {
> > > +		const struct rkisp1_ext_params_handler *hdlr;
> > > +		struct rkisp1_ext_params_block_header *block;
> > > +
> > > +		block = (struct rkisp1_ext_params_block_header *)
> > > +			&cfg->data[block_offset];
> > > +		block_offset += block->size;
>
> Move this line to the end of the loop to avoid block_offset ever
> pointing beyond the end of the buffer.
>
> > > +
> > > +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > > +			dev_dbg(params->rkisp1->dev,
> > > +				"Invalid parameters block type\n");
> > > +			return -EINVAL;
> > > +		}
> > > +
> > > +		hdlr = &rkisp1_ext_params_handlers[block->type];
> > > +		if (hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS &&
> > > +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC &&
> > > +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS) {
> > > +			dev_dbg(params->rkisp1->dev,
> > > +				"Invalid parameters block group\n");
> > > +			return -EINVAL;
> > > +		}
> >
> > I think this check can probably be dropped; those values are from the
> > kernel driver rather than userspace inputs.
> >
> > > +
> > > +		if (block->size != hdlr->size) {
> > > +			dev_dbg(params->rkisp1->dev,
> > > +				"Invalid parameters block size\n");
> > > +			return -EINVAL;
> > > +		}
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +static int rkisp1_params_vb2_buf_out_validate(struct vb2_buffer *vb)
> > > +{
> > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > +	struct rkisp1_params_buffer *params_buf =
> > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > > +	struct vb2_queue *vq = vb->vb2_queue;
> > > +	struct rkisp1_params *params = vq->drv_priv;
> > > +	struct rkisp1_ext_params_cfg *cfg =
> > > +		vb2_plane_vaddr(&params_buf->vb.vb2_buf, 0);
> > > +	int ret;
> > > +
> > > +	/* Fixed parameters format doesn't require validation. */
> > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > +		return 0;
>
> You need to add a check to rkisp1_params_s_fmt_meta_out() to reject
> format changes once the queue is busy, or you'll have bad surprises.
>
> > > +
> > > +	ret = rkisp1_params_validate_ext_params(params, cfg);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	/*
> > > +	 * If the parameters buffer is valid, copy it to the internal scratch
> > > +	 * buffer to avoid userspace modifying the buffer content while
> > > +	 * the driver processes it.
> > > +	 */
>
> You need to swap the validation and memcpy(), otherwise userspace could
> modify it after you have validated the contents and before the copy.
>
> > > +	memcpy(params_buf->cfg, cfg, sizeof(*cfg));
> >
> > I think that this part is something that we probably ought to handle
> > in vb2 core if we can, since the problem it's fixing isn't specific to
> > the extensible parameters format or even the rkisp1 itself (unless I'm
> > missing something). That doesn't have to block this set though, we can
> > change this over to a vb2-core implementation when that's done.
>
> I like the idea. I think we can experiment with it in rkisp1, and then
> move it to vb2 when adding a second implementation (likely C55 ?).
>

Again, not sure. Copying and buffer allocations are costly. If we risk
an invalid memory access like in the extensible format case then I
agree it's worth it. In all other cases, if userspace changes a buffer
content after qbuf we can't do much about it (is it any different than
userspace modifying the content of an output video buffer after it has
been queued ?)

> > > +
> > > +	return 0;
> > > +}
> > > +
> > >   static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
> > >   {
> > >   	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > @@ -2455,6 +2546,9 @@ static void rkisp1_params_vb2_stop_streaming(struct vb2_queue *vq)
> > >
> > >   static const struct vb2_ops rkisp1_params_vb2_ops = {
> > >   	.queue_setup = rkisp1_params_vb2_queue_setup,
> > > +	.buf_init = rkisp1_params_vb2_buf_init,
> > > +	.buf_cleanup = rkisp1_params_vb2_buf_cleanup,
> > > +	.buf_out_validate = rkisp1_params_vb2_buf_out_validate,
> > >   	.wait_prepare = vb2_ops_wait_prepare,
> > >   	.wait_finish = vb2_ops_wait_finish,
> > >   	.buf_queue = rkisp1_params_vb2_buf_queue,
>
> --
> Regards,
>
> Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-19 14:22       ` Jacopo Mondi
@ 2024-06-19 15:44         ` Laurent Pinchart
  2024-06-19 15:55           ` Jacopo Mondi
  0 siblings, 1 reply; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-19 15:44 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Dan Scally, Linux Media Mailing List, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi Jacopo,

On Wed, Jun 19, 2024 at 04:22:00PM +0200, Jacopo Mondi wrote:
> On Wed, Jun 12, 2024 at 07:20:48PM GMT, Laurent Pinchart wrote:
> > On Wed, Jun 12, 2024 at 03:28:05PM +0100, Daniel Scally wrote:
> > > Hi Jacopo. As mentioned in the last patch, I think that this could be
> > > squashed into 5/8, but a couple of comments below
> >
> > I think it should be moved earlier, yes, probably even before the
> > introduction of extended parameters.
> 
> I'm not sure I agree here
> 
> Squashing would create a patch that does too many things. Moving it
> before the introduction of the extensible format would not make much
> sense, as there would be nothing to validate/copy for the fixed
> format.

I think the copy should be done for the fixed format too. See below.

> I'll keep the ordering I have here if not a big deal.
>
> > > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > > With the introduction of the extensible parameters format support in the
> > > > rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
> > > > by parsing the content of a data buffer of variable size provided by
> > > > userspace through the V4L2 meta-output interface using the MMAP memory
> > > > handling mode.
> > > >
> > > > As the parameters buffer is mapped in the userspace process memory,
> > > > applications have access to the buffer content while the driver
> > > > parses it.
> > > >
> > > > To prevent potential issues during the parameters buffer parsing and
> > > > processing in the driver, implement three vb2_ops to
> > > >
> > > > 1) allocate a scratch buffer in the driver private buffer structure
> > > > 2) validate the buffer content at VIDIOC_QBUF time
> > > > 3) copy the content of the user provided configuration parameters
> > > >     in the driver-private scratch buffer
> > > >
> > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > ---
> > > >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
> > > >   1 file changed, 124 insertions(+), 30 deletions(-)
> > > >
> > > > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > index 4adaf084ce6e..003239e14511 100644
> > > > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > @@ -5,6 +5,8 @@
> > > >    * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
> > > >    */
> > > >
> > > > +#include <linux/string.h>
> > > > +
> > > >   #include <media/v4l2-common.h>
> > > >   #include <media/v4l2-event.h>
> > > >   #include <media/v4l2-ioctl.h>
> > > > @@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
> > > >   };
> > > >
> > > >   static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > -				      struct rkisp1_ext_params_cfg *cfg,
> > > > +				      struct rkisp1_params_buffer *buffer,
> > > >   				      u32 block_group_mask)
> > > >   {
> > > > +	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;
> >
> > Maybe do this in the callers to avoid changing the prototype of this
> > function and the other functions below.
> >
> > > >   	size_t block_offset = 0;
> > > >
> > > > -	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > > -		dev_dbg(params->rkisp1->dev,
> > > > -			"Invalid parameters buffer size %llu\n",
> > > > -			cfg->total_size);
> > > > -		return -EINVAL;
> > > > -	}
> > > > +	if (WARN_ON(!cfg))
> > > > +		return -ENOMEM;
> > > >
> > > >   	/* Walk the list of parameter blocks and process them. */
> > > >   	while (block_offset < cfg->total_size) {
> > > > @@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > >   		block_offset += block->size;
> > > >
> > > >   		/*
> > > > -		 * Validate the block id and make sure the block group is in
> > > > -		 * the list of groups to configure.
> > > > +		 * Make sure the block group is in  the list of groups to
> > > > +		 * configure.
> > > >   		 */
> > > > -		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > > > -			dev_dbg(params->rkisp1->dev,
> > > > -				"Invalid parameters block type\n");
> > > > -			return -EINVAL;
> > > > -		}
> > > > -
> > > >   		block_handler = &rkisp1_ext_params_handlers[block->type];
> > > >   		if (!(block_handler->group & block_group_mask))
> > > >   			continue;
> > > >
> > > > -		if (block->size != block_handler->size) {
> > > > -			dev_dbg(params->rkisp1->dev,
> > > > -				"Invalid parameters block size\n");
> > > > -			return -EINVAL;
> > > > -		}
> > > > -
> > > >   		block_handler->handler(params, block);
> > > >   	}
> > > >
> > > > @@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > >   }
> > > >
> > > >   static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > +				    struct rkisp1_params_buffer *buffer)
> > > >   {
> > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > @@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > >
> > > >   static int
> > > >   rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > +				    struct rkisp1_params_buffer *buffer)
> > > >   {
> > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > >   }
> > > >
> > > >   static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > > > -					struct rkisp1_ext_params_cfg *cfg)
> > > > +					struct rkisp1_params_buffer *buffer)
> > > >   {
> > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > > >   }
> > > >
> > > > @@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > >   	} else {
> > > > -		ret = rkisp1_ext_params_config(params, cfg);
> > > > +		ret = rkisp1_ext_params_config(params, buf);
> > > >   	}
> > > >
> > > >   	if (ret)
> > > > @@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
> > > >   		rkisp1_isp_isr_other_config(params, cfg);
> > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > >   	} else {
> > > > -		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > > > +		ret = rkisp1_ext_params_other_meas_config(params, buf);
> > > >   	}
> > > >
> > > >   	if (ret) {
> > > > @@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
> > > >   	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > >   	else
> > > > -		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > > > +		ret = rkisp1_ext_params_lsc_config(params, buf);
> > > >
> > > >   	if (ret)
> > > >   		goto complete_and_unlock;
> > > > @@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
> > > >   	return 0;
> > > >   }
> > > >
> > > > +static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
> > > > +{
> > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > +	struct rkisp1_params_buffer *params_buf =
> > > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> >
> > A to_rkisp1_params_buffer() inline function in the previous patch would
> > be nice.
> >
> > > > +	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
> > > > +	size_t buf_size = params->metafmt.buffersize;
> > > > +
> > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > > +		params_buf->cfg = NULL;
> > > > +		return 0;
> > > > +	}
> >
> > The problem is not restricted to the extensible format, how about doing
> > the same for the legacy format ?
> 
> Copying the buffer in ? Not sure I agree. The extensible format allows
> userspace to change the memory layout of the buffer, if this happens
> while the driver parses it, we'll ooops. With the fixed format the
> memory layout is fixed, if userspace changes the content of the buffer
> (ie the configuration parameters) after qbuf, well, it's their
> problem.

Can you guarantee that the driver will never ever crash or corrupt
memory if that happens ? Especially if you take into account the fact
that the compiler and CPU can reorder accesses, as well as split and
merge accesses (i.e. reading the same memory location multiple times
when it's accessed once only in the code), that seems hard to do.

> Copying (and validation) is costly and if it serves to avoid
> an ooops it's more than justified. In the other case I'm not sure.

How expensive is the copy ?

> > > > +
> > > > +	params_buf->cfg = kvmalloc(buf_size, GFP_KERNEL);
> > > > +	if (IS_ERR_OR_NULL(params_buf->cfg))
> >
> > Can kvmalloc() return an error pointer ?
> 
> I think I got a report somewhere about this but I can't find it
> anymore :/
> 
> > > > +		return -ENOMEM;
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static void rkisp1_params_vb2_buf_cleanup(struct vb2_buffer *vb)
> > > > +{
> > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > +	struct rkisp1_params_buffer *params_buf =
> > > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > > > +
> > > > +	kvfree(params_buf->cfg);
> >
> > 	params_buf->cfg = NULL;
> >
> > to be safe.
> >
> > > > +}
> > > > +
> > > > +static int rkisp1_params_validate_ext_params(struct rkisp1_params *params,
> > > > +					     struct rkisp1_ext_params_cfg *cfg)
> > > > +{
> > > > +	size_t block_offset = 0;
> > > > +
> > > > +	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > > +		dev_dbg(params->rkisp1->dev,
> > > > +			"Invalid parameters buffer size %llu\n",
> > > > +			cfg->total_size);
> > > > +		return -EINVAL;
> > > > +	}
> > > > +
> > > > +	/* Walk the list of parameter blocks and validate them. */
> > > > +	while (block_offset < cfg->total_size) {
> > > > +		const struct rkisp1_ext_params_handler *hdlr;
> > > > +		struct rkisp1_ext_params_block_header *block;
> > > > +
> > > > +		block = (struct rkisp1_ext_params_block_header *)
> > > > +			&cfg->data[block_offset];
> > > > +		block_offset += block->size;
> >
> > Move this line to the end of the loop to avoid block_offset ever
> > pointing beyond the end of the buffer.
> >
> > > > +
> > > > +		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > > > +			dev_dbg(params->rkisp1->dev,
> > > > +				"Invalid parameters block type\n");
> > > > +			return -EINVAL;
> > > > +		}
> > > > +
> > > > +		hdlr = &rkisp1_ext_params_handlers[block->type];
> > > > +		if (hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS &&
> > > > +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC &&
> > > > +		    hdlr->group != RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS) {
> > > > +			dev_dbg(params->rkisp1->dev,
> > > > +				"Invalid parameters block group\n");
> > > > +			return -EINVAL;
> > > > +		}
> > >
> > > I think this check can probably be dropped; those values are from the
> > > kernel driver rather than userspace inputs.
> > >
> > > > +
> > > > +		if (block->size != hdlr->size) {
> > > > +			dev_dbg(params->rkisp1->dev,
> > > > +				"Invalid parameters block size\n");
> > > > +			return -EINVAL;
> > > > +		}
> > > > +	}
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static int rkisp1_params_vb2_buf_out_validate(struct vb2_buffer *vb)
> > > > +{
> > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > +	struct rkisp1_params_buffer *params_buf =
> > > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > > > +	struct vb2_queue *vq = vb->vb2_queue;
> > > > +	struct rkisp1_params *params = vq->drv_priv;
> > > > +	struct rkisp1_ext_params_cfg *cfg =
> > > > +		vb2_plane_vaddr(&params_buf->vb.vb2_buf, 0);
> > > > +	int ret;
> > > > +
> > > > +	/* Fixed parameters format doesn't require validation. */
> > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > > +		return 0;
> >
> > You need to add a check to rkisp1_params_s_fmt_meta_out() to reject
> > format changes once the queue is busy, or you'll have bad surprises.
> >
> > > > +
> > > > +	ret = rkisp1_params_validate_ext_params(params, cfg);
> > > > +	if (ret)
> > > > +		return ret;
> > > > +
> > > > +	/*
> > > > +	 * If the parameters buffer is valid, copy it to the internal scratch
> > > > +	 * buffer to avoid userspace modifying the buffer content while
> > > > +	 * the driver processes it.
> > > > +	 */
> >
> > You need to swap the validation and memcpy(), otherwise userspace could
> > modify it after you have validated the contents and before the copy.
> >
> > > > +	memcpy(params_buf->cfg, cfg, sizeof(*cfg));
> > >
> > > I think that this part is something that we probably ought to handle
> > > in vb2 core if we can, since the problem it's fixing isn't specific to
> > > the extensible parameters format or even the rkisp1 itself (unless I'm
> > > missing something). That doesn't have to block this set though, we can
> > > change this over to a vb2-core implementation when that's done.
> >
> > I like the idea. I think we can experiment with it in rkisp1, and then
> > move it to vb2 when adding a second implementation (likely C55 ?).
> 
> Again, not sure. Copying and buffer allocations are costly. If we risk

I'm not saying we should do the copy unconditionally in vb2, but vb2
should provide the feature for drivers that want to use it.

> an invalid memory access like in the extensible format case then I
> agree it's worth it. In all other cases, if userspace changes a buffer
> content after qbuf we can't do much about it (is it any different than
> userspace modifying the content of an output video buffer after it has
> been queued ?)

It is. If you change the contents of a video buffer, you'll have corrupt
frames in the worst case. If you change the contents of memory used by
the CPU to influence kernel code flows, it's an entirely different
story.

> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > >   static void rkisp1_params_vb2_buf_queue(struct vb2_buffer *vb)
> > > >   {
> > > >   	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > @@ -2455,6 +2546,9 @@ static void rkisp1_params_vb2_stop_streaming(struct vb2_queue *vq)
> > > >
> > > >   static const struct vb2_ops rkisp1_params_vb2_ops = {
> > > >   	.queue_setup = rkisp1_params_vb2_queue_setup,
> > > > +	.buf_init = rkisp1_params_vb2_buf_init,
> > > > +	.buf_cleanup = rkisp1_params_vb2_buf_cleanup,
> > > > +	.buf_out_validate = rkisp1_params_vb2_buf_out_validate,
> > > >   	.wait_prepare = vb2_ops_wait_prepare,
> > > >   	.wait_finish = vb2_ops_wait_finish,
> > > >   	.buf_queue = rkisp1_params_vb2_buf_queue,

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-19 15:44         ` Laurent Pinchart
@ 2024-06-19 15:55           ` Jacopo Mondi
  2024-06-19 16:11             ` Laurent Pinchart
  0 siblings, 1 reply; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-19 15:55 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Jacopo Mondi, Dan Scally, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi Laurent

On Wed, Jun 19, 2024 at 06:44:20PM GMT, Laurent Pinchart wrote:
> Hi Jacopo,
>
> On Wed, Jun 19, 2024 at 04:22:00PM +0200, Jacopo Mondi wrote:
> > On Wed, Jun 12, 2024 at 07:20:48PM GMT, Laurent Pinchart wrote:
> > > On Wed, Jun 12, 2024 at 03:28:05PM +0100, Daniel Scally wrote:
> > > > Hi Jacopo. As mentioned in the last patch, I think that this could be
> > > > squashed into 5/8, but a couple of comments below
> > >
> > > I think it should be moved earlier, yes, probably even before the
> > > introduction of extended parameters.
> >
> > I'm not sure I agree here
> >
> > Squashing would create a patch that does too many things. Moving it
> > before the introduction of the extensible format would not make much
> > sense, as there would be nothing to validate/copy for the fixed
> > format.
>
> I think the copy should be done for the fixed format too. See below.
>
> > I'll keep the ordering I have here if not a big deal.
> >
> > > > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > > > With the introduction of the extensible parameters format support in the
> > > > > rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
> > > > > by parsing the content of a data buffer of variable size provided by
> > > > > userspace through the V4L2 meta-output interface using the MMAP memory
> > > > > handling mode.
> > > > >
> > > > > As the parameters buffer is mapped in the userspace process memory,
> > > > > applications have access to the buffer content while the driver
> > > > > parses it.
> > > > >
> > > > > To prevent potential issues during the parameters buffer parsing and
> > > > > processing in the driver, implement three vb2_ops to
> > > > >
> > > > > 1) allocate a scratch buffer in the driver private buffer structure
> > > > > 2) validate the buffer content at VIDIOC_QBUF time
> > > > > 3) copy the content of the user provided configuration parameters
> > > > >     in the driver-private scratch buffer
> > > > >
> > > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > > ---
> > > > >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
> > > > >   1 file changed, 124 insertions(+), 30 deletions(-)
> > > > >
> > > > > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > index 4adaf084ce6e..003239e14511 100644
> > > > > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > @@ -5,6 +5,8 @@
> > > > >    * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
> > > > >    */
> > > > >
> > > > > +#include <linux/string.h>
> > > > > +
> > > > >   #include <media/v4l2-common.h>
> > > > >   #include <media/v4l2-event.h>
> > > > >   #include <media/v4l2-ioctl.h>
> > > > > @@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
> > > > >   };
> > > > >
> > > > >   static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > -				      struct rkisp1_ext_params_cfg *cfg,
> > > > > +				      struct rkisp1_params_buffer *buffer,
> > > > >   				      u32 block_group_mask)
> > > > >   {
> > > > > +	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;
> > >
> > > Maybe do this in the callers to avoid changing the prototype of this
> > > function and the other functions below.
> > >
> > > > >   	size_t block_offset = 0;
> > > > >
> > > > > -	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > > > -		dev_dbg(params->rkisp1->dev,
> > > > > -			"Invalid parameters buffer size %llu\n",
> > > > > -			cfg->total_size);
> > > > > -		return -EINVAL;
> > > > > -	}
> > > > > +	if (WARN_ON(!cfg))
> > > > > +		return -ENOMEM;
> > > > >
> > > > >   	/* Walk the list of parameter blocks and process them. */
> > > > >   	while (block_offset < cfg->total_size) {
> > > > > @@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > >   		block_offset += block->size;
> > > > >
> > > > >   		/*
> > > > > -		 * Validate the block id and make sure the block group is in
> > > > > -		 * the list of groups to configure.
> > > > > +		 * Make sure the block group is in  the list of groups to
> > > > > +		 * configure.
> > > > >   		 */
> > > > > -		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > > > > -			dev_dbg(params->rkisp1->dev,
> > > > > -				"Invalid parameters block type\n");
> > > > > -			return -EINVAL;
> > > > > -		}
> > > > > -
> > > > >   		block_handler = &rkisp1_ext_params_handlers[block->type];
> > > > >   		if (!(block_handler->group & block_group_mask))
> > > > >   			continue;
> > > > >
> > > > > -		if (block->size != block_handler->size) {
> > > > > -			dev_dbg(params->rkisp1->dev,
> > > > > -				"Invalid parameters block size\n");
> > > > > -			return -EINVAL;
> > > > > -		}
> > > > > -
> > > > >   		block_handler->handler(params, block);
> > > > >   	}
> > > > >
> > > > > @@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > >   }
> > > > >
> > > > >   static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > > +				    struct rkisp1_params_buffer *buffer)
> > > > >   {
> > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > > @@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > >
> > > > >   static int
> > > > >   rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > > +				    struct rkisp1_params_buffer *buffer)
> > > > >   {
> > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > >   }
> > > > >
> > > > >   static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > > > > -					struct rkisp1_ext_params_cfg *cfg)
> > > > > +					struct rkisp1_params_buffer *buffer)
> > > > >   {
> > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > > > >   }
> > > > >
> > > > > @@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > > >   	} else {
> > > > > -		ret = rkisp1_ext_params_config(params, cfg);
> > > > > +		ret = rkisp1_ext_params_config(params, buf);
> > > > >   	}
> > > > >
> > > > >   	if (ret)
> > > > > @@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
> > > > >   		rkisp1_isp_isr_other_config(params, cfg);
> > > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > > >   	} else {
> > > > > -		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > > > > +		ret = rkisp1_ext_params_other_meas_config(params, buf);
> > > > >   	}
> > > > >
> > > > >   	if (ret) {
> > > > > @@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
> > > > >   	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > >   	else
> > > > > -		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > > > > +		ret = rkisp1_ext_params_lsc_config(params, buf);
> > > > >
> > > > >   	if (ret)
> > > > >   		goto complete_and_unlock;
> > > > > @@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
> > > > >   	return 0;
> > > > >   }
> > > > >
> > > > > +static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
> > > > > +{
> > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > +	struct rkisp1_params_buffer *params_buf =
> > > > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > >
> > > A to_rkisp1_params_buffer() inline function in the previous patch would
> > > be nice.
> > >
> > > > > +	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
> > > > > +	size_t buf_size = params->metafmt.buffersize;
> > > > > +
> > > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > > > +		params_buf->cfg = NULL;
> > > > > +		return 0;
> > > > > +	}
> > >
> > > The problem is not restricted to the extensible format, how about doing
> > > the same for the legacy format ?
> >
> > Copying the buffer in ? Not sure I agree. The extensible format allows
> > userspace to change the memory layout of the buffer, if this happens
> > while the driver parses it, we'll ooops. With the fixed format the
> > memory layout is fixed, if userspace changes the content of the buffer
> > (ie the configuration parameters) after qbuf, well, it's their
> > problem.
>
> Can you guarantee that the driver will never ever crash or corrupt
> memory if that happens ? Especially if you take into account the fact

All of the mainline driver work in this way today. But ok if most
people think it's worth it, I'll do so and re-order the patches.

> that the compiler and CPU can reorder accesses, as well as split and
> merge accesses (i.e. reading the same memory location multiple times
> when it's accessed once only in the code), that seems hard to do.
>
> > Copying (and validation) is costly and if it serves to avoid
> > an ooops it's more than justified. In the other case I'm not sure.
>
> How expensive is the copy ?
>

For sure it doubles the allocated memory as we've one backing memory
buffer for each buffer requested by userspace.

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-19 15:55           ` Jacopo Mondi
@ 2024-06-19 16:11             ` Laurent Pinchart
  2024-06-19 16:20               ` Jacopo Mondi
  0 siblings, 1 reply; 38+ messages in thread
From: Laurent Pinchart @ 2024-06-19 16:11 UTC (permalink / raw)
  To: Jacopo Mondi
  Cc: Dan Scally, Linux Media Mailing List, Sakari Ailus, Hans Verkuil,
	Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

On Wed, Jun 19, 2024 at 05:55:12PM +0200, Jacopo Mondi wrote:
> Hi Laurent
> 
> On Wed, Jun 19, 2024 at 06:44:20PM GMT, Laurent Pinchart wrote:
> > Hi Jacopo,
> >
> > On Wed, Jun 19, 2024 at 04:22:00PM +0200, Jacopo Mondi wrote:
> > > On Wed, Jun 12, 2024 at 07:20:48PM GMT, Laurent Pinchart wrote:
> > > > On Wed, Jun 12, 2024 at 03:28:05PM +0100, Daniel Scally wrote:
> > > > > Hi Jacopo. As mentioned in the last patch, I think that this could be
> > > > > squashed into 5/8, but a couple of comments below
> > > >
> > > > I think it should be moved earlier, yes, probably even before the
> > > > introduction of extended parameters.
> > >
> > > I'm not sure I agree here
> > >
> > > Squashing would create a patch that does too many things. Moving it
> > > before the introduction of the extensible format would not make much
> > > sense, as there would be nothing to validate/copy for the fixed
> > > format.
> >
> > I think the copy should be done for the fixed format too. See below.
> >
> > > I'll keep the ordering I have here if not a big deal.
> > >
> > > > > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > > > > With the introduction of the extensible parameters format support in the
> > > > > > rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
> > > > > > by parsing the content of a data buffer of variable size provided by
> > > > > > userspace through the V4L2 meta-output interface using the MMAP memory
> > > > > > handling mode.
> > > > > >
> > > > > > As the parameters buffer is mapped in the userspace process memory,
> > > > > > applications have access to the buffer content while the driver
> > > > > > parses it.
> > > > > >
> > > > > > To prevent potential issues during the parameters buffer parsing and
> > > > > > processing in the driver, implement three vb2_ops to
> > > > > >
> > > > > > 1) allocate a scratch buffer in the driver private buffer structure
> > > > > > 2) validate the buffer content at VIDIOC_QBUF time
> > > > > > 3) copy the content of the user provided configuration parameters
> > > > > >     in the driver-private scratch buffer
> > > > > >
> > > > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > > > ---
> > > > > >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
> > > > > >   1 file changed, 124 insertions(+), 30 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > > index 4adaf084ce6e..003239e14511 100644
> > > > > > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > > @@ -5,6 +5,8 @@
> > > > > >    * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
> > > > > >    */
> > > > > >
> > > > > > +#include <linux/string.h>
> > > > > > +
> > > > > >   #include <media/v4l2-common.h>
> > > > > >   #include <media/v4l2-event.h>
> > > > > >   #include <media/v4l2-ioctl.h>
> > > > > > @@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
> > > > > >   };
> > > > > >
> > > > > >   static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > > -				      struct rkisp1_ext_params_cfg *cfg,
> > > > > > +				      struct rkisp1_params_buffer *buffer,
> > > > > >   				      u32 block_group_mask)
> > > > > >   {
> > > > > > +	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;
> > > >
> > > > Maybe do this in the callers to avoid changing the prototype of this
> > > > function and the other functions below.
> > > >
> > > > > >   	size_t block_offset = 0;
> > > > > >
> > > > > > -	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > > > > -		dev_dbg(params->rkisp1->dev,
> > > > > > -			"Invalid parameters buffer size %llu\n",
> > > > > > -			cfg->total_size);
> > > > > > -		return -EINVAL;
> > > > > > -	}
> > > > > > +	if (WARN_ON(!cfg))
> > > > > > +		return -ENOMEM;
> > > > > >
> > > > > >   	/* Walk the list of parameter blocks and process them. */
> > > > > >   	while (block_offset < cfg->total_size) {
> > > > > > @@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > >   		block_offset += block->size;
> > > > > >
> > > > > >   		/*
> > > > > > -		 * Validate the block id and make sure the block group is in
> > > > > > -		 * the list of groups to configure.
> > > > > > +		 * Make sure the block group is in  the list of groups to
> > > > > > +		 * configure.
> > > > > >   		 */
> > > > > > -		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > > > > > -			dev_dbg(params->rkisp1->dev,
> > > > > > -				"Invalid parameters block type\n");
> > > > > > -			return -EINVAL;
> > > > > > -		}
> > > > > > -
> > > > > >   		block_handler = &rkisp1_ext_params_handlers[block->type];
> > > > > >   		if (!(block_handler->group & block_group_mask))
> > > > > >   			continue;
> > > > > >
> > > > > > -		if (block->size != block_handler->size) {
> > > > > > -			dev_dbg(params->rkisp1->dev,
> > > > > > -				"Invalid parameters block size\n");
> > > > > > -			return -EINVAL;
> > > > > > -		}
> > > > > > -
> > > > > >   		block_handler->handler(params, block);
> > > > > >   	}
> > > > > >
> > > > > > @@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > >   }
> > > > > >
> > > > > >   static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > > > +				    struct rkisp1_params_buffer *buffer)
> > > > > >   {
> > > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > > > @@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > >
> > > > > >   static int
> > > > > >   rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > > > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > > > +				    struct rkisp1_params_buffer *buffer)
> > > > > >   {
> > > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > > >   }
> > > > > >
> > > > > >   static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > > > > > -					struct rkisp1_ext_params_cfg *cfg)
> > > > > > +					struct rkisp1_params_buffer *buffer)
> > > > > >   {
> > > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > > > > >   }
> > > > > >
> > > > > > @@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > > > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > > > >   	} else {
> > > > > > -		ret = rkisp1_ext_params_config(params, cfg);
> > > > > > +		ret = rkisp1_ext_params_config(params, buf);
> > > > > >   	}
> > > > > >
> > > > > >   	if (ret)
> > > > > > @@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
> > > > > >   		rkisp1_isp_isr_other_config(params, cfg);
> > > > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > > > >   	} else {
> > > > > > -		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > > > > > +		ret = rkisp1_ext_params_other_meas_config(params, buf);
> > > > > >   	}
> > > > > >
> > > > > >   	if (ret) {
> > > > > > @@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
> > > > > >   	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > > >   	else
> > > > > > -		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > > > > > +		ret = rkisp1_ext_params_lsc_config(params, buf);
> > > > > >
> > > > > >   	if (ret)
> > > > > >   		goto complete_and_unlock;
> > > > > > @@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
> > > > > >   	return 0;
> > > > > >   }
> > > > > >
> > > > > > +static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
> > > > > > +{
> > > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > > +	struct rkisp1_params_buffer *params_buf =
> > > > > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > > >
> > > > A to_rkisp1_params_buffer() inline function in the previous patch would
> > > > be nice.
> > > >
> > > > > > +	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
> > > > > > +	size_t buf_size = params->metafmt.buffersize;
> > > > > > +
> > > > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > > > > +		params_buf->cfg = NULL;
> > > > > > +		return 0;
> > > > > > +	}
> > > >
> > > > The problem is not restricted to the extensible format, how about doing
> > > > the same for the legacy format ?
> > >
> > > Copying the buffer in ? Not sure I agree. The extensible format allows
> > > userspace to change the memory layout of the buffer, if this happens
> > > while the driver parses it, we'll ooops. With the fixed format the
> > > memory layout is fixed, if userspace changes the content of the buffer
> > > (ie the configuration parameters) after qbuf, well, it's their
> > > problem.
> >
> > Can you guarantee that the driver will never ever crash or corrupt
> > memory if that happens ? Especially if you take into account the fact
> 
> All of the mainline driver work in this way today. But ok if most
> people think it's worth it, I'll do so and re-order the patches.

Unless the memcpy really hinders performances, I think we should ensure
safety first and foremest.

> > that the compiler and CPU can reorder accesses, as well as split and
> > merge accesses (i.e. reading the same memory location multiple times
> > when it's accessed once only in the code), that seems hard to do.
> >
> > > Copying (and validation) is costly and if it serves to avoid
> > > an ooops it's more than justified. In the other case I'm not sure.
> >
> > How expensive is the copy ?
> 
> For sure it doubles the allocated memory as we've one backing memory
> buffer for each buffer requested by userspace.

True, but parameter buffers are quite small. I haven't checked but I
would expect everything to hold in one page.

-- 
Regards,

Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

* Re: [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer
  2024-06-19 16:11             ` Laurent Pinchart
@ 2024-06-19 16:20               ` Jacopo Mondi
  0 siblings, 0 replies; 38+ messages in thread
From: Jacopo Mondi @ 2024-06-19 16:20 UTC (permalink / raw)
  To: Laurent Pinchart
  Cc: Jacopo Mondi, Dan Scally, Linux Media Mailing List, Sakari Ailus,
	Hans Verkuil, Stefan Klug, Paul Elder, Kieran Bingham, Umang Jain,
	Dafna Hirschfeld, Mauro Carvalho Chehab, Heiko Stuebner

Hi Laurent

On Wed, Jun 19, 2024 at 07:11:01PM GMT, Laurent Pinchart wrote:
> On Wed, Jun 19, 2024 at 05:55:12PM +0200, Jacopo Mondi wrote:
> > Hi Laurent
> >
> > On Wed, Jun 19, 2024 at 06:44:20PM GMT, Laurent Pinchart wrote:
> > > Hi Jacopo,
> > >
> > > On Wed, Jun 19, 2024 at 04:22:00PM +0200, Jacopo Mondi wrote:
> > > > On Wed, Jun 12, 2024 at 07:20:48PM GMT, Laurent Pinchart wrote:
> > > > > On Wed, Jun 12, 2024 at 03:28:05PM +0100, Daniel Scally wrote:
> > > > > > Hi Jacopo. As mentioned in the last patch, I think that this could be
> > > > > > squashed into 5/8, but a couple of comments below
> > > > >
> > > > > I think it should be moved earlier, yes, probably even before the
> > > > > introduction of extended parameters.
> > > >
> > > > I'm not sure I agree here
> > > >
> > > > Squashing would create a patch that does too many things. Moving it
> > > > before the introduction of the extensible format would not make much
> > > > sense, as there would be nothing to validate/copy for the fixed
> > > > format.
> > >
> > > I think the copy should be done for the fixed format too. See below.
> > >
> > > > I'll keep the ordering I have here if not a big deal.
> > > >
> > > > > > On 05/06/2024 17:54, Jacopo Mondi wrote:
> > > > > > > With the introduction of the extensible parameters format support in the
> > > > > > > rkisp1-param.c module, the RkISP1 driver now configures the ISP blocks
> > > > > > > by parsing the content of a data buffer of variable size provided by
> > > > > > > userspace through the V4L2 meta-output interface using the MMAP memory
> > > > > > > handling mode.
> > > > > > >
> > > > > > > As the parameters buffer is mapped in the userspace process memory,
> > > > > > > applications have access to the buffer content while the driver
> > > > > > > parses it.
> > > > > > >
> > > > > > > To prevent potential issues during the parameters buffer parsing and
> > > > > > > processing in the driver, implement three vb2_ops to
> > > > > > >
> > > > > > > 1) allocate a scratch buffer in the driver private buffer structure
> > > > > > > 2) validate the buffer content at VIDIOC_QBUF time
> > > > > > > 3) copy the content of the user provided configuration parameters
> > > > > > >     in the driver-private scratch buffer
> > > > > > >
> > > > > > > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > > > > > > ---
> > > > > > >   .../platform/rockchip/rkisp1/rkisp1-params.c  | 154 ++++++++++++++----
> > > > > > >   1 file changed, 124 insertions(+), 30 deletions(-)
> > > > > > >
> > > > > > > diff --git a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > > > index 4adaf084ce6e..003239e14511 100644
> > > > > > > --- a/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > > > +++ b/drivers/media/platform/rockchip/rkisp1/rkisp1-params.c
> > > > > > > @@ -5,6 +5,8 @@
> > > > > > >    * Copyright (C) 2017 Rockchip Electronics Co., Ltd.
> > > > > > >    */
> > > > > > >
> > > > > > > +#include <linux/string.h>
> > > > > > > +
> > > > > > >   #include <media/v4l2-common.h>
> > > > > > >   #include <media/v4l2-event.h>
> > > > > > >   #include <media/v4l2-ioctl.h>
> > > > > > > @@ -1943,17 +1945,14 @@ static const struct rkisp1_ext_params_handler {
> > > > > > >   };
> > > > > > >
> > > > > > >   static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > > > -				      struct rkisp1_ext_params_cfg *cfg,
> > > > > > > +				      struct rkisp1_params_buffer *buffer,
> > > > > > >   				      u32 block_group_mask)
> > > > > > >   {
> > > > > > > +	struct rkisp1_ext_params_cfg *cfg = buffer->cfg;
> > > > >
> > > > > Maybe do this in the callers to avoid changing the prototype of this
> > > > > function and the other functions below.
> > > > >
> > > > > > >   	size_t block_offset = 0;
> > > > > > >
> > > > > > > -	if (cfg->total_size > RKISP1_EXT_PARAMS_MAX_SIZE) {
> > > > > > > -		dev_dbg(params->rkisp1->dev,
> > > > > > > -			"Invalid parameters buffer size %llu\n",
> > > > > > > -			cfg->total_size);
> > > > > > > -		return -EINVAL;
> > > > > > > -	}
> > > > > > > +	if (WARN_ON(!cfg))
> > > > > > > +		return -ENOMEM;
> > > > > > >
> > > > > > >   	/* Walk the list of parameter blocks and process them. */
> > > > > > >   	while (block_offset < cfg->total_size) {
> > > > > > > @@ -1965,25 +1964,13 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > > >   		block_offset += block->size;
> > > > > > >
> > > > > > >   		/*
> > > > > > > -		 * Validate the block id and make sure the block group is in
> > > > > > > -		 * the list of groups to configure.
> > > > > > > +		 * Make sure the block group is in  the list of groups to
> > > > > > > +		 * configure.
> > > > > > >   		 */
> > > > > > > -		if (block->type >= RKISP1_EXT_PARAMS_BLOCK_TYPE_SENTINEL) {
> > > > > > > -			dev_dbg(params->rkisp1->dev,
> > > > > > > -				"Invalid parameters block type\n");
> > > > > > > -			return -EINVAL;
> > > > > > > -		}
> > > > > > > -
> > > > > > >   		block_handler = &rkisp1_ext_params_handlers[block->type];
> > > > > > >   		if (!(block_handler->group & block_group_mask))
> > > > > > >   			continue;
> > > > > > >
> > > > > > > -		if (block->size != block_handler->size) {
> > > > > > > -			dev_dbg(params->rkisp1->dev,
> > > > > > > -				"Invalid parameters block size\n");
> > > > > > > -			return -EINVAL;
> > > > > > > -		}
> > > > > > > -
> > > > > > >   		block_handler->handler(params, block);
> > > > > > >   	}
> > > > > > >
> > > > > > > @@ -1991,9 +1978,9 @@ static int __rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > > >   }
> > > > > > >
> > > > > > >   static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > > > > +				    struct rkisp1_params_buffer *buffer)
> > > > > > >   {
> > > > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC |
> > > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > > > > @@ -2001,17 +1988,17 @@ static int rkisp1_ext_params_config(struct rkisp1_params *params,
> > > > > > >
> > > > > > >   static int
> > > > > > >   rkisp1_ext_params_other_meas_config(struct rkisp1_params *params,
> > > > > > > -				    struct rkisp1_ext_params_cfg *cfg)
> > > > > > > +				    struct rkisp1_params_buffer *buffer)
> > > > > > >   {
> > > > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_OTHERS |
> > > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_MEAS);
> > > > > > >   }
> > > > > > >
> > > > > > >   static int rkisp1_ext_params_lsc_config(struct rkisp1_params *params,
> > > > > > > -					struct rkisp1_ext_params_cfg *cfg)
> > > > > > > +					struct rkisp1_params_buffer *buffer)
> > > > > > >   {
> > > > > > > -	return __rkisp1_ext_params_config(params, cfg,
> > > > > > > +	return __rkisp1_ext_params_config(params, buffer,
> > > > > > >   					  RKISP1_EXT_PARAMS_BLOCK_GROUP_LSC);
> > > > > > >   }
> > > > > > >
> > > > > > > @@ -2057,7 +2044,7 @@ void rkisp1_params_isr(struct rkisp1_device *rkisp1)
> > > > > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > > > > >   	} else {
> > > > > > > -		ret = rkisp1_ext_params_config(params, cfg);
> > > > > > > +		ret = rkisp1_ext_params_config(params, buf);
> > > > > > >   	}
> > > > > > >
> > > > > > >   	if (ret)
> > > > > > > @@ -2168,7 +2155,7 @@ int rkisp1_params_pre_configure(struct rkisp1_params *params,
> > > > > > >   		rkisp1_isp_isr_other_config(params, cfg);
> > > > > > >   		rkisp1_isp_isr_meas_config(params, cfg);
> > > > > > >   	} else {
> > > > > > > -		ret = rkisp1_ext_params_other_meas_config(params, cfg);
> > > > > > > +		ret = rkisp1_ext_params_other_meas_config(params, buf);
> > > > > > >   	}
> > > > > > >
> > > > > > >   	if (ret) {
> > > > > > > @@ -2215,7 +2202,7 @@ int rkisp1_params_post_configure(struct rkisp1_params *params)
> > > > > > >   	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS)
> > > > > > >   		rkisp1_isp_isr_lsc_config(params, cfg);
> > > > > > >   	else
> > > > > > > -		ret = rkisp1_ext_params_lsc_config(params, cfg);
> > > > > > > +		ret = rkisp1_ext_params_lsc_config(params, buf);
> > > > > > >
> > > > > > >   	if (ret)
> > > > > > >   		goto complete_and_unlock;
> > > > > > > @@ -2407,6 +2394,110 @@ static int rkisp1_params_vb2_queue_setup(struct vb2_queue *vq,
> > > > > > >   	return 0;
> > > > > > >   }
> > > > > > >
> > > > > > > +static int rkisp1_params_vb2_buf_init(struct vb2_buffer *vb)
> > > > > > > +{
> > > > > > > +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > > > > > > +	struct rkisp1_params_buffer *params_buf =
> > > > > > > +		container_of(vbuf, struct rkisp1_params_buffer, vb);
> > > > >
> > > > > A to_rkisp1_params_buffer() inline function in the previous patch would
> > > > > be nice.
> > > > >
> > > > > > > +	struct rkisp1_params *params = vb->vb2_queue->drv_priv;
> > > > > > > +	size_t buf_size = params->metafmt.buffersize;
> > > > > > > +
> > > > > > > +	if (params->metafmt.dataformat == V4L2_META_FMT_RK_ISP1_PARAMS) {
> > > > > > > +		params_buf->cfg = NULL;
> > > > > > > +		return 0;
> > > > > > > +	}
> > > > >
> > > > > The problem is not restricted to the extensible format, how about doing
> > > > > the same for the legacy format ?
> > > >
> > > > Copying the buffer in ? Not sure I agree. The extensible format allows
> > > > userspace to change the memory layout of the buffer, if this happens
> > > > while the driver parses it, we'll ooops. With the fixed format the
> > > > memory layout is fixed, if userspace changes the content of the buffer
> > > > (ie the configuration parameters) after qbuf, well, it's their
> > > > problem.
> > >
> > > Can you guarantee that the driver will never ever crash or corrupt
> > > memory if that happens ? Especially if you take into account the fact
> >
> > All of the mainline driver work in this way today. But ok if most
> > people think it's worth it, I'll do so and re-order the patches.
>
> Unless the memcpy really hinders performances, I think we should ensure
> safety first and foremest.
>

I haven't run any test, but I doubt the performance penality is
terrible

> > > that the compiler and CPU can reorder accesses, as well as split and
> > > merge accesses (i.e. reading the same memory location multiple times
> > > when it's accessed once only in the code), that seems hard to do.
> > >
> > > > Copying (and validation) is costly and if it serves to avoid
> > > > an ooops it's more than justified. In the other case I'm not sure.
> > >
> > > How expensive is the copy ?
> >
> > For sure it doubles the allocated memory as we've one backing memory
> > buffer for each buffer requested by userspace.
>
> True, but parameter buffers are quite small. I haven't checked but I
> would expect everything to hold in one page.

True that.

Ok, safety first, I'll add support for copying in the fixed format as
well!

Thanks
  j

>
> --
> Regards,
>
> Laurent Pinchart

^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2024-06-20  9:42 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-06-05 16:54 [PATCH 0/8] media: rkisp1: Implement support for extensible parameters Jacopo Mondi
2024-06-05 16:54 ` [PATCH 1/8] uapi: rkisp1-config: Add extensible parameters format Jacopo Mondi
2024-06-12 10:02   ` Dan Scally
2024-06-12 12:56     ` Laurent Pinchart
2024-06-12 13:49       ` Dan Scally
2024-06-12 14:56         ` Laurent Pinchart
2024-06-19 12:44       ` Jacopo Mondi
2024-06-19 15:49         ` Laurent Pinchart
2024-06-05 16:54 ` [PATCH 2/8] uapi: videodev2: Add V4L2_META_FMT_RK_ISP1_EXT_PARAMS Jacopo Mondi
2024-06-12 10:00   ` Dan Scally
2024-06-12 14:35     ` Laurent Pinchart
2024-06-12 15:09       ` Dan Scally
2024-06-20  9:31   ` Paul Elder
2024-06-05 16:54 ` [PATCH 3/8] media: rkisp1: Remove cached format info Jacopo Mondi
2024-06-12 10:06   ` Dan Scally
2024-06-12 14:47   ` Laurent Pinchart
2024-06-20  9:41   ` Paul Elder
2024-06-05 16:54 ` [PATCH 4/8] media: rkisp1: Add support for ext format Jacopo Mondi
2024-06-12 10:46   ` Dan Scally
2024-06-12 14:51     ` Laurent Pinchart
2024-06-05 16:54 ` [PATCH 5/8] media: rkisp1: Implement extensible params support Jacopo Mondi
2024-06-12 13:50   ` Dan Scally
2024-06-12 15:42     ` Laurent Pinchart
2024-06-19 15:46       ` Jacopo Mondi
2024-06-19 16:09         ` Laurent Pinchart
2024-06-05 16:54 ` [PATCH 6/8] media: rkisp1: Propagate pre/post-config errors Jacopo Mondi
2024-06-12 13:35   ` Dan Scally
2024-06-12 15:46     ` Laurent Pinchart
2024-06-05 16:54 ` [PATCH 7/8] media: rkisp1: Add struct rkisp1_params_buffer Jacopo Mondi
2024-06-12 13:52   ` Dan Scally
2024-06-05 16:54 ` [PATCH 8/8] media: rkisp1: Copy and validate parameters buffer Jacopo Mondi
2024-06-12 14:28   ` Dan Scally
2024-06-12 16:20     ` Laurent Pinchart
2024-06-19 14:22       ` Jacopo Mondi
2024-06-19 15:44         ` Laurent Pinchart
2024-06-19 15:55           ` Jacopo Mondi
2024-06-19 16:11             ` Laurent Pinchart
2024-06-19 16:20               ` Jacopo Mondi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).