[PATCH v3 00/29] media: ov2680: Bugfixes + ACPI + selection(crop-tgt) API support

[PATCH v3 00/29] media: ov2680: Bugfixes + ACPI + selection(crop-tgt) API support

[Hans de Goede]

Hi All,

Here is v3 of my ov2680 sensor driver patch series.

Changes in v3:
- Add Rui Miguel Silva's Ack for the series
- 2 small fixes for remarks from Andy
- Add a new patch adding me as co-maintainer in MAINTAINERS

Changes in v2
- Drop "media: Add MIPI CCI register access helper functions"
  (being reviewed in its own thread / patch-submission)
- Drop "media: ov2680: Add g_skip_frames op support"
- Add "media: ov2680: Fix regulators being left enabled on
  ov2680_power_on() errors"
- Add "media: ov2680: Add link-freq and pixel-rate controls"
  with this the driver now works on IPU3 with ipu3-capture.sh
  (libcamera support requires adding a couple more controls)
- Limit line length to 80 chars everywhere
- Address various small remarks from Andy

During all the work done on the atomisp driver I have mostly been testing
on devices with an ov2680 sensor. As such I have also done a lot of work
on the atomisp-ov2680.c atomisp specific sensor driver.

With the latest atomisp code from:
https://git.kernel.org/pub/scm/linux/kernel/git/hansg/linux.git/tag/?h=media-atomisp-6.5-1

The atomisp code can now work with standard v4l2 sensor drivers using
the selections (crop-tgt) api and v4l2-async sensor driver registration.

This patch series modifies the main drivers/media/i2c/ov2680.c driver
to add bugfixes, ACPI enumeration, selection API support and further
improvments. After this the driver can be used with the atomisp driver
and atomisp-ov2680.c can be dropped.

This also gets the driver much closer to having everything needed for
use with IPU3 / libcamera. I have a Lenovo Miix 510 now with an IPU3 +
ov2680 sensor and with this series raw-capture using the ipu3-capture.sh
script works. I plan to work on libcamera support for this in the near
future.

This series consist of 3 parts:

1. Patches 1-8 are bugfixes these are put first for backporting

2. Patch 9 converts the ov2680 driver to the new CCI helpers,
the same has been done in the other series with the atomisp-ov2680
driver and this makes it much easier to sync things up.

Note this depends on the new CCI register helpers, these are being
reviewed here:

https://lore.kernel.org/linux-media/20230627125109.52354-1-hdegoede@redhat.com/

3. Patches 9 - 29 implement the ACPI enumeration,
selection API support and further improvments.

Regards,

Hans



Hans de Goede (29):
  media: ov2680: Remove auto-gain and auto-exposure controls
  media: ov2680: Fix ov2680_bayer_order()
  media: ov2680: Fix vflip / hflip set functions
  media: ov2680: Use select VIDEO_V4L2_SUBDEV_API
  media: ov2680: Don't take the lock for try_fmt calls
  media: ov2680: Add ov2680_fill_format() helper function
  media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY
    not working
  media: ov2680: Fix regulators being left enabled on ov2680_power_on()
    errors
  media: ov2680: Convert to new CCI register access helpers
  media: ov2680: Store dev instead of i2c_client in ov2680_dev
  media: ov2680: Check for "powerdown" GPIO con-id before checking for
    "reset" GPIO con-id
  media: ov2680: Add runtime-pm support
  media: ov2680: Drop is_enabled flag
  media: ov2680: Add support for more clk setups
  media: ov2680: Add support for 19.2 MHz clock
  media: ov2680: Add endpoint matching support
  media: ov2680: Add support for ACPI enumeration
  media: ov2680: Fix ov2680_enum_frame_interval()
  media: ov2680: Annotate the per mode register setting lists
  media: ov2680: Add ov2680_mode struct
  media: ov2680: Make setting the mode algorithm based
  media: ov2680: Add an __ov2680_get_pad_format() helper function
  media: ov2680: Implement selection support
  media: ov2680: Fix exposure and gain ctrls range and default value
  media: ov2680: Add a bunch of register tweaks
  media: ov2680: Drop unnecessary pad checks
  media: ov2680: Read and log sensor revision during probe
  media: ov2680: Add link-freq and pixel-rate controls
  MAINTAINERS: Add Hans de Goede as OV2680 sensor driver maintainer

 MAINTAINERS                |    1 +
 drivers/media/i2c/Kconfig  |    2 +
 drivers/media/i2c/ov2680.c | 1318 +++++++++++++++++++-----------------
 3 files changed, 691 insertions(+), 630 deletions(-)

-- 
2.41.0

[PATCH v3 01/29] media: ov2680: Remove auto-gain and auto-exposure controls

[Hans de Goede]

Quoting the OV2680 datasheet:

"3.2 exposure and gain control

In the OV2680, the exposure time and gain are set manually from an external
controller. The OV2680 supports manual gain and exposure control only for
normal applications, no auto mode."

And indeed testing with the atomisp_ov2680 fork of ov2680.c has shown that
auto-exposure and auto-gain do not work.

Note that the code setting the auto-exposure flag was broken, callers
of ov2680_exposure_set() were directly passing !!ctrls->auto_exp->val as
"bool auto_exp" value, but ctrls->auto_exp is a menu control with:

enum  v4l2_exposure_auto_type {
        V4L2_EXPOSURE_AUTO = 0,
        V4L2_EXPOSURE_MANUAL = 1,
	...

So instead of passing !!ctrls->auto_exp->val they should have been passing
ctrls->auto_exp->val == V4L2_EXPOSURE_AUTO, iow the passed value was
inverted of what it should have been.

Also remove ov2680_g_volatile_ctrl() since without auto support the gain
and exposure controls are not volatile.

This also fixes the control values not being properly applied in
ov2680_mode_set(). The 800x600 mode register-list also sets gain,
exposure and vflip overriding the last set ctrl values.

ov2680_mode_set() does call ov2680_gain_set() and ov2680_exposure_set()
but did this before writing the mode register-list, so these values
would still be overridden by the mode register-list.

Add a v4l2_ctrl_handler_setup() call after writing the mode register-list
to restore all ctrl values. Also remove the ctrls->gain->is_new check from
ov2680_gain_set() so that the gain always gets restored properly.

Last since ov2680_mode_set() now calls v4l2_ctrl_handler_setup(), remove
the v4l2_ctrl_handler_setup() call after ov2680_mode_restore() since
ov2680_mode_restore() calls ov2680_mode_set().

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 163 ++++---------------------------------
 1 file changed, 18 insertions(+), 145 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index d06e9fc37f77..049ca28b9663 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -85,15 +85,8 @@ struct ov2680_mode_info {
 
 struct ov2680_ctrls {
 	struct v4l2_ctrl_handler handler;
-	struct {
-		struct v4l2_ctrl *auto_exp;
-		struct v4l2_ctrl *exposure;
-	};
-	struct {
-		struct v4l2_ctrl *auto_gain;
-		struct v4l2_ctrl *gain;
-	};
-
+	struct v4l2_ctrl *exposure;
+	struct v4l2_ctrl *gain;
 	struct v4l2_ctrl *hflip;
 	struct v4l2_ctrl *vflip;
 	struct v4l2_ctrl *test_pattern;
@@ -143,6 +136,7 @@ static const struct reg_value ov2680_setting_30fps_QUXGA_800_600[] = {
 	{0x380e, 0x02}, {0x380f, 0x84}, {0x3811, 0x04}, {0x3813, 0x04},
 	{0x3814, 0x31}, {0x3815, 0x31}, {0x3820, 0xc0}, {0x4008, 0x00},
 	{0x4009, 0x03}, {0x4837, 0x1e}, {0x3501, 0x4e}, {0x3502, 0xe0},
+	{0x3503, 0x03},
 };
 
 static const struct reg_value ov2680_setting_30fps_720P_1280_720[] = {
@@ -405,69 +399,15 @@ static int ov2680_test_pattern_set(struct ov2680_dev *sensor, int value)
 	return 0;
 }
 
-static int ov2680_gain_set(struct ov2680_dev *sensor, bool auto_gain)
+static int ov2680_gain_set(struct ov2680_dev *sensor, u32 gain)
 {
-	struct ov2680_ctrls *ctrls = &sensor->ctrls;
-	u32 gain;
-	int ret;
-
-	ret = ov2680_mod_reg(sensor, OV2680_REG_R_MANUAL, BIT(1),
-			     auto_gain ? 0 : BIT(1));
-	if (ret < 0)
-		return ret;
-
-	if (auto_gain || !ctrls->gain->is_new)
-		return 0;
-
-	gain = ctrls->gain->val;
-
-	ret = ov2680_write_reg16(sensor, OV2680_REG_GAIN_PK, gain);
-
-	return 0;
+	return ov2680_write_reg16(sensor, OV2680_REG_GAIN_PK, gain);
 }
 
-static int ov2680_gain_get(struct ov2680_dev *sensor)
+static int ov2680_exposure_set(struct ov2680_dev *sensor, u32 exp)
 {
-	u32 gain;
-	int ret;
-
-	ret = ov2680_read_reg16(sensor, OV2680_REG_GAIN_PK, &gain);
-	if (ret)
-		return ret;
-
-	return gain;
-}
-
-static int ov2680_exposure_set(struct ov2680_dev *sensor, bool auto_exp)
-{
-	struct ov2680_ctrls *ctrls = &sensor->ctrls;
-	u32 exp;
-	int ret;
-
-	ret = ov2680_mod_reg(sensor, OV2680_REG_R_MANUAL, BIT(0),
-			     auto_exp ? 0 : BIT(0));
-	if (ret < 0)
-		return ret;
-
-	if (auto_exp || !ctrls->exposure->is_new)
-		return 0;
-
-	exp = (u32)ctrls->exposure->val;
-	exp <<= 4;
-
-	return ov2680_write_reg24(sensor, OV2680_REG_EXPOSURE_PK_HIGH, exp);
-}
-
-static int ov2680_exposure_get(struct ov2680_dev *sensor)
-{
-	int ret;
-	u32 exp;
-
-	ret = ov2680_read_reg24(sensor, OV2680_REG_EXPOSURE_PK_HIGH, &exp);
-	if (ret)
-		return ret;
-
-	return exp >> 4;
+	return ov2680_write_reg24(sensor, OV2680_REG_EXPOSURE_PK_HIGH,
+				  exp << 4);
 }
 
 static int ov2680_stream_enable(struct ov2680_dev *sensor)
@@ -482,32 +422,16 @@ static int ov2680_stream_disable(struct ov2680_dev *sensor)
 
 static int ov2680_mode_set(struct ov2680_dev *sensor)
 {
-	struct ov2680_ctrls *ctrls = &sensor->ctrls;
 	int ret;
 
-	ret = ov2680_gain_set(sensor, false);
-	if (ret < 0)
-		return ret;
-
-	ret = ov2680_exposure_set(sensor, false);
-	if (ret < 0)
-		return ret;
-
 	ret = ov2680_load_regs(sensor, sensor->current_mode);
 	if (ret < 0)
 		return ret;
 
-	if (ctrls->auto_gain->val) {
-		ret = ov2680_gain_set(sensor, true);
-		if (ret < 0)
-			return ret;
-	}
-
-	if (ctrls->auto_exp->val == V4L2_EXPOSURE_AUTO) {
-		ret = ov2680_exposure_set(sensor, true);
-		if (ret < 0)
-			return ret;
-	}
+	/* Restore value of all ctrls */
+	ret = __v4l2_ctrl_handler_setup(&sensor->ctrls.handler);
+	if (ret < 0)
+		return ret;
 
 	sensor->mode_pending_changes = false;
 
@@ -590,15 +514,10 @@ static int ov2680_s_power(struct v4l2_subdev *sd, int on)
 	else
 		ret = ov2680_power_off(sensor);
 
-	mutex_unlock(&sensor->lock);
-
-	if (on && ret == 0) {
-		ret = v4l2_ctrl_handler_setup(&sensor->ctrls.handler);
-		if (ret < 0)
-			return ret;
-
+	if (on && ret == 0)
 		ret = ov2680_mode_restore(sensor);
-	}
+
+	mutex_unlock(&sensor->lock);
 
 	return ret;
 }
@@ -794,52 +713,19 @@ static int ov2680_enum_frame_interval(struct v4l2_subdev *sd,
 	return 0;
 }
 
-static int ov2680_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
-{
-	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
-	struct ov2680_dev *sensor = to_ov2680_dev(sd);
-	struct ov2680_ctrls *ctrls = &sensor->ctrls;
-	int val;
-
-	if (!sensor->is_enabled)
-		return 0;
-
-	switch (ctrl->id) {
-	case V4L2_CID_GAIN:
-		val = ov2680_gain_get(sensor);
-		if (val < 0)
-			return val;
-		ctrls->gain->val = val;
-		break;
-	case V4L2_CID_EXPOSURE:
-		val = ov2680_exposure_get(sensor);
-		if (val < 0)
-			return val;
-		ctrls->exposure->val = val;
-		break;
-	}
-
-	return 0;
-}
-
 static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
-	struct ov2680_ctrls *ctrls = &sensor->ctrls;
 
 	if (!sensor->is_enabled)
 		return 0;
 
 	switch (ctrl->id) {
-	case V4L2_CID_AUTOGAIN:
-		return ov2680_gain_set(sensor, !!ctrl->val);
 	case V4L2_CID_GAIN:
-		return ov2680_gain_set(sensor, !!ctrls->auto_gain->val);
-	case V4L2_CID_EXPOSURE_AUTO:
-		return ov2680_exposure_set(sensor, !!ctrl->val);
+		return ov2680_gain_set(sensor, ctrl->val);
 	case V4L2_CID_EXPOSURE:
-		return ov2680_exposure_set(sensor, !!ctrls->auto_exp->val);
+		return ov2680_exposure_set(sensor, ctrl->val);
 	case V4L2_CID_VFLIP:
 		if (sensor->is_streaming)
 			return -EBUSY;
@@ -864,7 +750,6 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
 }
 
 static const struct v4l2_ctrl_ops ov2680_ctrl_ops = {
-	.g_volatile_ctrl = ov2680_g_volatile_ctrl,
 	.s_ctrl = ov2680_s_ctrl,
 };
 
@@ -936,7 +821,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 	if (ret < 0)
 		return ret;
 
-	v4l2_ctrl_handler_init(hdl, 7);
+	v4l2_ctrl_handler_init(hdl, 5);
 
 	hdl->lock = &sensor->lock;
 
@@ -948,16 +833,9 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 					ARRAY_SIZE(test_pattern_menu) - 1,
 					0, 0, test_pattern_menu);
 
-	ctrls->auto_exp = v4l2_ctrl_new_std_menu(hdl, ops,
-						 V4L2_CID_EXPOSURE_AUTO,
-						 V4L2_EXPOSURE_MANUAL, 0,
-						 V4L2_EXPOSURE_AUTO);
-
 	ctrls->exposure = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_EXPOSURE,
 					    0, 32767, 1, 0);
 
-	ctrls->auto_gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_AUTOGAIN,
-					     0, 1, 1, 1);
 	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN, 0, 2047, 1, 0);
 
 	if (hdl->error) {
@@ -965,14 +843,9 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 		goto cleanup_entity;
 	}
 
-	ctrls->gain->flags |= V4L2_CTRL_FLAG_VOLATILE;
-	ctrls->exposure->flags |= V4L2_CTRL_FLAG_VOLATILE;
 	ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
 	ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
 
-	v4l2_ctrl_auto_cluster(2, &ctrls->auto_gain, 0, true);
-	v4l2_ctrl_auto_cluster(2, &ctrls->auto_exp, 1, true);
-
 	sensor->sd.ctrl_handler = hdl;
 
 	ret = v4l2_async_register_subdev(&sensor->sd);
-- 
2.41.0

[PATCH v3 02/29] media: ov2680: Fix ov2680_bayer_order()

[Hans de Goede]

The index into ov2680_hv_flip_bayer_order[] should be 0-3, but
ov2680_bayer_order() was using 0 + BIT(2) + (BIT(2) << 1) as
max index, while the intention was to use: 0 + 1 + 2 as max index.

Fix the index calculation in ov2680_bayer_order(), while at it
also just use the ctrl values rather then reading them back using
a slow i2c-read transaction.

This also allows making the function void, since there now are
no more i2c-reads to error check.

Note the check for the ctrls being NULL is there to allow
adding an ov2680_fill_format() helper later, which will call
ov2680_set_bayer_order() during probe() before the ctrls are created.

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 27 ++++++++++-----------------
 1 file changed, 10 insertions(+), 17 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 049ca28b9663..2001e08253ef 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -315,26 +315,17 @@ static void ov2680_power_down(struct ov2680_dev *sensor)
 	usleep_range(5000, 10000);
 }
 
-static int ov2680_bayer_order(struct ov2680_dev *sensor)
+static void ov2680_set_bayer_order(struct ov2680_dev *sensor)
 {
-	u32 format1;
-	u32 format2;
-	u32 hv_flip;
-	int ret;
+	int hv_flip = 0;
 
-	ret = ov2680_read_reg(sensor, OV2680_REG_FORMAT1, &format1);
-	if (ret < 0)
-		return ret;
+	if (sensor->ctrls.vflip && sensor->ctrls.vflip->val)
+		hv_flip += 1;
 
-	ret = ov2680_read_reg(sensor, OV2680_REG_FORMAT2, &format2);
-	if (ret < 0)
-		return ret;
-
-	hv_flip = (format2 & BIT(2)  << 1) | (format1 & BIT(2));
+	if (sensor->ctrls.hflip && sensor->ctrls.hflip->val)
+		hv_flip += 2;
 
 	sensor->fmt.code = ov2680_hv_flip_bayer_order[hv_flip];
-
-	return 0;
 }
 
 static int ov2680_vflip_enable(struct ov2680_dev *sensor)
@@ -345,7 +336,8 @@ static int ov2680_vflip_enable(struct ov2680_dev *sensor)
 	if (ret < 0)
 		return ret;
 
-	return ov2680_bayer_order(sensor);
+	ov2680_set_bayer_order(sensor);
+	return 0;
 }
 
 static int ov2680_vflip_disable(struct ov2680_dev *sensor)
@@ -378,7 +370,8 @@ static int ov2680_hflip_disable(struct ov2680_dev *sensor)
 	if (ret < 0)
 		return ret;
 
-	return ov2680_bayer_order(sensor);
+	ov2680_set_bayer_order(sensor);
+	return 0;
 }
 
 static int ov2680_test_pattern_set(struct ov2680_dev *sensor, int value)
-- 
2.41.0

[PATCH v3 03/29] media: ov2680: Fix vflip / hflip set functions

[Hans de Goede]

ov2680_vflip_disable() / ov2680_hflip_disable() pass BIT(0) instead of
0 as value to ov2680_mod_reg().

While fixing this also:

1. Stop having separate enable/disable functions for hflip / vflip
2. Move the is_streaming check, which is unique to hflip / vflip
   into the ov2680_set_?flip() functions.

for a nice code cleanup.

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 50 ++++++++++----------------------------
 1 file changed, 13 insertions(+), 37 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 2001e08253ef..c93810f84ed7 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -328,11 +328,15 @@ static void ov2680_set_bayer_order(struct ov2680_dev *sensor)
 	sensor->fmt.code = ov2680_hv_flip_bayer_order[hv_flip];
 }
 
-static int ov2680_vflip_enable(struct ov2680_dev *sensor)
+static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
 {
 	int ret;
 
-	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(2));
+	if (sensor->is_streaming)
+		return -EBUSY;
+
+	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1,
+			     BIT(2), val ? BIT(2) : 0);
 	if (ret < 0)
 		return ret;
 
@@ -340,33 +344,15 @@ static int ov2680_vflip_enable(struct ov2680_dev *sensor)
 	return 0;
 }
 
-static int ov2680_vflip_disable(struct ov2680_dev *sensor)
+static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
 {
 	int ret;
 
-	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(0));
-	if (ret < 0)
-		return ret;
+	if (sensor->is_streaming)
+		return -EBUSY;
 
-	return ov2680_bayer_order(sensor);
-}
-
-static int ov2680_hflip_enable(struct ov2680_dev *sensor)
-{
-	int ret;
-
-	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(2));
-	if (ret < 0)
-		return ret;
-
-	return ov2680_bayer_order(sensor);
-}
-
-static int ov2680_hflip_disable(struct ov2680_dev *sensor)
-{
-	int ret;
-
-	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(0));
+	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2,
+			     BIT(2), val ? BIT(2) : 0);
 	if (ret < 0)
 		return ret;
 
@@ -720,19 +706,9 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
 	case V4L2_CID_EXPOSURE:
 		return ov2680_exposure_set(sensor, ctrl->val);
 	case V4L2_CID_VFLIP:
-		if (sensor->is_streaming)
-			return -EBUSY;
-		if (ctrl->val)
-			return ov2680_vflip_enable(sensor);
-		else
-			return ov2680_vflip_disable(sensor);
+		return ov2680_set_vflip(sensor, ctrl->val);
 	case V4L2_CID_HFLIP:
-		if (sensor->is_streaming)
-			return -EBUSY;
-		if (ctrl->val)
-			return ov2680_hflip_enable(sensor);
-		else
-			return ov2680_hflip_disable(sensor);
+		return ov2680_set_hflip(sensor, ctrl->val);
 	case V4L2_CID_TEST_PATTERN:
 		return ov2680_test_pattern_set(sensor, ctrl->val);
 	default:
-- 
2.41.0

Re: [PATCH v3 03/29] media: ov2680: Fix vflip / hflip set functions

[Jacopo Mondi]

Hi Hans

On Tue, Jun 27, 2023 at 03:18:04PM +0200, Hans de Goede wrote:
> ov2680_vflip_disable() / ov2680_hflip_disable() pass BIT(0) instead of
> 0 as value to ov2680_mod_reg().
>
> While fixing this also:
>
> 1. Stop having separate enable/disable functions for hflip / vflip
> 2. Move the is_streaming check, which is unique to hflip / vflip
>    into the ov2680_set_?flip() functions.

The patch looks good, but one little question on the controls update
procedure.

Usually s_ctrl() handlers checks for the sensor power state, like the
driver here is already doing by testing the is_enabled[*] flag, but they
usually call __v4l2_ctrl_handler_setup() unconditionally at
s_stream(1) time, not only if a new mode has been applied by calling
s_fmt(). Controls could be updated by userspace while the sensor is
powered off, and new values should be applied regardless if a new mode,
has been applied or not.

Does it make sense to you ?

[*] or better, if the sensor is ported to use pm_runtime first (by
dropping support for the deprecated .s_power(), or do you need
s_power() on your platform ?) you can use pm_runtime_get_if_in_use()
instead of keeping track manually of the is_enabled flag.

>
> for a nice code cleanup.
>
> Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>  drivers/media/i2c/ov2680.c | 50 ++++++++++----------------------------
>  1 file changed, 13 insertions(+), 37 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index 2001e08253ef..c93810f84ed7 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -328,11 +328,15 @@ static void ov2680_set_bayer_order(struct ov2680_dev *sensor)
>  	sensor->fmt.code = ov2680_hv_flip_bayer_order[hv_flip];
>  }
>
> -static int ov2680_vflip_enable(struct ov2680_dev *sensor)
> +static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
>  {
>  	int ret;
>
> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(2));
> +	if (sensor->is_streaming)
> +		return -EBUSY;
> +
> +	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1,
> +			     BIT(2), val ? BIT(2) : 0);
>  	if (ret < 0)
>  		return ret;
>
> @@ -340,33 +344,15 @@ static int ov2680_vflip_enable(struct ov2680_dev *sensor)
>  	return 0;
>  }
>
> -static int ov2680_vflip_disable(struct ov2680_dev *sensor)
> +static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
>  {
>  	int ret;
>
> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(0));
> -	if (ret < 0)
> -		return ret;
> +	if (sensor->is_streaming)
> +		return -EBUSY;
>
> -	return ov2680_bayer_order(sensor);
> -}
> -
> -static int ov2680_hflip_enable(struct ov2680_dev *sensor)
> -{
> -	int ret;
> -
> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(2));
> -	if (ret < 0)
> -		return ret;
> -
> -	return ov2680_bayer_order(sensor);
> -}
> -
> -static int ov2680_hflip_disable(struct ov2680_dev *sensor)
> -{
> -	int ret;
> -
> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(0));
> +	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2,
> +			     BIT(2), val ? BIT(2) : 0);
>  	if (ret < 0)
>  		return ret;
>
> @@ -720,19 +706,9 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
>  	case V4L2_CID_EXPOSURE:
>  		return ov2680_exposure_set(sensor, ctrl->val);
>  	case V4L2_CID_VFLIP:
> -		if (sensor->is_streaming)
> -			return -EBUSY;
> -		if (ctrl->val)
> -			return ov2680_vflip_enable(sensor);
> -		else
> -			return ov2680_vflip_disable(sensor);
> +		return ov2680_set_vflip(sensor, ctrl->val);
>  	case V4L2_CID_HFLIP:
> -		if (sensor->is_streaming)
> -			return -EBUSY;
> -		if (ctrl->val)
> -			return ov2680_hflip_enable(sensor);
> -		else
> -			return ov2680_hflip_disable(sensor);
> +		return ov2680_set_hflip(sensor, ctrl->val);
>  	case V4L2_CID_TEST_PATTERN:
>  		return ov2680_test_pattern_set(sensor, ctrl->val);
>  	default:
> --
> 2.41.0
>

Re: [PATCH v3 03/29] media: ov2680: Fix vflip / hflip set functions

[Hans de Goede]

Hi Jacopo,

On 6/27/23 16:35, Jacopo Mondi wrote:
> Hi Hans
> 
> On Tue, Jun 27, 2023 at 03:18:04PM +0200, Hans de Goede wrote:
>> ov2680_vflip_disable() / ov2680_hflip_disable() pass BIT(0) instead of
>> 0 as value to ov2680_mod_reg().
>>
>> While fixing this also:
>>
>> 1. Stop having separate enable/disable functions for hflip / vflip
>> 2. Move the is_streaming check, which is unique to hflip / vflip
>>    into the ov2680_set_?flip() functions.
> 
> The patch looks good, but one little question on the controls update
> procedure.
> 
> Usually s_ctrl() handlers checks for the sensor power state, like the
> driver here is already doing by testing the is_enabled[*] flag, but they
> usually call __v4l2_ctrl_handler_setup() unconditionally at
> s_stream(1) time, not only if a new mode has been applied by calling
> s_fmt(). Controls could be updated by userspace while the sensor is
> powered off, and new values should be applied regardless if a new mode,
> has been applied or not.
> 
> Does it make sense to you ?
> 
> [*] or better, if the sensor is ported to use pm_runtime first (by
> dropping support for the deprecated .s_power(), or do you need
> s_power() on your platform ?) you can use pm_runtime_get_if_in_use()
> instead of keeping track manually of the is_enabled flag.

What you are describing above is pretty much exactly what is done
later in this patchset.

Patches 1-8 are there to fix some glaring bugs, with the possibility
of backporting the fixes which is why they are the first patches
in the set.

Then after patch 9 the real work to get this driver into shape begins :)

Regards,

Hans





>> for a nice code cleanup.
>>
>> Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
>> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>> ---
>>  drivers/media/i2c/ov2680.c | 50 ++++++++++----------------------------
>>  1 file changed, 13 insertions(+), 37 deletions(-)
>>
>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>> index 2001e08253ef..c93810f84ed7 100644
>> --- a/drivers/media/i2c/ov2680.c
>> +++ b/drivers/media/i2c/ov2680.c
>> @@ -328,11 +328,15 @@ static void ov2680_set_bayer_order(struct ov2680_dev *sensor)
>>  	sensor->fmt.code = ov2680_hv_flip_bayer_order[hv_flip];
>>  }
>>
>> -static int ov2680_vflip_enable(struct ov2680_dev *sensor)
>> +static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
>>  {
>>  	int ret;
>>
>> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(2));
>> +	if (sensor->is_streaming)
>> +		return -EBUSY;
>> +
>> +	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1,
>> +			     BIT(2), val ? BIT(2) : 0);
>>  	if (ret < 0)
>>  		return ret;
>>
>> @@ -340,33 +344,15 @@ static int ov2680_vflip_enable(struct ov2680_dev *sensor)
>>  	return 0;
>>  }
>>
>> -static int ov2680_vflip_disable(struct ov2680_dev *sensor)
>> +static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
>>  {
>>  	int ret;
>>
>> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(0));
>> -	if (ret < 0)
>> -		return ret;
>> +	if (sensor->is_streaming)
>> +		return -EBUSY;
>>
>> -	return ov2680_bayer_order(sensor);
>> -}
>> -
>> -static int ov2680_hflip_enable(struct ov2680_dev *sensor)
>> -{
>> -	int ret;
>> -
>> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(2));
>> -	if (ret < 0)
>> -		return ret;
>> -
>> -	return ov2680_bayer_order(sensor);
>> -}
>> -
>> -static int ov2680_hflip_disable(struct ov2680_dev *sensor)
>> -{
>> -	int ret;
>> -
>> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(0));
>> +	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2,
>> +			     BIT(2), val ? BIT(2) : 0);
>>  	if (ret < 0)
>>  		return ret;
>>
>> @@ -720,19 +706,9 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
>>  	case V4L2_CID_EXPOSURE:
>>  		return ov2680_exposure_set(sensor, ctrl->val);
>>  	case V4L2_CID_VFLIP:
>> -		if (sensor->is_streaming)
>> -			return -EBUSY;
>> -		if (ctrl->val)
>> -			return ov2680_vflip_enable(sensor);
>> -		else
>> -			return ov2680_vflip_disable(sensor);
>> +		return ov2680_set_vflip(sensor, ctrl->val);
>>  	case V4L2_CID_HFLIP:
>> -		if (sensor->is_streaming)
>> -			return -EBUSY;
>> -		if (ctrl->val)
>> -			return ov2680_hflip_enable(sensor);
>> -		else
>> -			return ov2680_hflip_disable(sensor);
>> +		return ov2680_set_hflip(sensor, ctrl->val);
>>  	case V4L2_CID_TEST_PATTERN:
>>  		return ov2680_test_pattern_set(sensor, ctrl->val);
>>  	default:
>> --
>> 2.41.0
>>
> 

Re: [PATCH v3 03/29] media: ov2680: Fix vflip / hflip set functions

[Jacopo Mondi]

Hi Hans

On Tue, Jun 27, 2023 at 04:38:50PM +0200, Hans de Goede wrote:
> Hi Jacopo,
>
> On 6/27/23 16:35, Jacopo Mondi wrote:
> > Hi Hans
> >
> > On Tue, Jun 27, 2023 at 03:18:04PM +0200, Hans de Goede wrote:
> >> ov2680_vflip_disable() / ov2680_hflip_disable() pass BIT(0) instead of
> >> 0 as value to ov2680_mod_reg().
> >>
> >> While fixing this also:
> >>
> >> 1. Stop having separate enable/disable functions for hflip / vflip
> >> 2. Move the is_streaming check, which is unique to hflip / vflip
> >>    into the ov2680_set_?flip() functions.
> >
> > The patch looks good, but one little question on the controls update
> > procedure.
> >
> > Usually s_ctrl() handlers checks for the sensor power state, like the
> > driver here is already doing by testing the is_enabled[*] flag, but they
> > usually call __v4l2_ctrl_handler_setup() unconditionally at
> > s_stream(1) time, not only if a new mode has been applied by calling
> > s_fmt(). Controls could be updated by userspace while the sensor is
> > powered off, and new values should be applied regardless if a new mode,
> > has been applied or not.
> >
> > Does it make sense to you ?
> >
> > [*] or better, if the sensor is ported to use pm_runtime first (by
> > dropping support for the deprecated .s_power(), or do you need
> > s_power() on your platform ?) you can use pm_runtime_get_if_in_use()
> > instead of keeping track manually of the is_enabled flag.
>
> What you are describing above is pretty much exactly what is done
> later in this patchset.
>
> Patches 1-8 are there to fix some glaring bugs, with the possibility
> of backporting the fixes which is why they are the first patches
> in the set.
>
> Then after patch 9 the real work to get this driver into shape begins :)

Now that I see "media: ov2680: Add runtime-pm support" in the series
it all makes sense :)

Thanks and sorry for the noise!

(not adding my redundant tags, just skimming through the patches)

>
> Regards,
>
> Hans
>
>
>
>
>
> >> for a nice code cleanup.
> >>
> >> Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
> >> Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
> >> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> >> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> >> ---
> >>  drivers/media/i2c/ov2680.c | 50 ++++++++++----------------------------
> >>  1 file changed, 13 insertions(+), 37 deletions(-)
> >>
> >> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> >> index 2001e08253ef..c93810f84ed7 100644
> >> --- a/drivers/media/i2c/ov2680.c
> >> +++ b/drivers/media/i2c/ov2680.c
> >> @@ -328,11 +328,15 @@ static void ov2680_set_bayer_order(struct ov2680_dev *sensor)
> >>  	sensor->fmt.code = ov2680_hv_flip_bayer_order[hv_flip];
> >>  }
> >>
> >> -static int ov2680_vflip_enable(struct ov2680_dev *sensor)
> >> +static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
> >>  {
> >>  	int ret;
> >>
> >> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(2));
> >> +	if (sensor->is_streaming)
> >> +		return -EBUSY;
> >> +
> >> +	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1,
> >> +			     BIT(2), val ? BIT(2) : 0);
> >>  	if (ret < 0)
> >>  		return ret;
> >>
> >> @@ -340,33 +344,15 @@ static int ov2680_vflip_enable(struct ov2680_dev *sensor)
> >>  	return 0;
> >>  }
> >>
> >> -static int ov2680_vflip_disable(struct ov2680_dev *sensor)
> >> +static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
> >>  {
> >>  	int ret;
> >>
> >> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1, BIT(2), BIT(0));
> >> -	if (ret < 0)
> >> -		return ret;
> >> +	if (sensor->is_streaming)
> >> +		return -EBUSY;
> >>
> >> -	return ov2680_bayer_order(sensor);
> >> -}
> >> -
> >> -static int ov2680_hflip_enable(struct ov2680_dev *sensor)
> >> -{
> >> -	int ret;
> >> -
> >> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(2));
> >> -	if (ret < 0)
> >> -		return ret;
> >> -
> >> -	return ov2680_bayer_order(sensor);
> >> -}
> >> -
> >> -static int ov2680_hflip_disable(struct ov2680_dev *sensor)
> >> -{
> >> -	int ret;
> >> -
> >> -	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2, BIT(2), BIT(0));
> >> +	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2,
> >> +			     BIT(2), val ? BIT(2) : 0);
> >>  	if (ret < 0)
> >>  		return ret;
> >>
> >> @@ -720,19 +706,9 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
> >>  	case V4L2_CID_EXPOSURE:
> >>  		return ov2680_exposure_set(sensor, ctrl->val);
> >>  	case V4L2_CID_VFLIP:
> >> -		if (sensor->is_streaming)
> >> -			return -EBUSY;
> >> -		if (ctrl->val)
> >> -			return ov2680_vflip_enable(sensor);
> >> -		else
> >> -			return ov2680_vflip_disable(sensor);
> >> +		return ov2680_set_vflip(sensor, ctrl->val);
> >>  	case V4L2_CID_HFLIP:
> >> -		if (sensor->is_streaming)
> >> -			return -EBUSY;
> >> -		if (ctrl->val)
> >> -			return ov2680_hflip_enable(sensor);
> >> -		else
> >> -			return ov2680_hflip_disable(sensor);
> >> +		return ov2680_set_hflip(sensor, ctrl->val);
> >>  	case V4L2_CID_TEST_PATTERN:
> >>  		return ov2680_test_pattern_set(sensor, ctrl->val);
> >>  	default:
> >> --
> >> 2.41.0
> >>
> >
>

[PATCH v3 04/29] media: ov2680: Use select VIDEO_V4L2_SUBDEV_API

[Hans de Goede]

Select VIDEO_V4L2_SUBDEV_API in Kconfig and drop the
ifdef CONFIG_VIDEO_V4L2_SUBDEV_API checks, like other callers
of v4l2_subdev_get_try_format() do.

This is a preparation patch for fixing ov2680_set_fmt()
which == V4L2_SUBDEV_FORMAT_TRY calls not properly filling in
the passed in v4l2_mbus_framefmt struct.

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/Kconfig  |  1 +
 drivers/media/i2c/ov2680.c | 16 ++--------------
 2 files changed, 3 insertions(+), 14 deletions(-)

diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
index 6365c15bc4d4..1a6598561cf8 100644
--- a/drivers/media/i2c/Kconfig
+++ b/drivers/media/i2c/Kconfig
@@ -447,6 +447,7 @@ config VIDEO_OV2680
 	tristate "OmniVision OV2680 sensor support"
 	depends on VIDEO_DEV && I2C
 	select MEDIA_CONTROLLER
+	select VIDEO_V4L2_SUBDEV_API
 	select V4L2_FWNODE
 	help
 	  This is a Video4Linux2 sensor driver for the OmniVision
diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index c93810f84ed7..f6297874af3b 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -562,7 +562,6 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 	struct v4l2_mbus_framefmt *fmt = NULL;
-	int ret = 0;
 
 	if (format->pad != 0)
 		return -EINVAL;
@@ -570,22 +569,17 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
 	mutex_lock(&sensor->lock);
 
 	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
-#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
 		fmt = v4l2_subdev_get_try_format(&sensor->sd, sd_state,
 						 format->pad);
-#else
-		ret = -EINVAL;
-#endif
 	} else {
 		fmt = &sensor->fmt;
 	}
 
-	if (fmt)
-		format->format = *fmt;
+	format->format = *fmt;
 
 	mutex_unlock(&sensor->lock);
 
-	return ret;
+	return 0;
 }
 
 static int ov2680_set_fmt(struct v4l2_subdev *sd,
@@ -594,9 +588,7 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 	struct v4l2_mbus_framefmt *fmt = &format->format;
-#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
 	struct v4l2_mbus_framefmt *try_fmt;
-#endif
 	const struct ov2680_mode_info *mode;
 	int ret = 0;
 
@@ -619,10 +611,8 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 	}
 
 	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
-#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
 		try_fmt = v4l2_subdev_get_try_format(sd, sd_state, 0);
 		format->format = *try_fmt;
-#endif
 		goto unlock;
 	}
 
@@ -780,9 +770,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 	v4l2_i2c_subdev_init(&sensor->sd, sensor->i2c_client,
 			     &ov2680_subdev_ops);
 
-#ifdef CONFIG_VIDEO_V4L2_SUBDEV_API
 	sensor->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
-#endif
 	sensor->pad.flags = MEDIA_PAD_FL_SOURCE;
 	sensor->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
 
-- 
2.41.0

[PATCH v3 05/29] media: ov2680: Don't take the lock for try_fmt calls

[Hans de Goede]

On ov2680_set_fmt() calls with format->which == V4L2_SUBDEV_FORMAT_TRY,
ov2680_set_fmt() does not talk to the sensor.

So in this case there is no need to lock the sensor->lock mutex or
to check that the sensor is streaming.

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index f6297874af3b..2b20990f4cf5 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -595,24 +595,22 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 	if (format->pad != 0)
 		return -EINVAL;
 
-	mutex_lock(&sensor->lock);
-
-	if (sensor->is_streaming) {
-		ret = -EBUSY;
-		goto unlock;
-	}
-
 	mode = v4l2_find_nearest_size(ov2680_mode_data,
 				      ARRAY_SIZE(ov2680_mode_data), width,
 				      height, fmt->width, fmt->height);
-	if (!mode) {
-		ret = -EINVAL;
-		goto unlock;
-	}
+	if (!mode)
+		return -EINVAL;
 
 	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
 		try_fmt = v4l2_subdev_get_try_format(sd, sd_state, 0);
 		format->format = *try_fmt;
+		return 0;
+	}
+
+	mutex_lock(&sensor->lock);
+
+	if (sensor->is_streaming) {
+		ret = -EBUSY;
 		goto unlock;
 	}
 
-- 
2.41.0

[PATCH v3 06/29] media: ov2680: Add ov2680_fill_format() helper function

[Hans de Goede]

Add a ov2680_fill_format() helper function and use this everywhere were
a v4l2_mbus_framefmt struct needs to be filled in so that the driver always
fills it consistently.

This is a preparation patch for fixing ov2680_set_fmt()
which == V4L2_SUBDEV_FORMAT_TRY calls not properly filling in
the passed in v4l2_mbus_framefmt struct.

Note that for ov2680_init_cfg() this now simply always fills
the try_fmt struct of the passed in sd_state. This is correct because
ov2680_init_cfg() is never called with a NULL sd_state so the old
sd_state check is not necessary.

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 49 +++++++++++++++++++++-----------------
 1 file changed, 27 insertions(+), 22 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 2b20990f4cf5..c4a46c734d82 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -54,6 +54,9 @@
 #define OV2680_WIDTH_MAX		1600
 #define OV2680_HEIGHT_MAX		1200
 
+#define OV2680_DEFAULT_WIDTH			800
+#define OV2680_DEFAULT_HEIGHT			600
+
 enum ov2680_mode_id {
 	OV2680_MODE_QUXGA_800_600,
 	OV2680_MODE_720P_1280_720,
@@ -315,7 +318,8 @@ static void ov2680_power_down(struct ov2680_dev *sensor)
 	usleep_range(5000, 10000);
 }
 
-static void ov2680_set_bayer_order(struct ov2680_dev *sensor)
+static void ov2680_set_bayer_order(struct ov2680_dev *sensor,
+				   struct v4l2_mbus_framefmt *fmt)
 {
 	int hv_flip = 0;
 
@@ -325,7 +329,19 @@ static void ov2680_set_bayer_order(struct ov2680_dev *sensor)
 	if (sensor->ctrls.hflip && sensor->ctrls.hflip->val)
 		hv_flip += 2;
 
-	sensor->fmt.code = ov2680_hv_flip_bayer_order[hv_flip];
+	fmt->code = ov2680_hv_flip_bayer_order[hv_flip];
+}
+
+static void ov2680_fill_format(struct ov2680_dev *sensor,
+			       struct v4l2_mbus_framefmt *fmt,
+			       unsigned int width, unsigned int height)
+{
+	memset(fmt, 0, sizeof(*fmt));
+	fmt->width = width;
+	fmt->height = height;
+	fmt->field = V4L2_FIELD_NONE;
+	fmt->colorspace = V4L2_COLORSPACE_SRGB;
+	ov2680_set_bayer_order(sensor, fmt);
 }
 
 static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
@@ -340,7 +356,7 @@ static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
 	if (ret < 0)
 		return ret;
 
-	ov2680_set_bayer_order(sensor);
+	ov2680_set_bayer_order(sensor, &sensor->fmt);
 	return 0;
 }
 
@@ -356,7 +372,7 @@ static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
 	if (ret < 0)
 		return ret;
 
-	ov2680_set_bayer_order(sensor);
+	ov2680_set_bayer_order(sensor, &sensor->fmt);
 	return 0;
 }
 
@@ -614,10 +630,7 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 		goto unlock;
 	}
 
-	fmt->width = mode->width;
-	fmt->height = mode->height;
-	fmt->code = sensor->fmt.code;
-	fmt->colorspace = sensor->fmt.colorspace;
+	ov2680_fill_format(sensor, fmt, mode->width, mode->height);
 
 	sensor->current_mode = mode;
 	sensor->fmt = format->format;
@@ -632,16 +645,11 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 static int ov2680_init_cfg(struct v4l2_subdev *sd,
 			   struct v4l2_subdev_state *sd_state)
 {
-	struct v4l2_subdev_format fmt = {
-		.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY
-		: V4L2_SUBDEV_FORMAT_ACTIVE,
-		.format = {
-			.width = 800,
-			.height = 600,
-		}
-	};
+	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 
-	return ov2680_set_fmt(sd, sd_state, &fmt);
+	ov2680_fill_format(sensor, &sd_state->pads[0].try_fmt,
+			   OV2680_DEFAULT_WIDTH, OV2680_DEFAULT_HEIGHT);
+	return 0;
 }
 
 static int ov2680_enum_frame_size(struct v4l2_subdev *sd,
@@ -740,11 +748,8 @@ static int ov2680_mode_init(struct ov2680_dev *sensor)
 	const struct ov2680_mode_info *init_mode;
 
 	/* set initial mode */
-	sensor->fmt.code = MEDIA_BUS_FMT_SBGGR10_1X10;
-	sensor->fmt.width = 800;
-	sensor->fmt.height = 600;
-	sensor->fmt.field = V4L2_FIELD_NONE;
-	sensor->fmt.colorspace = V4L2_COLORSPACE_SRGB;
+	ov2680_fill_format(sensor, &sensor->fmt,
+			   OV2680_DEFAULT_WIDTH, OV2680_DEFAULT_HEIGHT);
 
 	sensor->frame_interval.denominator = OV2680_FRAME_RATE;
 	sensor->frame_interval.numerator = 1;
-- 
2.41.0

[PATCH v3 07/29] media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY not working

[Hans de Goede]

ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY was getting
the try_fmt v4l2_mbus_framefmt struct from the passed in sd_state
and then storing the contents of that into the return by reference
format->format struct.

While the right thing to do would be filling format->format based on
the just looked up mode and then store the results of that in
sd_state->pads[0].try_fmt .

Before the previous change introducing ov2680_fill_format() this
resulted in ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY always
returning the zero-ed out sd_state->pads[0].try_fmt in format->format
breaking callers using this.

After the introduction of ov2680_fill_format() which at least
initializes sd_state->pads[0].try_fmt properly, format->format
is now always being filled with the default 800x600 mode set by
ov2680_init_cfg() independent of the actual requested mode.

Move the filling of format->format with ov2680_fill_format() to
before the if (which == V4L2_SUBDEV_FORMAT_TRY) and then store
the filled in format->format in sd_state->pads[0].try_fmt to
fix this.

Note this removes the fmt local variable because IMHO having a local
variable which points to a sub-struct of one of the function arguments
just leads to confusion when reading the code.

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index c4a46c734d82..7fc4b39ebb37 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -603,7 +603,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 			  struct v4l2_subdev_format *format)
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
-	struct v4l2_mbus_framefmt *fmt = &format->format;
 	struct v4l2_mbus_framefmt *try_fmt;
 	const struct ov2680_mode_info *mode;
 	int ret = 0;
@@ -612,14 +611,18 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 		return -EINVAL;
 
 	mode = v4l2_find_nearest_size(ov2680_mode_data,
-				      ARRAY_SIZE(ov2680_mode_data), width,
-				      height, fmt->width, fmt->height);
+				      ARRAY_SIZE(ov2680_mode_data),
+				      width, height,
+				      format->format.width,
+				      format->format.height);
 	if (!mode)
 		return -EINVAL;
 
+	ov2680_fill_format(sensor, &format->format, mode->width, mode->height);
+
 	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
 		try_fmt = v4l2_subdev_get_try_format(sd, sd_state, 0);
-		format->format = *try_fmt;
+		*try_fmt = format->format;
 		return 0;
 	}
 
@@ -630,8 +633,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 		goto unlock;
 	}
 
-	ov2680_fill_format(sensor, fmt, mode->width, mode->height);
-
 	sensor->current_mode = mode;
 	sensor->fmt = format->format;
 	sensor->mode_pending_changes = true;
-- 
2.41.0

Re: [PATCH v3 07/29] media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY not working

[Dan Scally]

Hi Hans

On 27/06/2023 15:18, Hans de Goede wrote:
> ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY was getting
> the try_fmt v4l2_mbus_framefmt struct from the passed in sd_state
> and then storing the contents of that into the return by reference
> format->format struct.
>
> While the right thing to do would be filling format->format based on
> the just looked up mode and then store the results of that in
> sd_state->pads[0].try_fmt .
>
> Before the previous change introducing ov2680_fill_format() this
> resulted in ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY always
> returning the zero-ed out sd_state->pads[0].try_fmt in format->format
> breaking callers using this.
>
> After the introduction of ov2680_fill_format() which at least
> initializes sd_state->pads[0].try_fmt properly, format->format
> is now always being filled with the default 800x600 mode set by
> ov2680_init_cfg() independent of the actual requested mode.
>
> Move the filling of format->format with ov2680_fill_format() to
> before the if (which == V4L2_SUBDEV_FORMAT_TRY) and then store
> the filled in format->format in sd_state->pads[0].try_fmt to
> fix this.
>
> Note this removes the fmt local variable because IMHO having a local
> variable which points to a sub-struct of one of the function arguments
> just leads to confusion when reading the code.
>
> Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---


For #5, #6 and this one:


Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>


>   drivers/media/i2c/ov2680.c | 13 +++++++------
>   1 file changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index c4a46c734d82..7fc4b39ebb37 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -603,7 +603,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>   			  struct v4l2_subdev_format *format)
>   {
>   	struct ov2680_dev *sensor = to_ov2680_dev(sd);
> -	struct v4l2_mbus_framefmt *fmt = &format->format;
>   	struct v4l2_mbus_framefmt *try_fmt;
>   	const struct ov2680_mode_info *mode;
>   	int ret = 0;
> @@ -612,14 +611,18 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>   		return -EINVAL;
>   
>   	mode = v4l2_find_nearest_size(ov2680_mode_data,
> -				      ARRAY_SIZE(ov2680_mode_data), width,
> -				      height, fmt->width, fmt->height);
> +				      ARRAY_SIZE(ov2680_mode_data),
> +				      width, height,
> +				      format->format.width,
> +				      format->format.height);
>   	if (!mode)
>   		return -EINVAL;
>   
> +	ov2680_fill_format(sensor, &format->format, mode->width, mode->height);
> +
>   	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
>   		try_fmt = v4l2_subdev_get_try_format(sd, sd_state, 0);
> -		format->format = *try_fmt;
> +		*try_fmt = format->format;
>   		return 0;
>   	}
>   
> @@ -630,8 +633,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>   		goto unlock;
>   	}
>   
> -	ov2680_fill_format(sensor, fmt, mode->width, mode->height);
> -
>   	sensor->current_mode = mode;
>   	sensor->fmt = format->format;
>   	sensor->mode_pending_changes = true;

Re: [PATCH v3 07/29] media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY not working

[Jacopo Mondi]

Hi Hans

On Tue, Jun 27, 2023 at 03:18:08PM +0200, Hans de Goede wrote:
> ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY was getting
> the try_fmt v4l2_mbus_framefmt struct from the passed in sd_state
> and then storing the contents of that into the return by reference
> format->format struct.
>
> While the right thing to do would be filling format->format based on
> the just looked up mode and then store the results of that in
> sd_state->pads[0].try_fmt .
>
> Before the previous change introducing ov2680_fill_format() this
> resulted in ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY always
> returning the zero-ed out sd_state->pads[0].try_fmt in format->format
> breaking callers using this.
>
> After the introduction of ov2680_fill_format() which at least
> initializes sd_state->pads[0].try_fmt properly, format->format
> is now always being filled with the default 800x600 mode set by
> ov2680_init_cfg() independent of the actual requested mode.
>
> Move the filling of format->format with ov2680_fill_format() to
> before the if (which == V4L2_SUBDEV_FORMAT_TRY) and then store
> the filled in format->format in sd_state->pads[0].try_fmt to
> fix this.
>
> Note this removes the fmt local variable because IMHO having a local
> variable which points to a sub-struct of one of the function arguments
> just leads to confusion when reading the code.
>
> Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>  drivers/media/i2c/ov2680.c | 13 +++++++------
>  1 file changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index c4a46c734d82..7fc4b39ebb37 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -603,7 +603,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>  			  struct v4l2_subdev_format *format)
>  {
>  	struct ov2680_dev *sensor = to_ov2680_dev(sd);
> -	struct v4l2_mbus_framefmt *fmt = &format->format;
>  	struct v4l2_mbus_framefmt *try_fmt;
>  	const struct ov2680_mode_info *mode;
>  	int ret = 0;
> @@ -612,14 +611,18 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>  		return -EINVAL;
>
>  	mode = v4l2_find_nearest_size(ov2680_mode_data,
> -				      ARRAY_SIZE(ov2680_mode_data), width,
> -				      height, fmt->width, fmt->height);
> +				      ARRAY_SIZE(ov2680_mode_data),
> +				      width, height,
> +				      format->format.width,
> +				      format->format.height);
>  	if (!mode)
>  		return -EINVAL;

Nit: only if you have to resend, could this be dropped? mode will be NULL
only if ov2680_mode_data[] has no entries.

>
> +	ov2680_fill_format(sensor, &format->format, mode->width, mode->height);
> +
>  	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
>  		try_fmt = v4l2_subdev_get_try_format(sd, sd_state, 0);
> -		format->format = *try_fmt;
> +		*try_fmt = format->format;
>  		return 0;
>  	}
>
> @@ -630,8 +633,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>  		goto unlock;
>  	}
>
> -	ov2680_fill_format(sensor, fmt, mode->width, mode->height);
> -
>  	sensor->current_mode = mode;
>  	sensor->fmt = format->format;
>  	sensor->mode_pending_changes = true;
> --
> 2.41.0
>

Re: [PATCH v3 07/29] media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY not working

[Andy Shevchenko]

On Tue, Jun 27, 2023 at 05:08:39PM +0200, Jacopo Mondi wrote:
> On Tue, Jun 27, 2023 at 03:18:08PM +0200, Hans de Goede wrote:

...

> >  	mode = v4l2_find_nearest_size(ov2680_mode_data,
> > -				      ARRAY_SIZE(ov2680_mode_data), width,
> > -				      height, fmt->width, fmt->height);
> > +				      ARRAY_SIZE(ov2680_mode_data),
> > +				      width, height,
> > +				      format->format.width,
> > +				      format->format.height);
> >  	if (!mode)
> >  		return -EINVAL;
> 
> Nit: only if you have to resend, could this be dropped? mode will be NULL
> only if ov2680_mode_data[] has no entries.

We shouldn't rely on the implementation details of some API if it's not
advertised that way. Even if it is, the robustness of the code is better with
this check.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH v3 07/29] media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY not working

[Hans de Goede]

Hi,

On 6/27/23 18:55, Andy Shevchenko wrote:
> On Tue, Jun 27, 2023 at 05:08:39PM +0200, Jacopo Mondi wrote:
>> On Tue, Jun 27, 2023 at 03:18:08PM +0200, Hans de Goede wrote:
> 
> ...
> 
>>>  	mode = v4l2_find_nearest_size(ov2680_mode_data,
>>> -				      ARRAY_SIZE(ov2680_mode_data), width,
>>> -				      height, fmt->width, fmt->height);
>>> +				      ARRAY_SIZE(ov2680_mode_data),
>>> +				      width, height,
>>> +				      format->format.width,
>>> +				      format->format.height);
>>>  	if (!mode)
>>>  		return -EINVAL;
>>
>> Nit: only if you have to resend, could this be dropped? mode will be NULL
>> only if ov2680_mode_data[] has no entries.
> 
> We shouldn't rely on the implementation details of some API if it's not
> advertised that way. Even if it is, the robustness of the code is better with
> this check.

Note non of this is really important (as Jacopo already mentioned
it is just a nitpick) since this code is completely removed in
a later patch.

Also note that the check was already there, so removing it
would be out of scope for this patch.

Regards,

Hans

Re: [PATCH v3 07/29] media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY not working

[Jacopo Mondi]

Hi Andy

On Tue, Jun 27, 2023 at 07:55:48PM +0300, Andy Shevchenko wrote:
> On Tue, Jun 27, 2023 at 05:08:39PM +0200, Jacopo Mondi wrote:
> > On Tue, Jun 27, 2023 at 03:18:08PM +0200, Hans de Goede wrote:
>
> ...
>
> > >  	mode = v4l2_find_nearest_size(ov2680_mode_data,
> > > -				      ARRAY_SIZE(ov2680_mode_data), width,
> > > -				      height, fmt->width, fmt->height);
> > > +				      ARRAY_SIZE(ov2680_mode_data),
> > > +				      width, height,
> > > +				      format->format.width,
> > > +				      format->format.height);
> > >  	if (!mode)
> > >  		return -EINVAL;
> >
> > Nit: only if you have to resend, could this be dropped? mode will be NULL
> > only if ov2680_mode_data[] has no entries.
>
> We shouldn't rely on the implementation details of some API if it's not
> advertised that way. Even if it is, the robustness of the code is better with
> this check.

I don't 100% agree here. the s_fmt subdev operation is not supposed to
return -EINVAL if the requested configuration cannot be found. So when
I saw -EINVAL here it throw me off a little, until I didn't look into
v4l2_find_nearest_size() implementation and realized this can't
actually happen.

But as Hans said, this is going away in the next patches, something I
didn't notice when I wrote this comment, so no need to bother :)

>
> --
> With Best Regards,
> Andy Shevchenko
>
>

[PATCH v3 08/29] media: ov2680: Fix regulators being left enabled on ov2680_power_on() errors

[Hans de Goede]

When the ov2680_power_on() "sensor soft reset failed" path is hit during
probe() the WARN() about putting an enabled regulator at
drivers/regulator/core.c:2398 triggers 3 times (once for each regulator),
filling dmesg with backtraces.

Fix this by properly disabling the regulators on ov2680_power_on() errors.

Fixes: 3ee47cad3e69 ("media: ov2680: Add Omnivision OV2680 sensor driver")
Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>
Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 7fc4b39ebb37..55fc56ffad31 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -475,7 +475,7 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 		ret = ov2680_write_reg(sensor, OV2680_REG_SOFT_RESET, 0x01);
 		if (ret != 0) {
 			dev_err(dev, "sensor soft reset failed\n");
-			return ret;
+			goto err_disable_regulators;
 		}
 		usleep_range(1000, 2000);
 	} else {
@@ -485,7 +485,7 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 
 	ret = clk_prepare_enable(sensor->xvclk);
 	if (ret < 0)
-		return ret;
+		goto err_disable_regulators;
 
 	sensor->is_enabled = true;
 
@@ -495,6 +495,10 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 	ov2680_stream_disable(sensor);
 
 	return 0;
+
+err_disable_regulators:
+	regulator_bulk_disable(OV2680_NUM_SUPPLIES, sensor->supplies);
+	return ret;
 }
 
 static int ov2680_s_power(struct v4l2_subdev *sd, int on)
-- 
2.41.0

[PATCH v3 09/29] media: ov2680: Convert to new CCI register access helpers

[Hans de Goede]

Use the new comon CCI register access helpers to replace the private
register access helpers in the ov2680 driver.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/Kconfig  |   1 +
 drivers/media/i2c/ov2680.c | 224 ++++++++++---------------------------
 2 files changed, 58 insertions(+), 167 deletions(-)

diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
index 1a6598561cf8..7e21221fb9b1 100644
--- a/drivers/media/i2c/Kconfig
+++ b/drivers/media/i2c/Kconfig
@@ -449,6 +449,7 @@ config VIDEO_OV2680
 	select MEDIA_CONTROLLER
 	select VIDEO_V4L2_SUBDEV_API
 	select V4L2_FWNODE
+	select V4L2_CCI
 	help
 	  This is a Video4Linux2 sensor driver for the OmniVision
 	  OV2680 camera.
diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 55fc56ffad31..fb25ba446d52 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -10,49 +10,45 @@
  *
  */
 
-#include <asm/unaligned.h>
 #include <linux/clk.h>
 #include <linux/delay.h>
 #include <linux/err.h>
+#include <linux/gpio/consumer.h>
 #include <linux/i2c.h>
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/of_device.h>
-#include <linux/gpio/consumer.h>
+#include <linux/regmap.h>
 #include <linux/regulator/consumer.h>
 
+#include <media/v4l2-cci.h>
 #include <media/v4l2-common.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-subdev.h>
 
-#define OV2680_XVCLK_VALUE	24000000
+#define OV2680_XVCLK_VALUE			24000000
 
-#define OV2680_CHIP_ID		0x2680
+#define OV2680_CHIP_ID				0x2680
 
-#define OV2680_REG_STREAM_CTRL		0x0100
-#define OV2680_REG_SOFT_RESET		0x0103
+#define OV2680_REG_STREAM_CTRL			CCI_REG8(0x0100)
+#define OV2680_REG_SOFT_RESET			CCI_REG8(0x0103)
 
-#define OV2680_REG_CHIP_ID_HIGH		0x300a
-#define OV2680_REG_CHIP_ID_LOW		0x300b
+#define OV2680_REG_CHIP_ID			CCI_REG16(0x300a)
 
-#define OV2680_REG_R_MANUAL		0x3503
-#define OV2680_REG_GAIN_PK		0x350a
-#define OV2680_REG_EXPOSURE_PK_HIGH	0x3500
-#define OV2680_REG_TIMING_HTS		0x380c
-#define OV2680_REG_TIMING_VTS		0x380e
-#define OV2680_REG_FORMAT1		0x3820
-#define OV2680_REG_FORMAT2		0x3821
+#define OV2680_REG_EXPOSURE_PK			CCI_REG24(0x3500)
+#define OV2680_REG_R_MANUAL			CCI_REG8(0x3503)
+#define OV2680_REG_GAIN_PK			CCI_REG16(0x350a)
+#define OV2680_REG_TIMING_HTS			CCI_REG16(0x380c)
+#define OV2680_REG_TIMING_VTS			CCI_REG16(0x380e)
+#define OV2680_REG_FORMAT1			CCI_REG8(0x3820)
+#define OV2680_REG_FORMAT2			CCI_REG8(0x3821)
 
-#define OV2680_REG_ISP_CTRL00		0x5080
+#define OV2680_REG_ISP_CTRL00			CCI_REG8(0x5080)
 
-#define OV2680_FRAME_RATE		30
+#define OV2680_FRAME_RATE			30
 
-#define OV2680_REG_VALUE_8BIT		1
-#define OV2680_REG_VALUE_16BIT		2
-#define OV2680_REG_VALUE_24BIT		3
-
-#define OV2680_WIDTH_MAX		1600
-#define OV2680_HEIGHT_MAX		1200
+#define OV2680_WIDTH_MAX			1600
+#define OV2680_HEIGHT_MAX			1200
 
 #define OV2680_DEFAULT_WIDTH			800
 #define OV2680_DEFAULT_HEIGHT			600
@@ -64,11 +60,6 @@ enum ov2680_mode_id {
 	OV2680_MODE_MAX,
 };
 
-struct reg_value {
-	u16 reg_addr;
-	u8 val;
-};
-
 static const char * const ov2680_supply_name[] = {
 	"DOVDD",
 	"DVDD",
@@ -82,7 +73,7 @@ struct ov2680_mode_info {
 	enum ov2680_mode_id id;
 	u32 width;
 	u32 height;
-	const struct reg_value *reg_data;
+	const struct reg_sequence *reg_data;
 	u32 reg_data_size;
 };
 
@@ -97,6 +88,7 @@ struct ov2680_ctrls {
 
 struct ov2680_dev {
 	struct i2c_client		*i2c_client;
+	struct regmap			*regmap;
 	struct v4l2_subdev		sd;
 
 	struct media_pad		pad;
@@ -133,7 +125,7 @@ static const int ov2680_hv_flip_bayer_order[] = {
 	MEDIA_BUS_FMT_SRGGB10_1X10,
 };
 
-static const struct reg_value ov2680_setting_30fps_QUXGA_800_600[] = {
+static const struct reg_sequence ov2680_setting_30fps_QUXGA_800_600[] = {
 	{0x3086, 0x01}, {0x370a, 0x23}, {0x3808, 0x03}, {0x3809, 0x20},
 	{0x380a, 0x02}, {0x380b, 0x58}, {0x380c, 0x06}, {0x380d, 0xac},
 	{0x380e, 0x02}, {0x380f, 0x84}, {0x3811, 0x04}, {0x3813, 0x04},
@@ -142,14 +134,14 @@ static const struct reg_value ov2680_setting_30fps_QUXGA_800_600[] = {
 	{0x3503, 0x03},
 };
 
-static const struct reg_value ov2680_setting_30fps_720P_1280_720[] = {
+static const struct reg_sequence ov2680_setting_30fps_720P_1280_720[] = {
 	{0x3086, 0x00}, {0x3808, 0x05}, {0x3809, 0x00}, {0x380a, 0x02},
 	{0x380b, 0xd0}, {0x380c, 0x06}, {0x380d, 0xa8}, {0x380e, 0x05},
 	{0x380f, 0x0e}, {0x3811, 0x08}, {0x3813, 0x06}, {0x3814, 0x11},
 	{0x3815, 0x11}, {0x3820, 0xc0}, {0x4008, 0x00},
 };
 
-static const struct reg_value ov2680_setting_30fps_UXGA_1600_1200[] = {
+static const struct reg_sequence ov2680_setting_30fps_UXGA_1600_1200[] = {
 	{0x3086, 0x00}, {0x3501, 0x4e}, {0x3502, 0xe0}, {0x3808, 0x06},
 	{0x3809, 0x40}, {0x380a, 0x04}, {0x380b, 0xb0}, {0x380c, 0x06},
 	{0x380d, 0xa8}, {0x380e, 0x05}, {0x380f, 0x0e}, {0x3811, 0x00},
@@ -191,115 +183,6 @@ static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
 			     ctrls.handler)->sd;
 }
 
-static int __ov2680_write_reg(struct ov2680_dev *sensor, u16 reg,
-			      unsigned int len, u32 val)
-{
-	struct i2c_client *client = sensor->i2c_client;
-	u8 buf[6];
-	int ret;
-
-	if (len > 4)
-		return -EINVAL;
-
-	put_unaligned_be16(reg, buf);
-	put_unaligned_be32(val << (8 * (4 - len)), buf + 2);
-	ret = i2c_master_send(client, buf, len + 2);
-	if (ret != len + 2) {
-		dev_err(&client->dev, "write error: reg=0x%4x: %d\n", reg, ret);
-		return -EIO;
-	}
-
-	return 0;
-}
-
-#define ov2680_write_reg(s, r, v) \
-	__ov2680_write_reg(s, r, OV2680_REG_VALUE_8BIT, v)
-
-#define ov2680_write_reg16(s, r, v) \
-	__ov2680_write_reg(s, r, OV2680_REG_VALUE_16BIT, v)
-
-#define ov2680_write_reg24(s, r, v) \
-	__ov2680_write_reg(s, r, OV2680_REG_VALUE_24BIT, v)
-
-static int __ov2680_read_reg(struct ov2680_dev *sensor, u16 reg,
-			     unsigned int len, u32 *val)
-{
-	struct i2c_client *client = sensor->i2c_client;
-	struct i2c_msg msgs[2];
-	u8 addr_buf[2] = { reg >> 8, reg & 0xff };
-	u8 data_buf[4] = { 0, };
-	int ret;
-
-	if (len > 4)
-		return -EINVAL;
-
-	msgs[0].addr = client->addr;
-	msgs[0].flags = 0;
-	msgs[0].len = ARRAY_SIZE(addr_buf);
-	msgs[0].buf = addr_buf;
-
-	msgs[1].addr = client->addr;
-	msgs[1].flags = I2C_M_RD;
-	msgs[1].len = len;
-	msgs[1].buf = &data_buf[4 - len];
-
-	ret = i2c_transfer(client->adapter, msgs, ARRAY_SIZE(msgs));
-	if (ret != ARRAY_SIZE(msgs)) {
-		dev_err(&client->dev, "read error: reg=0x%4x: %d\n", reg, ret);
-		return -EIO;
-	}
-
-	*val = get_unaligned_be32(data_buf);
-
-	return 0;
-}
-
-#define ov2680_read_reg(s, r, v) \
-	__ov2680_read_reg(s, r, OV2680_REG_VALUE_8BIT, v)
-
-#define ov2680_read_reg16(s, r, v) \
-	__ov2680_read_reg(s, r, OV2680_REG_VALUE_16BIT, v)
-
-#define ov2680_read_reg24(s, r, v) \
-	__ov2680_read_reg(s, r, OV2680_REG_VALUE_24BIT, v)
-
-static int ov2680_mod_reg(struct ov2680_dev *sensor, u16 reg, u8 mask, u8 val)
-{
-	u32 readval;
-	int ret;
-
-	ret = ov2680_read_reg(sensor, reg, &readval);
-	if (ret < 0)
-		return ret;
-
-	readval &= ~mask;
-	val &= mask;
-	val |= readval;
-
-	return ov2680_write_reg(sensor, reg, val);
-}
-
-static int ov2680_load_regs(struct ov2680_dev *sensor,
-			    const struct ov2680_mode_info *mode)
-{
-	const struct reg_value *regs = mode->reg_data;
-	unsigned int i;
-	int ret = 0;
-	u16 reg_addr;
-	u8 val;
-
-	for (i = 0; i < mode->reg_data_size; ++i, ++regs) {
-		reg_addr = regs->reg_addr;
-		val = regs->val;
-
-		ret = ov2680_write_reg(sensor, reg_addr, val);
-		if (ret)
-			break;
-	}
-
-	return ret;
-}
-
 static void ov2680_power_up(struct ov2680_dev *sensor)
 {
 	if (!sensor->reset_gpio)
@@ -351,8 +234,8 @@ static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
 	if (sensor->is_streaming)
 		return -EBUSY;
 
-	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT1,
-			     BIT(2), val ? BIT(2) : 0);
+	ret = cci_update_bits(sensor->regmap, OV2680_REG_FORMAT1,
+			      BIT(2), val ? BIT(2) : 0, NULL);
 	if (ret < 0)
 		return ret;
 
@@ -367,8 +250,8 @@ static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
 	if (sensor->is_streaming)
 		return -EBUSY;
 
-	ret = ov2680_mod_reg(sensor, OV2680_REG_FORMAT2,
-			     BIT(2), val ? BIT(2) : 0);
+	ret = cci_update_bits(sensor->regmap, OV2680_REG_FORMAT2,
+			      BIT(2), val ? BIT(2) : 0, NULL);
 	if (ret < 0)
 		return ret;
 
@@ -378,48 +261,48 @@ static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
 
 static int ov2680_test_pattern_set(struct ov2680_dev *sensor, int value)
 {
-	int ret;
+	int ret = 0;
 
 	if (!value)
-		return ov2680_mod_reg(sensor, OV2680_REG_ISP_CTRL00, BIT(7), 0);
+		return cci_update_bits(sensor->regmap, OV2680_REG_ISP_CTRL00,
+				       BIT(7), 0, NULL);
 
-	ret = ov2680_mod_reg(sensor, OV2680_REG_ISP_CTRL00, 0x03, value - 1);
-	if (ret < 0)
-		return ret;
+	cci_update_bits(sensor->regmap, OV2680_REG_ISP_CTRL00,
+			0x03, value - 1, &ret);
+	cci_update_bits(sensor->regmap, OV2680_REG_ISP_CTRL00,
+			BIT(7), BIT(7), &ret);
 
-	ret = ov2680_mod_reg(sensor, OV2680_REG_ISP_CTRL00, BIT(7), BIT(7));
-	if (ret < 0)
-		return ret;
-
-	return 0;
+	return ret;
 }
 
 static int ov2680_gain_set(struct ov2680_dev *sensor, u32 gain)
 {
-	return ov2680_write_reg16(sensor, OV2680_REG_GAIN_PK, gain);
+	return cci_write(sensor->regmap, OV2680_REG_GAIN_PK, gain, NULL);
 }
 
 static int ov2680_exposure_set(struct ov2680_dev *sensor, u32 exp)
 {
-	return ov2680_write_reg24(sensor, OV2680_REG_EXPOSURE_PK_HIGH,
-				  exp << 4);
+	return cci_write(sensor->regmap, OV2680_REG_EXPOSURE_PK, exp << 4,
+			 NULL);
 }
 
 static int ov2680_stream_enable(struct ov2680_dev *sensor)
 {
-	return ov2680_write_reg(sensor, OV2680_REG_STREAM_CTRL, 1);
+	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 1, NULL);
 }
 
 static int ov2680_stream_disable(struct ov2680_dev *sensor)
 {
-	return ov2680_write_reg(sensor, OV2680_REG_STREAM_CTRL, 0);
+	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 0, NULL);
 }
 
 static int ov2680_mode_set(struct ov2680_dev *sensor)
 {
 	int ret;
 
-	ret = ov2680_load_regs(sensor, sensor->current_mode);
+	ret = regmap_multi_reg_write(sensor->regmap,
+				     sensor->current_mode->reg_data,
+				     sensor->current_mode->reg_data_size);
 	if (ret < 0)
 		return ret;
 
@@ -437,7 +320,9 @@ static int ov2680_mode_restore(struct ov2680_dev *sensor)
 {
 	int ret;
 
-	ret = ov2680_load_regs(sensor, &ov2680_mode_init_data);
+	ret = regmap_multi_reg_write(sensor->regmap,
+				     ov2680_mode_init_data.reg_data,
+				     ov2680_mode_init_data.reg_data_size);
 	if (ret < 0)
 		return ret;
 
@@ -472,7 +357,8 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 	}
 
 	if (!sensor->reset_gpio) {
-		ret = ov2680_write_reg(sensor, OV2680_REG_SOFT_RESET, 0x01);
+		ret = cci_write(sensor->regmap, OV2680_REG_SOFT_RESET, 0x01,
+				NULL);
 		if (ret != 0) {
 			dev_err(dev, "sensor soft reset failed\n");
 			goto err_disable_regulators;
@@ -841,19 +727,19 @@ static int ov2680_get_regulators(struct ov2680_dev *sensor)
 static int ov2680_check_id(struct ov2680_dev *sensor)
 {
 	struct device *dev = ov2680_to_dev(sensor);
-	u32 chip_id;
+	u64 chip_id;
 	int ret;
 
 	ov2680_power_on(sensor);
 
-	ret = ov2680_read_reg16(sensor, OV2680_REG_CHIP_ID_HIGH, &chip_id);
+	ret = cci_read(sensor->regmap, OV2680_REG_CHIP_ID, &chip_id, NULL);
 	if (ret < 0) {
-		dev_err(dev, "failed to read chip id high\n");
+		dev_err(dev, "failed to read chip id\n");
 		return -ENODEV;
 	}
 
 	if (chip_id != OV2680_CHIP_ID) {
-		dev_err(dev, "chip id: 0x%04x does not match expected 0x%04x\n",
+		dev_err(dev, "chip id: 0x%04llx does not match expected 0x%04x\n",
 			chip_id, OV2680_CHIP_ID);
 		return -ENODEV;
 	}
@@ -902,6 +788,10 @@ static int ov2680_probe(struct i2c_client *client)
 
 	sensor->i2c_client = client;
 
+	sensor->regmap = devm_cci_regmap_init_i2c(client, 16);
+	if (IS_ERR(sensor->regmap))
+		return PTR_ERR(sensor->regmap);
+
 	ret = ov2680_parse_dt(sensor);
 	if (ret < 0)
 		return -EINVAL;
-- 
2.41.0

[PATCH v3 10/29] media: ov2680: Store dev instead of i2c_client in ov2680_dev

[Hans de Goede]

Now that the cci_* register access helpers are used access to
the i2c_client after probe() is no longer necessary.

Directly store a struct device *dev pointing to &client->dev inside
ov2680_dev to make the code simpler.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
Changes in v2:
- Use to_i2c_client(sensor->dev) in ov2680_v4l2_register()
---
 drivers/media/i2c/ov2680.c | 30 +++++++++++-------------------
 1 file changed, 11 insertions(+), 19 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index fb25ba446d52..824e2962e7d5 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -87,7 +87,7 @@ struct ov2680_ctrls {
 };
 
 struct ov2680_dev {
-	struct i2c_client		*i2c_client;
+	struct device			*dev;
 	struct regmap			*regmap;
 	struct v4l2_subdev		sd;
 
@@ -172,11 +172,6 @@ static struct ov2680_dev *to_ov2680_dev(struct v4l2_subdev *sd)
 	return container_of(sd, struct ov2680_dev, sd);
 }
 
-static struct device *ov2680_to_dev(struct ov2680_dev *sensor)
-{
-	return &sensor->i2c_client->dev;
-}
-
 static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
 {
 	return &container_of(ctrl->handler, struct ov2680_dev,
@@ -344,7 +339,6 @@ static int ov2680_power_off(struct ov2680_dev *sensor)
 
 static int ov2680_power_on(struct ov2680_dev *sensor)
 {
-	struct device *dev = ov2680_to_dev(sensor);
 	int ret;
 
 	if (sensor->is_enabled)
@@ -352,7 +346,7 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 
 	ret = regulator_bulk_enable(OV2680_NUM_SUPPLIES, sensor->supplies);
 	if (ret < 0) {
-		dev_err(dev, "failed to enable regulators: %d\n", ret);
+		dev_err(sensor->dev, "failed to enable regulators: %d\n", ret);
 		return ret;
 	}
 
@@ -360,7 +354,7 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 		ret = cci_write(sensor->regmap, OV2680_REG_SOFT_RESET, 0x01,
 				NULL);
 		if (ret != 0) {
-			dev_err(dev, "sensor soft reset failed\n");
+			dev_err(sensor->dev, "sensor soft reset failed\n");
 			goto err_disable_regulators;
 		}
 		usleep_range(1000, 2000);
@@ -656,13 +650,13 @@ static int ov2680_mode_init(struct ov2680_dev *sensor)
 
 static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 {
+	struct i2c_client *client = to_i2c_client(sensor->dev);
 	const struct v4l2_ctrl_ops *ops = &ov2680_ctrl_ops;
 	struct ov2680_ctrls *ctrls = &sensor->ctrls;
 	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
 	int ret = 0;
 
-	v4l2_i2c_subdev_init(&sensor->sd, sensor->i2c_client,
-			     &ov2680_subdev_ops);
+	v4l2_i2c_subdev_init(&sensor->sd, client, &ov2680_subdev_ops);
 
 	sensor->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
 	sensor->pad.flags = MEDIA_PAD_FL_SOURCE;
@@ -719,14 +713,12 @@ static int ov2680_get_regulators(struct ov2680_dev *sensor)
 	for (i = 0; i < OV2680_NUM_SUPPLIES; i++)
 		sensor->supplies[i].supply = ov2680_supply_name[i];
 
-	return devm_regulator_bulk_get(&sensor->i2c_client->dev,
-				       OV2680_NUM_SUPPLIES,
-				       sensor->supplies);
+	return devm_regulator_bulk_get(sensor->dev,
+				       OV2680_NUM_SUPPLIES, sensor->supplies);
 }
 
 static int ov2680_check_id(struct ov2680_dev *sensor)
 {
-	struct device *dev = ov2680_to_dev(sensor);
 	u64 chip_id;
 	int ret;
 
@@ -734,12 +726,12 @@ static int ov2680_check_id(struct ov2680_dev *sensor)
 
 	ret = cci_read(sensor->regmap, OV2680_REG_CHIP_ID, &chip_id, NULL);
 	if (ret < 0) {
-		dev_err(dev, "failed to read chip id\n");
+		dev_err(sensor->dev, "failed to read chip id\n");
 		return -ENODEV;
 	}
 
 	if (chip_id != OV2680_CHIP_ID) {
-		dev_err(dev, "chip id: 0x%04llx does not match expected 0x%04x\n",
+		dev_err(sensor->dev, "chip id: 0x%04llx does not match expected 0x%04x\n",
 			chip_id, OV2680_CHIP_ID);
 		return -ENODEV;
 	}
@@ -749,7 +741,7 @@ static int ov2680_check_id(struct ov2680_dev *sensor)
 
 static int ov2680_parse_dt(struct ov2680_dev *sensor)
 {
-	struct device *dev = ov2680_to_dev(sensor);
+	struct device *dev = sensor->dev;
 	int ret;
 
 	sensor->reset_gpio = devm_gpiod_get_optional(dev, "reset",
@@ -786,7 +778,7 @@ static int ov2680_probe(struct i2c_client *client)
 	if (!sensor)
 		return -ENOMEM;
 
-	sensor->i2c_client = client;
+	sensor->dev = &client->dev;
 
 	sensor->regmap = devm_cci_regmap_init_i2c(client, 16);
 	if (IS_ERR(sensor->regmap))
-- 
2.41.0

[PATCH v3 11/29] media: ov2680: Check for "powerdown" GPIO con-id before checking for "reset" GPIO con-id

[Hans de Goede]

The datasheet of the OV2680 labels the single GPIO to put the sensor in
powersaving mode as XSHUTDN aka shutdown, _not_ reset.

This is important because some boards have standardized sensor connectors
which allow connecting various sensor modules. These connectors have both
reset and powerdown signals and the powerdown signal is routed to
the OV2680's XSHUTDN pin.

On x86/ACPI multiple Bay Trail, Cherry Trail, Sky Lake and Kaby Lake models
have an OV2680 connected to the ISP2 / IPU3. On these devices the GPIOS are
not described in DT instead the GPIOs are described with an Intel specific
DSM which labels them as either powerdown or reset. Often this DSM returns
both reset and powerdown pins even though the OV2680 has only 1 such pin.

For the ov2680 driver to work on these devices it must use the GPIO with
"powerdown" as con-id, matching the XSHUTDN name from the datasheet.

As for why "powerdown" vs say "shutdown" the ACPI DSM -> con-id mapping
code is shared, so we must use standardized names and currently all of
the following sensor drivers already use "powerdown":
adv7180, gc0310, isl7998x, ov02a10, ov2659, ov5640, ov5648, ov5670,
ov5693, ov7670, ov772x, ov7740, ov8858, ov8865 and ov9650 .

Where as the hi846 driver is the lonely standout using "shutdown".

Try the "powerdown" con-id first to make things work, falling back to
"reset" to keep existing DT setups working.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 29 ++++++++++++++++++++---------
 1 file changed, 20 insertions(+), 9 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 824e2962e7d5..0de047c49c31 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -96,7 +96,7 @@ struct ov2680_dev {
 	u32				xvclk_freq;
 	struct regulator_bulk_data	supplies[OV2680_NUM_SUPPLIES];
 
-	struct gpio_desc		*reset_gpio;
+	struct gpio_desc		*pwdn_gpio;
 	struct mutex			lock; /* protect members */
 
 	bool				mode_pending_changes;
@@ -180,19 +180,19 @@ static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
 
 static void ov2680_power_up(struct ov2680_dev *sensor)
 {
-	if (!sensor->reset_gpio)
+	if (!sensor->pwdn_gpio)
 		return;
 
-	gpiod_set_value(sensor->reset_gpio, 0);
+	gpiod_set_value(sensor->pwdn_gpio, 0);
 	usleep_range(5000, 10000);
 }
 
 static void ov2680_power_down(struct ov2680_dev *sensor)
 {
-	if (!sensor->reset_gpio)
+	if (!sensor->pwdn_gpio)
 		return;
 
-	gpiod_set_value(sensor->reset_gpio, 1);
+	gpiod_set_value(sensor->pwdn_gpio, 1);
 	usleep_range(5000, 10000);
 }
 
@@ -350,7 +350,7 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 		return ret;
 	}
 
-	if (!sensor->reset_gpio) {
+	if (!sensor->pwdn_gpio) {
 		ret = cci_write(sensor->regmap, OV2680_REG_SOFT_RESET, 0x01,
 				NULL);
 		if (ret != 0) {
@@ -742,16 +742,27 @@ static int ov2680_check_id(struct ov2680_dev *sensor)
 static int ov2680_parse_dt(struct ov2680_dev *sensor)
 {
 	struct device *dev = sensor->dev;
+	struct gpio_desc *gpio;
 	int ret;
 
-	sensor->reset_gpio = devm_gpiod_get_optional(dev, "reset",
-						     GPIOD_OUT_HIGH);
-	ret = PTR_ERR_OR_ZERO(sensor->reset_gpio);
+	/*
+	 * The pin we want is named XSHUTDN in the datasheet. Linux sensor
+	 * drivers have standardized on using "powerdown" as con-id name
+	 * for powerdown or shutdown pins. Older DTB files use "reset",
+	 * so fallback to that if there is no "powerdown" pin.
+	 */
+	gpio = devm_gpiod_get_optional(dev, "powerdown", GPIOD_OUT_HIGH);
+	if (!gpio)
+		gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
+
+	ret = PTR_ERR_OR_ZERO(gpio);
 	if (ret < 0) {
 		dev_dbg(dev, "error while getting reset gpio: %d\n", ret);
 		return ret;
 	}
 
+	sensor->pwdn_gpio = gpio;
+
 	sensor->xvclk = devm_clk_get(dev, "xvclk");
 	if (IS_ERR(sensor->xvclk)) {
 		dev_err(dev, "xvclk clock missing or invalid\n");
-- 
2.41.0

Re: [PATCH v3 11/29] media: ov2680: Check for "powerdown" GPIO con-id before checking for "reset" GPIO con-id

[Dan Scally]

Hi Hans

On 27/06/2023 15:18, Hans de Goede wrote:
> The datasheet of the OV2680 labels the single GPIO to put the sensor in
> powersaving mode as XSHUTDN aka shutdown, _not_ reset.
>
> This is important because some boards have standardized sensor connectors
> which allow connecting various sensor modules. These connectors have both
> reset and powerdown signals and the powerdown signal is routed to
> the OV2680's XSHUTDN pin.
>
> On x86/ACPI multiple Bay Trail, Cherry Trail, Sky Lake and Kaby Lake models
> have an OV2680 connected to the ISP2 / IPU3. On these devices the GPIOS are
> not described in DT instead the GPIOs are described with an Intel specific
> DSM which labels them as either powerdown or reset. Often this DSM returns
> both reset and powerdown pins even though the OV2680 has only 1 such pin.
>
> For the ov2680 driver to work on these devices it must use the GPIO with
> "powerdown" as con-id, matching the XSHUTDN name from the datasheet.
>
> As for why "powerdown" vs say "shutdown" the ACPI DSM -> con-id mapping
> code is shared, so we must use standardized names and currently all of
> the following sensor drivers already use "powerdown":
> adv7180, gc0310, isl7998x, ov02a10, ov2659, ov5640, ov5648, ov5670,
> ov5693, ov7670, ov772x, ov7740, ov8858, ov8865 and ov9650 .
>
> Where as the hi846 driver is the lonely standout using "shutdown".
>
> Try the "powerdown" con-id first to make things work, falling back to
> "reset" to keep existing DT setups working.
>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---


Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>

>   drivers/media/i2c/ov2680.c | 29 ++++++++++++++++++++---------
>   1 file changed, 20 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index 824e2962e7d5..0de047c49c31 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -96,7 +96,7 @@ struct ov2680_dev {
>   	u32				xvclk_freq;
>   	struct regulator_bulk_data	supplies[OV2680_NUM_SUPPLIES];
>   
> -	struct gpio_desc		*reset_gpio;
> +	struct gpio_desc		*pwdn_gpio;
>   	struct mutex			lock; /* protect members */
>   
>   	bool				mode_pending_changes;
> @@ -180,19 +180,19 @@ static inline struct v4l2_subdev *ctrl_to_sd(struct v4l2_ctrl *ctrl)
>   
>   static void ov2680_power_up(struct ov2680_dev *sensor)
>   {
> -	if (!sensor->reset_gpio)
> +	if (!sensor->pwdn_gpio)
>   		return;
>   
> -	gpiod_set_value(sensor->reset_gpio, 0);
> +	gpiod_set_value(sensor->pwdn_gpio, 0);
>   	usleep_range(5000, 10000);
>   }
>   
>   static void ov2680_power_down(struct ov2680_dev *sensor)
>   {
> -	if (!sensor->reset_gpio)
> +	if (!sensor->pwdn_gpio)
>   		return;
>   
> -	gpiod_set_value(sensor->reset_gpio, 1);
> +	gpiod_set_value(sensor->pwdn_gpio, 1);
>   	usleep_range(5000, 10000);
>   }
>   
> @@ -350,7 +350,7 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
>   		return ret;
>   	}
>   
> -	if (!sensor->reset_gpio) {
> +	if (!sensor->pwdn_gpio) {
>   		ret = cci_write(sensor->regmap, OV2680_REG_SOFT_RESET, 0x01,
>   				NULL);
>   		if (ret != 0) {
> @@ -742,16 +742,27 @@ static int ov2680_check_id(struct ov2680_dev *sensor)
>   static int ov2680_parse_dt(struct ov2680_dev *sensor)
>   {
>   	struct device *dev = sensor->dev;
> +	struct gpio_desc *gpio;
>   	int ret;
>   
> -	sensor->reset_gpio = devm_gpiod_get_optional(dev, "reset",
> -						     GPIOD_OUT_HIGH);
> -	ret = PTR_ERR_OR_ZERO(sensor->reset_gpio);
> +	/*
> +	 * The pin we want is named XSHUTDN in the datasheet. Linux sensor
> +	 * drivers have standardized on using "powerdown" as con-id name
> +	 * for powerdown or shutdown pins. Older DTB files use "reset",
> +	 * so fallback to that if there is no "powerdown" pin.
> +	 */
> +	gpio = devm_gpiod_get_optional(dev, "powerdown", GPIOD_OUT_HIGH);
> +	if (!gpio)
> +		gpio = devm_gpiod_get_optional(dev, "reset", GPIOD_OUT_HIGH);
> +
> +	ret = PTR_ERR_OR_ZERO(gpio);
>   	if (ret < 0) {
>   		dev_dbg(dev, "error while getting reset gpio: %d\n", ret);
>   		return ret;
>   	}
>   
> +	sensor->pwdn_gpio = gpio;
> +
>   	sensor->xvclk = devm_clk_get(dev, "xvclk");
>   	if (IS_ERR(sensor->xvclk)) {
>   		dev_err(dev, "xvclk clock missing or invalid\n");

[PATCH v3 12/29] media: ov2680: Add runtime-pm support

[Hans de Goede]

Remove the obsolete s_power() callback and instead use runtime-pm +
autosuspend, powering-on the sensor on s_stream(1) and releasing
the runtime-pm reference on s_stream(0).

This also removes the need for ov2680_mode_restore() instead
ov2680_stream_enable() now takes care of all sensor initalization
after power-on.

This is a preparation patch for adding ACPI support.

Note this also removes putting the clock lane into LP-11 state from
ov2680_power_on() since now streaming will start immediately after
powering on the sensor there is no need to put the clock lane
in a low power state.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 134 +++++++++++++++++--------------------
 1 file changed, 61 insertions(+), 73 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 0de047c49c31..56aaf67c1d82 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -18,6 +18,7 @@
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/of_device.h>
+#include <linux/pm_runtime.h>
 #include <linux/regmap.h>
 #include <linux/regulator/consumer.h>
 
@@ -99,7 +100,6 @@ struct ov2680_dev {
 	struct gpio_desc		*pwdn_gpio;
 	struct mutex			lock; /* protect members */
 
-	bool				mode_pending_changes;
 	bool				is_enabled;
 	bool				is_streaming;
 
@@ -282,19 +282,15 @@ static int ov2680_exposure_set(struct ov2680_dev *sensor, u32 exp)
 }
 
 static int ov2680_stream_enable(struct ov2680_dev *sensor)
-{
-	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 1, NULL);
-}
-
-static int ov2680_stream_disable(struct ov2680_dev *sensor)
-{
-	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 0, NULL);
-}
-
-static int ov2680_mode_set(struct ov2680_dev *sensor)
 {
 	int ret;
 
+	ret = regmap_multi_reg_write(sensor->regmap,
+				     ov2680_mode_init_data.reg_data,
+				     ov2680_mode_init_data.reg_data_size);
+	if (ret < 0)
+		return ret;
+
 	ret = regmap_multi_reg_write(sensor->regmap,
 				     sensor->current_mode->reg_data,
 				     sensor->current_mode->reg_data_size);
@@ -306,22 +302,12 @@ static int ov2680_mode_set(struct ov2680_dev *sensor)
 	if (ret < 0)
 		return ret;
 
-	sensor->mode_pending_changes = false;
-
-	return 0;
+	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 1, NULL);
 }
 
-static int ov2680_mode_restore(struct ov2680_dev *sensor)
+static int ov2680_stream_disable(struct ov2680_dev *sensor)
 {
-	int ret;
-
-	ret = regmap_multi_reg_write(sensor->regmap,
-				     ov2680_mode_init_data.reg_data,
-				     ov2680_mode_init_data.reg_data_size);
-	if (ret < 0)
-		return ret;
-
-	return ov2680_mode_set(sensor);
+	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 0, NULL);
 }
 
 static int ov2680_power_off(struct ov2680_dev *sensor)
@@ -369,11 +355,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 
 	sensor->is_enabled = true;
 
-	/* Set clock lane into LP-11 state */
-	ov2680_stream_enable(sensor);
-	usleep_range(1000, 2000);
-	ov2680_stream_disable(sensor);
-
 	return 0;
 
 err_disable_regulators:
@@ -381,26 +362,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 	return ret;
 }
 
-static int ov2680_s_power(struct v4l2_subdev *sd, int on)
-{
-	struct ov2680_dev *sensor = to_ov2680_dev(sd);
-	int ret = 0;
-
-	mutex_lock(&sensor->lock);
-
-	if (on)
-		ret = ov2680_power_on(sensor);
-	else
-		ret = ov2680_power_off(sensor);
-
-	if (on && ret == 0)
-		ret = ov2680_mode_restore(sensor);
-
-	mutex_unlock(&sensor->lock);
-
-	return ret;
-}
-
 static int ov2680_s_g_frame_interval(struct v4l2_subdev *sd,
 				     struct v4l2_subdev_frame_interval *fi)
 {
@@ -423,16 +384,20 @@ static int ov2680_s_stream(struct v4l2_subdev *sd, int enable)
 	if (sensor->is_streaming == !!enable)
 		goto unlock;
 
-	if (enable && sensor->mode_pending_changes) {
-		ret = ov2680_mode_set(sensor);
+	if (enable) {
+		ret = pm_runtime_resume_and_get(sensor->sd.dev);
 		if (ret < 0)
 			goto unlock;
-	}
 
-	if (enable)
 		ret = ov2680_stream_enable(sensor);
-	else
+		if (ret < 0) {
+			pm_runtime_put(sensor->sd.dev);
+			goto unlock;
+		}
+	} else {
 		ret = ov2680_stream_disable(sensor);
+		pm_runtime_put(sensor->sd.dev);
+	}
 
 	sensor->is_streaming = !!enable;
 
@@ -519,7 +484,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 
 	sensor->current_mode = mode;
 	sensor->fmt = format->format;
-	sensor->mode_pending_changes = true;
 
 unlock:
 	mutex_unlock(&sensor->lock);
@@ -603,10 +567,6 @@ static const struct v4l2_ctrl_ops ov2680_ctrl_ops = {
 	.s_ctrl = ov2680_s_ctrl,
 };
 
-static const struct v4l2_subdev_core_ops ov2680_core_ops = {
-	.s_power = ov2680_s_power,
-};
-
 static const struct v4l2_subdev_video_ops ov2680_video_ops = {
 	.g_frame_interval	= ov2680_s_g_frame_interval,
 	.s_frame_interval	= ov2680_s_g_frame_interval,
@@ -623,7 +583,6 @@ static const struct v4l2_subdev_pad_ops ov2680_pad_ops = {
 };
 
 static const struct v4l2_subdev_ops ov2680_subdev_ops = {
-	.core	= &ov2680_core_ops,
 	.video	= &ov2680_video_ops,
 	.pad	= &ov2680_pad_ops,
 };
@@ -643,8 +602,6 @@ static int ov2680_mode_init(struct ov2680_dev *sensor)
 
 	sensor->current_mode = init_mode;
 
-	sensor->mode_pending_changes = true;
-
 	return 0;
 }
 
@@ -722,8 +679,6 @@ static int ov2680_check_id(struct ov2680_dev *sensor)
 	u64 chip_id;
 	int ret;
 
-	ov2680_power_on(sensor);
-
 	ret = cci_read(sensor->regmap, OV2680_REG_CHIP_ID, &chip_id, NULL);
 	if (ret < 0) {
 		dev_err(sensor->dev, "failed to read chip id\n");
@@ -811,18 +766,39 @@ static int ov2680_probe(struct i2c_client *client)
 
 	mutex_init(&sensor->lock);
 
-	ret = ov2680_check_id(sensor);
+	/*
+	 * Power up and verify the chip now, so that if runtime pm is
+	 * disabled the chip is left on and streaming will work.
+	 */
+	ret = ov2680_power_on(sensor);
 	if (ret < 0)
 		goto lock_destroy;
 
+	ret = ov2680_check_id(sensor);
+	if (ret < 0)
+		goto err_powerdown;
+
+	pm_runtime_set_active(&client->dev);
+	pm_runtime_get_noresume(&client->dev);
+	pm_runtime_enable(&client->dev);
+
 	ret = ov2680_v4l2_register(sensor);
 	if (ret < 0)
-		goto lock_destroy;
+		goto err_pm_runtime;
+
+	pm_runtime_set_autosuspend_delay(&client->dev, 1000);
+	pm_runtime_use_autosuspend(&client->dev);
+	pm_runtime_put_autosuspend(&client->dev);
 
 	dev_info(dev, "ov2680 init correctly\n");
 
 	return 0;
 
+err_pm_runtime:
+	pm_runtime_disable(&client->dev);
+	pm_runtime_put_noidle(&client->dev);
+err_powerdown:
+	ov2680_power_off(sensor);
 lock_destroy:
 	dev_err(dev, "ov2680 init fail: %d\n", ret);
 	mutex_destroy(&sensor->lock);
@@ -839,9 +815,18 @@ static void ov2680_remove(struct i2c_client *client)
 	mutex_destroy(&sensor->lock);
 	media_entity_cleanup(&sensor->sd.entity);
 	v4l2_ctrl_handler_free(&sensor->ctrls.handler);
+
+	/*
+	 * Disable runtime PM. In case runtime PM is disabled in the kernel,
+	 * make sure to turn power off manually.
+	 */
+	pm_runtime_disable(&client->dev);
+	if (!pm_runtime_status_suspended(&client->dev))
+		ov2680_power_off(sensor);
+	pm_runtime_set_suspended(&client->dev);
 }
 
-static int __maybe_unused ov2680_suspend(struct device *dev)
+static int ov2680_suspend(struct device *dev)
 {
 	struct v4l2_subdev *sd = dev_get_drvdata(dev);
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
@@ -849,15 +834,19 @@ static int __maybe_unused ov2680_suspend(struct device *dev)
 	if (sensor->is_streaming)
 		ov2680_stream_disable(sensor);
 
-	return 0;
+	return ov2680_power_off(sensor);
 }
 
-static int __maybe_unused ov2680_resume(struct device *dev)
+static int ov2680_resume(struct device *dev)
 {
 	struct v4l2_subdev *sd = dev_get_drvdata(dev);
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 	int ret;
 
+	ret = ov2680_power_on(sensor);
+	if (ret < 0)
+		goto stream_disable;
+
 	if (sensor->is_streaming) {
 		ret = ov2680_stream_enable(sensor);
 		if (ret < 0)
@@ -873,9 +862,8 @@ static int __maybe_unused ov2680_resume(struct device *dev)
 	return ret;
 }
 
-static const struct dev_pm_ops ov2680_pm_ops = {
-	SET_SYSTEM_SLEEP_PM_OPS(ov2680_suspend, ov2680_resume)
-};
+static DEFINE_RUNTIME_DEV_PM_OPS(ov2680_pm_ops, ov2680_suspend, ov2680_resume,
+				 NULL);
 
 static const struct of_device_id ov2680_dt_ids[] = {
 	{ .compatible = "ovti,ov2680" },
@@ -886,7 +874,7 @@ MODULE_DEVICE_TABLE(of, ov2680_dt_ids);
 static struct i2c_driver ov2680_i2c_driver = {
 	.driver = {
 		.name  = "ov2680",
-		.pm = &ov2680_pm_ops,
+		.pm = pm_sleep_ptr(&ov2680_pm_ops),
 		.of_match_table	= of_match_ptr(ov2680_dt_ids),
 	},
 	.probe		= ov2680_probe,
-- 
2.41.0

[PATCH v3 13/29] media: ov2680: Drop is_enabled flag

[Hans de Goede]

With runtime-pm it is guaranteed that ov2680_power_on() and
ov2680_power_off() will always be called in a balanced way;
and the is_enabled check in ov2680_s_ctrl() can be replaced
by checking the runtime-suspend state.

So there is no more need for the is_enabled flag, remove it.

While at it also make sure that flip control changes while
suspended still lead to the bayer-order getting updated so
that get_fmt returns the correct bayer-order.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 36 ++++++++++++++++++------------------
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 56aaf67c1d82..b7c23286700e 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -100,7 +100,6 @@ struct ov2680_dev {
 	struct gpio_desc		*pwdn_gpio;
 	struct mutex			lock; /* protect members */
 
-	bool				is_enabled;
 	bool				is_streaming;
 
 	struct ov2680_ctrls		ctrls;
@@ -312,14 +311,9 @@ static int ov2680_stream_disable(struct ov2680_dev *sensor)
 
 static int ov2680_power_off(struct ov2680_dev *sensor)
 {
-	if (!sensor->is_enabled)
-		return 0;
-
 	clk_disable_unprepare(sensor->xvclk);
 	ov2680_power_down(sensor);
 	regulator_bulk_disable(OV2680_NUM_SUPPLIES, sensor->supplies);
-	sensor->is_enabled = false;
-
 	return 0;
 }
 
@@ -327,9 +321,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 {
 	int ret;
 
-	if (sensor->is_enabled)
-		return 0;
-
 	ret = regulator_bulk_enable(OV2680_NUM_SUPPLIES, sensor->supplies);
 	if (ret < 0) {
 		dev_err(sensor->dev, "failed to enable regulators: %d\n", ret);
@@ -353,8 +344,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 	if (ret < 0)
 		goto err_disable_regulators;
 
-	sensor->is_enabled = true;
-
 	return 0;
 
 err_disable_regulators:
@@ -541,26 +530,37 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
+	int ret;
 
-	if (!sensor->is_enabled)
+	/* Only apply changes to the controls if the device is powered up */
+	if (!pm_runtime_get_if_in_use(sensor->sd.dev)) {
+		ov2680_set_bayer_order(sensor, &sensor->fmt);
 		return 0;
+	}
 
 	switch (ctrl->id) {
 	case V4L2_CID_GAIN:
-		return ov2680_gain_set(sensor, ctrl->val);
+		ret = ov2680_gain_set(sensor, ctrl->val);
+		break;
 	case V4L2_CID_EXPOSURE:
-		return ov2680_exposure_set(sensor, ctrl->val);
+		ret = ov2680_exposure_set(sensor, ctrl->val);
+		break;
 	case V4L2_CID_VFLIP:
-		return ov2680_set_vflip(sensor, ctrl->val);
+		ret = ov2680_set_vflip(sensor, ctrl->val);
+		break;
 	case V4L2_CID_HFLIP:
-		return ov2680_set_hflip(sensor, ctrl->val);
+		ret = ov2680_set_hflip(sensor, ctrl->val);
+		break;
 	case V4L2_CID_TEST_PATTERN:
-		return ov2680_test_pattern_set(sensor, ctrl->val);
+		ret = ov2680_test_pattern_set(sensor, ctrl->val);
+		break;
 	default:
+		ret = -EINVAL;
 		break;
 	}
 
-	return -EINVAL;
+	pm_runtime_put(sensor->sd.dev);
+	return ret;
 }
 
 static const struct v4l2_ctrl_ops ov2680_ctrl_ops = {
-- 
2.41.0

Re: [PATCH v3 13/29] media: ov2680: Drop is_enabled flag

[Dan Scally]

Hi Hans

On 27/06/2023 15:18, Hans de Goede wrote:
> With runtime-pm it is guaranteed that ov2680_power_on() and
> ov2680_power_off() will always be called in a balanced way;
> and the is_enabled check in ov2680_s_ctrl() can be replaced
> by checking the runtime-suspend state.
>
> So there is no more need for the is_enabled flag, remove it.
>
> While at it also make sure that flip control changes while
> suspended still lead to the bayer-order getting updated so
> that get_fmt returns the correct bayer-order.


I think that this could benefit from a comment in the code - it's a departure from the usual and 
looks a bit strange. With that added:


Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>

>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>   drivers/media/i2c/ov2680.c | 36 ++++++++++++++++++------------------
>   1 file changed, 18 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index 56aaf67c1d82..b7c23286700e 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -100,7 +100,6 @@ struct ov2680_dev {
>   	struct gpio_desc		*pwdn_gpio;
>   	struct mutex			lock; /* protect members */
>   
> -	bool				is_enabled;
>   	bool				is_streaming;
>   
>   	struct ov2680_ctrls		ctrls;
> @@ -312,14 +311,9 @@ static int ov2680_stream_disable(struct ov2680_dev *sensor)
>   
>   static int ov2680_power_off(struct ov2680_dev *sensor)
>   {
> -	if (!sensor->is_enabled)
> -		return 0;
> -
>   	clk_disable_unprepare(sensor->xvclk);
>   	ov2680_power_down(sensor);
>   	regulator_bulk_disable(OV2680_NUM_SUPPLIES, sensor->supplies);
> -	sensor->is_enabled = false;
> -
>   	return 0;
>   }
>   
> @@ -327,9 +321,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
>   {
>   	int ret;
>   
> -	if (sensor->is_enabled)
> -		return 0;
> -
>   	ret = regulator_bulk_enable(OV2680_NUM_SUPPLIES, sensor->supplies);
>   	if (ret < 0) {
>   		dev_err(sensor->dev, "failed to enable regulators: %d\n", ret);
> @@ -353,8 +344,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
>   	if (ret < 0)
>   		goto err_disable_regulators;
>   
> -	sensor->is_enabled = true;
> -
>   	return 0;
>   
>   err_disable_regulators:
> @@ -541,26 +530,37 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
>   {
>   	struct v4l2_subdev *sd = ctrl_to_sd(ctrl);
>   	struct ov2680_dev *sensor = to_ov2680_dev(sd);
> +	int ret;
>   
> -	if (!sensor->is_enabled)
> +	/* Only apply changes to the controls if the device is powered up */
> +	if (!pm_runtime_get_if_in_use(sensor->sd.dev)) {
> +		ov2680_set_bayer_order(sensor, &sensor->fmt);
>   		return 0;
> +	}
>   
>   	switch (ctrl->id) {
>   	case V4L2_CID_GAIN:
> -		return ov2680_gain_set(sensor, ctrl->val);
> +		ret = ov2680_gain_set(sensor, ctrl->val);
> +		break;
>   	case V4L2_CID_EXPOSURE:
> -		return ov2680_exposure_set(sensor, ctrl->val);
> +		ret = ov2680_exposure_set(sensor, ctrl->val);
> +		break;
>   	case V4L2_CID_VFLIP:
> -		return ov2680_set_vflip(sensor, ctrl->val);
> +		ret = ov2680_set_vflip(sensor, ctrl->val);
> +		break;
>   	case V4L2_CID_HFLIP:
> -		return ov2680_set_hflip(sensor, ctrl->val);
> +		ret = ov2680_set_hflip(sensor, ctrl->val);
> +		break;
>   	case V4L2_CID_TEST_PATTERN:
> -		return ov2680_test_pattern_set(sensor, ctrl->val);
> +		ret = ov2680_test_pattern_set(sensor, ctrl->val);
> +		break;
>   	default:
> +		ret = -EINVAL;
>   		break;
>   	}
>   
> -	return -EINVAL;
> +	pm_runtime_put(sensor->sd.dev);
> +	return ret;
>   }
>   
>   static const struct v4l2_ctrl_ops ov2680_ctrl_ops = {

[PATCH v3 14/29] media: ov2680: Add support for more clk setups

[Hans de Goede]

On ACPI systems the following 2 scenarios are possible:

1. The xvclk is fully controlled by ACPI powermanagement, so there
   is no "xvclk" for the driver to get (since it is abstracted away).
   In this case there will be a "clock-frequency" device property
   to tell the driver the xvclk rate.

2. There is a xvclk modelled in the clk framework for the driver,
   but the clk-generator may not be set to the right frequency
   yet. In this case there will also be a "clock-frequency" device
   property and the driver is expected to set the rate of the xvclk
   through this frequency through the clk framework.

Handle both these scenarios by switching to devm_clk_get_optional()
and checking for a "clock-frequency" device property.

This is modelled after how the same issue was fixed for the ov8865 in
commit 73dcffeb2ff9 ("media: i2c: Support 19.2MHz input clock in ov8865").

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index b7c23286700e..a6a83f0e53f3 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -698,6 +698,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
 {
 	struct device *dev = sensor->dev;
 	struct gpio_desc *gpio;
+	unsigned int rate = 0;
 	int ret;
 
 	/*
@@ -718,13 +719,34 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
 
 	sensor->pwdn_gpio = gpio;
 
-	sensor->xvclk = devm_clk_get(dev, "xvclk");
+	sensor->xvclk = devm_clk_get_optional(dev, "xvclk");
 	if (IS_ERR(sensor->xvclk)) {
 		dev_err(dev, "xvclk clock missing or invalid\n");
 		return PTR_ERR(sensor->xvclk);
 	}
 
-	sensor->xvclk_freq = clk_get_rate(sensor->xvclk);
+	/*
+	 * We could have either a 24MHz or 19.2MHz clock rate from either DT or
+	 * ACPI... but we also need to support the weird IPU3 case which will
+	 * have an external clock AND a clock-frequency property. Check for the
+	 * clock-frequency property and if found, set that rate if we managed
+	 * to acquire a clock. This should cover the ACPI case. If the system
+	 * uses devicetree then the configured rate should already be set, so
+	 * we can just read it.
+	 */
+	ret = fwnode_property_read_u32(dev_fwnode(dev), "clock-frequency",
+				       &rate);
+	if (ret && !sensor->xvclk)
+		return dev_err_probe(dev, ret, "invalid clock config\n");
+
+	if (!ret && sensor->xvclk) {
+		ret = clk_set_rate(sensor->xvclk, rate);
+		if (ret)
+			return dev_err_probe(dev, ret,
+					     "failed to set clock rate\n");
+	}
+
+	sensor->xvclk_freq = rate ?: clk_get_rate(sensor->xvclk);
 	if (sensor->xvclk_freq != OV2680_XVCLK_VALUE) {
 		dev_err(dev, "wrong xvclk frequency %d HZ, expected: %d Hz\n",
 			sensor->xvclk_freq, OV2680_XVCLK_VALUE);
-- 
2.41.0

Re: [PATCH v3 14/29] media: ov2680: Add support for more clk setups

[Dan Scally]

Morning Hans

On 27/06/2023 15:18, Hans de Goede wrote:
> On ACPI systems the following 2 scenarios are possible:
>
> 1. The xvclk is fully controlled by ACPI powermanagement, so there
>     is no "xvclk" for the driver to get (since it is abstracted away).
>     In this case there will be a "clock-frequency" device property
>     to tell the driver the xvclk rate.
>
> 2. There is a xvclk modelled in the clk framework for the driver,
>     but the clk-generator may not be set to the right frequency
>     yet. In this case there will also be a "clock-frequency" device
>     property and the driver is expected to set the rate of the xvclk
>     through this frequency through the clk framework.
>
> Handle both these scenarios by switching to devm_clk_get_optional()
> and checking for a "clock-frequency" device property.
>
> This is modelled after how the same issue was fixed for the ov8865 in
> commit 73dcffeb2ff9 ("media: i2c: Support 19.2MHz input clock in ov8865").
>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---


Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>

>   drivers/media/i2c/ov2680.c | 26 ++++++++++++++++++++++++--
>   1 file changed, 24 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index b7c23286700e..a6a83f0e53f3 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -698,6 +698,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>   {
>   	struct device *dev = sensor->dev;
>   	struct gpio_desc *gpio;
> +	unsigned int rate = 0;
>   	int ret;
>   
>   	/*
> @@ -718,13 +719,34 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>   
>   	sensor->pwdn_gpio = gpio;
>   
> -	sensor->xvclk = devm_clk_get(dev, "xvclk");
> +	sensor->xvclk = devm_clk_get_optional(dev, "xvclk");
>   	if (IS_ERR(sensor->xvclk)) {
>   		dev_err(dev, "xvclk clock missing or invalid\n");
>   		return PTR_ERR(sensor->xvclk);
>   	}
>   
> -	sensor->xvclk_freq = clk_get_rate(sensor->xvclk);
> +	/*
> +	 * We could have either a 24MHz or 19.2MHz clock rate from either DT or
> +	 * ACPI... but we also need to support the weird IPU3 case which will
> +	 * have an external clock AND a clock-frequency property. Check for the
> +	 * clock-frequency property and if found, set that rate if we managed
> +	 * to acquire a clock. This should cover the ACPI case. If the system
> +	 * uses devicetree then the configured rate should already be set, so
> +	 * we can just read it.
> +	 */
> +	ret = fwnode_property_read_u32(dev_fwnode(dev), "clock-frequency",
> +				       &rate);
> +	if (ret && !sensor->xvclk)
> +		return dev_err_probe(dev, ret, "invalid clock config\n");
> +
> +	if (!ret && sensor->xvclk) {
> +		ret = clk_set_rate(sensor->xvclk, rate);
> +		if (ret)
> +			return dev_err_probe(dev, ret,
> +					     "failed to set clock rate\n");
> +	}
> +
> +	sensor->xvclk_freq = rate ?: clk_get_rate(sensor->xvclk);
>   	if (sensor->xvclk_freq != OV2680_XVCLK_VALUE) {
>   		dev_err(dev, "wrong xvclk frequency %d HZ, expected: %d Hz\n",
>   			sensor->xvclk_freq, OV2680_XVCLK_VALUE);

Re: [PATCH v3 14/29] media: ov2680: Add support for more clk setups

[Tommaso Merciai]

Hi Hans,

On Tue, Jun 27, 2023 at 03:18:15PM +0200, Hans de Goede wrote:
> On ACPI systems the following 2 scenarios are possible:
> 
> 1. The xvclk is fully controlled by ACPI powermanagement, so there
>    is no "xvclk" for the driver to get (since it is abstracted away).
>    In this case there will be a "clock-frequency" device property
>    to tell the driver the xvclk rate.
> 
> 2. There is a xvclk modelled in the clk framework for the driver,
>    but the clk-generator may not be set to the right frequency
>    yet. In this case there will also be a "clock-frequency" device
>    property and the driver is expected to set the rate of the xvclk
>    through this frequency through the clk framework.
> 
> Handle both these scenarios by switching to devm_clk_get_optional()
> and checking for a "clock-frequency" device property.
> 
> This is modelled after how the same issue was fixed for the ov8865 in
> commit 73dcffeb2ff9 ("media: i2c: Support 19.2MHz input clock in ov8865").
> 
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>  drivers/media/i2c/ov2680.c | 26 ++++++++++++++++++++++++--
>  1 file changed, 24 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index b7c23286700e..a6a83f0e53f3 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -698,6 +698,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>  {
>  	struct device *dev = sensor->dev;
>  	struct gpio_desc *gpio;
> +	unsigned int rate = 0;
>  	int ret;
>  
>  	/*
> @@ -718,13 +719,34 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>  
>  	sensor->pwdn_gpio = gpio;
>  
> -	sensor->xvclk = devm_clk_get(dev, "xvclk");
> +	sensor->xvclk = devm_clk_get_optional(dev, "xvclk");
>  	if (IS_ERR(sensor->xvclk)) {
>  		dev_err(dev, "xvclk clock missing or invalid\n");
>  		return PTR_ERR(sensor->xvclk);
>  	}
>  
> -	sensor->xvclk_freq = clk_get_rate(sensor->xvclk);
> +	/*
> +	 * We could have either a 24MHz or 19.2MHz clock rate from either DT or
> +	 * ACPI... but we also need to support the weird IPU3 case which will
> +	 * have an external clock AND a clock-frequency property. Check for the
> +	 * clock-frequency property and if found, set that rate if we managed
> +	 * to acquire a clock. This should cover the ACPI case. If the system
> +	 * uses devicetree then the configured rate should already be set, so
> +	 * we can just read it.
> +	 */

Nice comment :)

> +	ret = fwnode_property_read_u32(dev_fwnode(dev), "clock-frequency",
> +				       &rate);
> +	if (ret && !sensor->xvclk)
> +		return dev_err_probe(dev, ret, "invalid clock config\n");
> +
> +	if (!ret && sensor->xvclk) {
> +		ret = clk_set_rate(sensor->xvclk, rate);
> +		if (ret)
> +			return dev_err_probe(dev, ret,
> +					     "failed to set clock rate\n");
> +	}
> +
> +	sensor->xvclk_freq = rate ?: clk_get_rate(sensor->xvclk);
>  	if (sensor->xvclk_freq != OV2680_XVCLK_VALUE) {
>  		dev_err(dev, "wrong xvclk frequency %d HZ, expected: %d Hz\n",
>  			sensor->xvclk_freq, OV2680_XVCLK_VALUE);

Looks good to me.
Reviewed-by: Tommaso Merciai <tomm.merciai@gmail.com>

Regards,
Tommaso


> -- 
> 2.41.0
> 

Re: [PATCH v3 14/29] media: ov2680: Add support for more clk setups

[Sakari Ailus]

Hi Hans,

On Tue, Jun 27, 2023 at 03:18:15PM +0200, Hans de Goede wrote:
> On ACPI systems the following 2 scenarios are possible:
> 
> 1. The xvclk is fully controlled by ACPI powermanagement, so there
>    is no "xvclk" for the driver to get (since it is abstracted away).
>    In this case there will be a "clock-frequency" device property
>    to tell the driver the xvclk rate.
> 
> 2. There is a xvclk modelled in the clk framework for the driver,
>    but the clk-generator may not be set to the right frequency
>    yet. In this case there will also be a "clock-frequency" device
>    property and the driver is expected to set the rate of the xvclk
>    through this frequency through the clk framework.
> 
> Handle both these scenarios by switching to devm_clk_get_optional()
> and checking for a "clock-frequency" device property.
> 
> This is modelled after how the same issue was fixed for the ov8865 in
> commit 73dcffeb2ff9 ("media: i2c: Support 19.2MHz input clock in ov8865").
> 
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>  drivers/media/i2c/ov2680.c | 26 ++++++++++++++++++++++++--
>  1 file changed, 24 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index b7c23286700e..a6a83f0e53f3 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -698,6 +698,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>  {
>  	struct device *dev = sensor->dev;
>  	struct gpio_desc *gpio;
> +	unsigned int rate = 0;
>  	int ret;
>  
>  	/*
> @@ -718,13 +719,34 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>  
>  	sensor->pwdn_gpio = gpio;
>  
> -	sensor->xvclk = devm_clk_get(dev, "xvclk");
> +	sensor->xvclk = devm_clk_get_optional(dev, "xvclk");
>  	if (IS_ERR(sensor->xvclk)) {
>  		dev_err(dev, "xvclk clock missing or invalid\n");
>  		return PTR_ERR(sensor->xvclk);
>  	}
>  
> -	sensor->xvclk_freq = clk_get_rate(sensor->xvclk);
> +	/*
> +	 * We could have either a 24MHz or 19.2MHz clock rate from either DT or
> +	 * ACPI... but we also need to support the weird IPU3 case which will
> +	 * have an external clock AND a clock-frequency property. Check for the

Where does this happen? This puts the driver in an awful situation. :-(

> +	 * clock-frequency property and if found, set that rate if we managed
> +	 * to acquire a clock. This should cover the ACPI case. If the system
> +	 * uses devicetree then the configured rate should already be set, so
> +	 * we can just read it.
> +	 */
> +	ret = fwnode_property_read_u32(dev_fwnode(dev), "clock-frequency",
> +				       &rate);
> +	if (ret && !sensor->xvclk)
> +		return dev_err_probe(dev, ret, "invalid clock config\n");
> +
> +	if (!ret && sensor->xvclk) {
> +		ret = clk_set_rate(sensor->xvclk, rate);
> +		if (ret)
> +			return dev_err_probe(dev, ret,
> +					     "failed to set clock rate\n");
> +	}
> +
> +	sensor->xvclk_freq = rate ?: clk_get_rate(sensor->xvclk);
>  	if (sensor->xvclk_freq != OV2680_XVCLK_VALUE) {
>  		dev_err(dev, "wrong xvclk frequency %d HZ, expected: %d Hz\n",
>  			sensor->xvclk_freq, OV2680_XVCLK_VALUE);

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v3 14/29] media: ov2680: Add support for more clk setups

[Hans de Goede]

Hi,

On 7/31/23 14:44, Sakari Ailus wrote:
> Hi Hans,
> 
> On Tue, Jun 27, 2023 at 03:18:15PM +0200, Hans de Goede wrote:
>> On ACPI systems the following 2 scenarios are possible:
>>
>> 1. The xvclk is fully controlled by ACPI powermanagement, so there
>>    is no "xvclk" for the driver to get (since it is abstracted away).
>>    In this case there will be a "clock-frequency" device property
>>    to tell the driver the xvclk rate.
>>
>> 2. There is a xvclk modelled in the clk framework for the driver,
>>    but the clk-generator may not be set to the right frequency
>>    yet. In this case there will also be a "clock-frequency" device
>>    property and the driver is expected to set the rate of the xvclk
>>    through this frequency through the clk framework.
>>
>> Handle both these scenarios by switching to devm_clk_get_optional()
>> and checking for a "clock-frequency" device property.
>>
>> This is modelled after how the same issue was fixed for the ov8865 in
>> commit 73dcffeb2ff9 ("media: i2c: Support 19.2MHz input clock in ov8865").
>>
>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>> ---
>>  drivers/media/i2c/ov2680.c | 26 ++++++++++++++++++++++++--
>>  1 file changed, 24 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>> index b7c23286700e..a6a83f0e53f3 100644
>> --- a/drivers/media/i2c/ov2680.c
>> +++ b/drivers/media/i2c/ov2680.c
>> @@ -698,6 +698,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>  {
>>  	struct device *dev = sensor->dev;
>>  	struct gpio_desc *gpio;
>> +	unsigned int rate = 0;
>>  	int ret;
>>  
>>  	/*
>> @@ -718,13 +719,34 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>  
>>  	sensor->pwdn_gpio = gpio;
>>  
>> -	sensor->xvclk = devm_clk_get(dev, "xvclk");
>> +	sensor->xvclk = devm_clk_get_optional(dev, "xvclk");
>>  	if (IS_ERR(sensor->xvclk)) {
>>  		dev_err(dev, "xvclk clock missing or invalid\n");
>>  		return PTR_ERR(sensor->xvclk);
>>  	}
>>  
>> -	sensor->xvclk_freq = clk_get_rate(sensor->xvclk);
>> +	/*
>> +	 * We could have either a 24MHz or 19.2MHz clock rate from either DT or
>> +	 * ACPI... but we also need to support the weird IPU3 case which will
>> +	 * have an external clock AND a clock-frequency property. Check for the
> 
> Where does this happen? This puts the driver in an awful situation. :-(

This happens on IPU3 setups where the INT3472 device represents an actual
i2c attached sensor PMIC (rather then just some GPIOs) in this case
there is a clk generator inside the PMIC which is used and that is programmable,
so the driver needs to set the clk-rate.

Note this patch is pretty much a 1:1 copy of the same patch for the ov8865
and ov7251 drivers.

I guess it might be good to start a discussion about doing this more
elegantly but that seems out of scope for this series.

Regards,

Hans





> 
>> +	 * clock-frequency property and if found, set that rate if we managed
>> +	 * to acquire a clock. This should cover the ACPI case. If the system
>> +	 * uses devicetree then the configured rate should already be set, so
>> +	 * we can just read it.
>> +	 */
>> +	ret = fwnode_property_read_u32(dev_fwnode(dev), "clock-frequency",
>> +				       &rate);
>> +	if (ret && !sensor->xvclk)
>> +		return dev_err_probe(dev, ret, "invalid clock config\n");
>> +
>> +	if (!ret && sensor->xvclk) {
>> +		ret = clk_set_rate(sensor->xvclk, rate);
>> +		if (ret)
>> +			return dev_err_probe(dev, ret,
>> +					     "failed to set clock rate\n");
>> +	}
>> +
>> +	sensor->xvclk_freq = rate ?: clk_get_rate(sensor->xvclk);
>>  	if (sensor->xvclk_freq != OV2680_XVCLK_VALUE) {
>>  		dev_err(dev, "wrong xvclk frequency %d HZ, expected: %d Hz\n",
>>  			sensor->xvclk_freq, OV2680_XVCLK_VALUE);
> 

Re: [PATCH v3 14/29] media: ov2680: Add support for more clk setups

[Sakari Ailus]

Hi Hans,

On Mon, Jul 31, 2023 at 02:54:13PM +0200, Hans de Goede wrote:
> Hi,
> 
> On 7/31/23 14:44, Sakari Ailus wrote:
> > Hi Hans,
> > 
> > On Tue, Jun 27, 2023 at 03:18:15PM +0200, Hans de Goede wrote:
> >> On ACPI systems the following 2 scenarios are possible:
> >>
> >> 1. The xvclk is fully controlled by ACPI powermanagement, so there
> >>    is no "xvclk" for the driver to get (since it is abstracted away).
> >>    In this case there will be a "clock-frequency" device property
> >>    to tell the driver the xvclk rate.
> >>
> >> 2. There is a xvclk modelled in the clk framework for the driver,
> >>    but the clk-generator may not be set to the right frequency
> >>    yet. In this case there will also be a "clock-frequency" device
> >>    property and the driver is expected to set the rate of the xvclk
> >>    through this frequency through the clk framework.
> >>
> >> Handle both these scenarios by switching to devm_clk_get_optional()
> >> and checking for a "clock-frequency" device property.
> >>
> >> This is modelled after how the same issue was fixed for the ov8865 in
> >> commit 73dcffeb2ff9 ("media: i2c: Support 19.2MHz input clock in ov8865").
> >>
> >> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> >> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> >> ---
> >>  drivers/media/i2c/ov2680.c | 26 ++++++++++++++++++++++++--
> >>  1 file changed, 24 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> >> index b7c23286700e..a6a83f0e53f3 100644
> >> --- a/drivers/media/i2c/ov2680.c
> >> +++ b/drivers/media/i2c/ov2680.c
> >> @@ -698,6 +698,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
> >>  {
> >>  	struct device *dev = sensor->dev;
> >>  	struct gpio_desc *gpio;
> >> +	unsigned int rate = 0;
> >>  	int ret;
> >>  
> >>  	/*
> >> @@ -718,13 +719,34 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
> >>  
> >>  	sensor->pwdn_gpio = gpio;
> >>  
> >> -	sensor->xvclk = devm_clk_get(dev, "xvclk");
> >> +	sensor->xvclk = devm_clk_get_optional(dev, "xvclk");
> >>  	if (IS_ERR(sensor->xvclk)) {
> >>  		dev_err(dev, "xvclk clock missing or invalid\n");
> >>  		return PTR_ERR(sensor->xvclk);
> >>  	}
> >>  
> >> -	sensor->xvclk_freq = clk_get_rate(sensor->xvclk);
> >> +	/*
> >> +	 * We could have either a 24MHz or 19.2MHz clock rate from either DT or
> >> +	 * ACPI... but we also need to support the weird IPU3 case which will
> >> +	 * have an external clock AND a clock-frequency property. Check for the
> > 
> > Where does this happen? This puts the driver in an awful situation. :-(
> 
> This happens on IPU3 setups where the INT3472 device represents an actual
> i2c attached sensor PMIC (rather then just some GPIOs) in this case
> there is a clk generator inside the PMIC which is used and that is programmable,
> so the driver needs to set the clk-rate.
> 
> Note this patch is pretty much a 1:1 copy of the same patch for the ov8865
> and ov7251 drivers.
> 
> I guess it might be good to start a discussion about doing this more
> elegantly but that seems out of scope for this series.

Works for me.

Do you happen to know which systems use the clock generator feature of the
PMIC?

I guess it could be as simple as putting this to tps68470 platform data for
the clock. And then hope no other PMICs will be used with this format.

-- 
Kind regards,

Sakari Ailus

Re: [PATCH v3 14/29] media: ov2680: Add support for more clk setups

[Hans de Goede]

Hi,

On 7/31/23 15:35, Sakari Ailus wrote:
> Hi Hans,
> 
> On Mon, Jul 31, 2023 at 02:54:13PM +0200, Hans de Goede wrote:
>> Hi,
>>
>> On 7/31/23 14:44, Sakari Ailus wrote:
>>> Hi Hans,
>>>
>>> On Tue, Jun 27, 2023 at 03:18:15PM +0200, Hans de Goede wrote:
>>>> On ACPI systems the following 2 scenarios are possible:
>>>>
>>>> 1. The xvclk is fully controlled by ACPI powermanagement, so there
>>>>    is no "xvclk" for the driver to get (since it is abstracted away).
>>>>    In this case there will be a "clock-frequency" device property
>>>>    to tell the driver the xvclk rate.
>>>>
>>>> 2. There is a xvclk modelled in the clk framework for the driver,
>>>>    but the clk-generator may not be set to the right frequency
>>>>    yet. In this case there will also be a "clock-frequency" device
>>>>    property and the driver is expected to set the rate of the xvclk
>>>>    through this frequency through the clk framework.
>>>>
>>>> Handle both these scenarios by switching to devm_clk_get_optional()
>>>> and checking for a "clock-frequency" device property.
>>>>
>>>> This is modelled after how the same issue was fixed for the ov8865 in
>>>> commit 73dcffeb2ff9 ("media: i2c: Support 19.2MHz input clock in ov8865").
>>>>
>>>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>>>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>>>> ---
>>>>  drivers/media/i2c/ov2680.c | 26 ++++++++++++++++++++++++--
>>>>  1 file changed, 24 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>>>> index b7c23286700e..a6a83f0e53f3 100644
>>>> --- a/drivers/media/i2c/ov2680.c
>>>> +++ b/drivers/media/i2c/ov2680.c
>>>> @@ -698,6 +698,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>>>  {
>>>>  	struct device *dev = sensor->dev;
>>>>  	struct gpio_desc *gpio;
>>>> +	unsigned int rate = 0;
>>>>  	int ret;
>>>>  
>>>>  	/*
>>>> @@ -718,13 +719,34 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>>>  
>>>>  	sensor->pwdn_gpio = gpio;
>>>>  
>>>> -	sensor->xvclk = devm_clk_get(dev, "xvclk");
>>>> +	sensor->xvclk = devm_clk_get_optional(dev, "xvclk");
>>>>  	if (IS_ERR(sensor->xvclk)) {
>>>>  		dev_err(dev, "xvclk clock missing or invalid\n");
>>>>  		return PTR_ERR(sensor->xvclk);
>>>>  	}
>>>>  
>>>> -	sensor->xvclk_freq = clk_get_rate(sensor->xvclk);
>>>> +	/*
>>>> +	 * We could have either a 24MHz or 19.2MHz clock rate from either DT or
>>>> +	 * ACPI... but we also need to support the weird IPU3 case which will
>>>> +	 * have an external clock AND a clock-frequency property. Check for the
>>>
>>> Where does this happen? This puts the driver in an awful situation. :-(
>>
>> This happens on IPU3 setups where the INT3472 device represents an actual
>> i2c attached sensor PMIC (rather then just some GPIOs) in this case
>> there is a clk generator inside the PMIC which is used and that is programmable,
>> so the driver needs to set the clk-rate.
>>
>> Note this patch is pretty much a 1:1 copy of the same patch for the ov8865
>> and ov7251 drivers.
>>
>> I guess it might be good to start a discussion about doing this more
>> elegantly but that seems out of scope for this series.
> 
> Works for me.
> 
> Do you happen to know which systems use the clock generator feature of the
> PMIC?

This is used at least on the Microsoft Surface Go devices most folks use for IPU3 development.



 
 and have the tps68470 
> 
> I guess it could be as simple as putting this to tps68470 platform data for
> the clock. And then hope no other PMICs will be used with this format.

Right, after your email from earlier today I was actually thinking along the following lines to fix this:

1. There already is a struct tps68470_clk_platform_data which currently just contains the consumer dev_name() + con-id, we could extend this with an init_clk_rate member

2. Have the int3472 glue code fill init_clk_rate with info from the sensor's SSDB. This does require the int3472 code to make an extra SSDB() ACPI call. The ssdb struct definition has moved to include/media/ipu-bridge.h now, so that is already shared.

3. Make the tps68470 driver set the clk-rate to init_clk_rate if that is set to non 0

Then the clk_set_rate() call can be dropped from the drivers, note we do still need the other complexity with getting the clk + then getting the rate from the clk with a fallback to the property.

AFAICT doing this in a follow up series seems quite doable.

Regards,

Hans

[PATCH v3 15/29] media: ov2680: Add support for 19.2 MHz clock

[Hans de Goede]

Most x86/ACPI boards use the ov2680 with a 19.2 MHz xvclk,
rather then the expected 24MHz, add support for this.

Compensate for the lower clk by setting a higher PLL multiplier
of 69 when using 19.2 MHz vs the default multiplier of 55 for
a 24MHz xvclk.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 41 +++++++++++++++++++++++++++++++-------
 1 file changed, 34 insertions(+), 7 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index a6a83f0e53f3..d4ef34859914 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -27,14 +27,13 @@
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-subdev.h>
 
-#define OV2680_XVCLK_VALUE			24000000
-
 #define OV2680_CHIP_ID				0x2680
 
 #define OV2680_REG_STREAM_CTRL			CCI_REG8(0x0100)
 #define OV2680_REG_SOFT_RESET			CCI_REG8(0x0103)
 
 #define OV2680_REG_CHIP_ID			CCI_REG16(0x300a)
+#define OV2680_REG_PLL_MULTIPLIER		CCI_REG16(0x3081)
 
 #define OV2680_REG_EXPOSURE_PK			CCI_REG24(0x3500)
 #define OV2680_REG_R_MANUAL			CCI_REG8(0x3503)
@@ -69,6 +68,21 @@ static const char * const ov2680_supply_name[] = {
 
 #define OV2680_NUM_SUPPLIES ARRAY_SIZE(ov2680_supply_name)
 
+enum {
+	OV2680_19_2_MHZ,
+	OV2680_24_MHZ,
+};
+
+static const unsigned long ov2680_xvclk_freqs[] = {
+	[OV2680_19_2_MHZ] = 19200000,
+	[OV2680_24_MHZ] = 24000000,
+};
+
+static const u8 ov2680_pll_multipliers[] = {
+	[OV2680_19_2_MHZ] = 69,
+	[OV2680_24_MHZ] = 55,
+};
+
 struct ov2680_mode_info {
 	const char *name;
 	enum ov2680_mode_id id;
@@ -95,6 +109,7 @@ struct ov2680_dev {
 	struct media_pad		pad;
 	struct clk			*xvclk;
 	u32				xvclk_freq;
+	u8				pll_mult;
 	struct regulator_bulk_data	supplies[OV2680_NUM_SUPPLIES];
 
 	struct gpio_desc		*pwdn_gpio;
@@ -284,6 +299,11 @@ static int ov2680_stream_enable(struct ov2680_dev *sensor)
 {
 	int ret;
 
+	ret = cci_write(sensor->regmap, OV2680_REG_PLL_MULTIPLIER,
+			sensor->pll_mult, NULL);
+	if (ret < 0)
+		return ret;
+
 	ret = regmap_multi_reg_write(sensor->regmap,
 				     ov2680_mode_init_data.reg_data,
 				     ov2680_mode_init_data.reg_data_size);
@@ -699,7 +719,7 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
 	struct device *dev = sensor->dev;
 	struct gpio_desc *gpio;
 	unsigned int rate = 0;
-	int ret;
+	int i, ret;
 
 	/*
 	 * The pin we want is named XSHUTDN in the datasheet. Linux sensor
@@ -747,12 +767,19 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
 	}
 
 	sensor->xvclk_freq = rate ?: clk_get_rate(sensor->xvclk);
-	if (sensor->xvclk_freq != OV2680_XVCLK_VALUE) {
-		dev_err(dev, "wrong xvclk frequency %d HZ, expected: %d Hz\n",
-			sensor->xvclk_freq, OV2680_XVCLK_VALUE);
-		return -EINVAL;
+
+	for (i = 0; i < ARRAY_SIZE(ov2680_xvclk_freqs); i++) {
+		if (sensor->xvclk_freq == ov2680_xvclk_freqs[i])
+			break;
 	}
 
+	if (i == ARRAY_SIZE(ov2680_xvclk_freqs))
+		return dev_err_probe(dev, -EINVAL,
+				     "unsupported xvclk frequency %d Hz\n",
+				     sensor->xvclk_freq);
+
+	sensor->pll_mult = ov2680_pll_multipliers[i];
+
 	return 0;
 }
 
-- 
2.41.0

[PATCH v3 16/29] media: ov2680: Add endpoint matching support

[Hans de Goede]

Add endpoint matching support and defer probe() until
the endpoint fwnode is available. This is necessary on ACPI
platforms where the bridge code creating the fwnodes may also e.g.
set the "clock-frequency" device property and add GPIO mappings.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 28 +++++++++++++++++++++-------
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index d4ef34859914..52ae265612fa 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -103,6 +103,7 @@ struct ov2680_ctrls {
 
 struct ov2680_dev {
 	struct device			*dev;
+	struct fwnode_handle		*ep_fwnode;
 	struct regmap			*regmap;
 	struct v4l2_subdev		sd;
 
@@ -638,6 +639,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 	sensor->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
 	sensor->pad.flags = MEDIA_PAD_FL_SOURCE;
 	sensor->sd.entity.function = MEDIA_ENT_F_CAM_SENSOR;
+	sensor->sd.fwnode = sensor->ep_fwnode;
 
 	ret = media_entity_pads_init(&sensor->sd.entity, 1, &sensor->pad);
 	if (ret < 0)
@@ -799,29 +801,39 @@ static int ov2680_probe(struct i2c_client *client)
 	if (IS_ERR(sensor->regmap))
 		return PTR_ERR(sensor->regmap);
 
+	/*
+	 * Sometimes the fwnode graph is initialized by the bridge driver.
+	 * Bridge drivers doing this may also add GPIO mappings, wait for this.
+	 */
+	sensor->ep_fwnode = fwnode_graph_get_next_endpoint(dev_fwnode(dev),
+							   NULL);
+	if (!sensor->ep_fwnode)
+		return dev_err_probe(dev, -EPROBE_DEFER,
+				     "waiting for fwnode graph endpoint\n");
+
+	mutex_init(&sensor->lock);
+
 	ret = ov2680_parse_dt(sensor);
 	if (ret < 0)
-		return -EINVAL;
+		goto err_fwnode_put;
 
 	ret = ov2680_mode_init(sensor);
 	if (ret < 0)
-		return ret;
+		goto err_fwnode_put;
 
 	ret = ov2680_get_regulators(sensor);
 	if (ret < 0) {
 		dev_err(dev, "failed to get regulators\n");
-		return ret;
+		goto err_fwnode_put;
 	}
 
-	mutex_init(&sensor->lock);
-
 	/*
 	 * Power up and verify the chip now, so that if runtime pm is
 	 * disabled the chip is left on and streaming will work.
 	 */
 	ret = ov2680_power_on(sensor);
 	if (ret < 0)
-		goto lock_destroy;
+		goto err_fwnode_put;
 
 	ret = ov2680_check_id(sensor);
 	if (ret < 0)
@@ -848,9 +860,10 @@ static int ov2680_probe(struct i2c_client *client)
 	pm_runtime_put_noidle(&client->dev);
 err_powerdown:
 	ov2680_power_off(sensor);
-lock_destroy:
+err_fwnode_put:
 	dev_err(dev, "ov2680 init fail: %d\n", ret);
 	mutex_destroy(&sensor->lock);
+	fwnode_handle_put(sensor->ep_fwnode);
 
 	return ret;
 }
@@ -864,6 +877,7 @@ static void ov2680_remove(struct i2c_client *client)
 	mutex_destroy(&sensor->lock);
 	media_entity_cleanup(&sensor->sd.entity);
 	v4l2_ctrl_handler_free(&sensor->ctrls.handler);
+	fwnode_handle_put(sensor->ep_fwnode);
 
 	/*
 	 * Disable runtime PM. In case runtime PM is disabled in the kernel,
-- 
2.41.0

[PATCH v3 17/29] media: ov2680: Add support for ACPI enumeration

[Hans de Goede]

Add an acpi_match_table now that all the other bits necessary for
ACPI support are in place.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 52ae265612fa..b011dadbb98a 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -934,11 +934,18 @@ static const struct of_device_id ov2680_dt_ids[] = {
 };
 MODULE_DEVICE_TABLE(of, ov2680_dt_ids);
 
+static const struct acpi_device_id ov2680_acpi_ids[] = {
+	{ "OVTI2680" },
+	{ /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(acpi, ov2680_acpi_ids);
+
 static struct i2c_driver ov2680_i2c_driver = {
 	.driver = {
 		.name  = "ov2680",
 		.pm = pm_sleep_ptr(&ov2680_pm_ops),
 		.of_match_table	= of_match_ptr(ov2680_dt_ids),
+		.acpi_match_table = ov2680_acpi_ids,
 	},
 	.probe		= ov2680_probe,
 	.remove		= ov2680_remove,
-- 
2.41.0

[PATCH v3 18/29] media: ov2680: Fix ov2680_enum_frame_interval()

[Hans de Goede]

Fix and simplify ov2680_enum_frame_interval(), the index is not
an index into ov2680_mode_data[], so using OV2680_MODE_MAX is wrong.

Instead it is an index indexing the different framerates for
the resolution specified in fie->width, fie->height.

Since the ov2680 code only supports a single fixed framerate,
index must always be 0 and we don't need to check the other
fie input values.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index b011dadbb98a..7ca70877abf1 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -532,17 +532,13 @@ static int ov2680_enum_frame_interval(struct v4l2_subdev *sd,
 			      struct v4l2_subdev_state *sd_state,
 			      struct v4l2_subdev_frame_interval_enum *fie)
 {
-	struct v4l2_fract tpf;
+	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 
-	if (fie->index >= OV2680_MODE_MAX || fie->width > OV2680_WIDTH_MAX ||
-	    fie->height > OV2680_HEIGHT_MAX ||
-	    fie->which > V4L2_SUBDEV_FORMAT_ACTIVE)
+	/* Only 1 framerate */
+	if (fie->index)
 		return -EINVAL;
 
-	tpf.denominator = OV2680_FRAME_RATE;
-	tpf.numerator = 1;
-
-	fie->interval = tpf;
+	fie->interval = sensor->frame_interval;
 
 	return 0;
 }
-- 
2.41.0

Re: [PATCH v3 18/29] media: ov2680: Fix ov2680_enum_frame_interval()

[Dan Scally]

On 27/06/2023 15:18, Hans de Goede wrote:
> Fix and simplify ov2680_enum_frame_interval(), the index is not
> an index into ov2680_mode_data[], so using OV2680_MODE_MAX is wrong.
>
> Instead it is an index indexing the different framerates for
> the resolution specified in fie->width, fie->height.
>
> Since the ov2680 code only supports a single fixed framerate,
> index must always be 0 and we don't need to check the other
> fie input values.


But in this case the user could ask which single frame interval is supported for a frame size that 
is _not_ supported, and be told that the driver can give them 30fps. I think we still need to check 
the validity of the other inputs and return -EINVAL when they're invalid.

>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>   drivers/media/i2c/ov2680.c | 12 ++++--------
>   1 file changed, 4 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index b011dadbb98a..7ca70877abf1 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -532,17 +532,13 @@ static int ov2680_enum_frame_interval(struct v4l2_subdev *sd,
>   			      struct v4l2_subdev_state *sd_state,
>   			      struct v4l2_subdev_frame_interval_enum *fie)
>   {
> -	struct v4l2_fract tpf;
> +	struct ov2680_dev *sensor = to_ov2680_dev(sd);
>   
> -	if (fie->index >= OV2680_MODE_MAX || fie->width > OV2680_WIDTH_MAX ||
> -	    fie->height > OV2680_HEIGHT_MAX ||
> -	    fie->which > V4L2_SUBDEV_FORMAT_ACTIVE)
> +	/* Only 1 framerate */
> +	if (fie->index)
>   		return -EINVAL;
>   
> -	tpf.denominator = OV2680_FRAME_RATE;
> -	tpf.numerator = 1;
> -
> -	fie->interval = tpf;
> +	fie->interval = sensor->frame_interval;
>   
>   	return 0;
>   }

Re: [PATCH v3 18/29] media: ov2680: Fix ov2680_enum_frame_interval()

[Hans de Goede]

Hi,

On 7/3/23 09:26, Dan Scally wrote:
> 
> On 27/06/2023 15:18, Hans de Goede wrote:
>> Fix and simplify ov2680_enum_frame_interval(), the index is not
>> an index into ov2680_mode_data[], so using OV2680_MODE_MAX is wrong.
>>
>> Instead it is an index indexing the different framerates for
>> the resolution specified in fie->width, fie->height.
>>
>> Since the ov2680 code only supports a single fixed framerate,
>> index must always be 0 and we don't need to check the other
>> fie input values.
> 
> 
> But in this case the user could ask which single frame interval is supported for a frame size that is _not_ supported, and be told that the driver can give them 30fps. I think we still need to check the validity of the other inputs and return -EINVAL when they're invalid.

Ok for v4 I'll re-add the fie->which check and also add a check
that the passed in width + height are a valid combination from
ov2680_mode_data[].

Regards,

Hans

> 
>>
>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>> ---
>>   drivers/media/i2c/ov2680.c | 12 ++++--------
>>   1 file changed, 4 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>> index b011dadbb98a..7ca70877abf1 100644
>> --- a/drivers/media/i2c/ov2680.c
>> +++ b/drivers/media/i2c/ov2680.c
>> @@ -532,17 +532,13 @@ static int ov2680_enum_frame_interval(struct v4l2_subdev *sd,
>>                     struct v4l2_subdev_state *sd_state,
>>                     struct v4l2_subdev_frame_interval_enum *fie)
>>   {
>> -    struct v4l2_fract tpf;
>> +    struct ov2680_dev *sensor = to_ov2680_dev(sd);
>>   -    if (fie->index >= OV2680_MODE_MAX || fie->width > OV2680_WIDTH_MAX ||
>> -        fie->height > OV2680_HEIGHT_MAX ||
>> -        fie->which > V4L2_SUBDEV_FORMAT_ACTIVE)
>> +    /* Only 1 framerate */
>> +    if (fie->index)
>>           return -EINVAL;
>>   -    tpf.denominator = OV2680_FRAME_RATE;
>> -    tpf.numerator = 1;
>> -
>> -    fie->interval = tpf;
>> +    fie->interval = sensor->frame_interval;
>>         return 0;
>>   }
> 

[PATCH v3 19/29] media: ov2680: Annotate the per mode register setting lists

[Hans de Goede]

Annotate the per mode register setting lists.

This is a preparation patch for moving to calculating the per mode
settings, allowing to set any mode through cropping.

The annotations make it easier to see which registers are mode
dependent and which are fixed.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 118 ++++++++++++++++++++++++++++++++-----
 1 file changed, 104 insertions(+), 14 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 7ca70877abf1..09b74c15220b 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -141,27 +141,117 @@ static const int ov2680_hv_flip_bayer_order[] = {
 };
 
 static const struct reg_sequence ov2680_setting_30fps_QUXGA_800_600[] = {
-	{0x3086, 0x01}, {0x370a, 0x23}, {0x3808, 0x03}, {0x3809, 0x20},
-	{0x380a, 0x02}, {0x380b, 0x58}, {0x380c, 0x06}, {0x380d, 0xac},
-	{0x380e, 0x02}, {0x380f, 0x84}, {0x3811, 0x04}, {0x3813, 0x04},
-	{0x3814, 0x31}, {0x3815, 0x31}, {0x3820, 0xc0}, {0x4008, 0x00},
-	{0x4009, 0x03}, {0x4837, 0x1e}, {0x3501, 0x4e}, {0x3502, 0xe0},
+	/* Set PLL SP DIV to 1 for binning mode */
+	{0x3086, 0x01},
+
+	/* Sensor control register 0x0a to 0x23 for binning mode */
+	{0x370a, 0x23},
+
+	/* Set X and Y output size to 800x600 */
+	{0x3808, 0x03},
+	{0x3809, 0x20},
+	{0x380a, 0x02},
+	{0x380b, 0x58},
+
+	/* Set HTS + VTS to 1708x644 */
+	{0x380c, 0x06},
+	{0x380d, 0xac},
+	{0x380e, 0x02},
+	{0x380f, 0x84},
+
+	/* Set ISP WIN X and Y start to 4x4 */
+	{0x3811, 0x04},
+	{0x3813, 0x04},
+
+	/* Set X INC and Y INC for binning */
+	{0x3814, 0x31},
+	{0x3815, 0x31},
+
+	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
+	{0x3820, 0xc0},
+
+	/* Set black level compensation range to 0 - 3 (default 0 - 11) */
+	{0x4008, 0x00},
+	{0x4009, 0x03},
+
+	/* Set MIPI pclk period to 0x1e (default/reset is 0x18) */
+	{0x4837, 0x1e},
+
+	/* Initialize exposure to 0x4ee (overridden by the ctrl, drop this */
+	{0x3501, 0x4e},
+	{0x3502, 0xe0},
+
+	/* R MANUAL set exposure and gain to manual (hw does not do auto) */
 	{0x3503, 0x03},
 };
 
 static const struct reg_sequence ov2680_setting_30fps_720P_1280_720[] = {
-	{0x3086, 0x00}, {0x3808, 0x05}, {0x3809, 0x00}, {0x380a, 0x02},
-	{0x380b, 0xd0}, {0x380c, 0x06}, {0x380d, 0xa8}, {0x380e, 0x05},
-	{0x380f, 0x0e}, {0x3811, 0x08}, {0x3813, 0x06}, {0x3814, 0x11},
-	{0x3815, 0x11}, {0x3820, 0xc0}, {0x4008, 0x00},
+	/* Set PLL SP DIV to 0 for not binning mode */
+	{0x3086, 0x00},
+
+	/* Set X and Y output size to 1280x720 */
+	{0x3808, 0x05},
+	{0x3809, 0x00},
+	{0x380a, 0x02},
+	{0x380b, 0xd0},
+
+	/* Set HTS + VTS to 1704x1294 */
+	{0x380c, 0x06},
+	{0x380d, 0xa8},
+	{0x380e, 0x05},
+	{0x380f, 0x0e},
+
+	/* Set ISP WIN X and Y start to 8x6 */
+	{0x3811, 0x08},
+	{0x3813, 0x06},
+
+	/* Set X INC and Y INC for non binning */
+	{0x3814, 0x11},
+	{0x3815, 0x11},
+
+	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
+	{0x3820, 0xc0},
+
+	/* Set backlight compensation range start to 0 */
+	{0x4008, 0x00},
 };
 
 static const struct reg_sequence ov2680_setting_30fps_UXGA_1600_1200[] = {
-	{0x3086, 0x00}, {0x3501, 0x4e}, {0x3502, 0xe0}, {0x3808, 0x06},
-	{0x3809, 0x40}, {0x380a, 0x04}, {0x380b, 0xb0}, {0x380c, 0x06},
-	{0x380d, 0xa8}, {0x380e, 0x05}, {0x380f, 0x0e}, {0x3811, 0x00},
-	{0x3813, 0x00}, {0x3814, 0x11}, {0x3815, 0x11}, {0x3820, 0xc0},
-	{0x4008, 0x00}, {0x4837, 0x18}
+	/* Set PLL SP DIV to 0 for not binning mode */
+	{0x3086, 0x00},
+
+	/* Initialize exposure to 0x4ee (overridden by the ctrl, drop this */
+	{0x3501, 0x4e},
+	{0x3502, 0xe0},
+
+	/* Set X and Y output size to 1600x1200 */
+	{0x3808, 0x06},
+	{0x3809, 0x40},
+	{0x380a, 0x04},
+	{0x380b, 0xb0},
+
+	/* Set HTS + VTS to 1704x1294 */
+	{0x380c, 0x06},
+	{0x380d, 0xa8},
+	{0x380e, 0x05},
+	{0x380f, 0x0e},
+
+	/* Set ISP WIN X and Y start to 0x0 */
+	{0x3811, 0x00},
+	{0x3813, 0x00},
+
+	/* Set X INC and Y INC for non binning */
+	{0x3814, 0x11},
+	{0x3815, 0x11},
+
+	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
+	{0x3820, 0xc0},
+
+	/* Set backlight compensation range start to 0 */
+	{0x4008, 0x00},
+
+	/* Set MIPI pclk period to default/reset value of 0x18 */
+	{0x4837, 0x18}
 };
 
 static const struct ov2680_mode_info ov2680_mode_init_data = {
-- 
2.41.0

[PATCH v3 20/29] media: ov2680: Add ov2680_mode struct

[Hans de Goede]

Add an ov2680_mode struct to group together mode related state.

For now this only containst the v4l2_mbus_framefmt and
the frame_interval.

This is a preparation patch for moving to calculating the per mode
settings, which will store more info in the new ov2680_mode struct.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 30 +++++++++++++++++-------------
 1 file changed, 17 insertions(+), 13 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 09b74c15220b..79789b9efadf 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -101,6 +101,11 @@ struct ov2680_ctrls {
 	struct v4l2_ctrl *test_pattern;
 };
 
+struct ov2680_mode {
+	struct v4l2_mbus_framefmt	fmt;
+	struct v4l2_fract		frame_interval;
+};
+
 struct ov2680_dev {
 	struct device			*dev;
 	struct fwnode_handle		*ep_fwnode;
@@ -119,8 +124,7 @@ struct ov2680_dev {
 	bool				is_streaming;
 
 	struct ov2680_ctrls		ctrls;
-	struct v4l2_mbus_framefmt	fmt;
-	struct v4l2_fract		frame_interval;
+	struct ov2680_mode		mode;
 
 	const struct ov2680_mode_info	*current_mode;
 };
@@ -339,7 +343,7 @@ static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
 	if (ret < 0)
 		return ret;
 
-	ov2680_set_bayer_order(sensor, &sensor->fmt);
+	ov2680_set_bayer_order(sensor, &sensor->mode.fmt);
 	return 0;
 }
 
@@ -355,7 +359,7 @@ static int ov2680_set_hflip(struct ov2680_dev *sensor, s32 val)
 	if (ret < 0)
 		return ret;
 
-	ov2680_set_bayer_order(sensor, &sensor->fmt);
+	ov2680_set_bayer_order(sensor, &sensor->mode.fmt);
 	return 0;
 }
 
@@ -468,7 +472,7 @@ static int ov2680_s_g_frame_interval(struct v4l2_subdev *sd,
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 
 	mutex_lock(&sensor->lock);
-	fi->interval = sensor->frame_interval;
+	fi->interval = sensor->mode.frame_interval;
 	mutex_unlock(&sensor->lock);
 
 	return 0;
@@ -516,7 +520,7 @@ static int ov2680_enum_mbus_code(struct v4l2_subdev *sd,
 	if (code->pad != 0 || code->index != 0)
 		return -EINVAL;
 
-	code->code = sensor->fmt.code;
+	code->code = sensor->mode.fmt.code;
 
 	return 0;
 }
@@ -537,7 +541,7 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
 		fmt = v4l2_subdev_get_try_format(&sensor->sd, sd_state,
 						 format->pad);
 	} else {
-		fmt = &sensor->fmt;
+		fmt = &sensor->mode.fmt;
 	}
 
 	format->format = *fmt;
@@ -583,7 +587,7 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 	}
 
 	sensor->current_mode = mode;
-	sensor->fmt = format->format;
+	sensor->mode.fmt = format->format;
 
 unlock:
 	mutex_unlock(&sensor->lock);
@@ -628,7 +632,7 @@ static int ov2680_enum_frame_interval(struct v4l2_subdev *sd,
 	if (fie->index)
 		return -EINVAL;
 
-	fie->interval = sensor->frame_interval;
+	fie->interval = sensor->mode.frame_interval;
 
 	return 0;
 }
@@ -641,7 +645,7 @@ static int ov2680_s_ctrl(struct v4l2_ctrl *ctrl)
 
 	/* Only apply changes to the controls if the device is powered up */
 	if (!pm_runtime_get_if_in_use(sensor->sd.dev)) {
-		ov2680_set_bayer_order(sensor, &sensor->fmt);
+		ov2680_set_bayer_order(sensor, &sensor->mode.fmt);
 		return 0;
 	}
 
@@ -699,11 +703,11 @@ static int ov2680_mode_init(struct ov2680_dev *sensor)
 	const struct ov2680_mode_info *init_mode;
 
 	/* set initial mode */
-	ov2680_fill_format(sensor, &sensor->fmt,
+	ov2680_fill_format(sensor, &sensor->mode.fmt,
 			   OV2680_DEFAULT_WIDTH, OV2680_DEFAULT_HEIGHT);
 
-	sensor->frame_interval.denominator = OV2680_FRAME_RATE;
-	sensor->frame_interval.numerator = 1;
+	sensor->mode.frame_interval.denominator = OV2680_FRAME_RATE;
+	sensor->mode.frame_interval.numerator = 1;
 
 	init_mode = &ov2680_mode_init_data;
 
-- 
2.41.0

[PATCH v3 21/29] media: ov2680: Make setting the mode algorithm based

[Hans de Goede]

Instead of using a long fixed register settings list for each resolution,
calculate the register settings based on the requested width + height.

This is based on atomisp-ov2680 commit 0611888592df ("media: atomisp:
ov2680: Make setting the modes algorithm based").

This will allow future enhancements like adding hblank and vblank controls
and adding selection support.

This also adds properly prgramming the ISP window and setting
the manual ISP window control bit in register 0x5708, this is
necessary for the hflip and vflip conrols to work properly.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 315 +++++++++++++++++--------------------
 1 file changed, 143 insertions(+), 172 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 79789b9efadf..15e0b0ef5e8f 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -38,28 +38,44 @@
 #define OV2680_REG_EXPOSURE_PK			CCI_REG24(0x3500)
 #define OV2680_REG_R_MANUAL			CCI_REG8(0x3503)
 #define OV2680_REG_GAIN_PK			CCI_REG16(0x350a)
+
+#define OV2680_REG_SENSOR_CTRL_0A		CCI_REG8(0x370a)
+
+#define OV2680_REG_HORIZONTAL_START		CCI_REG16(0x3800)
+#define OV2680_REG_VERTICAL_START		CCI_REG16(0x3802)
+#define OV2680_REG_HORIZONTAL_END		CCI_REG16(0x3804)
+#define OV2680_REG_VERTICAL_END			CCI_REG16(0x3806)
+#define OV2680_REG_HORIZONTAL_OUTPUT_SIZE	CCI_REG16(0x3808)
+#define OV2680_REG_VERTICAL_OUTPUT_SIZE		CCI_REG16(0x380a)
 #define OV2680_REG_TIMING_HTS			CCI_REG16(0x380c)
 #define OV2680_REG_TIMING_VTS			CCI_REG16(0x380e)
+#define OV2680_REG_ISP_X_WIN			CCI_REG16(0x3810)
+#define OV2680_REG_ISP_Y_WIN			CCI_REG16(0x3812)
+#define OV2680_REG_X_INC			CCI_REG8(0x3814)
+#define OV2680_REG_Y_INC			CCI_REG8(0x3815)
 #define OV2680_REG_FORMAT1			CCI_REG8(0x3820)
 #define OV2680_REG_FORMAT2			CCI_REG8(0x3821)
 
 #define OV2680_REG_ISP_CTRL00			CCI_REG8(0x5080)
 
+#define OV2680_REG_X_WIN			CCI_REG16(0x5704)
+#define OV2680_REG_Y_WIN			CCI_REG16(0x5706)
+
 #define OV2680_FRAME_RATE			30
 
-#define OV2680_WIDTH_MAX			1600
-#define OV2680_HEIGHT_MAX			1200
+#define OV2680_NATIVE_WIDTH			1616
+#define OV2680_NATIVE_HEIGHT			1216
+
+/* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
+#define OV2680_PIXELS_PER_LINE			1704
+#define OV2680_LINES_PER_FRAME			1294
+
+/* If possible send 16 extra rows / lines to the ISP as padding */
+#define OV2680_END_MARGIN			16
 
 #define OV2680_DEFAULT_WIDTH			800
 #define OV2680_DEFAULT_HEIGHT			600
 
-enum ov2680_mode_id {
-	OV2680_MODE_QUXGA_800_600,
-	OV2680_MODE_720P_1280_720,
-	OV2680_MODE_UXGA_1600_1200,
-	OV2680_MODE_MAX,
-};
-
 static const char * const ov2680_supply_name[] = {
 	"DOVDD",
 	"DVDD",
@@ -83,15 +99,6 @@ static const u8 ov2680_pll_multipliers[] = {
 	[OV2680_24_MHZ] = 55,
 };
 
-struct ov2680_mode_info {
-	const char *name;
-	enum ov2680_mode_id id;
-	u32 width;
-	u32 height;
-	const struct reg_sequence *reg_data;
-	u32 reg_data_size;
-};
-
 struct ov2680_ctrls {
 	struct v4l2_ctrl_handler handler;
 	struct v4l2_ctrl *exposure;
@@ -104,6 +111,15 @@ struct ov2680_ctrls {
 struct ov2680_mode {
 	struct v4l2_mbus_framefmt	fmt;
 	struct v4l2_fract		frame_interval;
+	bool				binning;
+	u16				h_start;
+	u16				v_start;
+	u16				h_end;
+	u16				v_end;
+	u16				h_output_size;
+	u16				v_output_size;
+	u16				hts;
+	u16				vts;
 };
 
 struct ov2680_dev {
@@ -125,8 +141,6 @@ struct ov2680_dev {
 
 	struct ov2680_ctrls		ctrls;
 	struct ov2680_mode		mode;
-
-	const struct ov2680_mode_info	*current_mode;
 };
 
 static const char * const test_pattern_menu[] = {
@@ -144,136 +158,19 @@ static const int ov2680_hv_flip_bayer_order[] = {
 	MEDIA_BUS_FMT_SRGGB10_1X10,
 };
 
-static const struct reg_sequence ov2680_setting_30fps_QUXGA_800_600[] = {
-	/* Set PLL SP DIV to 1 for binning mode */
-	{0x3086, 0x01},
-
-	/* Sensor control register 0x0a to 0x23 for binning mode */
-	{0x370a, 0x23},
-
-	/* Set X and Y output size to 800x600 */
-	{0x3808, 0x03},
-	{0x3809, 0x20},
-	{0x380a, 0x02},
-	{0x380b, 0x58},
-
-	/* Set HTS + VTS to 1708x644 */
-	{0x380c, 0x06},
-	{0x380d, 0xac},
-	{0x380e, 0x02},
-	{0x380f, 0x84},
-
-	/* Set ISP WIN X and Y start to 4x4 */
-	{0x3811, 0x04},
-	{0x3813, 0x04},
-
-	/* Set X INC and Y INC for binning */
-	{0x3814, 0x31},
-	{0x3815, 0x31},
-
-	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
-	{0x3820, 0xc0},
+static const struct reg_sequence ov2680_global_setting[] = {
+	/* R MANUAL set exposure and gain to manual (hw does not do auto) */
+	{0x3503, 0x03},
 
 	/* Set black level compensation range to 0 - 3 (default 0 - 11) */
 	{0x4008, 0x00},
 	{0x4009, 0x03},
 
-	/* Set MIPI pclk period to 0x1e (default/reset is 0x18) */
-	{0x4837, 0x1e},
-
-	/* Initialize exposure to 0x4ee (overridden by the ctrl, drop this */
-	{0x3501, 0x4e},
-	{0x3502, 0xe0},
-
-	/* R MANUAL set exposure and gain to manual (hw does not do auto) */
-	{0x3503, 0x03},
-};
-
-static const struct reg_sequence ov2680_setting_30fps_720P_1280_720[] = {
-	/* Set PLL SP DIV to 0 for not binning mode */
-	{0x3086, 0x00},
-
-	/* Set X and Y output size to 1280x720 */
-	{0x3808, 0x05},
-	{0x3809, 0x00},
-	{0x380a, 0x02},
-	{0x380b, 0xd0},
-
-	/* Set HTS + VTS to 1704x1294 */
-	{0x380c, 0x06},
-	{0x380d, 0xa8},
-	{0x380e, 0x05},
-	{0x380f, 0x0e},
-
-	/* Set ISP WIN X and Y start to 8x6 */
-	{0x3811, 0x08},
-	{0x3813, 0x06},
-
-	/* Set X INC and Y INC for non binning */
-	{0x3814, 0x11},
-	{0x3815, 0x11},
-
-	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
-	{0x3820, 0xc0},
-
-	/* Set backlight compensation range start to 0 */
-	{0x4008, 0x00},
-};
-
-static const struct reg_sequence ov2680_setting_30fps_UXGA_1600_1200[] = {
-	/* Set PLL SP DIV to 0 for not binning mode */
-	{0x3086, 0x00},
-
-	/* Initialize exposure to 0x4ee (overridden by the ctrl, drop this */
-	{0x3501, 0x4e},
-	{0x3502, 0xe0},
-
-	/* Set X and Y output size to 1600x1200 */
-	{0x3808, 0x06},
-	{0x3809, 0x40},
-	{0x380a, 0x04},
-	{0x380b, 0xb0},
-
-	/* Set HTS + VTS to 1704x1294 */
-	{0x380c, 0x06},
-	{0x380d, 0xa8},
-	{0x380e, 0x05},
-	{0x380f, 0x0e},
-
-	/* Set ISP WIN X and Y start to 0x0 */
-	{0x3811, 0x00},
-	{0x3813, 0x00},
-
-	/* Set X INC and Y INC for non binning */
-	{0x3814, 0x11},
-	{0x3815, 0x11},
-
-	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
-	{0x3820, 0xc0},
-
-	/* Set backlight compensation range start to 0 */
-	{0x4008, 0x00},
-
-	/* Set MIPI pclk period to default/reset value of 0x18 */
-	{0x4837, 0x18}
-};
-
-static const struct ov2680_mode_info ov2680_mode_init_data = {
-	"mode_quxga_800_600", OV2680_MODE_QUXGA_800_600, 800, 600,
-	ov2680_setting_30fps_QUXGA_800_600,
-	ARRAY_SIZE(ov2680_setting_30fps_QUXGA_800_600),
-};
-
-static const struct ov2680_mode_info ov2680_mode_data[OV2680_MODE_MAX] = {
-	{"mode_quxga_800_600", OV2680_MODE_QUXGA_800_600,
-	 800, 600, ov2680_setting_30fps_QUXGA_800_600,
-	 ARRAY_SIZE(ov2680_setting_30fps_QUXGA_800_600)},
-	{"mode_720p_1280_720", OV2680_MODE_720P_1280_720,
-	 1280, 720, ov2680_setting_30fps_720P_1280_720,
-	 ARRAY_SIZE(ov2680_setting_30fps_720P_1280_720)},
-	{"mode_uxga_1600_1200", OV2680_MODE_UXGA_1600_1200,
-	 1600, 1200, ov2680_setting_30fps_UXGA_1600_1200,
-	 ARRAY_SIZE(ov2680_setting_30fps_UXGA_1600_1200)},
+	/*
+	 * Window CONTROL 0x00 -> 0x01, enable manual window control,
+	 * this is necessary for full size flip and mirror support.
+	 */
+	{0x5708, 0x01},
 };
 
 static struct ov2680_dev *to_ov2680_dev(struct v4l2_subdev *sd)
@@ -331,6 +228,85 @@ static void ov2680_fill_format(struct ov2680_dev *sensor,
 	ov2680_set_bayer_order(sensor, fmt);
 }
 
+static void ov2680_calc_mode(struct ov2680_dev *sensor)
+{
+	int width = sensor->mode.fmt.width;
+	int height = sensor->mode.fmt.height;
+	int orig_width = width;
+	int orig_height = height;
+
+	if (width  <= (OV2680_NATIVE_WIDTH / 2) &&
+	    height <= (OV2680_NATIVE_HEIGHT / 2)) {
+		sensor->mode.binning = true;
+		width *= 2;
+		height *= 2;
+	} else {
+		sensor->mode.binning = false;
+	}
+
+	sensor->mode.h_start = ((OV2680_NATIVE_WIDTH - width) / 2) & ~1;
+	sensor->mode.v_start = ((OV2680_NATIVE_HEIGHT - height) / 2) & ~1;
+	sensor->mode.h_end =
+		min(sensor->mode.h_start + width + OV2680_END_MARGIN - 1,
+		    OV2680_NATIVE_WIDTH - 1);
+	sensor->mode.v_end =
+		min(sensor->mode.v_start + height + OV2680_END_MARGIN - 1,
+		    OV2680_NATIVE_HEIGHT - 1);
+	sensor->mode.h_output_size = orig_width;
+	sensor->mode.v_output_size = orig_height;
+	sensor->mode.hts = OV2680_PIXELS_PER_LINE;
+	sensor->mode.vts = OV2680_LINES_PER_FRAME;
+}
+
+static int ov2680_set_mode(struct ov2680_dev *sensor)
+{
+	u8 sensor_ctrl_0a, inc, fmt1, fmt2;
+	int ret = 0;
+
+	if (sensor->mode.binning) {
+		sensor_ctrl_0a = 0x23;
+		inc = 0x31;
+		fmt1 = 0xc2;
+		fmt2 = 0x01;
+	} else {
+		sensor_ctrl_0a = 0x21;
+		inc = 0x11;
+		fmt1 = 0xc0;
+		fmt2 = 0x00;
+	}
+
+	cci_write(sensor->regmap, OV2680_REG_SENSOR_CTRL_0A,
+		  sensor_ctrl_0a, &ret);
+	cci_write(sensor->regmap, OV2680_REG_HORIZONTAL_START,
+		  sensor->mode.h_start, &ret);
+	cci_write(sensor->regmap, OV2680_REG_VERTICAL_START,
+		  sensor->mode.v_start, &ret);
+	cci_write(sensor->regmap, OV2680_REG_HORIZONTAL_END,
+		  sensor->mode.h_end, &ret);
+	cci_write(sensor->regmap, OV2680_REG_VERTICAL_END,
+		  sensor->mode.v_end, &ret);
+	cci_write(sensor->regmap, OV2680_REG_HORIZONTAL_OUTPUT_SIZE,
+		  sensor->mode.h_output_size, &ret);
+	cci_write(sensor->regmap, OV2680_REG_VERTICAL_OUTPUT_SIZE,
+		  sensor->mode.v_output_size, &ret);
+	cci_write(sensor->regmap, OV2680_REG_TIMING_HTS,
+		  sensor->mode.hts, &ret);
+	cci_write(sensor->regmap, OV2680_REG_TIMING_VTS,
+		  sensor->mode.vts, &ret);
+	cci_write(sensor->regmap, OV2680_REG_ISP_X_WIN, 0, &ret);
+	cci_write(sensor->regmap, OV2680_REG_ISP_Y_WIN, 0, &ret);
+	cci_write(sensor->regmap, OV2680_REG_X_INC, inc, &ret);
+	cci_write(sensor->regmap, OV2680_REG_Y_INC, inc, &ret);
+	cci_write(sensor->regmap, OV2680_REG_X_WIN,
+		  sensor->mode.h_output_size, &ret);
+	cci_write(sensor->regmap, OV2680_REG_Y_WIN,
+		  sensor->mode.v_output_size, &ret);
+	cci_write(sensor->regmap, OV2680_REG_FORMAT1, fmt1, &ret);
+	cci_write(sensor->regmap, OV2680_REG_FORMAT2, fmt2, &ret);
+
+	return ret;
+}
+
 static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
 {
 	int ret;
@@ -400,14 +376,12 @@ static int ov2680_stream_enable(struct ov2680_dev *sensor)
 		return ret;
 
 	ret = regmap_multi_reg_write(sensor->regmap,
-				     ov2680_mode_init_data.reg_data,
-				     ov2680_mode_init_data.reg_data_size);
+				     ov2680_global_setting,
+				     ARRAY_SIZE(ov2680_global_setting));
 	if (ret < 0)
 		return ret;
 
-	ret = regmap_multi_reg_write(sensor->regmap,
-				     sensor->current_mode->reg_data,
-				     sensor->current_mode->reg_data_size);
+	ret = ov2680_set_mode(sensor);
 	if (ret < 0)
 		return ret;
 
@@ -557,21 +531,18 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 	struct v4l2_mbus_framefmt *try_fmt;
-	const struct ov2680_mode_info *mode;
+	unsigned int width, height;
 	int ret = 0;
 
 	if (format->pad != 0)
 		return -EINVAL;
 
-	mode = v4l2_find_nearest_size(ov2680_mode_data,
-				      ARRAY_SIZE(ov2680_mode_data),
-				      width, height,
-				      format->format.width,
-				      format->format.height);
-	if (!mode)
-		return -EINVAL;
+	width = min_t(unsigned int, ALIGN(format->format.width, 2),
+		      OV2680_NATIVE_WIDTH);
+	height = min_t(unsigned int, ALIGN(format->format.height, 2),
+		       OV2680_NATIVE_HEIGHT);
 
-	ov2680_fill_format(sensor, &format->format, mode->width, mode->height);
+	ov2680_fill_format(sensor, &format->format, width, height);
 
 	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
 		try_fmt = v4l2_subdev_get_try_format(sd, sd_state, 0);
@@ -586,8 +557,8 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 		goto unlock;
 	}
 
-	sensor->current_mode = mode;
 	sensor->mode.fmt = format->format;
+	ov2680_calc_mode(sensor);
 
 unlock:
 	mutex_unlock(&sensor->lock);
@@ -609,15 +580,20 @@ static int ov2680_enum_frame_size(struct v4l2_subdev *sd,
 				  struct v4l2_subdev_state *sd_state,
 				  struct v4l2_subdev_frame_size_enum *fse)
 {
-	int index = fse->index;
+	static const struct v4l2_frmsize_discrete ov2680_frame_sizes[] = {
+		{ 1600, 1200 },
+		{ 1280,  720 },
+		{  800,  600 },
+	};
+	u32 index = fse->index;
 
-	if (index >= OV2680_MODE_MAX || index < 0)
+	if (index >= ARRAY_SIZE(ov2680_frame_sizes))
 		return -EINVAL;
 
-	fse->min_width = ov2680_mode_data[index].width;
-	fse->min_height = ov2680_mode_data[index].height;
-	fse->max_width = ov2680_mode_data[index].width;
-	fse->max_height = ov2680_mode_data[index].height;
+	fse->min_width = ov2680_frame_sizes[index].width;
+	fse->min_height = ov2680_frame_sizes[index].height;
+	fse->max_width = ov2680_frame_sizes[index].width;
+	fse->max_height = ov2680_frame_sizes[index].height;
 
 	return 0;
 }
@@ -700,19 +676,14 @@ static const struct v4l2_subdev_ops ov2680_subdev_ops = {
 
 static int ov2680_mode_init(struct ov2680_dev *sensor)
 {
-	const struct ov2680_mode_info *init_mode;
-
 	/* set initial mode */
 	ov2680_fill_format(sensor, &sensor->mode.fmt,
 			   OV2680_DEFAULT_WIDTH, OV2680_DEFAULT_HEIGHT);
+	ov2680_calc_mode(sensor);
 
 	sensor->mode.frame_interval.denominator = OV2680_FRAME_RATE;
 	sensor->mode.frame_interval.numerator = 1;
 
-	init_mode = &ov2680_mode_init_data;
-
-	sensor->current_mode = init_mode;
-
 	return 0;
 }
 
-- 
2.41.0

Re: [PATCH v3 21/29] media: ov2680: Make setting the mode algorithm based

[Dan Scally]

Hi Hans

On 27/06/2023 15:18, Hans de Goede wrote:
> Instead of using a long fixed register settings list for each resolution,
> calculate the register settings based on the requested width + height.
>
> This is based on atomisp-ov2680 commit 0611888592df ("media: atomisp:
> ov2680: Make setting the modes algorithm based").
>
> This will allow future enhancements like adding hblank and vblank controls
> and adding selection support.
>
> This also adds properly prgramming the ISP window and setting


s/prgramming/programming


This looks mostly good but I have one query down below.

> the manual ISP window control bit in register 0x5708, this is
> necessary for the hflip and vflip conrols to work properly.
>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>   drivers/media/i2c/ov2680.c | 315 +++++++++++++++++--------------------
>   1 file changed, 143 insertions(+), 172 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index 79789b9efadf..15e0b0ef5e8f 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -38,28 +38,44 @@
>   #define OV2680_REG_EXPOSURE_PK			CCI_REG24(0x3500)
>   #define OV2680_REG_R_MANUAL			CCI_REG8(0x3503)
>   #define OV2680_REG_GAIN_PK			CCI_REG16(0x350a)
> +
> +#define OV2680_REG_SENSOR_CTRL_0A		CCI_REG8(0x370a)
> +
> +#define OV2680_REG_HORIZONTAL_START		CCI_REG16(0x3800)
> +#define OV2680_REG_VERTICAL_START		CCI_REG16(0x3802)
> +#define OV2680_REG_HORIZONTAL_END		CCI_REG16(0x3804)
> +#define OV2680_REG_VERTICAL_END			CCI_REG16(0x3806)
> +#define OV2680_REG_HORIZONTAL_OUTPUT_SIZE	CCI_REG16(0x3808)
> +#define OV2680_REG_VERTICAL_OUTPUT_SIZE		CCI_REG16(0x380a)
>   #define OV2680_REG_TIMING_HTS			CCI_REG16(0x380c)
>   #define OV2680_REG_TIMING_VTS			CCI_REG16(0x380e)
> +#define OV2680_REG_ISP_X_WIN			CCI_REG16(0x3810)
> +#define OV2680_REG_ISP_Y_WIN			CCI_REG16(0x3812)
> +#define OV2680_REG_X_INC			CCI_REG8(0x3814)
> +#define OV2680_REG_Y_INC			CCI_REG8(0x3815)
>   #define OV2680_REG_FORMAT1			CCI_REG8(0x3820)
>   #define OV2680_REG_FORMAT2			CCI_REG8(0x3821)
>   
>   #define OV2680_REG_ISP_CTRL00			CCI_REG8(0x5080)
>   
> +#define OV2680_REG_X_WIN			CCI_REG16(0x5704)
> +#define OV2680_REG_Y_WIN			CCI_REG16(0x5706)
> +
>   #define OV2680_FRAME_RATE			30
>   
> -#define OV2680_WIDTH_MAX			1600
> -#define OV2680_HEIGHT_MAX			1200
> +#define OV2680_NATIVE_WIDTH			1616
> +#define OV2680_NATIVE_HEIGHT			1216
> +
> +/* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
> +#define OV2680_PIXELS_PER_LINE			1704
> +#define OV2680_LINES_PER_FRAME			1294
> +
> +/* If possible send 16 extra rows / lines to the ISP as padding */
> +#define OV2680_END_MARGIN			16
>   
>   #define OV2680_DEFAULT_WIDTH			800
>   #define OV2680_DEFAULT_HEIGHT			600
>   
> -enum ov2680_mode_id {
> -	OV2680_MODE_QUXGA_800_600,
> -	OV2680_MODE_720P_1280_720,
> -	OV2680_MODE_UXGA_1600_1200,
> -	OV2680_MODE_MAX,
> -};
> -
>   static const char * const ov2680_supply_name[] = {
>   	"DOVDD",
>   	"DVDD",
> @@ -83,15 +99,6 @@ static const u8 ov2680_pll_multipliers[] = {
>   	[OV2680_24_MHZ] = 55,
>   };
>   
> -struct ov2680_mode_info {
> -	const char *name;
> -	enum ov2680_mode_id id;
> -	u32 width;
> -	u32 height;
> -	const struct reg_sequence *reg_data;
> -	u32 reg_data_size;
> -};
> -
>   struct ov2680_ctrls {
>   	struct v4l2_ctrl_handler handler;
>   	struct v4l2_ctrl *exposure;
> @@ -104,6 +111,15 @@ struct ov2680_ctrls {
>   struct ov2680_mode {
>   	struct v4l2_mbus_framefmt	fmt;
>   	struct v4l2_fract		frame_interval;
> +	bool				binning;
> +	u16				h_start;
> +	u16				v_start;
> +	u16				h_end;
> +	u16				v_end;
> +	u16				h_output_size;
> +	u16				v_output_size;
> +	u16				hts;
> +	u16				vts;
>   };
>   
>   struct ov2680_dev {
> @@ -125,8 +141,6 @@ struct ov2680_dev {
>   
>   	struct ov2680_ctrls		ctrls;
>   	struct ov2680_mode		mode;
> -
> -	const struct ov2680_mode_info	*current_mode;
>   };
>   
>   static const char * const test_pattern_menu[] = {
> @@ -144,136 +158,19 @@ static const int ov2680_hv_flip_bayer_order[] = {
>   	MEDIA_BUS_FMT_SRGGB10_1X10,
>   };
>   
> -static const struct reg_sequence ov2680_setting_30fps_QUXGA_800_600[] = {
> -	/* Set PLL SP DIV to 1 for binning mode */
> -	{0x3086, 0x01},
> -
> -	/* Sensor control register 0x0a to 0x23 for binning mode */
> -	{0x370a, 0x23},
> -
> -	/* Set X and Y output size to 800x600 */
> -	{0x3808, 0x03},
> -	{0x3809, 0x20},
> -	{0x380a, 0x02},
> -	{0x380b, 0x58},
> -
> -	/* Set HTS + VTS to 1708x644 */
> -	{0x380c, 0x06},
> -	{0x380d, 0xac},
> -	{0x380e, 0x02},
> -	{0x380f, 0x84},
> -
> -	/* Set ISP WIN X and Y start to 4x4 */
> -	{0x3811, 0x04},
> -	{0x3813, 0x04},
> -
> -	/* Set X INC and Y INC for binning */
> -	{0x3814, 0x31},
> -	{0x3815, 0x31},
> -
> -	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
> -	{0x3820, 0xc0},
> +static const struct reg_sequence ov2680_global_setting[] = {
> +	/* R MANUAL set exposure and gain to manual (hw does not do auto) */
> +	{0x3503, 0x03},
>   
>   	/* Set black level compensation range to 0 - 3 (default 0 - 11) */
>   	{0x4008, 0x00},
>   	{0x4009, 0x03},
>   
> -	/* Set MIPI pclk period to 0x1e (default/reset is 0x18) */
> -	{0x4837, 0x1e},
> -
> -	/* Initialize exposure to 0x4ee (overridden by the ctrl, drop this */
> -	{0x3501, 0x4e},
> -	{0x3502, 0xe0},
> -
> -	/* R MANUAL set exposure and gain to manual (hw does not do auto) */
> -	{0x3503, 0x03},
> -};
> -
> -static const struct reg_sequence ov2680_setting_30fps_720P_1280_720[] = {
> -	/* Set PLL SP DIV to 0 for not binning mode */
> -	{0x3086, 0x00},
> -
> -	/* Set X and Y output size to 1280x720 */
> -	{0x3808, 0x05},
> -	{0x3809, 0x00},
> -	{0x380a, 0x02},
> -	{0x380b, 0xd0},
> -
> -	/* Set HTS + VTS to 1704x1294 */
> -	{0x380c, 0x06},
> -	{0x380d, 0xa8},
> -	{0x380e, 0x05},
> -	{0x380f, 0x0e},
> -
> -	/* Set ISP WIN X and Y start to 8x6 */
> -	{0x3811, 0x08},
> -	{0x3813, 0x06},
> -
> -	/* Set X INC and Y INC for non binning */
> -	{0x3814, 0x11},
> -	{0x3815, 0x11},
> -
> -	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
> -	{0x3820, 0xc0},
> -
> -	/* Set backlight compensation range start to 0 */
> -	{0x4008, 0x00},
> -};
> -
> -static const struct reg_sequence ov2680_setting_30fps_UXGA_1600_1200[] = {
> -	/* Set PLL SP DIV to 0 for not binning mode */
> -	{0x3086, 0x00},
> -
> -	/* Initialize exposure to 0x4ee (overridden by the ctrl, drop this */
> -	{0x3501, 0x4e},
> -	{0x3502, 0xe0},
> -
> -	/* Set X and Y output size to 1600x1200 */
> -	{0x3808, 0x06},
> -	{0x3809, 0x40},
> -	{0x380a, 0x04},
> -	{0x380b, 0xb0},
> -
> -	/* Set HTS + VTS to 1704x1294 */
> -	{0x380c, 0x06},
> -	{0x380d, 0xa8},
> -	{0x380e, 0x05},
> -	{0x380f, 0x0e},
> -
> -	/* Set ISP WIN X and Y start to 0x0 */
> -	{0x3811, 0x00},
> -	{0x3813, 0x00},
> -
> -	/* Set X INC and Y INC for non binning */
> -	{0x3814, 0x11},
> -	{0x3815, 0x11},
> -
> -	/* Initialize FORMAT1 to default/reset value (vflip disabled) */
> -	{0x3820, 0xc0},
> -
> -	/* Set backlight compensation range start to 0 */
> -	{0x4008, 0x00},
> -
> -	/* Set MIPI pclk period to default/reset value of 0x18 */
> -	{0x4837, 0x18}
> -};
> -
> -static const struct ov2680_mode_info ov2680_mode_init_data = {
> -	"mode_quxga_800_600", OV2680_MODE_QUXGA_800_600, 800, 600,
> -	ov2680_setting_30fps_QUXGA_800_600,
> -	ARRAY_SIZE(ov2680_setting_30fps_QUXGA_800_600),
> -};
> -
> -static const struct ov2680_mode_info ov2680_mode_data[OV2680_MODE_MAX] = {
> -	{"mode_quxga_800_600", OV2680_MODE_QUXGA_800_600,
> -	 800, 600, ov2680_setting_30fps_QUXGA_800_600,
> -	 ARRAY_SIZE(ov2680_setting_30fps_QUXGA_800_600)},
> -	{"mode_720p_1280_720", OV2680_MODE_720P_1280_720,
> -	 1280, 720, ov2680_setting_30fps_720P_1280_720,
> -	 ARRAY_SIZE(ov2680_setting_30fps_720P_1280_720)},
> -	{"mode_uxga_1600_1200", OV2680_MODE_UXGA_1600_1200,
> -	 1600, 1200, ov2680_setting_30fps_UXGA_1600_1200,
> -	 ARRAY_SIZE(ov2680_setting_30fps_UXGA_1600_1200)},
> +	/*
> +	 * Window CONTROL 0x00 -> 0x01, enable manual window control,
> +	 * this is necessary for full size flip and mirror support.
> +	 */
> +	{0x5708, 0x01},
>   };
>   
>   static struct ov2680_dev *to_ov2680_dev(struct v4l2_subdev *sd)
> @@ -331,6 +228,85 @@ static void ov2680_fill_format(struct ov2680_dev *sensor,
>   	ov2680_set_bayer_order(sensor, fmt);
>   }
>   
> +static void ov2680_calc_mode(struct ov2680_dev *sensor)
> +{
> +	int width = sensor->mode.fmt.width;
> +	int height = sensor->mode.fmt.height;
> +	int orig_width = width;
> +	int orig_height = height;
> +
> +	if (width  <= (OV2680_NATIVE_WIDTH / 2) &&
> +	    height <= (OV2680_NATIVE_HEIGHT / 2)) {
> +		sensor->mode.binning = true;
> +		width *= 2;
> +		height *= 2;
> +	} else {
> +		sensor->mode.binning = false;
> +	}
> +
> +	sensor->mode.h_start = ((OV2680_NATIVE_WIDTH - width) / 2) & ~1;
> +	sensor->mode.v_start = ((OV2680_NATIVE_HEIGHT - height) / 2) & ~1;
> +	sensor->mode.h_end =
> +		min(sensor->mode.h_start + width + OV2680_END_MARGIN - 1,
> +		    OV2680_NATIVE_WIDTH - 1);
> +	sensor->mode.v_end =
> +		min(sensor->mode.v_start + height + OV2680_END_MARGIN - 1,
> +		    OV2680_NATIVE_HEIGHT - 1);
> +	sensor->mode.h_output_size = orig_width;
> +	sensor->mode.v_output_size = orig_height;
> +	sensor->mode.hts = OV2680_PIXELS_PER_LINE;
> +	sensor->mode.vts = OV2680_LINES_PER_FRAME;
> +}
> +
> +static int ov2680_set_mode(struct ov2680_dev *sensor)
> +{
> +	u8 sensor_ctrl_0a, inc, fmt1, fmt2;
> +	int ret = 0;
> +
> +	if (sensor->mode.binning) {
> +		sensor_ctrl_0a = 0x23;
> +		inc = 0x31;
> +		fmt1 = 0xc2;
> +		fmt2 = 0x01;
> +	} else {
> +		sensor_ctrl_0a = 0x21;
> +		inc = 0x11;
> +		fmt1 = 0xc0;
> +		fmt2 = 0x00;
> +	}
> +
> +	cci_write(sensor->regmap, OV2680_REG_SENSOR_CTRL_0A,
> +		  sensor_ctrl_0a, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_HORIZONTAL_START,
> +		  sensor->mode.h_start, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_VERTICAL_START,
> +		  sensor->mode.v_start, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_HORIZONTAL_END,
> +		  sensor->mode.h_end, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_VERTICAL_END,
> +		  sensor->mode.v_end, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_HORIZONTAL_OUTPUT_SIZE,
> +		  sensor->mode.h_output_size, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_VERTICAL_OUTPUT_SIZE,
> +		  sensor->mode.v_output_size, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_TIMING_HTS,
> +		  sensor->mode.hts, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_TIMING_VTS,
> +		  sensor->mode.vts, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_ISP_X_WIN, 0, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_ISP_Y_WIN, 0, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_X_INC, inc, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_Y_INC, inc, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_X_WIN,
> +		  sensor->mode.h_output_size, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_Y_WIN,
> +		  sensor->mode.v_output_size, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_FORMAT1, fmt1, &ret);
> +	cci_write(sensor->regmap, OV2680_REG_FORMAT2, fmt2, &ret);
> +
> +	return ret;
> +}
> +
>   static int ov2680_set_vflip(struct ov2680_dev *sensor, s32 val)
>   {
>   	int ret;
> @@ -400,14 +376,12 @@ static int ov2680_stream_enable(struct ov2680_dev *sensor)
>   		return ret;
>   
>   	ret = regmap_multi_reg_write(sensor->regmap,
> -				     ov2680_mode_init_data.reg_data,
> -				     ov2680_mode_init_data.reg_data_size);
> +				     ov2680_global_setting,
> +				     ARRAY_SIZE(ov2680_global_setting));
>   	if (ret < 0)
>   		return ret;
>   
> -	ret = regmap_multi_reg_write(sensor->regmap,
> -				     sensor->current_mode->reg_data,
> -				     sensor->current_mode->reg_data_size);
> +	ret = ov2680_set_mode(sensor);
>   	if (ret < 0)
>   		return ret;
>   
> @@ -557,21 +531,18 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>   {
>   	struct ov2680_dev *sensor = to_ov2680_dev(sd);
>   	struct v4l2_mbus_framefmt *try_fmt;
> -	const struct ov2680_mode_info *mode;
> +	unsigned int width, height;
>   	int ret = 0;
>   
>   	if (format->pad != 0)
>   		return -EINVAL;
>   
> -	mode = v4l2_find_nearest_size(ov2680_mode_data,
> -				      ARRAY_SIZE(ov2680_mode_data),
> -				      width, height,
> -				      format->format.width,
> -				      format->format.height);
> -	if (!mode)
> -		return -EINVAL;
> +	width = min_t(unsigned int, ALIGN(format->format.width, 2),
> +		      OV2680_NATIVE_WIDTH);
> +	height = min_t(unsigned int, ALIGN(format->format.height, 2),
> +		       OV2680_NATIVE_HEIGHT);
>   
> -	ov2680_fill_format(sensor, &format->format, mode->width, mode->height);
> +	ov2680_fill_format(sensor, &format->format, width, height);
>   
>   	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
>   		try_fmt = v4l2_subdev_get_try_format(sd, sd_state, 0);
> @@ -586,8 +557,8 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
>   		goto unlock;
>   	}
>   
> -	sensor->current_mode = mode;
>   	sensor->mode.fmt = format->format;
> +	ov2680_calc_mode(sensor);
>   
>   unlock:
>   	mutex_unlock(&sensor->lock);
> @@ -609,15 +580,20 @@ static int ov2680_enum_frame_size(struct v4l2_subdev *sd,
>   				  struct v4l2_subdev_state *sd_state,
>   				  struct v4l2_subdev_frame_size_enum *fse)
>   {
> -	int index = fse->index;
> +	static const struct v4l2_frmsize_discrete ov2680_frame_sizes[] = {
> +		{ 1600, 1200 },
> +		{ 1280,  720 },
> +		{  800,  600 },
> +	};
> +	u32 index = fse->index;
>   
> -	if (index >= OV2680_MODE_MAX || index < 0)
> +	if (index >= ARRAY_SIZE(ov2680_frame_sizes))
>   		return -EINVAL;
>   
> -	fse->min_width = ov2680_mode_data[index].width;
> -	fse->min_height = ov2680_mode_data[index].height;
> -	fse->max_width = ov2680_mode_data[index].width;
> -	fse->max_height = ov2680_mode_data[index].height;
> +	fse->min_width = ov2680_frame_sizes[index].width;
> +	fse->min_height = ov2680_frame_sizes[index].height;
> +	fse->max_width = ov2680_frame_sizes[index].width;
> +	fse->max_height = ov2680_frame_sizes[index].height;
>   
>   	return 0;
>   }


Unless I'm missing something, .set_fmt() will let you set any arbitrary frame size you like within 
the bounds of the ov2680's pixel array. That's good, but why then should this callback only report 
the three discrete sizes?

> @@ -700,19 +676,14 @@ static const struct v4l2_subdev_ops ov2680_subdev_ops = {
>   
>   static int ov2680_mode_init(struct ov2680_dev *sensor)
>   {
> -	const struct ov2680_mode_info *init_mode;
> -
>   	/* set initial mode */
>   	ov2680_fill_format(sensor, &sensor->mode.fmt,
>   			   OV2680_DEFAULT_WIDTH, OV2680_DEFAULT_HEIGHT);
> +	ov2680_calc_mode(sensor);
>   
>   	sensor->mode.frame_interval.denominator = OV2680_FRAME_RATE;
>   	sensor->mode.frame_interval.numerator = 1;
>   
> -	init_mode = &ov2680_mode_init_data;
> -
> -	sensor->current_mode = init_mode;
> -
>   	return 0;
>   }
>   

Re: [PATCH v3 21/29] media: ov2680: Make setting the mode algorithm based

[Hans de Goede]

Hi,

On 7/3/23 10:50, Dan Scally wrote:
> Hi Hans
> 
> On 27/06/2023 15:18, Hans de Goede wrote:
>> Instead of using a long fixed register settings list for each resolution,
>> calculate the register settings based on the requested width + height.
>>
>> This is based on atomisp-ov2680 commit 0611888592df ("media: atomisp:
>> ov2680: Make setting the modes algorithm based").
>>
>> This will allow future enhancements like adding hblank and vblank controls
>> and adding selection support.
>>
>> This also adds properly prgramming the ISP window and setting
> 
> 
> s/prgramming/programming
> 
> 
> This looks mostly good but I have one query down below.

<snip>

>> @@ -609,15 +580,20 @@ static int ov2680_enum_frame_size(struct v4l2_subdev *sd,
>>                     struct v4l2_subdev_state *sd_state,
>>                     struct v4l2_subdev_frame_size_enum *fse)
>>   {
>> -    int index = fse->index;
>> +    static const struct v4l2_frmsize_discrete ov2680_frame_sizes[] = {
>> +        { 1600, 1200 },
>> +        { 1280,  720 },
>> +        {  800,  600 },
>> +    };
>> +    u32 index = fse->index;
>>   -    if (index >= OV2680_MODE_MAX || index < 0)
>> +    if (index >= ARRAY_SIZE(ov2680_frame_sizes))
>>           return -EINVAL;
>>   -    fse->min_width = ov2680_mode_data[index].width;
>> -    fse->min_height = ov2680_mode_data[index].height;
>> -    fse->max_width = ov2680_mode_data[index].width;
>> -    fse->max_height = ov2680_mode_data[index].height;
>> +    fse->min_width = ov2680_frame_sizes[index].width;
>> +    fse->min_height = ov2680_frame_sizes[index].height;
>> +    fse->max_width = ov2680_frame_sizes[index].width;
>> +    fse->max_height = ov2680_frame_sizes[index].height;
>>         return 0;
>>   }
> 
> 
> Unless I'm missing something, .set_fmt() will let you set any arbitrary frame size you like within the bounds of the ov2680's pixel array. That's good, but why then should this callback only report the three discrete sizes?

The later added selection (crop-target) support makes it possible for consumers of this driver to find out the real resolution + explicitly set a crop, rather then doing this through s_fmt.

Just like how the ov5693 driver is doing. The ov5693 driver just lists the full + binned size in ov2680_enum_frame_size().

I was a bit reluctant to do this here to break any potential existing users of the drivers, but :

1. Given the poor state of the driver before I doubt that there are any actual users
2. Jacopo has asked me to change the gain control CID which is also an API break

I think we might just as well go ahead here and only list the full + binned sizes like the ov5693 code does. So I'll go an update that for v4.

Regards,

Hans

[PATCH v3 22/29] media: ov2680: Add an __ov2680_get_pad_format() helper function

[Hans de Goede]

Add an __ov2680_get_pad_format() helper function.

This is a preparation patch for adding selections support.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 15e0b0ef5e8f..c692e0457770 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -216,6 +216,18 @@ static void ov2680_set_bayer_order(struct ov2680_dev *sensor,
 	fmt->code = ov2680_hv_flip_bayer_order[hv_flip];
 }
 
+static struct v4l2_mbus_framefmt *
+__ov2680_get_pad_format(struct ov2680_dev *sensor,
+			struct v4l2_subdev_state *state,
+			unsigned int pad,
+			enum v4l2_subdev_format_whence which)
+{
+	if (which == V4L2_SUBDEV_FORMAT_TRY)
+		return v4l2_subdev_get_try_format(&sensor->sd, state, pad);
+
+	return &sensor->mode.fmt;
+}
+
 static void ov2680_fill_format(struct ov2680_dev *sensor,
 			       struct v4l2_mbus_framefmt *fmt,
 			       unsigned int width, unsigned int height)
@@ -504,22 +516,16 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
 			  struct v4l2_subdev_format *format)
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
-	struct v4l2_mbus_framefmt *fmt = NULL;
+	struct v4l2_mbus_framefmt *fmt;
 
 	if (format->pad != 0)
 		return -EINVAL;
 
+	fmt = __ov2680_get_pad_format(sensor, sd_state, format->pad,
+				      format->which);
+
 	mutex_lock(&sensor->lock);
-
-	if (format->which == V4L2_SUBDEV_FORMAT_TRY) {
-		fmt = v4l2_subdev_get_try_format(&sensor->sd, sd_state,
-						 format->pad);
-	} else {
-		fmt = &sensor->mode.fmt;
-	}
-
 	format->format = *fmt;
-
 	mutex_unlock(&sensor->lock);
 
 	return 0;
-- 
2.41.0

[PATCH v3 23/29] media: ov2680: Implement selection support

[Hans de Goede]

Implement selection support. Modelled after ov5693 selection support,
but allow setting sizes smaller than crop-size through set_fmt() since
that was already allowed.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
Changes in v2:
- Use clamp_val() instead of clamp() / clamp_t()
---
 drivers/media/i2c/ov2680.c | 144 ++++++++++++++++++++++++++++++++++---
 1 file changed, 134 insertions(+), 10 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index c692e0457770..6591ce3b9244 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -65,6 +65,14 @@
 
 #define OV2680_NATIVE_WIDTH			1616
 #define OV2680_NATIVE_HEIGHT			1216
+#define OV2680_NATIVE_START_LEFT		0
+#define OV2680_NATIVE_START_TOP			0
+#define OV2680_ACTIVE_WIDTH			1600
+#define OV2680_ACTIVE_HEIGHT			1200
+#define OV2680_ACTIVE_START_LEFT		8
+#define OV2680_ACTIVE_START_TOP			8
+#define OV2680_MIN_CROP_WIDTH			2
+#define OV2680_MIN_CROP_HEIGHT			2
 
 /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
 #define OV2680_PIXELS_PER_LINE			1704
@@ -109,6 +117,7 @@ struct ov2680_ctrls {
 };
 
 struct ov2680_mode {
+	struct v4l2_rect		crop;
 	struct v4l2_mbus_framefmt	fmt;
 	struct v4l2_fract		frame_interval;
 	bool				binning;
@@ -143,6 +152,13 @@ struct ov2680_dev {
 	struct ov2680_mode		mode;
 };
 
+static const struct v4l2_rect ov2680_default_crop = {
+	.left = OV2680_ACTIVE_START_LEFT,
+	.top = OV2680_ACTIVE_START_TOP,
+	.width = OV2680_ACTIVE_WIDTH,
+	.height = OV2680_ACTIVE_HEIGHT,
+};
+
 static const char * const test_pattern_menu[] = {
 	"Disabled",
 	"Color Bars",
@@ -228,6 +244,18 @@ __ov2680_get_pad_format(struct ov2680_dev *sensor,
 	return &sensor->mode.fmt;
 }
 
+static struct v4l2_rect *
+__ov2680_get_pad_crop(struct ov2680_dev *sensor,
+		      struct v4l2_subdev_state *state,
+		      unsigned int pad,
+		      enum v4l2_subdev_format_whence which)
+{
+	if (which == V4L2_SUBDEV_FORMAT_TRY)
+		return v4l2_subdev_get_try_crop(&sensor->sd, state, pad);
+
+	return &sensor->mode.crop;
+}
+
 static void ov2680_fill_format(struct ov2680_dev *sensor,
 			       struct v4l2_mbus_framefmt *fmt,
 			       unsigned int width, unsigned int height)
@@ -247,8 +275,8 @@ static void ov2680_calc_mode(struct ov2680_dev *sensor)
 	int orig_width = width;
 	int orig_height = height;
 
-	if (width  <= (OV2680_NATIVE_WIDTH / 2) &&
-	    height <= (OV2680_NATIVE_HEIGHT / 2)) {
+	if (width  <= (sensor->mode.crop.width / 2) &&
+	    height <= (sensor->mode.crop.height / 2)) {
 		sensor->mode.binning = true;
 		width *= 2;
 		height *= 2;
@@ -256,8 +284,10 @@ static void ov2680_calc_mode(struct ov2680_dev *sensor)
 		sensor->mode.binning = false;
 	}
 
-	sensor->mode.h_start = ((OV2680_NATIVE_WIDTH - width) / 2) & ~1;
-	sensor->mode.v_start = ((OV2680_NATIVE_HEIGHT - height) / 2) & ~1;
+	sensor->mode.h_start = (sensor->mode.crop.left +
+				(sensor->mode.crop.width - width) / 2) & ~1;
+	sensor->mode.v_start = (sensor->mode.crop.top +
+				(sensor->mode.crop.height - height) / 2) & ~1;
 	sensor->mode.h_end =
 		min(sensor->mode.h_start + width + OV2680_END_MARGIN - 1,
 		    OV2680_NATIVE_WIDTH - 1);
@@ -537,16 +567,21 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 	struct v4l2_mbus_framefmt *try_fmt;
+	const struct v4l2_rect *crop;
 	unsigned int width, height;
 	int ret = 0;
 
 	if (format->pad != 0)
 		return -EINVAL;
 
-	width = min_t(unsigned int, ALIGN(format->format.width, 2),
-		      OV2680_NATIVE_WIDTH);
-	height = min_t(unsigned int, ALIGN(format->format.height, 2),
-		       OV2680_NATIVE_HEIGHT);
+	crop = __ov2680_get_pad_crop(sensor, sd_state, format->pad,
+				     format->which);
+
+	/* Limit set_fmt max size to crop width / height */
+	width = clamp_val(ALIGN(format->format.width, 2),
+			  OV2680_MIN_CROP_WIDTH, crop->width);
+	height = clamp_val(ALIGN(format->format.height, 2),
+			   OV2680_MIN_CROP_HEIGHT, crop->height);
 
 	ov2680_fill_format(sensor, &format->format, width, height);
 
@@ -572,11 +607,97 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 	return ret;
 }
 
+static int ov2680_get_selection(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *state,
+				struct v4l2_subdev_selection *sel)
+{
+	struct ov2680_dev *sensor = to_ov2680_dev(sd);
+
+	switch (sel->target) {
+	case V4L2_SEL_TGT_CROP:
+		mutex_lock(&sensor->lock);
+		sel->r = *__ov2680_get_pad_crop(sensor, state, sel->pad,
+						sel->which);
+		mutex_unlock(&sensor->lock);
+		break;
+	case V4L2_SEL_TGT_NATIVE_SIZE:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+		sel->r.top = 0;
+		sel->r.left = 0;
+		sel->r.width = OV2680_NATIVE_WIDTH;
+		sel->r.height = OV2680_NATIVE_HEIGHT;
+		break;
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		sel->r = ov2680_default_crop;
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int ov2680_set_selection(struct v4l2_subdev *sd,
+				struct v4l2_subdev_state *state,
+				struct v4l2_subdev_selection *sel)
+{
+	struct ov2680_dev *sensor = to_ov2680_dev(sd);
+	struct v4l2_mbus_framefmt *format;
+	struct v4l2_rect *crop;
+	struct v4l2_rect rect;
+
+	if (sel->target != V4L2_SEL_TGT_CROP)
+		return -EINVAL;
+
+	/*
+	 * Clamp the boundaries of the crop rectangle to the size of the sensor
+	 * pixel array. Align to multiples of 2 to ensure Bayer pattern isn't
+	 * disrupted.
+	 */
+	rect.left = clamp_val(ALIGN(sel->r.left, 2),
+			      OV2680_NATIVE_START_LEFT, OV2680_NATIVE_WIDTH);
+	rect.top = clamp_val(ALIGN(sel->r.top, 2),
+			     OV2680_NATIVE_START_TOP, OV2680_NATIVE_HEIGHT);
+	rect.width = clamp_val(ALIGN(sel->r.width, 2),
+			       OV2680_MIN_CROP_WIDTH, OV2680_NATIVE_WIDTH);
+	rect.height = clamp_val(ALIGN(sel->r.height, 2),
+				OV2680_MIN_CROP_HEIGHT, OV2680_NATIVE_HEIGHT);
+
+	/* Make sure the crop rectangle isn't outside the bounds of the array */
+	rect.width = min_t(unsigned int, rect.width,
+			   OV2680_NATIVE_WIDTH - rect.left);
+	rect.height = min_t(unsigned int, rect.height,
+			    OV2680_NATIVE_HEIGHT - rect.top);
+
+	crop = __ov2680_get_pad_crop(sensor, state, sel->pad, sel->which);
+
+	mutex_lock(&sensor->lock);
+	if (rect.width != crop->width || rect.height != crop->height) {
+		/*
+		 * Reset the output image size if the crop rectangle size has
+		 * been modified.
+		 */
+		format = __ov2680_get_pad_format(sensor, state, sel->pad,
+						 sel->which);
+		format->width = rect.width;
+		format->height = rect.height;
+	}
+
+	*crop = rect;
+	mutex_unlock(&sensor->lock);
+
+	sel->r = rect;
+
+	return 0;
+}
+
 static int ov2680_init_cfg(struct v4l2_subdev *sd,
 			   struct v4l2_subdev_state *sd_state)
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 
+	sd_state->pads[0].try_crop = ov2680_default_crop;
+
 	ov2680_fill_format(sensor, &sd_state->pads[0].try_fmt,
 			   OV2680_DEFAULT_WIDTH, OV2680_DEFAULT_HEIGHT);
 	return 0;
@@ -669,10 +790,12 @@ static const struct v4l2_subdev_video_ops ov2680_video_ops = {
 static const struct v4l2_subdev_pad_ops ov2680_pad_ops = {
 	.init_cfg		= ov2680_init_cfg,
 	.enum_mbus_code		= ov2680_enum_mbus_code,
-	.get_fmt		= ov2680_get_fmt,
-	.set_fmt		= ov2680_set_fmt,
 	.enum_frame_size	= ov2680_enum_frame_size,
 	.enum_frame_interval	= ov2680_enum_frame_interval,
+	.get_fmt		= ov2680_get_fmt,
+	.set_fmt		= ov2680_set_fmt,
+	.get_selection		= ov2680_get_selection,
+	.set_selection		= ov2680_set_selection,
 };
 
 static const struct v4l2_subdev_ops ov2680_subdev_ops = {
@@ -683,6 +806,7 @@ static const struct v4l2_subdev_ops ov2680_subdev_ops = {
 static int ov2680_mode_init(struct ov2680_dev *sensor)
 {
 	/* set initial mode */
+	sensor->mode.crop = ov2680_default_crop;
 	ov2680_fill_format(sensor, &sensor->mode.fmt,
 			   OV2680_DEFAULT_WIDTH, OV2680_DEFAULT_HEIGHT);
 	ov2680_calc_mode(sensor);
-- 
2.41.0

[PATCH v3 24/29] media: ov2680: Fix exposure and gain ctrls range and default value

[Hans de Goede]

The exposure control's max effective value is VTS - 8, set the control
range to match this. Thas means that if/when a future commit makes VTS
configurable as a control itself the exposure range needs to be
updated dynamically to match the VTS value.

The gain control goes from 0 - 1023, higher values wrap around to
the lowest gain setting.

Also stop setting 0 as default for both controls this leads to
a totally black picture which is no good. This is esp. important
for tests of the sensor driver without (userspace driven)
auto exposure / gain.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 6591ce3b9244..e26a6487d421 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -81,6 +81,9 @@
 /* If possible send 16 extra rows / lines to the ISP as padding */
 #define OV2680_END_MARGIN			16
 
+/* Max exposure time is VTS - 8 */
+#define OV2680_INTEGRATION_TIME_MARGIN		8
+
 #define OV2680_DEFAULT_WIDTH			800
 #define OV2680_DEFAULT_HEIGHT			600
 
@@ -823,6 +826,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 	const struct v4l2_ctrl_ops *ops = &ov2680_ctrl_ops;
 	struct ov2680_ctrls *ctrls = &sensor->ctrls;
 	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
+	int exp_max = OV2680_LINES_PER_FRAME - OV2680_INTEGRATION_TIME_MARGIN;
 	int ret = 0;
 
 	v4l2_i2c_subdev_init(&sensor->sd, client, &ov2680_subdev_ops);
@@ -849,9 +853,10 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 					0, 0, test_pattern_menu);
 
 	ctrls->exposure = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_EXPOSURE,
-					    0, 32767, 1, 0);
+					    0, exp_max, 1, exp_max);
 
-	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN, 0, 2047, 1, 0);
+	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
+					0, 1023, 1, 250);
 
 	if (hdl->error) {
 		ret = hdl->error;
-- 
2.41.0

Re: [PATCH v3 24/29] media: ov2680: Fix exposure and gain ctrls range and default value

[Jacopo Mondi]

Hi Hans
  another drive-by question

On Tue, Jun 27, 2023 at 03:18:25PM +0200, Hans de Goede wrote:
> The exposure control's max effective value is VTS - 8, set the control
> range to match this. Thas means that if/when a future commit makes VTS
> configurable as a control itself the exposure range needs to be
> updated dynamically to match the VTS value.
>
> The gain control goes from 0 - 1023, higher values wrap around to
> the lowest gain setting.
>
> Also stop setting 0 as default for both controls this leads to
> a totally black picture which is no good. This is esp. important
> for tests of the sensor driver without (userspace driven)
> auto exposure / gain.

I see the driver uses V4L2_CID_GAIN. Is this intentional or should
this be V4L2_CID_ANALOGUE_GAIN? As you're plumbing libcamera support
in, this is the control libcamera expects to use to control analogue
gain.

>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>  drivers/media/i2c/ov2680.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index 6591ce3b9244..e26a6487d421 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -81,6 +81,9 @@
>  /* If possible send 16 extra rows / lines to the ISP as padding */
>  #define OV2680_END_MARGIN			16
>
> +/* Max exposure time is VTS - 8 */
> +#define OV2680_INTEGRATION_TIME_MARGIN		8
> +
>  #define OV2680_DEFAULT_WIDTH			800
>  #define OV2680_DEFAULT_HEIGHT			600
>
> @@ -823,6 +826,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>  	const struct v4l2_ctrl_ops *ops = &ov2680_ctrl_ops;
>  	struct ov2680_ctrls *ctrls = &sensor->ctrls;
>  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> +	int exp_max = OV2680_LINES_PER_FRAME - OV2680_INTEGRATION_TIME_MARGIN;
>  	int ret = 0;
>
>  	v4l2_i2c_subdev_init(&sensor->sd, client, &ov2680_subdev_ops);
> @@ -849,9 +853,10 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>  					0, 0, test_pattern_menu);
>
>  	ctrls->exposure = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_EXPOSURE,
> -					    0, 32767, 1, 0);
> +					    0, exp_max, 1, exp_max);
>
> -	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN, 0, 2047, 1, 0);
> +	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
> +					0, 1023, 1, 250);
>
>  	if (hdl->error) {
>  		ret = hdl->error;
> --
> 2.41.0
>

Re: [PATCH v3 24/29] media: ov2680: Fix exposure and gain ctrls range and default value

[Hans de Goede]

Hi Jacopo,

On 6/27/23 17:16, Jacopo Mondi wrote:
> Hi Hans
>   another drive-by question
> 
> On Tue, Jun 27, 2023 at 03:18:25PM +0200, Hans de Goede wrote:
>> The exposure control's max effective value is VTS - 8, set the control
>> range to match this. Thas means that if/when a future commit makes VTS
>> configurable as a control itself the exposure range needs to be
>> updated dynamically to match the VTS value.
>>
>> The gain control goes from 0 - 1023, higher values wrap around to
>> the lowest gain setting.
>>
>> Also stop setting 0 as default for both controls this leads to
>> a totally black picture which is no good. This is esp. important
>> for tests of the sensor driver without (userspace driven)
>> auto exposure / gain.
> 
> I see the driver uses V4L2_CID_GAIN. Is this intentional or should
> this be V4L2_CID_ANALOGUE_GAIN? As you're plumbing libcamera support
> in, this is the control libcamera expects to use to control analogue
> gain.

That is a good question. I've not changed this for worries about
existing users. Although given the previous state of the existing
code I wonder if there are any existing users?

So what is the policy on this ?

Also I still need to figure out what the actual range
(as in amplification at lowest + highest setting) of the gain
control is, because AFAIk libcamera wants to know this.

Any hints on how to do this ? Also are there any docs on
how a driver should implement V4L2_CID_ANALOGUE_GAIN wrt range?

E.g. is the driver expected do to some conversion of values
to make the control always set a specific amplification for
a specific value?

Regards,

Hans



> 
>>
>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>> ---
>>  drivers/media/i2c/ov2680.c | 9 +++++++--
>>  1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>> index 6591ce3b9244..e26a6487d421 100644
>> --- a/drivers/media/i2c/ov2680.c
>> +++ b/drivers/media/i2c/ov2680.c
>> @@ -81,6 +81,9 @@
>>  /* If possible send 16 extra rows / lines to the ISP as padding */
>>  #define OV2680_END_MARGIN			16
>>
>> +/* Max exposure time is VTS - 8 */
>> +#define OV2680_INTEGRATION_TIME_MARGIN		8
>> +
>>  #define OV2680_DEFAULT_WIDTH			800
>>  #define OV2680_DEFAULT_HEIGHT			600
>>
>> @@ -823,6 +826,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>  	const struct v4l2_ctrl_ops *ops = &ov2680_ctrl_ops;
>>  	struct ov2680_ctrls *ctrls = &sensor->ctrls;
>>  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
>> +	int exp_max = OV2680_LINES_PER_FRAME - OV2680_INTEGRATION_TIME_MARGIN;
>>  	int ret = 0;
>>
>>  	v4l2_i2c_subdev_init(&sensor->sd, client, &ov2680_subdev_ops);
>> @@ -849,9 +853,10 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>  					0, 0, test_pattern_menu);
>>
>>  	ctrls->exposure = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_EXPOSURE,
>> -					    0, 32767, 1, 0);
>> +					    0, exp_max, 1, exp_max);
>>
>> -	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN, 0, 2047, 1, 0);
>> +	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
>> +					0, 1023, 1, 250);
>>
>>  	if (hdl->error) {
>>  		ret = hdl->error;
>> --
>> 2.41.0
>>
> 

Re: [PATCH v3 24/29] media: ov2680: Fix exposure and gain ctrls range and default value

[Jacopo Mondi]

Hi Han

On Tue, Jun 27, 2023 at 06:26:08PM +0200, Hans de Goede wrote:
> Hi Jacopo,
>
> On 6/27/23 17:16, Jacopo Mondi wrote:
> > Hi Hans
> >   another drive-by question
> >
> > On Tue, Jun 27, 2023 at 03:18:25PM +0200, Hans de Goede wrote:
> >> The exposure control's max effective value is VTS - 8, set the control
> >> range to match this. Thas means that if/when a future commit makes VTS
> >> configurable as a control itself the exposure range needs to be
> >> updated dynamically to match the VTS value.
> >>
> >> The gain control goes from 0 - 1023, higher values wrap around to
> >> the lowest gain setting.
> >>
> >> Also stop setting 0 as default for both controls this leads to
> >> a totally black picture which is no good. This is esp. important
> >> for tests of the sensor driver without (userspace driven)
> >> auto exposure / gain.
> >
> > I see the driver uses V4L2_CID_GAIN. Is this intentional or should
> > this be V4L2_CID_ANALOGUE_GAIN? As you're plumbing libcamera support
> > in, this is the control libcamera expects to use to control analogue
> > gain.
>
> That is a good question. I've not changed this for worries about
> existing users. Although given the previous state of the existing
> code I wonder if there are any existing users?

That's a fair concern

>
> So what is the policy on this ?
>
> Also I still need to figure out what the actual range
> (as in amplification at lowest + highest setting) of the gain
> control is, because AFAIk libcamera wants to know this.

libcamera has helpers that translate the scalar gain values computed
by the AEGC algorithms (or supplied by the application) to sensor's
device specific gain codes (and the other way around)
https://git.libcamera.org/libcamera/libcamera.git/tree/src/ipa/libipa/camera_sensor_helper.cpp

So it's fine if the driver exposes the device specific gain codes from
a libcamera perspective. True, there are sensors drivers where the
gain codes interval has been linearized and made continuous to make it
easier for userspace to control it. I tried to dig it out an example,
but so far memory is failing me...

Does this answer your question or have I misinterpreted it ?

When it comes to gain vs analogue gain, it should be clarified if
OV2680_REG_GAIN_PK only controls the analogue gain amplification or
also the digital gain part. I don't have a datasheet so I can't help
much there....

Also see afa4805799c1 ("media: ov5640: Fix analogue gain control") as an
example for a driver being ported from GAIN to ANALOGUE_GAIN (so far
nobody complained, so I presume this doesn't count as a regression :)

>
> Any hints on how to do this ? Also are there any docs on
> how a driver should implement V4L2_CID_ANALOGUE_GAIN wrt range?
>
> E.g. is the driver expected do to some conversion of values
> to make the control always set a specific amplification for
> a specific value?
>
> Regards,
>
> Hans
>
>
>
> >
> >>
> >> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> >> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> >> ---
> >>  drivers/media/i2c/ov2680.c | 9 +++++++--
> >>  1 file changed, 7 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> >> index 6591ce3b9244..e26a6487d421 100644
> >> --- a/drivers/media/i2c/ov2680.c
> >> +++ b/drivers/media/i2c/ov2680.c
> >> @@ -81,6 +81,9 @@
> >>  /* If possible send 16 extra rows / lines to the ISP as padding */
> >>  #define OV2680_END_MARGIN			16
> >>
> >> +/* Max exposure time is VTS - 8 */
> >> +#define OV2680_INTEGRATION_TIME_MARGIN		8
> >> +
> >>  #define OV2680_DEFAULT_WIDTH			800
> >>  #define OV2680_DEFAULT_HEIGHT			600
> >>
> >> @@ -823,6 +826,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
> >>  	const struct v4l2_ctrl_ops *ops = &ov2680_ctrl_ops;
> >>  	struct ov2680_ctrls *ctrls = &sensor->ctrls;
> >>  	struct v4l2_ctrl_handler *hdl = &ctrls->handler;
> >> +	int exp_max = OV2680_LINES_PER_FRAME - OV2680_INTEGRATION_TIME_MARGIN;
> >>  	int ret = 0;
> >>
> >>  	v4l2_i2c_subdev_init(&sensor->sd, client, &ov2680_subdev_ops);
> >> @@ -849,9 +853,10 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
> >>  					0, 0, test_pattern_menu);
> >>
> >>  	ctrls->exposure = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_EXPOSURE,
> >> -					    0, 32767, 1, 0);
> >> +					    0, exp_max, 1, exp_max);
> >>
> >> -	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN, 0, 2047, 1, 0);
> >> +	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
> >> +					0, 1023, 1, 250);
> >>
> >>  	if (hdl->error) {
> >>  		ret = hdl->error;
> >> --
> >> 2.41.0
> >>
> >
>

[PATCH v3 25/29] media: ov2680: Add a bunch of register tweaks

[Hans de Goede]

Usually when developing a sensor driver with help from the vendor
the vendor will provide a bunch of register tweaks for optimal
performance of the sensor.

The atomisp-ov2680.c driver was (presumably) developed by Intel with
help from OmniVision and indeed contains a bunch of register tweaks.

Add these register tweaks to the "main" ov2680.c driver.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 53 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index e26a6487d421..6c9f038c4631 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -178,18 +178,71 @@ static const int ov2680_hv_flip_bayer_order[] = {
 };
 
 static const struct reg_sequence ov2680_global_setting[] = {
+	/* MIPI PHY, 0x10 -> 0x1c enable bp_c_hs_en_lat and bp_d_hs_en_lat */
+	{0x3016, 0x1c},
+
 	/* R MANUAL set exposure and gain to manual (hw does not do auto) */
 	{0x3503, 0x03},
 
+	/* Analog control register tweaks */
+	{0x3603, 0x39}, /* Reset value 0x99 */
+	{0x3604, 0x24}, /* Reset value 0x74 */
+	{0x3621, 0x37}, /* Reset value 0x44 */
+
+	/* Sensor control register tweaks */
+	{0x3701, 0x64}, /* Reset value 0x61 */
+	{0x3705, 0x3c}, /* Reset value 0x21 */
+	{0x370c, 0x50}, /* Reset value 0x10 */
+	{0x370d, 0xc0}, /* Reset value 0x00 */
+	{0x3718, 0x88}, /* Reset value 0x80 */
+
+	/* PSRAM tweaks */
+	{0x3781, 0x80}, /* Reset value 0x00 */
+	{0x3784, 0x0c}, /* Reset value 0x00, based on OV2680_R1A_AM10.ovt */
+	{0x3789, 0x60}, /* Reset value 0x50 */
+
+	/* BLC CTRL00 0x01 -> 0x81 set avg_weight to 8 */
+	{0x4000, 0x81},
+
 	/* Set black level compensation range to 0 - 3 (default 0 - 11) */
 	{0x4008, 0x00},
 	{0x4009, 0x03},
 
+	/* VFIFO R2 0x00 -> 0x02 set Frame reset enable */
+	{0x4602, 0x02},
+
+	/* MIPI ctrl CLK PREPARE MIN change from 0x26 (38) -> 0x36 (54) */
+	{0x481f, 0x36},
+
+	/* MIPI ctrl CLK LPX P MIN change from 0x32 (50) -> 0x36 (54) */
+	{0x4825, 0x36},
+
+	/* R ISP CTRL2 0x20 -> 0x30, set sof_sel bit */
+	{0x5002, 0x30},
+
 	/*
 	 * Window CONTROL 0x00 -> 0x01, enable manual window control,
 	 * this is necessary for full size flip and mirror support.
 	 */
 	{0x5708, 0x01},
+
+	/*
+	 * DPC CTRL0 0x14 -> 0x3e, set enable_tail, enable_3x3_cluster
+	 * and enable_general_tail bits based OV2680_R1A_AM10.ovt.
+	 */
+	{0x5780, 0x3e},
+
+	/* DPC MORE CONNECTION CASE THRE 0x0c (12) -> 0x02 (2) */
+	{0x5788, 0x02},
+
+	/* DPC GAIN LIST1 0x0f (15) -> 0x08 (8) */
+	{0x578e, 0x08},
+
+	/* DPC GAIN LIST2 0x3f (63) -> 0x0c (12) */
+	{0x578f, 0x0c},
+
+	/* DPC THRE RATIO 0x04 (4) -> 0x00 (0) */
+	{0x5792, 0x00},
 };
 
 static struct ov2680_dev *to_ov2680_dev(struct v4l2_subdev *sd)
-- 
2.41.0

[PATCH v3 26/29] media: ov2680: Drop unnecessary pad checks

[Hans de Goede]

Drop unnecessary pad checks in enum_mbus_code, get_fmt, set_fmt
this is already checked by check_pad() from
drivers/media/v4l2-core/v4l2-subdev.c.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 6c9f038c4631..d1de2fc50f4e 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -589,7 +589,7 @@ static int ov2680_enum_mbus_code(struct v4l2_subdev *sd,
 {
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 
-	if (code->pad != 0 || code->index != 0)
+	if (code->index != 0)
 		return -EINVAL;
 
 	code->code = sensor->mode.fmt.code;
@@ -604,9 +604,6 @@ static int ov2680_get_fmt(struct v4l2_subdev *sd,
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 	struct v4l2_mbus_framefmt *fmt;
 
-	if (format->pad != 0)
-		return -EINVAL;
-
 	fmt = __ov2680_get_pad_format(sensor, sd_state, format->pad,
 				      format->which);
 
@@ -627,9 +624,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 	unsigned int width, height;
 	int ret = 0;
 
-	if (format->pad != 0)
-		return -EINVAL;
-
 	crop = __ov2680_get_pad_crop(sensor, sd_state, format->pad,
 				     format->which);
 
-- 
2.41.0

[PATCH v3 27/29] media: ov2680: Read and log sensor revision during probe

[Hans de Goede]

Read and log sensor revision during probe.

Since this means that the driver will now already log a message on
successful probe drop the "ov2680 init correctly" log message.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index d1de2fc50f4e..8bc542df1890 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -33,6 +33,7 @@
 #define OV2680_REG_SOFT_RESET			CCI_REG8(0x0103)
 
 #define OV2680_REG_CHIP_ID			CCI_REG16(0x300a)
+#define OV2680_REG_SC_CMMN_SUB_ID		CCI_REG8(0x302a)
 #define OV2680_REG_PLL_MULTIPLIER		CCI_REG16(0x3081)
 
 #define OV2680_REG_EXPOSURE_PK			CCI_REG24(0x3500)
@@ -941,13 +942,14 @@ static int ov2680_get_regulators(struct ov2680_dev *sensor)
 
 static int ov2680_check_id(struct ov2680_dev *sensor)
 {
-	u64 chip_id;
-	int ret;
+	u64 chip_id, rev;
+	int ret = 0;
 
-	ret = cci_read(sensor->regmap, OV2680_REG_CHIP_ID, &chip_id, NULL);
+	cci_read(sensor->regmap, OV2680_REG_CHIP_ID, &chip_id, &ret);
+	cci_read(sensor->regmap, OV2680_REG_SC_CMMN_SUB_ID, &rev, &ret);
 	if (ret < 0) {
 		dev_err(sensor->dev, "failed to read chip id\n");
-		return -ENODEV;
+		return ret;
 	}
 
 	if (chip_id != OV2680_CHIP_ID) {
@@ -956,6 +958,9 @@ static int ov2680_check_id(struct ov2680_dev *sensor)
 		return -ENODEV;
 	}
 
+	dev_info(sensor->dev, "sensor_revision id = 0x%llx, rev= %lld\n",
+		 chip_id, rev & 0x0f);
+
 	return 0;
 }
 
@@ -1094,8 +1099,6 @@ static int ov2680_probe(struct i2c_client *client)
 	pm_runtime_use_autosuspend(&client->dev);
 	pm_runtime_put_autosuspend(&client->dev);
 
-	dev_info(dev, "ov2680 init correctly\n");
-
 	return 0;
 
 err_pm_runtime:
-- 
2.41.0

[PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Hans de Goede]

Add read-only link-freq and pixel-rate controls. This is necessary for
the sensor to work with the ipu3-cio2 driver and for libcamera.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 8bc542df1890..95d3152ddd22 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -75,6 +75,12 @@
 #define OV2680_MIN_CROP_WIDTH			2
 #define OV2680_MIN_CROP_HEIGHT			2
 
+/* Fixed pre-div of 1/2 */
+#define OV2680_PLL_PREDIV0			2
+
+/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
+#define OV2680_PLL_PREDIV			2
+
 /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
 #define OV2680_PIXELS_PER_LINE			1704
 #define OV2680_LINES_PER_FRAME			1294
@@ -118,6 +124,8 @@ struct ov2680_ctrls {
 	struct v4l2_ctrl *hflip;
 	struct v4l2_ctrl *vflip;
 	struct v4l2_ctrl *test_pattern;
+	struct v4l2_ctrl *link_freq;
+	struct v4l2_ctrl *pixel_rate;
 };
 
 struct ov2680_mode {
@@ -145,6 +153,8 @@ struct ov2680_dev {
 	struct clk			*xvclk;
 	u32				xvclk_freq;
 	u8				pll_mult;
+	s64				link_freq[1];
+	s64				pixel_rate;
 	struct regulator_bulk_data	supplies[OV2680_NUM_SUPPLIES];
 
 	struct gpio_desc		*pwdn_gpio;
@@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
 					0, 1023, 1, 250);
 
+	ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
+						  0, 0, sensor->link_freq);
+	ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
+					      0, sensor->pixel_rate,
+					      1, sensor->pixel_rate);
+
 	if (hdl->error) {
 		ret = hdl->error;
 		goto cleanup_entity;
@@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
 
 	ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
 	ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
+	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
 
 	sensor->sd.ctrl_handler = hdl;
 
@@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
 
 	sensor->pll_mult = ov2680_pll_multipliers[i];
 
+	sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
+			       OV2680_PLL_PREDIV * sensor->pll_mult;
+
+	/* CSI-2 is double data rate, bus-format is 10 bpp */
+	sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;
+
 	return 0;
 }
 
-- 
2.41.0

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Dan Scally]

Morning Hans

On 27/06/2023 15:18, Hans de Goede wrote:
> Add read-only link-freq and pixel-rate controls. This is necessary for
> the sensor to work with the ipu3-cio2 driver and for libcamera.
>
> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---
>   drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
>   1 file changed, 23 insertions(+)
>
> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> index 8bc542df1890..95d3152ddd22 100644
> --- a/drivers/media/i2c/ov2680.c
> +++ b/drivers/media/i2c/ov2680.c
> @@ -75,6 +75,12 @@
>   #define OV2680_MIN_CROP_WIDTH			2
>   #define OV2680_MIN_CROP_HEIGHT			2
>   
> +/* Fixed pre-div of 1/2 */
> +#define OV2680_PLL_PREDIV0			2
> +
> +/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
> +#define OV2680_PLL_PREDIV			2
> +
>   /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
>   #define OV2680_PIXELS_PER_LINE			1704
>   #define OV2680_LINES_PER_FRAME			1294
> @@ -118,6 +124,8 @@ struct ov2680_ctrls {
>   	struct v4l2_ctrl *hflip;
>   	struct v4l2_ctrl *vflip;
>   	struct v4l2_ctrl *test_pattern;
> +	struct v4l2_ctrl *link_freq;
> +	struct v4l2_ctrl *pixel_rate;
>   };
>   
>   struct ov2680_mode {
> @@ -145,6 +153,8 @@ struct ov2680_dev {
>   	struct clk			*xvclk;
>   	u32				xvclk_freq;
>   	u8				pll_mult;
> +	s64				link_freq[1];
> +	s64				pixel_rate;
>   	struct regulator_bulk_data	supplies[OV2680_NUM_SUPPLIES];
>   
>   	struct gpio_desc		*pwdn_gpio;
> @@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>   	ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
>   					0, 1023, 1, 250);
>   
> +	ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
> +						  0, 0, sensor->link_freq);
> +	ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
> +					      0, sensor->pixel_rate,
> +					      1, sensor->pixel_rate);
> +
>   	if (hdl->error) {
>   		ret = hdl->error;
>   		goto cleanup_entity;
> @@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>   
>   	ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>   	ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
> +	ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>   
>   	sensor->sd.ctrl_handler = hdl;
>   
> @@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>   
>   	sensor->pll_mult = ov2680_pll_multipliers[i];
>   
> +	sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
> +			       OV2680_PLL_PREDIV * sensor->pll_mult;
> +
> +	/* CSI-2 is double data rate, bus-format is 10 bpp */
> +	sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;


I'm a little unsure on this one. My understanding is that the link frequency really ought to come 
from the endpoint properties (which in our case would be set by the ipu-bridge; though it doesn't 
for this sensor at the moment because I didn't understand it properly back then) because it's a 
platform specific thing. What the value should be, I have been determining by reading the PLL 
settings for the sensor whilst the laptop's running Windows. So whilst this is probably technically 
fine in supporting the link frequency that the driver already expects to configure for whatever 
platform this was originally designed for, my guess would be that the Miix expects a different link 
frequency and ideally we'd support that instead. For example see these commits for the ov7251:


ed9566ce1946 media: i2c: Add support for new frequencies to ov7251
df057b0dd99b media: i2c: Add ov7251_pll_configure()
1757b44eb6bb media: i2c: Remove per-mode frequencies from ov7251
cc125aaa5a78 media: i2c: Provide ov7251_check_hwcfg()


Which keeps support for the link frequency that the driver already configured and adds support for 
the new one, for both 19.2 and 24MHz input clocks.


What would be your thoughts on moving #15 and this patch to something like that?


> +
>   	return 0;
>   }
>   

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Hans de Goede]

Hi Dan,

On 7/4/23 11:22, Dan Scally wrote:
> Morning Hans
> 
> On 27/06/2023 15:18, Hans de Goede wrote:
>> Add read-only link-freq and pixel-rate controls. This is necessary for
>> the sensor to work with the ipu3-cio2 driver and for libcamera.
>>
>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>> ---
>>   drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
>>   1 file changed, 23 insertions(+)
>>
>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>> index 8bc542df1890..95d3152ddd22 100644
>> --- a/drivers/media/i2c/ov2680.c
>> +++ b/drivers/media/i2c/ov2680.c
>> @@ -75,6 +75,12 @@
>>   #define OV2680_MIN_CROP_WIDTH            2
>>   #define OV2680_MIN_CROP_HEIGHT            2
>>   +/* Fixed pre-div of 1/2 */
>> +#define OV2680_PLL_PREDIV0            2
>> +
>> +/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
>> +#define OV2680_PLL_PREDIV            2
>> +
>>   /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
>>   #define OV2680_PIXELS_PER_LINE            1704
>>   #define OV2680_LINES_PER_FRAME            1294
>> @@ -118,6 +124,8 @@ struct ov2680_ctrls {
>>       struct v4l2_ctrl *hflip;
>>       struct v4l2_ctrl *vflip;
>>       struct v4l2_ctrl *test_pattern;
>> +    struct v4l2_ctrl *link_freq;
>> +    struct v4l2_ctrl *pixel_rate;
>>   };
>>     struct ov2680_mode {
>> @@ -145,6 +153,8 @@ struct ov2680_dev {
>>       struct clk            *xvclk;
>>       u32                xvclk_freq;
>>       u8                pll_mult;
>> +    s64                link_freq[1];
>> +    s64                pixel_rate;
>>       struct regulator_bulk_data    supplies[OV2680_NUM_SUPPLIES];
>>         struct gpio_desc        *pwdn_gpio;
>> @@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>       ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
>>                       0, 1023, 1, 250);
>>   +    ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
>> +                          0, 0, sensor->link_freq);
>> +    ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
>> +                          0, sensor->pixel_rate,
>> +                          1, sensor->pixel_rate);
>> +
>>       if (hdl->error) {
>>           ret = hdl->error;
>>           goto cleanup_entity;
>> @@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>         ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>>       ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>> +    ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>>         sensor->sd.ctrl_handler = hdl;
>>   @@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>         sensor->pll_mult = ov2680_pll_multipliers[i];
>>   +    sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
>> +                   OV2680_PLL_PREDIV * sensor->pll_mult;
>> +
>> +    /* CSI-2 is double data rate, bus-format is 10 bpp */
>> +    sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;
> 
> 
> I'm a little unsure on this one. My understanding is that the link frequency really ought to come from the endpoint properties (which in our case would be set by the ipu-bridge; though it doesn't for this sensor at the moment because I didn't understand it properly back then) because it's a platform specific thing. What the value should be, I have been determining by reading the PLL settings for the sensor whilst the laptop's running Windows. So whilst this is probably technically fine in supporting the link frequency that the driver already expects to configure for whatever platform this was originally designed for, my guess would be that the Miix expects a different link frequency and ideally we'd support that instead. For example see these commits for the ov7251:

The datasheet is clear that the ov2680 is intended to be used with
a fixed pixelclock of 66 MHz:

"2.2 architecture

The OV2680 sensor core generates streaming pixel data at a constant
frame rate to a pixel clock of 66 MHz."

and the ov2680 always operates in single lane mode. So there really is
not much to configure here.

Also the datasheet only contains a rudimentary description of the PLL,
which is not really enough to write a function to configure the PLL for
arbitrary link-frequencies.

The adjustment to make the sensor work with a 19.2MHz xvclk instead
of the default 24 MHz comes from the atomisp code. How to make other
adjustments would pretty much be guess work.

I guess we could add code to check the link-frequencies and check
that there is only 1 and it matches the expected 330 MHz then the driver
still honors the link-frequencies property while at the same time
sticking with the fixed setup the sensor is intended to be used with.

Would adding a link-frequency check like that work for you ?

And if yes what should the link-frequency control return then,
the actual achieved frequency (this would be better IMHO) or
the one from the property ?

Regards,

Hans

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Dave Stevenson]

Hi Hans and Dan

On Tue, 4 Jul 2023 at 10:45, Hans de Goede <hdegoede@redhat.com> wrote:
>
> Hi Dan,
>
> On 7/4/23 11:22, Dan Scally wrote:
> > Morning Hans
> >
> > On 27/06/2023 15:18, Hans de Goede wrote:
> >> Add read-only link-freq and pixel-rate controls. This is necessary for
> >> the sensor to work with the ipu3-cio2 driver and for libcamera.
> >>
> >> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> >> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> >> ---
> >>   drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
> >>   1 file changed, 23 insertions(+)
> >>
> >> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> >> index 8bc542df1890..95d3152ddd22 100644
> >> --- a/drivers/media/i2c/ov2680.c
> >> +++ b/drivers/media/i2c/ov2680.c
> >> @@ -75,6 +75,12 @@
> >>   #define OV2680_MIN_CROP_WIDTH            2
> >>   #define OV2680_MIN_CROP_HEIGHT            2
> >>   +/* Fixed pre-div of 1/2 */
> >> +#define OV2680_PLL_PREDIV0            2
> >> +
> >> +/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
> >> +#define OV2680_PLL_PREDIV            2
> >> +
> >>   /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
> >>   #define OV2680_PIXELS_PER_LINE            1704
> >>   #define OV2680_LINES_PER_FRAME            1294
> >> @@ -118,6 +124,8 @@ struct ov2680_ctrls {
> >>       struct v4l2_ctrl *hflip;
> >>       struct v4l2_ctrl *vflip;
> >>       struct v4l2_ctrl *test_pattern;
> >> +    struct v4l2_ctrl *link_freq;
> >> +    struct v4l2_ctrl *pixel_rate;
> >>   };
> >>     struct ov2680_mode {
> >> @@ -145,6 +153,8 @@ struct ov2680_dev {
> >>       struct clk            *xvclk;
> >>       u32                xvclk_freq;
> >>       u8                pll_mult;
> >> +    s64                link_freq[1];
> >> +    s64                pixel_rate;
> >>       struct regulator_bulk_data    supplies[OV2680_NUM_SUPPLIES];
> >>         struct gpio_desc        *pwdn_gpio;
> >> @@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
> >>       ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
> >>                       0, 1023, 1, 250);
> >>   +    ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
> >> +                          0, 0, sensor->link_freq);
> >> +    ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
> >> +                          0, sensor->pixel_rate,
> >> +                          1, sensor->pixel_rate);
> >> +
> >>       if (hdl->error) {
> >>           ret = hdl->error;
> >>           goto cleanup_entity;
> >> @@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
> >>         ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
> >>       ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
> >> +    ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> >>         sensor->sd.ctrl_handler = hdl;
> >>   @@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
> >>         sensor->pll_mult = ov2680_pll_multipliers[i];
> >>   +    sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
> >> +                   OV2680_PLL_PREDIV * sensor->pll_mult;
> >> +
> >> +    /* CSI-2 is double data rate, bus-format is 10 bpp */
> >> +    sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;
> >
> >
> > I'm a little unsure on this one. My understanding is that the link frequency really ought to come from the endpoint properties (which in our case would be set by the ipu-bridge; though it doesn't for this sensor at the moment because I didn't understand it properly back then) because it's a platform specific thing. What the value should be, I have been determining by reading the PLL settings for the sensor whilst the laptop's running Windows. So whilst this is probably technically fine in supporting the link frequency that the driver already expects to configure for whatever platform this was originally designed for, my guess would be that the Miix expects a different link frequency and ideally we'd support that instead. For example see these commits for the ov7251:
>
> The datasheet is clear that the ov2680 is intended to be used with
> a fixed pixelclock of 66 MHz:
>
> "2.2 architecture
>
> The OV2680 sensor core generates streaming pixel data at a constant
> frame rate to a pixel clock of 66 MHz."
>
> and the ov2680 always operates in single lane mode. So there really is
> not much to configure here.
>
> Also the datasheet only contains a rudimentary description of the PLL,
> which is not really enough to write a function to configure the PLL for
> arbitrary link-frequencies.

Working from an OV2680 datasheet I've found on the internet [1] I'll
agree with you that the pixel clock is intended only ever to be 66MHz.

With the FIFO at the front of the "image output interface" in fig 2-1,
and the dividers shown in fig 2-8, the link frequencies could differ
from one fixed ratio from the pixel clock.
Not being able to produce fully arbitrary link frequencies is totally
acceptable in a driver, so there's no obligation to understand or
control those dividers.

> The adjustment to make the sensor work with a 19.2MHz xvclk instead
> of the default 24 MHz comes from the atomisp code. How to make other
> adjustments would pretty much be guess work.
>
> I guess we could add code to check the link-frequencies and check
> that there is only 1 and it matches the expected 330 MHz then the driver
> still honors the link-frequencies property while at the same time
> sticking with the fixed setup the sensor is intended to be used with.
>
> Would adding a link-frequency check like that work for you ?

This is the approach taken in other drivers - see imx214 and imx219
for 2 examples (I've not gone looking for more).

> And if yes what should the link-frequency control return then,
> the actual achieved frequency (this would be better IMHO) or
> the one from the property ?

You've checked that the link-frequency matches the expected 330MHz, so
isn't the actual achieved frequency the same as the one configured?
I'm confused by your statement.

I'm assuming that it's the ipu3-cio that needs link-frequency, as
AFAIK libcamera doesn't.

  Dave

[1] http://www.datasheet.es/PDF/1045403/OV2680-pdf.html

> Regards,
>
> Hans
>
>

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Hans de Goede]

Hi Dave,

On 7/4/23 13:23, Dave Stevenson wrote:
> Hi Hans and Dan
> 
> On Tue, 4 Jul 2023 at 10:45, Hans de Goede <hdegoede@redhat.com> wrote:
>>
>> Hi Dan,
>>
>> On 7/4/23 11:22, Dan Scally wrote:
>>> Morning Hans
>>>
>>> On 27/06/2023 15:18, Hans de Goede wrote:
>>>> Add read-only link-freq and pixel-rate controls. This is necessary for
>>>> the sensor to work with the ipu3-cio2 driver and for libcamera.
>>>>
>>>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>>>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>>>> ---
>>>>   drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
>>>>   1 file changed, 23 insertions(+)
>>>>
>>>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>>>> index 8bc542df1890..95d3152ddd22 100644
>>>> --- a/drivers/media/i2c/ov2680.c
>>>> +++ b/drivers/media/i2c/ov2680.c
>>>> @@ -75,6 +75,12 @@
>>>>   #define OV2680_MIN_CROP_WIDTH            2
>>>>   #define OV2680_MIN_CROP_HEIGHT            2
>>>>   +/* Fixed pre-div of 1/2 */
>>>> +#define OV2680_PLL_PREDIV0            2
>>>> +
>>>> +/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
>>>> +#define OV2680_PLL_PREDIV            2
>>>> +
>>>>   /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
>>>>   #define OV2680_PIXELS_PER_LINE            1704
>>>>   #define OV2680_LINES_PER_FRAME            1294
>>>> @@ -118,6 +124,8 @@ struct ov2680_ctrls {
>>>>       struct v4l2_ctrl *hflip;
>>>>       struct v4l2_ctrl *vflip;
>>>>       struct v4l2_ctrl *test_pattern;
>>>> +    struct v4l2_ctrl *link_freq;
>>>> +    struct v4l2_ctrl *pixel_rate;
>>>>   };
>>>>     struct ov2680_mode {
>>>> @@ -145,6 +153,8 @@ struct ov2680_dev {
>>>>       struct clk            *xvclk;
>>>>       u32                xvclk_freq;
>>>>       u8                pll_mult;
>>>> +    s64                link_freq[1];
>>>> +    s64                pixel_rate;
>>>>       struct regulator_bulk_data    supplies[OV2680_NUM_SUPPLIES];
>>>>         struct gpio_desc        *pwdn_gpio;
>>>> @@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>>>       ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
>>>>                       0, 1023, 1, 250);
>>>>   +    ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
>>>> +                          0, 0, sensor->link_freq);
>>>> +    ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
>>>> +                          0, sensor->pixel_rate,
>>>> +                          1, sensor->pixel_rate);
>>>> +
>>>>       if (hdl->error) {
>>>>           ret = hdl->error;
>>>>           goto cleanup_entity;
>>>> @@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>>>         ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>>>>       ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>>>> +    ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>>>>         sensor->sd.ctrl_handler = hdl;
>>>>   @@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>>>         sensor->pll_mult = ov2680_pll_multipliers[i];
>>>>   +    sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
>>>> +                   OV2680_PLL_PREDIV * sensor->pll_mult;
>>>> +
>>>> +    /* CSI-2 is double data rate, bus-format is 10 bpp */
>>>> +    sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;
>>>
>>>
>>> I'm a little unsure on this one. My understanding is that the link frequency really ought to come from the endpoint properties (which in our case would be set by the ipu-bridge; though it doesn't for this sensor at the moment because I didn't understand it properly back then) because it's a platform specific thing. What the value should be, I have been determining by reading the PLL settings for the sensor whilst the laptop's running Windows. So whilst this is probably technically fine in supporting the link frequency that the driver already expects to configure for whatever platform this was originally designed for, my guess would be that the Miix expects a different link frequency and ideally we'd support that instead. For example see these commits for the ov7251:
>>
>> The datasheet is clear that the ov2680 is intended to be used with
>> a fixed pixelclock of 66 MHz:
>>
>> "2.2 architecture
>>
>> The OV2680 sensor core generates streaming pixel data at a constant
>> frame rate to a pixel clock of 66 MHz."
>>
>> and the ov2680 always operates in single lane mode. So there really is
>> not much to configure here.
>>
>> Also the datasheet only contains a rudimentary description of the PLL,
>> which is not really enough to write a function to configure the PLL for
>> arbitrary link-frequencies.
> 
> Working from an OV2680 datasheet I've found on the internet [1] I'll
> agree with you that the pixel clock is intended only ever to be 66MHz.
> 
> With the FIFO at the front of the "image output interface" in fig 2-1,
> and the dividers shown in fig 2-8, the link frequencies could differ
> from one fixed ratio from the pixel clock.
> Not being able to produce fully arbitrary link frequencies is totally
> acceptable in a driver, so there's no obligation to understand or
> control those dividers.
> 
>> The adjustment to make the sensor work with a 19.2MHz xvclk instead
>> of the default 24 MHz comes from the atomisp code. How to make other
>> adjustments would pretty much be guess work.
>>
>> I guess we could add code to check the link-frequencies and check
>> that there is only 1 and it matches the expected 330 MHz then the driver
>> still honors the link-frequencies property while at the same time
>> sticking with the fixed setup the sensor is intended to be used with.
>>
>> Would adding a link-frequency check like that work for you ?
> 
> This is the approach taken in other drivers - see imx214 and imx219
> for 2 examples (I've not gone looking for more).
> 
>> And if yes what should the link-frequency control return then,
>> the actual achieved frequency (this would be better IMHO) or
>> the one from the property ?
> 
> You've checked that the link-frequency matches the expected 330MHz, so
> isn't the actual achieved frequency the same as the one configured?
> I'm confused by your statement.

On x86 devices the ov2680 is used with a 19.2 MHz clock from
the PMC rather then with a 24 MHz clk.

This results in a link-frequency of: 331.2 MHz instead of 330
MHz (and a matching pixel-clk of 66.24 MHz instead of 66 MHz).

So my question is do we make the link-frequency control
return 331.2 MHz or 330 MHz when a 19.2 MHz xvclk is used ?

330 MHz is what will be requested in the link-frequency
device property. 331.2 MHz will be what is actually used.

(the 24MHz xvclk scenario results in an exact link-freq of
330MHz)

Regards,

Hans




> 
> I'm assuming that it's the ipu3-cio that needs link-frequency, as
> AFAIK libcamera doesn't.
> 
>   Dave
> 
> [1] http://www.datasheet.es/PDF/1045403/OV2680-pdf.html
> 
>> Regards,
>>
>> Hans
>>
>>
> 

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Hans de Goede]

p.s.

On 7/4/23 13:29, Hans de Goede wrote:
> Hi Dave,
> 
> On 7/4/23 13:23, Dave Stevenson wrote:
>> Hi Hans and Dan
>>
>> On Tue, 4 Jul 2023 at 10:45, Hans de Goede <hdegoede@redhat.com> wrote:
>>>
>>> Hi Dan,
>>>
>>> On 7/4/23 11:22, Dan Scally wrote:
>>>> Morning Hans
>>>>
>>>> On 27/06/2023 15:18, Hans de Goede wrote:
>>>>> Add read-only link-freq and pixel-rate controls. This is necessary for
>>>>> the sensor to work with the ipu3-cio2 driver and for libcamera.
>>>>>
>>>>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>>>>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>>>>> ---
>>>>>   drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
>>>>>   1 file changed, 23 insertions(+)
>>>>>
>>>>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>>>>> index 8bc542df1890..95d3152ddd22 100644
>>>>> --- a/drivers/media/i2c/ov2680.c
>>>>> +++ b/drivers/media/i2c/ov2680.c
>>>>> @@ -75,6 +75,12 @@
>>>>>   #define OV2680_MIN_CROP_WIDTH            2
>>>>>   #define OV2680_MIN_CROP_HEIGHT            2
>>>>>   +/* Fixed pre-div of 1/2 */
>>>>> +#define OV2680_PLL_PREDIV0            2
>>>>> +
>>>>> +/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
>>>>> +#define OV2680_PLL_PREDIV            2
>>>>> +
>>>>>   /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
>>>>>   #define OV2680_PIXELS_PER_LINE            1704
>>>>>   #define OV2680_LINES_PER_FRAME            1294
>>>>> @@ -118,6 +124,8 @@ struct ov2680_ctrls {
>>>>>       struct v4l2_ctrl *hflip;
>>>>>       struct v4l2_ctrl *vflip;
>>>>>       struct v4l2_ctrl *test_pattern;
>>>>> +    struct v4l2_ctrl *link_freq;
>>>>> +    struct v4l2_ctrl *pixel_rate;
>>>>>   };
>>>>>     struct ov2680_mode {
>>>>> @@ -145,6 +153,8 @@ struct ov2680_dev {
>>>>>       struct clk            *xvclk;
>>>>>       u32                xvclk_freq;
>>>>>       u8                pll_mult;
>>>>> +    s64                link_freq[1];
>>>>> +    s64                pixel_rate;
>>>>>       struct regulator_bulk_data    supplies[OV2680_NUM_SUPPLIES];
>>>>>         struct gpio_desc        *pwdn_gpio;
>>>>> @@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>>>>       ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
>>>>>                       0, 1023, 1, 250);
>>>>>   +    ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
>>>>> +                          0, 0, sensor->link_freq);
>>>>> +    ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
>>>>> +                          0, sensor->pixel_rate,
>>>>> +                          1, sensor->pixel_rate);
>>>>> +
>>>>>       if (hdl->error) {
>>>>>           ret = hdl->error;
>>>>>           goto cleanup_entity;
>>>>> @@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>>>>         ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>>>>>       ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>>>>> +    ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>>>>>         sensor->sd.ctrl_handler = hdl;
>>>>>   @@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>>>>         sensor->pll_mult = ov2680_pll_multipliers[i];
>>>>>   +    sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
>>>>> +                   OV2680_PLL_PREDIV * sensor->pll_mult;
>>>>> +
>>>>> +    /* CSI-2 is double data rate, bus-format is 10 bpp */
>>>>> +    sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;
>>>>
>>>>
>>>> I'm a little unsure on this one. My understanding is that the link frequency really ought to come from the endpoint properties (which in our case would be set by the ipu-bridge; though it doesn't for this sensor at the moment because I didn't understand it properly back then) because it's a platform specific thing. What the value should be, I have been determining by reading the PLL settings for the sensor whilst the laptop's running Windows. So whilst this is probably technically fine in supporting the link frequency that the driver already expects to configure for whatever platform this was originally designed for, my guess would be that the Miix expects a different link frequency and ideally we'd support that instead. For example see these commits for the ov7251:
>>>
>>> The datasheet is clear that the ov2680 is intended to be used with
>>> a fixed pixelclock of 66 MHz:
>>>
>>> "2.2 architecture
>>>
>>> The OV2680 sensor core generates streaming pixel data at a constant
>>> frame rate to a pixel clock of 66 MHz."
>>>
>>> and the ov2680 always operates in single lane mode. So there really is
>>> not much to configure here.
>>>
>>> Also the datasheet only contains a rudimentary description of the PLL,
>>> which is not really enough to write a function to configure the PLL for
>>> arbitrary link-frequencies.
>>
>> Working from an OV2680 datasheet I've found on the internet [1] I'll
>> agree with you that the pixel clock is intended only ever to be 66MHz.
>>
>> With the FIFO at the front of the "image output interface" in fig 2-1,
>> and the dividers shown in fig 2-8, the link frequencies could differ
>> from one fixed ratio from the pixel clock.
>> Not being able to produce fully arbitrary link frequencies is totally
>> acceptable in a driver, so there's no obligation to understand or
>> control those dividers.
>>
>>> The adjustment to make the sensor work with a 19.2MHz xvclk instead
>>> of the default 24 MHz comes from the atomisp code. How to make other
>>> adjustments would pretty much be guess work.
>>>
>>> I guess we could add code to check the link-frequencies and check
>>> that there is only 1 and it matches the expected 330 MHz then the driver
>>> still honors the link-frequencies property while at the same time
>>> sticking with the fixed setup the sensor is intended to be used with.
>>>
>>> Would adding a link-frequency check like that work for you ?
>>
>> This is the approach taken in other drivers - see imx214 and imx219
>> for 2 examples (I've not gone looking for more).
>>
>>> And if yes what should the link-frequency control return then,
>>> the actual achieved frequency (this would be better IMHO) or
>>> the one from the property ?
>>
>> You've checked that the link-frequency matches the expected 330MHz, so
>> isn't the actual achieved frequency the same as the one configured?
>> I'm confused by your statement.
> 
> On x86 devices the ov2680 is used with a 19.2 MHz clock from
> the PMC rather then with a 24 MHz clk.
> 
> This results in a link-frequency of: 331.2 MHz instead of 330
> MHz (and a matching pixel-clk of 66.24 MHz instead of 66 MHz).
> 
> So my question is do we make the link-frequency control
> return 331.2 MHz or 330 MHz when a 19.2 MHz xvclk is used ?
> 
> 330 MHz is what will be requested in the link-frequency
> device property. 331.2 MHz will be what is actually used.
> 
> (the 24MHz xvclk scenario results in an exact link-freq of
> 330MHz)

I just realized that since both the xvclk frequency and the
link-frequency property will both come from the same ACPI
glue we could fix this be actually putting  331.2 MHz
in the property and making the driver accept this as valid
when a 19.2 MHz xvclk is used.

That would make the property and the actually achieved
link-frequency be in sync again. So this is probably the
best way to handle this.

Does this solution work for everyone ?

Regards,
 
Hans





>> I'm assuming that it's the ipu3-cio that needs link-frequency, as
>> AFAIK libcamera doesn't.
>>
>>   Dave
>>
>> [1] http://www.datasheet.es/PDF/1045403/OV2680-pdf.html
>>
>>> Regards,
>>>
>>> Hans
>>>
>>>
>>
> 

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Dan Scally]

Hello

On 04/07/2023 13:32, Hans de Goede wrote:
> p.s.
>
> On 7/4/23 13:29, Hans de Goede wrote:
>> Hi Dave,
>>
>> On 7/4/23 13:23, Dave Stevenson wrote:
>>> Hi Hans and Dan
>>>
>>> On Tue, 4 Jul 2023 at 10:45, Hans de Goede <hdegoede@redhat.com> wrote:
>>>> Hi Dan,
>>>>
>>>> On 7/4/23 11:22, Dan Scally wrote:
>>>>> Morning Hans
>>>>>
>>>>> On 27/06/2023 15:18, Hans de Goede wrote:
>>>>>> Add read-only link-freq and pixel-rate controls. This is necessary for
>>>>>> the sensor to work with the ipu3-cio2 driver and for libcamera.
>>>>>>
>>>>>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
>>>>>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
>>>>>> ---
>>>>>>    drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
>>>>>>    1 file changed, 23 insertions(+)
>>>>>>
>>>>>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
>>>>>> index 8bc542df1890..95d3152ddd22 100644
>>>>>> --- a/drivers/media/i2c/ov2680.c
>>>>>> +++ b/drivers/media/i2c/ov2680.c
>>>>>> @@ -75,6 +75,12 @@
>>>>>>    #define OV2680_MIN_CROP_WIDTH            2
>>>>>>    #define OV2680_MIN_CROP_HEIGHT            2
>>>>>>    +/* Fixed pre-div of 1/2 */
>>>>>> +#define OV2680_PLL_PREDIV0            2
>>>>>> +
>>>>>> +/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
>>>>>> +#define OV2680_PLL_PREDIV            2
>>>>>> +
>>>>>>    /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
>>>>>>    #define OV2680_PIXELS_PER_LINE            1704
>>>>>>    #define OV2680_LINES_PER_FRAME            1294
>>>>>> @@ -118,6 +124,8 @@ struct ov2680_ctrls {
>>>>>>        struct v4l2_ctrl *hflip;
>>>>>>        struct v4l2_ctrl *vflip;
>>>>>>        struct v4l2_ctrl *test_pattern;
>>>>>> +    struct v4l2_ctrl *link_freq;
>>>>>> +    struct v4l2_ctrl *pixel_rate;
>>>>>>    };
>>>>>>      struct ov2680_mode {
>>>>>> @@ -145,6 +153,8 @@ struct ov2680_dev {
>>>>>>        struct clk            *xvclk;
>>>>>>        u32                xvclk_freq;
>>>>>>        u8                pll_mult;
>>>>>> +    s64                link_freq[1];
>>>>>> +    s64                pixel_rate;
>>>>>>        struct regulator_bulk_data    supplies[OV2680_NUM_SUPPLIES];
>>>>>>          struct gpio_desc        *pwdn_gpio;
>>>>>> @@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>>>>>        ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
>>>>>>                        0, 1023, 1, 250);
>>>>>>    +    ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
>>>>>> +                          0, 0, sensor->link_freq);
>>>>>> +    ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
>>>>>> +                          0, sensor->pixel_rate,
>>>>>> +                          1, sensor->pixel_rate);
>>>>>> +
>>>>>>        if (hdl->error) {
>>>>>>            ret = hdl->error;
>>>>>>            goto cleanup_entity;
>>>>>> @@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
>>>>>>          ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>>>>>>        ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
>>>>>> +    ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
>>>>>>          sensor->sd.ctrl_handler = hdl;
>>>>>>    @@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
>>>>>>          sensor->pll_mult = ov2680_pll_multipliers[i];
>>>>>>    +    sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
>>>>>> +                   OV2680_PLL_PREDIV * sensor->pll_mult;
>>>>>> +
>>>>>> +    /* CSI-2 is double data rate, bus-format is 10 bpp */
>>>>>> +    sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;
>>>>>
>>>>> I'm a little unsure on this one. My understanding is that the link frequency really ought to come from the endpoint properties (which in our case would be set by the ipu-bridge; though it doesn't for this sensor at the moment because I didn't understand it properly back then) because it's a platform specific thing. What the value should be, I have been determining by reading the PLL settings for the sensor whilst the laptop's running Windows. So whilst this is probably technically fine in supporting the link frequency that the driver already expects to configure for whatever platform this was originally designed for, my guess would be that the Miix expects a different link frequency and ideally we'd support that instead. For example see these commits for the ov7251:
>>>> The datasheet is clear that the ov2680 is intended to be used with
>>>> a fixed pixelclock of 66 MHz:
>>>>
>>>> "2.2 architecture
>>>>
>>>> The OV2680 sensor core generates streaming pixel data at a constant
>>>> frame rate to a pixel clock of 66 MHz."
>>>>
>>>> and the ov2680 always operates in single lane mode. So there really is
>>>> not much to configure here.
>>>>
>>>> Also the datasheet only contains a rudimentary description of the PLL,
>>>> which is not really enough to write a function to configure the PLL for
>>>> arbitrary link-frequencies.
>>> Working from an OV2680 datasheet I've found on the internet [1] I'll
>>> agree with you that the pixel clock is intended only ever to be 66MHz.
>>>
>>> With the FIFO at the front of the "image output interface" in fig 2-1,
>>> and the dividers shown in fig 2-8, the link frequencies could differ
>>> from one fixed ratio from the pixel clock.
>>> Not being able to produce fully arbitrary link frequencies is totally
>>> acceptable in a driver, so there's no obligation to understand or
>>> control those dividers.
>>>
>>>> The adjustment to make the sensor work with a 19.2MHz xvclk instead
>>>> of the default 24 MHz comes from the atomisp code. How to make other
>>>> adjustments would pretty much be guess work.
>>>>
>>>> I guess we could add code to check the link-frequencies and check
>>>> that there is only 1 and it matches the expected 330 MHz then the driver
>>>> still honors the link-frequencies property while at the same time
>>>> sticking with the fixed setup the sensor is intended to be used with.
>>>>
>>>> Would adding a link-frequency check like that work for you ?
>>> This is the approach taken in other drivers - see imx214 and imx219
>>> for 2 examples (I've not gone looking for more).
>>>
>>>> And if yes what should the link-frequency control return then,
>>>> the actual achieved frequency (this would be better IMHO) or
>>>> the one from the property ?
>>> You've checked that the link-frequency matches the expected 330MHz, so
>>> isn't the actual achieved frequency the same as the one configured?
>>> I'm confused by your statement.
>> On x86 devices the ov2680 is used with a 19.2 MHz clock from
>> the PMC rather then with a 24 MHz clk.
>>
>> This results in a link-frequency of: 331.2 MHz instead of 330
>> MHz (and a matching pixel-clk of 66.24 MHz instead of 66 MHz).
>>
>> So my question is do we make the link-frequency control
>> return 331.2 MHz or 330 MHz when a 19.2 MHz xvclk is used ?
>>
>> 330 MHz is what will be requested in the link-frequency
>> device property. 331.2 MHz will be what is actually used.
>>
>> (the 24MHz xvclk scenario results in an exact link-freq of
>> 330MHz)
> I just realized that since both the xvclk frequency and the
> link-frequency property will both come from the same ACPI
> glue we could fix this be actually putting  331.2 MHz
> in the property and making the driver accept this as valid
> when a 19.2 MHz xvclk is used.
>
> That would make the property and the actually achieved
> link-frequency be in sync again. So this is probably the
> best way to handle this.
>
> Does this solution work for everyone ?

Yep that sounds good to me then.
>
> Regards,
>   
> Hans
>
>
>
>
>
>>> I'm assuming that it's the ipu3-cio that needs link-frequency, as
>>> AFAIK libcamera doesn't.
>>>
>>>    Dave
>>>
>>> [1] http://www.datasheet.es/PDF/1045403/OV2680-pdf.html
>>>
>>>> Regards,
>>>>
>>>> Hans
>>>>
>>>>

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Dave Stevenson]

On Tue, 4 Jul 2023 at 12:32, Hans de Goede <hdegoede@redhat.com> wrote:
>
> p.s.
>
> On 7/4/23 13:29, Hans de Goede wrote:
> > Hi Dave,
> >
> > On 7/4/23 13:23, Dave Stevenson wrote:
> >> Hi Hans and Dan
> >>
> >> On Tue, 4 Jul 2023 at 10:45, Hans de Goede <hdegoede@redhat.com> wrote:
> >>>
> >>> Hi Dan,
> >>>
> >>> On 7/4/23 11:22, Dan Scally wrote:
> >>>> Morning Hans
> >>>>
> >>>> On 27/06/2023 15:18, Hans de Goede wrote:
> >>>>> Add read-only link-freq and pixel-rate controls. This is necessary for
> >>>>> the sensor to work with the ipu3-cio2 driver and for libcamera.
> >>>>>
> >>>>> Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
> >>>>> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> >>>>> ---
> >>>>>   drivers/media/i2c/ov2680.c | 23 +++++++++++++++++++++++
> >>>>>   1 file changed, 23 insertions(+)
> >>>>>
> >>>>> diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
> >>>>> index 8bc542df1890..95d3152ddd22 100644
> >>>>> --- a/drivers/media/i2c/ov2680.c
> >>>>> +++ b/drivers/media/i2c/ov2680.c
> >>>>> @@ -75,6 +75,12 @@
> >>>>>   #define OV2680_MIN_CROP_WIDTH            2
> >>>>>   #define OV2680_MIN_CROP_HEIGHT            2
> >>>>>   +/* Fixed pre-div of 1/2 */
> >>>>> +#define OV2680_PLL_PREDIV0            2
> >>>>> +
> >>>>> +/* Pre-div configurable through reg 0x3080, left at its default of 0x02 : 1/2 */
> >>>>> +#define OV2680_PLL_PREDIV            2
> >>>>> +
> >>>>>   /* 66MHz pixel clock: 66MHz / 1704 * 1294 = 30fps */
> >>>>>   #define OV2680_PIXELS_PER_LINE            1704
> >>>>>   #define OV2680_LINES_PER_FRAME            1294
> >>>>> @@ -118,6 +124,8 @@ struct ov2680_ctrls {
> >>>>>       struct v4l2_ctrl *hflip;
> >>>>>       struct v4l2_ctrl *vflip;
> >>>>>       struct v4l2_ctrl *test_pattern;
> >>>>> +    struct v4l2_ctrl *link_freq;
> >>>>> +    struct v4l2_ctrl *pixel_rate;
> >>>>>   };
> >>>>>     struct ov2680_mode {
> >>>>> @@ -145,6 +153,8 @@ struct ov2680_dev {
> >>>>>       struct clk            *xvclk;
> >>>>>       u32                xvclk_freq;
> >>>>>       u8                pll_mult;
> >>>>> +    s64                link_freq[1];
> >>>>> +    s64                pixel_rate;
> >>>>>       struct regulator_bulk_data    supplies[OV2680_NUM_SUPPLIES];
> >>>>>         struct gpio_desc        *pwdn_gpio;
> >>>>> @@ -906,6 +916,12 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
> >>>>>       ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
> >>>>>                       0, 1023, 1, 250);
> >>>>>   +    ctrls->link_freq = v4l2_ctrl_new_int_menu(hdl, NULL, V4L2_CID_LINK_FREQ,
> >>>>> +                          0, 0, sensor->link_freq);
> >>>>> +    ctrls->pixel_rate = v4l2_ctrl_new_std(hdl, NULL, V4L2_CID_PIXEL_RATE,
> >>>>> +                          0, sensor->pixel_rate,
> >>>>> +                          1, sensor->pixel_rate);
> >>>>> +
> >>>>>       if (hdl->error) {
> >>>>>           ret = hdl->error;
> >>>>>           goto cleanup_entity;
> >>>>> @@ -913,6 +929,7 @@ static int ov2680_v4l2_register(struct ov2680_dev *sensor)
> >>>>>         ctrls->vflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
> >>>>>       ctrls->hflip->flags |= V4L2_CTRL_FLAG_MODIFY_LAYOUT;
> >>>>> +    ctrls->link_freq->flags |= V4L2_CTRL_FLAG_READ_ONLY;
> >>>>>         sensor->sd.ctrl_handler = hdl;
> >>>>>   @@ -1030,6 +1047,12 @@ static int ov2680_parse_dt(struct ov2680_dev *sensor)
> >>>>>         sensor->pll_mult = ov2680_pll_multipliers[i];
> >>>>>   +    sensor->link_freq[0] = sensor->xvclk_freq / OV2680_PLL_PREDIV0 /
> >>>>> +                   OV2680_PLL_PREDIV * sensor->pll_mult;
> >>>>> +
> >>>>> +    /* CSI-2 is double data rate, bus-format is 10 bpp */
> >>>>> +    sensor->pixel_rate = sensor->link_freq[0] * 2 / 10;
> >>>>
> >>>>
> >>>> I'm a little unsure on this one. My understanding is that the link frequency really ought to come from the endpoint properties (which in our case would be set by the ipu-bridge; though it doesn't for this sensor at the moment because I didn't understand it properly back then) because it's a platform specific thing. What the value should be, I have been determining by reading the PLL settings for the sensor whilst the laptop's running Windows. So whilst this is probably technically fine in supporting the link frequency that the driver already expects to configure for whatever platform this was originally designed for, my guess would be that the Miix expects a different link frequency and ideally we'd support that instead. For example see these commits for the ov7251:
> >>>
> >>> The datasheet is clear that the ov2680 is intended to be used with
> >>> a fixed pixelclock of 66 MHz:
> >>>
> >>> "2.2 architecture
> >>>
> >>> The OV2680 sensor core generates streaming pixel data at a constant
> >>> frame rate to a pixel clock of 66 MHz."
> >>>
> >>> and the ov2680 always operates in single lane mode. So there really is
> >>> not much to configure here.
> >>>
> >>> Also the datasheet only contains a rudimentary description of the PLL,
> >>> which is not really enough to write a function to configure the PLL for
> >>> arbitrary link-frequencies.
> >>
> >> Working from an OV2680 datasheet I've found on the internet [1] I'll
> >> agree with you that the pixel clock is intended only ever to be 66MHz.
> >>
> >> With the FIFO at the front of the "image output interface" in fig 2-1,
> >> and the dividers shown in fig 2-8, the link frequencies could differ
> >> from one fixed ratio from the pixel clock.
> >> Not being able to produce fully arbitrary link frequencies is totally
> >> acceptable in a driver, so there's no obligation to understand or
> >> control those dividers.
> >>
> >>> The adjustment to make the sensor work with a 19.2MHz xvclk instead
> >>> of the default 24 MHz comes from the atomisp code. How to make other
> >>> adjustments would pretty much be guess work.
> >>>
> >>> I guess we could add code to check the link-frequencies and check
> >>> that there is only 1 and it matches the expected 330 MHz then the driver
> >>> still honors the link-frequencies property while at the same time
> >>> sticking with the fixed setup the sensor is intended to be used with.
> >>>
> >>> Would adding a link-frequency check like that work for you ?
> >>
> >> This is the approach taken in other drivers - see imx214 and imx219
> >> for 2 examples (I've not gone looking for more).
> >>
> >>> And if yes what should the link-frequency control return then,
> >>> the actual achieved frequency (this would be better IMHO) or
> >>> the one from the property ?
> >>
> >> You've checked that the link-frequency matches the expected 330MHz, so
> >> isn't the actual achieved frequency the same as the one configured?
> >> I'm confused by your statement.
> >
> > On x86 devices the ov2680 is used with a 19.2 MHz clock from
> > the PMC rather then with a 24 MHz clk.
> >
> > This results in a link-frequency of: 331.2 MHz instead of 330
> > MHz (and a matching pixel-clk of 66.24 MHz instead of 66 MHz).
> >
> > So my question is do we make the link-frequency control
> > return 331.2 MHz or 330 MHz when a 19.2 MHz xvclk is used ?
> >
> > 330 MHz is what will be requested in the link-frequency
> > device property. 331.2 MHz will be what is actually used.
> >
> > (the 24MHz xvclk scenario results in an exact link-freq of
> > 330MHz)

OK, that wasn't clear from the earlier discussion.

> I just realized that since both the xvclk frequency and the
> link-frequency property will both come from the same ACPI
> glue we could fix this be actually putting  331.2 MHz
> in the property and making the driver accept this as valid
> when a 19.2 MHz xvclk is used.
>
> That would make the property and the actually achieved
> link-frequency be in sync again. So this is probably the
> best way to handle this.
>
> Does this solution work for everyone ?

Yes, asking for the right thing sounds like a good solution.

It's the same route as I went for with imx258 in [1]. There 24MHz
results in a slightly different link freq to the existing 19.2MHz
configuration.

  Dave

[1] https://patchwork.linuxtv.org/project/linux-media/patch/20230530173000.3060865-9-dave.stevenson@raspberrypi.com/

> Regards,
>
> Hans
>
>
>
>
>
> >> I'm assuming that it's the ipu3-cio that needs link-frequency, as
> >> AFAIK libcamera doesn't.
> >>
> >>   Dave
> >>
> >> [1] http://www.datasheet.es/PDF/1045403/OV2680-pdf.html
> >>
> >>> Regards,
> >>>
> >>> Hans
> >>>
> >>>
> >>
> >
>

Re: [PATCH v3 28/29] media: ov2680: Add link-freq and pixel-rate controls

[Andy Shevchenko]

On Tue, Jul 04, 2023 at 01:19:28PM +0100, Dave Stevenson wrote:
> On Tue, 4 Jul 2023 at 12:32, Hans de Goede <hdegoede@redhat.com> wrote:
> > On 7/4/23 13:29, Hans de Goede wrote:
> > > On 7/4/23 13:23, Dave Stevenson wrote:

...

> > > So my question is do we make the link-frequency control
> > > return 331.2 MHz or 330 MHz when a 19.2 MHz xvclk is used ?
> > >
> > > 330 MHz is what will be requested in the link-frequency
> > > device property. 331.2 MHz will be what is actually used.
> > >
> > > (the 24MHz xvclk scenario results in an exact link-freq of
> > > 330MHz)
> 
> OK, that wasn't clear from the earlier discussion.
> 
> > I just realized that since both the xvclk frequency and the
> > link-frequency property will both come from the same ACPI
> > glue we could fix this be actually putting  331.2 MHz
> > in the property and making the driver accept this as valid
> > when a 19.2 MHz xvclk is used.
> >
> > That would make the property and the actually achieved
> > link-frequency be in sync again. So this is probably the
> > best way to handle this.
> >
> > Does this solution work for everyone ?
> 
> Yes, asking for the right thing sounds like a good solution.

+1 here.

> It's the same route as I went for with imx258 in [1]. There 24MHz
> results in a slightly different link freq to the existing 19.2MHz
> configuration.

-- 
With Best Regards,
Andy Shevchenko

[PATCH v3 29/29] MAINTAINERS: Add Hans de Goede as OV2680 sensor driver maintainer

[Hans de Goede]

Add myself as OV2680 sensor driver maintainer.

Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index c6a7b43321d9..2ec35c3569dd 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -15518,6 +15518,7 @@ F:	drivers/media/i2c/ov13b10.c
 
 OMNIVISION OV2680 SENSOR DRIVER
 M:	Rui Miguel Silva <rmfrfs@gmail.com>
+M:	Hans de Goede <hansg@kernel.org>
 L:	linux-media@vger.kernel.org
 S:	Maintained
 T:	git git://linuxtv.org/media_tree.git
-- 
2.41.0

Re: [PATCH v3 29/29] MAINTAINERS: Add Hans de Goede as OV2680 sensor driver maintainer

[Rui Miguel Silva]

Hi Hans,
Hans de Goede <hdegoede@redhat.com> writes:

> Add myself as OV2680 sensor driver maintainer.

Thanks for this.

>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>

Reviewed-by: Rui Miguel Silva <rmfrfs@gmail.com>

Cheers,
   Rui

> ---
>  MAINTAINERS | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index c6a7b43321d9..2ec35c3569dd 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -15518,6 +15518,7 @@ F:	drivers/media/i2c/ov13b10.c
>  
>  OMNIVISION OV2680 SENSOR DRIVER
>  M:	Rui Miguel Silva <rmfrfs@gmail.com>
> +M:	Hans de Goede <hansg@kernel.org>
>  L:	linux-media@vger.kernel.org
>  S:	Maintained
>  T:	git git://linuxtv.org/media_tree.git
> -- 
> 2.41.0

Re: [PATCH v3 29/29] MAINTAINERS: Add Hans de Goede as OV2680 sensor driver maintainer

[Dan Scally]

Hi Hans

On 27/06/2023 15:18, Hans de Goede wrote:
> Add myself as OV2680 sensor driver maintainer.
>
> Signed-off-by: Hans de Goede <hdegoede@redhat.com>
> ---


This is a really nice series. I had some comments on 15, 18, 21 and 28 but for all of the others:


Reviewed-by: Daniel Scally <dan.scally@ideasonboard.com>

>   MAINTAINERS | 1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index c6a7b43321d9..2ec35c3569dd 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -15518,6 +15518,7 @@ F:	drivers/media/i2c/ov13b10.c
>   
>   OMNIVISION OV2680 SENSOR DRIVER
>   M:	Rui Miguel Silva <rmfrfs@gmail.com>
> +M:	Hans de Goede <hansg@kernel.org>
>   L:	linux-media@vger.kernel.org
>   S:	Maintained
>   T:	git git://linuxtv.org/media_tree.git

Re: [PATCH v3 00/29] media: ov2680: Bugfixes + ACPI + selection(crop-tgt) API support

[Jacopo Mondi]

Hi Hans

  seems like [12/29] is missing, it doesn't show up in patchwork
neither:
https://patchwork.linuxtv.org/project/linux-media/list/?series=10723

have I missed it ?

On Tue, Jun 27, 2023 at 03:18:01PM +0200, Hans de Goede wrote:
> Hi All,
>
> Here is v3 of my ov2680 sensor driver patch series.
>
> Changes in v3:
> - Add Rui Miguel Silva's Ack for the series
> - 2 small fixes for remarks from Andy
> - Add a new patch adding me as co-maintainer in MAINTAINERS
>
> Changes in v2
> - Drop "media: Add MIPI CCI register access helper functions"
>   (being reviewed in its own thread / patch-submission)
> - Drop "media: ov2680: Add g_skip_frames op support"
> - Add "media: ov2680: Fix regulators being left enabled on
>   ov2680_power_on() errors"
> - Add "media: ov2680: Add link-freq and pixel-rate controls"
>   with this the driver now works on IPU3 with ipu3-capture.sh
>   (libcamera support requires adding a couple more controls)
> - Limit line length to 80 chars everywhere
> - Address various small remarks from Andy
>
> During all the work done on the atomisp driver I have mostly been testing
> on devices with an ov2680 sensor. As such I have also done a lot of work
> on the atomisp-ov2680.c atomisp specific sensor driver.
>
> With the latest atomisp code from:
> https://git.kernel.org/pub/scm/linux/kernel/git/hansg/linux.git/tag/?h=media-atomisp-6.5-1
>
> The atomisp code can now work with standard v4l2 sensor drivers using
> the selections (crop-tgt) api and v4l2-async sensor driver registration.
>
> This patch series modifies the main drivers/media/i2c/ov2680.c driver
> to add bugfixes, ACPI enumeration, selection API support and further
> improvments. After this the driver can be used with the atomisp driver
> and atomisp-ov2680.c can be dropped.
>
> This also gets the driver much closer to having everything needed for
> use with IPU3 / libcamera. I have a Lenovo Miix 510 now with an IPU3 +
> ov2680 sensor and with this series raw-capture using the ipu3-capture.sh
> script works. I plan to work on libcamera support for this in the near
> future.
>
> This series consist of 3 parts:
>
> 1. Patches 1-8 are bugfixes these are put first for backporting
>
> 2. Patch 9 converts the ov2680 driver to the new CCI helpers,
> the same has been done in the other series with the atomisp-ov2680
> driver and this makes it much easier to sync things up.
>
> Note this depends on the new CCI register helpers, these are being
> reviewed here:
>
> https://lore.kernel.org/linux-media/20230627125109.52354-1-hdegoede@redhat.com/
>
> 3. Patches 9 - 29 implement the ACPI enumeration,
> selection API support and further improvments.
>
> Regards,
>
> Hans
>
>
>
> Hans de Goede (29):
>   media: ov2680: Remove auto-gain and auto-exposure controls
>   media: ov2680: Fix ov2680_bayer_order()
>   media: ov2680: Fix vflip / hflip set functions
>   media: ov2680: Use select VIDEO_V4L2_SUBDEV_API
>   media: ov2680: Don't take the lock for try_fmt calls
>   media: ov2680: Add ov2680_fill_format() helper function
>   media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY
>     not working
>   media: ov2680: Fix regulators being left enabled on ov2680_power_on()
>     errors
>   media: ov2680: Convert to new CCI register access helpers
>   media: ov2680: Store dev instead of i2c_client in ov2680_dev
>   media: ov2680: Check for "powerdown" GPIO con-id before checking for
>     "reset" GPIO con-id
>   media: ov2680: Add runtime-pm support
>   media: ov2680: Drop is_enabled flag
>   media: ov2680: Add support for more clk setups
>   media: ov2680: Add support for 19.2 MHz clock
>   media: ov2680: Add endpoint matching support
>   media: ov2680: Add support for ACPI enumeration
>   media: ov2680: Fix ov2680_enum_frame_interval()
>   media: ov2680: Annotate the per mode register setting lists
>   media: ov2680: Add ov2680_mode struct
>   media: ov2680: Make setting the mode algorithm based
>   media: ov2680: Add an __ov2680_get_pad_format() helper function
>   media: ov2680: Implement selection support
>   media: ov2680: Fix exposure and gain ctrls range and default value
>   media: ov2680: Add a bunch of register tweaks
>   media: ov2680: Drop unnecessary pad checks
>   media: ov2680: Read and log sensor revision during probe
>   media: ov2680: Add link-freq and pixel-rate controls
>   MAINTAINERS: Add Hans de Goede as OV2680 sensor driver maintainer
>
>  MAINTAINERS                |    1 +
>  drivers/media/i2c/Kconfig  |    2 +
>  drivers/media/i2c/ov2680.c | 1318 +++++++++++++++++++-----------------
>  3 files changed, 691 insertions(+), 630 deletions(-)
>
> --
> 2.41.0
>

Re: [PATCH v3 00/29] media: ov2680: Bugfixes + ACPI + selection(crop-tgt) API support

[Hans de Goede]

Hi Jacopo,

On 6/27/23 16:11, Jacopo Mondi wrote:
> Hi Hans
> 
>   seems like [12/29] is missing, it doesn't show up in patchwork
> neither:
> https://patchwork.linuxtv.org/project/linux-media/list/?series=10723
> 
> have I missed it ?

Nope looks like something ate it. I've just resend it using
"--in-reply-to=20230627131830.54601-1-hdegoede@redhat.com"
so that it should show up in the thread / patch-set.

Regards,

Hans




> 
> On Tue, Jun 27, 2023 at 03:18:01PM +0200, Hans de Goede wrote:
>> Hi All,
>>
>> Here is v3 of my ov2680 sensor driver patch series.
>>
>> Changes in v3:
>> - Add Rui Miguel Silva's Ack for the series
>> - 2 small fixes for remarks from Andy
>> - Add a new patch adding me as co-maintainer in MAINTAINERS
>>
>> Changes in v2
>> - Drop "media: Add MIPI CCI register access helper functions"
>>   (being reviewed in its own thread / patch-submission)
>> - Drop "media: ov2680: Add g_skip_frames op support"
>> - Add "media: ov2680: Fix regulators being left enabled on
>>   ov2680_power_on() errors"
>> - Add "media: ov2680: Add link-freq and pixel-rate controls"
>>   with this the driver now works on IPU3 with ipu3-capture.sh
>>   (libcamera support requires adding a couple more controls)
>> - Limit line length to 80 chars everywhere
>> - Address various small remarks from Andy
>>
>> During all the work done on the atomisp driver I have mostly been testing
>> on devices with an ov2680 sensor. As such I have also done a lot of work
>> on the atomisp-ov2680.c atomisp specific sensor driver.
>>
>> With the latest atomisp code from:
>> https://git.kernel.org/pub/scm/linux/kernel/git/hansg/linux.git/tag/?h=media-atomisp-6.5-1
>>
>> The atomisp code can now work with standard v4l2 sensor drivers using
>> the selections (crop-tgt) api and v4l2-async sensor driver registration.
>>
>> This patch series modifies the main drivers/media/i2c/ov2680.c driver
>> to add bugfixes, ACPI enumeration, selection API support and further
>> improvments. After this the driver can be used with the atomisp driver
>> and atomisp-ov2680.c can be dropped.
>>
>> This also gets the driver much closer to having everything needed for
>> use with IPU3 / libcamera. I have a Lenovo Miix 510 now with an IPU3 +
>> ov2680 sensor and with this series raw-capture using the ipu3-capture.sh
>> script works. I plan to work on libcamera support for this in the near
>> future.
>>
>> This series consist of 3 parts:
>>
>> 1. Patches 1-8 are bugfixes these are put first for backporting
>>
>> 2. Patch 9 converts the ov2680 driver to the new CCI helpers,
>> the same has been done in the other series with the atomisp-ov2680
>> driver and this makes it much easier to sync things up.
>>
>> Note this depends on the new CCI register helpers, these are being
>> reviewed here:
>>
>> https://lore.kernel.org/linux-media/20230627125109.52354-1-hdegoede@redhat.com/
>>
>> 3. Patches 9 - 29 implement the ACPI enumeration,
>> selection API support and further improvments.
>>
>> Regards,
>>
>> Hans
>>
>>
>>
>> Hans de Goede (29):
>>   media: ov2680: Remove auto-gain and auto-exposure controls
>>   media: ov2680: Fix ov2680_bayer_order()
>>   media: ov2680: Fix vflip / hflip set functions
>>   media: ov2680: Use select VIDEO_V4L2_SUBDEV_API
>>   media: ov2680: Don't take the lock for try_fmt calls
>>   media: ov2680: Add ov2680_fill_format() helper function
>>   media: ov2680: Fix ov2680_set_fmt() which == V4L2_SUBDEV_FORMAT_TRY
>>     not working
>>   media: ov2680: Fix regulators being left enabled on ov2680_power_on()
>>     errors
>>   media: ov2680: Convert to new CCI register access helpers
>>   media: ov2680: Store dev instead of i2c_client in ov2680_dev
>>   media: ov2680: Check for "powerdown" GPIO con-id before checking for
>>     "reset" GPIO con-id
>>   media: ov2680: Add runtime-pm support
>>   media: ov2680: Drop is_enabled flag
>>   media: ov2680: Add support for more clk setups
>>   media: ov2680: Add support for 19.2 MHz clock
>>   media: ov2680: Add endpoint matching support
>>   media: ov2680: Add support for ACPI enumeration
>>   media: ov2680: Fix ov2680_enum_frame_interval()
>>   media: ov2680: Annotate the per mode register setting lists
>>   media: ov2680: Add ov2680_mode struct
>>   media: ov2680: Make setting the mode algorithm based
>>   media: ov2680: Add an __ov2680_get_pad_format() helper function
>>   media: ov2680: Implement selection support
>>   media: ov2680: Fix exposure and gain ctrls range and default value
>>   media: ov2680: Add a bunch of register tweaks
>>   media: ov2680: Drop unnecessary pad checks
>>   media: ov2680: Read and log sensor revision during probe
>>   media: ov2680: Add link-freq and pixel-rate controls
>>   MAINTAINERS: Add Hans de Goede as OV2680 sensor driver maintainer
>>
>>  MAINTAINERS                |    1 +
>>  drivers/media/i2c/Kconfig  |    2 +
>>  drivers/media/i2c/ov2680.c | 1318 +++++++++++++++++++-----------------
>>  3 files changed, 691 insertions(+), 630 deletions(-)
>>
>> --
>> 2.41.0
>>
> 

[PATCH v3 12/29] media: ov2680: Add runtime-pm support

[Hans de Goede]

Remove the obsolete s_power() callback and instead use runtime-pm +
autosuspend, powering-on the sensor on s_stream(1) and releasing
the runtime-pm reference on s_stream(0).

This also removes the need for ov2680_mode_restore() instead
ov2680_stream_enable() now takes care of all sensor initalization
after power-on.

This is a preparation patch for adding ACPI support.

Note this also removes putting the clock lane into LP-11 state from
ov2680_power_on() since now streaming will start immediately after
powering on the sensor there is no need to put the clock lane
in a low power state.

Acked-by: Rui Miguel Silva <rmfrfs@gmail.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
---
 drivers/media/i2c/ov2680.c | 134 +++++++++++++++++--------------------
 1 file changed, 61 insertions(+), 73 deletions(-)

diff --git a/drivers/media/i2c/ov2680.c b/drivers/media/i2c/ov2680.c
index 0de047c49c31..56aaf67c1d82 100644
--- a/drivers/media/i2c/ov2680.c
+++ b/drivers/media/i2c/ov2680.c
@@ -18,6 +18,7 @@
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/of_device.h>
+#include <linux/pm_runtime.h>
 #include <linux/regmap.h>
 #include <linux/regulator/consumer.h>
 
@@ -99,7 +100,6 @@ struct ov2680_dev {
 	struct gpio_desc		*pwdn_gpio;
 	struct mutex			lock; /* protect members */
 
-	bool				mode_pending_changes;
 	bool				is_enabled;
 	bool				is_streaming;
 
@@ -282,19 +282,15 @@ static int ov2680_exposure_set(struct ov2680_dev *sensor, u32 exp)
 }
 
 static int ov2680_stream_enable(struct ov2680_dev *sensor)
-{
-	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 1, NULL);
-}
-
-static int ov2680_stream_disable(struct ov2680_dev *sensor)
-{
-	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 0, NULL);
-}
-
-static int ov2680_mode_set(struct ov2680_dev *sensor)
 {
 	int ret;
 
+	ret = regmap_multi_reg_write(sensor->regmap,
+				     ov2680_mode_init_data.reg_data,
+				     ov2680_mode_init_data.reg_data_size);
+	if (ret < 0)
+		return ret;
+
 	ret = regmap_multi_reg_write(sensor->regmap,
 				     sensor->current_mode->reg_data,
 				     sensor->current_mode->reg_data_size);
@@ -306,22 +302,12 @@ static int ov2680_mode_set(struct ov2680_dev *sensor)
 	if (ret < 0)
 		return ret;
 
-	sensor->mode_pending_changes = false;
-
-	return 0;
+	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 1, NULL);
 }
 
-static int ov2680_mode_restore(struct ov2680_dev *sensor)
+static int ov2680_stream_disable(struct ov2680_dev *sensor)
 {
-	int ret;
-
-	ret = regmap_multi_reg_write(sensor->regmap,
-				     ov2680_mode_init_data.reg_data,
-				     ov2680_mode_init_data.reg_data_size);
-	if (ret < 0)
-		return ret;
-
-	return ov2680_mode_set(sensor);
+	return cci_write(sensor->regmap, OV2680_REG_STREAM_CTRL, 0, NULL);
 }
 
 static int ov2680_power_off(struct ov2680_dev *sensor)
@@ -369,11 +355,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 
 	sensor->is_enabled = true;
 
-	/* Set clock lane into LP-11 state */
-	ov2680_stream_enable(sensor);
-	usleep_range(1000, 2000);
-	ov2680_stream_disable(sensor);
-
 	return 0;
 
 err_disable_regulators:
@@ -381,26 +362,6 @@ static int ov2680_power_on(struct ov2680_dev *sensor)
 	return ret;
 }
 
-static int ov2680_s_power(struct v4l2_subdev *sd, int on)
-{
-	struct ov2680_dev *sensor = to_ov2680_dev(sd);
-	int ret = 0;
-
-	mutex_lock(&sensor->lock);
-
-	if (on)
-		ret = ov2680_power_on(sensor);
-	else
-		ret = ov2680_power_off(sensor);
-
-	if (on && ret == 0)
-		ret = ov2680_mode_restore(sensor);
-
-	mutex_unlock(&sensor->lock);
-
-	return ret;
-}
-
 static int ov2680_s_g_frame_interval(struct v4l2_subdev *sd,
 				     struct v4l2_subdev_frame_interval *fi)
 {
@@ -423,16 +384,20 @@ static int ov2680_s_stream(struct v4l2_subdev *sd, int enable)
 	if (sensor->is_streaming == !!enable)
 		goto unlock;
 
-	if (enable && sensor->mode_pending_changes) {
-		ret = ov2680_mode_set(sensor);
+	if (enable) {
+		ret = pm_runtime_resume_and_get(sensor->sd.dev);
 		if (ret < 0)
 			goto unlock;
-	}
 
-	if (enable)
 		ret = ov2680_stream_enable(sensor);
-	else
+		if (ret < 0) {
+			pm_runtime_put(sensor->sd.dev);
+			goto unlock;
+		}
+	} else {
 		ret = ov2680_stream_disable(sensor);
+		pm_runtime_put(sensor->sd.dev);
+	}
 
 	sensor->is_streaming = !!enable;
 
@@ -519,7 +484,6 @@ static int ov2680_set_fmt(struct v4l2_subdev *sd,
 
 	sensor->current_mode = mode;
 	sensor->fmt = format->format;
-	sensor->mode_pending_changes = true;
 
 unlock:
 	mutex_unlock(&sensor->lock);
@@ -603,10 +567,6 @@ static const struct v4l2_ctrl_ops ov2680_ctrl_ops = {
 	.s_ctrl = ov2680_s_ctrl,
 };
 
-static const struct v4l2_subdev_core_ops ov2680_core_ops = {
-	.s_power = ov2680_s_power,
-};
-
 static const struct v4l2_subdev_video_ops ov2680_video_ops = {
 	.g_frame_interval	= ov2680_s_g_frame_interval,
 	.s_frame_interval	= ov2680_s_g_frame_interval,
@@ -623,7 +583,6 @@ static const struct v4l2_subdev_pad_ops ov2680_pad_ops = {
 };
 
 static const struct v4l2_subdev_ops ov2680_subdev_ops = {
-	.core	= &ov2680_core_ops,
 	.video	= &ov2680_video_ops,
 	.pad	= &ov2680_pad_ops,
 };
@@ -643,8 +602,6 @@ static int ov2680_mode_init(struct ov2680_dev *sensor)
 
 	sensor->current_mode = init_mode;
 
-	sensor->mode_pending_changes = true;
-
 	return 0;
 }
 
@@ -722,8 +679,6 @@ static int ov2680_check_id(struct ov2680_dev *sensor)
 	u64 chip_id;
 	int ret;
 
-	ov2680_power_on(sensor);
-
 	ret = cci_read(sensor->regmap, OV2680_REG_CHIP_ID, &chip_id, NULL);
 	if (ret < 0) {
 		dev_err(sensor->dev, "failed to read chip id\n");
@@ -811,18 +766,39 @@ static int ov2680_probe(struct i2c_client *client)
 
 	mutex_init(&sensor->lock);
 
-	ret = ov2680_check_id(sensor);
+	/*
+	 * Power up and verify the chip now, so that if runtime pm is
+	 * disabled the chip is left on and streaming will work.
+	 */
+	ret = ov2680_power_on(sensor);
 	if (ret < 0)
 		goto lock_destroy;
 
+	ret = ov2680_check_id(sensor);
+	if (ret < 0)
+		goto err_powerdown;
+
+	pm_runtime_set_active(&client->dev);
+	pm_runtime_get_noresume(&client->dev);
+	pm_runtime_enable(&client->dev);
+
 	ret = ov2680_v4l2_register(sensor);
 	if (ret < 0)
-		goto lock_destroy;
+		goto err_pm_runtime;
+
+	pm_runtime_set_autosuspend_delay(&client->dev, 1000);
+	pm_runtime_use_autosuspend(&client->dev);
+	pm_runtime_put_autosuspend(&client->dev);
 
 	dev_info(dev, "ov2680 init correctly\n");
 
 	return 0;
 
+err_pm_runtime:
+	pm_runtime_disable(&client->dev);
+	pm_runtime_put_noidle(&client->dev);
+err_powerdown:
+	ov2680_power_off(sensor);
 lock_destroy:
 	dev_err(dev, "ov2680 init fail: %d\n", ret);
 	mutex_destroy(&sensor->lock);
@@ -839,9 +815,18 @@ static void ov2680_remove(struct i2c_client *client)
 	mutex_destroy(&sensor->lock);
 	media_entity_cleanup(&sensor->sd.entity);
 	v4l2_ctrl_handler_free(&sensor->ctrls.handler);
+
+	/*
+	 * Disable runtime PM. In case runtime PM is disabled in the kernel,
+	 * make sure to turn power off manually.
+	 */
+	pm_runtime_disable(&client->dev);
+	if (!pm_runtime_status_suspended(&client->dev))
+		ov2680_power_off(sensor);
+	pm_runtime_set_suspended(&client->dev);
 }
 
-static int __maybe_unused ov2680_suspend(struct device *dev)
+static int ov2680_suspend(struct device *dev)
 {
 	struct v4l2_subdev *sd = dev_get_drvdata(dev);
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
@@ -849,15 +834,19 @@ static int __maybe_unused ov2680_suspend(struct device *dev)
 	if (sensor->is_streaming)
 		ov2680_stream_disable(sensor);
 
-	return 0;
+	return ov2680_power_off(sensor);
 }
 
-static int __maybe_unused ov2680_resume(struct device *dev)
+static int ov2680_resume(struct device *dev)
 {
 	struct v4l2_subdev *sd = dev_get_drvdata(dev);
 	struct ov2680_dev *sensor = to_ov2680_dev(sd);
 	int ret;
 
+	ret = ov2680_power_on(sensor);
+	if (ret < 0)
+		goto stream_disable;
+
 	if (sensor->is_streaming) {
 		ret = ov2680_stream_enable(sensor);
 		if (ret < 0)
@@ -873,9 +862,8 @@ static int __maybe_unused ov2680_resume(struct device *dev)
 	return ret;
 }
 
-static const struct dev_pm_ops ov2680_pm_ops = {
-	SET_SYSTEM_SLEEP_PM_OPS(ov2680_suspend, ov2680_resume)
-};
+static DEFINE_RUNTIME_DEV_PM_OPS(ov2680_pm_ops, ov2680_suspend, ov2680_resume,
+				 NULL);
 
 static const struct of_device_id ov2680_dt_ids[] = {
 	{ .compatible = "ovti,ov2680" },
@@ -886,7 +874,7 @@ MODULE_DEVICE_TABLE(of, ov2680_dt_ids);
 static struct i2c_driver ov2680_i2c_driver = {
 	.driver = {
 		.name  = "ov2680",
-		.pm = &ov2680_pm_ops,
+		.pm = pm_sleep_ptr(&ov2680_pm_ops),
 		.of_match_table	= of_match_ptr(ov2680_dt_ids),
 	},
 	.probe		= ov2680_probe,
-- 
2.41.0