* [PATCH v4 0/7] Add AMD ISP4 driver
@ 2025-09-11 10:08 Bin Du
2025-09-11 10:08 ` [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver Bin Du
` (7 more replies)
0 siblings, 8 replies; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du
Hello,
AMD ISP4 is the AMD image processing gen 4 which can be found in HP ZBook Ultra G1a 14 inch Mobile Workstation PC (Ryzen AI Max 300 Series)
(https://ubuntu.com/certified/202411-36043)
This patch series introduces the initial driver support for the AMD ISP4.
Patch summary:
- Powers up/off and initializes ISP HW
- Configures and kicks off ISP FW
- Interacts with APP using standard V4l2 interface by video node
- Controls ISP HW and interacts with ISP FW to do image processing
- Supports enum/set output image format and resolution
- Supports queueing buffer from app and dequeuing ISP filled buffer to App
- It is verified on qv4l2, cheese and qcam
- It is verified together with following patches
platform/x86: Add AMD ISP platform config (https://lore.kernel.org/all/20250514215623.522746-1-pratap.nirujogi@amd.com/)
pinctrl: amd: isp411: Add amdisp GPIO pinctrl (https://github.com/torvalds/linux/commit/e97435ab09f3ad7b6a588dd7c4e45a96699bbb4a)
drm/amd/amdgpu: Add GPIO resources required for amdisp (https://gitlab.freedesktop.org/agd5f/linux/-/commit/ad0f5966ed8297aa47b3184192b00b7379ae0758)
drm/amd/amdgpu: Declare isp firmware binary file (https://gitlab.freedesktop.org/agd5f/linux/-/commit/35345917bc9f7c86152b270d9d93c220230b667f)
AMD ISP4 Key features:
- Processes bayer raw data from the connected sensor and output them to different YUV formats
- Downscale input image to different output image resolution
- Pipeline to do image processing on the input image including demosaic, denoise, 3A, etc.
----------
Changes v3 -> v4:
- Replace one mutex with guard mutex.
- Remove unnecessary bus_info initialization of v4l2_capability.
- Drop V4L2_CAP_IO_MC from capabilities of v4l2_capability.
- Modify document with better SOC description.
- Fix Test x86 failure in Media CI test https://linux-media.pages.freedesktop.org/-/users/patchwork/-/jobs/83470456/artifacts/report.htm
- Modify some commit messages by describing changes in imperative mood.
- Add media-ctl output in cover letter.
- Create separated dedicated amdgpu patch to add declaration MODULE_FIRMWARE("amdgpu/isp_4_1_1.bin");
- Fix typo errors and other cosmetic issues.
- Add DRM_AMD_ISP dependency in Kconfig.
Changes v2 -> v3:
- All the dependent patches in other modules (drm/amd/amdgpu, platform/x86, pinctrl/amd) merged on upstream mainline kernel (https://github.com/torvalds/linux) v6.17.
- Removed usage of amdgpu structs in ISP driver. Added helper functions in amdgpu accepting opaque params from ISP driver to allocate and release ISP GART buffers.
- Moved sensor and MIPI phy control entirely into ISP FW instead of the previous hybrid approach controlling sensor from both FW and x86 (sensor driver).
- Removed phy configuration and sensor binding as x86 (sensor driver) had relinquished the sensor control for ISP FW. With this approach the driver will be exposed as web camera like interface.
- New FW with built-in sensor driver is submitted on upstream linux-firmware repo (https://gitlab.com/kernel-firmware/linux-firmware/).
- Please note the new FW submitted is not directly compatible with OEM Kernel ISP4.0 (https://github.com/amd/Linux_ISP_Kernel/tree/4.0) and the previous ISP V2 patch series.
- If intend to use the new FW, please rebuild OEM ISP4.0 Kernel with CONFIG_VIDEO_OV05C10=N and CONFIG_PINCTRL_AMDISP=Y.
- Included critical fixes from Sultan Alsawaf branch (https://github.com/kerneltoast/kernel_x86_laptop.git) related to managing lifetime of isp buffers.
media: amd: isp4: Add missing refcount tracking to mmap memop
media: amd: isp4: Don't put or unmap the dmabuf when detaching
media: amd: isp4: Don't increment refcount when dmabuf export fails
media: amd: isp4: Fix possible use-after-free in isp4vid_vb2_put()
media: amd: isp4: Always export a new dmabuf from get_dmabuf memop
media: amd: isp4: Fix implicit dmabuf lifetime tracking
media: amd: isp4: Fix possible use-after-free when putting implicit dmabuf
media: amd: isp4: Simplify isp4vid_get_dmabuf() arguments
media: amd: isp4: Move up buf->vaddr check in isp4vid_get_dmabuf()
media: amd: isp4: Remove unused userptr memops
media: amd: isp4: Add missing cleanup on error in isp4vid_vb2_alloc()
media: amd: isp4: Release queued buffers on error in start_streaming
- Addressed all code related upstream comments
- Fix typo errors and other cosmetic issues.
Changes v1 -> v2:
- Fix media CI test errors and valid warnings
- Reduce patch number in the series from 9 to 8 by merging MAINTAINERS adding patch to the first patch
- In patch 5
- do modification to use remote endpoint instead of local endpoint
- use link frequency and port number as start phy parameter instead of extra added phy-id and phy-bit-rate property of endpoint
----------
It passes v4l2 compliance test, the test reports for:
(a) amd_isp_capture device /dev/video0
Compliance test for amd_isp_capture device /dev/video0:
-------------------------------------------------------
atg@atg-HP-PV:~/bin$ ./v4l2-compliance -d /dev/video0
v4l2-compliance 1.29.0-5348, 64 bits, 64-bit time_t
v4l2-compliance SHA: 75e3f0e2c2cb 2025-03-17 18:12:17
Compliance test for amd_isp_capture device /dev/video0:
Driver Info:
Driver name : amd_isp_capture
Card type : amd_isp_capture
Bus info : platform:amd_isp_capture
Driver version : 6.14.0
Capabilities : 0xa4200001
Video Capture
I/O MC
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x24200001
Video Capture
I/O MC
Streaming
Extended Pix Format
Media Driver Info:
Driver name : amd_isp_capture
Model : amd_isp41_mdev
Serial :
Bus info : platform:amd_isp_capture
Media version : 6.14.0
Hardware revision: 0x00000000 (0)
Driver version : 6.14.0
Interface Info:
ID : 0x03000005
Type : V4L Video
Entity Info:
ID : 0x00000003 (3)
Name : Preview
Function : V4L2 I/O
Pad 0x01000004 : 0: Sink
Link 0x02000007: from remote pad 0x1000002 of entity 'amd isp4' (Image Signal Processor): Data, Enabled, Immutable
Required ioctls:
test MC information (see 'Media Driver Info' above): OK
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
test VIDIOC_QUERYCTRL: OK (Not Supported)
test VIDIOC_G/S_CTRL: OK (Not Supported)
test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
Standard Controls: 0 Private Controls: 0
Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
test VIDIOC_G_ENC_INDEX: OK (Not Supported)
test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
test CREATE_BUFS maximum buffers: OK
test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
test blocking wait: OK
Total for amd_isp_capture device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
The media-ctl output of media device /dev/media0:
-------------------------------------------------------
atg@atg-HP-PV:~$ media-ctl -p -d /dev/media0
Media controller API version 6.17.0
Media device information
------------------------
driver amd_isp_capture
model amd_isp41_mdev
serial
bus info platform:amd_isp_capture
hw revision 0x0
driver version 6.17.0
Device topology
- entity 1: amd isp4 (1 pad, 1 link, 0 routes)
type V4L2 subdev subtype Unknown flags 0
pad0: Source
-> "Preview":0 [ENABLED,IMMUTABLE]
- entity 3: Preview (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "amd isp4":0 [ENABLED,IMMUTABLE]
Please review and provide feedback.
Many thanks,
Bin Du (7):
media: platform: amd: Introduce amd isp4 capture driver
media: platform: amd: low level support for isp4 firmware
media: platform: amd: Add isp4 fw and hw interface
media: platform: amd: isp4 subdev and firmware loading handling added
media: platform: amd: isp4 video node and buffers handling added
media: platform: amd: isp4 debug fs logging and more descriptive
errors
Documentation: add documentation of AMD isp 4 driver
Documentation/admin-guide/media/amdisp4-1.rst | 63 +
Documentation/admin-guide/media/amdisp4.dot | 6 +
.../admin-guide/media/v4l-drivers.rst | 1 +
MAINTAINERS | 25 +
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/amd/Kconfig | 3 +
drivers/media/platform/amd/Makefile | 3 +
drivers/media/platform/amd/isp4/Kconfig | 13 +
drivers/media/platform/amd/isp4/Makefile | 10 +
drivers/media/platform/amd/isp4/isp4.c | 237 ++++
drivers/media/platform/amd/isp4/isp4.h | 26 +
drivers/media/platform/amd/isp4/isp4_debug.c | 272 ++++
drivers/media/platform/amd/isp4/isp4_debug.h | 41 +
.../platform/amd/isp4/isp4_fw_cmd_resp.h | 314 +++++
drivers/media/platform/amd/isp4/isp4_hw_reg.h | 125 ++
.../media/platform/amd/isp4/isp4_interface.c | 966 +++++++++++++
.../media/platform/amd/isp4/isp4_interface.h | 149 ++
drivers/media/platform/amd/isp4/isp4_subdev.c | 1197 ++++++++++++++++
drivers/media/platform/amd/isp4/isp4_subdev.h | 133 ++
drivers/media/platform/amd/isp4/isp4_video.c | 1207 +++++++++++++++++
drivers/media/platform/amd/isp4/isp4_video.h | 87 ++
22 files changed, 4880 insertions(+)
create mode 100644 Documentation/admin-guide/media/amdisp4-1.rst
create mode 100644 Documentation/admin-guide/media/amdisp4.dot
create mode 100644 drivers/media/platform/amd/Kconfig
create mode 100644 drivers/media/platform/amd/Makefile
create mode 100644 drivers/media/platform/amd/isp4/Kconfig
create mode 100644 drivers/media/platform/amd/isp4/Makefile
create mode 100644 drivers/media/platform/amd/isp4/isp4.c
create mode 100644 drivers/media/platform/amd/isp4/isp4.h
create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.h
create mode 100644 drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
create mode 100644 drivers/media/platform/amd/isp4/isp4_hw_reg.h
create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.h
create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.h
create mode 100644 drivers/media/platform/amd/isp4/isp4_video.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_video.h
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
@ 2025-09-11 10:08 ` Bin Du
2025-09-21 20:23 ` Sultan Alsawaf
2025-09-11 10:08 ` [PATCH v4 2/7] media: platform: amd: low level support for isp4 firmware Bin Du
` (6 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du, Svetoslav Stoilov, Mario Limonciello,
Alexey Zagorodnikov
AMD isp4 capture is a v4l2 media device which implements media controller
interface. It has one sub-device (AMD ISP4 sub-device) endpoint which can
be connected to a remote CSI2 TX endpoint. It supports only one physical
interface for now. Also add ISP4 driver related entry info into the
MAINTAINERS file
Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Bin Du <Bin.Du@amd.com>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
---
MAINTAINERS | 13 +++
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/amd/Kconfig | 3 +
drivers/media/platform/amd/Makefile | 3 +
drivers/media/platform/amd/isp4/Kconfig | 13 +++
drivers/media/platform/amd/isp4/Makefile | 6 ++
drivers/media/platform/amd/isp4/isp4.c | 121 +++++++++++++++++++++++
drivers/media/platform/amd/isp4/isp4.h | 24 +++++
9 files changed, 185 insertions(+)
create mode 100644 drivers/media/platform/amd/Kconfig
create mode 100644 drivers/media/platform/amd/Makefile
create mode 100644 drivers/media/platform/amd/isp4/Kconfig
create mode 100644 drivers/media/platform/amd/isp4/Makefile
create mode 100644 drivers/media/platform/amd/isp4/isp4.c
create mode 100644 drivers/media/platform/amd/isp4/isp4.h
diff --git a/MAINTAINERS b/MAINTAINERS
index cd7ff55b5d32..3640a1e3262c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1133,6 +1133,19 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
F: drivers/iommu/amd/
F: include/linux/amd-iommu.h
+AMD ISP4 DRIVER
+M: Bin Du <bin.du@amd.com>
+M: Nirujogi Pratap <pratap.nirujogi@amd.com>
+L: linux-media@vger.kernel.org
+S: Maintained
+T: git git://linuxtv.org/media.git
+F: drivers/media/platform/amd/Kconfig
+F: drivers/media/platform/amd/Makefile
+F: drivers/media/platform/amd/isp4/Kconfig
+F: drivers/media/platform/amd/isp4/Makefile
+F: drivers/media/platform/amd/isp4/isp4.c
+F: drivers/media/platform/amd/isp4/isp4.h
+
AMD KFD
M: Felix Kuehling <Felix.Kuehling@amd.com>
L: amd-gfx@lists.freedesktop.org
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 9287faafdce5..772c70665510 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -63,6 +63,7 @@ config VIDEO_MUX
# Platform drivers - Please keep it alphabetically sorted
source "drivers/media/platform/allegro-dvt/Kconfig"
+source "drivers/media/platform/amd/Kconfig"
source "drivers/media/platform/amlogic/Kconfig"
source "drivers/media/platform/amphion/Kconfig"
source "drivers/media/platform/aspeed/Kconfig"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 6fd7db0541c7..b207bd8d8022 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -6,6 +6,7 @@
# Place here, alphabetically sorted by directory
# (e. g. LC_ALL=C sort Makefile)
obj-y += allegro-dvt/
+obj-y += amd/
obj-y += amlogic/
obj-y += amphion/
obj-y += aspeed/
diff --git a/drivers/media/platform/amd/Kconfig b/drivers/media/platform/amd/Kconfig
new file mode 100644
index 000000000000..25af49f246b2
--- /dev/null
+++ b/drivers/media/platform/amd/Kconfig
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0+
+
+source "drivers/media/platform/amd/isp4/Kconfig"
diff --git a/drivers/media/platform/amd/Makefile b/drivers/media/platform/amd/Makefile
new file mode 100644
index 000000000000..8bfc1955f22e
--- /dev/null
+++ b/drivers/media/platform/amd/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0+
+
+obj-y += isp4/
diff --git a/drivers/media/platform/amd/isp4/Kconfig b/drivers/media/platform/amd/isp4/Kconfig
new file mode 100644
index 000000000000..52962eee8564
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/Kconfig
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0+
+
+config AMD_ISP4
+ tristate "AMD ISP4 and camera driver"
+ depends on VIDEO_DEV && VIDEO_V4L2_SUBDEV_API && DRM_AMD_ISP
+ select VIDEOBUF2_CORE
+ select VIDEOBUF2_V4L2
+ select VIDEOBUF2_MEMOPS
+ help
+ This is support for AMD ISP4 and camera subsystem driver.
+ Say Y here to enable the ISP4 and camera device for video capture.
+ To compile this driver as a module, choose M here. The module will
+ be called amd_capture.
diff --git a/drivers/media/platform/amd/isp4/Makefile b/drivers/media/platform/amd/isp4/Makefile
new file mode 100644
index 000000000000..de0092dad26f
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Copyright (C) 2025 Advanced Micro Devices, Inc.
+
+obj-$(CONFIG_AMD_ISP4) += amd_capture.o
+amd_capture-objs := isp4.o
diff --git a/drivers/media/platform/amd/isp4/isp4.c b/drivers/media/platform/amd/isp4/isp4.c
new file mode 100644
index 000000000000..6ff3ded4310a
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4.c
@@ -0,0 +1,121 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#include <linux/pm_runtime.h>
+#include <linux/vmalloc.h>
+#include <media/v4l2-ioctl.h>
+
+#include "isp4.h"
+
+#define VIDEO_BUF_NUM 5
+
+#define ISP4_DRV_NAME "amd_isp_capture"
+
+/* interrupt num */
+static const u32 isp4_ringbuf_interrupt_num[] = {
+ 0, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT9 */
+ 1, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT10 */
+ 3, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT11 */
+ 4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
+};
+
+#define to_isp4_device(dev) \
+ ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
+
+static irqreturn_t isp4_irq_handler(int irq, void *arg)
+{
+ return IRQ_HANDLED;
+}
+
+static int isp4_capture_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct isp4_device *isp_dev;
+ int i, irq, ret;
+
+ isp_dev = devm_kzalloc(&pdev->dev, sizeof(*isp_dev), GFP_KERNEL);
+ if (!isp_dev)
+ return -ENOMEM;
+
+ isp_dev->pdev = pdev;
+ dev->init_name = ISP4_DRV_NAME;
+
+ for (i = 0; i < ARRAY_SIZE(isp4_ringbuf_interrupt_num); i++) {
+ irq = platform_get_irq(pdev, isp4_ringbuf_interrupt_num[i]);
+ if (irq < 0)
+ return dev_err_probe(dev, -ENODEV,
+ "fail to get irq %d\n",
+ isp4_ringbuf_interrupt_num[i]);
+ ret = devm_request_irq(&pdev->dev, irq, isp4_irq_handler, 0,
+ "ISP_IRQ", &pdev->dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "fail to req irq %d\n",
+ irq);
+ }
+
+ /* Link the media device within the v4l2_device */
+ isp_dev->v4l2_dev.mdev = &isp_dev->mdev;
+
+ /* Initialize media device */
+ strscpy(isp_dev->mdev.model, "amd_isp41_mdev",
+ sizeof(isp_dev->mdev.model));
+ snprintf(isp_dev->mdev.bus_info, sizeof(isp_dev->mdev.bus_info),
+ "platform:%s", ISP4_DRV_NAME);
+ isp_dev->mdev.dev = &pdev->dev;
+ media_device_init(&isp_dev->mdev);
+
+ /* register v4l2 device */
+ snprintf(isp_dev->v4l2_dev.name, sizeof(isp_dev->v4l2_dev.name),
+ "AMD-V4L2-ROOT");
+ ret = v4l2_device_register(&pdev->dev, &isp_dev->v4l2_dev);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "fail register v4l2 device\n");
+
+ ret = media_device_register(&isp_dev->mdev);
+ if (ret) {
+ dev_err(dev, "fail to register media device %d\n", ret);
+ goto err_unreg_v4l2;
+ }
+
+ platform_set_drvdata(pdev, isp_dev);
+
+ pm_runtime_set_suspended(dev);
+ pm_runtime_enable(dev);
+
+ return 0;
+
+err_unreg_v4l2:
+ v4l2_device_unregister(&isp_dev->v4l2_dev);
+
+ return dev_err_probe(dev, ret, "isp probe fail\n");
+}
+
+static void isp4_capture_remove(struct platform_device *pdev)
+{
+ struct isp4_device *isp_dev = platform_get_drvdata(pdev);
+
+ media_device_unregister(&isp_dev->mdev);
+ v4l2_device_unregister(&isp_dev->v4l2_dev);
+}
+
+static struct platform_driver isp4_capture_drv = {
+ .probe = isp4_capture_probe,
+ .remove = isp4_capture_remove,
+ .driver = {
+ .name = ISP4_DRV_NAME,
+ .owner = THIS_MODULE,
+ }
+};
+
+module_platform_driver(isp4_capture_drv);
+
+MODULE_ALIAS("platform:" ISP4_DRV_NAME);
+MODULE_IMPORT_NS("DMA_BUF");
+
+MODULE_DESCRIPTION("AMD ISP4 Driver");
+MODULE_AUTHOR("Bin Du <bin.du@amd.com>");
+MODULE_AUTHOR("Pratap Nirujogi <pratap.nirujogi@amd.com>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/amd/isp4/isp4.h b/drivers/media/platform/amd/isp4/isp4.h
new file mode 100644
index 000000000000..8535f662ab49
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#ifndef _ISP4_H_
+#define _ISP4_H_
+
+#include <linux/mutex.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-memops.h>
+#include <media/videobuf2-vmalloc.h>
+
+#define ISP4_GET_ISP_REG_BASE(isp4sd) (((isp4sd))->mmio)
+
+struct isp4_device {
+ struct v4l2_device v4l2_dev;
+ struct media_device mdev;
+
+ struct platform_device *pdev;
+ struct notifier_block i2c_nb;
+};
+
+#endif /* _ISP4_H_ */
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 2/7] media: platform: amd: low level support for isp4 firmware
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
2025-09-11 10:08 ` [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver Bin Du
@ 2025-09-11 10:08 ` Bin Du
2025-09-21 20:31 ` Sultan Alsawaf
2025-09-11 10:08 ` [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface Bin Du
` (5 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du, Svetoslav Stoilov,
Alexey Zagorodnikov
Low level functions for accessing the registers and mapping to their
ranges. This change also includes register definitions for ring buffer
used to communicate with ISP Firmware. Ring buffer is the communication
interface between driver and ISP Firmware. Command and responses are
exchanged through the ring buffer.
Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Bin Du <Bin.Du@amd.com>
Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
---
MAINTAINERS | 1 +
drivers/media/platform/amd/isp4/Makefile | 2 +-
drivers/media/platform/amd/isp4/isp4_hw_reg.h | 125 ++++++++++++++++++
3 files changed, 127 insertions(+), 1 deletion(-)
create mode 100644 drivers/media/platform/amd/isp4/isp4_hw_reg.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 3640a1e3262c..7aa17c7e71d6 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1145,6 +1145,7 @@ F: drivers/media/platform/amd/isp4/Kconfig
F: drivers/media/platform/amd/isp4/Makefile
F: drivers/media/platform/amd/isp4/isp4.c
F: drivers/media/platform/amd/isp4/isp4.h
+F: drivers/media/platform/amd/isp4/isp4_hw_reg.h
AMD KFD
M: Felix Kuehling <Felix.Kuehling@amd.com>
diff --git a/drivers/media/platform/amd/isp4/Makefile b/drivers/media/platform/amd/isp4/Makefile
index de0092dad26f..eb130647fbe2 100644
--- a/drivers/media/platform/amd/isp4/Makefile
+++ b/drivers/media/platform/amd/isp4/Makefile
@@ -3,4 +3,4 @@
# Copyright (C) 2025 Advanced Micro Devices, Inc.
obj-$(CONFIG_AMD_ISP4) += amd_capture.o
-amd_capture-objs := isp4.o
+amd_capture-objs := isp4.o \
diff --git a/drivers/media/platform/amd/isp4/isp4_hw_reg.h b/drivers/media/platform/amd/isp4/isp4_hw_reg.h
new file mode 100644
index 000000000000..16a02a14d985
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_hw_reg.h
@@ -0,0 +1,125 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#ifndef _ISP4_HW_REG_H_
+#define _ISP4_HW_REG_H_
+
+#include <linux/io.h>
+#include <linux/types.h>
+
+#define ISP_SOFT_RESET 0x62000
+#define ISP_SYS_INT0_EN 0x62010
+#define ISP_SYS_INT0_STATUS 0x62014
+#define ISP_SYS_INT0_ACK 0x62018
+#define ISP_CCPU_CNTL 0x62054
+#define ISP_STATUS 0x62058
+#define ISP_LOG_RB_BASE_LO0 0x62148
+#define ISP_LOG_RB_BASE_HI0 0x6214c
+#define ISP_LOG_RB_SIZE0 0x62150
+#define ISP_LOG_RB_RPTR0 0x62154
+#define ISP_LOG_RB_WPTR0 0x62158
+#define ISP_RB_BASE_LO1 0x62170
+#define ISP_RB_BASE_HI1 0x62174
+#define ISP_RB_SIZE1 0x62178
+#define ISP_RB_RPTR1 0x6217c
+#define ISP_RB_WPTR1 0x62180
+#define ISP_RB_BASE_LO2 0x62184
+#define ISP_RB_BASE_HI2 0x62188
+#define ISP_RB_SIZE2 0x6218c
+#define ISP_RB_RPTR2 0x62190
+#define ISP_RB_WPTR2 0x62194
+#define ISP_RB_BASE_LO3 0x62198
+#define ISP_RB_BASE_HI3 0x6219c
+#define ISP_RB_SIZE3 0x621a0
+#define ISP_RB_RPTR3 0x621a4
+#define ISP_RB_WPTR3 0x621a8
+#define ISP_RB_BASE_LO4 0x621ac
+#define ISP_RB_BASE_HI4 0x621b0
+#define ISP_RB_SIZE4 0x621b4
+#define ISP_RB_RPTR4 0x621b8
+#define ISP_RB_WPTR4 0x621bc
+#define ISP_RB_BASE_LO5 0x621c0
+#define ISP_RB_BASE_HI5 0x621c4
+#define ISP_RB_SIZE5 0x621c8
+#define ISP_RB_RPTR5 0x621cc
+#define ISP_RB_WPTR5 0x621d0
+#define ISP_RB_BASE_LO6 0x621d4
+#define ISP_RB_BASE_HI6 0x621d8
+#define ISP_RB_SIZE6 0x621dc
+#define ISP_RB_RPTR6 0x621e0
+#define ISP_RB_WPTR6 0x621e4
+#define ISP_RB_BASE_LO7 0x621e8
+#define ISP_RB_BASE_HI7 0x621ec
+#define ISP_RB_SIZE7 0x621f0
+#define ISP_RB_RPTR7 0x621f4
+#define ISP_RB_WPTR7 0x621f8
+#define ISP_RB_BASE_LO8 0x621fc
+#define ISP_RB_BASE_HI8 0x62200
+#define ISP_RB_SIZE8 0x62204
+#define ISP_RB_RPTR8 0x62208
+#define ISP_RB_WPTR8 0x6220c
+#define ISP_RB_BASE_LO9 0x62210
+#define ISP_RB_BASE_HI9 0x62214
+#define ISP_RB_SIZE9 0x62218
+#define ISP_RB_RPTR9 0x6221c
+#define ISP_RB_WPTR9 0x62220
+#define ISP_RB_BASE_LO10 0x62224
+#define ISP_RB_BASE_HI10 0x62228
+#define ISP_RB_SIZE10 0x6222c
+#define ISP_RB_RPTR10 0x62230
+#define ISP_RB_WPTR10 0x62234
+#define ISP_RB_BASE_LO11 0x62238
+#define ISP_RB_BASE_HI11 0x6223c
+#define ISP_RB_SIZE11 0x62240
+#define ISP_RB_RPTR11 0x62244
+#define ISP_RB_WPTR11 0x62248
+#define ISP_RB_BASE_LO12 0x6224c
+#define ISP_RB_BASE_HI12 0x62250
+#define ISP_RB_SIZE12 0x62254
+#define ISP_RB_RPTR12 0x62258
+#define ISP_RB_WPTR12 0x6225c
+
+#define ISP_POWER_STATUS 0x60000
+
+/* ISP_SOFT_RESET */
+#define ISP_SOFT_RESET__CCPU_SOFT_RESET_MASK 0x00000001UL
+
+/* ISP_CCPU_CNTL */
+#define ISP_CCPU_CNTL__CCPU_HOST_SOFT_RST_MASK 0x00040000UL
+
+/* ISP_STATUS */
+#define ISP_STATUS__CCPU_REPORT_MASK 0x000000feUL
+
+/* ISP_SYS_INT0_STATUS */
+#define ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK 0x00010000UL
+#define ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK 0x00040000UL
+#define ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK 0x00100000UL
+#define ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK 0x00400000UL
+
+/* ISP_SYS_INT0_EN */
+#define ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT9_EN_MASK 0x00010000UL
+#define ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT10_EN_MASK 0x00040000UL
+#define ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT11_EN_MASK 0x00100000UL
+#define ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT12_EN_MASK 0x00400000UL
+
+/* ISP_SYS_INT0_ACK */
+#define ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT9_ACK_MASK 0x00010000UL
+#define ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT10_ACK_MASK 0x00040000UL
+#define ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT11_ACK_MASK 0x00100000UL
+#define ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT12_ACK_MASK 0x00400000UL
+
+/* Helper functions for reading isp registers */
+static inline u32 isp4hw_rreg(void __iomem *base, u32 reg)
+{
+ return readl(base + reg);
+}
+
+/* Helper functions for writing isp registers */
+static inline void isp4hw_wreg(void __iomem *base, u32 reg, u32 val)
+{
+ return writel(val, base + reg);
+}
+
+#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
2025-09-11 10:08 ` [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver Bin Du
2025-09-11 10:08 ` [PATCH v4 2/7] media: platform: amd: low level support for isp4 firmware Bin Du
@ 2025-09-11 10:08 ` Bin Du
2025-09-21 21:55 ` Sultan Alsawaf
2025-09-11 10:08 ` [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added Bin Du
` (4 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du, Svetoslav Stoilov,
Alexey Zagorodnikov
ISP firmware controls ISP HW pipeline using dedicated embedded processor
called ccpu. The communication between ISP FW and driver is using commands
and response messages sent through the ring buffer. Command buffers support
either global setting that is not specific to the steam and support stream
specific parameters. Response buffers contain ISP FW notification
information such as frame buffer done and command done. IRQ is used for
receiving response buffer from ISP firmware, which is handled in the main
isp4 media device. ISP ccpu is booted up through the firmware loading
helper function prior to stream start. Memory used for command buffer and
response buffer needs to be allocated from amdgpu buffer manager because
isp4 is a child device of amdgpu.
Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Bin Du <Bin.Du@amd.com>
Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
---
MAINTAINERS | 3 +
drivers/media/platform/amd/isp4/Makefile | 1 +
.../platform/amd/isp4/isp4_fw_cmd_resp.h | 314 ++++++
.../media/platform/amd/isp4/isp4_interface.c | 955 ++++++++++++++++++
.../media/platform/amd/isp4/isp4_interface.h | 149 +++
5 files changed, 1422 insertions(+)
create mode 100644 drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 7aa17c7e71d6..cccae369c876 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1145,7 +1145,10 @@ F: drivers/media/platform/amd/isp4/Kconfig
F: drivers/media/platform/amd/isp4/Makefile
F: drivers/media/platform/amd/isp4/isp4.c
F: drivers/media/platform/amd/isp4/isp4.h
+F: drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
F: drivers/media/platform/amd/isp4/isp4_hw_reg.h
+F: drivers/media/platform/amd/isp4/isp4_interface.c
+F: drivers/media/platform/amd/isp4/isp4_interface.h
AMD KFD
M: Felix Kuehling <Felix.Kuehling@amd.com>
diff --git a/drivers/media/platform/amd/isp4/Makefile b/drivers/media/platform/amd/isp4/Makefile
index eb130647fbe2..327ed1157076 100644
--- a/drivers/media/platform/amd/isp4/Makefile
+++ b/drivers/media/platform/amd/isp4/Makefile
@@ -4,3 +4,4 @@
obj-$(CONFIG_AMD_ISP4) += amd_capture.o
amd_capture-objs := isp4.o \
+ isp4_interface.o \
diff --git a/drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h b/drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
new file mode 100644
index 000000000000..2a21cc4b80e9
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
@@ -0,0 +1,314 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#ifndef _ISP4_CMD_RESP_H_
+#define _ISP4_CMD_RESP_H_
+
+/*
+ * @brief Host and Firmware command & response channel.
+ * Two types of command/response channel.
+ * Type Global Command has one command/response channel.
+ * Type Stream Command has one command/response channel.
+ *----------- ------------
+ *| | --------------------------- | |
+ *| | ---->| Global Command |----> | |
+ *| | --------------------------- | |
+ *| | | |
+ *| | | |
+ *| | --------------------------- | |
+ *| | ---->| Stream Command |----> | |
+ *| | --------------------------- | |
+ *| | | |
+ *| | | |
+ *| | | |
+ *| HOST | | Firmware |
+ *| | | |
+ *| | | |
+ *| | -------------------------- | |
+ *| | <----| Global Response |<---- | |
+ *| | -------------------------- | |
+ *| | | |
+ *| | | |
+ *| | -------------------------- | |
+ *| | <----| Stream Response |<---- | |
+ *| | -------------------------- | |
+ *| | | |
+ *| | | |
+ *----------- ------------
+ */
+
+/*
+ * @brief command ID format
+ * cmd_id is in the format of following type:
+ * type: indicate command type, global/stream commands.
+ * group: indicate the command group.
+ * id: A unique command identification in one type and group.
+ * |<-Bit31 ~ Bit24->|<-Bit23 ~ Bit16->|<-Bit15 ~ Bit0->|
+ * | type | group | id |
+ */
+
+#define CMD_TYPE_SHIFT 24
+#define CMD_GROUP_SHIFT 16
+#define CMD_TYPE_STREAM_CTRL (0x2U << CMD_TYPE_SHIFT)
+
+#define CMD_GROUP_STREAM_CTRL (0x1U << CMD_GROUP_SHIFT)
+#define CMD_GROUP_STREAM_BUFFER (0x4U << CMD_GROUP_SHIFT)
+
+/* Stream Command */
+#define CMD_ID_SET_STREAM_CONFIG (CMD_TYPE_STREAM_CTRL | CMD_GROUP_STREAM_CTRL | 0x1)
+#define CMD_ID_SET_OUT_CHAN_PROP (CMD_TYPE_STREAM_CTRL | CMD_GROUP_STREAM_CTRL | 0x3)
+#define CMD_ID_ENABLE_OUT_CHAN (CMD_TYPE_STREAM_CTRL | CMD_GROUP_STREAM_CTRL | 0x5)
+#define CMD_ID_START_STREAM (CMD_TYPE_STREAM_CTRL | CMD_GROUP_STREAM_CTRL | 0x7)
+#define CMD_ID_STOP_STREAM (CMD_TYPE_STREAM_CTRL | CMD_GROUP_STREAM_CTRL | 0x8)
+
+/* Stream Buffer Command */
+#define CMD_ID_SEND_BUFFER (CMD_TYPE_STREAM_CTRL | CMD_GROUP_STREAM_BUFFER | 0x1)
+
+/*
+ * @brief response ID format
+ * resp_id is in the format of following type:
+ * type: indicate command type, global/stream commands.
+ * group: indicate the command group.
+ * id: A unique command identification in one type and group.
+ * |<-Bit31 ~ Bit24->|<-Bit23 ~ Bit16->|<-Bit15 ~ Bit0->|
+ * | type | group | id |
+ */
+
+#define RESP_GROUP_SHIFT 16
+#define RESP_GROUP_MASK (0xff << RESP_GROUP_SHIFT)
+
+#define GET_RESP_GROUP_VALUE(resp_id) (((resp_id) & RESP_GROUP_MASK) >> RESP_GROUP_SHIFT)
+#define GET_RESP_ID_VALUE(resp_id) ((resp_id) & 0xffff)
+
+#define RESP_GROUP_GENERAL (0x1 << RESP_GROUP_SHIFT)
+#define RESP_GROUP_NOTIFICATION (0x3 << RESP_GROUP_SHIFT)
+
+/* General Response */
+#define RESP_ID_CMD_DONE (RESP_GROUP_GENERAL | 0x1)
+
+/* Notification */
+#define RESP_ID_NOTI_FRAME_DONE (RESP_GROUP_NOTIFICATION | 0x1)
+
+#define CMD_STATUS_SUCCESS 0
+#define CMD_STATUS_FAIL 1
+#define CMD_STATUS_SKIPPED 2
+
+#define ADDR_SPACE_TYPE_GPU_VA 4
+
+#define FW_MEMORY_POOL_SIZE (200 * 1024 * 1024)
+
+/*
+ * standard ISP mipicsi=>isp
+ */
+#define MIPI0_ISP_PIPELINE_ID 0x5f91
+
+enum isp4fw_sensor_id {
+ SENSOR_ID_ON_MIPI0 = 0, /* Sensor id for ISP input from MIPI port 0 */
+};
+
+enum isp4fw_stream_id {
+ STREAM_ID_INVALID = -1, /* STREAM_ID_INVALID. */
+ STREAM_ID_1 = 0, /* STREAM_ID_1. */
+ STREAM_ID_2 = 1, /* STREAM_ID_2. */
+ STREAM_ID_3 = 2, /* STREAM_ID_3. */
+ STREAM_ID_MAXIMUM /* STREAM_ID_MAXIMUM. */
+};
+
+enum isp4fw_image_format {
+ IMAGE_FORMAT_NV12 = 1, /* 4:2:0,semi-planar, 8-bit */
+ IMAGE_FORMAT_YUV422INTERLEAVED = 7, /* interleave, 4:2:2, 8-bit */
+};
+
+enum isp4fw_pipe_out_ch {
+ ISP_PIPE_OUT_CH_PREVIEW = 0,
+};
+
+enum isp4fw_yuv_range {
+ ISP_YUV_RANGE_FULL = 0, /* YUV value range in 0~255 */
+ ISP_YUV_RANGE_NARROW = 1, /* YUV value range in 16~235 */
+ ISP_YUV_RANGE_MAX
+};
+
+enum isp4fw_buffer_type {
+ BUFFER_TYPE_PREVIEW = 8,
+ BUFFER_TYPE_META_INFO = 10,
+ BUFFER_TYPE_MEM_POOL = 15,
+};
+
+enum isp4fw_buffer_status {
+ BUFFER_STATUS_INVALID, /* The buffer is INVALID */
+ BUFFER_STATUS_SKIPPED, /* The buffer is not filled with image data */
+ BUFFER_STATUS_EXIST, /* The buffer is exist and waiting for filled */
+ BUFFER_STATUS_DONE, /* The buffer is filled with image data */
+ BUFFER_STATUS_LACK, /* The buffer is unavailable */
+ BUFFER_STATUS_DIRTY, /* The buffer is dirty, probably caused by
+ * LMI leakage
+ */
+ BUFFER_STATUS_MAX /* The buffer STATUS_MAX */
+};
+
+enum isp4fw_buffer_source {
+ /* The buffer is from the stream buffer queue */
+ BUFFER_SOURCE_STREAM,
+};
+
+struct isp4fw_error_code {
+ u32 code1;
+ u32 code2;
+ u32 code3;
+ u32 code4;
+ u32 code5;
+};
+
+/*
+ * Command Structure for FW
+ */
+
+struct isp4fw_cmd {
+ u32 cmd_seq_num;
+ u32 cmd_id;
+ u32 cmd_param[12];
+ u16 cmd_stream_id;
+ u8 cmd_silent_resp;
+ u8 reserved;
+ u32 cmd_check_sum;
+};
+
+struct isp4fw_resp_cmd_done {
+ /*
+ * The host2fw command seqNum.
+ * To indicate which command this response refers to.
+ */
+ u32 cmd_seq_num;
+ /* The host2fw command id for host double check. */
+ u32 cmd_id;
+ /*
+ * Indicate the command process status.
+ * 0 means success. 1 means fail. 2 means skipped
+ */
+ u16 cmd_status;
+ /*
+ * If the cmd_status is 1, that means the command is processed fail,
+ * host can check the isp4fw_error_code to get the details
+ * error information
+ */
+ u16 isp4fw_error_code;
+ /* The response payload will be in different struct type */
+ /* according to different cmd done response. */
+ u8 payload[36];
+};
+
+struct isp4fw_resp_param_package {
+ u32 package_addr_lo; /* The low 32 bit addr of the pkg address. */
+ u32 package_addr_hi; /* The high 32 bit addr of the pkg address. */
+ u32 package_size; /* The total pkg size in bytes. */
+ u32 package_check_sum; /* The byte sum of the pkg. */
+};
+
+struct isp4fw_resp {
+ u32 resp_seq_num;
+ u32 resp_id;
+ union {
+ struct isp4fw_resp_cmd_done cmd_done;
+ struct isp4fw_resp_param_package frame_done;
+ u32 resp_param[12];
+ } param;
+ u8 reserved[4];
+ u32 resp_check_sum;
+};
+
+struct isp4fw_mipi_pipe_path_cfg {
+ u32 b_enable;
+ enum isp4fw_sensor_id isp4fw_sensor_id;
+};
+
+struct isp4fw_isp_pipe_path_cfg {
+ u32 isp_pipe_id; /* pipe ids for pipeline construction */
+};
+
+struct isp4fw_isp_stream_cfg {
+ /* Isp mipi path */
+ struct isp4fw_mipi_pipe_path_cfg mipi_pipe_path_cfg;
+ /* Isp pipe path */
+ struct isp4fw_isp_pipe_path_cfg isp_pipe_path_cfg;
+ /* enable TNR */
+ u32 b_enable_tnr;
+ /*
+ * number of frame rta per-processing,
+ * set to 0 to use fw default value
+ */
+ u32 rta_frames_per_proc;
+};
+
+struct isp4fw_image_prop {
+ enum isp4fw_image_format image_format; /* Image format */
+ u32 width; /* Width */
+ u32 height; /* Height */
+ u32 luma_pitch; /* Luma pitch */
+ u32 chroma_pitch; /* Chrom pitch */
+ enum isp4fw_yuv_range yuv_range; /* YUV value range */
+};
+
+struct isp4fw_buffer {
+ /* A check num for debug usage, host need to */
+ /* set the buf_tags to different number */
+ u32 buf_tags;
+ union {
+ u32 value;
+ struct {
+ u32 space : 16;
+ u32 vmid : 16;
+ } bit;
+ } vmid_space;
+ u32 buf_base_a_lo; /* Low address of buffer A */
+ u32 buf_base_a_hi; /* High address of buffer A */
+ u32 buf_size_a; /* Buffer size of buffer A */
+
+ u32 buf_base_b_lo; /* Low address of buffer B */
+ u32 buf_base_b_hi; /* High address of buffer B */
+ u32 buf_size_b; /* Buffer size of buffer B */
+
+ u32 buf_base_c_lo; /* Low address of buffer C */
+ u32 buf_base_c_hi; /* High address of buffer C */
+ u32 buf_size_c; /* Buffer size of buffer C */
+};
+
+struct isp4fw_buffer_meta_info {
+ u32 enabled; /* enabled flag */
+ enum isp4fw_buffer_status status; /* BufferStatus */
+ struct isp4fw_error_code err; /* err code */
+ enum isp4fw_buffer_source source; /* BufferSource */
+ struct isp4fw_image_prop image_prop; /* image_prop */
+ struct isp4fw_buffer buffer; /* buffer */
+};
+
+struct isp4fw_meta_info {
+ u32 poc; /* frame id */
+ u32 fc_id; /* frame ctl id */
+ u32 time_stamp_lo; /* time_stamp_lo */
+ u32 time_stamp_hi; /* time_stamp_hi */
+ struct isp4fw_buffer_meta_info preview; /* preview BufferMetaInfo */
+};
+
+struct isp4fw_cmd_send_buffer {
+ enum isp4fw_buffer_type buffer_type; /* buffer Type */
+ struct isp4fw_buffer buffer; /* buffer info */
+};
+
+struct isp4fw_cmd_set_out_ch_prop {
+ enum isp4fw_pipe_out_ch ch; /* ISP pipe out channel */
+ struct isp4fw_image_prop image_prop; /* image property */
+};
+
+struct isp4fw_cmd_enable_out_ch {
+ enum isp4fw_pipe_out_ch ch; /* ISP pipe out channel */
+ u32 is_enable; /* If enable channel or not */
+};
+
+struct isp4fw_cmd_set_stream_cfg {
+ struct isp4fw_isp_stream_cfg stream_cfg; /* stream path config */
+};
+
+#endif
diff --git a/drivers/media/platform/amd/isp4/isp4_interface.c b/drivers/media/platform/amd/isp4/isp4_interface.c
new file mode 100644
index 000000000000..52dcca57ce2e
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_interface.c
@@ -0,0 +1,955 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#include <drm/amd/isp.h>
+#include <linux/iopoll.h>
+#include <linux/mutex.h>
+#include <linux/platform_device.h>
+
+#include "isp4_fw_cmd_resp.h"
+#include "isp4_hw_reg.h"
+#include "isp4_interface.h"
+
+#define ISP4IF_FW_RESP_RB_IRQ_EN_MASK \
+ (ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT9_EN_MASK | \
+ ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT10_EN_MASK | \
+ ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT11_EN_MASK | \
+ ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT12_EN_MASK)
+
+struct isp4if_rb_config {
+ const char *name;
+ u32 index;
+ u32 reg_rptr;
+ u32 reg_wptr;
+ u32 reg_base_lo;
+ u32 reg_base_hi;
+ u32 reg_size;
+ u32 val_size;
+ u64 base_mc_addr;
+ void *base_sys_addr;
+};
+
+/* FW cmd ring buffer configuration */
+static struct isp4if_rb_config
+ isp4if_cmd_rb_config[ISP4IF_STREAM_ID_MAX] = {
+ {
+ .name = "CMD_RB_GBL0",
+ .index = 3,
+ .reg_rptr = ISP_RB_RPTR4,
+ .reg_wptr = ISP_RB_WPTR4,
+ .reg_base_lo = ISP_RB_BASE_LO4,
+ .reg_base_hi = ISP_RB_BASE_HI4,
+ .reg_size = ISP_RB_SIZE4,
+ },
+ {
+ .name = "CMD_RB_STR1",
+ .index = 0,
+ .reg_rptr = ISP_RB_RPTR1,
+ .reg_wptr = ISP_RB_WPTR1,
+ .reg_base_lo = ISP_RB_BASE_LO1,
+ .reg_base_hi = ISP_RB_BASE_HI1,
+ .reg_size = ISP_RB_SIZE1,
+ },
+ {
+ .name = "CMD_RB_STR2",
+ .index = 1,
+ .reg_rptr = ISP_RB_RPTR2,
+ .reg_wptr = ISP_RB_WPTR2,
+ .reg_base_lo = ISP_RB_BASE_LO2,
+ .reg_base_hi = ISP_RB_BASE_HI2,
+ .reg_size = ISP_RB_SIZE2,
+ },
+ {
+ .name = "CMD_RB_STR3",
+ .index = 2,
+ .reg_rptr = ISP_RB_RPTR3,
+ .reg_wptr = ISP_RB_WPTR3,
+ .reg_base_lo = ISP_RB_BASE_LO3,
+ .reg_base_hi = ISP_RB_BASE_HI3,
+ .reg_size = ISP_RB_SIZE3,
+ },
+};
+
+/* FW resp ring buffer configuration */
+static struct isp4if_rb_config
+ isp4if_resp_rb_config[ISP4IF_STREAM_ID_MAX] = {
+ {
+ .name = "RES_RB_GBL0",
+ .index = 3,
+ .reg_rptr = ISP_RB_RPTR12,
+ .reg_wptr = ISP_RB_WPTR12,
+ .reg_base_lo = ISP_RB_BASE_LO12,
+ .reg_base_hi = ISP_RB_BASE_HI12,
+ .reg_size = ISP_RB_SIZE12,
+ },
+ {
+ .name = "RES_RB_STR1",
+ .index = 0,
+ .reg_rptr = ISP_RB_RPTR9,
+ .reg_wptr = ISP_RB_WPTR9,
+ .reg_base_lo = ISP_RB_BASE_LO9,
+ .reg_base_hi = ISP_RB_BASE_HI9,
+ .reg_size = ISP_RB_SIZE9,
+ },
+ {
+ .name = "RES_RB_STR2",
+ .index = 1,
+ .reg_rptr = ISP_RB_RPTR10,
+ .reg_wptr = ISP_RB_WPTR10,
+ .reg_base_lo = ISP_RB_BASE_LO10,
+ .reg_base_hi = ISP_RB_BASE_HI10,
+ .reg_size = ISP_RB_SIZE10,
+ },
+ {
+ .name = "RES_RB_STR3",
+ .index = 2,
+ .reg_rptr = ISP_RB_RPTR11,
+ .reg_wptr = ISP_RB_WPTR11,
+ .reg_base_lo = ISP_RB_BASE_LO11,
+ .reg_base_hi = ISP_RB_BASE_HI11,
+ .reg_size = ISP_RB_SIZE11,
+ },
+};
+
+/* FW log ring buffer configuration */
+static struct isp4if_rb_config isp4if_log_rb_config = {
+ .name = "LOG_RB",
+ .index = 0,
+ .reg_rptr = ISP_LOG_RB_RPTR0,
+ .reg_wptr = ISP_LOG_RB_WPTR0,
+ .reg_base_lo = ISP_LOG_RB_BASE_LO0,
+ .reg_base_hi = ISP_LOG_RB_BASE_HI0,
+ .reg_size = ISP_LOG_RB_SIZE0,
+};
+
+static struct isp4if_gpu_mem_info *isp4if_gpu_mem_alloc(struct isp4_interface *ispif, u32 mem_size)
+{
+ struct isp4if_gpu_mem_info *mem_info;
+ struct device *dev = ispif->dev;
+ int ret;
+
+ if (!mem_size)
+ return NULL;
+
+ mem_info = kzalloc(sizeof(*mem_info), GFP_KERNEL);
+ if (!mem_info)
+ return NULL;
+
+ mem_info->mem_size = mem_size;
+ ret = isp_kernel_buffer_alloc(dev, mem_info->mem_size, &mem_info->mem_handle,
+ &mem_info->gpu_mc_addr, &mem_info->sys_addr);
+ if (ret) {
+ kfree(mem_info);
+ return NULL;
+ }
+
+ return mem_info;
+}
+
+static int isp4if_gpu_mem_free(struct isp4_interface *ispif, struct isp4if_gpu_mem_info *mem_info)
+{
+ struct device *dev = ispif->dev;
+
+ if (!mem_info) {
+ dev_err(dev, "invalid mem_info\n");
+ return -EINVAL;
+ }
+
+ isp_kernel_buffer_free(&mem_info->mem_handle, &mem_info->gpu_mc_addr, &mem_info->sys_addr);
+
+ kfree(mem_info);
+
+ return 0;
+}
+
+static int isp4if_dealloc_fw_gpumem(struct isp4_interface *ispif)
+{
+ int i;
+
+ if (ispif->fw_mem_pool) {
+ isp4if_gpu_mem_free(ispif, ispif->fw_mem_pool);
+ ispif->fw_mem_pool = NULL;
+ }
+
+ if (ispif->fw_cmd_resp_buf) {
+ isp4if_gpu_mem_free(ispif, ispif->fw_cmd_resp_buf);
+ ispif->fw_cmd_resp_buf = NULL;
+ }
+
+ if (ispif->fw_log_buf) {
+ isp4if_gpu_mem_free(ispif, ispif->fw_log_buf);
+ ispif->fw_log_buf = NULL;
+ }
+
+ for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
+ if (ispif->metainfo_buf_pool[i]) {
+ isp4if_gpu_mem_free(ispif, ispif->metainfo_buf_pool[i]);
+ ispif->metainfo_buf_pool[i] = NULL;
+ }
+ }
+
+ return 0;
+}
+
+static int isp4if_alloc_fw_gpumem(struct isp4_interface *ispif)
+{
+ struct device *dev = ispif->dev;
+ unsigned int i;
+
+ ispif->fw_mem_pool = isp4if_gpu_mem_alloc(ispif, FW_MEMORY_POOL_SIZE);
+ if (!ispif->fw_mem_pool)
+ goto error_no_memory;
+
+ ispif->fw_cmd_resp_buf =
+ isp4if_gpu_mem_alloc(ispif, ISP4IF_RB_PMBMAP_MEM_SIZE);
+ if (!ispif->fw_cmd_resp_buf)
+ goto error_no_memory;
+
+ ispif->fw_log_buf =
+ isp4if_gpu_mem_alloc(ispif, ISP4IF_FW_LOG_RINGBUF_SIZE);
+ if (!ispif->fw_log_buf)
+ goto error_no_memory;
+
+ for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
+ ispif->metainfo_buf_pool[i] =
+ isp4if_gpu_mem_alloc(ispif,
+ ISP4IF_META_INFO_BUF_SIZE);
+ if (!ispif->metainfo_buf_pool[i])
+ goto error_no_memory;
+ }
+
+ return 0;
+
+error_no_memory:
+ dev_err(dev, "failed to allocate gpu memory\n");
+ return -ENOMEM;
+}
+
+static u32 isp4if_compute_check_sum(u8 *buf, u32 buf_size)
+{
+ u32 checksum = 0;
+ u8 *surplus_ptr;
+ u32 *buffer;
+ u32 i;
+
+ buffer = (u32 *)buf;
+ for (i = 0; i < buf_size / sizeof(u32); i++)
+ checksum += buffer[i];
+
+ surplus_ptr = (u8 *)&buffer[i];
+ /* add surplus data crc checksum */
+ for (i = 0; i < buf_size % sizeof(u32); i++)
+ checksum += surplus_ptr[i];
+
+ return checksum;
+}
+
+void isp4if_clear_cmdq(struct isp4_interface *ispif)
+{
+ struct isp4if_cmd_element *buf_node = NULL;
+ struct isp4if_cmd_element *tmp_node = NULL;
+
+ guard(mutex)(&ispif->cmdq_mutex);
+
+ list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
+ list_del(&buf_node->list);
+ kfree(buf_node);
+ }
+}
+
+static bool isp4if_is_cmdq_rb_full(struct isp4_interface *ispif, enum isp4if_stream_id cmd_buf_idx)
+{
+ struct isp4if_rb_config *rb_config;
+ u32 rd_ptr, wr_ptr;
+ u32 new_wr_ptr;
+ u32 rreg;
+ u32 wreg;
+ u32 len;
+
+ rb_config = &isp4if_cmd_rb_config[cmd_buf_idx];
+ rreg = rb_config->reg_rptr;
+ wreg = rb_config->reg_wptr;
+ len = rb_config->val_size;
+
+ rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
+ wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
+
+ new_wr_ptr = wr_ptr + sizeof(struct isp4fw_cmd);
+
+ if (wr_ptr >= rd_ptr) {
+ if (new_wr_ptr < len) {
+ return false;
+ } else if (new_wr_ptr == len) {
+ if (rd_ptr == 0)
+ return true;
+
+ return false;
+ }
+
+ new_wr_ptr -= len;
+ if (new_wr_ptr < rd_ptr)
+ return false;
+
+ return true;
+ }
+
+ if (new_wr_ptr < rd_ptr)
+ return false;
+
+ return true;
+}
+
+static struct isp4if_cmd_element *isp4if_append_cmd_2_cmdq(struct isp4_interface *ispif,
+ struct isp4if_cmd_element *cmd_ele)
+{
+ struct isp4if_cmd_element *copy_command = NULL;
+
+ copy_command = kmemdup(cmd_ele, sizeof(*cmd_ele), GFP_KERNEL);
+ if (!copy_command)
+ return NULL;
+
+ guard(mutex)(&ispif->cmdq_mutex);
+
+ list_add_tail(©_command->list, &ispif->cmdq);
+
+ return copy_command;
+}
+
+struct isp4if_cmd_element *isp4if_rm_cmd_from_cmdq(struct isp4_interface *ispif, u32 seq_num,
+ u32 cmd_id)
+{
+ struct isp4if_cmd_element *buf_node = NULL;
+ struct isp4if_cmd_element *tmp_node = NULL;
+
+ guard(mutex)(&ispif->cmdq_mutex);
+
+ list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
+ if (buf_node->seq_num == seq_num &&
+ buf_node->cmd_id == cmd_id) {
+ list_del(&buf_node->list);
+ return buf_node;
+ }
+ }
+
+ return NULL;
+}
+
+static int isp4if_insert_isp_fw_cmd(struct isp4_interface *ispif, enum isp4if_stream_id stream,
+ struct isp4fw_cmd *cmd)
+{
+ struct isp4if_rb_config *rb_config;
+ struct device *dev = ispif->dev;
+ u8 *mem_sys;
+ u32 wr_ptr;
+ u32 rd_ptr;
+ u32 rreg;
+ u32 wreg;
+ u32 len;
+
+ rb_config = &isp4if_cmd_rb_config[stream];
+ rreg = rb_config->reg_rptr;
+ wreg = rb_config->reg_wptr;
+ mem_sys = (u8 *)rb_config->base_sys_addr;
+ len = rb_config->val_size;
+
+ if (isp4if_is_cmdq_rb_full(ispif, stream)) {
+ dev_err(dev, "fail no cmdslot (%d)\n", stream);
+ return -EINVAL;
+ }
+
+ wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
+ rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
+
+ if (rd_ptr > len) {
+ dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
+ stream, rd_ptr, len, wr_ptr);
+ return -EINVAL;
+ }
+
+ if (wr_ptr > len) {
+ dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
+ stream, wr_ptr, len, rd_ptr);
+ return -EINVAL;
+ }
+
+ if (wr_ptr < rd_ptr) {
+ memcpy((mem_sys + wr_ptr),
+ (u8 *)cmd, sizeof(struct isp4fw_cmd));
+ } else {
+ if ((len - wr_ptr) >= (sizeof(struct isp4fw_cmd))) {
+ memcpy((mem_sys + wr_ptr),
+ (u8 *)cmd, sizeof(struct isp4fw_cmd));
+ } else {
+ u32 size;
+ u8 *src;
+
+ src = (u8 *)cmd;
+ size = len - wr_ptr;
+
+ memcpy((mem_sys + wr_ptr), src, size);
+
+ src += size;
+ size = sizeof(struct isp4fw_cmd) - size;
+ memcpy((mem_sys), src, size);
+ }
+ }
+
+ wr_ptr += sizeof(struct isp4fw_cmd);
+ if (wr_ptr >= len)
+ wr_ptr -= len;
+
+ isp4hw_wreg(ispif->mmio, wreg, wr_ptr);
+
+ return 0;
+}
+
+static inline enum isp4if_stream_id isp4if_get_fw_stream(u32 cmd_id)
+{
+ return ISP4IF_STREAM_ID_1;
+}
+
+static int isp4if_send_fw_cmd(struct isp4_interface *ispif, u32 cmd_id, void *package,
+ u32 package_size, wait_queue_head_t *wq, u32 *wq_cond, u32 *seq)
+{
+ enum isp4if_stream_id stream = isp4if_get_fw_stream(cmd_id);
+ struct isp4if_cmd_element command_element = {};
+ struct isp4if_gpu_mem_info *gpu_mem = NULL;
+ struct isp4if_cmd_element *cmd_ele = NULL;
+ struct isp4if_rb_config *rb_config;
+ struct device *dev = ispif->dev;
+ struct isp4fw_cmd cmd = {};
+ u64 package_base = 0;
+ u32 seq_num;
+ u32 rreg;
+ u32 wreg;
+ int ret;
+
+ if (package_size > sizeof(cmd.cmd_param)) {
+ dev_err(dev, "fail pkgsize(%u)>%zu cmd:0x%x,stream %d\n",
+ package_size, sizeof(cmd.cmd_param), cmd_id, stream);
+ return -EINVAL;
+ }
+
+ rb_config = &isp4if_resp_rb_config[stream];
+ rreg = rb_config->reg_rptr;
+ wreg = rb_config->reg_wptr;
+
+ guard(mutex)(&ispif->isp4if_mutex);
+
+ ret = read_poll_timeout(isp4if_is_cmdq_rb_full, ret, !ret, ISP4IF_MAX_SLEEP_TIME * 1000,
+ ISP4IF_MAX_SLEEP_COUNT * ISP4IF_MAX_SLEEP_TIME * 1000, false,
+ ispif, stream);
+
+ if (ret) {
+ u32 rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
+ u32 wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
+
+ dev_err(dev,
+ "failed to get free cmdq slot, stream (%d),rd %u, wr %u\n",
+ stream, rd_ptr, wr_ptr);
+ return -ETIMEDOUT;
+ }
+
+ cmd.cmd_id = cmd_id;
+ switch (stream) {
+ case ISP4IF_STREAM_ID_GLOBAL:
+ cmd.cmd_stream_id = STREAM_ID_INVALID;
+ break;
+ case ISP4IF_STREAM_ID_1:
+ cmd.cmd_stream_id = STREAM_ID_1;
+ break;
+ default:
+ dev_err(dev, "fail bad stream id %d\n", stream);
+ return -EINVAL;
+ }
+
+ if (package && package_size)
+ memcpy(cmd.cmd_param, package, package_size);
+
+ seq_num = ispif->host2fw_seq_num++;
+ cmd.cmd_seq_num = seq_num;
+ cmd.cmd_check_sum =
+ isp4if_compute_check_sum((u8 *)&cmd, sizeof(cmd) - 4);
+
+ if (seq)
+ *seq = seq_num;
+ command_element.seq_num = seq_num;
+ command_element.cmd_id = cmd_id;
+ command_element.mc_addr = package_base;
+ command_element.wq = wq;
+ command_element.wq_cond = wq_cond;
+ command_element.gpu_pkg = gpu_mem;
+ command_element.stream = stream;
+ /*
+ * only append the fw cmd to queue when its response needs to be waited for,
+ * currently there are only two such commands, disable channel and stop stream
+ * which are only sent after close camera
+ */
+ if (wq && wq_cond) {
+ cmd_ele = isp4if_append_cmd_2_cmdq(ispif, &command_element);
+ if (!cmd_ele) {
+ dev_err(dev, "fail for isp_append_cmd_2_cmdq\n");
+ return -ENOMEM;
+ }
+ }
+
+ ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
+ if (ret) {
+ dev_err(dev, "fail for insert_isp_fw_cmd camId (0x%08x)\n", cmd_id);
+ if (cmd_ele) {
+ isp4if_rm_cmd_from_cmdq(ispif, cmd_ele->seq_num, cmd_ele->cmd_id);
+ kfree(cmd_ele);
+ }
+ }
+
+ return ret;
+}
+
+static int isp4if_send_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_info *buf_info)
+{
+ struct isp4fw_cmd_send_buffer cmd = {};
+
+ cmd.buffer_type = BUFFER_TYPE_PREVIEW;
+ cmd.buffer.vmid_space.bit.vmid = 0;
+ cmd.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
+ isp4if_split_addr64(buf_info->planes[0].mc_addr,
+ &cmd.buffer.buf_base_a_lo,
+ &cmd.buffer.buf_base_a_hi);
+ cmd.buffer.buf_size_a = buf_info->planes[0].len;
+
+ isp4if_split_addr64(buf_info->planes[1].mc_addr,
+ &cmd.buffer.buf_base_b_lo,
+ &cmd.buffer.buf_base_b_hi);
+ cmd.buffer.buf_size_b = buf_info->planes[1].len;
+
+ isp4if_split_addr64(buf_info->planes[2].mc_addr,
+ &cmd.buffer.buf_base_c_lo,
+ &cmd.buffer.buf_base_c_hi);
+ cmd.buffer.buf_size_c = buf_info->planes[2].len;
+
+ return isp4if_send_fw_cmd(ispif, CMD_ID_SEND_BUFFER, &cmd,
+ sizeof(cmd), NULL, NULL, NULL);
+}
+
+static void isp4if_init_rb_config(struct isp4_interface *ispif, struct isp4if_rb_config *rb_config)
+{
+ u32 lo;
+ u32 hi;
+
+ isp4if_split_addr64(rb_config->base_mc_addr, &lo, &hi);
+
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rb_config->reg_rptr, 0x0);
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rb_config->reg_wptr, 0x0);
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rb_config->reg_base_lo, lo);
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rb_config->reg_base_hi, hi);
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rb_config->reg_size, rb_config->val_size);
+}
+
+static int isp4if_fw_init(struct isp4_interface *ispif)
+{
+ struct isp4if_rb_config *rb_config;
+ u32 offset;
+ int i;
+
+ /* initialize CMD_RB streams */
+ for (i = 0; i < ISP4IF_STREAM_ID_MAX; i++) {
+ rb_config = (isp4if_cmd_rb_config + i);
+ offset = ispif->aligned_rb_chunk_size *
+ (rb_config->index + ispif->cmd_rb_base_index);
+
+ rb_config->val_size = ISP4IF_FW_CMD_BUF_SIZE;
+ rb_config->base_sys_addr =
+ (u8 *)ispif->fw_cmd_resp_buf->sys_addr + offset;
+ rb_config->base_mc_addr =
+ ispif->fw_cmd_resp_buf->gpu_mc_addr + offset;
+
+ isp4if_init_rb_config(ispif, rb_config);
+ }
+
+ /* initialize RESP_RB streams */
+ for (i = 0; i < ISP4IF_STREAM_ID_MAX; i++) {
+ rb_config = (isp4if_resp_rb_config + i);
+ offset = ispif->aligned_rb_chunk_size *
+ (rb_config->index + ispif->resp_rb_base_index);
+
+ rb_config->val_size = ISP4IF_FW_CMD_BUF_SIZE;
+ rb_config->base_sys_addr =
+ (u8 *)ispif->fw_cmd_resp_buf->sys_addr + offset;
+ rb_config->base_mc_addr =
+ ispif->fw_cmd_resp_buf->gpu_mc_addr + offset;
+
+ isp4if_init_rb_config(ispif, rb_config);
+ }
+
+ /* initialize LOG_RB stream */
+ rb_config = &isp4if_log_rb_config;
+ rb_config->val_size = ISP4IF_FW_LOG_RINGBUF_SIZE;
+ rb_config->base_mc_addr = ispif->fw_log_buf->gpu_mc_addr;
+ rb_config->base_sys_addr = ispif->fw_log_buf->sys_addr;
+
+ isp4if_init_rb_config(ispif, rb_config);
+
+ return 0;
+}
+
+static int isp4if_wait_fw_ready(struct isp4_interface *ispif, u32 isp_status_addr)
+{
+ struct device *dev = ispif->dev;
+ u32 timeout_ms = 100;
+ u32 interval_ms = 1;
+ u32 reg_val;
+
+ /* wait for FW initialize done! */
+ if (!read_poll_timeout(isp4hw_rreg, reg_val, reg_val & ISP_STATUS__CCPU_REPORT_MASK,
+ interval_ms * 1000, timeout_ms * 1000, false,
+ GET_ISP4IF_REG_BASE(ispif), isp_status_addr))
+ return 0;
+
+ dev_err(dev, "ISP CCPU FW boot failed\n");
+
+ return -ETIME;
+}
+
+static void isp4if_enable_ccpu(struct isp4_interface *ispif)
+{
+ u32 reg_val;
+
+ reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET);
+ reg_val &= (~ISP_SOFT_RESET__CCPU_SOFT_RESET_MASK);
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET, reg_val);
+
+ usleep_range(100, 150);
+
+ reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL);
+ reg_val &= (~ISP_CCPU_CNTL__CCPU_HOST_SOFT_RST_MASK);
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL, reg_val);
+}
+
+static void isp4if_disable_ccpu(struct isp4_interface *ispif)
+{
+ u32 reg_val;
+
+ reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL);
+ reg_val |= ISP_CCPU_CNTL__CCPU_HOST_SOFT_RST_MASK;
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL, reg_val);
+
+ usleep_range(100, 150);
+
+ reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET);
+ reg_val |= ISP_SOFT_RESET__CCPU_SOFT_RESET_MASK;
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET, reg_val);
+}
+
+static int isp4if_fw_boot(struct isp4_interface *ispif)
+{
+ struct device *dev = ispif->dev;
+
+ if (ispif->status != ISP4IF_STATUS_PWR_ON) {
+ dev_err(dev, "invalid isp power status %d\n", ispif->status);
+ return -EINVAL;
+ }
+
+ isp4if_disable_ccpu(ispif);
+
+ isp4if_fw_init(ispif);
+
+ /* clear ccpu status */
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_STATUS, 0x0);
+
+ isp4if_enable_ccpu(ispif);
+
+ if (isp4if_wait_fw_ready(ispif, ISP_STATUS)) {
+ isp4if_disable_ccpu(ispif);
+ return -EINVAL;
+ }
+
+ /* enable interrupts */
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SYS_INT0_EN,
+ ISP4IF_FW_RESP_RB_IRQ_EN_MASK);
+
+ ispif->status = ISP4IF_STATUS_FW_RUNNING;
+
+ dev_dbg(dev, "ISP CCPU FW boot success\n");
+
+ return 0;
+}
+
+int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream, void *resp)
+{
+ struct isp4fw_resp *response = resp;
+ struct isp4if_rb_config *rb_config;
+ struct device *dev = ispif->dev;
+ u32 rd_ptr_dbg;
+ u32 wr_ptr_dbg;
+ void *mem_sys;
+ u64 mem_addr;
+ u32 checksum;
+ u32 rd_ptr;
+ u32 wr_ptr;
+ u32 rreg;
+ u32 wreg;
+ u32 len;
+
+ rb_config = &isp4if_resp_rb_config[stream];
+ rreg = rb_config->reg_rptr;
+ wreg = rb_config->reg_wptr;
+ mem_sys = rb_config->base_sys_addr;
+ mem_addr = rb_config->base_mc_addr;
+ len = rb_config->val_size;
+
+ rd_ptr = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), rreg);
+ wr_ptr = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), wreg);
+ rd_ptr_dbg = rd_ptr;
+ wr_ptr_dbg = wr_ptr;
+
+ if (rd_ptr > len) {
+ dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
+ stream, rd_ptr, len, wr_ptr);
+ return -EINVAL;
+ }
+
+ if (wr_ptr > len) {
+ dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
+ stream, wr_ptr, len, rd_ptr);
+ return -EINVAL;
+ }
+
+ if (rd_ptr < wr_ptr) {
+ if ((wr_ptr - rd_ptr) >= (sizeof(struct isp4fw_resp))) {
+ memcpy((u8 *)response, (u8 *)mem_sys + rd_ptr,
+ sizeof(struct isp4fw_resp));
+
+ rd_ptr += sizeof(struct isp4fw_resp);
+ if (rd_ptr < len) {
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rreg, rd_ptr);
+ } else {
+ dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
+ stream, rd_ptr, len, wr_ptr);
+ return -EINVAL;
+ }
+
+ } else {
+ dev_err(dev, "sth wrong with wptr and rptr\n");
+ return -EINVAL;
+ }
+ } else if (rd_ptr > wr_ptr) {
+ u32 size;
+ u8 *dst;
+
+ dst = (u8 *)response;
+
+ size = len - rd_ptr;
+ if (size > sizeof(struct isp4fw_resp)) {
+ mem_addr += rd_ptr;
+ memcpy((u8 *)response,
+ (u8 *)(mem_sys) + rd_ptr,
+ sizeof(struct isp4fw_resp));
+ rd_ptr += sizeof(struct isp4fw_resp);
+ if (rd_ptr < len) {
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rreg, rd_ptr);
+ } else {
+ dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
+ stream, rd_ptr, len, wr_ptr);
+ return -EINVAL;
+ }
+
+ } else {
+ if ((size + wr_ptr) < (sizeof(struct isp4fw_resp))) {
+ dev_err(dev, "sth wrong with wptr and rptr1\n");
+ return -EINVAL;
+ }
+
+ memcpy(dst, (u8 *)(mem_sys) + rd_ptr, size);
+
+ dst += size;
+ size = sizeof(struct isp4fw_resp) - size;
+ if (size)
+ memcpy(dst, (u8 *)(mem_sys), size);
+ rd_ptr = size;
+ if (rd_ptr < len) {
+ isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
+ rreg, rd_ptr);
+ } else {
+ dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
+ stream, rd_ptr, len, wr_ptr);
+ return -EINVAL;
+ }
+ }
+ } else {
+ return -ETIME;
+ }
+
+ checksum = isp4if_compute_check_sum((u8 *)response, sizeof(struct isp4fw_resp) - 4);
+
+ if (checksum != response->resp_check_sum) {
+ dev_err(dev, "resp checksum 0x%x,should 0x%x,rptr %u,wptr %u\n",
+ checksum, response->resp_check_sum, rd_ptr_dbg, wr_ptr_dbg);
+
+ dev_err(dev, "(%u), seqNo %u, resp_id (0x%x)\n", stream,
+ response->resp_seq_num,
+ response->resp_id);
+
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int isp4if_send_command(struct isp4_interface *ispif, u32 cmd_id, void *package, u32 package_size)
+{
+ return isp4if_send_fw_cmd(ispif, cmd_id, package, package_size, NULL, NULL, NULL);
+}
+
+int isp4if_send_command_sync(struct isp4_interface *ispif, u32 cmd_id, void *package,
+ u32 package_size, u32 timeout)
+{
+ struct device *dev = ispif->dev;
+ DECLARE_WAIT_QUEUE_HEAD(cmd_wq);
+ u32 wq_cond = 0;
+ int ret;
+ u32 seq;
+
+ ret = isp4if_send_fw_cmd(ispif, cmd_id, package, package_size, &cmd_wq, &wq_cond, &seq);
+
+ if (ret) {
+ dev_err(dev, "send fw cmd fail %d\n", ret);
+ return ret;
+ }
+
+ ret = wait_event_timeout(cmd_wq, wq_cond != 0, msecs_to_jiffies(timeout));
+ if (ret == 0) {
+ struct isp4if_cmd_element *ele;
+
+ ele = isp4if_rm_cmd_from_cmdq(ispif, seq, cmd_id);
+ kfree(ele);
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
+void isp4if_clear_bufq(struct isp4_interface *ispif)
+{
+ struct isp4if_img_buf_node *buf_node = NULL;
+ struct isp4if_img_buf_node *tmp_node = NULL;
+
+ guard(mutex)(&ispif->bufq_mutex);
+
+ list_for_each_entry_safe(buf_node, tmp_node, &ispif->bufq, node) {
+ list_del(&buf_node->node);
+ kfree(buf_node);
+ }
+}
+
+void isp4if_dealloc_buffer_node(struct isp4if_img_buf_node *buf_node)
+{
+ kfree(buf_node);
+}
+
+struct isp4if_img_buf_node *isp4if_alloc_buffer_node(struct isp4if_img_buf_info *buf_info)
+{
+ struct isp4if_img_buf_node *node = NULL;
+
+ node = kmalloc(sizeof(*node), GFP_KERNEL);
+ if (node)
+ node->buf_info = *buf_info;
+
+ return node;
+};
+
+struct isp4if_img_buf_node *isp4if_dequeue_buffer(struct isp4_interface *ispif)
+{
+ struct isp4if_img_buf_node *buf_node = NULL;
+
+ guard(mutex)(&ispif->bufq_mutex);
+
+ buf_node = list_first_entry_or_null(&ispif->bufq, typeof(*buf_node), node);
+ if (buf_node)
+ list_del(&buf_node->node);
+
+ return buf_node;
+}
+
+int isp4if_queue_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_node *buf_node)
+{
+ int ret;
+
+ ret = isp4if_send_buffer(ispif, &buf_node->buf_info);
+ if (ret)
+ return ret;
+
+ guard(mutex)(&ispif->bufq_mutex);
+
+ list_add_tail(&buf_node->node, &ispif->bufq);
+
+ return 0;
+}
+
+int isp4if_stop(struct isp4_interface *ispif)
+{
+ isp4if_disable_ccpu(ispif);
+
+ isp4if_dealloc_fw_gpumem(ispif);
+
+ return 0;
+}
+
+int isp4if_start(struct isp4_interface *ispif)
+{
+ int ret;
+
+ ret = isp4if_alloc_fw_gpumem(ispif);
+ if (ret)
+ return -ENOMEM;
+
+ ret = isp4if_fw_boot(ispif);
+ if (ret)
+ goto failed_fw_boot;
+
+ return 0;
+
+failed_fw_boot:
+ isp4if_dealloc_fw_gpumem(ispif);
+ return ret;
+}
+
+int isp4if_deinit(struct isp4_interface *ispif)
+{
+ isp4if_clear_cmdq(ispif);
+
+ isp4if_clear_bufq(ispif);
+
+ mutex_destroy(&ispif->cmdq_mutex);
+ mutex_destroy(&ispif->bufq_mutex);
+ mutex_destroy(&ispif->isp4if_mutex);
+
+ return 0;
+}
+
+int isp4if_init(struct isp4_interface *ispif, struct device *dev, void __iomem *isp_mmip)
+{
+ ispif->dev = dev;
+ ispif->mmio = isp_mmip;
+
+ ispif->cmd_rb_base_index = 0;
+ ispif->resp_rb_base_index = ISP4IF_RESP_CHAN_TO_RB_OFFSET - 1;
+ ispif->aligned_rb_chunk_size = ISP4IF_RB_PMBMAP_MEM_CHUNK & 0xffffffc0;
+
+ mutex_init(&ispif->cmdq_mutex); /* used for cmdq access */
+ mutex_init(&ispif->bufq_mutex); /* used for bufq access */
+ mutex_init(&ispif->isp4if_mutex); /* used for commands sent to ispfw */
+
+ INIT_LIST_HEAD(&ispif->cmdq);
+ INIT_LIST_HEAD(&ispif->bufq);
+
+ return 0;
+}
diff --git a/drivers/media/platform/amd/isp4/isp4_interface.h b/drivers/media/platform/amd/isp4/isp4_interface.h
new file mode 100644
index 000000000000..5b94985cdc44
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_interface.h
@@ -0,0 +1,149 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#ifndef _ISP4_INTERFACE_
+#define _ISP4_INTERFACE_
+
+#define ISP4IF_RB_MAX (25)
+#define ISP4IF_RESP_CHAN_TO_RB_OFFSET (9)
+#define ISP4IF_RB_PMBMAP_MEM_SIZE (16 * 1024 * 1024 - 1)
+#define ISP4IF_RB_PMBMAP_MEM_CHUNK (ISP4IF_RB_PMBMAP_MEM_SIZE \
+ / (ISP4IF_RB_MAX - 1))
+#define ISP4IF_HOST2FW_COMMAND_SIZE (sizeof(struct isp4fw_cmd))
+#define ISP4IF_FW_CMD_BUF_COUNT 4
+#define ISP4IF_FW_RESP_BUF_COUNT 4
+#define ISP4IF_MAX_NUM_HOST2FW_COMMAND (40)
+#define ISP4IF_FW_CMD_BUF_SIZE (ISP4IF_MAX_NUM_HOST2FW_COMMAND \
+ * ISP4IF_HOST2FW_COMMAND_SIZE)
+#define ISP4IF_MAX_SLEEP_COUNT (10)
+#define ISP4IF_MAX_SLEEP_TIME (33)
+
+#define ISP4IF_META_INFO_BUF_SIZE ALIGN(sizeof(struct isp4fw_meta_info), 0x8000)
+#define ISP4IF_MAX_STREAM_BUF_COUNT 8
+
+#define ISP4IF_FW_LOG_RINGBUF_SIZE (2 * 1024 * 1024)
+
+#define ISP4IF_MAX_CMD_RESPONSE_BUF_SIZE (4 * 1024)
+
+#define GET_ISP4IF_REG_BASE(ispif) (((ispif))->mmio)
+
+enum isp4if_stream_id {
+ ISP4IF_STREAM_ID_GLOBAL = 0,
+ ISP4IF_STREAM_ID_1 = 1,
+ ISP4IF_STREAM_ID_MAX = 4
+};
+
+enum isp4if_status {
+ ISP4IF_STATUS_PWR_OFF,
+ ISP4IF_STATUS_PWR_ON,
+ ISP4IF_STATUS_FW_RUNNING,
+ ISP4IF_FSM_STATUS_MAX
+};
+
+struct isp4if_gpu_mem_info {
+ u32 mem_domain;
+ u64 mem_size;
+ u32 mem_align;
+ u64 gpu_mc_addr;
+ void *sys_addr;
+ void *mem_handle;
+};
+
+struct isp4if_img_buf_info {
+ struct {
+ void *sys_addr;
+ u64 mc_addr;
+ u32 len;
+ } planes[3];
+};
+
+struct isp4if_img_buf_node {
+ struct list_head node;
+ struct isp4if_img_buf_info buf_info;
+};
+
+struct isp4if_cmd_element {
+ struct list_head list;
+ u32 seq_num;
+ u32 cmd_id;
+ enum isp4if_stream_id stream;
+ u64 mc_addr;
+ wait_queue_head_t *wq;
+ u32 *wq_cond;
+ struct isp4if_gpu_mem_info *gpu_pkg;
+};
+
+struct isp4_interface {
+ struct device *dev;
+ void __iomem *mmio;
+
+ struct mutex cmdq_mutex; /* used for cmdq access */
+ struct mutex bufq_mutex; /* used for bufq access */
+ struct mutex isp4if_mutex; /* used to send fw cmd and read fw log */
+
+ struct list_head cmdq; /* commands sent to fw */
+ struct list_head bufq; /* buffers sent to fw */
+
+ enum isp4if_status status;
+ u32 host2fw_seq_num;
+
+ /* FW ring buffer configs */
+ u32 cmd_rb_base_index;
+ u32 resp_rb_base_index;
+ u32 aligned_rb_chunk_size;
+
+ /* ISP fw buffers */
+ struct isp4if_gpu_mem_info *fw_log_buf;
+ struct isp4if_gpu_mem_info *fw_cmd_resp_buf;
+ struct isp4if_gpu_mem_info *fw_mem_pool;
+ struct isp4if_gpu_mem_info *
+ metainfo_buf_pool[ISP4IF_MAX_STREAM_BUF_COUNT];
+};
+
+static inline void isp4if_split_addr64(u64 addr, u32 *lo, u32 *hi)
+{
+ if (lo)
+ *lo = addr & 0xffffffff;
+ if (hi)
+ *hi = addr >> 32;
+}
+
+static inline u64 isp4if_join_addr64(u32 lo, u32 hi)
+{
+ return (((u64)hi) << 32) | (u64)lo;
+}
+
+int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream, void *response);
+
+int isp4if_send_command(struct isp4_interface *ispif, u32 cmd_id, void *package,
+ u32 package_size);
+
+int isp4if_send_command_sync(struct isp4_interface *ispif, u32 cmd_id, void *package,
+ u32 package_size, u32 timeout);
+
+struct isp4if_cmd_element *isp4if_rm_cmd_from_cmdq(struct isp4_interface *ispif, u32 seq_num,
+ u32 cmd_id);
+
+void isp4if_clear_cmdq(struct isp4_interface *ispif);
+
+void isp4if_clear_bufq(struct isp4_interface *ispif);
+
+void isp4if_dealloc_buffer_node(struct isp4if_img_buf_node *buf_node);
+
+struct isp4if_img_buf_node *isp4if_alloc_buffer_node(struct isp4if_img_buf_info *buf_info);
+
+struct isp4if_img_buf_node *isp4if_dequeue_buffer(struct isp4_interface *ispif);
+
+int isp4if_queue_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_node *buf_node);
+
+int isp4if_stop(struct isp4_interface *ispif);
+
+int isp4if_start(struct isp4_interface *ispif);
+
+int isp4if_deinit(struct isp4_interface *ispif);
+
+int isp4if_init(struct isp4_interface *ispif, struct device *dev, void __iomem *isp_mmip);
+
+#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
` (2 preceding siblings ...)
2025-09-11 10:08 ` [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface Bin Du
@ 2025-09-11 10:08 ` Bin Du
2025-09-23 7:23 ` Sultan Alsawaf
2025-09-11 10:08 ` [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers " Bin Du
` (3 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du, Svetoslav Stoilov,
Alexey Zagorodnikov
Isp4 sub-device is implementing v4l2 sub-device interface. It has one
capture video node, and supports only preview stream. It manages firmware
states, stream configuration. Add interrupt handling and notification for
isp firmware to isp-subdevice.
Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Bin Du <Bin.Du@amd.com>
Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
---
MAINTAINERS | 2 +
drivers/media/platform/amd/isp4/Makefile | 3 +-
drivers/media/platform/amd/isp4/isp4.c | 120 +-
drivers/media/platform/amd/isp4/isp4.h | 8 +-
drivers/media/platform/amd/isp4/isp4_subdev.c | 1095 +++++++++++++++++
drivers/media/platform/amd/isp4/isp4_subdev.h | 131 ++
6 files changed, 1346 insertions(+), 13 deletions(-)
create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.h
diff --git a/MAINTAINERS b/MAINTAINERS
index cccae369c876..48ffc8bbdcee 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1149,6 +1149,8 @@ F: drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
F: drivers/media/platform/amd/isp4/isp4_hw_reg.h
F: drivers/media/platform/amd/isp4/isp4_interface.c
F: drivers/media/platform/amd/isp4/isp4_interface.h
+F: drivers/media/platform/amd/isp4/isp4_subdev.c
+F: drivers/media/platform/amd/isp4/isp4_subdev.h
AMD KFD
M: Felix Kuehling <Felix.Kuehling@amd.com>
diff --git a/drivers/media/platform/amd/isp4/Makefile b/drivers/media/platform/amd/isp4/Makefile
index 327ed1157076..905788bc6a1e 100644
--- a/drivers/media/platform/amd/isp4/Makefile
+++ b/drivers/media/platform/amd/isp4/Makefile
@@ -3,5 +3,6 @@
# Copyright (C) 2025 Advanced Micro Devices, Inc.
obj-$(CONFIG_AMD_ISP4) += amd_capture.o
-amd_capture-objs := isp4.o \
+amd_capture-objs := isp4_subdev.o \
isp4_interface.o \
+ isp4.o \
diff --git a/drivers/media/platform/amd/isp4/isp4.c b/drivers/media/platform/amd/isp4/isp4.c
index 6ff3ded4310a..8cec27228710 100644
--- a/drivers/media/platform/amd/isp4/isp4.c
+++ b/drivers/media/platform/amd/isp4/isp4.c
@@ -5,13 +5,19 @@
#include <linux/pm_runtime.h>
#include <linux/vmalloc.h>
+
+#include <media/v4l2-fwnode.h>
#include <media/v4l2-ioctl.h>
#include "isp4.h"
-
-#define VIDEO_BUF_NUM 5
+#include "isp4_hw_reg.h"
#define ISP4_DRV_NAME "amd_isp_capture"
+#define ISP4_FW_RESP_RB_IRQ_STATUS_MASK \
+ (ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK | \
+ ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK | \
+ ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK | \
+ ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK)
/* interrupt num */
static const u32 isp4_ringbuf_interrupt_num[] = {
@@ -21,19 +27,95 @@ static const u32 isp4_ringbuf_interrupt_num[] = {
4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
};
-#define to_isp4_device(dev) \
- ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
+#define to_isp4_device(dev) container_of(dev, struct isp4_device, v4l2_dev)
+
+static void isp4_wake_up_resp_thread(struct isp4_subdev *isp, u32 index)
+{
+ if (isp && index < ISP4SD_MAX_FW_RESP_STREAM_NUM) {
+ struct isp4sd_thread_handler *thread_ctx =
+ &isp->fw_resp_thread[index];
+
+ thread_ctx->wq_cond = 1;
+ wake_up_interruptible(&thread_ctx->waitq);
+ }
+}
+
+static void isp4_resp_interrupt_notify(struct isp4_subdev *isp, u32 intr_status)
+{
+ bool wake = (isp->ispif.status == ISP4IF_STATUS_FW_RUNNING);
+
+ u32 intr_ack = 0;
+
+ /* global response */
+ if (intr_status &
+ ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK) {
+ if (wake)
+ isp4_wake_up_resp_thread(isp, 0);
+
+ intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT12_ACK_MASK;
+ }
+
+ /* stream 1 response */
+ if (intr_status &
+ ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK) {
+ if (wake)
+ isp4_wake_up_resp_thread(isp, 1);
+
+ intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT9_ACK_MASK;
+ }
+
+ /* stream 2 response */
+ if (intr_status &
+ ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK) {
+ if (wake)
+ isp4_wake_up_resp_thread(isp, 2);
+
+ intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT10_ACK_MASK;
+ }
+
+ /* stream 3 response */
+ if (intr_status &
+ ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK) {
+ if (wake)
+ isp4_wake_up_resp_thread(isp, 3);
+
+ intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT11_ACK_MASK;
+ }
+
+ /* clear ISP_SYS interrupts */
+ isp4hw_wreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_ACK, intr_ack);
+}
static irqreturn_t isp4_irq_handler(int irq, void *arg)
{
+ struct isp4_device *isp_dev = dev_get_drvdata(arg);
+ struct isp4_subdev *isp = NULL;
+ u32 isp_sys_irq_status = 0x0;
+ u32 r1;
+
+ if (!isp_dev)
+ goto error_drv_data;
+
+ isp = &isp_dev->isp_sdev;
+ /* check ISP_SYS interrupts status */
+ r1 = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_STATUS);
+
+ isp_sys_irq_status = r1 & ISP4_FW_RESP_RB_IRQ_STATUS_MASK;
+
+ isp4_resp_interrupt_notify(isp, isp_sys_irq_status);
+
+error_drv_data:
return IRQ_HANDLED;
}
static int isp4_capture_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
+ struct isp4_subdev *isp_sdev;
struct isp4_device *isp_dev;
- int i, irq, ret;
+ size_t i;
+ int irq;
+ int ret;
isp_dev = devm_kzalloc(&pdev->dev, sizeof(*isp_dev), GFP_KERNEL);
if (!isp_dev)
@@ -42,6 +124,12 @@ static int isp4_capture_probe(struct platform_device *pdev)
isp_dev->pdev = pdev;
dev->init_name = ISP4_DRV_NAME;
+ isp_sdev = &isp_dev->isp_sdev;
+ isp_sdev->mmio = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(isp_sdev->mmio))
+ return dev_err_probe(dev, PTR_ERR(isp_sdev->mmio),
+ "isp ioremap fail\n");
+
for (i = 0; i < ARRAY_SIZE(isp4_ringbuf_interrupt_num); i++) {
irq = platform_get_irq(pdev, isp4_ringbuf_interrupt_num[i]);
if (irq < 0)
@@ -55,6 +143,8 @@ static int isp4_capture_probe(struct platform_device *pdev)
irq);
}
+ isp_dev->pltf_data = pdev->dev.platform_data;
+
/* Link the media device within the v4l2_device */
isp_dev->v4l2_dev.mdev = &isp_dev->mdev;
@@ -66,6 +156,8 @@ static int isp4_capture_probe(struct platform_device *pdev)
isp_dev->mdev.dev = &pdev->dev;
media_device_init(&isp_dev->mdev);
+ pm_runtime_set_suspended(dev);
+ pm_runtime_enable(dev);
/* register v4l2 device */
snprintf(isp_dev->v4l2_dev.name, sizeof(isp_dev->v4l2_dev.name),
"AMD-V4L2-ROOT");
@@ -74,19 +166,24 @@ static int isp4_capture_probe(struct platform_device *pdev)
return dev_err_probe(dev, ret,
"fail register v4l2 device\n");
+ ret = isp4sd_init(&isp_dev->isp_sdev, &isp_dev->v4l2_dev);
+ if (ret) {
+ dev_err(dev, "fail init isp4 sub dev %d\n", ret);
+ goto err_unreg_v4l2;
+ }
+
ret = media_device_register(&isp_dev->mdev);
if (ret) {
dev_err(dev, "fail to register media device %d\n", ret);
- goto err_unreg_v4l2;
+ goto err_isp4_deinit;
}
platform_set_drvdata(pdev, isp_dev);
- pm_runtime_set_suspended(dev);
- pm_runtime_enable(dev);
-
return 0;
+err_isp4_deinit:
+ isp4sd_deinit(&isp_dev->isp_sdev);
err_unreg_v4l2:
v4l2_device_unregister(&isp_dev->v4l2_dev);
@@ -97,8 +194,13 @@ static void isp4_capture_remove(struct platform_device *pdev)
{
struct isp4_device *isp_dev = platform_get_drvdata(pdev);
+ v4l2_device_unregister_subdev(&isp_dev->isp_sdev.sdev);
+
media_device_unregister(&isp_dev->mdev);
+ media_entity_cleanup(&isp_dev->isp_sdev.sdev.entity);
v4l2_device_unregister(&isp_dev->v4l2_dev);
+
+ isp4sd_deinit(&isp_dev->isp_sdev);
}
static struct platform_driver isp4_capture_drv = {
diff --git a/drivers/media/platform/amd/isp4/isp4.h b/drivers/media/platform/amd/isp4/isp4.h
index 8535f662ab49..00ac11ed8fb0 100644
--- a/drivers/media/platform/amd/isp4/isp4.h
+++ b/drivers/media/platform/amd/isp4/isp4.h
@@ -6,19 +6,21 @@
#ifndef _ISP4_H_
#define _ISP4_H_
+#include <drm/amd/isp.h>
#include <linux/mutex.h>
-#include <media/v4l2-device.h>
-#include <media/videobuf2-memops.h>
-#include <media/videobuf2-vmalloc.h>
+#include "isp4_subdev.h"
#define ISP4_GET_ISP_REG_BASE(isp4sd) (((isp4sd))->mmio)
struct isp4_device {
struct v4l2_device v4l2_dev;
+ struct isp4_subdev isp_sdev;
struct media_device mdev;
+ struct isp_platform_data *pltf_data;
struct platform_device *pdev;
struct notifier_block i2c_nb;
+ struct v4l2_async_notifier notifier;
};
#endif /* _ISP4_H_ */
diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.c b/drivers/media/platform/amd/isp4/isp4_subdev.c
new file mode 100644
index 000000000000..a9cb14de04ca
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
@@ -0,0 +1,1095 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#include <linux/mutex.h>
+#include <linux/pm_domain.h>
+#include <linux/pm_runtime.h>
+
+#include "isp4_fw_cmd_resp.h"
+#include "isp4_interface.h"
+#include "isp4_subdev.h"
+#include <linux/units.h>
+
+#define ISP4SD_MAX_CMD_RESP_BUF_SIZE (4 * 1024)
+#define ISP4SD_MIN_BUF_CNT_BEF_START_STREAM 4
+
+#define ISP4SD_PERFORMANCE_STATE_LOW 0
+#define ISP4SD_PERFORMANCE_STATE_HIGH 1
+
+#define ISP4SD_FW_CMD_TIMEOUT_IN_MS 500
+#define ISP4SD_WAIT_RESP_IRQ_TIMEOUT 5 /* ms */
+/* align 32KB */
+#define ISP4SD_META_BUF_SIZE ALIGN(sizeof(struct isp4fw_meta_info), 0x8000)
+
+#define to_isp4_subdev(v4l2_sdev) \
+ container_of(v4l2_sdev, struct isp4_subdev, sdev)
+
+static const char *isp4sd_entity_name = "amd isp4";
+
+static void isp4sd_module_enable(struct isp4_subdev *isp_subdev, bool enable)
+{
+ if (isp_subdev->enable_gpio) {
+ gpiod_set_value(isp_subdev->enable_gpio, enable ? 1 : 0);
+ dev_dbg(isp_subdev->dev, "%s isp_subdev module\n",
+ enable ? "enable" : "disable");
+ }
+}
+
+static int isp4sd_setup_fw_mem_pool(struct isp4_subdev *isp_subdev)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4fw_cmd_send_buffer buf_type = {};
+ struct device *dev = isp_subdev->dev;
+ int ret;
+
+ if (!ispif->fw_mem_pool) {
+ dev_err(dev, "fail to alloc mem pool\n");
+ return -ENOMEM;
+ }
+
+ buf_type.buffer_type = BUFFER_TYPE_MEM_POOL;
+ buf_type.buffer.buf_tags = 0;
+ buf_type.buffer.vmid_space.bit.vmid = 0;
+ buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
+ isp4if_split_addr64(ispif->fw_mem_pool->gpu_mc_addr,
+ &buf_type.buffer.buf_base_a_lo,
+ &buf_type.buffer.buf_base_a_hi);
+ buf_type.buffer.buf_size_a = (u32)ispif->fw_mem_pool->mem_size;
+
+ ret = isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
+ &buf_type, sizeof(buf_type));
+ if (ret) {
+ dev_err(dev, "send fw mem pool 0x%llx(%u) fail %d\n",
+ ispif->fw_mem_pool->gpu_mc_addr,
+ buf_type.buffer.buf_size_a,
+ ret);
+ return ret;
+ }
+
+ dev_dbg(dev, "send fw mem pool 0x%llx(%u) suc\n",
+ ispif->fw_mem_pool->gpu_mc_addr,
+ buf_type.buffer.buf_size_a);
+
+ return 0;
+};
+
+static int isp4sd_set_stream_path(struct isp4_subdev *isp_subdev)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4fw_cmd_set_stream_cfg cmd = {};
+ struct device *dev = isp_subdev->dev;
+
+ cmd.stream_cfg.mipi_pipe_path_cfg.isp4fw_sensor_id = SENSOR_ID_ON_MIPI0;
+ cmd.stream_cfg.mipi_pipe_path_cfg.b_enable = true;
+ cmd.stream_cfg.isp_pipe_path_cfg.isp_pipe_id = MIPI0_ISP_PIPELINE_ID;
+
+ cmd.stream_cfg.b_enable_tnr = true;
+ dev_dbg(dev, "isp4fw_sensor_id %d, pipeId 0x%x EnableTnr %u\n",
+ cmd.stream_cfg.mipi_pipe_path_cfg.isp4fw_sensor_id,
+ cmd.stream_cfg.isp_pipe_path_cfg.isp_pipe_id,
+ cmd.stream_cfg.b_enable_tnr);
+
+ return isp4if_send_command(ispif, CMD_ID_SET_STREAM_CONFIG,
+ &cmd, sizeof(cmd));
+}
+
+static int isp4sd_send_meta_buf(struct isp4_subdev *isp_subdev)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4fw_cmd_send_buffer buf_type = {};
+ struct isp4sd_sensor_info *sensor_info;
+ struct device *dev = isp_subdev->dev;
+ u32 i;
+
+ sensor_info = &isp_subdev->sensor_info;
+ for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
+ int ret;
+
+ if (!sensor_info->meta_info_buf[i]) {
+ dev_err(dev, "fail for no meta info buf(%u)\n", i);
+ return -ENOMEM;
+ }
+ buf_type.buffer_type = BUFFER_TYPE_META_INFO;
+ buf_type.buffer.buf_tags = 0;
+ buf_type.buffer.vmid_space.bit.vmid = 0;
+ buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
+ isp4if_split_addr64(sensor_info->meta_info_buf[i]->gpu_mc_addr,
+ &buf_type.buffer.buf_base_a_lo,
+ &buf_type.buffer.buf_base_a_hi);
+ buf_type.buffer.buf_size_a =
+ (u32)sensor_info->meta_info_buf[i]->mem_size;
+ ret = isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
+ &buf_type,
+ sizeof(buf_type));
+ if (ret) {
+ dev_err(dev, "send meta info(%u) fail\n", i);
+ return ret;
+ }
+ }
+
+ dev_dbg(dev, "send meta info suc\n");
+ return 0;
+}
+
+static bool isp4sd_get_str_out_prop(struct isp4_subdev *isp_subdev,
+ struct isp4fw_image_prop *out_prop,
+ struct v4l2_subdev_state *state, u32 pad)
+{
+ struct v4l2_mbus_framefmt *format = NULL;
+ struct device *dev = isp_subdev->dev;
+ bool ret;
+
+ format = v4l2_subdev_state_get_format(state, pad, 0);
+ if (!format) {
+ dev_err(dev, "fail get subdev state format\n");
+ return false;
+ }
+
+ switch (format->code) {
+ case MEDIA_BUS_FMT_YUYV8_1_5X8:
+ out_prop->image_format = IMAGE_FORMAT_NV12;
+ out_prop->width = format->width;
+ out_prop->height = format->height;
+ out_prop->luma_pitch = format->width;
+ out_prop->chroma_pitch = out_prop->width;
+ ret = true;
+ break;
+ case MEDIA_BUS_FMT_YUYV8_1X16:
+ out_prop->image_format = IMAGE_FORMAT_YUV422INTERLEAVED;
+ out_prop->width = format->width;
+ out_prop->height = format->height;
+ out_prop->luma_pitch = format->width * 2;
+ out_prop->chroma_pitch = 0;
+ ret = true;
+ break;
+ default:
+ dev_err(dev, "fail for bad image format:0x%x\n",
+ format->code);
+ ret = false;
+ break;
+ }
+
+ if (!out_prop->width || !out_prop->height)
+ ret = false;
+ return ret;
+}
+
+static int isp4sd_kickoff_stream(struct isp4_subdev *isp_subdev, u32 w, u32 h)
+{
+ struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct device *dev = isp_subdev->dev;
+
+ if (sensor_info->status == ISP4SD_START_STATUS_STARTED) {
+ return 0;
+ } else if (sensor_info->status == ISP4SD_START_STATUS_START_FAIL) {
+ dev_err(dev, "fail for previous start fail\n");
+ return -EINVAL;
+ }
+
+ dev_dbg(dev, "w:%u,h:%u\n", w, h);
+
+ sensor_info->status = ISP4SD_START_STATUS_START_FAIL;
+
+ if (isp4sd_send_meta_buf(isp_subdev)) {
+ dev_err(dev, "fail to send meta buf\n");
+ return -EINVAL;
+ };
+
+ sensor_info->status = ISP4SD_START_STATUS_NOT_START;
+
+ if (!sensor_info->start_stream_cmd_sent &&
+ sensor_info->buf_sent_cnt >=
+ ISP4SD_MIN_BUF_CNT_BEF_START_STREAM) {
+ int ret = isp4if_send_command(ispif, CMD_ID_START_STREAM,
+ NULL, 0);
+ if (ret) {
+ dev_err(dev, "fail to start stream\n");
+ return ret;
+ }
+
+ sensor_info->start_stream_cmd_sent = true;
+ } else {
+ dev_dbg(dev,
+ "no send START_STREAM, start_sent %u, buf_sent %u\n",
+ sensor_info->start_stream_cmd_sent,
+ sensor_info->buf_sent_cnt);
+ }
+
+ return 0;
+}
+
+static int isp4sd_setup_output(struct isp4_subdev *isp_subdev,
+ struct v4l2_subdev_state *state, u32 pad)
+{
+ struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
+ struct isp4sd_output_info *output_info =
+ &isp_subdev->sensor_info.output_info;
+ struct isp4fw_cmd_set_out_ch_prop cmd_ch_prop = {};
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4fw_cmd_enable_out_ch cmd_ch_en = {};
+ struct device *dev = isp_subdev->dev;
+ struct isp4fw_image_prop *out_prop;
+ int ret;
+
+ if (output_info->start_status == ISP4SD_START_STATUS_STARTED)
+ return 0;
+
+ if (output_info->start_status == ISP4SD_START_STATUS_START_FAIL) {
+ dev_err(dev, "fail for previous start fail\n");
+ return -EINVAL;
+ }
+
+ out_prop = &cmd_ch_prop.image_prop;
+ cmd_ch_prop.ch = ISP_PIPE_OUT_CH_PREVIEW;
+ cmd_ch_en.ch = ISP_PIPE_OUT_CH_PREVIEW;
+ cmd_ch_en.is_enable = true;
+
+ if (!isp4sd_get_str_out_prop(isp_subdev, out_prop, state, pad)) {
+ dev_err(dev, "fail to get out prop\n");
+ return -EINVAL;
+ }
+
+ dev_dbg(dev, "channel: w:h=%u:%u,lp:%u,cp%u\n",
+ cmd_ch_prop.image_prop.width, cmd_ch_prop.image_prop.height,
+ cmd_ch_prop.image_prop.luma_pitch,
+ cmd_ch_prop.image_prop.chroma_pitch);
+
+ ret = isp4if_send_command(ispif, CMD_ID_SET_OUT_CHAN_PROP,
+ &cmd_ch_prop,
+ sizeof(cmd_ch_prop));
+ if (ret) {
+ output_info->start_status = ISP4SD_START_STATUS_START_FAIL;
+ dev_err(dev, "fail to set out prop\n");
+ return ret;
+ };
+
+ ret = isp4if_send_command(ispif, CMD_ID_ENABLE_OUT_CHAN,
+ &cmd_ch_en, sizeof(cmd_ch_en));
+
+ if (ret) {
+ output_info->start_status = ISP4SD_START_STATUS_START_FAIL;
+ dev_err(dev, "fail to enable channel\n");
+ return ret;
+ }
+
+ if (!sensor_info->start_stream_cmd_sent) {
+ ret = isp4sd_kickoff_stream(isp_subdev, out_prop->width,
+ out_prop->height);
+ if (ret) {
+ dev_err(dev, "kickoff stream fail %d\n", ret);
+ return ret;
+ }
+ /*
+ * sensor_info->start_stream_cmd_sent will be set to true
+ * 1. in isp4sd_kickoff_stream, if app first send buffer then
+ * start stream
+ * 2. in isp_set_stream_buf, if app first start stream, then
+ * send buffer
+ * because ISP FW has the requirement, host needs to send buffer
+ * before send start stream cmd
+ */
+ if (sensor_info->start_stream_cmd_sent) {
+ sensor_info->status = ISP4SD_START_STATUS_STARTED;
+ output_info->start_status = ISP4SD_START_STATUS_STARTED;
+ dev_dbg(dev, "kickoff stream suc,start cmd sent\n");
+ }
+ } else {
+ dev_dbg(dev, "stream running, no need kickoff\n");
+ output_info->start_status = ISP4SD_START_STATUS_STARTED;
+ }
+
+ dev_dbg(dev, "setup output suc\n");
+ return 0;
+}
+
+static int isp4sd_init_meta_buf(struct isp4_subdev *isp_subdev)
+{
+ struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct device *dev = isp_subdev->dev;
+ u32 i;
+
+ for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
+ if (!sensor_info->meta_info_buf[i]) {
+ sensor_info->meta_info_buf[i] = ispif->metainfo_buf_pool[i];
+ if (!sensor_info->meta_info_buf[i]) {
+ dev_err(dev, "invalid %u meta_info_buf fail\n", i);
+ return -ENOMEM;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int isp4sd_init_stream(struct isp4_subdev *isp_subdev)
+{
+ struct device *dev = isp_subdev->dev;
+ int ret;
+
+ ret = isp4sd_setup_fw_mem_pool(isp_subdev);
+ if (ret) {
+ dev_err(dev, "fail to setup fw mem pool\n");
+ return ret;
+ }
+
+ ret = isp4sd_init_meta_buf(isp_subdev);
+ if (ret) {
+ dev_err(dev, "fail to alloc fw driver shared buf\n");
+ return ret;
+ }
+
+ ret = isp4sd_set_stream_path(isp_subdev);
+ if (ret) {
+ dev_err(dev, "fail to setup stream path\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void isp4sd_reset_stream_info(struct isp4_subdev *isp_subdev,
+ struct v4l2_subdev_state *state, u32 pad)
+{
+ struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
+ struct v4l2_mbus_framefmt *format = NULL;
+ struct isp4sd_output_info *str_info;
+ int i;
+
+ format = v4l2_subdev_state_get_format(state, pad, 0);
+
+ if (!format) {
+ dev_err(isp_subdev->dev, "fail to setup stream path\n");
+ } else {
+ memset(format, 0, sizeof(*format));
+ format->code = MEDIA_BUS_FMT_YUYV8_1_5X8;
+ }
+
+ for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++)
+ sensor_info->meta_info_buf[i] = NULL;
+
+ str_info = &sensor_info->output_info;
+ str_info->start_status = ISP4SD_START_STATUS_NOT_START;
+}
+
+static bool isp4sd_is_stream_running(struct isp4_subdev *isp_subdev)
+{
+ struct isp4sd_sensor_info *sif;
+ enum isp4sd_start_status stat;
+
+ sif = &isp_subdev->sensor_info;
+ stat = sif->output_info.start_status;
+ if (stat == ISP4SD_START_STATUS_STARTED)
+ return true;
+
+ return false;
+}
+
+static void isp4sd_reset_camera_info(struct isp4_subdev *isp_subdev,
+ struct v4l2_subdev_state *state, u32 pad)
+{
+ struct isp4sd_sensor_info *info = &isp_subdev->sensor_info;
+
+ info->status = ISP4SD_START_STATUS_NOT_START;
+ isp4sd_reset_stream_info(isp_subdev, state, pad);
+
+ info->start_stream_cmd_sent = false;
+}
+
+static int isp4sd_uninit_stream(struct isp4_subdev *isp_subdev,
+ struct v4l2_subdev_state *state, u32 pad)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct device *dev = isp_subdev->dev;
+ bool running;
+
+ running = isp4sd_is_stream_running(isp_subdev);
+
+ if (running) {
+ dev_dbg(dev, "fail for stream is still running\n");
+ return -EINVAL;
+ }
+
+ isp4sd_reset_camera_info(isp_subdev, state, pad);
+
+ isp4if_clear_cmdq(ispif);
+ return 0;
+}
+
+static void isp4sd_fw_resp_cmd_done(struct isp4_subdev *isp_subdev,
+ enum isp4if_stream_id stream_id,
+ struct isp4fw_resp_cmd_done *para)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4if_cmd_element *ele =
+ isp4if_rm_cmd_from_cmdq(ispif, para->cmd_seq_num, para->cmd_id);
+ struct device *dev = isp_subdev->dev;
+
+ dev_dbg(dev, "stream %d,cmd (0x%08x)(%d),seq %u, ele %p\n",
+ stream_id,
+ para->cmd_id, para->cmd_status, para->cmd_seq_num,
+ ele);
+
+ if (!ele)
+ return;
+
+ if (ele->wq) {
+ dev_dbg(dev, "signal event %p\n", ele->wq);
+ if (ele->wq_cond)
+ *ele->wq_cond = 1;
+ wake_up(ele->wq);
+ }
+
+ kfree(ele);
+}
+
+static struct isp4fw_meta_info *
+isp4sd_get_meta_by_mc(struct isp4_subdev *isp_subdev,
+ u64 mc)
+{
+ u32 i;
+
+ for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
+ struct isp4if_gpu_mem_info *meta_info_buf =
+ isp_subdev->sensor_info.meta_info_buf[i];
+
+ if (meta_info_buf) {
+ if (mc == meta_info_buf->gpu_mc_addr)
+ return meta_info_buf->sys_addr;
+ }
+ }
+ return NULL;
+};
+
+static struct isp4if_img_buf_node *
+isp4sd_preview_done(struct isp4_subdev *isp_subdev,
+ struct isp4fw_meta_info *meta)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4if_img_buf_node *prev = NULL;
+ struct device *dev = isp_subdev->dev;
+
+ if (meta->preview.enabled &&
+ (meta->preview.status == BUFFER_STATUS_SKIPPED ||
+ meta->preview.status == BUFFER_STATUS_DONE ||
+ meta->preview.status == BUFFER_STATUS_DIRTY)) {
+ prev = isp4if_dequeue_buffer(ispif);
+ if (!prev)
+ dev_err(dev, "fail null prev buf\n");
+
+ } else if (meta->preview.enabled) {
+ dev_err(dev, "fail bad preview status %u\n",
+ meta->preview.status);
+ }
+
+ return prev;
+}
+
+static void isp4sd_send_meta_info(struct isp4_subdev *isp_subdev,
+ u64 meta_info_mc)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4fw_cmd_send_buffer buf_type = {};
+ struct device *dev = isp_subdev->dev;
+
+ if (isp_subdev->sensor_info.status != ISP4SD_START_STATUS_STARTED) {
+ dev_warn(dev, "not working status %i, meta_info 0x%llx\n",
+ isp_subdev->sensor_info.status, meta_info_mc);
+ return;
+ }
+
+ if (meta_info_mc) {
+ buf_type.buffer_type = BUFFER_TYPE_META_INFO;
+ buf_type.buffer.buf_tags = 0;
+ buf_type.buffer.vmid_space.bit.vmid = 0;
+ buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
+ isp4if_split_addr64(meta_info_mc,
+ &buf_type.buffer.buf_base_a_lo,
+ &buf_type.buffer.buf_base_a_hi);
+
+ buf_type.buffer.buf_size_a = ISP4SD_META_BUF_SIZE;
+ if (isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
+ &buf_type, sizeof(buf_type))) {
+ dev_err(dev, "fail send meta_info 0x%llx\n",
+ meta_info_mc);
+ } else {
+ dev_dbg(dev, "resend meta_info 0x%llx\n", meta_info_mc);
+ }
+ }
+}
+
+static void isp4sd_fw_resp_frame_done(struct isp4_subdev *isp_subdev,
+ enum isp4if_stream_id stream_id,
+ struct isp4fw_resp_param_package *para)
+{
+ struct isp4if_img_buf_node *prev = NULL;
+ struct device *dev = isp_subdev->dev;
+ struct isp4fw_meta_info *meta;
+ u64 mc = 0;
+
+ mc = isp4if_join_addr64(para->package_addr_lo, para->package_addr_hi);
+ meta = isp4sd_get_meta_by_mc(isp_subdev, mc);
+ if (mc == 0 || !meta) {
+ dev_err(dev, "fail to get meta from mc %llx\n", mc);
+ return;
+ }
+
+ dev_dbg(dev, "ts:%llu,streamId:%d,poc:%u,preview_en:%u,(%i)\n",
+ ktime_get_ns(), stream_id, meta->poc,
+ meta->preview.enabled,
+ meta->preview.status);
+
+ prev = isp4sd_preview_done(isp_subdev, meta);
+
+ isp4if_dealloc_buffer_node(prev);
+
+ if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)
+ isp4sd_send_meta_info(isp_subdev, mc);
+
+ dev_dbg(dev, "stream_id:%d, status:%d\n", stream_id,
+ isp_subdev->sensor_info.status);
+}
+
+static void isp4sd_fw_resp_func(struct isp4_subdev *isp_subdev,
+ enum isp4if_stream_id stream_id)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct device *dev = isp_subdev->dev;
+ struct isp4fw_resp resp;
+
+ if (ispif->status < ISP4IF_STATUS_FW_RUNNING)
+ return;
+
+ while (true) {
+ s32 ret;
+
+ ret = isp4if_f2h_resp(ispif, stream_id, &resp);
+ if (ret)
+ break;
+
+ switch (resp.resp_id) {
+ case RESP_ID_CMD_DONE:
+ isp4sd_fw_resp_cmd_done(isp_subdev, stream_id,
+ &resp.param.cmd_done);
+ break;
+ case RESP_ID_NOTI_FRAME_DONE:
+ isp4sd_fw_resp_frame_done(isp_subdev, stream_id,
+ &resp.param.frame_done);
+ break;
+ default:
+ dev_err(dev, "-><- fail respid (0x%x)\n",
+ resp.resp_id);
+ break;
+ }
+ }
+}
+
+static s32 isp4sd_fw_resp_thread_wrapper(void *context)
+{
+ struct isp4_subdev_thread_param *para = context;
+ struct isp4sd_thread_handler *thread_ctx;
+ enum isp4if_stream_id stream_id;
+
+ struct isp4_subdev *isp_subdev;
+ struct device *dev;
+ u64 timeout;
+
+ if (!para)
+ return -EINVAL;
+
+ isp_subdev = para->isp_subdev;
+ dev = isp_subdev->dev;
+
+ switch (para->idx) {
+ case 0:
+ stream_id = ISP4IF_STREAM_ID_GLOBAL;
+ break;
+ case 1:
+ stream_id = ISP4IF_STREAM_ID_1;
+ break;
+ default:
+ dev_err(dev, "fail invalid %d\n", para->idx);
+ return -EINVAL;
+ }
+
+ thread_ctx = &isp_subdev->fw_resp_thread[para->idx];
+
+ thread_ctx->wq_cond = 0;
+ mutex_init(&thread_ctx->mutex);
+ init_waitqueue_head(&thread_ctx->waitq);
+ timeout = msecs_to_jiffies(ISP4SD_WAIT_RESP_IRQ_TIMEOUT);
+
+ dev_dbg(dev, "[%u] started\n", para->idx);
+
+ while (true) {
+ wait_event_interruptible_timeout(thread_ctx->waitq,
+ thread_ctx->wq_cond != 0,
+ timeout);
+ thread_ctx->wq_cond = 0;
+
+ if (kthread_should_stop()) {
+ dev_dbg(dev, "[%u] quit\n", para->idx);
+ break;
+ }
+
+ guard(mutex)(&thread_ctx->mutex);
+ isp4sd_fw_resp_func(isp_subdev, stream_id);
+ }
+
+ mutex_destroy(&thread_ctx->mutex);
+
+ return 0;
+}
+
+static int isp4sd_start_resp_proc_threads(struct isp4_subdev *isp_subdev)
+{
+ struct device *dev = isp_subdev->dev;
+ int i;
+
+ for (i = 0; i < ISP4SD_MAX_FW_RESP_STREAM_NUM; i++) {
+ struct isp4sd_thread_handler *thread_ctx =
+ &isp_subdev->fw_resp_thread[i];
+
+ isp_subdev->isp_resp_para[i].idx = i;
+ isp_subdev->isp_resp_para[i].isp_subdev = isp_subdev;
+
+ thread_ctx->thread = kthread_run(isp4sd_fw_resp_thread_wrapper,
+ &isp_subdev->isp_resp_para[i],
+ "amd_isp4_thread");
+ if (IS_ERR(thread_ctx->thread)) {
+ dev_err(dev, "create thread [%d] fail\n", i);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int isp4sd_stop_resp_proc_threads(struct isp4_subdev *isp_subdev)
+{
+ int i;
+
+ for (i = 0; i < ISP4SD_MAX_FW_RESP_STREAM_NUM; i++) {
+ struct isp4sd_thread_handler *thread_ctx =
+ &isp_subdev->fw_resp_thread[i];
+
+ if (thread_ctx->thread) {
+ kthread_stop(thread_ctx->thread);
+ thread_ctx->thread = NULL;
+ }
+ }
+
+ return 0;
+}
+
+static u32 isp4sd_get_started_stream_count(struct isp4_subdev *isp_subdev)
+{
+ u32 cnt = 0;
+
+ if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)
+ cnt++;
+ return cnt;
+}
+
+static int isp4sd_pwroff_and_deinit(struct isp4_subdev *isp_subdev)
+{
+ struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
+ unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_LOW;
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+
+ struct device *dev = isp_subdev->dev;
+ u32 cnt;
+ int ret;
+
+ mutex_lock(&isp_subdev->ops_mutex);
+
+ if (sensor_info->status == ISP4SD_START_STATUS_STARTED) {
+ dev_err(dev, "fail for stream still running\n");
+ mutex_unlock(&isp_subdev->ops_mutex);
+ return -EINVAL;
+ }
+
+ sensor_info->status = ISP4SD_START_STATUS_NOT_START;
+ cnt = isp4sd_get_started_stream_count(isp_subdev);
+ if (cnt > 0) {
+ dev_dbg(dev, "no need power off isp_subdev\n");
+ mutex_unlock(&isp_subdev->ops_mutex);
+ return 0;
+ }
+
+ isp4if_stop(ispif);
+
+ ret = dev_pm_genpd_set_performance_state(dev, perf_state);
+ if (ret)
+ dev_err(dev,
+ "fail to set isp_subdev performance state %u,ret %d\n",
+ perf_state, ret);
+ isp4sd_stop_resp_proc_threads(isp_subdev);
+ dev_dbg(dev, "isp_subdev stop resp proc streads suc");
+ /* hold ccpu reset */
+ isp4hw_wreg(isp_subdev->mmio, ISP_SOFT_RESET, 0x0);
+ isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0);
+ ret = pm_runtime_put_sync(dev);
+ if (ret)
+ dev_err(dev, "power off isp_subdev fail %d\n", ret);
+ else
+ dev_dbg(dev, "power off isp_subdev suc\n");
+
+ ispif->status = ISP4IF_STATUS_PWR_OFF;
+ isp4if_clear_cmdq(ispif);
+ isp4sd_module_enable(isp_subdev, false);
+
+ msleep(20);
+
+ mutex_unlock(&isp_subdev->ops_mutex);
+
+ return 0;
+}
+
+static int isp4sd_pwron_and_init(struct isp4_subdev *isp_subdev)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct device *dev = isp_subdev->dev;
+ int ret;
+
+ if (ispif->status == ISP4IF_STATUS_FW_RUNNING) {
+ dev_dbg(dev, "camera already opened, do nothing\n");
+ return 0;
+ }
+
+ mutex_lock(&isp_subdev->ops_mutex);
+
+ isp4sd_module_enable(isp_subdev, true);
+
+ isp_subdev->sensor_info.start_stream_cmd_sent = false;
+ isp_subdev->sensor_info.buf_sent_cnt = 0;
+
+ if (ispif->status < ISP4IF_STATUS_PWR_ON) {
+ unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_HIGH;
+
+ ret = pm_runtime_resume_and_get(dev);
+ if (ret) {
+ dev_err(dev, "fail to power on isp_subdev ret %d\n",
+ ret);
+ goto err_unlock_and_close;
+ }
+
+ /* ISPPG ISP Power Status */
+ isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0x7FF);
+ ret = dev_pm_genpd_set_performance_state(dev, perf_state);
+ if (ret) {
+ dev_err(dev,
+ "fail to set performance state %u, ret %d\n",
+ perf_state, ret);
+ goto err_unlock_and_close;
+ }
+
+ ispif->status = ISP4IF_STATUS_PWR_ON;
+
+ if (isp4sd_start_resp_proc_threads(isp_subdev)) {
+ dev_err(dev, "isp_start_resp_proc_threads fail");
+ goto err_unlock_and_close;
+ } else {
+ dev_dbg(dev, "create resp threads ok");
+ }
+ }
+
+ isp_subdev->sensor_info.start_stream_cmd_sent = false;
+ isp_subdev->sensor_info.buf_sent_cnt = 0;
+
+ ret = isp4if_start(ispif);
+ if (ret) {
+ dev_err(dev, "fail to start isp_subdev interface\n");
+ goto err_unlock_and_close;
+ }
+
+ mutex_unlock(&isp_subdev->ops_mutex);
+ return 0;
+err_unlock_and_close:
+ mutex_unlock(&isp_subdev->ops_mutex);
+ isp4sd_pwroff_and_deinit(isp_subdev);
+ return -EINVAL;
+}
+
+static int isp4sd_stop_stream(struct isp4_subdev *isp_subdev,
+ struct v4l2_subdev_state *state, u32 pad)
+{
+ struct isp4sd_output_info *output_info =
+ &isp_subdev->sensor_info.output_info;
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct device *dev = isp_subdev->dev;
+ int ret = 0;
+
+ dev_dbg(dev, "status %i\n", output_info->start_status);
+ mutex_lock(&isp_subdev->ops_mutex);
+
+ if (output_info->start_status == ISP4SD_START_STATUS_STARTED) {
+ struct isp4fw_cmd_enable_out_ch cmd_ch_disable;
+
+ cmd_ch_disable.ch = ISP_PIPE_OUT_CH_PREVIEW;
+ cmd_ch_disable.is_enable = false;
+ ret = isp4if_send_command_sync(ispif,
+ CMD_ID_ENABLE_OUT_CHAN,
+ &cmd_ch_disable,
+ sizeof(cmd_ch_disable),
+ ISP4SD_FW_CMD_TIMEOUT_IN_MS);
+ if (ret)
+ dev_err(dev, "fail to disable stream\n");
+ else
+ dev_dbg(dev, "wait disable stream suc\n");
+
+ ret = isp4if_send_command_sync(ispif, CMD_ID_STOP_STREAM,
+ NULL,
+ 0,
+ ISP4SD_FW_CMD_TIMEOUT_IN_MS);
+ if (ret)
+ dev_err(dev, "fail to stop steam\n");
+ else
+ dev_dbg(dev, "wait stop stream suc\n");
+ }
+
+ isp4if_clear_bufq(ispif);
+
+ output_info->start_status = ISP4SD_START_STATUS_NOT_START;
+ isp4sd_reset_stream_info(isp_subdev, state, pad);
+
+ mutex_unlock(&isp_subdev->ops_mutex);
+
+ isp4sd_uninit_stream(isp_subdev, state, pad);
+
+ return ret;
+}
+
+static int isp4sd_start_stream(struct isp4_subdev *isp_subdev,
+ struct v4l2_subdev_state *state, u32 pad)
+{
+ struct isp4sd_output_info *output_info =
+ &isp_subdev->sensor_info.output_info;
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct device *dev = isp_subdev->dev;
+ int ret;
+
+ mutex_lock(&isp_subdev->ops_mutex);
+
+ if (ispif->status != ISP4IF_STATUS_FW_RUNNING) {
+ mutex_unlock(&isp_subdev->ops_mutex);
+ dev_err(dev, "fail, bad fsm %d", ispif->status);
+ return -EINVAL;
+ }
+
+ ret = isp4sd_init_stream(isp_subdev);
+
+ if (ret) {
+ dev_err(dev, "fail to init isp_subdev stream\n");
+ ret = -EINVAL;
+ goto unlock_and_check_ret;
+ }
+
+ if (output_info->start_status == ISP4SD_START_STATUS_STARTED) {
+ ret = 0;
+ dev_dbg(dev, "stream started, do nothing\n");
+ goto unlock_and_check_ret;
+ } else if (output_info->start_status ==
+ ISP4SD_START_STATUS_START_FAIL) {
+ ret = -EINVAL;
+ dev_err(dev, "stream fail to start before\n");
+ goto unlock_and_check_ret;
+ }
+
+ if (isp4sd_setup_output(isp_subdev, state, pad)) {
+ dev_err(dev, "fail to setup output\n");
+ ret = -EINVAL;
+ } else {
+ ret = 0;
+ dev_dbg(dev, "suc to setup out\n");
+ }
+
+unlock_and_check_ret:
+ mutex_unlock(&isp_subdev->ops_mutex);
+ if (ret) {
+ isp4sd_stop_stream(isp_subdev, state, pad);
+ dev_err(dev, "start stream fail\n");
+ }
+
+ return ret;
+}
+
+static int isp4sd_set_power(struct v4l2_subdev *sd, int on)
+{
+ struct isp4_subdev *ispsd = to_isp4_subdev(sd);
+
+ if (on)
+ return isp4sd_pwron_and_init(ispsd);
+ else
+ return isp4sd_pwroff_and_deinit(ispsd);
+};
+
+static const struct v4l2_subdev_core_ops isp4sd_core_ops = {
+ .s_power = isp4sd_set_power,
+};
+
+static const struct v4l2_subdev_video_ops isp4sd_video_ops = {
+ .s_stream = v4l2_subdev_s_stream_helper,
+};
+
+static int isp4sd_set_pad_format(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *sd_state,
+ struct v4l2_subdev_format *fmt)
+{
+ struct isp4sd_output_info *steam_info =
+ &(to_isp4_subdev(sd)->sensor_info.output_info);
+ struct v4l2_mbus_framefmt *format;
+
+ format = v4l2_subdev_state_get_format(sd_state, fmt->pad);
+
+ if (!format) {
+ dev_err(sd->dev, "fail to get state format\n");
+ return -EINVAL;
+ }
+
+ *format = fmt->format;
+ switch (format->code) {
+ case MEDIA_BUS_FMT_YUYV8_1_5X8:
+ steam_info->image_size = format->width * format->height * 3 / 2;
+ break;
+ case MEDIA_BUS_FMT_YUYV8_1X16:
+ steam_info->image_size = format->width * format->height * 2;
+ break;
+ default:
+ steam_info->image_size = 0;
+ break;
+ }
+ if (!steam_info->image_size) {
+ dev_err(sd->dev,
+ "fail set pad format,code 0x%x,width %u, height %u\n",
+ format->code, format->width, format->height);
+ return -EINVAL;
+ }
+ dev_dbg(sd->dev,
+ "set pad format suc, code:%x w:%u h:%u size:%u\n", format->code,
+ format->width, format->height, steam_info->image_size);
+
+ return 0;
+}
+
+static int isp4sd_enable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct isp4_subdev *ispsd = to_isp4_subdev(sd);
+
+ return isp4sd_start_stream(ispsd, state, pad);
+}
+
+static int isp4sd_disable_streams(struct v4l2_subdev *sd,
+ struct v4l2_subdev_state *state, u32 pad,
+ u64 streams_mask)
+{
+ struct isp4_subdev *ispsd = to_isp4_subdev(sd);
+
+ return isp4sd_stop_stream(ispsd, state, pad);
+}
+
+static const struct v4l2_subdev_pad_ops isp4sd_pad_ops = {
+ .get_fmt = v4l2_subdev_get_fmt,
+ .set_fmt = isp4sd_set_pad_format,
+ .enable_streams = isp4sd_enable_streams,
+ .disable_streams = isp4sd_disable_streams,
+};
+
+static const struct v4l2_subdev_ops isp4sd_subdev_ops = {
+ .core = &isp4sd_core_ops,
+ .video = &isp4sd_video_ops,
+ .pad = &isp4sd_pad_ops,
+};
+
+static int isp4sd_sdev_link_validate(struct media_link *link)
+{
+ return 0;
+}
+
+static const struct media_entity_operations isp4sd_sdev_ent_ops = {
+ .link_validate = isp4sd_sdev_link_validate,
+};
+
+int isp4sd_init(struct isp4_subdev *isp_subdev,
+ struct v4l2_device *v4l2_dev)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4sd_sensor_info *sensor_info;
+ struct device *dev = v4l2_dev->dev;
+ int ret;
+
+ isp_subdev->dev = dev;
+ v4l2_subdev_init(&isp_subdev->sdev, &isp4sd_subdev_ops);
+ isp_subdev->sdev.owner = THIS_MODULE;
+ isp_subdev->sdev.dev = dev;
+ snprintf(isp_subdev->sdev.name, sizeof(isp_subdev->sdev.name), "%s",
+ dev_name(dev));
+
+ isp_subdev->sdev.entity.name = isp4sd_entity_name;
+ isp_subdev->sdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
+ isp_subdev->sdev.entity.ops = &isp4sd_sdev_ent_ops;
+ isp_subdev->sdev_pad.flags = MEDIA_PAD_FL_SOURCE;
+ ret = media_entity_pads_init(&isp_subdev->sdev.entity, 1,
+ &isp_subdev->sdev_pad);
+ if (ret) {
+ dev_err(dev, "fail to init isp4 subdev entity pad %d\n", ret);
+ return ret;
+ }
+ ret = v4l2_subdev_init_finalize(&isp_subdev->sdev);
+ if (ret < 0) {
+ dev_err(dev, "fail to init finalize isp4 subdev %d\n",
+ ret);
+ return ret;
+ }
+ ret = v4l2_device_register_subdev(v4l2_dev, &isp_subdev->sdev);
+ if (ret) {
+ dev_err(dev, "fail to register isp4 subdev to V4L2 device %d\n",
+ ret);
+ goto err_media_clean_up;
+ }
+
+ sensor_info = &isp_subdev->sensor_info;
+
+ isp4if_init(ispif, dev, isp_subdev->mmio);
+
+ mutex_init(&isp_subdev->ops_mutex);
+ sensor_info->start_stream_cmd_sent = false;
+ sensor_info->status = ISP4SD_START_STATUS_NOT_START;
+
+ /* create ISP enable gpio control */
+ isp_subdev->enable_gpio = devm_gpiod_get(isp_subdev->dev,
+ "enable_isp",
+ GPIOD_OUT_LOW);
+ if (IS_ERR(isp_subdev->enable_gpio)) {
+ dev_err(dev, "fail to get gpiod %d\n", ret);
+ media_entity_cleanup(&isp_subdev->sdev.entity);
+ return PTR_ERR(isp_subdev->enable_gpio);
+ }
+
+ isp_subdev->host2fw_seq_num = 1;
+ ispif->status = ISP4IF_STATUS_PWR_OFF;
+
+ if (ret)
+ goto err_media_clean_up;
+ return ret;
+
+err_media_clean_up:
+ media_entity_cleanup(&isp_subdev->sdev.entity);
+ return ret;
+}
+
+void isp4sd_deinit(struct isp4_subdev *isp_subdev)
+{
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+
+ media_entity_cleanup(&isp_subdev->sdev.entity);
+ isp4if_deinit(ispif);
+ isp4sd_module_enable(isp_subdev, false);
+
+ ispif->status = ISP4IF_STATUS_PWR_OFF;
+}
diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.h b/drivers/media/platform/amd/isp4/isp4_subdev.h
new file mode 100644
index 000000000000..524a8de5e18d
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_subdev.h
@@ -0,0 +1,131 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#ifndef _ISP4_CONTEXT_H_
+#define _ISP4_CONTEXT_H_
+
+#include <linux/delay.h>
+#include <linux/firmware.h>
+#include <linux/platform_device.h>
+#include <linux/uaccess.h>
+#include <linux/types.h>
+#include <linux/debugfs.h>
+#include <media/v4l2-device.h>
+
+#include "isp4_fw_cmd_resp.h"
+#include "isp4_hw_reg.h"
+#include "isp4_interface.h"
+
+/*
+ * one is for none sesnor specefic response which is not used now
+ * another is for sensor specific response
+ */
+#define ISP4SD_MAX_FW_RESP_STREAM_NUM 2
+
+/*
+ * cmd used to register frame done callback, parameter is
+ * struct isp4sd_register_framedone_cb_param *
+ * when a image buffer is filled by ISP, ISP will call the registered callback.
+ * callback func prototype is isp4sd_framedone_cb, cb_ctx can be anything
+ * provided by caller which will be provided back as the first parameter of the
+ * callback function.
+ * both cb_func and cb_ctx are provide by caller, set cb_func to NULL to
+ * unregister the callback
+ */
+
+/* used to indicate the ISP status */
+enum isp4sd_status {
+ ISP4SD_STATUS_PWR_OFF,
+ ISP4SD_STATUS_PWR_ON,
+ ISP4SD_STATUS_FW_RUNNING,
+ ISP4SD_STATUS_MAX
+};
+
+/* used to indicate the status of sensor, output stream */
+enum isp4sd_start_status {
+ ISP4SD_START_STATUS_NOT_START,
+ ISP4SD_START_STATUS_STARTED,
+ ISP4SD_START_STATUS_START_FAIL,
+};
+
+struct isp4sd_img_buf_node {
+ struct list_head node;
+ struct isp4if_img_buf_info buf_info;
+};
+
+/* this is isp output after processing bayer raw input from sensor */
+struct isp4sd_output_info {
+ enum isp4sd_start_status start_status;
+ u32 image_size;
+};
+
+/*
+ * This struct represents the sensor info which is input or source of ISP,
+ * meta_info_buf is the buffer store the fw to driver metainfo response
+ * status is the sensor status
+ * output_info is the isp output info after ISP processing the sensor input,
+ * start_stream_cmd_sent mean if CMD_ID_START_STREAM has sent to fw.
+ * buf_sent_cnt is buffer count app has sent to receive the images
+ */
+struct isp4sd_sensor_info {
+ struct isp4if_gpu_mem_info *
+ meta_info_buf[ISP4IF_MAX_STREAM_BUF_COUNT];
+ struct isp4sd_output_info output_info;
+ enum isp4sd_start_status status;
+ bool start_stream_cmd_sent;
+ u32 buf_sent_cnt;
+};
+
+/*
+ * Thread created by driver to receive fw response
+ * thread will be wakeup by fw to driver response interrupt
+ */
+struct isp4sd_thread_handler {
+ struct task_struct *thread;
+ struct mutex mutex; /* mutex */
+ wait_queue_head_t waitq;
+ int wq_cond;
+};
+
+struct isp4_subdev_thread_param {
+ u32 idx;
+ struct isp4_subdev *isp_subdev;
+};
+
+struct isp4_subdev {
+ struct v4l2_subdev sdev;
+ struct isp4_interface ispif;
+
+ struct media_pad sdev_pad;
+
+ enum isp4sd_status isp_status;
+ struct mutex ops_mutex; /* ops_mutex */
+
+ /* Used to store fw cmds sent to FW whose response driver needs to wait for */
+ struct isp4sd_thread_handler
+ fw_resp_thread[ISP4SD_MAX_FW_RESP_STREAM_NUM];
+
+ u32 host2fw_seq_num;
+
+ struct isp4sd_sensor_info sensor_info;
+
+ /* gpio descriptor */
+ struct gpio_desc *enable_gpio;
+ struct device *dev;
+ void __iomem *mmio;
+ struct isp4_subdev_thread_param
+ isp_resp_para[ISP4SD_MAX_FW_RESP_STREAM_NUM];
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debugfs_dir;
+ bool enable_fw_log;
+ char *fw_log_output;
+#endif
+};
+
+int isp4sd_init(struct isp4_subdev *isp_subdev,
+ struct v4l2_device *v4l2_dev);
+void isp4sd_deinit(struct isp4_subdev *isp_subdev);
+
+#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
` (3 preceding siblings ...)
2025-09-11 10:08 ` [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added Bin Du
@ 2025-09-11 10:08 ` Bin Du
2025-10-01 6:53 ` Sultan Alsawaf
2025-09-11 10:08 ` [PATCH v4 6/7] media: platform: amd: isp4 debug fs logging and more descriptive errors Bin Du
` (2 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du, Svetoslav Stoilov,
Alexey Zagorodnikov
Isp video implements v4l2 video interface and supports NV12 and YUYV. It
manages buffers, pipeline power and state. Cherry-picked Sultan's DMA
buffer related fix from branch v6.16-drm-tip-isp4-for-amd on
https://github.com/kerneltoast/kernel_x86_laptop.git
Co-developed-by: Sultan Alsawaf <sultan@kerneltoast.com>
Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Bin Du <Bin.Du@amd.com>
Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
---
MAINTAINERS | 2 +
drivers/media/platform/amd/isp4/Makefile | 1 +
drivers/media/platform/amd/isp4/isp4.c | 10 +
drivers/media/platform/amd/isp4/isp4_subdev.c | 95 +-
drivers/media/platform/amd/isp4/isp4_subdev.h | 2 +
drivers/media/platform/amd/isp4/isp4_video.c | 1207 +++++++++++++++++
drivers/media/platform/amd/isp4/isp4_video.h | 87 ++
7 files changed, 1400 insertions(+), 4 deletions(-)
create mode 100644 drivers/media/platform/amd/isp4/isp4_video.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_video.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 48ffc8bbdcee..80c966fde0b4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1151,6 +1151,8 @@ F: drivers/media/platform/amd/isp4/isp4_interface.c
F: drivers/media/platform/amd/isp4/isp4_interface.h
F: drivers/media/platform/amd/isp4/isp4_subdev.c
F: drivers/media/platform/amd/isp4/isp4_subdev.h
+F: drivers/media/platform/amd/isp4/isp4_video.c
+F: drivers/media/platform/amd/isp4/isp4_video.h
AMD KFD
M: Felix Kuehling <Felix.Kuehling@amd.com>
diff --git a/drivers/media/platform/amd/isp4/Makefile b/drivers/media/platform/amd/isp4/Makefile
index 905788bc6a1e..33589091ca96 100644
--- a/drivers/media/platform/amd/isp4/Makefile
+++ b/drivers/media/platform/amd/isp4/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_AMD_ISP4) += amd_capture.o
amd_capture-objs := isp4_subdev.o \
isp4_interface.o \
isp4.o \
+ isp4_video.o
\ No newline at end of file
diff --git a/drivers/media/platform/amd/isp4/isp4.c b/drivers/media/platform/amd/isp4/isp4.c
index 8cec27228710..a46e110a396f 100644
--- a/drivers/media/platform/amd/isp4/isp4.c
+++ b/drivers/media/platform/amd/isp4/isp4.c
@@ -178,6 +178,16 @@ static int isp4_capture_probe(struct platform_device *pdev)
goto err_isp4_deinit;
}
+ ret = media_create_pad_link(&isp_dev->isp_sdev.sdev.entity,
+ 0, &isp_dev->isp_sdev.isp_vdev.vdev.entity,
+ 0,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret) {
+ dev_err(dev, "fail to create pad link %d\n", ret);
+ goto err_isp4_deinit;
+ }
+
platform_set_drvdata(pdev, isp_dev);
return 0;
diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.c b/drivers/media/platform/amd/isp4/isp4_subdev.c
index a9cb14de04ca..7d3339c915eb 100644
--- a/drivers/media/platform/amd/isp4/isp4_subdev.c
+++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
@@ -466,20 +466,25 @@ isp4sd_get_meta_by_mc(struct isp4_subdev *isp_subdev,
static struct isp4if_img_buf_node *
isp4sd_preview_done(struct isp4_subdev *isp_subdev,
- struct isp4fw_meta_info *meta)
+ struct isp4fw_meta_info *meta,
+ struct isp4vid_framedone_param *pcb)
{
struct isp4_interface *ispif = &isp_subdev->ispif;
struct isp4if_img_buf_node *prev = NULL;
struct device *dev = isp_subdev->dev;
+ pcb->preview.status = ISP4VID_BUF_DONE_STATUS_ABSENT;
if (meta->preview.enabled &&
(meta->preview.status == BUFFER_STATUS_SKIPPED ||
meta->preview.status == BUFFER_STATUS_DONE ||
meta->preview.status == BUFFER_STATUS_DIRTY)) {
prev = isp4if_dequeue_buffer(ispif);
- if (!prev)
+ if (!prev) {
dev_err(dev, "fail null prev buf\n");
-
+ } else {
+ pcb->preview.buf = prev->buf_info;
+ pcb->preview.status = ISP4VID_BUF_DONE_STATUS_SUCCESS;
+ }
} else if (meta->preview.enabled) {
dev_err(dev, "fail bad preview status %u\n",
meta->preview.status);
@@ -525,6 +530,7 @@ static void isp4sd_fw_resp_frame_done(struct isp4_subdev *isp_subdev,
enum isp4if_stream_id stream_id,
struct isp4fw_resp_param_package *para)
{
+ struct isp4vid_framedone_param pcb = {};
struct isp4if_img_buf_node *prev = NULL;
struct device *dev = isp_subdev->dev;
struct isp4fw_meta_info *meta;
@@ -537,12 +543,17 @@ static void isp4sd_fw_resp_frame_done(struct isp4_subdev *isp_subdev,
return;
}
+ pcb.poc = meta->poc;
+ pcb.cam_id = 0;
+
dev_dbg(dev, "ts:%llu,streamId:%d,poc:%u,preview_en:%u,(%i)\n",
ktime_get_ns(), stream_id, meta->poc,
meta->preview.enabled,
meta->preview.status);
- prev = isp4sd_preview_done(isp_subdev, meta);
+ prev = isp4sd_preview_done(isp_subdev, meta, &pcb);
+ if (pcb.preview.status != ISP4VID_BUF_DONE_STATUS_ABSENT)
+ isp4vid_notify(&isp_subdev->isp_vdev, &pcb);
isp4if_dealloc_buffer_node(prev);
@@ -917,6 +928,75 @@ static int isp4sd_start_stream(struct isp4_subdev *isp_subdev,
return ret;
}
+static int isp4sd_ioc_send_img_buf(struct v4l2_subdev *sd,
+ struct isp4if_img_buf_info *buf_info)
+{
+ struct isp4_subdev *isp_subdev = to_isp4_subdev(sd);
+ struct isp4_interface *ispif = &isp_subdev->ispif;
+ struct isp4if_img_buf_node *buf_node = NULL;
+ struct device *dev = isp_subdev->dev;
+ int ret = -EINVAL;
+
+ mutex_lock(&isp_subdev->ops_mutex);
+ /* TODO: remove isp_status */
+ if (ispif->status != ISP4IF_STATUS_FW_RUNNING) {
+ dev_err(dev, "fail send img buf for bad fsm %d\n",
+ ispif->status);
+ mutex_unlock(&isp_subdev->ops_mutex);
+ return -EINVAL;
+ }
+
+ buf_node = isp4if_alloc_buffer_node(buf_info);
+ if (!buf_node) {
+ dev_err(dev, "fail alloc sys img buf info node\n");
+ ret = -ENOMEM;
+ goto unlock_and_return;
+ }
+
+ ret = isp4if_queue_buffer(ispif, buf_node);
+ if (ret) {
+ dev_err(dev, "fail to queue image buf, %d\n", ret);
+ goto error_release_buf_node;
+ }
+
+ if (!isp_subdev->sensor_info.start_stream_cmd_sent) {
+ isp_subdev->sensor_info.buf_sent_cnt++;
+
+ if (isp_subdev->sensor_info.buf_sent_cnt >=
+ ISP4SD_MIN_BUF_CNT_BEF_START_STREAM) {
+ ret = isp4if_send_command(ispif, CMD_ID_START_STREAM,
+ NULL, 0);
+
+ if (ret) {
+ dev_err(dev, "fail to START_STREAM");
+ goto error_release_buf_node;
+ }
+ isp_subdev->sensor_info.start_stream_cmd_sent = true;
+ isp_subdev->sensor_info.output_info.start_status =
+ ISP4SD_START_STATUS_STARTED;
+ isp_subdev->sensor_info.status =
+ ISP4SD_START_STATUS_STARTED;
+ } else {
+ dev_dbg(dev,
+ "no send start,required %u,buf sent %u\n",
+ ISP4SD_MIN_BUF_CNT_BEF_START_STREAM,
+ isp_subdev->sensor_info.buf_sent_cnt);
+ }
+ }
+
+ mutex_unlock(&isp_subdev->ops_mutex);
+
+ return 0;
+
+error_release_buf_node:
+ isp4if_dealloc_buffer_node(buf_node);
+
+unlock_and_return:
+ mutex_unlock(&isp_subdev->ops_mutex);
+
+ return ret;
+}
+
static int isp4sd_set_power(struct v4l2_subdev *sd, int on)
{
struct isp4_subdev *ispsd = to_isp4_subdev(sd);
@@ -1015,6 +1095,10 @@ static const struct media_entity_operations isp4sd_sdev_ent_ops = {
.link_validate = isp4sd_sdev_link_validate,
};
+static const struct isp4vid_ops isp4sd_isp4vid_ops = {
+ .send_buffer = isp4sd_ioc_send_img_buf,
+};
+
int isp4sd_init(struct isp4_subdev *isp_subdev,
struct v4l2_device *v4l2_dev)
{
@@ -1074,6 +1158,8 @@ int isp4sd_init(struct isp4_subdev *isp_subdev,
isp_subdev->host2fw_seq_num = 1;
ispif->status = ISP4IF_STATUS_PWR_OFF;
+ ret = isp4vid_dev_init(&isp_subdev->isp_vdev, &isp_subdev->sdev,
+ &isp4sd_isp4vid_ops);
if (ret)
goto err_media_clean_up;
return ret;
@@ -1087,6 +1173,7 @@ void isp4sd_deinit(struct isp4_subdev *isp_subdev)
{
struct isp4_interface *ispif = &isp_subdev->ispif;
+ isp4vid_dev_deinit(&isp_subdev->isp_vdev);
media_entity_cleanup(&isp_subdev->sdev.entity);
isp4if_deinit(ispif);
isp4sd_module_enable(isp_subdev, false);
diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.h b/drivers/media/platform/amd/isp4/isp4_subdev.h
index 524a8de5e18d..322bd990bbaa 100644
--- a/drivers/media/platform/amd/isp4/isp4_subdev.h
+++ b/drivers/media/platform/amd/isp4/isp4_subdev.h
@@ -17,6 +17,7 @@
#include "isp4_fw_cmd_resp.h"
#include "isp4_hw_reg.h"
#include "isp4_interface.h"
+#include "isp4_video.h"
/*
* one is for none sesnor specefic response which is not used now
@@ -97,6 +98,7 @@ struct isp4_subdev_thread_param {
struct isp4_subdev {
struct v4l2_subdev sdev;
struct isp4_interface ispif;
+ struct isp4vid_dev isp_vdev;
struct media_pad sdev_pad;
diff --git a/drivers/media/platform/amd/isp4/isp4_video.c b/drivers/media/platform/amd/isp4/isp4_video.c
new file mode 100644
index 000000000000..38252854012e
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_video.c
@@ -0,0 +1,1207 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#include <drm/amd/isp.h>
+#include <linux/pm_runtime.h>
+#include <linux/vmalloc.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+
+#include "isp4_interface.h"
+#include "isp4_subdev.h"
+#include "isp4_video.h"
+
+#define ISP4VID_ISP_DRV_NAME "amd_isp_capture"
+#define ISP4VID_MAX_PREVIEW_FPS 30
+#define ISP4VID_DEFAULT_FMT isp4vid_formats[0]
+
+#define ISP4VID_PAD_VIDEO_OUTPUT 0
+
+/* timeperframe default */
+#define ISP4VID_ISP_TPF_DEFAULT isp4vid_tpfs[0]
+
+/* amdisp buffer for vb2 operations */
+struct isp4vid_vb2_buf {
+ struct device *dev;
+ void *vaddr;
+ unsigned long size;
+ refcount_t refcount;
+ struct dma_buf *dbuf;
+ void *bo;
+ u64 gpu_addr;
+ struct vb2_vmarea_handler handler;
+};
+
+static int isp4vid_vb2_mmap(void *buf_priv, struct vm_area_struct *vma);
+
+static void isp4vid_vb2_put(void *buf_priv);
+
+static const char *isp4vid_video_dev_name = "Preview";
+
+/* Sizes must be in increasing order */
+static const struct v4l2_frmsize_discrete isp4vid_frmsize[] = {
+ {640, 360},
+ {640, 480},
+ {1280, 720},
+ {1280, 960},
+ {1920, 1080},
+ {1920, 1440},
+ {2560, 1440},
+ {2880, 1620},
+ {2880, 1624},
+ {2888, 1808},
+};
+
+static const u32 isp4vid_formats[] = {
+ V4L2_PIX_FMT_NV12,
+ V4L2_PIX_FMT_YUYV
+};
+
+/* timeperframe list */
+static const struct v4l2_fract isp4vid_tpfs[] = {
+ {.numerator = 1, .denominator = ISP4VID_MAX_PREVIEW_FPS}
+};
+
+static void isp4vid_handle_frame_done(struct isp4vid_dev *isp_vdev,
+ const struct isp4if_img_buf_info *img_buf,
+ bool done_suc)
+{
+ struct isp4vid_capture_buffer *isp4vid_buf;
+ enum vb2_buffer_state state;
+ void *vbuf;
+
+ mutex_lock(&isp_vdev->buf_list_lock);
+
+ /* Get the first entry of the list */
+ isp4vid_buf = list_first_entry_or_null(&isp_vdev->buf_list, typeof(*isp4vid_buf), list);
+ if (!isp4vid_buf) {
+ mutex_unlock(&isp_vdev->buf_list_lock);
+ return;
+ }
+
+ vbuf = vb2_plane_vaddr(&isp4vid_buf->vb2.vb2_buf, 0);
+
+ if (vbuf != img_buf->planes[0].sys_addr) {
+ dev_err(isp_vdev->dev, "Invalid vbuf");
+ mutex_unlock(&isp_vdev->buf_list_lock);
+ return;
+ }
+
+ /* Remove this entry from the list */
+ list_del(&isp4vid_buf->list);
+
+ mutex_unlock(&isp_vdev->buf_list_lock);
+
+ /* Fill the buffer */
+ isp4vid_buf->vb2.vb2_buf.timestamp = ktime_get_ns();
+ isp4vid_buf->vb2.sequence = isp_vdev->sequence++;
+ isp4vid_buf->vb2.field = V4L2_FIELD_ANY;
+
+ /* at most 2 planes */
+ vb2_set_plane_payload(&isp4vid_buf->vb2.vb2_buf,
+ 0, isp_vdev->format.sizeimage);
+
+ state = done_suc ? VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+ vb2_buffer_done(&isp4vid_buf->vb2.vb2_buf, state);
+
+ dev_dbg(isp_vdev->dev, "call vb2_buffer_done(size=%u)\n",
+ isp_vdev->format.sizeimage);
+}
+
+s32 isp4vid_notify(void *cb_ctx, struct isp4vid_framedone_param *evt_param)
+{
+ struct isp4vid_dev *isp4vid_vdev = cb_ctx;
+
+ if (evt_param->preview.status != ISP4VID_BUF_DONE_STATUS_ABSENT) {
+ bool suc;
+
+ suc = (evt_param->preview.status ==
+ ISP4VID_BUF_DONE_STATUS_SUCCESS);
+ isp4vid_handle_frame_done(isp4vid_vdev,
+ &evt_param->preview.buf,
+ suc);
+ }
+
+ return 0;
+}
+
+static unsigned int isp4vid_vb2_num_users(void *buf_priv)
+{
+ struct isp4vid_vb2_buf *buf = buf_priv;
+
+ return refcount_read(&buf->refcount);
+}
+
+static int isp4vid_vb2_mmap(void *buf_priv, struct vm_area_struct *vma)
+{
+ struct isp4vid_vb2_buf *buf = buf_priv;
+ int ret;
+
+ if (!buf) {
+ pr_err("fail no memory to map\n");
+ return -EINVAL;
+ }
+
+ ret = remap_vmalloc_range(vma, buf->vaddr, 0);
+ if (ret) {
+ dev_err(buf->dev, "fail remap vmalloc mem, %d\n", ret);
+ return ret;
+ }
+
+ /*
+ * Make sure that vm_areas for 2 buffers won't be merged together
+ */
+ vm_flags_set(vma, VM_DONTEXPAND);
+
+ /*
+ * Use common vm_area operations to track buffer refcount.
+ */
+ vma->vm_private_data = &buf->handler;
+ vma->vm_ops = &vb2_common_vm_ops;
+
+ vma->vm_ops->open(vma);
+
+ dev_dbg(buf->dev, "mmap isp user bo 0x%llx size %ld refcount %d\n",
+ buf->gpu_addr, buf->size, refcount_read(&buf->refcount));
+
+ return 0;
+}
+
+static void *isp4vid_vb2_vaddr(struct vb2_buffer *vb, void *buf_priv)
+{
+ struct isp4vid_vb2_buf *buf = buf_priv;
+
+ if (!buf->vaddr) {
+ dev_err(buf->dev,
+ "fail for buf vaddr is null\n");
+ return NULL;
+ }
+ return buf->vaddr;
+}
+
+static void isp4vid_vb2_detach_dmabuf(void *mem_priv)
+{
+ struct isp4vid_vb2_buf *buf = mem_priv;
+
+ dev_dbg(buf->dev, "detach dmabuf of isp user bo 0x%llx size %ld",
+ buf->gpu_addr, buf->size);
+
+ kfree(buf);
+}
+
+static void *isp4vid_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
+ struct dma_buf *dbuf,
+ unsigned long size)
+{
+ struct isp4vid_vb2_buf *buf;
+
+ if (dbuf->size < size) {
+ dev_err(dev, "Invalid dmabuf size %zu %lu", dbuf->size, size);
+ return ERR_PTR(-EFAULT);
+ }
+
+ buf = kzalloc(sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ return ERR_PTR(-ENOMEM);
+
+ struct isp4vid_vb2_buf *dbg_buf = dbuf->priv;
+
+ buf->dev = dev;
+ buf->dbuf = dbuf;
+ buf->size = size;
+
+ dev_dbg(dev, "attach dmabuf of isp user bo 0x%llx size %ld",
+ dbg_buf->gpu_addr, dbg_buf->size);
+
+ return buf;
+}
+
+static void isp4vid_vb2_unmap_dmabuf(void *mem_priv)
+{
+ struct isp4vid_vb2_buf *buf = mem_priv;
+ struct iosys_map map = IOSYS_MAP_INIT_VADDR(buf->vaddr);
+
+ dev_dbg(buf->dev, "unmap dmabuf of isp user bo 0x%llx size %ld",
+ buf->gpu_addr, buf->size);
+
+ dma_buf_vunmap_unlocked(buf->dbuf, &map);
+ buf->vaddr = NULL;
+}
+
+static int isp4vid_vb2_map_dmabuf(void *mem_priv)
+{
+ struct isp4vid_vb2_buf *mmap_buf = NULL;
+ struct isp4vid_vb2_buf *buf = mem_priv;
+ struct iosys_map map = {};
+ int ret;
+
+ ret = dma_buf_vmap_unlocked(buf->dbuf, &map);
+ if (ret) {
+ dev_err(buf->dev, "vmap_unlocked fail");
+ return -EFAULT;
+ }
+ buf->vaddr = map.vaddr;
+
+ mmap_buf = buf->dbuf->priv;
+ buf->gpu_addr = mmap_buf->gpu_addr;
+
+ dev_dbg(buf->dev, "map dmabuf of isp user bo 0x%llx size %ld",
+ buf->gpu_addr, buf->size);
+
+ return 0;
+}
+
+#ifdef CONFIG_HAS_DMA
+struct isp4vid_vb2_amdgpu_attachment {
+ struct sg_table sgt;
+ enum dma_data_direction dma_dir;
+};
+
+static int isp4vid_dmabuf_ops_attach(struct dma_buf *dbuf,
+ struct dma_buf_attachment *dbuf_attach)
+{
+ struct isp4vid_vb2_buf *buf = dbuf->priv;
+ int num_pages = PAGE_ALIGN(buf->size) / PAGE_SIZE;
+ struct isp4vid_vb2_amdgpu_attachment *attach;
+ void *vaddr = buf->vaddr;
+ struct scatterlist *sg;
+ struct sg_table *sgt;
+ int ret;
+ int i;
+
+ attach = kzalloc(sizeof(*attach), GFP_KERNEL);
+ if (!attach)
+ return -ENOMEM;
+
+ sgt = &attach->sgt;
+ ret = sg_alloc_table(sgt, num_pages, GFP_KERNEL);
+ if (ret) {
+ kfree(attach);
+ return ret;
+ }
+ for_each_sgtable_sg(sgt, sg, i) {
+ struct page *page = vmalloc_to_page(vaddr);
+
+ if (!page) {
+ sg_free_table(sgt);
+ kfree(attach);
+ return -ENOMEM;
+ }
+ sg_set_page(sg, page, PAGE_SIZE, 0);
+ vaddr = ((char *)vaddr) + PAGE_SIZE;
+ }
+
+ attach->dma_dir = DMA_NONE;
+ dbuf_attach->priv = attach;
+
+ return 0;
+}
+
+static void isp4vid_dmabuf_ops_detach(struct dma_buf *dbuf,
+ struct dma_buf_attachment *dbuf_attach)
+{
+ struct isp4vid_vb2_amdgpu_attachment *attach = dbuf_attach->priv;
+ struct sg_table *sgt;
+
+ if (!attach) {
+ pr_err("fail invalid attach handler\n");
+ return;
+ }
+
+ sgt = &attach->sgt;
+
+ /* release the scatterlist cache */
+ if (attach->dma_dir != DMA_NONE)
+ dma_unmap_sgtable(dbuf_attach->dev, sgt, attach->dma_dir, 0);
+ sg_free_table(sgt);
+ kfree(attach);
+ dbuf_attach->priv = NULL;
+}
+
+static struct sg_table *isp4vid_dmabuf_ops_map(struct dma_buf_attachment *dbuf_attach,
+ enum dma_data_direction dma_dir)
+{
+ struct isp4vid_vb2_amdgpu_attachment *attach = dbuf_attach->priv;
+ struct sg_table *sgt;
+
+ sgt = &attach->sgt;
+ /* return previously mapped sg table */
+ if (attach->dma_dir == dma_dir)
+ return sgt;
+
+ /* release any previous cache */
+ if (attach->dma_dir != DMA_NONE) {
+ dma_unmap_sgtable(dbuf_attach->dev, sgt, attach->dma_dir, 0);
+ attach->dma_dir = DMA_NONE;
+ }
+
+ /* mapping to the client with new direction */
+ if (dma_map_sgtable(dbuf_attach->dev, sgt, dma_dir, 0)) {
+ dev_err(dbuf_attach->dev, "fail to map scatterlist");
+ return ERR_PTR(-EIO);
+ }
+
+ attach->dma_dir = dma_dir;
+
+ return sgt;
+}
+
+static void isp4vid_dmabuf_ops_unmap(struct dma_buf_attachment *dbuf_attach,
+ struct sg_table *sgt,
+ enum dma_data_direction dma_dir)
+{
+ /* nothing to be done here */
+}
+
+static int isp4vid_dmabuf_ops_vmap(struct dma_buf *dbuf,
+ struct iosys_map *map)
+{
+ struct isp4vid_vb2_buf *buf = dbuf->priv;
+
+ iosys_map_set_vaddr(map, buf->vaddr);
+
+ return 0;
+}
+
+static void isp4vid_dmabuf_ops_release(struct dma_buf *dbuf)
+{
+ struct isp4vid_vb2_buf *buf = dbuf->priv;
+
+ /* drop reference obtained in isp4vid_vb2_get_dmabuf */
+ if (dbuf != buf->dbuf)
+ isp4vid_vb2_put(buf);
+ else
+ kfree(buf);
+}
+
+static int isp4vid_dmabuf_ops_mmap(struct dma_buf *dbuf, struct vm_area_struct *vma)
+{
+ return isp4vid_vb2_mmap(dbuf->priv, vma);
+}
+
+static const struct dma_buf_ops isp4vid_dmabuf_ops = {
+ .attach = isp4vid_dmabuf_ops_attach,
+ .detach = isp4vid_dmabuf_ops_detach,
+ .map_dma_buf = isp4vid_dmabuf_ops_map,
+ .unmap_dma_buf = isp4vid_dmabuf_ops_unmap,
+ .vmap = isp4vid_dmabuf_ops_vmap,
+ .mmap = isp4vid_dmabuf_ops_mmap,
+ .release = isp4vid_dmabuf_ops_release,
+};
+
+static struct dma_buf *isp4vid_get_dmabuf(struct isp4vid_vb2_buf *buf, unsigned long flags)
+{
+ DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct dma_buf *dbuf;
+
+ if (WARN_ON(!buf->vaddr))
+ return NULL;
+
+ exp_info.ops = &isp4vid_dmabuf_ops;
+ exp_info.size = buf->size;
+ exp_info.flags = flags;
+ exp_info.priv = buf;
+
+ dbuf = dma_buf_export(&exp_info);
+ if (IS_ERR(dbuf))
+ return NULL;
+
+ return dbuf;
+}
+
+static struct dma_buf *isp4vid_vb2_get_dmabuf(struct vb2_buffer *vb, void *buf_priv,
+ unsigned long flags)
+{
+ struct isp4vid_vb2_buf *buf = buf_priv;
+ struct dma_buf *dbuf;
+
+ dbuf = isp4vid_get_dmabuf(buf, flags);
+ if (!dbuf) {
+ dev_err(buf->dev, "fail to get isp dma buffer\n");
+ return NULL;
+ }
+
+ refcount_inc(&buf->refcount);
+
+ dev_dbg(buf->dev, "buf exported, refcount %d\n",
+ refcount_read(&buf->refcount));
+
+ return dbuf;
+}
+#endif /* CONFIG_HAS_DMA */
+
+static void isp4vid_vb2_put(void *buf_priv)
+{
+ struct isp4vid_vb2_buf *buf = buf_priv;
+
+ dev_dbg(buf->dev,
+ "release isp user bo 0x%llx size %ld refcount %d",
+ buf->gpu_addr, buf->size,
+ refcount_read(&buf->refcount));
+
+ if (refcount_dec_and_test(&buf->refcount)) {
+ isp_user_buffer_free(buf->bo);
+ vfree(buf->vaddr);
+ /*
+ * Putting the implicit dmabuf frees `buf`. Freeing `buf` must
+ * be done from the dmabuf release callback since dma_buf_put()
+ * isn't always synchronous; it's just an fput(), which may be
+ * deferred. Since the dmabuf release callback needs to access
+ * `buf`, this means `buf` cannot be freed until then.
+ */
+ dma_buf_put(buf->dbuf);
+ }
+}
+
+static void *isp4vid_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size)
+{
+ struct isp4vid_vb2_buf *buf;
+ u64 gpu_addr;
+ void *bo;
+ int ret;
+
+ buf = kzalloc(sizeof(*buf), GFP_KERNEL | vb->vb2_queue->gfp_flags);
+ if (!buf)
+ return ERR_PTR(-ENOMEM);
+
+ buf->dev = dev;
+ buf->size = size;
+ buf->vaddr = vmalloc_user(buf->size);
+ if (!buf->vaddr) {
+ dev_err(dev, "fail to vmalloc buffer\n");
+ goto free_buf;
+ }
+
+ buf->handler.refcount = &buf->refcount;
+ buf->handler.put = isp4vid_vb2_put;
+ buf->handler.arg = buf;
+
+ /* get implicit dmabuf */
+ buf->dbuf = isp4vid_get_dmabuf(buf, 0);
+ if (!buf->dbuf) {
+ dev_err(dev, "fail to get implicit dmabuf\n");
+ goto free_user_vmem;
+ }
+
+ /* create isp user BO and obtain gpu_addr */
+ ret = isp_user_buffer_alloc(dev, buf->dbuf, &bo, &gpu_addr);
+ if (ret) {
+ dev_err(dev, "fail to create isp user BO\n");
+ goto put_dmabuf;
+ }
+
+ buf->bo = bo;
+ buf->gpu_addr = gpu_addr;
+
+ refcount_set(&buf->refcount, 1);
+
+ dev_dbg(dev, "allocated isp user bo 0x%llx size %ld refcount %d\n",
+ buf->gpu_addr, buf->size, refcount_read(&buf->refcount));
+
+ return buf;
+
+put_dmabuf:
+ dma_buf_put(buf->dbuf);
+free_user_vmem:
+ vfree(buf->vaddr);
+free_buf:
+ ret = buf->vaddr ? -EINVAL : -ENOMEM;
+ kfree(buf);
+ return ERR_PTR(ret);
+}
+
+static const struct vb2_mem_ops isp4vid_vb2_memops = {
+ .alloc = isp4vid_vb2_alloc,
+ .put = isp4vid_vb2_put,
+#ifdef CONFIG_HAS_DMA
+ .get_dmabuf = isp4vid_vb2_get_dmabuf,
+#endif
+ .map_dmabuf = isp4vid_vb2_map_dmabuf,
+ .unmap_dmabuf = isp4vid_vb2_unmap_dmabuf,
+ .attach_dmabuf = isp4vid_vb2_attach_dmabuf,
+ .detach_dmabuf = isp4vid_vb2_detach_dmabuf,
+ .vaddr = isp4vid_vb2_vaddr,
+ .mmap = isp4vid_vb2_mmap,
+ .num_users = isp4vid_vb2_num_users,
+};
+
+static const struct v4l2_pix_format isp4vid_fmt_default = {
+ .width = 1920,
+ .height = 1080,
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .field = V4L2_FIELD_NONE,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+};
+
+static void isp4vid_capture_return_all_buffers(struct isp4vid_dev *isp_vdev,
+ enum vb2_buffer_state state)
+{
+ struct isp4vid_capture_buffer *vbuf, *node;
+
+ mutex_lock(&isp_vdev->buf_list_lock);
+
+ list_for_each_entry_safe(vbuf, node, &isp_vdev->buf_list, list) {
+ list_del(&vbuf->list);
+ vb2_buffer_done(&vbuf->vb2.vb2_buf, state);
+ }
+ mutex_unlock(&isp_vdev->buf_list_lock);
+
+ dev_dbg(isp_vdev->dev, "call vb2_buffer_done(%d)\n", state);
+}
+
+static int isp4vid_vdev_link_validate(struct media_link *link)
+{
+ return 0;
+}
+
+static const struct media_entity_operations isp4vid_vdev_ent_ops = {
+ .link_validate = isp4vid_vdev_link_validate,
+};
+
+static const struct v4l2_file_operations isp4vid_vdev_fops = {
+ .owner = THIS_MODULE,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .read = vb2_fop_read,
+ .poll = vb2_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = vb2_fop_mmap,
+};
+
+static int isp4vid_ioctl_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
+{
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+
+ strscpy(cap->driver, ISP4VID_ISP_DRV_NAME, sizeof(cap->driver));
+ snprintf(cap->card, sizeof(cap->card), "%s", ISP4VID_ISP_DRV_NAME);
+ cap->capabilities |= V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
+
+ dev_dbg(isp_vdev->dev, "%s|capabilities=0x%X", isp_vdev->vdev.name, cap->capabilities);
+
+ return 0;
+}
+
+static int isp4vid_g_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f)
+{
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+
+ f->fmt.pix = isp_vdev->format;
+
+ return 0;
+}
+
+static int isp4vid_try_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f)
+{
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+ struct v4l2_pix_format *format = &f->fmt.pix;
+ unsigned int i;
+
+ if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ /*
+ * Check if the hardware supports the requested format, use the default
+ * format otherwise.
+ */
+ for (i = 0; i < ARRAY_SIZE(isp4vid_formats); i++)
+ if (isp4vid_formats[i] == format->pixelformat)
+ break;
+
+ if (i == ARRAY_SIZE(isp4vid_formats))
+ format->pixelformat = ISP4VID_DEFAULT_FMT;
+
+ switch (format->pixelformat) {
+ case V4L2_PIX_FMT_NV12: {
+ const struct v4l2_frmsize_discrete *fsz =
+ v4l2_find_nearest_size(isp4vid_frmsize,
+ ARRAY_SIZE(isp4vid_frmsize),
+ width, height,
+ format->width, format->height);
+
+ format->width = fsz->width;
+ format->height = fsz->height;
+
+ format->bytesperline = format->width;
+ format->sizeimage = format->bytesperline *
+ format->height * 3 / 2;
+ }
+ break;
+ case V4L2_PIX_FMT_YUYV: {
+ const struct v4l2_frmsize_discrete *fsz =
+ v4l2_find_nearest_size(isp4vid_frmsize,
+ ARRAY_SIZE(isp4vid_frmsize),
+ width, height,
+ format->width, format->height);
+
+ format->width = fsz->width;
+ format->height = fsz->height;
+
+ format->bytesperline = format->width * 2;
+ format->sizeimage = format->bytesperline * format->height;
+ }
+ break;
+ default:
+ dev_err(isp_vdev->dev, "%s|unsupported fmt=%u",
+ isp_vdev->vdev.name, format->pixelformat);
+ return -EINVAL;
+ }
+
+ if (format->field == V4L2_FIELD_ANY)
+ format->field = isp4vid_fmt_default.field;
+
+ if (format->colorspace == V4L2_COLORSPACE_DEFAULT)
+ format->colorspace = isp4vid_fmt_default.colorspace;
+
+ return 0;
+}
+
+static int isp4vid_set_fmt_2_isp(struct v4l2_subdev *sdev, struct v4l2_pix_format *pix_fmt)
+{
+ struct v4l2_subdev_format fmt = {};
+
+ switch (pix_fmt->pixelformat) {
+ case V4L2_PIX_FMT_NV12:
+ fmt.format.code = MEDIA_BUS_FMT_YUYV8_1_5X8;
+ break;
+ case V4L2_PIX_FMT_YUYV:
+ fmt.format.code = MEDIA_BUS_FMT_YUYV8_1X16;
+ break;
+ default:
+ return -EINVAL;
+ };
+ fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ fmt.pad = ISP4VID_PAD_VIDEO_OUTPUT;
+ fmt.format.width = pix_fmt->width;
+ fmt.format.height = pix_fmt->height;
+ return v4l2_subdev_call(sdev, pad, set_fmt, NULL, &fmt);
+}
+
+static int isp4vid_s_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f)
+{
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+ int ret;
+
+ /* Do not change the format while stream is on */
+ if (vb2_is_busy(&isp_vdev->vbq))
+ return -EBUSY;
+
+ if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ ret = isp4vid_try_fmt_vid_cap(file, priv, f);
+ if (ret)
+ return ret;
+
+ dev_dbg(isp_vdev->dev, "%s|width height:%ux%u->%ux%u",
+ isp_vdev->vdev.name,
+ isp_vdev->format.width, isp_vdev->format.height,
+ f->fmt.pix.width, f->fmt.pix.height);
+ dev_dbg(isp_vdev->dev, "%s|pixelformat:0x%x-0x%x",
+ isp_vdev->vdev.name, isp_vdev->format.pixelformat,
+ f->fmt.pix.pixelformat);
+ dev_dbg(isp_vdev->dev, "%s|bytesperline:%u->%u",
+ isp_vdev->vdev.name, isp_vdev->format.bytesperline,
+ f->fmt.pix.bytesperline);
+ dev_dbg(isp_vdev->dev, "%s|sizeimage:%u->%u",
+ isp_vdev->vdev.name, isp_vdev->format.sizeimage,
+ f->fmt.pix.sizeimage);
+
+ isp_vdev->format = f->fmt.pix;
+ ret = isp4vid_set_fmt_2_isp(isp_vdev->isp_sdev, &isp_vdev->format);
+
+ return ret;
+}
+
+static int isp4vid_enum_fmt_vid_cap(struct file *file, void *priv, struct v4l2_fmtdesc *f)
+{
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+
+ switch (f->index) {
+ case 0:
+ f->pixelformat = V4L2_PIX_FMT_NV12;
+ break;
+ case 1:
+ f->pixelformat = V4L2_PIX_FMT_YUYV;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ dev_dbg(isp_vdev->dev, "%s|index=%d, pixelformat=0x%X",
+ isp_vdev->vdev.name, f->index, f->pixelformat);
+
+ return 0;
+}
+
+static int isp4vid_enum_framesizes(struct file *file, void *fh, struct v4l2_frmsizeenum *fsize)
+{
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(isp4vid_formats); i++) {
+ if (isp4vid_formats[i] == fsize->pixel_format)
+ break;
+ }
+ if (i == ARRAY_SIZE(isp4vid_formats))
+ return -EINVAL;
+
+ if (fsize->index < ARRAY_SIZE(isp4vid_frmsize)) {
+ fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+ fsize->discrete = isp4vid_frmsize[fsize->index];
+ dev_dbg(isp_vdev->dev, "%s|size[%d]=%dx%d",
+ isp_vdev->vdev.name, fsize->index,
+ fsize->discrete.width, fsize->discrete.height);
+ } else {
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int isp4vid_ioctl_enum_frameintervals(struct file *file, void *priv,
+ struct v4l2_frmivalenum *fival)
+{
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+ int i;
+
+ if (fival->index >= ARRAY_SIZE(isp4vid_tpfs))
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(isp4vid_formats); i++)
+ if (isp4vid_formats[i] == fival->pixel_format)
+ break;
+ if (i == ARRAY_SIZE(isp4vid_formats))
+ return -EINVAL;
+
+ for (i = 0; i < ARRAY_SIZE(isp4vid_frmsize); i++)
+ if (isp4vid_frmsize[i].width == fival->width &&
+ isp4vid_frmsize[i].height == fival->height)
+ break;
+ if (i == ARRAY_SIZE(isp4vid_frmsize))
+ return -EINVAL;
+
+ fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
+ fival->discrete = isp4vid_tpfs[fival->index];
+ v4l2_simplify_fraction(&fival->discrete.numerator,
+ &fival->discrete.denominator, 8, 333);
+
+ dev_dbg(isp_vdev->dev, "%s|interval[%d]=%d/%d",
+ isp_vdev->vdev.name, fival->index,
+ fival->discrete.numerator,
+ fival->discrete.denominator);
+
+ return 0;
+}
+
+static int isp4vid_ioctl_g_param(struct file *file, void *priv, struct v4l2_streamparm *param)
+{
+ struct v4l2_captureparm *capture = ¶m->parm.capture;
+ struct isp4vid_dev *isp_vdev = video_drvdata(file);
+
+ if (param->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ capture->capability = V4L2_CAP_TIMEPERFRAME;
+ capture->timeperframe = isp_vdev->timeperframe;
+ capture->readbuffers = 0;
+
+ dev_dbg(isp_vdev->dev, "%s|timeperframe=%d/%d", isp_vdev->vdev.name,
+ capture->timeperframe.numerator,
+ capture->timeperframe.denominator);
+
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops isp4vid_vdev_ioctl_ops = {
+ /* VIDIOC_QUERYCAP handler */
+ .vidioc_querycap = isp4vid_ioctl_querycap,
+
+ /* VIDIOC_ENUM_FMT handlers */
+ .vidioc_enum_fmt_vid_cap = isp4vid_enum_fmt_vid_cap,
+
+ /* VIDIOC_G_FMT handlers */
+ .vidioc_g_fmt_vid_cap = isp4vid_g_fmt_vid_cap,
+
+ /* VIDIOC_S_FMT handlers */
+ .vidioc_s_fmt_vid_cap = isp4vid_s_fmt_vid_cap,
+
+ /* VIDIOC_TRY_FMT handlers */
+ .vidioc_try_fmt_vid_cap = isp4vid_try_fmt_vid_cap,
+
+ /* Buffer handlers */
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+
+ /* Stream on/off */
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+
+ /* Stream type-dependent parameter ioctls */
+ .vidioc_g_parm = isp4vid_ioctl_g_param,
+ .vidioc_s_parm = isp4vid_ioctl_g_param,
+
+ .vidioc_enum_framesizes = isp4vid_enum_framesizes,
+ .vidioc_enum_frameintervals = isp4vid_ioctl_enum_frameintervals,
+
+};
+
+static unsigned int isp4vid_get_image_size(struct v4l2_pix_format *fmt)
+{
+ switch (fmt->pixelformat) {
+ case V4L2_PIX_FMT_NV12:
+ return fmt->width * fmt->height * 3 / 2;
+ case V4L2_PIX_FMT_YUYV:
+ return fmt->width * fmt->height * 2;
+ default:
+ return 0;
+ }
+};
+
+static int isp4vid_qops_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers,
+ unsigned int *nplanes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct isp4vid_dev *isp_vdev = vb2_get_drv_priv(vq);
+ unsigned int q_num_bufs = vb2_get_num_buffers(vq);
+
+ if (*nplanes > 1) {
+ dev_err(isp_vdev->dev,
+ "fail to setup queue, no mplane supported %u\n",
+ *nplanes);
+ return -EINVAL;
+ };
+
+ if (*nplanes == 1) {
+ unsigned int size;
+
+ size = isp4vid_get_image_size(&isp_vdev->format);
+ if (sizes[0] < size) {
+ dev_err(isp_vdev->dev,
+ "fail for small plane size %u, %u expected\n",
+ sizes[0], size);
+ return -EINVAL;
+ }
+ }
+
+ if (q_num_bufs + *nbuffers < ISP4IF_MAX_STREAM_BUF_COUNT)
+ *nbuffers = ISP4IF_MAX_STREAM_BUF_COUNT - q_num_bufs;
+
+ switch (isp_vdev->format.pixelformat) {
+ case V4L2_PIX_FMT_NV12:
+ case V4L2_PIX_FMT_YUYV: {
+ *nplanes = 1;
+ sizes[0] = max(sizes[0], isp_vdev->format.sizeimage);
+ isp_vdev->format.sizeimage = sizes[0];
+ }
+ break;
+ default:
+ dev_err(isp_vdev->dev, "%s|unsupported fmt=%u\n",
+ isp_vdev->vdev.name, isp_vdev->format.pixelformat);
+ return -EINVAL;
+ }
+
+ dev_dbg(isp_vdev->dev, "%s|*nbuffers=%u *nplanes=%u sizes[0]=%u\n",
+ isp_vdev->vdev.name,
+ *nbuffers, *nplanes, sizes[0]);
+
+ return 0;
+}
+
+static void isp4vid_qops_buffer_queue(struct vb2_buffer *vb)
+{
+ struct isp4vid_capture_buffer *buf =
+ container_of(vb, struct isp4vid_capture_buffer, vb2.vb2_buf);
+ struct isp4vid_dev *isp_vdev = vb2_get_drv_priv(vb->vb2_queue);
+
+ struct isp4vid_vb2_buf *priv_buf = vb->planes[0].mem_priv;
+ struct isp4if_img_buf_info *img_buf = &buf->img_buf;
+
+ dev_dbg(isp_vdev->dev, "%s|index=%u", isp_vdev->vdev.name, vb->index);
+
+ dev_dbg(isp_vdev->dev, "queue isp user bo 0x%llx size=%lu",
+ priv_buf->gpu_addr,
+ priv_buf->size);
+
+ switch (isp_vdev->format.pixelformat) {
+ case V4L2_PIX_FMT_NV12: {
+ u32 y_size = isp_vdev->format.sizeimage / 3 * 2;
+ u32 uv_size = isp_vdev->format.sizeimage / 3;
+
+ img_buf->planes[0].len = y_size;
+ img_buf->planes[0].sys_addr = priv_buf->vaddr;
+ img_buf->planes[0].mc_addr = priv_buf->gpu_addr;
+
+ dev_dbg(isp_vdev->dev, "img_buf[0]: mc=0x%llx size=%u",
+ img_buf->planes[0].mc_addr,
+ img_buf->planes[0].len);
+
+ img_buf->planes[1].len = uv_size;
+ img_buf->planes[1].sys_addr = ((u8 *)priv_buf->vaddr + y_size);
+ img_buf->planes[1].mc_addr = priv_buf->gpu_addr + y_size;
+
+ dev_dbg(isp_vdev->dev, "img_buf[1]: mc=0x%llx size=%u",
+ img_buf->planes[1].mc_addr,
+ img_buf->planes[1].len);
+
+ img_buf->planes[2].len = 0;
+ }
+ break;
+ case V4L2_PIX_FMT_YUYV: {
+ img_buf->planes[0].len = isp_vdev->format.sizeimage;
+ img_buf->planes[0].sys_addr = priv_buf->vaddr;
+ img_buf->planes[0].mc_addr = priv_buf->gpu_addr;
+
+ dev_dbg(isp_vdev->dev, "img_buf[0]: mc=0x%llx size=%u",
+ img_buf->planes[0].mc_addr,
+ img_buf->planes[0].len);
+
+ img_buf->planes[1].len = 0;
+ img_buf->planes[2].len = 0;
+ }
+ break;
+ default:
+ dev_err(isp_vdev->dev, "%s|unsupported fmt=%u",
+ isp_vdev->vdev.name, isp_vdev->format.pixelformat);
+ return;
+ }
+
+ if (isp_vdev->stream_started)
+ isp_vdev->ops->send_buffer(isp_vdev->isp_sdev, img_buf);
+
+ mutex_lock(&isp_vdev->buf_list_lock);
+ list_add_tail(&buf->list, &isp_vdev->buf_list);
+ mutex_unlock(&isp_vdev->buf_list_lock);
+}
+
+static int isp4vid_qops_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct isp4vid_dev *isp_vdev = vb2_get_drv_priv(vq);
+ struct isp4vid_capture_buffer *isp_buf;
+ struct media_entity *entity;
+ struct v4l2_subdev *subdev;
+ struct media_pad *pad;
+ int ret = 0;
+
+ isp_vdev->sequence = 0;
+ ret = v4l2_pipeline_pm_get(&isp_vdev->vdev.entity);
+ if (ret) {
+ dev_err(isp_vdev->dev, "power up isp fail %d\n", ret);
+ goto release_buffers;
+ }
+
+ entity = &isp_vdev->vdev.entity;
+ while (1) {
+ pad = &entity->pads[0];
+ if (!(pad->flags & MEDIA_PAD_FL_SINK))
+ break;
+
+ pad = media_pad_remote_pad_first(pad);
+ if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
+ break;
+
+ entity = pad->entity;
+ subdev = media_entity_to_v4l2_subdev(entity);
+
+ ret = v4l2_subdev_call(subdev, video, s_stream, 1);
+ if (ret < 0 && ret != -ENOIOCTLCMD) {
+ dev_dbg(isp_vdev->dev, "fail start streaming: %s %d\n",
+ subdev->name, ret);
+ goto release_buffers;
+ }
+ }
+
+ list_for_each_entry(isp_buf, &isp_vdev->buf_list, list) {
+ isp_vdev->ops->send_buffer(isp_vdev->isp_sdev,
+ &isp_buf->img_buf);
+ }
+
+ /* Start the media pipeline */
+ ret = video_device_pipeline_start(&isp_vdev->vdev, &isp_vdev->pipe);
+ if (ret) {
+ dev_err(isp_vdev->dev, "video_device_pipeline_start fail:%d",
+ ret);
+ goto release_buffers;
+ }
+ isp_vdev->stream_started = true;
+
+ return 0;
+
+release_buffers:
+ isp4vid_capture_return_all_buffers(isp_vdev, VB2_BUF_STATE_QUEUED);
+ return ret;
+}
+
+static void isp4vid_qops_stop_streaming(struct vb2_queue *vq)
+{
+ struct isp4vid_dev *isp_vdev = vb2_get_drv_priv(vq);
+ struct v4l2_subdev *subdev;
+ struct media_entity *entity;
+ struct media_pad *pad;
+ int ret;
+
+ entity = &isp_vdev->vdev.entity;
+ while (1) {
+ pad = &entity->pads[0];
+ if (!(pad->flags & MEDIA_PAD_FL_SINK))
+ break;
+
+ pad = media_pad_remote_pad_first(pad);
+ if (!pad || !is_media_entity_v4l2_subdev(pad->entity))
+ break;
+
+ entity = pad->entity;
+ subdev = media_entity_to_v4l2_subdev(entity);
+
+ ret = v4l2_subdev_call(subdev, video, s_stream, 0);
+
+ if (ret < 0 && ret != -ENOIOCTLCMD)
+ dev_dbg(isp_vdev->dev, "fail stop streaming: %s %d\n",
+ subdev->name, ret);
+ }
+
+ isp_vdev->stream_started = false;
+ v4l2_pipeline_pm_put(&isp_vdev->vdev.entity);
+
+ /* Stop the media pipeline */
+ video_device_pipeline_stop(&isp_vdev->vdev);
+
+ /* Release all active buffers */
+ isp4vid_capture_return_all_buffers(isp_vdev, VB2_BUF_STATE_ERROR);
+}
+
+static int isp4vid_fill_buffer_size(struct isp4vid_dev *isp_vdev)
+{
+ int ret = 0;
+
+ switch (isp_vdev->format.pixelformat) {
+ case V4L2_PIX_FMT_NV12:
+ isp_vdev->format.bytesperline = isp_vdev->format.width;
+ isp_vdev->format.sizeimage = isp_vdev->format.bytesperline *
+ isp_vdev->format.height * 3 / 2;
+ break;
+ case V4L2_PIX_FMT_YUYV:
+ isp_vdev->format.bytesperline = isp_vdev->format.width;
+ isp_vdev->format.sizeimage = isp_vdev->format.bytesperline *
+ isp_vdev->format.height * 2;
+ break;
+ default:
+ dev_err(isp_vdev->dev, "fail for invalid default format 0x%x",
+ isp_vdev->format.pixelformat);
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static const struct vb2_ops isp4vid_qops = {
+ .queue_setup = isp4vid_qops_queue_setup,
+ .buf_queue = isp4vid_qops_buffer_queue,
+ .start_streaming = isp4vid_qops_start_streaming,
+ .stop_streaming = isp4vid_qops_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+int isp4vid_dev_init(struct isp4vid_dev *isp_vdev, struct v4l2_subdev *isp_sdev,
+ const struct isp4vid_ops *ops)
+{
+ const char *vdev_name = isp4vid_video_dev_name;
+ struct v4l2_device *v4l2_dev;
+ struct video_device *vdev;
+ struct vb2_queue *q;
+ int ret;
+
+ if (!isp_vdev || !isp_sdev || !isp_sdev->v4l2_dev)
+ return -EINVAL;
+
+ v4l2_dev = isp_sdev->v4l2_dev;
+ vdev = &isp_vdev->vdev;
+
+ isp_vdev->isp_sdev = isp_sdev;
+ isp_vdev->dev = v4l2_dev->dev;
+ isp_vdev->ops = ops;
+
+ /* Initialize the vb2_queue struct */
+ mutex_init(&isp_vdev->vbq_lock);
+ q = &isp_vdev->vbq;
+ q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ q->io_modes = VB2_MMAP | VB2_DMABUF;
+ q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ q->buf_struct_size = sizeof(struct isp4vid_capture_buffer);
+ q->min_queued_buffers = 2;
+ q->ops = &isp4vid_qops;
+ q->drv_priv = isp_vdev;
+ q->mem_ops = &isp4vid_vb2_memops;
+ q->lock = &isp_vdev->vbq_lock;
+ q->dev = v4l2_dev->dev;
+ ret = vb2_queue_init(q);
+ if (ret) {
+ dev_err(v4l2_dev->dev, "vb2_queue_init error:%d", ret);
+ return ret;
+ }
+ /* Initialize buffer list and its lock */
+ mutex_init(&isp_vdev->buf_list_lock);
+ INIT_LIST_HEAD(&isp_vdev->buf_list);
+
+ /* Set default frame format */
+ isp_vdev->format = isp4vid_fmt_default;
+ isp_vdev->timeperframe = ISP4VID_ISP_TPF_DEFAULT;
+ v4l2_simplify_fraction(&isp_vdev->timeperframe.numerator,
+ &isp_vdev->timeperframe.denominator, 8, 333);
+
+ ret = isp4vid_fill_buffer_size(isp_vdev);
+ if (ret) {
+ dev_err(v4l2_dev->dev, "fail to fill buffer size: %d\n", ret);
+ return ret;
+ }
+
+ ret = isp4vid_set_fmt_2_isp(isp_sdev, &isp_vdev->format);
+ if (ret) {
+ dev_err(v4l2_dev->dev, "fail init format :%d\n", ret);
+ return ret;
+ }
+
+ /* Initialize the video_device struct */
+ isp_vdev->vdev.entity.name = vdev_name;
+ isp_vdev->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
+ isp_vdev->vdev_pad.flags = MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(&isp_vdev->vdev.entity, 1,
+ &isp_vdev->vdev_pad);
+
+ if (ret) {
+ dev_err(v4l2_dev->dev, "init media entity pad fail:%d\n", ret);
+ return ret;
+ }
+
+ vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE |
+ V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
+ vdev->entity.ops = &isp4vid_vdev_ent_ops;
+ vdev->release = video_device_release_empty;
+ vdev->fops = &isp4vid_vdev_fops;
+ vdev->ioctl_ops = &isp4vid_vdev_ioctl_ops;
+ vdev->lock = NULL;
+ vdev->queue = q;
+ vdev->v4l2_dev = v4l2_dev;
+ vdev->vfl_dir = VFL_DIR_RX;
+ strscpy(vdev->name, vdev_name, sizeof(vdev->name));
+ video_set_drvdata(vdev, isp_vdev);
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret)
+ dev_err(v4l2_dev->dev, "register video device fail:%d\n", ret);
+
+ return ret;
+}
+
+void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev)
+{
+ video_unregister_device(&isp_vdev->vdev);
+}
diff --git a/drivers/media/platform/amd/isp4/isp4_video.h b/drivers/media/platform/amd/isp4/isp4_video.h
new file mode 100644
index 000000000000..fae7dbdedd18
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_video.h
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#ifndef _ISP4_VIDEO_H_
+#define _ISP4_VIDEO_H_
+
+#include <linux/mutex.h>
+#include <media/videobuf2-memops.h>
+#include <media/v4l2-dev.h>
+#include "isp4_interface.h"
+
+enum isp4vid_buf_done_status {
+ /* It means no corresponding image buf in fw response */
+ ISP4VID_BUF_DONE_STATUS_ABSENT,
+ ISP4VID_BUF_DONE_STATUS_SUCCESS,
+ ISP4VID_BUF_DONE_STATUS_FAILED
+};
+
+struct isp4vid_buf_done_info {
+ enum isp4vid_buf_done_status status;
+ struct isp4if_img_buf_info buf;
+};
+
+/* call back parameter for CB_EVT_ID_FRAME_DONE */
+struct isp4vid_framedone_param {
+ s32 poc;
+ s32 cam_id;
+ s64 time_stamp;
+ struct isp4vid_buf_done_info preview;
+};
+
+struct isp4vid_capture_buffer {
+ /*
+ * struct vb2_v4l2_buffer must be the first element
+ * the videobuf2 framework will allocate this struct based on
+ * buf_struct_size and use the first sizeof(struct vb2_buffer) bytes of
+ * memory as a vb2_buffer
+ */
+ struct vb2_v4l2_buffer vb2;
+ struct isp4if_img_buf_info img_buf;
+ struct list_head list;
+};
+
+struct isp4vid_dev;
+
+struct isp4vid_ops {
+ int (*send_buffer)(struct v4l2_subdev *sd,
+ struct isp4if_img_buf_info *img_buf);
+};
+
+struct isp4vid_dev {
+ struct video_device vdev;
+ struct media_pad vdev_pad;
+ struct v4l2_pix_format format;
+
+ /* mutex that protects vbq */
+ struct mutex vbq_lock;
+ struct vb2_queue vbq;
+
+ /* mutex that protects buf_list */
+ struct mutex buf_list_lock;
+ struct list_head buf_list;
+
+ u32 sequence;
+ bool stream_started;
+ struct task_struct *kthread;
+
+ struct media_pipeline pipe;
+ struct device *dev;
+ struct v4l2_subdev *isp_sdev;
+ struct v4l2_fract timeperframe;
+
+ /* Callback operations */
+ const struct isp4vid_ops *ops;
+};
+
+int isp4vid_dev_init(struct isp4vid_dev *isp_vdev,
+ struct v4l2_subdev *isp_sdev,
+ const struct isp4vid_ops *ops);
+
+void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev);
+
+s32 isp4vid_notify(void *cb_ctx, struct isp4vid_framedone_param *evt_param);
+
+#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 6/7] media: platform: amd: isp4 debug fs logging and more descriptive errors
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
` (4 preceding siblings ...)
2025-09-11 10:08 ` [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers " Bin Du
@ 2025-09-11 10:08 ` Bin Du
2025-09-11 10:08 ` [PATCH v4 7/7] Documentation: add documentation of AMD isp 4 driver Bin Du
2025-09-19 3:24 ` [PATCH v4 0/7] Add AMD ISP4 driver Du, Bin
7 siblings, 0 replies; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du, Svetoslav Stoilov,
Alexey Zagorodnikov
Add debug fs for isp4 driver and add more detailed descriptive error info
to some of the log message
Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Bin Du <Bin.Du@amd.com>
Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
---
MAINTAINERS | 2 +
drivers/media/platform/amd/isp4/Makefile | 1 +
drivers/media/platform/amd/isp4/isp4.c | 4 +
drivers/media/platform/amd/isp4/isp4_debug.c | 272 ++++++++++++++++++
drivers/media/platform/amd/isp4/isp4_debug.h | 41 +++
.../media/platform/amd/isp4/isp4_interface.c | 41 ++-
drivers/media/platform/amd/isp4/isp4_subdev.c | 29 +-
7 files changed, 368 insertions(+), 22 deletions(-)
create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.c
create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 80c966fde0b4..8478789ac265 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1145,6 +1145,8 @@ F: drivers/media/platform/amd/isp4/Kconfig
F: drivers/media/platform/amd/isp4/Makefile
F: drivers/media/platform/amd/isp4/isp4.c
F: drivers/media/platform/amd/isp4/isp4.h
+F: drivers/media/platform/amd/isp4/isp4_debug.c
+F: drivers/media/platform/amd/isp4/isp4_debug.h
F: drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
F: drivers/media/platform/amd/isp4/isp4_hw_reg.h
F: drivers/media/platform/amd/isp4/isp4_interface.c
diff --git a/drivers/media/platform/amd/isp4/Makefile b/drivers/media/platform/amd/isp4/Makefile
index 33589091ca96..c89355fb2374 100644
--- a/drivers/media/platform/amd/isp4/Makefile
+++ b/drivers/media/platform/amd/isp4/Makefile
@@ -4,6 +4,7 @@
obj-$(CONFIG_AMD_ISP4) += amd_capture.o
amd_capture-objs := isp4_subdev.o \
+ isp4_debug.o \
isp4_interface.o \
isp4.o \
isp4_video.o
\ No newline at end of file
diff --git a/drivers/media/platform/amd/isp4/isp4.c b/drivers/media/platform/amd/isp4/isp4.c
index a46e110a396f..f129a9fbf315 100644
--- a/drivers/media/platform/amd/isp4/isp4.c
+++ b/drivers/media/platform/amd/isp4/isp4.c
@@ -10,6 +10,7 @@
#include <media/v4l2-ioctl.h>
#include "isp4.h"
+#include "isp4_debug.h"
#include "isp4_hw_reg.h"
#define ISP4_DRV_NAME "amd_isp_capture"
@@ -189,6 +190,7 @@ static int isp4_capture_probe(struct platform_device *pdev)
}
platform_set_drvdata(pdev, isp_dev);
+ isp_debugfs_create(isp_dev);
return 0;
@@ -204,6 +206,8 @@ static void isp4_capture_remove(struct platform_device *pdev)
{
struct isp4_device *isp_dev = platform_get_drvdata(pdev);
+ isp_debugfs_remove(isp_dev);
+
v4l2_device_unregister_subdev(&isp_dev->isp_sdev.sdev);
media_device_unregister(&isp_dev->mdev);
diff --git a/drivers/media/platform/amd/isp4/isp4_debug.c b/drivers/media/platform/amd/isp4/isp4_debug.c
new file mode 100644
index 000000000000..cc99b357df27
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_debug.c
@@ -0,0 +1,272 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#include "isp4.h"
+#include "isp4_debug.h"
+#include "isp4_hw_reg.h"
+#include "isp4_interface.h"
+
+#define ISP4DBG_FW_LOG_RINGBUF_SIZE (2 * 1024 * 1024)
+#define ISP4DBG_MACRO_2_STR(X) #X
+#define ISP4DBG_MAX_ONE_TIME_LOG_LEN 510
+
+#ifdef CONFIG_DEBUG_FS
+
+void isp_debugfs_create(struct isp4_device *isp_dev)
+{
+ isp_dev->isp_sdev.debugfs_dir = debugfs_create_dir("amd_isp", NULL);
+ debugfs_create_bool("fw_log_enable", 0644,
+ isp_dev->isp_sdev.debugfs_dir,
+ &isp_dev->isp_sdev.enable_fw_log);
+ isp_dev->isp_sdev.fw_log_output =
+ devm_kzalloc(&isp_dev->pdev->dev,
+ ISP4DBG_FW_LOG_RINGBUF_SIZE + 32,
+ GFP_KERNEL);
+}
+
+void isp_debugfs_remove(struct isp4_device *isp_dev)
+{
+ debugfs_remove_recursive(isp_dev->isp_sdev.debugfs_dir);
+ isp_dev->isp_sdev.debugfs_dir = NULL;
+}
+
+static u32 isp_fw_fill_rb_log(struct isp4_subdev *isp, u8 *sys, u32 rb_size)
+{
+ struct isp4_interface *ispif = &isp->ispif;
+ struct device *dev = isp->dev;
+ u8 *buf = isp->fw_log_output;
+ u32 rd_ptr, wr_ptr;
+ u32 total_cnt = 0;
+ u32 offset = 0;
+ u32 cnt;
+
+ if (!sys || rb_size == 0)
+ return 0;
+
+ mutex_lock(&ispif->isp4if_mutex);
+
+ rd_ptr = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_LOG_RB_RPTR0);
+ wr_ptr = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_LOG_RB_WPTR0);
+
+ do {
+ if (wr_ptr > rd_ptr)
+ cnt = wr_ptr - rd_ptr;
+ else if (wr_ptr < rd_ptr)
+ cnt = rb_size - rd_ptr;
+ else
+ goto unlock_and_quit;
+
+ if (cnt > rb_size) {
+ dev_err(dev, "fail bad fw log size %u\n", cnt);
+ goto unlock_and_quit;
+ }
+
+ memcpy(buf + offset, (u8 *)(sys + rd_ptr), cnt);
+
+ offset += cnt;
+ total_cnt += cnt;
+ rd_ptr = (rd_ptr + cnt) % rb_size;
+ } while (rd_ptr < wr_ptr);
+
+ isp4hw_wreg(ISP4_GET_ISP_REG_BASE(isp), ISP_LOG_RB_RPTR0, rd_ptr);
+
+unlock_and_quit:
+ mutex_unlock(&ispif->isp4if_mutex);
+ return total_cnt;
+}
+
+void isp_fw_log_print(struct isp4_subdev *isp)
+{
+ struct isp4_interface *ispif = &isp->ispif;
+ char *fw_log_buf = isp->fw_log_output;
+ u32 cnt;
+
+ if (!isp->enable_fw_log || !fw_log_buf)
+ return;
+
+ cnt = isp_fw_fill_rb_log(isp, ispif->fw_log_buf->sys_addr,
+ ispif->fw_log_buf->mem_size);
+
+ if (cnt) {
+ char *line_end;
+ char temp_ch;
+ char *str;
+ char *end;
+
+ str = (char *)fw_log_buf;
+ end = ((char *)fw_log_buf + cnt);
+ fw_log_buf[cnt] = 0;
+
+ while (str < end) {
+ line_end = strchr(str, 0x0A);
+ if ((line_end && (str + ISP4DBG_MAX_ONE_TIME_LOG_LEN) >= line_end) ||
+ (!line_end && (str + ISP4DBG_MAX_ONE_TIME_LOG_LEN) >= end)) {
+ if (line_end)
+ *line_end = 0;
+
+ if (*str != '\0')
+ dev_dbg(isp->dev,
+ "%s", str);
+
+ if (line_end) {
+ *line_end = 0x0A;
+ str = line_end + 1;
+ } else {
+ break;
+ }
+ } else {
+ u32 tmp_len = ISP4DBG_MAX_ONE_TIME_LOG_LEN;
+
+ temp_ch = str[tmp_len];
+ str[tmp_len] = 0;
+ dev_dbg(isp->dev, "%s", str);
+ str[tmp_len] = temp_ch;
+ str = &str[tmp_len];
+ }
+ }
+ }
+}
+#endif
+
+char *isp4dbg_get_buf_src_str(u32 src)
+{
+ switch (src) {
+ case BUFFER_SOURCE_STREAM:
+ return ISP4DBG_MACRO_2_STR(BUFFER_SOURCE_STREAM);
+ default:
+ return "Unknown buf source";
+ }
+}
+
+char *isp4dbg_get_buf_done_str(u32 status)
+{
+ switch (status) {
+ case BUFFER_STATUS_INVALID:
+ return ISP4DBG_MACRO_2_STR(BUFFER_STATUS_INVALID);
+ case BUFFER_STATUS_SKIPPED:
+ return ISP4DBG_MACRO_2_STR(BUFFER_STATUS_SKIPPED);
+ case BUFFER_STATUS_EXIST:
+ return ISP4DBG_MACRO_2_STR(BUFFER_STATUS_EXIST);
+ case BUFFER_STATUS_DONE:
+ return ISP4DBG_MACRO_2_STR(BUFFER_STATUS_DONE);
+ case BUFFER_STATUS_LACK:
+ return ISP4DBG_MACRO_2_STR(BUFFER_STATUS_LACK);
+ case BUFFER_STATUS_DIRTY:
+ return ISP4DBG_MACRO_2_STR(BUFFER_STATUS_DIRTY);
+ case BUFFER_STATUS_MAX:
+ return ISP4DBG_MACRO_2_STR(BUFFER_STATUS_MAX);
+ default:
+ return "Unknown Buf Done Status";
+ }
+};
+
+char *isp4dbg_get_img_fmt_str(int fmt /* enum isp4fw_image_format * */)
+{
+ switch (fmt) {
+ case IMAGE_FORMAT_NV12:
+ return "NV12";
+ case IMAGE_FORMAT_YUV422INTERLEAVED:
+ return "YUV422INTERLEAVED";
+ default:
+ return "unknown fmt";
+ }
+}
+
+void isp4dbg_show_bufmeta_info(struct device *dev, char *pre,
+ void *in, void *orig_buf)
+{
+ struct isp4fw_buffer_meta_info *p;
+ struct isp4if_img_buf_info *orig;
+
+ if (!in)
+ return;
+
+ if (!pre)
+ pre = "";
+
+ p = in;
+ orig = orig_buf;
+
+ dev_dbg(dev, "%s(%s) en:%d,stat:%s(%u),src:%s\n", pre,
+ isp4dbg_get_img_fmt_str(p->image_prop.image_format),
+ p->enabled, isp4dbg_get_buf_done_str(p->status), p->status,
+ isp4dbg_get_buf_src_str(p->source));
+
+ dev_dbg(dev, "%p,0x%llx(%u) %p,0x%llx(%u) %p,0x%llx(%u)\n",
+ orig->planes[0].sys_addr, orig->planes[0].mc_addr,
+ orig->planes[0].len, orig->planes[1].sys_addr,
+ orig->planes[1].mc_addr, orig->planes[1].len,
+ orig->planes[2].sys_addr, orig->planes[2].mc_addr,
+ orig->planes[2].len);
+}
+
+char *isp4dbg_get_buf_type(u32 type)
+{
+ /* enum isp4fw_buffer_type */
+ switch (type) {
+ case BUFFER_TYPE_PREVIEW:
+ return ISP4DBG_MACRO_2_STR(BUFFER_TYPE_PREVIEW);
+ case BUFFER_TYPE_META_INFO:
+ return ISP4DBG_MACRO_2_STR(BUFFER_TYPE_META_INFO);
+ case BUFFER_TYPE_MEM_POOL:
+ return ISP4DBG_MACRO_2_STR(BUFFER_TYPE_MEM_POOL);
+ default:
+ return "unknown type";
+ }
+}
+
+char *isp4dbg_get_cmd_str(u32 cmd)
+{
+ switch (cmd) {
+ case CMD_ID_START_STREAM:
+ return ISP4DBG_MACRO_2_STR(CMD_ID_START_STREAM);
+ case CMD_ID_STOP_STREAM:
+ return ISP4DBG_MACRO_2_STR(CMD_ID_STOP_STREAM);
+ case CMD_ID_SEND_BUFFER:
+ return ISP4DBG_MACRO_2_STR(CMD_ID_SEND_BUFFER);
+ case CMD_ID_SET_STREAM_CONFIG:
+ return ISP4DBG_MACRO_2_STR(CMD_ID_SET_STREAM_CONFIG);
+ case CMD_ID_SET_OUT_CHAN_PROP:
+ return ISP4DBG_MACRO_2_STR(CMD_ID_SET_OUT_CHAN_PROP);
+ case CMD_ID_ENABLE_OUT_CHAN:
+ return ISP4DBG_MACRO_2_STR(CMD_ID_ENABLE_OUT_CHAN);
+ default:
+ return "unknown cmd";
+ };
+}
+
+char *isp4dbg_get_resp_str(u32 cmd)
+{
+ switch (cmd) {
+ case RESP_ID_CMD_DONE:
+ return ISP4DBG_MACRO_2_STR(RESP_ID_CMD_DONE);
+ case RESP_ID_NOTI_FRAME_DONE:
+ return ISP4DBG_MACRO_2_STR(RESP_ID_NOTI_FRAME_DONE);
+ default:
+ return "unknown respid";
+ };
+}
+
+char *isp4dbg_get_if_stream_str(u32 stream /* enum fw_cmd_resp_stream_id */)
+{
+ switch (stream) {
+ case ISP4IF_STREAM_ID_GLOBAL:
+ return "STREAM_GLOBAL";
+ case ISP4IF_STREAM_ID_1:
+ return "STREAM1";
+ default:
+ return "unknown streamID";
+ }
+}
+
+char *isp4dbg_get_out_ch_str(int ch /* enum isp4fw_pipe_out_ch */)
+{
+ switch ((enum isp4fw_pipe_out_ch)ch) {
+ case ISP_PIPE_OUT_CH_PREVIEW:
+ return "prev";
+ default:
+ return "unknown channel";
+ }
+}
diff --git a/drivers/media/platform/amd/isp4/isp4_debug.h b/drivers/media/platform/amd/isp4/isp4_debug.h
new file mode 100644
index 000000000000..0c75ddae4882
--- /dev/null
+++ b/drivers/media/platform/amd/isp4/isp4_debug.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2025 Advanced Micro Devices, Inc.
+ */
+
+#ifndef _ISP4_DEBUG_H_
+#define _ISP4_DEBUG_H_
+
+#include <linux/printk.h>
+#include <linux/dev_printk.h>
+
+#include "isp4_subdev.h"
+
+#ifdef CONFIG_DEBUG_FS
+struct isp4_device;
+
+void isp_debugfs_create(struct isp4_device *isp_dev);
+void isp_debugfs_remove(struct isp4_device *isp_dev);
+void isp_fw_log_print(struct isp4_subdev *isp);
+
+#else
+
+/* to avoid checkpatch warning */
+#define isp_debugfs_create(cam) ((void)(cam))
+#define isp_debugfs_remove(cam) ((void)(cam))
+#define isp_fw_log_print(isp) ((void)(isp))
+
+#endif /* CONFIG_DEBUG_FS */
+
+void isp4dbg_show_bufmeta_info(struct device *dev, char *pre, void *p,
+ void *orig_buf /* struct sys_img_buf_handle */);
+char *isp4dbg_get_img_fmt_str(int fmt /* enum _image_format_t */);
+char *isp4dbg_get_out_ch_str(int ch /* enum _isp_pipe_out_ch_t */);
+char *isp4dbg_get_cmd_str(u32 cmd);
+char *isp4dbg_get_buf_type(u32 type);/* enum _buffer_type_t */
+char *isp4dbg_get_resp_str(u32 resp);
+char *isp4dbg_get_buf_src_str(u32 src);
+char *isp4dbg_get_buf_done_str(u32 status);
+char *isp4dbg_get_if_stream_str(u32 stream);
+
+#endif
diff --git a/drivers/media/platform/amd/isp4/isp4_interface.c b/drivers/media/platform/amd/isp4/isp4_interface.c
index 52dcca57ce2e..914bd02a26e6 100644
--- a/drivers/media/platform/amd/isp4/isp4_interface.c
+++ b/drivers/media/platform/amd/isp4/isp4_interface.c
@@ -6,8 +6,8 @@
#include <drm/amd/isp.h>
#include <linux/iopoll.h>
#include <linux/mutex.h>
-#include <linux/platform_device.h>
+#include "isp4_debug.h"
#include "isp4_fw_cmd_resp.h"
#include "isp4_hw_reg.h"
#include "isp4_interface.h"
@@ -355,7 +355,8 @@ static int isp4if_insert_isp_fw_cmd(struct isp4_interface *ispif, enum isp4if_st
len = rb_config->val_size;
if (isp4if_is_cmdq_rb_full(ispif, stream)) {
- dev_err(dev, "fail no cmdslot (%d)\n", stream);
+ dev_err(dev, "fail no cmdslot %s(%d)\n",
+ isp4dbg_get_if_stream_str(stream), stream);
return -EINVAL;
}
@@ -363,13 +364,15 @@ static int isp4if_insert_isp_fw_cmd(struct isp4_interface *ispif, enum isp4if_st
rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
if (rd_ptr > len) {
- dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
+ dev_err(dev, "fail %s(%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, rd_ptr, len, wr_ptr);
return -EINVAL;
}
if (wr_ptr > len) {
- dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
+ dev_err(dev, "fail %s(%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, wr_ptr, len, rd_ptr);
return -EINVAL;
}
@@ -447,7 +450,8 @@ static int isp4if_send_fw_cmd(struct isp4_interface *ispif, u32 cmd_id, void *pa
u32 wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
dev_err(dev,
- "failed to get free cmdq slot, stream (%d),rd %u, wr %u\n",
+ "failed to get free cmdq slot, stream %s(%d),rd %u, wr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, rd_ptr, wr_ptr);
return -ETIMEDOUT;
}
@@ -497,9 +501,11 @@ static int isp4if_send_fw_cmd(struct isp4_interface *ispif, u32 cmd_id, void *pa
ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
if (ret) {
- dev_err(dev, "fail for insert_isp_fw_cmd camId (0x%08x)\n", cmd_id);
+ dev_err(dev, "fail for insert_isp_fw_cmd camId %s(0x%08x)\n",
+ isp4dbg_get_cmd_str(cmd_id), cmd_id);
if (cmd_ele) {
- isp4if_rm_cmd_from_cmdq(ispif, cmd_ele->seq_num, cmd_ele->cmd_id);
+ isp4if_rm_cmd_from_cmdq(ispif, cmd_ele->seq_num,
+ cmd_ele->cmd_id);
kfree(cmd_ele);
}
}
@@ -710,13 +716,15 @@ int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream,
wr_ptr_dbg = wr_ptr;
if (rd_ptr > len) {
- dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
+ dev_err(dev, "fail %s(%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, rd_ptr, len, wr_ptr);
return -EINVAL;
}
if (wr_ptr > len) {
- dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
+ dev_err(dev, "fail %s(%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, wr_ptr, len, rd_ptr);
return -EINVAL;
}
@@ -731,7 +739,8 @@ int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream,
isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
rreg, rd_ptr);
} else {
- dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
+ dev_err(dev, "%s(%u),rd %u(should<=%u),wr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, rd_ptr, len, wr_ptr);
return -EINVAL;
}
@@ -757,7 +766,8 @@ int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream,
isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
rreg, rd_ptr);
} else {
- dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
+ dev_err(dev, "%s(%u),rd %u(should<=%u),wr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, rd_ptr, len, wr_ptr);
return -EINVAL;
}
@@ -779,7 +789,8 @@ int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream,
isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
rreg, rd_ptr);
} else {
- dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
+ dev_err(dev, "%s(%u),rd %u(should<=%u),wr %u\n",
+ isp4dbg_get_if_stream_str(stream),
stream, rd_ptr, len, wr_ptr);
return -EINVAL;
}
@@ -794,9 +805,9 @@ int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream,
dev_err(dev, "resp checksum 0x%x,should 0x%x,rptr %u,wptr %u\n",
checksum, response->resp_check_sum, rd_ptr_dbg, wr_ptr_dbg);
- dev_err(dev, "(%u), seqNo %u, resp_id (0x%x)\n", stream,
- response->resp_seq_num,
- response->resp_id);
+ dev_err(dev, "%s(%u), seqNo %u, resp_id %s(0x%x)\n",
+ isp4dbg_get_if_stream_str(stream), stream, response->resp_seq_num,
+ isp4dbg_get_resp_str(response->resp_id), response->resp_id);
return -EINVAL;
}
diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.c b/drivers/media/platform/amd/isp4/isp4_subdev.c
index 7d3339c915eb..bc63311c2490 100644
--- a/drivers/media/platform/amd/isp4/isp4_subdev.c
+++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
@@ -7,6 +7,7 @@
#include <linux/pm_domain.h>
#include <linux/pm_runtime.h>
+#include "isp4_debug.h"
#include "isp4_fw_cmd_resp.h"
#include "isp4_interface.h"
#include "isp4_subdev.h"
@@ -252,7 +253,9 @@ static int isp4sd_setup_output(struct isp4_subdev *isp_subdev,
return -EINVAL;
}
- dev_dbg(dev, "channel: w:h=%u:%u,lp:%u,cp%u\n",
+ dev_dbg(dev, "channel:%s,fmt %s,w:h=%u:%u,lp:%u,cp%u\n",
+ isp4dbg_get_out_ch_str(cmd_ch_prop.ch),
+ isp4dbg_get_img_fmt_str(cmd_ch_prop.image_prop.image_format),
cmd_ch_prop.image_prop.width, cmd_ch_prop.image_prop.height,
cmd_ch_prop.image_prop.luma_pitch,
cmd_ch_prop.image_prop.chroma_pitch);
@@ -275,6 +278,9 @@ static int isp4sd_setup_output(struct isp4_subdev *isp_subdev,
return ret;
}
+ dev_dbg(dev, "enable channel %s\n",
+ isp4dbg_get_out_ch_str(cmd_ch_en.ch));
+
if (!sensor_info->start_stream_cmd_sent) {
ret = isp4sd_kickoff_stream(isp_subdev, out_prop->width,
out_prop->height);
@@ -428,8 +434,9 @@ static void isp4sd_fw_resp_cmd_done(struct isp4_subdev *isp_subdev,
isp4if_rm_cmd_from_cmdq(ispif, para->cmd_seq_num, para->cmd_id);
struct device *dev = isp_subdev->dev;
- dev_dbg(dev, "stream %d,cmd (0x%08x)(%d),seq %u, ele %p\n",
+ dev_dbg(dev, "stream %d,cmd %s(0x%08x)(%d),seq %u, ele %p\n",
stream_id,
+ isp4dbg_get_cmd_str(para->cmd_id),
para->cmd_id, para->cmd_status, para->cmd_seq_num,
ele);
@@ -486,8 +493,9 @@ isp4sd_preview_done(struct isp4_subdev *isp_subdev,
pcb->preview.status = ISP4VID_BUF_DONE_STATUS_SUCCESS;
}
} else if (meta->preview.enabled) {
- dev_err(dev, "fail bad preview status %u\n",
- meta->preview.status);
+ dev_err(dev, "fail bad preview status %u(%s)\n",
+ meta->preview.status,
+ isp4dbg_get_buf_done_str(meta->preview.status));
}
return prev;
@@ -546,14 +554,18 @@ static void isp4sd_fw_resp_frame_done(struct isp4_subdev *isp_subdev,
pcb.poc = meta->poc;
pcb.cam_id = 0;
- dev_dbg(dev, "ts:%llu,streamId:%d,poc:%u,preview_en:%u,(%i)\n",
+ dev_dbg(dev, "ts:%llu,streamId:%d,poc:%u,preview_en:%u,%s(%i)\n",
ktime_get_ns(), stream_id, meta->poc,
meta->preview.enabled,
+ isp4dbg_get_buf_done_str(meta->preview.status),
meta->preview.status);
prev = isp4sd_preview_done(isp_subdev, meta, &pcb);
- if (pcb.preview.status != ISP4VID_BUF_DONE_STATUS_ABSENT)
+ if (pcb.preview.status != ISP4VID_BUF_DONE_STATUS_ABSENT) {
+ isp4dbg_show_bufmeta_info(dev, "prev", &meta->preview,
+ &pcb.preview.buf);
isp4vid_notify(&isp_subdev->isp_vdev, &pcb);
+ }
isp4if_dealloc_buffer_node(prev);
@@ -574,6 +586,8 @@ static void isp4sd_fw_resp_func(struct isp4_subdev *isp_subdev,
if (ispif->status < ISP4IF_STATUS_FW_RUNNING)
return;
+ isp_fw_log_print(isp_subdev);
+
while (true) {
s32 ret;
@@ -591,7 +605,8 @@ static void isp4sd_fw_resp_func(struct isp4_subdev *isp_subdev,
&resp.param.frame_done);
break;
default:
- dev_err(dev, "-><- fail respid (0x%x)\n",
+ dev_err(dev, "-><- fail respid %s(0x%x)\n",
+ isp4dbg_get_resp_str(resp.resp_id),
resp.resp_id);
break;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH v4 7/7] Documentation: add documentation of AMD isp 4 driver
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
` (5 preceding siblings ...)
2025-09-11 10:08 ` [PATCH v4 6/7] media: platform: amd: isp4 debug fs logging and more descriptive errors Bin Du
@ 2025-09-11 10:08 ` Bin Du
2025-09-19 3:24 ` [PATCH v4 0/7] Add AMD ISP4 driver Du, Bin
7 siblings, 0 replies; 35+ messages in thread
From: Bin Du @ 2025-09-11 10:08 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, bin.du, Bin Du, Svetoslav Stoilov, Mario Limonciello,
Alexey Zagorodnikov
Add documentation for AMD ISP 4 and describe the main components
Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
Signed-off-by: Bin Du <Bin.Du@amd.com>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
---
Documentation/admin-guide/media/amdisp4-1.rst | 63 +++++++++++++++++++
Documentation/admin-guide/media/amdisp4.dot | 6 ++
.../admin-guide/media/v4l-drivers.rst | 1 +
MAINTAINERS | 2 +
4 files changed, 72 insertions(+)
create mode 100644 Documentation/admin-guide/media/amdisp4-1.rst
create mode 100644 Documentation/admin-guide/media/amdisp4.dot
diff --git a/Documentation/admin-guide/media/amdisp4-1.rst b/Documentation/admin-guide/media/amdisp4-1.rst
new file mode 100644
index 000000000000..878141154f96
--- /dev/null
+++ b/Documentation/admin-guide/media/amdisp4-1.rst
@@ -0,0 +1,63 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. include:: <isonum.txt>
+
+====================================
+AMD Image Signal Processor (amdisp4)
+====================================
+
+Introduction
+============
+
+This file documents the driver for the AMD ISP4 that is part of
+AMD Ryzen AI Max 300 Series.
+
+The driver is located under drivers/media/platform/amd/isp4 and uses
+the Media-Controller API.
+
+The driver exposes one video capture device to userspace and provide
+web camera like interface. Internally the video device is connected
+to the isp4 sub-device responsible for communication with the CCPU FW.
+
+Topology
+========
+
+.. _amdisp4_topology_graph:
+
+.. kernel-figure:: amdisp4.dot
+ :alt: Diagram of the media pipeline topology
+ :align: center
+
+
+
+The driver has 1 sub-device: Representing isp4 image signal processor.
+The driver has 1 video device: Capture device for retrieving images.
+
+- ISP4 Image Signal Processing Subdevice Node
+
+---------------------------------------------
+
+The isp4 is represented as a single V4L2 subdev, the sub-device does not
+provide interface to the user space. The sub-device is connected to one video node
+(isp4_capture) with immutable active link. The sub-device represents ISP with
+connected sensor similar to smart cameras (sensors with integrated ISP).
+sub-device has only one link to the video device for capturing the frames.
+The sub-device communicates with CCPU FW for streaming configuration and
+buffer management.
+
+
+- isp4_capture - Frames Capture Video Node
+
+------------------------------------------
+
+Isp4_capture is a capture device to capture frames to memory.
+The entity is connected to isp4 sub-device. The video device
+provides web camera like interface to userspace. It supports
+mmap and dma buf types of memory.
+
+Capturing Video Frames Example
+==============================
+
+.. code-block:: bash
+
+ v4l2-ctl "-d" "/dev/video0" "--set-fmt-video=width=1920,height=1080,pixelformat=NV12" "--stream-mmap" "--stream-count=10"
diff --git a/Documentation/admin-guide/media/amdisp4.dot b/Documentation/admin-guide/media/amdisp4.dot
new file mode 100644
index 000000000000..978f30c1a31a
--- /dev/null
+++ b/Documentation/admin-guide/media/amdisp4.dot
@@ -0,0 +1,6 @@
+digraph board {
+ rankdir=TB
+ n00000001 [label="{{} | amd isp4\n | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000001:port0 -> n00000003 [style=bold]
+ n00000003 [label="Preview\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+}
diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
index 3bac5165b134..6027416e5373 100644
--- a/Documentation/admin-guide/media/v4l-drivers.rst
+++ b/Documentation/admin-guide/media/v4l-drivers.rst
@@ -9,6 +9,7 @@ Video4Linux (V4L) driver-specific documentation
.. toctree::
:maxdepth: 2
+ amdisp4-1
bttv
c3-isp
cafe_ccic
diff --git a/MAINTAINERS b/MAINTAINERS
index 8478789ac265..c34137e27b55 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1139,6 +1139,8 @@ M: Nirujogi Pratap <pratap.nirujogi@amd.com>
L: linux-media@vger.kernel.org
S: Maintained
T: git git://linuxtv.org/media.git
+F: Documentation/admin-guide/media/amdisp4-1.rst
+F: Documentation/admin-guide/media/amdisp4.dot
F: drivers/media/platform/amd/Kconfig
F: drivers/media/platform/amd/Makefile
F: drivers/media/platform/amd/isp4/Kconfig
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/7] Add AMD ISP4 driver
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
` (6 preceding siblings ...)
2025-09-11 10:08 ` [PATCH v4 7/7] Documentation: add documentation of AMD isp 4 driver Bin Du
@ 2025-09-19 3:24 ` Du, Bin
2025-09-19 12:23 ` Laurent Pinchart
7 siblings, 1 reply; 35+ messages in thread
From: Du, Bin @ 2025-09-19 3:24 UTC (permalink / raw)
To: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
sultan
Cc: pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao
Hi Laurent, Sakari,
Thank you for your previous review feedback, we have addressed all
comments and listed the detailed changes in this cover letter below.
Would you mind taking a look at the v4 series if you have time. Any
feedback would be greatly appreciated.
Regards,
Bin
On 9/11/2025 6:08 PM, Bin Du wrote:
> Hello,
>
> AMD ISP4 is the AMD image processing gen 4 which can be found in HP ZBook Ultra G1a 14 inch Mobile Workstation PC (Ryzen AI Max 300 Series)
> (https://ubuntu.com/certified/202411-36043)
> This patch series introduces the initial driver support for the AMD ISP4.
>
> Patch summary:
> - Powers up/off and initializes ISP HW
> - Configures and kicks off ISP FW
> - Interacts with APP using standard V4l2 interface by video node
> - Controls ISP HW and interacts with ISP FW to do image processing
> - Supports enum/set output image format and resolution
> - Supports queueing buffer from app and dequeuing ISP filled buffer to App
> - It is verified on qv4l2, cheese and qcam
> - It is verified together with following patches
> platform/x86: Add AMD ISP platform config (https://lore.kernel.org/all/20250514215623.522746-1-pratap.nirujogi@amd.com/)
> pinctrl: amd: isp411: Add amdisp GPIO pinctrl (https://github.com/torvalds/linux/commit/e97435ab09f3ad7b6a588dd7c4e45a96699bbb4a)
> drm/amd/amdgpu: Add GPIO resources required for amdisp (https://gitlab.freedesktop.org/agd5f/linux/-/commit/ad0f5966ed8297aa47b3184192b00b7379ae0758)
> drm/amd/amdgpu: Declare isp firmware binary file (https://gitlab.freedesktop.org/agd5f/linux/-/commit/35345917bc9f7c86152b270d9d93c220230b667f)
>
> AMD ISP4 Key features:
> - Processes bayer raw data from the connected sensor and output them to different YUV formats
> - Downscale input image to different output image resolution
> - Pipeline to do image processing on the input image including demosaic, denoise, 3A, etc.
>
> ----------
>
> Changes v3 -> v4:
>
> - Replace one mutex with guard mutex.
> - Remove unnecessary bus_info initialization of v4l2_capability.
> - Drop V4L2_CAP_IO_MC from capabilities of v4l2_capability.
> - Modify document with better SOC description.
> - Fix Test x86 failure in Media CI test https://linux-media.pages.freedesktop.org/-/users/patchwork/-/jobs/83470456/artifacts/report.htm
> - Modify some commit messages by describing changes in imperative mood.
> - Add media-ctl output in cover letter.
> - Create separated dedicated amdgpu patch to add declaration MODULE_FIRMWARE("amdgpu/isp_4_1_1.bin");
> - Fix typo errors and other cosmetic issues.
> - Add DRM_AMD_ISP dependency in Kconfig.
>
>
> Changes v2 -> v3:
>
> - All the dependent patches in other modules (drm/amd/amdgpu, platform/x86, pinctrl/amd) merged on upstream mainline kernel (https://github.com/torvalds/linux) v6.17.
> - Removed usage of amdgpu structs in ISP driver. Added helper functions in amdgpu accepting opaque params from ISP driver to allocate and release ISP GART buffers.
> - Moved sensor and MIPI phy control entirely into ISP FW instead of the previous hybrid approach controlling sensor from both FW and x86 (sensor driver).
> - Removed phy configuration and sensor binding as x86 (sensor driver) had relinquished the sensor control for ISP FW. With this approach the driver will be exposed as web camera like interface.
> - New FW with built-in sensor driver is submitted on upstream linux-firmware repo (https://gitlab.com/kernel-firmware/linux-firmware/).
> - Please note the new FW submitted is not directly compatible with OEM Kernel ISP4.0 (https://github.com/amd/Linux_ISP_Kernel/tree/4.0) and the previous ISP V2 patch series.
> - If intend to use the new FW, please rebuild OEM ISP4.0 Kernel with CONFIG_VIDEO_OV05C10=N and CONFIG_PINCTRL_AMDISP=Y.
> - Included critical fixes from Sultan Alsawaf branch (https://github.com/kerneltoast/kernel_x86_laptop.git) related to managing lifetime of isp buffers.
> media: amd: isp4: Add missing refcount tracking to mmap memop
> media: amd: isp4: Don't put or unmap the dmabuf when detaching
> media: amd: isp4: Don't increment refcount when dmabuf export fails
> media: amd: isp4: Fix possible use-after-free in isp4vid_vb2_put()
> media: amd: isp4: Always export a new dmabuf from get_dmabuf memop
> media: amd: isp4: Fix implicit dmabuf lifetime tracking
> media: amd: isp4: Fix possible use-after-free when putting implicit dmabuf
> media: amd: isp4: Simplify isp4vid_get_dmabuf() arguments
> media: amd: isp4: Move up buf->vaddr check in isp4vid_get_dmabuf()
> media: amd: isp4: Remove unused userptr memops
> media: amd: isp4: Add missing cleanup on error in isp4vid_vb2_alloc()
> media: amd: isp4: Release queued buffers on error in start_streaming
> - Addressed all code related upstream comments
> - Fix typo errors and other cosmetic issues.
>
>
> Changes v1 -> v2:
>
> - Fix media CI test errors and valid warnings
> - Reduce patch number in the series from 9 to 8 by merging MAINTAINERS adding patch to the first patch
> - In patch 5
> - do modification to use remote endpoint instead of local endpoint
> - use link frequency and port number as start phy parameter instead of extra added phy-id and phy-bit-rate property of endpoint
>
> ----------
>
> It passes v4l2 compliance test, the test reports for:
>
> (a) amd_isp_capture device /dev/video0
>
> Compliance test for amd_isp_capture device /dev/video0:
> -------------------------------------------------------
>
> atg@atg-HP-PV:~/bin$ ./v4l2-compliance -d /dev/video0
> v4l2-compliance 1.29.0-5348, 64 bits, 64-bit time_t
> v4l2-compliance SHA: 75e3f0e2c2cb 2025-03-17 18:12:17
>
> Compliance test for amd_isp_capture device /dev/video0:
>
> Driver Info:
> Driver name : amd_isp_capture
> Card type : amd_isp_capture
> Bus info : platform:amd_isp_capture
> Driver version : 6.14.0
> Capabilities : 0xa4200001
> Video Capture
> I/O MC
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x24200001
> Video Capture
> I/O MC
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : amd_isp_capture
> Model : amd_isp41_mdev
> Serial :
> Bus info : platform:amd_isp_capture
> Media version : 6.14.0
> Hardware revision: 0x00000000 (0)
> Driver version : 6.14.0
> Interface Info:
> ID : 0x03000005
> Type : V4L Video
> Entity Info:
> ID : 0x00000003 (3)
> Name : Preview
> Function : V4L2 I/O
> Pad 0x01000004 : 0: Sink
> Link 0x02000007: from remote pad 0x1000002 of entity 'amd isp4' (Image Signal Processor): Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
> test invalid ioctls: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 1 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls (Input 0):
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls (Input 0):
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK (Not Supported)
>
> Codec ioctls (Input 0):
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls (Input 0):
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test CREATE_BUFS maximum buffers: OK
> test VIDIOC_REMOVE_BUFS: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
> test blocking wait: OK
>
> Total for amd_isp_capture device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
>
> The media-ctl output of media device /dev/media0:
> -------------------------------------------------------
>
> atg@atg-HP-PV:~$ media-ctl -p -d /dev/media0
> Media controller API version 6.17.0
>
> Media device information
> ------------------------
> driver amd_isp_capture
> model amd_isp41_mdev
> serial
> bus info platform:amd_isp_capture
> hw revision 0x0
> driver version 6.17.0
>
> Device topology
> - entity 1: amd isp4 (1 pad, 1 link, 0 routes)
> type V4L2 subdev subtype Unknown flags 0
> pad0: Source
> -> "Preview":0 [ENABLED,IMMUTABLE]
>
> - entity 3: Preview (1 pad, 1 link)
> type Node subtype V4L flags 0
> device node name /dev/video0
> pad0: Sink
> <- "amd isp4":0 [ENABLED,IMMUTABLE]
>
> Please review and provide feedback.
>
> Many thanks,
>
> Bin Du (7):
> media: platform: amd: Introduce amd isp4 capture driver
> media: platform: amd: low level support for isp4 firmware
> media: platform: amd: Add isp4 fw and hw interface
> media: platform: amd: isp4 subdev and firmware loading handling added
> media: platform: amd: isp4 video node and buffers handling added
> media: platform: amd: isp4 debug fs logging and more descriptive
> errors
> Documentation: add documentation of AMD isp 4 driver
>
> Documentation/admin-guide/media/amdisp4-1.rst | 63 +
> Documentation/admin-guide/media/amdisp4.dot | 6 +
> .../admin-guide/media/v4l-drivers.rst | 1 +
> MAINTAINERS | 25 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 1 +
> drivers/media/platform/amd/Kconfig | 3 +
> drivers/media/platform/amd/Makefile | 3 +
> drivers/media/platform/amd/isp4/Kconfig | 13 +
> drivers/media/platform/amd/isp4/Makefile | 10 +
> drivers/media/platform/amd/isp4/isp4.c | 237 ++++
> drivers/media/platform/amd/isp4/isp4.h | 26 +
> drivers/media/platform/amd/isp4/isp4_debug.c | 272 ++++
> drivers/media/platform/amd/isp4/isp4_debug.h | 41 +
> .../platform/amd/isp4/isp4_fw_cmd_resp.h | 314 +++++
> drivers/media/platform/amd/isp4/isp4_hw_reg.h | 125 ++
> .../media/platform/amd/isp4/isp4_interface.c | 966 +++++++++++++
> .../media/platform/amd/isp4/isp4_interface.h | 149 ++
> drivers/media/platform/amd/isp4/isp4_subdev.c | 1197 ++++++++++++++++
> drivers/media/platform/amd/isp4/isp4_subdev.h | 133 ++
> drivers/media/platform/amd/isp4/isp4_video.c | 1207 +++++++++++++++++
> drivers/media/platform/amd/isp4/isp4_video.h | 87 ++
> 22 files changed, 4880 insertions(+)
> create mode 100644 Documentation/admin-guide/media/amdisp4-1.rst
> create mode 100644 Documentation/admin-guide/media/amdisp4.dot
> create mode 100644 drivers/media/platform/amd/Kconfig
> create mode 100644 drivers/media/platform/amd/Makefile
> create mode 100644 drivers/media/platform/amd/isp4/Kconfig
> create mode 100644 drivers/media/platform/amd/isp4/Makefile
> create mode 100644 drivers/media/platform/amd/isp4/isp4.c
> create mode 100644 drivers/media/platform/amd/isp4/isp4.h
> create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.c
> create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.h
> create mode 100644 drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
> create mode 100644 drivers/media/platform/amd/isp4/isp4_hw_reg.h
> create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.c
> create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.h
> create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.c
> create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.h
> create mode 100644 drivers/media/platform/amd/isp4/isp4_video.c
> create mode 100644 drivers/media/platform/amd/isp4/isp4_video.h
>
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/7] Add AMD ISP4 driver
2025-09-19 3:24 ` [PATCH v4 0/7] Add AMD ISP4 driver Du, Bin
@ 2025-09-19 12:23 ` Laurent Pinchart
2025-09-22 2:50 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Laurent Pinchart @ 2025-09-19 12:23 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, bryan.odonoghue, sakari.ailus,
prabhakar.mahadev-lad.rj, linux-media, linux-kernel, sultan,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao
Hi Bin,
On Fri, Sep 19, 2025 at 11:24:36AM +0800, Du, Bin wrote:
> Hi Laurent, Sakari,
>
> Thank you for your previous review feedback, we have addressed all
> comments and listed the detailed changes in this cover letter below.
> Would you mind taking a look at the v4 series if you have time. Any
> feedback would be greatly appreciated.
I'm really overloaded at the moment, so I can't guarantee any review
time frame. Sorry about that.
> On 9/11/2025 6:08 PM, Bin Du wrote:
> > Hello,
> >
> > AMD ISP4 is the AMD image processing gen 4 which can be found in HP ZBook Ultra G1a 14 inch Mobile Workstation PC (Ryzen AI Max 300 Series)
> > (https://ubuntu.com/certified/202411-36043)
> > This patch series introduces the initial driver support for the AMD ISP4.
> >
> > Patch summary:
> > - Powers up/off and initializes ISP HW
> > - Configures and kicks off ISP FW
> > - Interacts with APP using standard V4l2 interface by video node
> > - Controls ISP HW and interacts with ISP FW to do image processing
> > - Supports enum/set output image format and resolution
> > - Supports queueing buffer from app and dequeuing ISP filled buffer to App
> > - It is verified on qv4l2, cheese and qcam
> > - It is verified together with following patches
> > platform/x86: Add AMD ISP platform config (https://lore.kernel.org/all/20250514215623.522746-1-pratap.nirujogi@amd.com/)
> > pinctrl: amd: isp411: Add amdisp GPIO pinctrl (https://github.com/torvalds/linux/commit/e97435ab09f3ad7b6a588dd7c4e45a96699bbb4a)
> > drm/amd/amdgpu: Add GPIO resources required for amdisp (https://gitlab.freedesktop.org/agd5f/linux/-/commit/ad0f5966ed8297aa47b3184192b00b7379ae0758)
> > drm/amd/amdgpu: Declare isp firmware binary file (https://gitlab.freedesktop.org/agd5f/linux/-/commit/35345917bc9f7c86152b270d9d93c220230b667f)
> >
> > AMD ISP4 Key features:
> > - Processes bayer raw data from the connected sensor and output them to different YUV formats
> > - Downscale input image to different output image resolution
> > - Pipeline to do image processing on the input image including demosaic, denoise, 3A, etc.
> >
> > ----------
> >
> > Changes v3 -> v4:
> >
> > - Replace one mutex with guard mutex.
> > - Remove unnecessary bus_info initialization of v4l2_capability.
> > - Drop V4L2_CAP_IO_MC from capabilities of v4l2_capability.
> > - Modify document with better SOC description.
> > - Fix Test x86 failure in Media CI test https://linux-media.pages.freedesktop.org/-/users/patchwork/-/jobs/83470456/artifacts/report.htm
> > - Modify some commit messages by describing changes in imperative mood.
> > - Add media-ctl output in cover letter.
> > - Create separated dedicated amdgpu patch to add declaration MODULE_FIRMWARE("amdgpu/isp_4_1_1.bin");
> > - Fix typo errors and other cosmetic issues.
> > - Add DRM_AMD_ISP dependency in Kconfig.
> >
> >
> > Changes v2 -> v3:
> >
> > - All the dependent patches in other modules (drm/amd/amdgpu, platform/x86, pinctrl/amd) merged on upstream mainline kernel (https://github.com/torvalds/linux) v6.17.
> > - Removed usage of amdgpu structs in ISP driver. Added helper functions in amdgpu accepting opaque params from ISP driver to allocate and release ISP GART buffers.
> > - Moved sensor and MIPI phy control entirely into ISP FW instead of the previous hybrid approach controlling sensor from both FW and x86 (sensor driver).
> > - Removed phy configuration and sensor binding as x86 (sensor driver) had relinquished the sensor control for ISP FW. With this approach the driver will be exposed as web camera like interface.
> > - New FW with built-in sensor driver is submitted on upstream linux-firmware repo (https://gitlab.com/kernel-firmware/linux-firmware/).
> > - Please note the new FW submitted is not directly compatible with OEM Kernel ISP4.0 (https://github.com/amd/Linux_ISP_Kernel/tree/4.0) and the previous ISP V2 patch series.
> > - If intend to use the new FW, please rebuild OEM ISP4.0 Kernel with CONFIG_VIDEO_OV05C10=N and CONFIG_PINCTRL_AMDISP=Y.
> > - Included critical fixes from Sultan Alsawaf branch (https://github.com/kerneltoast/kernel_x86_laptop.git) related to managing lifetime of isp buffers.
> > media: amd: isp4: Add missing refcount tracking to mmap memop
> > media: amd: isp4: Don't put or unmap the dmabuf when detaching
> > media: amd: isp4: Don't increment refcount when dmabuf export fails
> > media: amd: isp4: Fix possible use-after-free in isp4vid_vb2_put()
> > media: amd: isp4: Always export a new dmabuf from get_dmabuf memop
> > media: amd: isp4: Fix implicit dmabuf lifetime tracking
> > media: amd: isp4: Fix possible use-after-free when putting implicit dmabuf
> > media: amd: isp4: Simplify isp4vid_get_dmabuf() arguments
> > media: amd: isp4: Move up buf->vaddr check in isp4vid_get_dmabuf()
> > media: amd: isp4: Remove unused userptr memops
> > media: amd: isp4: Add missing cleanup on error in isp4vid_vb2_alloc()
> > media: amd: isp4: Release queued buffers on error in start_streaming
> > - Addressed all code related upstream comments
> > - Fix typo errors and other cosmetic issues.
> >
> >
> > Changes v1 -> v2:
> >
> > - Fix media CI test errors and valid warnings
> > - Reduce patch number in the series from 9 to 8 by merging MAINTAINERS adding patch to the first patch
> > - In patch 5
> > - do modification to use remote endpoint instead of local endpoint
> > - use link frequency and port number as start phy parameter instead of extra added phy-id and phy-bit-rate property of endpoint
> >
> > ----------
> >
> > It passes v4l2 compliance test, the test reports for:
> >
> > (a) amd_isp_capture device /dev/video0
> >
> > Compliance test for amd_isp_capture device /dev/video0:
> > -------------------------------------------------------
> >
> > atg@atg-HP-PV:~/bin$ ./v4l2-compliance -d /dev/video0
> > v4l2-compliance 1.29.0-5348, 64 bits, 64-bit time_t
> > v4l2-compliance SHA: 75e3f0e2c2cb 2025-03-17 18:12:17
> >
> > Compliance test for amd_isp_capture device /dev/video0:
> >
> > Driver Info:
> > Driver name : amd_isp_capture
> > Card type : amd_isp_capture
> > Bus info : platform:amd_isp_capture
> > Driver version : 6.14.0
> > Capabilities : 0xa4200001
> > Video Capture
> > I/O MC
> > Streaming
> > Extended Pix Format
> > Device Capabilities
> > Device Caps : 0x24200001
> > Video Capture
> > I/O MC
> > Streaming
> > Extended Pix Format
> > Media Driver Info:
> > Driver name : amd_isp_capture
> > Model : amd_isp41_mdev
> > Serial :
> > Bus info : platform:amd_isp_capture
> > Media version : 6.14.0
> > Hardware revision: 0x00000000 (0)
> > Driver version : 6.14.0
> > Interface Info:
> > ID : 0x03000005
> > Type : V4L Video
> > Entity Info:
> > ID : 0x00000003 (3)
> > Name : Preview
> > Function : V4L2 I/O
> > Pad 0x01000004 : 0: Sink
> > Link 0x02000007: from remote pad 0x1000002 of entity 'amd isp4' (Image Signal Processor): Data, Enabled, Immutable
> >
> > Required ioctls:
> > test MC information (see 'Media Driver Info' above): OK
> > test VIDIOC_QUERYCAP: OK
> > test invalid ioctls: OK
> >
> > Allow for multiple opens:
> > test second /dev/video0 open: OK
> > test VIDIOC_QUERYCAP: OK
> > test VIDIOC_G/S_PRIORITY: OK
> > test for unlimited opens: OK
> >
> > Debug ioctls:
> > test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> > test VIDIOC_LOG_STATUS: OK (Not Supported)
> >
> > Input ioctls:
> > test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> > test VIDIOC_ENUMAUDIO: OK (Not Supported)
> > test VIDIOC_G/S/ENUMINPUT: OK
> > test VIDIOC_G/S_AUDIO: OK (Not Supported)
> > Inputs: 1 Audio Inputs: 0 Tuners: 0
> >
> > Output ioctls:
> > test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> > test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> > test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> > test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> > test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> > Outputs: 0 Audio Outputs: 0 Modulators: 0
> >
> > Input/Output configuration ioctls:
> > test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> > test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> > test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> > test VIDIOC_G/S_EDID: OK (Not Supported)
> >
> > Control ioctls (Input 0):
> > test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> > test VIDIOC_QUERYCTRL: OK (Not Supported)
> > test VIDIOC_G/S_CTRL: OK (Not Supported)
> > test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> > test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> > test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> > Standard Controls: 0 Private Controls: 0
> >
> > Format ioctls (Input 0):
> > test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> > test VIDIOC_G/S_PARM: OK
> > test VIDIOC_G_FBUF: OK (Not Supported)
> > test VIDIOC_G_FMT: OK
> > test VIDIOC_TRY_FMT: OK
> > test VIDIOC_S_FMT: OK
> > test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> > test Cropping: OK (Not Supported)
> > test Composing: OK (Not Supported)
> > test Scaling: OK (Not Supported)
> >
> > Codec ioctls (Input 0):
> > test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> > test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> > test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
> >
> > Buffer ioctls (Input 0):
> > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> > test CREATE_BUFS maximum buffers: OK
> > test VIDIOC_REMOVE_BUFS: OK
> > test VIDIOC_EXPBUF: OK
> > test Requests: OK (Not Supported)
> > test blocking wait: OK
> >
> > Total for amd_isp_capture device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
> >
> > The media-ctl output of media device /dev/media0:
> > -------------------------------------------------------
> >
> > atg@atg-HP-PV:~$ media-ctl -p -d /dev/media0
> > Media controller API version 6.17.0
> >
> > Media device information
> > ------------------------
> > driver amd_isp_capture
> > model amd_isp41_mdev
> > serial
> > bus info platform:amd_isp_capture
> > hw revision 0x0
> > driver version 6.17.0
> >
> > Device topology
> > - entity 1: amd isp4 (1 pad, 1 link, 0 routes)
> > type V4L2 subdev subtype Unknown flags 0
> > pad0: Source
> > -> "Preview":0 [ENABLED,IMMUTABLE]
> >
> > - entity 3: Preview (1 pad, 1 link)
> > type Node subtype V4L flags 0
> > device node name /dev/video0
> > pad0: Sink
> > <- "amd isp4":0 [ENABLED,IMMUTABLE]
> >
> > Please review and provide feedback.
> >
> > Many thanks,
> >
> > Bin Du (7):
> > media: platform: amd: Introduce amd isp4 capture driver
> > media: platform: amd: low level support for isp4 firmware
> > media: platform: amd: Add isp4 fw and hw interface
> > media: platform: amd: isp4 subdev and firmware loading handling added
> > media: platform: amd: isp4 video node and buffers handling added
> > media: platform: amd: isp4 debug fs logging and more descriptive
> > errors
> > Documentation: add documentation of AMD isp 4 driver
> >
> > Documentation/admin-guide/media/amdisp4-1.rst | 63 +
> > Documentation/admin-guide/media/amdisp4.dot | 6 +
> > .../admin-guide/media/v4l-drivers.rst | 1 +
> > MAINTAINERS | 25 +
> > drivers/media/platform/Kconfig | 1 +
> > drivers/media/platform/Makefile | 1 +
> > drivers/media/platform/amd/Kconfig | 3 +
> > drivers/media/platform/amd/Makefile | 3 +
> > drivers/media/platform/amd/isp4/Kconfig | 13 +
> > drivers/media/platform/amd/isp4/Makefile | 10 +
> > drivers/media/platform/amd/isp4/isp4.c | 237 ++++
> > drivers/media/platform/amd/isp4/isp4.h | 26 +
> > drivers/media/platform/amd/isp4/isp4_debug.c | 272 ++++
> > drivers/media/platform/amd/isp4/isp4_debug.h | 41 +
> > .../platform/amd/isp4/isp4_fw_cmd_resp.h | 314 +++++
> > drivers/media/platform/amd/isp4/isp4_hw_reg.h | 125 ++
> > .../media/platform/amd/isp4/isp4_interface.c | 966 +++++++++++++
> > .../media/platform/amd/isp4/isp4_interface.h | 149 ++
> > drivers/media/platform/amd/isp4/isp4_subdev.c | 1197 ++++++++++++++++
> > drivers/media/platform/amd/isp4/isp4_subdev.h | 133 ++
> > drivers/media/platform/amd/isp4/isp4_video.c | 1207 +++++++++++++++++
> > drivers/media/platform/amd/isp4/isp4_video.h | 87 ++
> > 22 files changed, 4880 insertions(+)
> > create mode 100644 Documentation/admin-guide/media/amdisp4-1.rst
> > create mode 100644 Documentation/admin-guide/media/amdisp4.dot
> > create mode 100644 drivers/media/platform/amd/Kconfig
> > create mode 100644 drivers/media/platform/amd/Makefile
> > create mode 100644 drivers/media/platform/amd/isp4/Kconfig
> > create mode 100644 drivers/media/platform/amd/isp4/Makefile
> > create mode 100644 drivers/media/platform/amd/isp4/isp4.c
> > create mode 100644 drivers/media/platform/amd/isp4/isp4.h
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.c
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.h
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_hw_reg.h
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.c
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.h
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.c
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.h
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_video.c
> > create mode 100644 drivers/media/platform/amd/isp4/isp4_video.h
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver
2025-09-11 10:08 ` [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver Bin Du
@ 2025-09-21 20:23 ` Sultan Alsawaf
2025-09-23 7:56 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-09-21 20:23 UTC (permalink / raw)
To: Bin Du
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Svetoslav Stoilov, Mario Limonciello,
Alexey Zagorodnikov
Hi Bin,
On Thu, Sep 11, 2025 at 06:08:41PM +0800, Bin Du wrote:
> AMD isp4 capture is a v4l2 media device which implements media controller
> interface. It has one sub-device (AMD ISP4 sub-device) endpoint which can
> be connected to a remote CSI2 TX endpoint. It supports only one physical
> interface for now. Also add ISP4 driver related entry info into the
> MAINTAINERS file
>
> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Bin Du <Bin.Du@amd.com>
> Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
[snip]
> +++ b/drivers/media/platform/amd/isp4/Kconfig
> @@ -0,0 +1,13 @@
> +# SPDX-License-Identifier: GPL-2.0+
> +
> +config AMD_ISP4
> + tristate "AMD ISP4 and camera driver"
> + depends on VIDEO_DEV && VIDEO_V4L2_SUBDEV_API && DRM_AMD_ISP
> + select VIDEOBUF2_CORE
> + select VIDEOBUF2_V4L2
> + select VIDEOBUF2_MEMOPS
VIDEO_V4L2_SUBDEV_API should be selected rather than set as a dependency, per
what other drivers do. You can also sort the dependencies to look cleaner.
Change to:
depends on VIDEO_DEV && DRM_AMD_ISP
select VIDEOBUF2_CORE
select VIDEOBUF2_MEMOPS
select VIDEOBUF2_V4L2
select VIDEO_V4L2_SUBDEV_API
> + help
> + This is support for AMD ISP4 and camera subsystem driver.
> + Say Y here to enable the ISP4 and camera device for video capture.
> + To compile this driver as a module, choose M here. The module will
> + be called amd_capture.
[snip]
> +++ b/drivers/media/platform/amd/isp4/isp4.c
> @@ -0,0 +1,121 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#include <linux/pm_runtime.h>
> +#include <linux/vmalloc.h>
> +#include <media/v4l2-ioctl.h>
> +
> +#include "isp4.h"
> +
> +#define VIDEO_BUF_NUM 5
> +
> +#define ISP4_DRV_NAME "amd_isp_capture"
> +
> +/* interrupt num */
> +static const u32 isp4_ringbuf_interrupt_num[] = {
> + 0, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT9 */
> + 1, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT10 */
> + 3, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT11 */
> + 4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
> +};
> +
> +#define to_isp4_device(dev) \
> + ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
The unnecessary cast on container_of() is removed later in "[PATCH v4 4/7]
media: platform: amd: isp4 subdev and firmware loading handling added".
Remove the cast in this patch instead.
> +
> +static irqreturn_t isp4_irq_handler(int irq, void *arg)
> +{
> + return IRQ_HANDLED;
> +}
> +
> +static int isp4_capture_probe(struct platform_device *pdev)
> +{
> + struct device *dev = &pdev->dev;
> + struct isp4_device *isp_dev;
> + int i, irq, ret;
> +
> + isp_dev = devm_kzalloc(&pdev->dev, sizeof(*isp_dev), GFP_KERNEL);
s/&pdev->dev/dev/
> + if (!isp_dev)
> + return -ENOMEM;
> +
> + isp_dev->pdev = pdev;
> + dev->init_name = ISP4_DRV_NAME;
> +
> + for (i = 0; i < ARRAY_SIZE(isp4_ringbuf_interrupt_num); i++) {
> + irq = platform_get_irq(pdev, isp4_ringbuf_interrupt_num[i]);
> + if (irq < 0)
> + return dev_err_probe(dev, -ENODEV,
> + "fail to get irq %d\n",
> + isp4_ringbuf_interrupt_num[i]);
Return the error from platform_get_irq() here instead of -ENODEV.
> + ret = devm_request_irq(&pdev->dev, irq, isp4_irq_handler, 0,
> + "ISP_IRQ", &pdev->dev);
s/&pdev->dev/dev/
> + if (ret)
> + return dev_err_probe(dev, ret, "fail to req irq %d\n",
> + irq);
> + }
> +
> + /* Link the media device within the v4l2_device */
> + isp_dev->v4l2_dev.mdev = &isp_dev->mdev;
> +
> + /* Initialize media device */
> + strscpy(isp_dev->mdev.model, "amd_isp41_mdev",
> + sizeof(isp_dev->mdev.model));
> + snprintf(isp_dev->mdev.bus_info, sizeof(isp_dev->mdev.bus_info),
> + "platform:%s", ISP4_DRV_NAME);
> + isp_dev->mdev.dev = &pdev->dev;
s/&pdev->dev/dev/
> + media_device_init(&isp_dev->mdev);
> +
> + /* register v4l2 device */
> + snprintf(isp_dev->v4l2_dev.name, sizeof(isp_dev->v4l2_dev.name),
> + "AMD-V4L2-ROOT");
> + ret = v4l2_device_register(&pdev->dev, &isp_dev->v4l2_dev);
s/&pdev->dev/dev/
> + if (ret)
> + return dev_err_probe(dev, ret,
> + "fail register v4l2 device\n");
> +
> + ret = media_device_register(&isp_dev->mdev);
> + if (ret) {
> + dev_err(dev, "fail to register media device %d\n", ret);
> + goto err_unreg_v4l2;
> + }
> +
> + platform_set_drvdata(pdev, isp_dev);
> +
> + pm_runtime_set_suspended(dev);
> + pm_runtime_enable(dev);
> +
> + return 0;
> +
> +err_unreg_v4l2:
> + v4l2_device_unregister(&isp_dev->v4l2_dev);
> +
> + return dev_err_probe(dev, ret, "isp probe fail\n");
> +}
[snip]
> +++ b/drivers/media/platform/amd/isp4/isp4.h
> @@ -0,0 +1,24 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#ifndef _ISP4_H_
> +#define _ISP4_H_
> +
> +#include <linux/mutex.h>
Remove this linux/mutex.h include. It should be moved to isp4_subdev.h instead.
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-memops.h>
> +#include <media/videobuf2-vmalloc.h>
This media/videobuf2-vmalloc.h include is removed in "[PATCH v4 4/7] media:
platform: amd: isp4 subdev and firmware loading handling added".
Remove it in this patch instead.
> +
> +#define ISP4_GET_ISP_REG_BASE(isp4sd) (((isp4sd))->mmio)
> +
> +struct isp4_device {
> + struct v4l2_device v4l2_dev;
> + struct media_device mdev;
> +
> + struct platform_device *pdev;
> + struct notifier_block i2c_nb;
i2c_nb is unused, remove it.
> +};
> +
> +#endif /* _ISP4_H_ */
> --
> 2.34.1
>
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 2/7] media: platform: amd: low level support for isp4 firmware
2025-09-11 10:08 ` [PATCH v4 2/7] media: platform: amd: low level support for isp4 firmware Bin Du
@ 2025-09-21 20:31 ` Sultan Alsawaf
2025-09-23 8:05 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-09-21 20:31 UTC (permalink / raw)
To: Bin Du
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Hi Bin,
On Thu, Sep 11, 2025 at 06:08:42PM +0800, Bin Du wrote:
> Low level functions for accessing the registers and mapping to their
> ranges. This change also includes register definitions for ring buffer
> used to communicate with ISP Firmware. Ring buffer is the communication
> interface between driver and ISP Firmware. Command and responses are
> exchanged through the ring buffer.
>
> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Bin Du <Bin.Du@amd.com>
> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
[snip]
> +++ b/drivers/media/platform/amd/isp4/Makefile
> @@ -3,4 +3,4 @@
> # Copyright (C) 2025 Advanced Micro Devices, Inc.
>
> obj-$(CONFIG_AMD_ISP4) += amd_capture.o
> -amd_capture-objs := isp4.o
> +amd_capture-objs := isp4.o \
Remove this hunk since there are no new objects added in this patch.
> +++ b/drivers/media/platform/amd/isp4/isp4_hw_reg.h
> @@ -0,0 +1,125 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#ifndef _ISP4_HW_REG_H_
> +#define _ISP4_HW_REG_H_
> +
> +#include <linux/io.h>
> +#include <linux/types.h>
Remove redundant linux/types.h include, as it is included by linux/io.h.
> +
> +#define ISP_SOFT_RESET 0x62000
[snip]
> +
> +#endif
Add /* _ISP4_HW_REG_H_ */
> --
> 2.34.1
>
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface
2025-09-11 10:08 ` [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface Bin Du
@ 2025-09-21 21:55 ` Sultan Alsawaf
2025-09-23 9:24 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-09-21 21:55 UTC (permalink / raw)
To: Bin Du
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Hi Bin,
On Thu, Sep 11, 2025 at 06:08:43PM +0800, Bin Du wrote:
> ISP firmware controls ISP HW pipeline using dedicated embedded processor
> called ccpu. The communication between ISP FW and driver is using commands
> and response messages sent through the ring buffer. Command buffers support
> either global setting that is not specific to the steam and support stream
> specific parameters. Response buffers contain ISP FW notification
> information such as frame buffer done and command done. IRQ is used for
> receiving response buffer from ISP firmware, which is handled in the main
> isp4 media device. ISP ccpu is booted up through the firmware loading
> helper function prior to stream start. Memory used for command buffer and
> response buffer needs to be allocated from amdgpu buffer manager because
> isp4 is a child device of amdgpu.
>
> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Bin Du <Bin.Du@amd.com>
> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
[snip]
> +++ b/drivers/media/platform/amd/isp4/Makefile
> @@ -4,3 +4,4 @@
>
> obj-$(CONFIG_AMD_ISP4) += amd_capture.o
> amd_capture-objs := isp4.o \
> + isp4_interface.o \
Align the objects with spaces and remove the trailing backslash.
I.e., change to:
amd_capture-objs := isp4_subdev.o \
isp4_interface.o
> +++ b/drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
> @@ -0,0 +1,314 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#ifndef _ISP4_CMD_RESP_H_
> +#define _ISP4_CMD_RESP_H_
> +
[snip]
> +
> +#endif
Add /* _ISP4_CMD_RESP_H_ */
> diff --git a/drivers/media/platform/amd/isp4/isp4_interface.c b/drivers/media/platform/amd/isp4/isp4_interface.c
> new file mode 100644
> index 000000000000..52dcca57ce2e
> --- /dev/null
> +++ b/drivers/media/platform/amd/isp4/isp4_interface.c
> @@ -0,0 +1,955 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#include <drm/amd/isp.h>
> +#include <linux/iopoll.h>
> +#include <linux/mutex.h>
Remove this linux/mutex.h include, it should come from isp4_interface.h instead
since it is used for a mutex from a isp4_interface.h struct.
> +#include <linux/platform_device.h>
> +
> +#include "isp4_fw_cmd_resp.h"
> +#include "isp4_hw_reg.h"
> +#include "isp4_interface.h"
> +
> +#define ISP4IF_FW_RESP_RB_IRQ_EN_MASK \
> + (ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT9_EN_MASK | \
> + ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT10_EN_MASK | \
> + ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT11_EN_MASK | \
> + ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT12_EN_MASK)
> +
> +struct isp4if_rb_config {
> + const char *name;
> + u32 index;
> + u32 reg_rptr;
> + u32 reg_wptr;
> + u32 reg_base_lo;
> + u32 reg_base_hi;
> + u32 reg_size;
> + u32 val_size;
> + u64 base_mc_addr;
> + void *base_sys_addr;
> +};
> +
> +/* FW cmd ring buffer configuration */
> +static struct isp4if_rb_config
> + isp4if_cmd_rb_config[ISP4IF_STREAM_ID_MAX] = {
> + {
> + .name = "CMD_RB_GBL0",
> + .index = 3,
> + .reg_rptr = ISP_RB_RPTR4,
> + .reg_wptr = ISP_RB_WPTR4,
> + .reg_base_lo = ISP_RB_BASE_LO4,
> + .reg_base_hi = ISP_RB_BASE_HI4,
> + .reg_size = ISP_RB_SIZE4,
> + },
> + {
> + .name = "CMD_RB_STR1",
> + .index = 0,
> + .reg_rptr = ISP_RB_RPTR1,
> + .reg_wptr = ISP_RB_WPTR1,
> + .reg_base_lo = ISP_RB_BASE_LO1,
> + .reg_base_hi = ISP_RB_BASE_HI1,
> + .reg_size = ISP_RB_SIZE1,
> + },
> + {
> + .name = "CMD_RB_STR2",
> + .index = 1,
> + .reg_rptr = ISP_RB_RPTR2,
> + .reg_wptr = ISP_RB_WPTR2,
> + .reg_base_lo = ISP_RB_BASE_LO2,
> + .reg_base_hi = ISP_RB_BASE_HI2,
> + .reg_size = ISP_RB_SIZE2,
> + },
> + {
> + .name = "CMD_RB_STR3",
> + .index = 2,
> + .reg_rptr = ISP_RB_RPTR3,
> + .reg_wptr = ISP_RB_WPTR3,
> + .reg_base_lo = ISP_RB_BASE_LO3,
> + .reg_base_hi = ISP_RB_BASE_HI3,
> + .reg_size = ISP_RB_SIZE3,
> + },
> +};
> +
> +/* FW resp ring buffer configuration */
> +static struct isp4if_rb_config
> + isp4if_resp_rb_config[ISP4IF_STREAM_ID_MAX] = {
> + {
> + .name = "RES_RB_GBL0",
> + .index = 3,
> + .reg_rptr = ISP_RB_RPTR12,
> + .reg_wptr = ISP_RB_WPTR12,
> + .reg_base_lo = ISP_RB_BASE_LO12,
> + .reg_base_hi = ISP_RB_BASE_HI12,
> + .reg_size = ISP_RB_SIZE12,
> + },
> + {
> + .name = "RES_RB_STR1",
> + .index = 0,
> + .reg_rptr = ISP_RB_RPTR9,
> + .reg_wptr = ISP_RB_WPTR9,
> + .reg_base_lo = ISP_RB_BASE_LO9,
> + .reg_base_hi = ISP_RB_BASE_HI9,
> + .reg_size = ISP_RB_SIZE9,
> + },
> + {
> + .name = "RES_RB_STR2",
> + .index = 1,
> + .reg_rptr = ISP_RB_RPTR10,
> + .reg_wptr = ISP_RB_WPTR10,
> + .reg_base_lo = ISP_RB_BASE_LO10,
> + .reg_base_hi = ISP_RB_BASE_HI10,
> + .reg_size = ISP_RB_SIZE10,
> + },
> + {
> + .name = "RES_RB_STR3",
> + .index = 2,
> + .reg_rptr = ISP_RB_RPTR11,
> + .reg_wptr = ISP_RB_WPTR11,
> + .reg_base_lo = ISP_RB_BASE_LO11,
> + .reg_base_hi = ISP_RB_BASE_HI11,
> + .reg_size = ISP_RB_SIZE11,
> + },
> +};
> +
> +/* FW log ring buffer configuration */
> +static struct isp4if_rb_config isp4if_log_rb_config = {
> + .name = "LOG_RB",
> + .index = 0,
> + .reg_rptr = ISP_LOG_RB_RPTR0,
> + .reg_wptr = ISP_LOG_RB_WPTR0,
> + .reg_base_lo = ISP_LOG_RB_BASE_LO0,
> + .reg_base_hi = ISP_LOG_RB_BASE_HI0,
> + .reg_size = ISP_LOG_RB_SIZE0,
> +};
> +
> +static struct isp4if_gpu_mem_info *isp4if_gpu_mem_alloc(struct isp4_interface *ispif, u32 mem_size)
> +{
> + struct isp4if_gpu_mem_info *mem_info;
> + struct device *dev = ispif->dev;
> + int ret;
> +
> + if (!mem_size)
> + return NULL;
mem_size is never zero, remove this check.
> +
> + mem_info = kzalloc(sizeof(*mem_info), GFP_KERNEL);
No need for kzalloc, use kmalloc here instead.
> + if (!mem_info)
> + return NULL;
> +
> + mem_info->mem_size = mem_size;
mem_info->mem_size is not used anywhere, remove it.
> + ret = isp_kernel_buffer_alloc(dev, mem_info->mem_size, &mem_info->mem_handle,
> + &mem_info->gpu_mc_addr, &mem_info->sys_addr);
> + if (ret) {
> + kfree(mem_info);
> + return NULL;
> + }
> +
> + return mem_info;
> +}
[snip]
> +static int isp4if_gpu_mem_free(struct isp4_interface *ispif, struct isp4if_gpu_mem_info *mem_info)
> +{
> + struct device *dev = ispif->dev;
> +
> + if (!mem_info) {
> + dev_err(dev, "invalid mem_info\n");
> + return -EINVAL;
> + }
> +
> + isp_kernel_buffer_free(&mem_info->mem_handle, &mem_info->gpu_mc_addr, &mem_info->sys_addr);
> +
> + kfree(mem_info);
> +
> + return 0;
> +}
> +
> +static int isp4if_dealloc_fw_gpumem(struct isp4_interface *ispif)
> +{
> + int i;
> +
> + if (ispif->fw_mem_pool) {
> + isp4if_gpu_mem_free(ispif, ispif->fw_mem_pool);
> + ispif->fw_mem_pool = NULL;
> + }
> +
> + if (ispif->fw_cmd_resp_buf) {
> + isp4if_gpu_mem_free(ispif, ispif->fw_cmd_resp_buf);
> + ispif->fw_cmd_resp_buf = NULL;
> + }
> +
> + if (ispif->fw_log_buf) {
> + isp4if_gpu_mem_free(ispif, ispif->fw_log_buf);
> + ispif->fw_log_buf = NULL;
> + }
> +
> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
> + if (ispif->metainfo_buf_pool[i]) {
> + isp4if_gpu_mem_free(ispif, ispif->metainfo_buf_pool[i]);
> + ispif->metainfo_buf_pool[i] = NULL;
> + }
> + }
> +
> + return 0;
> +}
isp4if_gpu_mem_free() and isp4if_dealloc_fw_gpumem() can be simplified and made
more robust against copy+paste errors, and their return values are not used
anywhere. Plus, the mem_info argument to isp4if_gpu_mem_free() is never NULL, so
there is redundant NULL pointer checking.
Change to the following:
static void isp4if_gpu_mem_free(struct isp4if_gpu_mem_info **mem_info_ptr)
{
struct isp4if_gpu_mem_info *mem_info = *mem_info_ptr;
if (!mem_info)
return;
*mem_info_ptr = NULL;
isp_kernel_buffer_free(&mem_info->mem_handle, &mem_info->gpu_mc_addr,
&mem_info->sys_addr);
kfree(mem_info);
}
static void isp4if_dealloc_fw_gpumem(struct isp4_interface *ispif)
{
int i;
isp4if_gpu_mem_free(&ispif->fw_mem_pool);
isp4if_gpu_mem_free(&ispif->fw_cmd_resp_buf);
isp4if_gpu_mem_free(&ispif->fw_log_buf);
for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++)
isp4if_gpu_mem_free(&ispif->metainfo_buf_pool[i]);
}
> +static int isp4if_alloc_fw_gpumem(struct isp4_interface *ispif)
> +{
> + struct device *dev = ispif->dev;
> + unsigned int i;
`i` doesn't need to be unsigned. Remove unsigned to make it consistent with
other ISP4IF_MAX_STREAM_BUF_COUNT loops.
> +
> + ispif->fw_mem_pool = isp4if_gpu_mem_alloc(ispif, FW_MEMORY_POOL_SIZE);
> + if (!ispif->fw_mem_pool)
> + goto error_no_memory;
> +
> + ispif->fw_cmd_resp_buf =
> + isp4if_gpu_mem_alloc(ispif, ISP4IF_RB_PMBMAP_MEM_SIZE);
> + if (!ispif->fw_cmd_resp_buf)
> + goto error_no_memory;
> +
> + ispif->fw_log_buf =
> + isp4if_gpu_mem_alloc(ispif, ISP4IF_FW_LOG_RINGBUF_SIZE);
> + if (!ispif->fw_log_buf)
> + goto error_no_memory;
> +
> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
> + ispif->metainfo_buf_pool[i] =
> + isp4if_gpu_mem_alloc(ispif,
> + ISP4IF_META_INFO_BUF_SIZE);
> + if (!ispif->metainfo_buf_pool[i])
> + goto error_no_memory;
> + }
> +
> + return 0;
> +
> +error_no_memory:
> + dev_err(dev, "failed to allocate gpu memory\n");
> + return -ENOMEM;
> +}
> +
> +static u32 isp4if_compute_check_sum(u8 *buf, u32 buf_size)
Change `u8 *buf` to `const u8 *buf`.
Change `u32 buf_size` to `size_t buf_size` just to be consistent with buf_size
coming from sizeof().
> +{
> + u32 checksum = 0;
> + u8 *surplus_ptr;
> + u32 *buffer;
> + u32 i;
Change the 3 variables above to:
const u8 *surplus_ptr;
const u32 *buffer;
size_t i;
> +
> + buffer = (u32 *)buf;
Change cast to `(const u32 *)`
> + for (i = 0; i < buf_size / sizeof(u32); i++)
> + checksum += buffer[i];
> +
> + surplus_ptr = (u8 *)&buffer[i];
Change cast to `(const u32 *)`
> + /* add surplus data crc checksum */
> + for (i = 0; i < buf_size % sizeof(u32); i++)
> + checksum += surplus_ptr[i];
> +
> + return checksum;
> +}
> +
> +void isp4if_clear_cmdq(struct isp4_interface *ispif)
> +{
> + struct isp4if_cmd_element *buf_node = NULL;
> + struct isp4if_cmd_element *tmp_node = NULL;
Remove unnecessary initialization of buf_node and tmp_node.
> +
> + guard(mutex)(&ispif->cmdq_mutex);
> +
> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
> + list_del(&buf_node->list);
> + kfree(buf_node);
> + }
Move the whole list to a local LIST_HEAD(free_list) variable and then release
the lock. Then you can list_for_each_entry_safe() without needing to do a
list_del() every time, and you won't need to hold the lock the whole time.
> +}
[snip]
> +static struct isp4if_cmd_element *isp4if_append_cmd_2_cmdq(struct isp4_interface *ispif,
> + struct isp4if_cmd_element *cmd_ele)
> +{
> + struct isp4if_cmd_element *copy_command = NULL;
Remove unnecessary initialization of copy_command.
> +
> + copy_command = kmemdup(cmd_ele, sizeof(*cmd_ele), GFP_KERNEL);
> + if (!copy_command)
> + return NULL;
> +
> + guard(mutex)(&ispif->cmdq_mutex);
> +
> + list_add_tail(©_command->list, &ispif->cmdq);
> +
> + return copy_command;
> +}
> +
> +struct isp4if_cmd_element *isp4if_rm_cmd_from_cmdq(struct isp4_interface *ispif, u32 seq_num,
> + u32 cmd_id)
> +{
> + struct isp4if_cmd_element *buf_node = NULL;
> + struct isp4if_cmd_element *tmp_node = NULL;
Remove unnecessary initialization of buf_node and tmp_node.
> +
> + guard(mutex)(&ispif->cmdq_mutex);
> +
> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
> + if (buf_node->seq_num == seq_num &&
> + buf_node->cmd_id == cmd_id) {
> + list_del(&buf_node->list);
> + return buf_node;
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static int isp4if_insert_isp_fw_cmd(struct isp4_interface *ispif, enum isp4if_stream_id stream,
> + struct isp4fw_cmd *cmd)
> +{
> + struct isp4if_rb_config *rb_config;
> + struct device *dev = ispif->dev;
> + u8 *mem_sys;
> + u32 wr_ptr;
> + u32 rd_ptr;
> + u32 rreg;
> + u32 wreg;
> + u32 len;
> +
> + rb_config = &isp4if_cmd_rb_config[stream];
> + rreg = rb_config->reg_rptr;
> + wreg = rb_config->reg_wptr;
> + mem_sys = (u8 *)rb_config->base_sys_addr;
> + len = rb_config->val_size;
> +
> + if (isp4if_is_cmdq_rb_full(ispif, stream)) {
> + dev_err(dev, "fail no cmdslot (%d)\n", stream);
> + return -EINVAL;
> + }
> +
> + wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
> + rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
> +
> + if (rd_ptr > len) {
> + dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
> + stream, rd_ptr, len, wr_ptr);
> + return -EINVAL;
> + }
> +
> + if (wr_ptr > len) {
> + dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
> + stream, wr_ptr, len, rd_ptr);
> + return -EINVAL;
> + }
> +
> + if (wr_ptr < rd_ptr) {
> + memcpy((mem_sys + wr_ptr),
> + (u8 *)cmd, sizeof(struct isp4fw_cmd));
> + } else {
> + if ((len - wr_ptr) >= (sizeof(struct isp4fw_cmd))) {
> + memcpy((mem_sys + wr_ptr),
> + (u8 *)cmd, sizeof(struct isp4fw_cmd));
> + } else {
> + u32 size;
> + u8 *src;
> +
> + src = (u8 *)cmd;
> + size = len - wr_ptr;
> +
> + memcpy((mem_sys + wr_ptr), src, size);
> +
> + src += size;
> + size = sizeof(struct isp4fw_cmd) - size;
> + memcpy((mem_sys), src, size);
> + }
> + }
> +
> + wr_ptr += sizeof(struct isp4fw_cmd);
> + if (wr_ptr >= len)
> + wr_ptr -= len;
> +
> + isp4hw_wreg(ispif->mmio, wreg, wr_ptr);
> +
> + return 0;
> +}
> +
> +static inline enum isp4if_stream_id isp4if_get_fw_stream(u32 cmd_id)
> +{
> + return ISP4IF_STREAM_ID_1;
> +}
> +
> +static int isp4if_send_fw_cmd(struct isp4_interface *ispif, u32 cmd_id, void *package,
> + u32 package_size, wait_queue_head_t *wq, u32 *wq_cond, u32 *seq)
> +{
> + enum isp4if_stream_id stream = isp4if_get_fw_stream(cmd_id);
> + struct isp4if_cmd_element command_element = {};
Remove command_element per comments further down.
> + struct isp4if_gpu_mem_info *gpu_mem = NULL;
gpu_mem is never changed from NULL, remove this variable.
> + struct isp4if_cmd_element *cmd_ele = NULL;
> + struct isp4if_rb_config *rb_config;
> + struct device *dev = ispif->dev;
> + struct isp4fw_cmd cmd = {};
Use memset() to guarantee padding bits of cmd are zeroed, since this may not
guarantee it on all compilers.
> + u64 package_base = 0;
> + u32 seq_num;
> + u32 rreg;
> + u32 wreg;
> + int ret;
> +
> + if (package_size > sizeof(cmd.cmd_param)) {
> + dev_err(dev, "fail pkgsize(%u)>%zu cmd:0x%x,stream %d\n",
> + package_size, sizeof(cmd.cmd_param), cmd_id, stream);
> + return -EINVAL;
> + }
> +
> + rb_config = &isp4if_resp_rb_config[stream];
> + rreg = rb_config->reg_rptr;
> + wreg = rb_config->reg_wptr;
> +
> + guard(mutex)(&ispif->isp4if_mutex);
> +
> + ret = read_poll_timeout(isp4if_is_cmdq_rb_full, ret, !ret, ISP4IF_MAX_SLEEP_TIME * 1000,
> + ISP4IF_MAX_SLEEP_COUNT * ISP4IF_MAX_SLEEP_TIME * 1000, false,
> + ispif, stream);
> +
> + if (ret) {
> + u32 rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
> + u32 wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
> +
> + dev_err(dev,
> + "failed to get free cmdq slot, stream (%d),rd %u, wr %u\n",
> + stream, rd_ptr, wr_ptr);
> + return -ETIMEDOUT;
> + }
> +
> + cmd.cmd_id = cmd_id;
> + switch (stream) {
> + case ISP4IF_STREAM_ID_GLOBAL:
> + cmd.cmd_stream_id = STREAM_ID_INVALID;
> + break;
> + case ISP4IF_STREAM_ID_1:
> + cmd.cmd_stream_id = STREAM_ID_1;
> + break;
> + default:
> + dev_err(dev, "fail bad stream id %d\n", stream);
> + return -EINVAL;
> + }
> +
> + if (package && package_size)
> + memcpy(cmd.cmd_param, package, package_size);
> +
> + seq_num = ispif->host2fw_seq_num++;
> + cmd.cmd_seq_num = seq_num;
> + cmd.cmd_check_sum =
> + isp4if_compute_check_sum((u8 *)&cmd, sizeof(cmd) - 4);
Change `- 4` to `- sizeof(u32)`
> +
> + if (seq)
> + *seq = seq_num;
> + command_element.seq_num = seq_num;
> + command_element.cmd_id = cmd_id;
> + command_element.mc_addr = package_base;
> + command_element.wq = wq;
> + command_element.wq_cond = wq_cond;
> + command_element.gpu_pkg = gpu_mem;
> + command_element.stream = stream;
> + /*
> + * only append the fw cmd to queue when its response needs to be waited for,
> + * currently there are only two such commands, disable channel and stop stream
> + * which are only sent after close camera
> + */
> + if (wq && wq_cond) {
> + cmd_ele = isp4if_append_cmd_2_cmdq(ispif, &command_element);
> + if (!cmd_ele) {
> + dev_err(dev, "fail for isp_append_cmd_2_cmdq\n");
> + return -ENOMEM;
> + }
> + }
The kmemdup() is unnecessary. Remove the isp4if_append_cmd_2_cmdq() function and
change this to:
if (wq && wq_cond) {
cmd_ele = kmalloc(sizeof(*cmd_ele), GFP_KERNEL);
if (!cmd_ele) {
dev_err(dev, "fail for allocating cmd_ele\n");
return -ENOMEM;
}
cmd_ele->seq_num = seq_num;
cmd_ele->cmd_id = cmd_id;
cmd_ele->wq = wq;
cmd_ele->wq_cond = wq_cond;
guard(mutex)(&ispif->cmdq_mutex);
list_add_tail(©_command->list, &ispif->cmdq);
}
> +
> + ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
> + if (ret) {
> + dev_err(dev, "fail for insert_isp_fw_cmd camId (0x%08x)\n", cmd_id);
> + if (cmd_ele) {
> + isp4if_rm_cmd_from_cmdq(ispif, cmd_ele->seq_num, cmd_ele->cmd_id);
Using isp4if_rm_cmd_from_cmdq() sort of implies that there is a risk that
cmd_ele may have been removed from the list somehow, even though the fw cmd
insertion failed? In any case, either make it truly safe by assuming that it's
unsafe to dereference cmd_ele right now, or just delete cmd_ele directly from
the list under the lock.
> + kfree(cmd_ele);
> + }
> + }
> +
> + return ret;
> +}
> +
> +static int isp4if_send_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_info *buf_info)
> +{
> + struct isp4fw_cmd_send_buffer cmd = {};
Use memset() to guarantee padding bits are zeroed, since this may not guarantee
it on all compilers.
> +
> + cmd.buffer_type = BUFFER_TYPE_PREVIEW;
> + cmd.buffer.vmid_space.bit.vmid = 0;
> + cmd.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
> + isp4if_split_addr64(buf_info->planes[0].mc_addr,
> + &cmd.buffer.buf_base_a_lo,
> + &cmd.buffer.buf_base_a_hi);
> + cmd.buffer.buf_size_a = buf_info->planes[0].len;
> +
> + isp4if_split_addr64(buf_info->planes[1].mc_addr,
> + &cmd.buffer.buf_base_b_lo,
> + &cmd.buffer.buf_base_b_hi);
> + cmd.buffer.buf_size_b = buf_info->planes[1].len;
> +
> + isp4if_split_addr64(buf_info->planes[2].mc_addr,
> + &cmd.buffer.buf_base_c_lo,
> + &cmd.buffer.buf_base_c_hi);
> + cmd.buffer.buf_size_c = buf_info->planes[2].len;
> +
> + return isp4if_send_fw_cmd(ispif, CMD_ID_SEND_BUFFER, &cmd,
> + sizeof(cmd), NULL, NULL, NULL);
> +}
> +
> +static void isp4if_init_rb_config(struct isp4_interface *ispif, struct isp4if_rb_config *rb_config)
> +{
> + u32 lo;
> + u32 hi;
> +
> + isp4if_split_addr64(rb_config->base_mc_addr, &lo, &hi);
> +
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rb_config->reg_rptr, 0x0);
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rb_config->reg_wptr, 0x0);
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rb_config->reg_base_lo, lo);
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rb_config->reg_base_hi, hi);
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rb_config->reg_size, rb_config->val_size);
> +}
> +
> +static int isp4if_fw_init(struct isp4_interface *ispif)
> +{
> + struct isp4if_rb_config *rb_config;
> + u32 offset;
> + int i;
> +
> + /* initialize CMD_RB streams */
> + for (i = 0; i < ISP4IF_STREAM_ID_MAX; i++) {
> + rb_config = (isp4if_cmd_rb_config + i);
> + offset = ispif->aligned_rb_chunk_size *
> + (rb_config->index + ispif->cmd_rb_base_index);
> +
> + rb_config->val_size = ISP4IF_FW_CMD_BUF_SIZE;
> + rb_config->base_sys_addr =
> + (u8 *)ispif->fw_cmd_resp_buf->sys_addr + offset;
> + rb_config->base_mc_addr =
> + ispif->fw_cmd_resp_buf->gpu_mc_addr + offset;
> +
> + isp4if_init_rb_config(ispif, rb_config);
> + }
> +
> + /* initialize RESP_RB streams */
> + for (i = 0; i < ISP4IF_STREAM_ID_MAX; i++) {
> + rb_config = (isp4if_resp_rb_config + i);
> + offset = ispif->aligned_rb_chunk_size *
> + (rb_config->index + ispif->resp_rb_base_index);
> +
> + rb_config->val_size = ISP4IF_FW_CMD_BUF_SIZE;
> + rb_config->base_sys_addr =
> + (u8 *)ispif->fw_cmd_resp_buf->sys_addr + offset;
> + rb_config->base_mc_addr =
> + ispif->fw_cmd_resp_buf->gpu_mc_addr + offset;
> +
> + isp4if_init_rb_config(ispif, rb_config);
> + }
> +
> + /* initialize LOG_RB stream */
> + rb_config = &isp4if_log_rb_config;
> + rb_config->val_size = ISP4IF_FW_LOG_RINGBUF_SIZE;
> + rb_config->base_mc_addr = ispif->fw_log_buf->gpu_mc_addr;
> + rb_config->base_sys_addr = ispif->fw_log_buf->sys_addr;
> +
> + isp4if_init_rb_config(ispif, rb_config);
> +
> + return 0;
> +}
> +
> +static int isp4if_wait_fw_ready(struct isp4_interface *ispif, u32 isp_status_addr)
> +{
> + struct device *dev = ispif->dev;
> + u32 timeout_ms = 100;
> + u32 interval_ms = 1;
> + u32 reg_val;
> +
> + /* wait for FW initialize done! */
> + if (!read_poll_timeout(isp4hw_rreg, reg_val, reg_val & ISP_STATUS__CCPU_REPORT_MASK,
> + interval_ms * 1000, timeout_ms * 1000, false,
> + GET_ISP4IF_REG_BASE(ispif), isp_status_addr))
> + return 0;
> +
> + dev_err(dev, "ISP CCPU FW boot failed\n");
> +
> + return -ETIME;
> +}
> +
> +static void isp4if_enable_ccpu(struct isp4_interface *ispif)
> +{
> + u32 reg_val;
> +
> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET);
> + reg_val &= (~ISP_SOFT_RESET__CCPU_SOFT_RESET_MASK);
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET, reg_val);
> +
> + usleep_range(100, 150);
> +
> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL);
> + reg_val &= (~ISP_CCPU_CNTL__CCPU_HOST_SOFT_RST_MASK);
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL, reg_val);
> +}
> +
> +static void isp4if_disable_ccpu(struct isp4_interface *ispif)
> +{
> + u32 reg_val;
> +
> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL);
> + reg_val |= ISP_CCPU_CNTL__CCPU_HOST_SOFT_RST_MASK;
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL, reg_val);
> +
> + usleep_range(100, 150);
> +
> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET);
> + reg_val |= ISP_SOFT_RESET__CCPU_SOFT_RESET_MASK;
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET, reg_val);
> +}
> +
> +static int isp4if_fw_boot(struct isp4_interface *ispif)
> +{
> + struct device *dev = ispif->dev;
> +
> + if (ispif->status != ISP4IF_STATUS_PWR_ON) {
> + dev_err(dev, "invalid isp power status %d\n", ispif->status);
> + return -EINVAL;
> + }
> +
> + isp4if_disable_ccpu(ispif);
> +
> + isp4if_fw_init(ispif);
> +
> + /* clear ccpu status */
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_STATUS, 0x0);
> +
> + isp4if_enable_ccpu(ispif);
> +
> + if (isp4if_wait_fw_ready(ispif, ISP_STATUS)) {
> + isp4if_disable_ccpu(ispif);
> + return -EINVAL;
> + }
> +
> + /* enable interrupts */
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SYS_INT0_EN,
> + ISP4IF_FW_RESP_RB_IRQ_EN_MASK);
> +
> + ispif->status = ISP4IF_STATUS_FW_RUNNING;
> +
> + dev_dbg(dev, "ISP CCPU FW boot success\n");
> +
> + return 0;
> +}
> +
> +int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream, void *resp)
> +{
> + struct isp4fw_resp *response = resp;
> + struct isp4if_rb_config *rb_config;
> + struct device *dev = ispif->dev;
> + u32 rd_ptr_dbg;
> + u32 wr_ptr_dbg;
> + void *mem_sys;
> + u64 mem_addr;
> + u32 checksum;
> + u32 rd_ptr;
> + u32 wr_ptr;
> + u32 rreg;
> + u32 wreg;
> + u32 len;
> +
> + rb_config = &isp4if_resp_rb_config[stream];
> + rreg = rb_config->reg_rptr;
> + wreg = rb_config->reg_wptr;
> + mem_sys = rb_config->base_sys_addr;
> + mem_addr = rb_config->base_mc_addr;
> + len = rb_config->val_size;
> +
> + rd_ptr = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), rreg);
> + wr_ptr = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), wreg);
> + rd_ptr_dbg = rd_ptr;
> + wr_ptr_dbg = wr_ptr;
> +
> + if (rd_ptr > len) {
> + dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
> + stream, rd_ptr, len, wr_ptr);
> + return -EINVAL;
> + }
> +
> + if (wr_ptr > len) {
> + dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
> + stream, wr_ptr, len, rd_ptr);
> + return -EINVAL;
> + }
> +
> + if (rd_ptr < wr_ptr) {
> + if ((wr_ptr - rd_ptr) >= (sizeof(struct isp4fw_resp))) {
> + memcpy((u8 *)response, (u8 *)mem_sys + rd_ptr,
> + sizeof(struct isp4fw_resp));
> +
> + rd_ptr += sizeof(struct isp4fw_resp);
> + if (rd_ptr < len) {
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rreg, rd_ptr);
> + } else {
> + dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
> + stream, rd_ptr, len, wr_ptr);
> + return -EINVAL;
> + }
> +
> + } else {
> + dev_err(dev, "sth wrong with wptr and rptr\n");
> + return -EINVAL;
> + }
> + } else if (rd_ptr > wr_ptr) {
> + u32 size;
> + u8 *dst;
> +
> + dst = (u8 *)response;
> +
> + size = len - rd_ptr;
> + if (size > sizeof(struct isp4fw_resp)) {
> + mem_addr += rd_ptr;
> + memcpy((u8 *)response,
> + (u8 *)(mem_sys) + rd_ptr,
> + sizeof(struct isp4fw_resp));
> + rd_ptr += sizeof(struct isp4fw_resp);
> + if (rd_ptr < len) {
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rreg, rd_ptr);
> + } else {
> + dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
> + stream, rd_ptr, len, wr_ptr);
> + return -EINVAL;
> + }
> +
> + } else {
> + if ((size + wr_ptr) < (sizeof(struct isp4fw_resp))) {
> + dev_err(dev, "sth wrong with wptr and rptr1\n");
> + return -EINVAL;
> + }
> +
> + memcpy(dst, (u8 *)(mem_sys) + rd_ptr, size);
> +
> + dst += size;
> + size = sizeof(struct isp4fw_resp) - size;
> + if (size)
> + memcpy(dst, (u8 *)(mem_sys), size);
> + rd_ptr = size;
> + if (rd_ptr < len) {
> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
> + rreg, rd_ptr);
> + } else {
> + dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
> + stream, rd_ptr, len, wr_ptr);
> + return -EINVAL;
> + }
> + }
> + } else {
> + return -ETIME;
> + }
> +
> + checksum = isp4if_compute_check_sum((u8 *)response, sizeof(struct isp4fw_resp) - 4);
Change `- 4` to `- sizeof(u32)`
> +
> + if (checksum != response->resp_check_sum) {
> + dev_err(dev, "resp checksum 0x%x,should 0x%x,rptr %u,wptr %u\n",
> + checksum, response->resp_check_sum, rd_ptr_dbg, wr_ptr_dbg);
> +
> + dev_err(dev, "(%u), seqNo %u, resp_id (0x%x)\n", stream,
> + response->resp_seq_num,
> + response->resp_id);
> +
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +int isp4if_send_command(struct isp4_interface *ispif, u32 cmd_id, void *package, u32 package_size)
> +{
> + return isp4if_send_fw_cmd(ispif, cmd_id, package, package_size, NULL, NULL, NULL);
> +}
> +
> +int isp4if_send_command_sync(struct isp4_interface *ispif, u32 cmd_id, void *package,
> + u32 package_size, u32 timeout)
> +{
> + struct device *dev = ispif->dev;
> + DECLARE_WAIT_QUEUE_HEAD(cmd_wq);
> + u32 wq_cond = 0;
> + int ret;
> + u32 seq;
> +
> + ret = isp4if_send_fw_cmd(ispif, cmd_id, package, package_size, &cmd_wq, &wq_cond, &seq);
> +
> + if (ret) {
> + dev_err(dev, "send fw cmd fail %d\n", ret);
> + return ret;
> + }
> +
> + ret = wait_event_timeout(cmd_wq, wq_cond != 0, msecs_to_jiffies(timeout));
Instead of wq and wq_cond, use a `struct completion`.
> + if (ret == 0) {
> + struct isp4if_cmd_element *ele;
> +
> + ele = isp4if_rm_cmd_from_cmdq(ispif, seq, cmd_id);
> + kfree(ele);
> + return -ETIMEDOUT;
> + }
> +
> + return 0;
> +}
> +
> +void isp4if_clear_bufq(struct isp4_interface *ispif)
> +{
> + struct isp4if_img_buf_node *buf_node = NULL;
> + struct isp4if_img_buf_node *tmp_node = NULL;
Remove unnecessary initialization of buf_node and tmp_node.
> +
> + guard(mutex)(&ispif->bufq_mutex);
> +
> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->bufq, node) {
> + list_del(&buf_node->node);
> + kfree(buf_node);
> + }
Move the whole list to a local LIST_HEAD(free_list) variable and then release
the lock. Then you can list_for_each_entry_safe() without needing to do a
list_del() every time, and you won't need to hold the lock the whole time.
> +}
> +
> +void isp4if_dealloc_buffer_node(struct isp4if_img_buf_node *buf_node)
> +{
> + kfree(buf_node);
> +}
> +
> +struct isp4if_img_buf_node *isp4if_alloc_buffer_node(struct isp4if_img_buf_info *buf_info)
> +{
> + struct isp4if_img_buf_node *node = NULL;
> +
> + node = kmalloc(sizeof(*node), GFP_KERNEL);
> + if (node)
> + node->buf_info = *buf_info;
> +
> + return node;
> +};
Remove superfluous ; after the }.
> +
> +struct isp4if_img_buf_node *isp4if_dequeue_buffer(struct isp4_interface *ispif)
> +{
> + struct isp4if_img_buf_node *buf_node = NULL;
Remove the unnecessary initialization of buf_node.
> +
> + guard(mutex)(&ispif->bufq_mutex);
> +
> + buf_node = list_first_entry_or_null(&ispif->bufq, typeof(*buf_node), node);
> + if (buf_node)
> + list_del(&buf_node->node);
> +
> + return buf_node;
> +}
> +
> +int isp4if_queue_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_node *buf_node)
> +{
> + int ret;
> +
> + ret = isp4if_send_buffer(ispif, &buf_node->buf_info);
> + if (ret)
> + return ret;
> +
> + guard(mutex)(&ispif->bufq_mutex);
> +
> + list_add_tail(&buf_node->node, &ispif->bufq);
> +
> + return 0;
> +}
> +
> +int isp4if_stop(struct isp4_interface *ispif)
> +{
> + isp4if_disable_ccpu(ispif);
> +
> + isp4if_dealloc_fw_gpumem(ispif);
> +
> + return 0;
> +}
> +
> +int isp4if_start(struct isp4_interface *ispif)
> +{
> + int ret;
> +
> + ret = isp4if_alloc_fw_gpumem(ispif);
> + if (ret)
> + return -ENOMEM;
Return ret instead of -ENOMEM, since isp4if_alloc_fw_gpumem() returns -ENOMEM.
> +
> + ret = isp4if_fw_boot(ispif);
> + if (ret)
> + goto failed_fw_boot;
> +
> + return 0;
> +
> +failed_fw_boot:
> + isp4if_dealloc_fw_gpumem(ispif);
> + return ret;
> +}
> +
> +int isp4if_deinit(struct isp4_interface *ispif)
> +{
> + isp4if_clear_cmdq(ispif);
> +
> + isp4if_clear_bufq(ispif);
> +
> + mutex_destroy(&ispif->cmdq_mutex);
> + mutex_destroy(&ispif->bufq_mutex);
> + mutex_destroy(&ispif->isp4if_mutex);
> +
> + return 0;
> +}
> +
> +int isp4if_init(struct isp4_interface *ispif, struct device *dev, void __iomem *isp_mmip)
> +{
> + ispif->dev = dev;
> + ispif->mmio = isp_mmip;
> +
> + ispif->cmd_rb_base_index = 0;
> + ispif->resp_rb_base_index = ISP4IF_RESP_CHAN_TO_RB_OFFSET - 1;
> + ispif->aligned_rb_chunk_size = ISP4IF_RB_PMBMAP_MEM_CHUNK & 0xffffffc0;
> +
> + mutex_init(&ispif->cmdq_mutex); /* used for cmdq access */
> + mutex_init(&ispif->bufq_mutex); /* used for bufq access */
> + mutex_init(&ispif->isp4if_mutex); /* used for commands sent to ispfw */
> +
> + INIT_LIST_HEAD(&ispif->cmdq);
> + INIT_LIST_HEAD(&ispif->bufq);
> +
> + return 0;
> +}
> diff --git a/drivers/media/platform/amd/isp4/isp4_interface.h b/drivers/media/platform/amd/isp4/isp4_interface.h
> new file mode 100644
> index 000000000000..5b94985cdc44
> --- /dev/null
> +++ b/drivers/media/platform/amd/isp4/isp4_interface.h
> @@ -0,0 +1,149 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#ifndef _ISP4_INTERFACE_
> +#define _ISP4_INTERFACE_
It seems strange that isp4_interface.h has so many include dependencies and yet
doesn't #include anything on its own. Maybe the includes needed by
isp4_interface.h should be moved into isp4_interface.h?
> +
> +#define ISP4IF_RB_MAX (25)
> +#define ISP4IF_RESP_CHAN_TO_RB_OFFSET (9)
> +#define ISP4IF_RB_PMBMAP_MEM_SIZE (16 * 1024 * 1024 - 1)
> +#define ISP4IF_RB_PMBMAP_MEM_CHUNK (ISP4IF_RB_PMBMAP_MEM_SIZE \
> + / (ISP4IF_RB_MAX - 1))
> +#define ISP4IF_HOST2FW_COMMAND_SIZE (sizeof(struct isp4fw_cmd))
> +#define ISP4IF_FW_CMD_BUF_COUNT 4
> +#define ISP4IF_FW_RESP_BUF_COUNT 4
> +#define ISP4IF_MAX_NUM_HOST2FW_COMMAND (40)
> +#define ISP4IF_FW_CMD_BUF_SIZE (ISP4IF_MAX_NUM_HOST2FW_COMMAND \
> + * ISP4IF_HOST2FW_COMMAND_SIZE)
> +#define ISP4IF_MAX_SLEEP_COUNT (10)
> +#define ISP4IF_MAX_SLEEP_TIME (33)
> +
> +#define ISP4IF_META_INFO_BUF_SIZE ALIGN(sizeof(struct isp4fw_meta_info), 0x8000)
> +#define ISP4IF_MAX_STREAM_BUF_COUNT 8
> +
> +#define ISP4IF_FW_LOG_RINGBUF_SIZE (2 * 1024 * 1024)
> +
> +#define ISP4IF_MAX_CMD_RESPONSE_BUF_SIZE (4 * 1024)
> +
> +#define GET_ISP4IF_REG_BASE(ispif) (((ispif))->mmio)
> +
> +enum isp4if_stream_id {
> + ISP4IF_STREAM_ID_GLOBAL = 0,
> + ISP4IF_STREAM_ID_1 = 1,
> + ISP4IF_STREAM_ID_MAX = 4
> +};
> +
> +enum isp4if_status {
> + ISP4IF_STATUS_PWR_OFF,
> + ISP4IF_STATUS_PWR_ON,
> + ISP4IF_STATUS_FW_RUNNING,
> + ISP4IF_FSM_STATUS_MAX
> +};
> +
> +struct isp4if_gpu_mem_info {
> + u32 mem_domain;
> + u64 mem_size;
> + u32 mem_align;
mem_domain, mem_size, and mem_align are all unused. Remove them.
> + u64 gpu_mc_addr;
> + void *sys_addr;
> + void *mem_handle;
> +};
> +
> +struct isp4if_img_buf_info {
> + struct {
> + void *sys_addr;
> + u64 mc_addr;
> + u32 len;
> + } planes[3];
> +};
> +
> +struct isp4if_img_buf_node {
> + struct list_head node;
> + struct isp4if_img_buf_info buf_info;
> +};
> +
> +struct isp4if_cmd_element {
> + struct list_head list;
> + u32 seq_num;
> + u32 cmd_id;
> + enum isp4if_stream_id stream;
> + u64 mc_addr;
stream and mc_addr are not used for anything, remove them.
> + wait_queue_head_t *wq;
> + u32 *wq_cond;
> + struct isp4if_gpu_mem_info *gpu_pkg;
gpu_pkg is not used for anything, remove it.
> +};
> +
> +struct isp4_interface {
> + struct device *dev;
> + void __iomem *mmio;
> +
> + struct mutex cmdq_mutex; /* used for cmdq access */
> + struct mutex bufq_mutex; /* used for bufq access */
It makes more sense for cmdq_mutex and bufq_mutex to be spin locks since they
are only held briefly for list traversal.
> + struct mutex isp4if_mutex; /* used to send fw cmd and read fw log */
> +
> + struct list_head cmdq; /* commands sent to fw */
> + struct list_head bufq; /* buffers sent to fw */
> +
> + enum isp4if_status status;
> + u32 host2fw_seq_num;
> +
> + /* FW ring buffer configs */
> + u32 cmd_rb_base_index;
> + u32 resp_rb_base_index;
> + u32 aligned_rb_chunk_size;
> +
> + /* ISP fw buffers */
> + struct isp4if_gpu_mem_info *fw_log_buf;
> + struct isp4if_gpu_mem_info *fw_cmd_resp_buf;
> + struct isp4if_gpu_mem_info *fw_mem_pool;
> + struct isp4if_gpu_mem_info *
> + metainfo_buf_pool[ISP4IF_MAX_STREAM_BUF_COUNT];
> +};
[snip]
> +
> +#endif
Add /* _ISP4_INTERFACE_ */
> --
> 2.34.1
>
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 0/7] Add AMD ISP4 driver
2025-09-19 12:23 ` Laurent Pinchart
@ 2025-09-22 2:50 ` Du, Bin
0 siblings, 0 replies; 35+ messages in thread
From: Du, Bin @ 2025-09-22 2:50 UTC (permalink / raw)
To: Laurent Pinchart
Cc: mchehab, hverkuil, bryan.odonoghue, sakari.ailus,
prabhakar.mahadev-lad.rj, linux-media, linux-kernel, sultan,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao
Thanks Laurent, totally understood, no worries at all and please take
your time.
On 9/19/2025 8:23 PM, Laurent Pinchart wrote:
> Hi Bin,
>
> On Fri, Sep 19, 2025 at 11:24:36AM +0800, Du, Bin wrote:
>> Hi Laurent, Sakari,
>>
>> Thank you for your previous review feedback, we have addressed all
>> comments and listed the detailed changes in this cover letter below.
>> Would you mind taking a look at the v4 series if you have time. Any
>> feedback would be greatly appreciated.
>
> I'm really overloaded at the moment, so I can't guarantee any review
> time frame. Sorry about that.
>
>> On 9/11/2025 6:08 PM, Bin Du wrote:
>>> Hello,
>>>
>>> AMD ISP4 is the AMD image processing gen 4 which can be found in HP ZBook Ultra G1a 14 inch Mobile Workstation PC (Ryzen AI Max 300 Series)
>>> (https://ubuntu.com/certified/202411-36043)
>>> This patch series introduces the initial driver support for the AMD ISP4.
>>>
>>> Patch summary:
>>> - Powers up/off and initializes ISP HW
>>> - Configures and kicks off ISP FW
>>> - Interacts with APP using standard V4l2 interface by video node
>>> - Controls ISP HW and interacts with ISP FW to do image processing
>>> - Supports enum/set output image format and resolution
>>> - Supports queueing buffer from app and dequeuing ISP filled buffer to App
>>> - It is verified on qv4l2, cheese and qcam
>>> - It is verified together with following patches
>>> platform/x86: Add AMD ISP platform config (https://lore.kernel.org/all/20250514215623.522746-1-pratap.nirujogi@amd.com/)
>>> pinctrl: amd: isp411: Add amdisp GPIO pinctrl (https://github.com/torvalds/linux/commit/e97435ab09f3ad7b6a588dd7c4e45a96699bbb4a)
>>> drm/amd/amdgpu: Add GPIO resources required for amdisp (https://gitlab.freedesktop.org/agd5f/linux/-/commit/ad0f5966ed8297aa47b3184192b00b7379ae0758)
>>> drm/amd/amdgpu: Declare isp firmware binary file (https://gitlab.freedesktop.org/agd5f/linux/-/commit/35345917bc9f7c86152b270d9d93c220230b667f)
>>>
>>> AMD ISP4 Key features:
>>> - Processes bayer raw data from the connected sensor and output them to different YUV formats
>>> - Downscale input image to different output image resolution
>>> - Pipeline to do image processing on the input image including demosaic, denoise, 3A, etc.
>>>
>>> ----------
>>>
>>> Changes v3 -> v4:
>>>
>>> - Replace one mutex with guard mutex.
>>> - Remove unnecessary bus_info initialization of v4l2_capability.
>>> - Drop V4L2_CAP_IO_MC from capabilities of v4l2_capability.
>>> - Modify document with better SOC description.
>>> - Fix Test x86 failure in Media CI test https://linux-media.pages.freedesktop.org/-/users/patchwork/-/jobs/83470456/artifacts/report.htm
>>> - Modify some commit messages by describing changes in imperative mood.
>>> - Add media-ctl output in cover letter.
>>> - Create separated dedicated amdgpu patch to add declaration MODULE_FIRMWARE("amdgpu/isp_4_1_1.bin");
>>> - Fix typo errors and other cosmetic issues.
>>> - Add DRM_AMD_ISP dependency in Kconfig.
>>>
>>>
>>> Changes v2 -> v3:
>>>
>>> - All the dependent patches in other modules (drm/amd/amdgpu, platform/x86, pinctrl/amd) merged on upstream mainline kernel (https://github.com/torvalds/linux) v6.17.
>>> - Removed usage of amdgpu structs in ISP driver. Added helper functions in amdgpu accepting opaque params from ISP driver to allocate and release ISP GART buffers.
>>> - Moved sensor and MIPI phy control entirely into ISP FW instead of the previous hybrid approach controlling sensor from both FW and x86 (sensor driver).
>>> - Removed phy configuration and sensor binding as x86 (sensor driver) had relinquished the sensor control for ISP FW. With this approach the driver will be exposed as web camera like interface.
>>> - New FW with built-in sensor driver is submitted on upstream linux-firmware repo (https://gitlab.com/kernel-firmware/linux-firmware/).
>>> - Please note the new FW submitted is not directly compatible with OEM Kernel ISP4.0 (https://github.com/amd/Linux_ISP_Kernel/tree/4.0) and the previous ISP V2 patch series.
>>> - If intend to use the new FW, please rebuild OEM ISP4.0 Kernel with CONFIG_VIDEO_OV05C10=N and CONFIG_PINCTRL_AMDISP=Y.
>>> - Included critical fixes from Sultan Alsawaf branch (https://github.com/kerneltoast/kernel_x86_laptop.git) related to managing lifetime of isp buffers.
>>> media: amd: isp4: Add missing refcount tracking to mmap memop
>>> media: amd: isp4: Don't put or unmap the dmabuf when detaching
>>> media: amd: isp4: Don't increment refcount when dmabuf export fails
>>> media: amd: isp4: Fix possible use-after-free in isp4vid_vb2_put()
>>> media: amd: isp4: Always export a new dmabuf from get_dmabuf memop
>>> media: amd: isp4: Fix implicit dmabuf lifetime tracking
>>> media: amd: isp4: Fix possible use-after-free when putting implicit dmabuf
>>> media: amd: isp4: Simplify isp4vid_get_dmabuf() arguments
>>> media: amd: isp4: Move up buf->vaddr check in isp4vid_get_dmabuf()
>>> media: amd: isp4: Remove unused userptr memops
>>> media: amd: isp4: Add missing cleanup on error in isp4vid_vb2_alloc()
>>> media: amd: isp4: Release queued buffers on error in start_streaming
>>> - Addressed all code related upstream comments
>>> - Fix typo errors and other cosmetic issues.
>>>
>>>
>>> Changes v1 -> v2:
>>>
>>> - Fix media CI test errors and valid warnings
>>> - Reduce patch number in the series from 9 to 8 by merging MAINTAINERS adding patch to the first patch
>>> - In patch 5
>>> - do modification to use remote endpoint instead of local endpoint
>>> - use link frequency and port number as start phy parameter instead of extra added phy-id and phy-bit-rate property of endpoint
>>>
>>> ----------
>>>
>>> It passes v4l2 compliance test, the test reports for:
>>>
>>> (a) amd_isp_capture device /dev/video0
>>>
>>> Compliance test for amd_isp_capture device /dev/video0:
>>> -------------------------------------------------------
>>>
>>> atg@atg-HP-PV:~/bin$ ./v4l2-compliance -d /dev/video0
>>> v4l2-compliance 1.29.0-5348, 64 bits, 64-bit time_t
>>> v4l2-compliance SHA: 75e3f0e2c2cb 2025-03-17 18:12:17
>>>
>>> Compliance test for amd_isp_capture device /dev/video0:
>>>
>>> Driver Info:
>>> Driver name : amd_isp_capture
>>> Card type : amd_isp_capture
>>> Bus info : platform:amd_isp_capture
>>> Driver version : 6.14.0
>>> Capabilities : 0xa4200001
>>> Video Capture
>>> I/O MC
>>> Streaming
>>> Extended Pix Format
>>> Device Capabilities
>>> Device Caps : 0x24200001
>>> Video Capture
>>> I/O MC
>>> Streaming
>>> Extended Pix Format
>>> Media Driver Info:
>>> Driver name : amd_isp_capture
>>> Model : amd_isp41_mdev
>>> Serial :
>>> Bus info : platform:amd_isp_capture
>>> Media version : 6.14.0
>>> Hardware revision: 0x00000000 (0)
>>> Driver version : 6.14.0
>>> Interface Info:
>>> ID : 0x03000005
>>> Type : V4L Video
>>> Entity Info:
>>> ID : 0x00000003 (3)
>>> Name : Preview
>>> Function : V4L2 I/O
>>> Pad 0x01000004 : 0: Sink
>>> Link 0x02000007: from remote pad 0x1000002 of entity 'amd isp4' (Image Signal Processor): Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> test MC information (see 'Media Driver Info' above): OK
>>> test VIDIOC_QUERYCAP: OK
>>> test invalid ioctls: OK
>>>
>>> Allow for multiple opens:
>>> test second /dev/video0 open: OK
>>> test VIDIOC_QUERYCAP: OK
>>> test VIDIOC_G/S_PRIORITY: OK
>>> test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> test VIDIOC_G/S/ENUMINPUT: OK
>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> Inputs: 1 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls (Input 0):
>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
>>> test VIDIOC_QUERYCTRL: OK (Not Supported)
>>> test VIDIOC_G/S_CTRL: OK (Not Supported)
>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> Standard Controls: 0 Private Controls: 0
>>>
>>> Format ioctls (Input 0):
>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> test VIDIOC_G/S_PARM: OK
>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>> test VIDIOC_G_FMT: OK
>>> test VIDIOC_TRY_FMT: OK
>>> test VIDIOC_S_FMT: OK
>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> test Cropping: OK (Not Supported)
>>> test Composing: OK (Not Supported)
>>> test Scaling: OK (Not Supported)
>>>
>>> Codec ioctls (Input 0):
>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls (Input 0):
>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>>> test CREATE_BUFS maximum buffers: OK
>>> test VIDIOC_REMOVE_BUFS: OK
>>> test VIDIOC_EXPBUF: OK
>>> test Requests: OK (Not Supported)
>>> test blocking wait: OK
>>>
>>> Total for amd_isp_capture device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
>>>
>>> The media-ctl output of media device /dev/media0:
>>> -------------------------------------------------------
>>>
>>> atg@atg-HP-PV:~$ media-ctl -p -d /dev/media0
>>> Media controller API version 6.17.0
>>>
>>> Media device information
>>> ------------------------
>>> driver amd_isp_capture
>>> model amd_isp41_mdev
>>> serial
>>> bus info platform:amd_isp_capture
>>> hw revision 0x0
>>> driver version 6.17.0
>>>
>>> Device topology
>>> - entity 1: amd isp4 (1 pad, 1 link, 0 routes)
>>> type V4L2 subdev subtype Unknown flags 0
>>> pad0: Source
>>> -> "Preview":0 [ENABLED,IMMUTABLE]
>>>
>>> - entity 3: Preview (1 pad, 1 link)
>>> type Node subtype V4L flags 0
>>> device node name /dev/video0
>>> pad0: Sink
>>> <- "amd isp4":0 [ENABLED,IMMUTABLE]
>>>
>>> Please review and provide feedback.
>>>
>>> Many thanks,
>>>
>>> Bin Du (7):
>>> media: platform: amd: Introduce amd isp4 capture driver
>>> media: platform: amd: low level support for isp4 firmware
>>> media: platform: amd: Add isp4 fw and hw interface
>>> media: platform: amd: isp4 subdev and firmware loading handling added
>>> media: platform: amd: isp4 video node and buffers handling added
>>> media: platform: amd: isp4 debug fs logging and more descriptive
>>> errors
>>> Documentation: add documentation of AMD isp 4 driver
>>>
>>> Documentation/admin-guide/media/amdisp4-1.rst | 63 +
>>> Documentation/admin-guide/media/amdisp4.dot | 6 +
>>> .../admin-guide/media/v4l-drivers.rst | 1 +
>>> MAINTAINERS | 25 +
>>> drivers/media/platform/Kconfig | 1 +
>>> drivers/media/platform/Makefile | 1 +
>>> drivers/media/platform/amd/Kconfig | 3 +
>>> drivers/media/platform/amd/Makefile | 3 +
>>> drivers/media/platform/amd/isp4/Kconfig | 13 +
>>> drivers/media/platform/amd/isp4/Makefile | 10 +
>>> drivers/media/platform/amd/isp4/isp4.c | 237 ++++
>>> drivers/media/platform/amd/isp4/isp4.h | 26 +
>>> drivers/media/platform/amd/isp4/isp4_debug.c | 272 ++++
>>> drivers/media/platform/amd/isp4/isp4_debug.h | 41 +
>>> .../platform/amd/isp4/isp4_fw_cmd_resp.h | 314 +++++
>>> drivers/media/platform/amd/isp4/isp4_hw_reg.h | 125 ++
>>> .../media/platform/amd/isp4/isp4_interface.c | 966 +++++++++++++
>>> .../media/platform/amd/isp4/isp4_interface.h | 149 ++
>>> drivers/media/platform/amd/isp4/isp4_subdev.c | 1197 ++++++++++++++++
>>> drivers/media/platform/amd/isp4/isp4_subdev.h | 133 ++
>>> drivers/media/platform/amd/isp4/isp4_video.c | 1207 +++++++++++++++++
>>> drivers/media/platform/amd/isp4/isp4_video.h | 87 ++
>>> 22 files changed, 4880 insertions(+)
>>> create mode 100644 Documentation/admin-guide/media/amdisp4-1.rst
>>> create mode 100644 Documentation/admin-guide/media/amdisp4.dot
>>> create mode 100644 drivers/media/platform/amd/Kconfig
>>> create mode 100644 drivers/media/platform/amd/Makefile
>>> create mode 100644 drivers/media/platform/amd/isp4/Kconfig
>>> create mode 100644 drivers/media/platform/amd/isp4/Makefile
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4.c
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4.h
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.c
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_debug.h
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_hw_reg.h
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.c
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_interface.h
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.c
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_subdev.h
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_video.c
>>> create mode 100644 drivers/media/platform/amd/isp4/isp4_video.h
>
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added
2025-09-11 10:08 ` [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added Bin Du
@ 2025-09-23 7:23 ` Sultan Alsawaf
2025-09-30 7:30 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-09-23 7:23 UTC (permalink / raw)
To: Bin Du
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Hi Bin,
On Thu, Sep 11, 2025 at 06:08:44PM +0800, Bin Du wrote:
> Isp4 sub-device is implementing v4l2 sub-device interface. It has one
> capture video node, and supports only preview stream. It manages firmware
> states, stream configuration. Add interrupt handling and notification for
> isp firmware to isp-subdevice.
>
> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Bin Du <Bin.Du@amd.com>
> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
[snip]
> +++ b/drivers/media/platform/amd/isp4/isp4.c
> @@ -5,13 +5,19 @@
>
> #include <linux/pm_runtime.h>
> #include <linux/vmalloc.h>
> +
> +#include <media/v4l2-fwnode.h>
> #include <media/v4l2-ioctl.h>
>
> #include "isp4.h"
> -
> -#define VIDEO_BUF_NUM 5
> +#include "isp4_hw_reg.h"
>
> #define ISP4_DRV_NAME "amd_isp_capture"
> +#define ISP4_FW_RESP_RB_IRQ_STATUS_MASK \
> + (ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK | \
> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK | \
> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK | \
> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK)
>
> /* interrupt num */
> static const u32 isp4_ringbuf_interrupt_num[] = {
> @@ -21,19 +27,95 @@ static const u32 isp4_ringbuf_interrupt_num[] = {
> 4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
> };
>
> -#define to_isp4_device(dev) \
> - ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
> +#define to_isp4_device(dev) container_of(dev, struct isp4_device, v4l2_dev)
> +
> +static void isp4_wake_up_resp_thread(struct isp4_subdev *isp, u32 index)
> +{
> + if (isp && index < ISP4SD_MAX_FW_RESP_STREAM_NUM) {
> + struct isp4sd_thread_handler *thread_ctx =
> + &isp->fw_resp_thread[index];
> +
> + thread_ctx->wq_cond = 1;
> + wake_up_interruptible(&thread_ctx->waitq);
> + }
> +}
> +
> +static void isp4_resp_interrupt_notify(struct isp4_subdev *isp, u32 intr_status)
> +{
> + bool wake = (isp->ispif.status == ISP4IF_STATUS_FW_RUNNING);
> +
> + u32 intr_ack = 0;
> +
> + /* global response */
> + if (intr_status &
> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK) {
> + if (wake)
> + isp4_wake_up_resp_thread(isp, 0);
> +
> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT12_ACK_MASK;
The INT_MASKs and ACK_MASKs are the same; perhaps the ACK_MASKs can just be
removed so you can just write intr_status to ISP_SYS_INT0_ACK instead?
> + }
> +
> + /* stream 1 response */
> + if (intr_status &
> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK) {
> + if (wake)
> + isp4_wake_up_resp_thread(isp, 1);
> +
> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT9_ACK_MASK;
> + }
> +
> + /* stream 2 response */
> + if (intr_status &
> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK) {
> + if (wake)
> + isp4_wake_up_resp_thread(isp, 2);
> +
> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT10_ACK_MASK;
> + }
> +
> + /* stream 3 response */
> + if (intr_status &
> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK) {
> + if (wake)
> + isp4_wake_up_resp_thread(isp, 3);
> +
> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT11_ACK_MASK;
> + }
I think it'd be cleaner to put these masks into an array and loop over them in
here instead of writing them all out by hand.
> +
> + /* clear ISP_SYS interrupts */
> + isp4hw_wreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_ACK, intr_ack);
> +}
>
> static irqreturn_t isp4_irq_handler(int irq, void *arg)
> {
> + struct isp4_device *isp_dev = dev_get_drvdata(arg);
This is technically a data race because setting drvdata and reading drvdata do
not use WRITE_ONCE() and READ_ONCE(), respectively. And enabling the IRQ before
the handler is allowed to do anything is why that `if (!isp_dev)` check exists,
because that is another race.
Instead, pass the isp_dev pointer through the private pointer of
devm_request_irq() and add IRQ_NOAUTOEN so the IRQ is enabled by default. Then,
when it is safe for the IRQ to run, enable it with enable_irq().
That way you can delete the `if (!isp_dev)` check and resolve the two races.
> + struct isp4_subdev *isp = NULL;
> + u32 isp_sys_irq_status = 0x0;
Remove unnecessary initialization of `isp` and `isp_sys_irq_status` variables.
> + u32 r1;
> +
> + if (!isp_dev)
> + goto error_drv_data;
> +
> + isp = &isp_dev->isp_sdev;
> + /* check ISP_SYS interrupts status */
> + r1 = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_STATUS);
> +
> + isp_sys_irq_status = r1 & ISP4_FW_RESP_RB_IRQ_STATUS_MASK;
There are four IRQs (one for each stream) but any one of the IRQs can result in
a notification for _all_ streams. Each IRQ should only do the work of its own
stream.
You can do this by passing devm_request_irq() a private pointer to indicate the
mapping between a stream and its IRQ, so that isp4_irq_handler() can know which
stream it should look at.
> +
> + isp4_resp_interrupt_notify(isp, isp_sys_irq_status);
> +
> +error_drv_data:
> return IRQ_HANDLED;
> }
>
> static int isp4_capture_probe(struct platform_device *pdev)
> {
> struct device *dev = &pdev->dev;
> + struct isp4_subdev *isp_sdev;
> struct isp4_device *isp_dev;
> - int i, irq, ret;
> + size_t i;
> + int irq;
> + int ret;
>
> isp_dev = devm_kzalloc(&pdev->dev, sizeof(*isp_dev), GFP_KERNEL);
> if (!isp_dev)
> @@ -42,6 +124,12 @@ static int isp4_capture_probe(struct platform_device *pdev)
> isp_dev->pdev = pdev;
> dev->init_name = ISP4_DRV_NAME;
>
> + isp_sdev = &isp_dev->isp_sdev;
> + isp_sdev->mmio = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(isp_sdev->mmio))
> + return dev_err_probe(dev, PTR_ERR(isp_sdev->mmio),
> + "isp ioremap fail\n");
> +
> for (i = 0; i < ARRAY_SIZE(isp4_ringbuf_interrupt_num); i++) {
> irq = platform_get_irq(pdev, isp4_ringbuf_interrupt_num[i]);
> if (irq < 0)
> @@ -55,6 +143,8 @@ static int isp4_capture_probe(struct platform_device *pdev)
> irq);
> }
>
> + isp_dev->pltf_data = pdev->dev.platform_data;
> +
> /* Link the media device within the v4l2_device */
> isp_dev->v4l2_dev.mdev = &isp_dev->mdev;
>
> @@ -66,6 +156,8 @@ static int isp4_capture_probe(struct platform_device *pdev)
> isp_dev->mdev.dev = &pdev->dev;
> media_device_init(&isp_dev->mdev);
>
> + pm_runtime_set_suspended(dev);
> + pm_runtime_enable(dev);
> /* register v4l2 device */
> snprintf(isp_dev->v4l2_dev.name, sizeof(isp_dev->v4l2_dev.name),
> "AMD-V4L2-ROOT");
> @@ -74,19 +166,24 @@ static int isp4_capture_probe(struct platform_device *pdev)
> return dev_err_probe(dev, ret,
> "fail register v4l2 device\n");
>
> + ret = isp4sd_init(&isp_dev->isp_sdev, &isp_dev->v4l2_dev);
> + if (ret) {
> + dev_err(dev, "fail init isp4 sub dev %d\n", ret);
> + goto err_unreg_v4l2;
> + }
> +
> ret = media_device_register(&isp_dev->mdev);
> if (ret) {
> dev_err(dev, "fail to register media device %d\n", ret);
> - goto err_unreg_v4l2;
> + goto err_isp4_deinit;
> }
>
> platform_set_drvdata(pdev, isp_dev);
>
> - pm_runtime_set_suspended(dev);
> - pm_runtime_enable(dev);
> -
> return 0;
>
> +err_isp4_deinit:
> + isp4sd_deinit(&isp_dev->isp_sdev);
> err_unreg_v4l2:
> v4l2_device_unregister(&isp_dev->v4l2_dev);
>
> @@ -97,8 +194,13 @@ static void isp4_capture_remove(struct platform_device *pdev)
> {
> struct isp4_device *isp_dev = platform_get_drvdata(pdev);
>
> + v4l2_device_unregister_subdev(&isp_dev->isp_sdev.sdev);
> +
> media_device_unregister(&isp_dev->mdev);
> + media_entity_cleanup(&isp_dev->isp_sdev.sdev.entity);
Why is isp4_capture_remove() handling the cleanup responsibility of
isp4sd_deinit()? The v4l2_device_unregister_subdev() and media_entity_cleanup()
on the subdev should only be done by isp4sd_deinit().
Since v4l2_device_unregister_subdev() is not called by isp4sd_deinit(), this
results in missing cleanup on error from isp4_capture_probe() when
isp4sd_deinit() is called.
> v4l2_device_unregister(&isp_dev->v4l2_dev);
> +
> + isp4sd_deinit(&isp_dev->isp_sdev);
> }
>
> static struct platform_driver isp4_capture_drv = {
[snip]
> +++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
> @@ -0,0 +1,1095 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#include <linux/mutex.h>
> +#include <linux/pm_domain.h>
> +#include <linux/pm_runtime.h>
> +
> +#include "isp4_fw_cmd_resp.h"
> +#include "isp4_interface.h"
> +#include "isp4_subdev.h"
> +#include <linux/units.h>
> +
> +#define ISP4SD_MAX_CMD_RESP_BUF_SIZE (4 * 1024)
> +#define ISP4SD_MIN_BUF_CNT_BEF_START_STREAM 4
> +
> +#define ISP4SD_PERFORMANCE_STATE_LOW 0
> +#define ISP4SD_PERFORMANCE_STATE_HIGH 1
> +
> +#define ISP4SD_FW_CMD_TIMEOUT_IN_MS 500
> +#define ISP4SD_WAIT_RESP_IRQ_TIMEOUT 5 /* ms */
> +/* align 32KB */
> +#define ISP4SD_META_BUF_SIZE ALIGN(sizeof(struct isp4fw_meta_info), 0x8000)
> +
> +#define to_isp4_subdev(v4l2_sdev) \
> + container_of(v4l2_sdev, struct isp4_subdev, sdev)
> +
> +static const char *isp4sd_entity_name = "amd isp4";
> +
> +static void isp4sd_module_enable(struct isp4_subdev *isp_subdev, bool enable)
> +{
> + if (isp_subdev->enable_gpio) {
> + gpiod_set_value(isp_subdev->enable_gpio, enable ? 1 : 0);
> + dev_dbg(isp_subdev->dev, "%s isp_subdev module\n",
> + enable ? "enable" : "disable");
> + }
> +}
> +
> +static int isp4sd_setup_fw_mem_pool(struct isp4_subdev *isp_subdev)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4fw_cmd_send_buffer buf_type = {};
Use memset to guarantee zeroing of padding bits.
> + struct device *dev = isp_subdev->dev;
> + int ret;
> +
> + if (!ispif->fw_mem_pool) {
> + dev_err(dev, "fail to alloc mem pool\n");
> + return -ENOMEM;
> + }
> +
> + buf_type.buffer_type = BUFFER_TYPE_MEM_POOL;
> + buf_type.buffer.buf_tags = 0;
> + buf_type.buffer.vmid_space.bit.vmid = 0;
> + buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
> + isp4if_split_addr64(ispif->fw_mem_pool->gpu_mc_addr,
> + &buf_type.buffer.buf_base_a_lo,
> + &buf_type.buffer.buf_base_a_hi);
> + buf_type.buffer.buf_size_a = (u32)ispif->fw_mem_pool->mem_size;
> +
> + ret = isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
> + &buf_type, sizeof(buf_type));
> + if (ret) {
> + dev_err(dev, "send fw mem pool 0x%llx(%u) fail %d\n",
> + ispif->fw_mem_pool->gpu_mc_addr,
> + buf_type.buffer.buf_size_a,
> + ret);
> + return ret;
> + }
> +
> + dev_dbg(dev, "send fw mem pool 0x%llx(%u) suc\n",
> + ispif->fw_mem_pool->gpu_mc_addr,
> + buf_type.buffer.buf_size_a);
> +
> + return 0;
> +};
> +
> +static int isp4sd_set_stream_path(struct isp4_subdev *isp_subdev)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4fw_cmd_set_stream_cfg cmd = {};
Use memset to guarantee zeroing of padding bits.
> + struct device *dev = isp_subdev->dev;
> +
> + cmd.stream_cfg.mipi_pipe_path_cfg.isp4fw_sensor_id = SENSOR_ID_ON_MIPI0;
> + cmd.stream_cfg.mipi_pipe_path_cfg.b_enable = true;
> + cmd.stream_cfg.isp_pipe_path_cfg.isp_pipe_id = MIPI0_ISP_PIPELINE_ID;
> +
> + cmd.stream_cfg.b_enable_tnr = true;
> + dev_dbg(dev, "isp4fw_sensor_id %d, pipeId 0x%x EnableTnr %u\n",
> + cmd.stream_cfg.mipi_pipe_path_cfg.isp4fw_sensor_id,
> + cmd.stream_cfg.isp_pipe_path_cfg.isp_pipe_id,
> + cmd.stream_cfg.b_enable_tnr);
> +
> + return isp4if_send_command(ispif, CMD_ID_SET_STREAM_CONFIG,
> + &cmd, sizeof(cmd));
> +}
> +
> +static int isp4sd_send_meta_buf(struct isp4_subdev *isp_subdev)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4fw_cmd_send_buffer buf_type = {};
Use memset to guarantee zeroing of padding bits.
> + struct isp4sd_sensor_info *sensor_info;
> + struct device *dev = isp_subdev->dev;
> + u32 i;
> +
> + sensor_info = &isp_subdev->sensor_info;
> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
> + int ret;
> +
> + if (!sensor_info->meta_info_buf[i]) {
> + dev_err(dev, "fail for no meta info buf(%u)\n", i);
> + return -ENOMEM;
> + }
> + buf_type.buffer_type = BUFFER_TYPE_META_INFO;
> + buf_type.buffer.buf_tags = 0;
> + buf_type.buffer.vmid_space.bit.vmid = 0;
> + buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
> + isp4if_split_addr64(sensor_info->meta_info_buf[i]->gpu_mc_addr,
> + &buf_type.buffer.buf_base_a_lo,
> + &buf_type.buffer.buf_base_a_hi);
> + buf_type.buffer.buf_size_a =
> + (u32)sensor_info->meta_info_buf[i]->mem_size;
> + ret = isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
> + &buf_type,
> + sizeof(buf_type));
> + if (ret) {
> + dev_err(dev, "send meta info(%u) fail\n", i);
> + return ret;
> + }
> + }
> +
> + dev_dbg(dev, "send meta info suc\n");
> + return 0;
> +}
[snip]
> +static int isp4sd_setup_output(struct isp4_subdev *isp_subdev,
> + struct v4l2_subdev_state *state, u32 pad)
> +{
> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
> + struct isp4sd_output_info *output_info =
> + &isp_subdev->sensor_info.output_info;
> + struct isp4fw_cmd_set_out_ch_prop cmd_ch_prop = {};
Use memset to guarantee zeroing of padding bits.
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4fw_cmd_enable_out_ch cmd_ch_en = {};
Use memset to guarantee zeroing of padding bits.
> + struct device *dev = isp_subdev->dev;
> + struct isp4fw_image_prop *out_prop;
> + int ret;
> +
> + if (output_info->start_status == ISP4SD_START_STATUS_STARTED)
> + return 0;
> +
> + if (output_info->start_status == ISP4SD_START_STATUS_START_FAIL) {
> + dev_err(dev, "fail for previous start fail\n");
> + return -EINVAL;
> + }
> +
> + out_prop = &cmd_ch_prop.image_prop;
> + cmd_ch_prop.ch = ISP_PIPE_OUT_CH_PREVIEW;
> + cmd_ch_en.ch = ISP_PIPE_OUT_CH_PREVIEW;
> + cmd_ch_en.is_enable = true;
> +
> + if (!isp4sd_get_str_out_prop(isp_subdev, out_prop, state, pad)) {
> + dev_err(dev, "fail to get out prop\n");
> + return -EINVAL;
> + }
> +
> + dev_dbg(dev, "channel: w:h=%u:%u,lp:%u,cp%u\n",
> + cmd_ch_prop.image_prop.width, cmd_ch_prop.image_prop.height,
> + cmd_ch_prop.image_prop.luma_pitch,
> + cmd_ch_prop.image_prop.chroma_pitch);
> +
> + ret = isp4if_send_command(ispif, CMD_ID_SET_OUT_CHAN_PROP,
> + &cmd_ch_prop,
> + sizeof(cmd_ch_prop));
> + if (ret) {
> + output_info->start_status = ISP4SD_START_STATUS_START_FAIL;
> + dev_err(dev, "fail to set out prop\n");
> + return ret;
> + };
> +
> + ret = isp4if_send_command(ispif, CMD_ID_ENABLE_OUT_CHAN,
> + &cmd_ch_en, sizeof(cmd_ch_en));
> +
> + if (ret) {
> + output_info->start_status = ISP4SD_START_STATUS_START_FAIL;
> + dev_err(dev, "fail to enable channel\n");
> + return ret;
> + }
> +
> + if (!sensor_info->start_stream_cmd_sent) {
> + ret = isp4sd_kickoff_stream(isp_subdev, out_prop->width,
> + out_prop->height);
> + if (ret) {
> + dev_err(dev, "kickoff stream fail %d\n", ret);
> + return ret;
> + }
> + /*
> + * sensor_info->start_stream_cmd_sent will be set to true
> + * 1. in isp4sd_kickoff_stream, if app first send buffer then
> + * start stream
> + * 2. in isp_set_stream_buf, if app first start stream, then
> + * send buffer
> + * because ISP FW has the requirement, host needs to send buffer
> + * before send start stream cmd
> + */
> + if (sensor_info->start_stream_cmd_sent) {
> + sensor_info->status = ISP4SD_START_STATUS_STARTED;
> + output_info->start_status = ISP4SD_START_STATUS_STARTED;
> + dev_dbg(dev, "kickoff stream suc,start cmd sent\n");
> + }
> + } else {
> + dev_dbg(dev, "stream running, no need kickoff\n");
> + output_info->start_status = ISP4SD_START_STATUS_STARTED;
> + }
> +
> + dev_dbg(dev, "setup output suc\n");
> + return 0;
> +}
> +
> +static int isp4sd_init_meta_buf(struct isp4_subdev *isp_subdev)
> +{
> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct device *dev = isp_subdev->dev;
> + u32 i;
> +
> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
> + if (!sensor_info->meta_info_buf[i]) {
> + sensor_info->meta_info_buf[i] = ispif->metainfo_buf_pool[i];
> + if (!sensor_info->meta_info_buf[i]) {
> + dev_err(dev, "invalid %u meta_info_buf fail\n", i);
> + return -ENOMEM;
> + }
> + }
> + }
> +
> + return 0;
> +}
What is the point of metainfo_buf_pool? Especially since metainfo_buf_pool[i] is
not set to NULL after this "allocation" occurs.
I think isp4sd_init_meta_buf() and metainfo_buf_pool are unnecessary and can be
factored out.
> +
> +static int isp4sd_init_stream(struct isp4_subdev *isp_subdev)
> +{
> + struct device *dev = isp_subdev->dev;
> + int ret;
> +
> + ret = isp4sd_setup_fw_mem_pool(isp_subdev);
> + if (ret) {
> + dev_err(dev, "fail to setup fw mem pool\n");
> + return ret;
> + }
> +
> + ret = isp4sd_init_meta_buf(isp_subdev);
> + if (ret) {
> + dev_err(dev, "fail to alloc fw driver shared buf\n");
> + return ret;
> + }
> +
> + ret = isp4sd_set_stream_path(isp_subdev);
> + if (ret) {
> + dev_err(dev, "fail to setup stream path\n");
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static void isp4sd_reset_stream_info(struct isp4_subdev *isp_subdev,
> + struct v4l2_subdev_state *state, u32 pad)
> +{
> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
> + struct v4l2_mbus_framefmt *format = NULL;
Remove unnecessary initialization of `format`.
> + struct isp4sd_output_info *str_info;
> + int i;
> +
> + format = v4l2_subdev_state_get_format(state, pad, 0);
> +
> + if (!format) {
> + dev_err(isp_subdev->dev, "fail to setup stream path\n");
> + } else {
> + memset(format, 0, sizeof(*format));
> + format->code = MEDIA_BUS_FMT_YUYV8_1_5X8;
> + }
> +
> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++)
> + sensor_info->meta_info_buf[i] = NULL;
> +
> + str_info = &sensor_info->output_info;
> + str_info->start_status = ISP4SD_START_STATUS_NOT_START;
> +}
[snip]
> +static struct isp4fw_meta_info *
> +isp4sd_get_meta_by_mc(struct isp4_subdev *isp_subdev,
> + u64 mc)
> +{
> + u32 i;
Change u32 to int for `i` to match other ISP4IF_MAX_STREAM_BUF_COUNT loops.
> +
> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
> + struct isp4if_gpu_mem_info *meta_info_buf =
> + isp_subdev->sensor_info.meta_info_buf[i];
> +
> + if (meta_info_buf) {
> + if (mc == meta_info_buf->gpu_mc_addr)
> + return meta_info_buf->sys_addr;
> + }
meta_info_buf is never NULL. Also it's easier to read with the constant operand
on the right side of the comparison. Change to:
if (meta_info_buf->gpu_mc_addr == mc)
return meta_info_buf->sys_addr;
> + }
> + return NULL;
> +};
Remove unnecessary ; after }.
> +
> +static struct isp4if_img_buf_node *
> +isp4sd_preview_done(struct isp4_subdev *isp_subdev,
> + struct isp4fw_meta_info *meta)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4if_img_buf_node *prev = NULL;
> + struct device *dev = isp_subdev->dev;
> +
> + if (meta->preview.enabled &&
> + (meta->preview.status == BUFFER_STATUS_SKIPPED ||
> + meta->preview.status == BUFFER_STATUS_DONE ||
> + meta->preview.status == BUFFER_STATUS_DIRTY)) {
> + prev = isp4if_dequeue_buffer(ispif);
> + if (!prev)
> + dev_err(dev, "fail null prev buf\n");
> +
> + } else if (meta->preview.enabled) {
> + dev_err(dev, "fail bad preview status %u\n",
> + meta->preview.status);
> + }
> +
> + return prev;
> +}
> +
> +static void isp4sd_send_meta_info(struct isp4_subdev *isp_subdev,
> + u64 meta_info_mc)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4fw_cmd_send_buffer buf_type = {};
Use memset to guarantee zeroing of padding bits.
> + struct device *dev = isp_subdev->dev;
> +
> + if (isp_subdev->sensor_info.status != ISP4SD_START_STATUS_STARTED) {
> + dev_warn(dev, "not working status %i, meta_info 0x%llx\n",
> + isp_subdev->sensor_info.status, meta_info_mc);
> + return;
> + }
> +
> + if (meta_info_mc) {
> + buf_type.buffer_type = BUFFER_TYPE_META_INFO;
> + buf_type.buffer.buf_tags = 0;
> + buf_type.buffer.vmid_space.bit.vmid = 0;
> + buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
> + isp4if_split_addr64(meta_info_mc,
> + &buf_type.buffer.buf_base_a_lo,
> + &buf_type.buffer.buf_base_a_hi);
> +
> + buf_type.buffer.buf_size_a = ISP4SD_META_BUF_SIZE;
> + if (isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
> + &buf_type, sizeof(buf_type))) {
> + dev_err(dev, "fail send meta_info 0x%llx\n",
> + meta_info_mc);
> + } else {
> + dev_dbg(dev, "resend meta_info 0x%llx\n", meta_info_mc);
> + }
> + }
> +}
> +
> +static void isp4sd_fw_resp_frame_done(struct isp4_subdev *isp_subdev,
> + enum isp4if_stream_id stream_id,
> + struct isp4fw_resp_param_package *para)
> +{
> + struct isp4if_img_buf_node *prev = NULL;
> + struct device *dev = isp_subdev->dev;
> + struct isp4fw_meta_info *meta;
> + u64 mc = 0;
> +
> + mc = isp4if_join_addr64(para->package_addr_lo, para->package_addr_hi);
> + meta = isp4sd_get_meta_by_mc(isp_subdev, mc);
> + if (mc == 0 || !meta) {
If `mc == 0` is always an error then why pass it to isp4sd_get_meta_by_mc()?
Change it to skip isp4sd_get_meta_by_mc() when mc is 0, or make
isp4sd_get_meta_by_mc() return NULL when mc is 0 and then you can remove the
`mc == 0` check from here.
> + dev_err(dev, "fail to get meta from mc %llx\n", mc);
> + return;
> + }
> +
> + dev_dbg(dev, "ts:%llu,streamId:%d,poc:%u,preview_en:%u,(%i)\n",
> + ktime_get_ns(), stream_id, meta->poc,
> + meta->preview.enabled,
> + meta->preview.status);
> +
> + prev = isp4sd_preview_done(isp_subdev, meta);
> +
> + isp4if_dealloc_buffer_node(prev);
> +
> + if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)
> + isp4sd_send_meta_info(isp_subdev, mc);
> +
> + dev_dbg(dev, "stream_id:%d, status:%d\n", stream_id,
> + isp_subdev->sensor_info.status);
> +}
> +
> +static void isp4sd_fw_resp_func(struct isp4_subdev *isp_subdev,
> + enum isp4if_stream_id stream_id)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct device *dev = isp_subdev->dev;
> + struct isp4fw_resp resp;
> +
> + if (ispif->status < ISP4IF_STATUS_FW_RUNNING)
> + return;
If the lifetime of the kthread is managed correctly, this check shouldn't be
needed.
> +
> + while (true) {
> + s32 ret;
Change from s32 to int to match isp4if_f2h_resp().
> +
> + ret = isp4if_f2h_resp(ispif, stream_id, &resp);
> + if (ret)
> + break;
This loop just parses responses from firmware until the ringbuffer has nothing
new left, which kind of makes the IRQ useless when this scenario occurs, meaning
that you can mask the IRQ from the IRQ handler and then unmask it when the
resp kthread goes back to sleep. So the IRQ doesn't fire needlessly.
> +
> + switch (resp.resp_id) {
> + case RESP_ID_CMD_DONE:
> + isp4sd_fw_resp_cmd_done(isp_subdev, stream_id,
> + &resp.param.cmd_done);
> + break;
> + case RESP_ID_NOTI_FRAME_DONE:
> + isp4sd_fw_resp_frame_done(isp_subdev, stream_id,
> + &resp.param.frame_done);
> + break;
> + default:
> + dev_err(dev, "-><- fail respid (0x%x)\n",
> + resp.resp_id);
> + break;
> + }
> + }
> +}
> +
> +static s32 isp4sd_fw_resp_thread_wrapper(void *context)
> +{
> + struct isp4_subdev_thread_param *para = context;
> + struct isp4sd_thread_handler *thread_ctx;
> + enum isp4if_stream_id stream_id;
> +
> + struct isp4_subdev *isp_subdev;
> + struct device *dev;
> + u64 timeout;
> +
> + if (!para)
> + return -EINVAL;
> +
> + isp_subdev = para->isp_subdev;
> + dev = isp_subdev->dev;
> +
> + switch (para->idx) {
> + case 0:
> + stream_id = ISP4IF_STREAM_ID_GLOBAL;
> + break;
> + case 1:
> + stream_id = ISP4IF_STREAM_ID_1;
> + break;
> + default:
> + dev_err(dev, "fail invalid %d\n", para->idx);
> + return -EINVAL;
> + }
> +
> + thread_ctx = &isp_subdev->fw_resp_thread[para->idx];
> +
> + thread_ctx->wq_cond = 0;
> + mutex_init(&thread_ctx->mutex);
This mutex doesn't protect anything. Remove it.
> + init_waitqueue_head(&thread_ctx->waitq);
> + timeout = msecs_to_jiffies(ISP4SD_WAIT_RESP_IRQ_TIMEOUT);
> +
> + dev_dbg(dev, "[%u] started\n", para->idx);
> +
> + while (true) {
> + wait_event_interruptible_timeout(thread_ctx->waitq,
> + thread_ctx->wq_cond != 0,
> + timeout);
Why is there a timeout? What does the timeout even do since the return value of
wait_event_interruptible_timeout() is not checked? Doesn't that mean that once
the timeout is hit, isp4sd_fw_resp_func() will be called for nothing?
I observe that most of the time spent by these kthreads is due to the constant
wake-ups from the very short 5 ms timeout. This is bad for energy efficiency and
creates needless overhead.
> + thread_ctx->wq_cond = 0;
> +
> + if (kthread_should_stop()) {
> + dev_dbg(dev, "[%u] quit\n", para->idx);
> + break;
> + }
> +
> + guard(mutex)(&thread_ctx->mutex);
> + isp4sd_fw_resp_func(isp_subdev, stream_id);
> + }
> +
> + mutex_destroy(&thread_ctx->mutex);
> +
> + return 0;
> +}
> +
> +static int isp4sd_start_resp_proc_threads(struct isp4_subdev *isp_subdev)
> +{
> + struct device *dev = isp_subdev->dev;
> + int i;
> +
> + for (i = 0; i < ISP4SD_MAX_FW_RESP_STREAM_NUM; i++) {
> + struct isp4sd_thread_handler *thread_ctx =
> + &isp_subdev->fw_resp_thread[i];
> +
> + isp_subdev->isp_resp_para[i].idx = i;
> + isp_subdev->isp_resp_para[i].isp_subdev = isp_subdev;
> +
> + thread_ctx->thread = kthread_run(isp4sd_fw_resp_thread_wrapper,
> + &isp_subdev->isp_resp_para[i],
> + "amd_isp4_thread");
The kthread name and also the IRQ name can be made more informative by appending
the index number.
> + if (IS_ERR(thread_ctx->thread)) {
> + dev_err(dev, "create thread [%d] fail\n", i);
> + return -EINVAL;
> + }
> + }
> +
> + return 0;
> +}
[snip]
> +static u32 isp4sd_get_started_stream_count(struct isp4_subdev *isp_subdev)
> +{
> + u32 cnt = 0;
> +
> + if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)
> + cnt++;
> + return cnt;
> +}
isp4sd_get_started_stream_count() is unnecessary, remove it and just use
`if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)` instead.
> +
> +static int isp4sd_pwroff_and_deinit(struct isp4_subdev *isp_subdev)
> +{
> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
> + unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_LOW;
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> +
> + struct device *dev = isp_subdev->dev;
> + u32 cnt;
> + int ret;
> +
> + mutex_lock(&isp_subdev->ops_mutex);
> +
> + if (sensor_info->status == ISP4SD_START_STATUS_STARTED) {
> + dev_err(dev, "fail for stream still running\n");
> + mutex_unlock(&isp_subdev->ops_mutex);
> + return -EINVAL;
> + }
> +
> + sensor_info->status = ISP4SD_START_STATUS_NOT_START;
> + cnt = isp4sd_get_started_stream_count(isp_subdev);
> + if (cnt > 0) {
> + dev_dbg(dev, "no need power off isp_subdev\n");
> + mutex_unlock(&isp_subdev->ops_mutex);
> + return 0;
> + }
> +
> + isp4if_stop(ispif);
> +
> + ret = dev_pm_genpd_set_performance_state(dev, perf_state);
> + if (ret)
> + dev_err(dev,
> + "fail to set isp_subdev performance state %u,ret %d\n",
> + perf_state, ret);
> + isp4sd_stop_resp_proc_threads(isp_subdev);
> + dev_dbg(dev, "isp_subdev stop resp proc streads suc");
> + /* hold ccpu reset */
> + isp4hw_wreg(isp_subdev->mmio, ISP_SOFT_RESET, 0x0);
> + isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0);
> + ret = pm_runtime_put_sync(dev);
> + if (ret)
> + dev_err(dev, "power off isp_subdev fail %d\n", ret);
> + else
> + dev_dbg(dev, "power off isp_subdev suc\n");
> +
> + ispif->status = ISP4IF_STATUS_PWR_OFF;
> + isp4if_clear_cmdq(ispif);
> + isp4sd_module_enable(isp_subdev, false);
> +
> + msleep(20);
What is this msleep for?
> +
> + mutex_unlock(&isp_subdev->ops_mutex);
> +
> + return 0;
> +}
> +
> +static int isp4sd_pwron_and_init(struct isp4_subdev *isp_subdev)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct device *dev = isp_subdev->dev;
> + int ret;
> +
> + if (ispif->status == ISP4IF_STATUS_FW_RUNNING) {
`ispif->status` is checked under ops_mutex elsewhere but not in this function?
> + dev_dbg(dev, "camera already opened, do nothing\n");
> + return 0;
> + }
> +
> + mutex_lock(&isp_subdev->ops_mutex);
> +
> + isp4sd_module_enable(isp_subdev, true);
> +
> + isp_subdev->sensor_info.start_stream_cmd_sent = false;
> + isp_subdev->sensor_info.buf_sent_cnt = 0;
> +
> + if (ispif->status < ISP4IF_STATUS_PWR_ON) {
> + unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_HIGH;
> +
> + ret = pm_runtime_resume_and_get(dev);
> + if (ret) {
> + dev_err(dev, "fail to power on isp_subdev ret %d\n",
> + ret);
> + goto err_unlock_and_close;
> + }
> +
> + /* ISPPG ISP Power Status */
> + isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0x7FF);
> + ret = dev_pm_genpd_set_performance_state(dev, perf_state);
> + if (ret) {
> + dev_err(dev,
> + "fail to set performance state %u, ret %d\n",
> + perf_state, ret);
> + goto err_unlock_and_close;
> + }
> +
> + ispif->status = ISP4IF_STATUS_PWR_ON;
> +
> + if (isp4sd_start_resp_proc_threads(isp_subdev)) {
> + dev_err(dev, "isp_start_resp_proc_threads fail");
> + goto err_unlock_and_close;
> + } else {
> + dev_dbg(dev, "create resp threads ok");
> + }
> + }
> +
> + isp_subdev->sensor_info.start_stream_cmd_sent = false;
> + isp_subdev->sensor_info.buf_sent_cnt = 0;
> +
> + ret = isp4if_start(ispif);
> + if (ret) {
> + dev_err(dev, "fail to start isp_subdev interface\n");
> + goto err_unlock_and_close;
> + }
> +
> + mutex_unlock(&isp_subdev->ops_mutex);
> + return 0;
> +err_unlock_and_close:
> + mutex_unlock(&isp_subdev->ops_mutex);
> + isp4sd_pwroff_and_deinit(isp_subdev);
> + return -EINVAL;
> +}
[snip]
> +static int isp4sd_set_power(struct v4l2_subdev *sd, int on)
> +{
> + struct isp4_subdev *ispsd = to_isp4_subdev(sd);
`ispsd` is the variable name used here and in a couple other functions but
`isp_subdev` is the name used everywhere else. Make the variable name consistent
for `struct isp4_subdev *`.
> +
Add `guard(mutex)(&ispsd->ops_mutex);` here and remove all uses of ops_mutex
from isp4sd_pwron_and_init() and isp4sd_pwroff_and_deinit(). This simplifies the
locking and also ensures that the lock is not released and reacquired when
isp4sd_pwron_and_init() needs to call isp4sd_pwroff_and_deinit() on error.
> + if (on)
> + return isp4sd_pwron_and_init(ispsd);
> + else
> + return isp4sd_pwroff_and_deinit(ispsd);
> +};
Remove unnecessary ; after }.
> +
> +static const struct v4l2_subdev_core_ops isp4sd_core_ops = {
> + .s_power = isp4sd_set_power,
> +};
[snip]
> +int isp4sd_init(struct isp4_subdev *isp_subdev,
> + struct v4l2_device *v4l2_dev)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4sd_sensor_info *sensor_info;
> + struct device *dev = v4l2_dev->dev;
> + int ret;
> +
> + isp_subdev->dev = dev;
> + v4l2_subdev_init(&isp_subdev->sdev, &isp4sd_subdev_ops);
> + isp_subdev->sdev.owner = THIS_MODULE;
> + isp_subdev->sdev.dev = dev;
> + snprintf(isp_subdev->sdev.name, sizeof(isp_subdev->sdev.name), "%s",
> + dev_name(dev));
> +
> + isp_subdev->sdev.entity.name = isp4sd_entity_name;
> + isp_subdev->sdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
> + isp_subdev->sdev.entity.ops = &isp4sd_sdev_ent_ops;
> + isp_subdev->sdev_pad.flags = MEDIA_PAD_FL_SOURCE;
> + ret = media_entity_pads_init(&isp_subdev->sdev.entity, 1,
> + &isp_subdev->sdev_pad);
> + if (ret) {
> + dev_err(dev, "fail to init isp4 subdev entity pad %d\n", ret);
> + return ret;
> + }
> + ret = v4l2_subdev_init_finalize(&isp_subdev->sdev);
> + if (ret < 0) {
> + dev_err(dev, "fail to init finalize isp4 subdev %d\n",
> + ret);
> + return ret;
> + }
> + ret = v4l2_device_register_subdev(v4l2_dev, &isp_subdev->sdev);
> + if (ret) {
> + dev_err(dev, "fail to register isp4 subdev to V4L2 device %d\n",
> + ret);
> + goto err_media_clean_up;
Missing error handling: v4l2_subdev_cleanup() is not called.
> + }
> +
> + sensor_info = &isp_subdev->sensor_info;
> +
> + isp4if_init(ispif, dev, isp_subdev->mmio);
> +
> + mutex_init(&isp_subdev->ops_mutex);
> + sensor_info->start_stream_cmd_sent = false;
> + sensor_info->status = ISP4SD_START_STATUS_NOT_START;
> +
> + /* create ISP enable gpio control */
> + isp_subdev->enable_gpio = devm_gpiod_get(isp_subdev->dev,
> + "enable_isp",
> + GPIOD_OUT_LOW);
> + if (IS_ERR(isp_subdev->enable_gpio)) {
> + dev_err(dev, "fail to get gpiod %d\n", ret);
This prints ret instead of the actual error, PTR_ERR(isp_subdev->enable_gpio).
Instead, add `ret = PTR_ERR(isp_subdev->enable_gpio);` before the dev_err().
> + media_entity_cleanup(&isp_subdev->sdev.entity);
> + return PTR_ERR(isp_subdev->enable_gpio);
Missing error handling: v4l2_device_unregister_subdev() is not called.
Add another goto label and use that instead of returning here.
> + }
> +
> + isp_subdev->host2fw_seq_num = 1;
> + ispif->status = ISP4IF_STATUS_PWR_OFF;
> +
> + if (ret)
> + goto err_media_clean_up;
> + return ret;
ret is always zero at this point. Remove this `if (ret) ...` and change the
return to `return 0`.
> +
> +err_media_clean_up:
> + media_entity_cleanup(&isp_subdev->sdev.entity);
> + return ret;
> +}
> +
> +void isp4sd_deinit(struct isp4_subdev *isp_subdev)
> +{
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> +
> + media_entity_cleanup(&isp_subdev->sdev.entity);
> + isp4if_deinit(ispif);
> + isp4sd_module_enable(isp_subdev, false);
> +
> + ispif->status = ISP4IF_STATUS_PWR_OFF;
> +}
> diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.h b/drivers/media/platform/amd/isp4/isp4_subdev.h
> new file mode 100644
> index 000000000000..524a8de5e18d
> --- /dev/null
> +++ b/drivers/media/platform/amd/isp4/isp4_subdev.h
> @@ -0,0 +1,131 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#ifndef _ISP4_CONTEXT_H_
> +#define _ISP4_CONTEXT_H_
> +
> +#include <linux/delay.h>
> +#include <linux/firmware.h>
> +#include <linux/platform_device.h>
> +#include <linux/uaccess.h>
> +#include <linux/types.h>
> +#include <linux/debugfs.h>
> +#include <media/v4l2-device.h>
> +
> +#include "isp4_fw_cmd_resp.h"
> +#include "isp4_hw_reg.h"
> +#include "isp4_interface.h"
> +
> +/*
> + * one is for none sesnor specefic response which is not used now
> + * another is for sensor specific response
> + */
> +#define ISP4SD_MAX_FW_RESP_STREAM_NUM 2
Only two out of four possible streams are used, yet IRQs are requested for all
four streams in isp4_capture_probe(). ISP4SD_MAX_FW_RESP_STREAM_NUM should be
checked and used to limit the number of IRQs requested in isp4_capture_probe().
> +
> +/*
> + * cmd used to register frame done callback, parameter is
> + * struct isp4sd_register_framedone_cb_param *
> + * when a image buffer is filled by ISP, ISP will call the registered callback.
> + * callback func prototype is isp4sd_framedone_cb, cb_ctx can be anything
> + * provided by caller which will be provided back as the first parameter of the
> + * callback function.
> + * both cb_func and cb_ctx are provide by caller, set cb_func to NULL to
> + * unregister the callback
> + */
[snip]
> +void isp4sd_deinit(struct isp4_subdev *isp_subdev);
> +
> +#endif
Add /* _ISP4_CONTEXT_H_ */
> --
> 2.34.1
>
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver
2025-09-21 20:23 ` Sultan Alsawaf
@ 2025-09-23 7:56 ` Du, Bin
0 siblings, 0 replies; 35+ messages in thread
From: Du, Bin @ 2025-09-23 7:56 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Svetoslav Stoilov, Mario Limonciello,
Alexey Zagorodnikov
Many thanks Sultan for your careful review.
On 9/22/2025 4:23 AM, Sultan Alsawaf wrote:
> Hi Bin,
>
> On Thu, Sep 11, 2025 at 06:08:41PM +0800, Bin Du wrote:
>> AMD isp4 capture is a v4l2 media device which implements media controller
>> interface. It has one sub-device (AMD ISP4 sub-device) endpoint which can
>> be connected to a remote CSI2 TX endpoint. It supports only one physical
>> interface for now. Also add ISP4 driver related entry info into the
>> MAINTAINERS file
>>
>> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Bin Du <Bin.Du@amd.com>
>> Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
>> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/Kconfig
>> @@ -0,0 +1,13 @@
>> +# SPDX-License-Identifier: GPL-2.0+
>> +
>> +config AMD_ISP4
>> + tristate "AMD ISP4 and camera driver"
>> + depends on VIDEO_DEV && VIDEO_V4L2_SUBDEV_API && DRM_AMD_ISP
>> + select VIDEOBUF2_CORE
>> + select VIDEOBUF2_V4L2
>> + select VIDEOBUF2_MEMOPS
>
> VIDEO_V4L2_SUBDEV_API should be selected rather than set as a dependency, per
> what other drivers do. You can also sort the dependencies to look cleaner.
>
> Change to:
>
> depends on VIDEO_DEV && DRM_AMD_ISP
> select VIDEOBUF2_CORE
> select VIDEOBUF2_MEMOPS
> select VIDEOBUF2_V4L2
> select VIDEO_V4L2_SUBDEV_API
>
Sure, will change VIDEO_V4L2_SUBDEV_API from depends to select. Had a
check, yes, you are correct, almost all driver are doing like this with
only one exception drivers/media/platform/ti/Kconfig
>> + help
>> + This is support for AMD ISP4 and camera subsystem driver.
>> + Say Y here to enable the ISP4 and camera device for video capture.
>> + To compile this driver as a module, choose M here. The module will
>> + be called amd_capture.
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/isp4.c
>> @@ -0,0 +1,121 @@
>> +// SPDX-License-Identifier: GPL-2.0+
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#include <linux/pm_runtime.h>
>> +#include <linux/vmalloc.h>
>> +#include <media/v4l2-ioctl.h>
>> +
>> +#include "isp4.h"
>> +
>> +#define VIDEO_BUF_NUM 5
>> +
>> +#define ISP4_DRV_NAME "amd_isp_capture"
>> +
>> +/* interrupt num */
>> +static const u32 isp4_ringbuf_interrupt_num[] = {
>> + 0, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT9 */
>> + 1, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT10 */
>> + 3, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT11 */
>> + 4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
>> +};
>> +
>> +#define to_isp4_device(dev) \
>> + ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
>
> The unnecessary cast on container_of() is removed later in "[PATCH v4 4/7]
> media: platform: amd: isp4 subdev and firmware loading handling added".
>
> Remove the cast in this patch instead.
>
Sure, will remove it directly from this patch
>> +
>> +static irqreturn_t isp4_irq_handler(int irq, void *arg)
>> +{
>> + return IRQ_HANDLED;
>> +}
>> +
>> +static int isp4_capture_probe(struct platform_device *pdev)
>> +{
>> + struct device *dev = &pdev->dev;
>> + struct isp4_device *isp_dev;
>> + int i, irq, ret;
>> +
>> + isp_dev = devm_kzalloc(&pdev->dev, sizeof(*isp_dev), GFP_KERNEL);
>
> s/&pdev->dev/dev/
>
Sure, will update in the next version.
>> + if (!isp_dev)
>> + return -ENOMEM;
>> +
>> + isp_dev->pdev = pdev;
>> + dev->init_name = ISP4_DRV_NAME;
>> +
>> + for (i = 0; i < ARRAY_SIZE(isp4_ringbuf_interrupt_num); i++) {
>> + irq = platform_get_irq(pdev, isp4_ringbuf_interrupt_num[i]);
>> + if (irq < 0)
>> + return dev_err_probe(dev, -ENODEV,
>> + "fail to get irq %d\n",
>> + isp4_ringbuf_interrupt_num[i]);
>
> Return the error from platform_get_irq() here instead of -ENODEV.
>
Thanks, will update in the next version.
>> + ret = devm_request_irq(&pdev->dev, irq, isp4_irq_handler, 0,
>> + "ISP_IRQ", &pdev->dev);
>
> s/&pdev->dev/dev/
>
Sure, will update in the next version.
>> + if (ret)
>> + return dev_err_probe(dev, ret, "fail to req irq %d\n",
>> + irq);
>> + }
>> +
>> + /* Link the media device within the v4l2_device */
>> + isp_dev->v4l2_dev.mdev = &isp_dev->mdev;
>> +
>> + /* Initialize media device */
>> + strscpy(isp_dev->mdev.model, "amd_isp41_mdev",
>> + sizeof(isp_dev->mdev.model));
>> + snprintf(isp_dev->mdev.bus_info, sizeof(isp_dev->mdev.bus_info),
>> + "platform:%s", ISP4_DRV_NAME);
>> + isp_dev->mdev.dev = &pdev->dev;
>
> s/&pdev->dev/dev/
>
Sure, will update in the next version.
>> + media_device_init(&isp_dev->mdev);
>> +
>> + /* register v4l2 device */
>> + snprintf(isp_dev->v4l2_dev.name, sizeof(isp_dev->v4l2_dev.name),
>> + "AMD-V4L2-ROOT");
>> + ret = v4l2_device_register(&pdev->dev, &isp_dev->v4l2_dev);
>
> s/&pdev->dev/dev/
>
Sure, will update in the next version.
>> + if (ret)
>> + return dev_err_probe(dev, ret,
>> + "fail register v4l2 device\n");
>> +
>> + ret = media_device_register(&isp_dev->mdev);
>> + if (ret) {
>> + dev_err(dev, "fail to register media device %d\n", ret);
>> + goto err_unreg_v4l2;
>> + }
>> +
>> + platform_set_drvdata(pdev, isp_dev);
>> +
>> + pm_runtime_set_suspended(dev);
>> + pm_runtime_enable(dev);
>> +
>> + return 0;
>> +
>> +err_unreg_v4l2:
>> + v4l2_device_unregister(&isp_dev->v4l2_dev);
>> +
>> + return dev_err_probe(dev, ret, "isp probe fail\n");
>> +}
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/isp4.h
>> @@ -0,0 +1,24 @@
>> +/* SPDX-License-Identifier: GPL-2.0+ */
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#ifndef _ISP4_H_
>> +#define _ISP4_H_
>> +
>> +#include <linux/mutex.h>
>
> Remove this linux/mutex.h include. It should be moved to isp4_subdev.h instead.
>
Sure, will update in the next version.
>> +#include <media/v4l2-device.h>
>> +#include <media/videobuf2-memops.h>
>> +#include <media/videobuf2-vmalloc.h>
>
> This media/videobuf2-vmalloc.h include is removed in "[PATCH v4 4/7] media:
> platform: amd: isp4 subdev and firmware loading handling added".
>
> Remove it in this patch instead.
>
Sure, will remove it directly from this patch.
>> +
>> +#define ISP4_GET_ISP_REG_BASE(isp4sd) (((isp4sd))->mmio)
>> +
>> +struct isp4_device {
>> + struct v4l2_device v4l2_dev;
>> + struct media_device mdev;
>> +
>> + struct platform_device *pdev;
>> + struct notifier_block i2c_nb;
>
> i2c_nb is unused, remove it.
>
Sure, will remove it and other unused fields in this struct.
>> +};
>> +
>> +#endif /* _ISP4_H_ */
>> --
>> 2.34.1
>>
>
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 2/7] media: platform: amd: low level support for isp4 firmware
2025-09-21 20:31 ` Sultan Alsawaf
@ 2025-09-23 8:05 ` Du, Bin
0 siblings, 0 replies; 35+ messages in thread
From: Du, Bin @ 2025-09-23 8:05 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Many thanks Sultan for the careful review.
On 9/22/2025 4:31 AM, Sultan Alsawaf wrote:
> Hi Bin,
>
> On Thu, Sep 11, 2025 at 06:08:42PM +0800, Bin Du wrote:
>> Low level functions for accessing the registers and mapping to their
>> ranges. This change also includes register definitions for ring buffer
>> used to communicate with ISP Firmware. Ring buffer is the communication
>> interface between driver and ISP Firmware. Command and responses are
>> exchanged through the ring buffer.
>>
>> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Bin Du <Bin.Du@amd.com>
>> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/Makefile
>> @@ -3,4 +3,4 @@
>> # Copyright (C) 2025 Advanced Micro Devices, Inc.
>>
>> obj-$(CONFIG_AMD_ISP4) += amd_capture.o
>> -amd_capture-objs := isp4.o
>> +amd_capture-objs := isp4.o \
>
> Remove this hunk since there are no new objects added in this patch.
>
Yes, you are correct, will remove it in the next version
>> +++ b/drivers/media/platform/amd/isp4/isp4_hw_reg.h
>> @@ -0,0 +1,125 @@
>> +/* SPDX-License-Identifier: GPL-2.0+ */
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#ifndef _ISP4_HW_REG_H_
>> +#define _ISP4_HW_REG_H_
>> +
>> +#include <linux/io.h>
>> +#include <linux/types.h>
>
> Remove redundant linux/types.h include, as it is included by linux/io.h.
>
Sure, will remove it in the next version
>> +
>> +#define ISP_SOFT_RESET 0x62000
>
> [snip]
>
>> +
>> +#endif
>
> Add /* _ISP4_HW_REG_H_ */
>
Sure, will add it in the next version and also check all other header files.
>> --
>> 2.34.1
>>
>
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface
2025-09-21 21:55 ` Sultan Alsawaf
@ 2025-09-23 9:24 ` Du, Bin
2025-09-24 7:09 ` Sultan Alsawaf
0 siblings, 1 reply; 35+ messages in thread
From: Du, Bin @ 2025-09-23 9:24 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Many thanks Sultan for the careful review, really appreciate that.
On 9/22/2025 5:55 AM, Sultan Alsawaf wrote:
> Hi Bin,
>
> On Thu, Sep 11, 2025 at 06:08:43PM +0800, Bin Du wrote:
>> ISP firmware controls ISP HW pipeline using dedicated embedded processor
>> called ccpu. The communication between ISP FW and driver is using commands
>> and response messages sent through the ring buffer. Command buffers support
>> either global setting that is not specific to the steam and support stream
>> specific parameters. Response buffers contain ISP FW notification
>> information such as frame buffer done and command done. IRQ is used for
>> receiving response buffer from ISP firmware, which is handled in the main
>> isp4 media device. ISP ccpu is booted up through the firmware loading
>> helper function prior to stream start. Memory used for command buffer and
>> response buffer needs to be allocated from amdgpu buffer manager because
>> isp4 is a child device of amdgpu.
>>
>> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Bin Du <Bin.Du@amd.com>
>> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/Makefile
>> @@ -4,3 +4,4 @@
>>
>> obj-$(CONFIG_AMD_ISP4) += amd_capture.o
>> amd_capture-objs := isp4.o \
>> + isp4_interface.o \
>
> Align the objects with spaces and remove the trailing backslash.
>
> I.e., change to:
>
> amd_capture-objs := isp4_subdev.o \
> isp4_interface.o
>
Sure, will align the bojects and remove the trailing backslash, besides
that, will arrange the objects in alphabetical order.
>> +++ b/drivers/media/platform/amd/isp4/isp4_fw_cmd_resp.h
>> @@ -0,0 +1,314 @@
>> +/* SPDX-License-Identifier: GPL-2.0+ */
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#ifndef _ISP4_CMD_RESP_H_
>> +#define _ISP4_CMD_RESP_H_
>> +
>
> [snip]
>
>> +
>> +#endif
>
> Add /* _ISP4_CMD_RESP_H_ */
Sure, will add it and check all other header files.
>
>> diff --git a/drivers/media/platform/amd/isp4/isp4_interface.c b/drivers/media/platform/amd/isp4/isp4_interface.c
>> new file mode 100644
>> index 000000000000..52dcca57ce2e
>> --- /dev/null
>> +++ b/drivers/media/platform/amd/isp4/isp4_interface.c
>> @@ -0,0 +1,955 @@
>> +// SPDX-License-Identifier: GPL-2.0+
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#include <drm/amd/isp.h>
>> +#include <linux/iopoll.h>
>> +#include <linux/mutex.h>
>
> Remove this linux/mutex.h include, it should come from isp4_interface.h instead
> since it is used for a mutex from a isp4_interface.h struct.
>
Sure, will do that.
>> +#include <linux/platform_device.h>
>> +
>> +#include "isp4_fw_cmd_resp.h"
>> +#include "isp4_hw_reg.h"
>> +#include "isp4_interface.h"
>> +
>> +#define ISP4IF_FW_RESP_RB_IRQ_EN_MASK \
>> + (ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT9_EN_MASK | \
>> + ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT10_EN_MASK | \
>> + ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT11_EN_MASK | \
>> + ISP_SYS_INT0_EN__SYS_INT_RINGBUFFER_WPT12_EN_MASK)
>> +
>> +struct isp4if_rb_config {
>> + const char *name;
>> + u32 index;
>> + u32 reg_rptr;
>> + u32 reg_wptr;
>> + u32 reg_base_lo;
>> + u32 reg_base_hi;
>> + u32 reg_size;
>> + u32 val_size;
>> + u64 base_mc_addr;
>> + void *base_sys_addr;
>> +};
>> +
>> +/* FW cmd ring buffer configuration */
>> +static struct isp4if_rb_config
>> + isp4if_cmd_rb_config[ISP4IF_STREAM_ID_MAX] = {
>> + {
>> + .name = "CMD_RB_GBL0",
>> + .index = 3,
>> + .reg_rptr = ISP_RB_RPTR4,
>> + .reg_wptr = ISP_RB_WPTR4,
>> + .reg_base_lo = ISP_RB_BASE_LO4,
>> + .reg_base_hi = ISP_RB_BASE_HI4,
>> + .reg_size = ISP_RB_SIZE4,
>> + },
>> + {
>> + .name = "CMD_RB_STR1",
>> + .index = 0,
>> + .reg_rptr = ISP_RB_RPTR1,
>> + .reg_wptr = ISP_RB_WPTR1,
>> + .reg_base_lo = ISP_RB_BASE_LO1,
>> + .reg_base_hi = ISP_RB_BASE_HI1,
>> + .reg_size = ISP_RB_SIZE1,
>> + },
>> + {
>> + .name = "CMD_RB_STR2",
>> + .index = 1,
>> + .reg_rptr = ISP_RB_RPTR2,
>> + .reg_wptr = ISP_RB_WPTR2,
>> + .reg_base_lo = ISP_RB_BASE_LO2,
>> + .reg_base_hi = ISP_RB_BASE_HI2,
>> + .reg_size = ISP_RB_SIZE2,
>> + },
>> + {
>> + .name = "CMD_RB_STR3",
>> + .index = 2,
>> + .reg_rptr = ISP_RB_RPTR3,
>> + .reg_wptr = ISP_RB_WPTR3,
>> + .reg_base_lo = ISP_RB_BASE_LO3,
>> + .reg_base_hi = ISP_RB_BASE_HI3,
>> + .reg_size = ISP_RB_SIZE3,
>> + },
>> +};
>> +
>> +/* FW resp ring buffer configuration */
>> +static struct isp4if_rb_config
>> + isp4if_resp_rb_config[ISP4IF_STREAM_ID_MAX] = {
>> + {
>> + .name = "RES_RB_GBL0",
>> + .index = 3,
>> + .reg_rptr = ISP_RB_RPTR12,
>> + .reg_wptr = ISP_RB_WPTR12,
>> + .reg_base_lo = ISP_RB_BASE_LO12,
>> + .reg_base_hi = ISP_RB_BASE_HI12,
>> + .reg_size = ISP_RB_SIZE12,
>> + },
>> + {
>> + .name = "RES_RB_STR1",
>> + .index = 0,
>> + .reg_rptr = ISP_RB_RPTR9,
>> + .reg_wptr = ISP_RB_WPTR9,
>> + .reg_base_lo = ISP_RB_BASE_LO9,
>> + .reg_base_hi = ISP_RB_BASE_HI9,
>> + .reg_size = ISP_RB_SIZE9,
>> + },
>> + {
>> + .name = "RES_RB_STR2",
>> + .index = 1,
>> + .reg_rptr = ISP_RB_RPTR10,
>> + .reg_wptr = ISP_RB_WPTR10,
>> + .reg_base_lo = ISP_RB_BASE_LO10,
>> + .reg_base_hi = ISP_RB_BASE_HI10,
>> + .reg_size = ISP_RB_SIZE10,
>> + },
>> + {
>> + .name = "RES_RB_STR3",
>> + .index = 2,
>> + .reg_rptr = ISP_RB_RPTR11,
>> + .reg_wptr = ISP_RB_WPTR11,
>> + .reg_base_lo = ISP_RB_BASE_LO11,
>> + .reg_base_hi = ISP_RB_BASE_HI11,
>> + .reg_size = ISP_RB_SIZE11,
>> + },
>> +};
>> +
>> +/* FW log ring buffer configuration */
>> +static struct isp4if_rb_config isp4if_log_rb_config = {
>> + .name = "LOG_RB",
>> + .index = 0,
>> + .reg_rptr = ISP_LOG_RB_RPTR0,
>> + .reg_wptr = ISP_LOG_RB_WPTR0,
>> + .reg_base_lo = ISP_LOG_RB_BASE_LO0,
>> + .reg_base_hi = ISP_LOG_RB_BASE_HI0,
>> + .reg_size = ISP_LOG_RB_SIZE0,
>> +};
>> +
>> +static struct isp4if_gpu_mem_info *isp4if_gpu_mem_alloc(struct isp4_interface *ispif, u32 mem_size)
>> +{
>> + struct isp4if_gpu_mem_info *mem_info;
>> + struct device *dev = ispif->dev;
>> + int ret;
>> +
>> + if (!mem_size)
>> + return NULL;
>
> mem_size is never zero, remove this check.
>
Yes, in our implementation, it will never be zero, will remove it.
>> +
>> + mem_info = kzalloc(sizeof(*mem_info), GFP_KERNEL);
>
> No need for kzalloc, use kmalloc here instead.
>
Sure, will update in the next version
>> + if (!mem_info)
>> + return NULL;
>> +
>> + mem_info->mem_size = mem_size;
>
> mem_info->mem_size is not used anywhere, remove it.
>
Actually, mem_size will be used in following patches in isp4_subdev.c,
so, i'd like to keep it.
>> + ret = isp_kernel_buffer_alloc(dev, mem_info->mem_size, &mem_info->mem_handle,
>> + &mem_info->gpu_mc_addr, &mem_info->sys_addr);
>> + if (ret) {
>> + kfree(mem_info);
>> + return NULL;
>> + }
>> +
>> + return mem_info;
>> +}
>
> [snip]
>
>> +static int isp4if_gpu_mem_free(struct isp4_interface *ispif, struct isp4if_gpu_mem_info *mem_info)
>> +{
>> + struct device *dev = ispif->dev;
>> +
>> + if (!mem_info) {
>> + dev_err(dev, "invalid mem_info\n");
>> + return -EINVAL;
>> + }
>> +
>> + isp_kernel_buffer_free(&mem_info->mem_handle, &mem_info->gpu_mc_addr, &mem_info->sys_addr);
>> +
>> + kfree(mem_info);
>> +
>> + return 0;
>> +}
>> +
>> +static int isp4if_dealloc_fw_gpumem(struct isp4_interface *ispif)
>> +{
>> + int i;
>> +
>> + if (ispif->fw_mem_pool) {
>> + isp4if_gpu_mem_free(ispif, ispif->fw_mem_pool);
>> + ispif->fw_mem_pool = NULL;
>> + }
>> +
>> + if (ispif->fw_cmd_resp_buf) {
>> + isp4if_gpu_mem_free(ispif, ispif->fw_cmd_resp_buf);
>> + ispif->fw_cmd_resp_buf = NULL;
>> + }
>> +
>> + if (ispif->fw_log_buf) {
>> + isp4if_gpu_mem_free(ispif, ispif->fw_log_buf);
>> + ispif->fw_log_buf = NULL;
>> + }
>> +
>> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
>> + if (ispif->metainfo_buf_pool[i]) {
>> + isp4if_gpu_mem_free(ispif, ispif->metainfo_buf_pool[i]);
>> + ispif->metainfo_buf_pool[i] = NULL;
>> + }
>> + }
>> +
>> + return 0;
>> +}
>
> isp4if_gpu_mem_free() and isp4if_dealloc_fw_gpumem() can be simplified and made
> more robust against copy+paste errors, and their return values are not used
> anywhere. Plus, the mem_info argument to isp4if_gpu_mem_free() is never NULL, so
> there is redundant NULL pointer checking.
>
> Change to the following:
>
> static void isp4if_gpu_mem_free(struct isp4if_gpu_mem_info **mem_info_ptr)
> {
> struct isp4if_gpu_mem_info *mem_info = *mem_info_ptr;
>
> if (!mem_info)
> return;
>
> *mem_info_ptr = NULL;
> isp_kernel_buffer_free(&mem_info->mem_handle, &mem_info->gpu_mc_addr,
> &mem_info->sys_addr);
> kfree(mem_info);
> }
>
> static void isp4if_dealloc_fw_gpumem(struct isp4_interface *ispif)
> {
> int i;
>
> isp4if_gpu_mem_free(&ispif->fw_mem_pool);
> isp4if_gpu_mem_free(&ispif->fw_cmd_resp_buf);
> isp4if_gpu_mem_free(&ispif->fw_log_buf);
>
> for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++)
> isp4if_gpu_mem_free(&ispif->metainfo_buf_pool[i]);
> }
>
Really good optimization, will take it in the next version.
>> +static int isp4if_alloc_fw_gpumem(struct isp4_interface *ispif)
>> +{
>> + struct device *dev = ispif->dev;
>> + unsigned int i;
>
> `i` doesn't need to be unsigned. Remove unsigned to make it consistent with
> other ISP4IF_MAX_STREAM_BUF_COUNT loops.
>
Sure, will modify in the next version.
>> +
>> + ispif->fw_mem_pool = isp4if_gpu_mem_alloc(ispif, FW_MEMORY_POOL_SIZE);
>> + if (!ispif->fw_mem_pool)
>> + goto error_no_memory;
>> +
>> + ispif->fw_cmd_resp_buf =
>> + isp4if_gpu_mem_alloc(ispif, ISP4IF_RB_PMBMAP_MEM_SIZE);
>> + if (!ispif->fw_cmd_resp_buf)
>> + goto error_no_memory;
>> +
>> + ispif->fw_log_buf =
>> + isp4if_gpu_mem_alloc(ispif, ISP4IF_FW_LOG_RINGBUF_SIZE);
>> + if (!ispif->fw_log_buf)
>> + goto error_no_memory;
>> +
>> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
>> + ispif->metainfo_buf_pool[i] =
>> + isp4if_gpu_mem_alloc(ispif,
>> + ISP4IF_META_INFO_BUF_SIZE);
>> + if (!ispif->metainfo_buf_pool[i])
>> + goto error_no_memory;
>> + }
>> +
>> + return 0;
>> +
>> +error_no_memory:
>> + dev_err(dev, "failed to allocate gpu memory\n");
>> + return -ENOMEM;
>> +}
>> +
>> +static u32 isp4if_compute_check_sum(u8 *buf, u32 buf_size)
>
> Change `u8 *buf` to `const u8 *buf`.
>
> Change `u32 buf_size` to `size_t buf_size` just to be consistent with buf_size
> coming from sizeof().
>
Sure, will modify in the next version.
>> +{
>> + u32 checksum = 0;
>> + u8 *surplus_ptr;
>> + u32 *buffer;
>> + u32 i;
>
> Change the 3 variables above to:
>
> const u8 *surplus_ptr;
> const u32 *buffer;
> size_t i;
>
Sure, will modify in the next version.
>> +
>> + buffer = (u32 *)buf;
>
> Change cast to `(const u32 *)`
>
Sure, will modify in the next version.
>> + for (i = 0; i < buf_size / sizeof(u32); i++)
>> + checksum += buffer[i];
>> +
>> + surplus_ptr = (u8 *)&buffer[i];
>
> Change cast to `(const u32 *)`
>
Sure, will modify in the next version. I guess you mean `(const u8 *)`
>> + /* add surplus data crc checksum */
>> + for (i = 0; i < buf_size % sizeof(u32); i++)
>> + checksum += surplus_ptr[i];
>> +
>> + return checksum;
>> +}
>> +
>> +void isp4if_clear_cmdq(struct isp4_interface *ispif)
>> +{
>> + struct isp4if_cmd_element *buf_node = NULL;
>> + struct isp4if_cmd_element *tmp_node = NULL;
>
> Remove unnecessary initialization of buf_node and tmp_node.
>
Sure, will remove it in the next version.
>> +
>> + guard(mutex)(&ispif->cmdq_mutex);
>> +
>> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
>> + list_del(&buf_node->list);
>> + kfree(buf_node);
>> + }
>
> Move the whole list to a local LIST_HEAD(free_list) variable and then release
> the lock. Then you can list_for_each_entry_safe() without needing to do a
> list_del() every time, and you won't need to hold the lock the whole time.
>
Thanks for the suggestion, seems that will make code complicated, e.g.
this is the suggested implementation in my mind if i don't get you wrong,
void isp4if_clear_cmdq(struct isp4_interface *ispif)
{
struct isp4if_cmd_element *buf_node;
struct isp4if_cmd_element *tmp_node;
LIST_HEAD(free_list);
{
guard(mutex)(&ispif->cmdq_mutex);
if (list_empty(&ispif->cmdq))
return;
free_list = ispif->cmdq;
INIT_LIST_HEAD(&ispif->cmdq);
}
list_for_each_entry_safe(buf_node, tmp_node, &free_list, list) {
bool quit = false;
if (buf_node->list.next == &ispif->cmdq)
quit = true;
kfree(buf_node);
if (quit)
return;
}
}
So, I'd like to keep previous implementation by removing list_del and
adding INIT_LIST_HEAD, so this will be the code after refine,
void isp4if_clear_cmdq(struct isp4_interface *ispif)
{
struct isp4if_cmd_element *buf_node;
struct isp4if_cmd_element *tmp_node;
guard(mutex)(&ispif->cmdq_mutex);
list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list)
kfree(buf_node);
INIT_LIST_HEAD(&ispif->cmdq);
}
It's much simpler, and based on our test, for command and buffer queue,
the elements in the queue won't exceed 5 when run to here, so the lock
time will be very short. What do you think?
>> +}
>
> [snip]
>
>> +static struct isp4if_cmd_element *isp4if_append_cmd_2_cmdq(struct isp4_interface *ispif,
>> + struct isp4if_cmd_element *cmd_ele)
>> +{
>> + struct isp4if_cmd_element *copy_command = NULL;
>
> Remove unnecessary initialization of copy_command.
>
Sure, will remove it in the next version.
>> +
>> + copy_command = kmemdup(cmd_ele, sizeof(*cmd_ele), GFP_KERNEL);
>> + if (!copy_command)
>> + return NULL;
>> +
>> + guard(mutex)(&ispif->cmdq_mutex);
>> +
>> + list_add_tail(©_command->list, &ispif->cmdq);
>> +
>> + return copy_command;
>> +}
>> +
>> +struct isp4if_cmd_element *isp4if_rm_cmd_from_cmdq(struct isp4_interface *ispif, u32 seq_num,
>> + u32 cmd_id)
>> +{
>> + struct isp4if_cmd_element *buf_node = NULL;
>> + struct isp4if_cmd_element *tmp_node = NULL;
>
> Remove unnecessary initialization of buf_node and tmp_node.
>
Sure, will remove it in the next version.
>> +
>> + guard(mutex)(&ispif->cmdq_mutex);
>> +
>> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
>> + if (buf_node->seq_num == seq_num &&
>> + buf_node->cmd_id == cmd_id) {
>> + list_del(&buf_node->list);
>> + return buf_node;
>> + }
>> + }
>> +
>> + return NULL;
>> +}
>> +
>> +static int isp4if_insert_isp_fw_cmd(struct isp4_interface *ispif, enum isp4if_stream_id stream,
>> + struct isp4fw_cmd *cmd)
>> +{
>> + struct isp4if_rb_config *rb_config;
>> + struct device *dev = ispif->dev;
>> + u8 *mem_sys;
>> + u32 wr_ptr;
>> + u32 rd_ptr;
>> + u32 rreg;
>> + u32 wreg;
>> + u32 len;
>> +
>> + rb_config = &isp4if_cmd_rb_config[stream];
>> + rreg = rb_config->reg_rptr;
>> + wreg = rb_config->reg_wptr;
>> + mem_sys = (u8 *)rb_config->base_sys_addr;
>> + len = rb_config->val_size;
>> +
>> + if (isp4if_is_cmdq_rb_full(ispif, stream)) {
>> + dev_err(dev, "fail no cmdslot (%d)\n", stream);
>> + return -EINVAL;
>> + }
>> +
>> + wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
>> + rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
>> +
>> + if (rd_ptr > len) {
>> + dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
>> + stream, rd_ptr, len, wr_ptr);
>> + return -EINVAL;
>> + }
>> +
>> + if (wr_ptr > len) {
>> + dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
>> + stream, wr_ptr, len, rd_ptr);
>> + return -EINVAL;
>> + }
>> +
>> + if (wr_ptr < rd_ptr) {
>> + memcpy((mem_sys + wr_ptr),
>> + (u8 *)cmd, sizeof(struct isp4fw_cmd));
>> + } else {
>> + if ((len - wr_ptr) >= (sizeof(struct isp4fw_cmd))) {
>> + memcpy((mem_sys + wr_ptr),
>> + (u8 *)cmd, sizeof(struct isp4fw_cmd));
>> + } else {
>> + u32 size;
>> + u8 *src;
>> +
>> + src = (u8 *)cmd;
>> + size = len - wr_ptr;
>> +
>> + memcpy((mem_sys + wr_ptr), src, size);
>> +
>> + src += size;
>> + size = sizeof(struct isp4fw_cmd) - size;
>> + memcpy((mem_sys), src, size);
>> + }
>> + }
>> +
>> + wr_ptr += sizeof(struct isp4fw_cmd);
>> + if (wr_ptr >= len)
>> + wr_ptr -= len;
>> +
>> + isp4hw_wreg(ispif->mmio, wreg, wr_ptr);
>> +
>> + return 0;
>> +}
>> +
>> +static inline enum isp4if_stream_id isp4if_get_fw_stream(u32 cmd_id)
>> +{
>> + return ISP4IF_STREAM_ID_1;
>> +}
>> +
>> +static int isp4if_send_fw_cmd(struct isp4_interface *ispif, u32 cmd_id, void *package,
>> + u32 package_size, wait_queue_head_t *wq, u32 *wq_cond, u32 *seq)
>> +{
>> + enum isp4if_stream_id stream = isp4if_get_fw_stream(cmd_id);
>> + struct isp4if_cmd_element command_element = {};
>
> Remove command_element per comments further down.
>
Sure, will remove it in the next version.
>> + struct isp4if_gpu_mem_info *gpu_mem = NULL;
>
> gpu_mem is never changed from NULL, remove this variable.
>
Sure, will remove it in the next version.
>> + struct isp4if_cmd_element *cmd_ele = NULL;
>> + struct isp4if_rb_config *rb_config;
>> + struct device *dev = ispif->dev;
>> + struct isp4fw_cmd cmd = {};
>
> Use memset() to guarantee padding bits of cmd are zeroed, since this may not
> guarantee it on all compilers.
>
Sure, will do it in the next version. Just a question, padding bits seem
never to be used, will it cause any problem if they are not zeroed?
>> + u64 package_base = 0;
>> + u32 seq_num;
>> + u32 rreg;
>> + u32 wreg;
>> + int ret;
>> +
>> + if (package_size > sizeof(cmd.cmd_param)) {
>> + dev_err(dev, "fail pkgsize(%u)>%zu cmd:0x%x,stream %d\n",
>> + package_size, sizeof(cmd.cmd_param), cmd_id, stream);
>> + return -EINVAL;
>> + }
>> +
>> + rb_config = &isp4if_resp_rb_config[stream];
>> + rreg = rb_config->reg_rptr;
>> + wreg = rb_config->reg_wptr;
>> +
>> + guard(mutex)(&ispif->isp4if_mutex);
>> +
>> + ret = read_poll_timeout(isp4if_is_cmdq_rb_full, ret, !ret, ISP4IF_MAX_SLEEP_TIME * 1000,
>> + ISP4IF_MAX_SLEEP_COUNT * ISP4IF_MAX_SLEEP_TIME * 1000, false,
>> + ispif, stream);
>> +
>> + if (ret) {
>> + u32 rd_ptr = isp4hw_rreg(ispif->mmio, rreg);
>> + u32 wr_ptr = isp4hw_rreg(ispif->mmio, wreg);
>> +
>> + dev_err(dev,
>> + "failed to get free cmdq slot, stream (%d),rd %u, wr %u\n",
>> + stream, rd_ptr, wr_ptr);
>> + return -ETIMEDOUT;
>> + }
>> +
>> + cmd.cmd_id = cmd_id;
>> + switch (stream) {
>> + case ISP4IF_STREAM_ID_GLOBAL:
>> + cmd.cmd_stream_id = STREAM_ID_INVALID;
>> + break;
>> + case ISP4IF_STREAM_ID_1:
>> + cmd.cmd_stream_id = STREAM_ID_1;
>> + break;
>> + default:
>> + dev_err(dev, "fail bad stream id %d\n", stream);
>> + return -EINVAL;
>> + }
>> +
>> + if (package && package_size)
>> + memcpy(cmd.cmd_param, package, package_size);
>> +
>> + seq_num = ispif->host2fw_seq_num++;
>> + cmd.cmd_seq_num = seq_num;
>> + cmd.cmd_check_sum =
>> + isp4if_compute_check_sum((u8 *)&cmd, sizeof(cmd) - 4);
>
> Change `- 4` to `- sizeof(u32)`
>
Sure, will change it in the next verion.
>> +
>> + if (seq)
>> + *seq = seq_num;
>> + command_element.seq_num = seq_num;
>> + command_element.cmd_id = cmd_id;
>> + command_element.mc_addr = package_base;
>> + command_element.wq = wq;
>> + command_element.wq_cond = wq_cond;
>> + command_element.gpu_pkg = gpu_mem;
>> + command_element.stream = stream;
>> + /*
>> + * only append the fw cmd to queue when its response needs to be waited for,
>> + * currently there are only two such commands, disable channel and stop stream
>> + * which are only sent after close camera
>> + */
>> + if (wq && wq_cond) {
>> + cmd_ele = isp4if_append_cmd_2_cmdq(ispif, &command_element);
>> + if (!cmd_ele) {
>> + dev_err(dev, "fail for isp_append_cmd_2_cmdq\n");
>> + return -ENOMEM;
>> + }
>> + }
>
> The kmemdup() is unnecessary. Remove the isp4if_append_cmd_2_cmdq() function and
> change this to:
>
> if (wq && wq_cond) {
> cmd_ele = kmalloc(sizeof(*cmd_ele), GFP_KERNEL);
> if (!cmd_ele) {
> dev_err(dev, "fail for allocating cmd_ele\n");
> return -ENOMEM;
> }
>
> cmd_ele->seq_num = seq_num;
> cmd_ele->cmd_id = cmd_id;
> cmd_ele->wq = wq;
> cmd_ele->wq_cond = wq_cond;
>
> guard(mutex)(&ispif->cmdq_mutex);
> list_add_tail(©_command->list, &ispif->cmdq);
> }
>
Really good optimization, will modify in the next version.
>> +
>> + ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
>> + if (ret) {
>> + dev_err(dev, "fail for insert_isp_fw_cmd camId (0x%08x)\n", cmd_id);
>> + if (cmd_ele) {
>> + isp4if_rm_cmd_from_cmdq(ispif, cmd_ele->seq_num, cmd_ele->cmd_id);
>
> Using isp4if_rm_cmd_from_cmdq() sort of implies that there is a risk that
> cmd_ele may have been removed from the list somehow, even though the fw cmd
> insertion failed? In any case, either make it truly safe by assuming that it's
> unsafe to dereference cmd_ele right now, or just delete cmd_ele directly from
> the list under the lock.
>
Good consideration, so will change it to following in the next version,
ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
if (ret) {
dev_err(dev, "fail for insert_isp_fw_cmd camId %s(0x%08x)\n",
isp4dbg_get_cmd_str(cmd_id), cmd_id);
if (cmd_ele) {
cmd_ele = isp4if_rm_cmd_from_cmdq(ispif, seq_num, cmd_id);
kfree(cmd_ele);
}
}
The final cmd_ele to be freed will come from cmdq which will be
protected by mutex, so it will be safe.
>> + kfree(cmd_ele);
>> + }
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static int isp4if_send_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_info *buf_info)
>> +{
>> + struct isp4fw_cmd_send_buffer cmd = {};
>
> Use memset() to guarantee padding bits are zeroed, since this may not guarantee
> it on all compilers.
>
Sure, will do it in the next version. as mentioned above, padding bits
seem never to be used, will it cause any problem if they are not zeroed?
>> +
>> + cmd.buffer_type = BUFFER_TYPE_PREVIEW;
>> + cmd.buffer.vmid_space.bit.vmid = 0;
>> + cmd.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
>> + isp4if_split_addr64(buf_info->planes[0].mc_addr,
>> + &cmd.buffer.buf_base_a_lo,
>> + &cmd.buffer.buf_base_a_hi);
>> + cmd.buffer.buf_size_a = buf_info->planes[0].len;
>> +
>> + isp4if_split_addr64(buf_info->planes[1].mc_addr,
>> + &cmd.buffer.buf_base_b_lo,
>> + &cmd.buffer.buf_base_b_hi);
>> + cmd.buffer.buf_size_b = buf_info->planes[1].len;
>> +
>> + isp4if_split_addr64(buf_info->planes[2].mc_addr,
>> + &cmd.buffer.buf_base_c_lo,
>> + &cmd.buffer.buf_base_c_hi);
>> + cmd.buffer.buf_size_c = buf_info->planes[2].len;
>> +
>> + return isp4if_send_fw_cmd(ispif, CMD_ID_SEND_BUFFER, &cmd,
>> + sizeof(cmd), NULL, NULL, NULL);
>> +}
>> +
>> +static void isp4if_init_rb_config(struct isp4_interface *ispif, struct isp4if_rb_config *rb_config)
>> +{
>> + u32 lo;
>> + u32 hi;
>> +
>> + isp4if_split_addr64(rb_config->base_mc_addr, &lo, &hi);
>> +
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rb_config->reg_rptr, 0x0);
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rb_config->reg_wptr, 0x0);
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rb_config->reg_base_lo, lo);
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rb_config->reg_base_hi, hi);
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rb_config->reg_size, rb_config->val_size);
>> +}
>> +
>> +static int isp4if_fw_init(struct isp4_interface *ispif)
>> +{
>> + struct isp4if_rb_config *rb_config;
>> + u32 offset;
>> + int i;
>> +
>> + /* initialize CMD_RB streams */
>> + for (i = 0; i < ISP4IF_STREAM_ID_MAX; i++) {
>> + rb_config = (isp4if_cmd_rb_config + i);
>> + offset = ispif->aligned_rb_chunk_size *
>> + (rb_config->index + ispif->cmd_rb_base_index);
>> +
>> + rb_config->val_size = ISP4IF_FW_CMD_BUF_SIZE;
>> + rb_config->base_sys_addr =
>> + (u8 *)ispif->fw_cmd_resp_buf->sys_addr + offset;
>> + rb_config->base_mc_addr =
>> + ispif->fw_cmd_resp_buf->gpu_mc_addr + offset;
>> +
>> + isp4if_init_rb_config(ispif, rb_config);
>> + }
>> +
>> + /* initialize RESP_RB streams */
>> + for (i = 0; i < ISP4IF_STREAM_ID_MAX; i++) {
>> + rb_config = (isp4if_resp_rb_config + i);
>> + offset = ispif->aligned_rb_chunk_size *
>> + (rb_config->index + ispif->resp_rb_base_index);
>> +
>> + rb_config->val_size = ISP4IF_FW_CMD_BUF_SIZE;
>> + rb_config->base_sys_addr =
>> + (u8 *)ispif->fw_cmd_resp_buf->sys_addr + offset;
>> + rb_config->base_mc_addr =
>> + ispif->fw_cmd_resp_buf->gpu_mc_addr + offset;
>> +
>> + isp4if_init_rb_config(ispif, rb_config);
>> + }
>> +
>> + /* initialize LOG_RB stream */
>> + rb_config = &isp4if_log_rb_config;
>> + rb_config->val_size = ISP4IF_FW_LOG_RINGBUF_SIZE;
>> + rb_config->base_mc_addr = ispif->fw_log_buf->gpu_mc_addr;
>> + rb_config->base_sys_addr = ispif->fw_log_buf->sys_addr;
>> +
>> + isp4if_init_rb_config(ispif, rb_config);
>> +
>> + return 0;
>> +}
>> +
>> +static int isp4if_wait_fw_ready(struct isp4_interface *ispif, u32 isp_status_addr)
>> +{
>> + struct device *dev = ispif->dev;
>> + u32 timeout_ms = 100;
>> + u32 interval_ms = 1;
>> + u32 reg_val;
>> +
>> + /* wait for FW initialize done! */
>> + if (!read_poll_timeout(isp4hw_rreg, reg_val, reg_val & ISP_STATUS__CCPU_REPORT_MASK,
>> + interval_ms * 1000, timeout_ms * 1000, false,
>> + GET_ISP4IF_REG_BASE(ispif), isp_status_addr))
>> + return 0;
>> +
>> + dev_err(dev, "ISP CCPU FW boot failed\n");
>> +
>> + return -ETIME;
>> +}
>> +
>> +static void isp4if_enable_ccpu(struct isp4_interface *ispif)
>> +{
>> + u32 reg_val;
>> +
>> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET);
>> + reg_val &= (~ISP_SOFT_RESET__CCPU_SOFT_RESET_MASK);
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET, reg_val);
>> +
>> + usleep_range(100, 150);
>> +
>> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL);
>> + reg_val &= (~ISP_CCPU_CNTL__CCPU_HOST_SOFT_RST_MASK);
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL, reg_val);
>> +}
>> +
>> +static void isp4if_disable_ccpu(struct isp4_interface *ispif)
>> +{
>> + u32 reg_val;
>> +
>> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL);
>> + reg_val |= ISP_CCPU_CNTL__CCPU_HOST_SOFT_RST_MASK;
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_CCPU_CNTL, reg_val);
>> +
>> + usleep_range(100, 150);
>> +
>> + reg_val = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET);
>> + reg_val |= ISP_SOFT_RESET__CCPU_SOFT_RESET_MASK;
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SOFT_RESET, reg_val);
>> +}
>> +
>> +static int isp4if_fw_boot(struct isp4_interface *ispif)
>> +{
>> + struct device *dev = ispif->dev;
>> +
>> + if (ispif->status != ISP4IF_STATUS_PWR_ON) {
>> + dev_err(dev, "invalid isp power status %d\n", ispif->status);
>> + return -EINVAL;
>> + }
>> +
>> + isp4if_disable_ccpu(ispif);
>> +
>> + isp4if_fw_init(ispif);
>> +
>> + /* clear ccpu status */
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_STATUS, 0x0);
>> +
>> + isp4if_enable_ccpu(ispif);
>> +
>> + if (isp4if_wait_fw_ready(ispif, ISP_STATUS)) {
>> + isp4if_disable_ccpu(ispif);
>> + return -EINVAL;
>> + }
>> +
>> + /* enable interrupts */
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif), ISP_SYS_INT0_EN,
>> + ISP4IF_FW_RESP_RB_IRQ_EN_MASK);
>> +
>> + ispif->status = ISP4IF_STATUS_FW_RUNNING;
>> +
>> + dev_dbg(dev, "ISP CCPU FW boot success\n");
>> +
>> + return 0;
>> +}
>> +
>> +int isp4if_f2h_resp(struct isp4_interface *ispif, enum isp4if_stream_id stream, void *resp)
>> +{
>> + struct isp4fw_resp *response = resp;
>> + struct isp4if_rb_config *rb_config;
>> + struct device *dev = ispif->dev;
>> + u32 rd_ptr_dbg;
>> + u32 wr_ptr_dbg;
>> + void *mem_sys;
>> + u64 mem_addr;
>> + u32 checksum;
>> + u32 rd_ptr;
>> + u32 wr_ptr;
>> + u32 rreg;
>> + u32 wreg;
>> + u32 len;
>> +
>> + rb_config = &isp4if_resp_rb_config[stream];
>> + rreg = rb_config->reg_rptr;
>> + wreg = rb_config->reg_wptr;
>> + mem_sys = rb_config->base_sys_addr;
>> + mem_addr = rb_config->base_mc_addr;
>> + len = rb_config->val_size;
>> +
>> + rd_ptr = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), rreg);
>> + wr_ptr = isp4hw_rreg(GET_ISP4IF_REG_BASE(ispif), wreg);
>> + rd_ptr_dbg = rd_ptr;
>> + wr_ptr_dbg = wr_ptr;
>> +
>> + if (rd_ptr > len) {
>> + dev_err(dev, "fail (%u),rd_ptr %u(should<=%u),wr_ptr %u\n",
>> + stream, rd_ptr, len, wr_ptr);
>> + return -EINVAL;
>> + }
>> +
>> + if (wr_ptr > len) {
>> + dev_err(dev, "fail (%u),wr_ptr %u(should<=%u), rd_ptr %u\n",
>> + stream, wr_ptr, len, rd_ptr);
>> + return -EINVAL;
>> + }
>> +
>> + if (rd_ptr < wr_ptr) {
>> + if ((wr_ptr - rd_ptr) >= (sizeof(struct isp4fw_resp))) {
>> + memcpy((u8 *)response, (u8 *)mem_sys + rd_ptr,
>> + sizeof(struct isp4fw_resp));
>> +
>> + rd_ptr += sizeof(struct isp4fw_resp);
>> + if (rd_ptr < len) {
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rreg, rd_ptr);
>> + } else {
>> + dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
>> + stream, rd_ptr, len, wr_ptr);
>> + return -EINVAL;
>> + }
>> +
>> + } else {
>> + dev_err(dev, "sth wrong with wptr and rptr\n");
>> + return -EINVAL;
>> + }
>> + } else if (rd_ptr > wr_ptr) {
>> + u32 size;
>> + u8 *dst;
>> +
>> + dst = (u8 *)response;
>> +
>> + size = len - rd_ptr;
>> + if (size > sizeof(struct isp4fw_resp)) {
>> + mem_addr += rd_ptr;
>> + memcpy((u8 *)response,
>> + (u8 *)(mem_sys) + rd_ptr,
>> + sizeof(struct isp4fw_resp));
>> + rd_ptr += sizeof(struct isp4fw_resp);
>> + if (rd_ptr < len) {
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rreg, rd_ptr);
>> + } else {
>> + dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
>> + stream, rd_ptr, len, wr_ptr);
>> + return -EINVAL;
>> + }
>> +
>> + } else {
>> + if ((size + wr_ptr) < (sizeof(struct isp4fw_resp))) {
>> + dev_err(dev, "sth wrong with wptr and rptr1\n");
>> + return -EINVAL;
>> + }
>> +
>> + memcpy(dst, (u8 *)(mem_sys) + rd_ptr, size);
>> +
>> + dst += size;
>> + size = sizeof(struct isp4fw_resp) - size;
>> + if (size)
>> + memcpy(dst, (u8 *)(mem_sys), size);
>> + rd_ptr = size;
>> + if (rd_ptr < len) {
>> + isp4hw_wreg(GET_ISP4IF_REG_BASE(ispif),
>> + rreg, rd_ptr);
>> + } else {
>> + dev_err(dev, "(%u),rd %u(should<=%u),wr %u\n",
>> + stream, rd_ptr, len, wr_ptr);
>> + return -EINVAL;
>> + }
>> + }
>> + } else {
>> + return -ETIME;
>> + }
>> +
>> + checksum = isp4if_compute_check_sum((u8 *)response, sizeof(struct isp4fw_resp) - 4);
>
> Change `- 4` to `- sizeof(u32)`
>
Sure, will modify in the next version.
>> +
>> + if (checksum != response->resp_check_sum) {
>> + dev_err(dev, "resp checksum 0x%x,should 0x%x,rptr %u,wptr %u\n",
>> + checksum, response->resp_check_sum, rd_ptr_dbg, wr_ptr_dbg);
>> +
>> + dev_err(dev, "(%u), seqNo %u, resp_id (0x%x)\n", stream,
>> + response->resp_seq_num,
>> + response->resp_id);
>> +
>> + return -EINVAL;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +int isp4if_send_command(struct isp4_interface *ispif, u32 cmd_id, void *package, u32 package_size)
>> +{
>> + return isp4if_send_fw_cmd(ispif, cmd_id, package, package_size, NULL, NULL, NULL);
>> +}
>> +
>> +int isp4if_send_command_sync(struct isp4_interface *ispif, u32 cmd_id, void *package,
>> + u32 package_size, u32 timeout)
>> +{
>> + struct device *dev = ispif->dev;
>> + DECLARE_WAIT_QUEUE_HEAD(cmd_wq);
>> + u32 wq_cond = 0;
>> + int ret;
>> + u32 seq;
>> +
>> + ret = isp4if_send_fw_cmd(ispif, cmd_id, package, package_size, &cmd_wq, &wq_cond, &seq);
>> +
>> + if (ret) {
>> + dev_err(dev, "send fw cmd fail %d\n", ret);
>> + return ret;
>> + }
>> +
>> + ret = wait_event_timeout(cmd_wq, wq_cond != 0, msecs_to_jiffies(timeout));
>
> Instead of wq and wq_cond, use a `struct completion`.
>
Sure, will optimize to use completion.
>> + if (ret == 0) {
>> + struct isp4if_cmd_element *ele;
>> +
>> + ele = isp4if_rm_cmd_from_cmdq(ispif, seq, cmd_id);
>> + kfree(ele);
>> + return -ETIMEDOUT;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +void isp4if_clear_bufq(struct isp4_interface *ispif)
>> +{
>> + struct isp4if_img_buf_node *buf_node = NULL;
>> + struct isp4if_img_buf_node *tmp_node = NULL;
>
> Remove unnecessary initialization of buf_node and tmp_node.
>
Sure, will do it in the next version
>> +
>> + guard(mutex)(&ispif->bufq_mutex);
>> +
>> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->bufq, node) {
>> + list_del(&buf_node->node);
>> + kfree(buf_node);
>> + }
>
> Move the whole list to a local LIST_HEAD(free_list) variable and then release
> the lock. Then you can list_for_each_entry_safe() without needing to do a
> list_del() every time, and you won't need to hold the lock the whole time.
>
Same comments as above cmdq
>> +}
>> +
>> +void isp4if_dealloc_buffer_node(struct isp4if_img_buf_node *buf_node)
>> +{
>> + kfree(buf_node);
>> +}
>> +
>> +struct isp4if_img_buf_node *isp4if_alloc_buffer_node(struct isp4if_img_buf_info *buf_info)
>> +{
>> + struct isp4if_img_buf_node *node = NULL;
>> +
>> + node = kmalloc(sizeof(*node), GFP_KERNEL);
>> + if (node)
>> + node->buf_info = *buf_info;
>> +
>> + return node;
>> +};
>
> Remove superfluous ; after the }.
>
Yes, will remove it in the next version.
>> +
>> +struct isp4if_img_buf_node *isp4if_dequeue_buffer(struct isp4_interface *ispif)
>> +{
>> + struct isp4if_img_buf_node *buf_node = NULL;
>
> Remove the unnecessary initialization of buf_node.
>
Sure, will remove it in the next version.
>> +
>> + guard(mutex)(&ispif->bufq_mutex);
>> +
>> + buf_node = list_first_entry_or_null(&ispif->bufq, typeof(*buf_node), node);
>> + if (buf_node)
>> + list_del(&buf_node->node);
>> +
>> + return buf_node;
>> +}
>> +
>> +int isp4if_queue_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_node *buf_node)
>> +{
>> + int ret;
>> +
>> + ret = isp4if_send_buffer(ispif, &buf_node->buf_info);
>> + if (ret)
>> + return ret;
>> +
>> + guard(mutex)(&ispif->bufq_mutex);
>> +
>> + list_add_tail(&buf_node->node, &ispif->bufq);
>> +
>> + return 0;
>> +}
>> +
>> +int isp4if_stop(struct isp4_interface *ispif)
>> +{
>> + isp4if_disable_ccpu(ispif);
>> +
>> + isp4if_dealloc_fw_gpumem(ispif);
>> +
>> + return 0;
>> +}
>> +
>> +int isp4if_start(struct isp4_interface *ispif)
>> +{
>> + int ret;
>> +
>> + ret = isp4if_alloc_fw_gpumem(ispif);
>> + if (ret)
>> + return -ENOMEM;
>
> Return ret instead of -ENOMEM, since isp4if_alloc_fw_gpumem() returns -ENOMEM.
>
Sure, will modify in the next version.
>> +
>> + ret = isp4if_fw_boot(ispif);
>> + if (ret)
>> + goto failed_fw_boot;
>> +
>> + return 0;
>> +
>> +failed_fw_boot:
>> + isp4if_dealloc_fw_gpumem(ispif);
>> + return ret;
>> +}
>> +
>> +int isp4if_deinit(struct isp4_interface *ispif)
>> +{
>> + isp4if_clear_cmdq(ispif);
>> +
>> + isp4if_clear_bufq(ispif);
>> +
>> + mutex_destroy(&ispif->cmdq_mutex);
>> + mutex_destroy(&ispif->bufq_mutex);
>> + mutex_destroy(&ispif->isp4if_mutex);
>> +
>> + return 0;
>> +}
>> +
>> +int isp4if_init(struct isp4_interface *ispif, struct device *dev, void __iomem *isp_mmip)
>> +{
>> + ispif->dev = dev;
>> + ispif->mmio = isp_mmip;
>> +
>> + ispif->cmd_rb_base_index = 0;
>> + ispif->resp_rb_base_index = ISP4IF_RESP_CHAN_TO_RB_OFFSET - 1;
>> + ispif->aligned_rb_chunk_size = ISP4IF_RB_PMBMAP_MEM_CHUNK & 0xffffffc0;
>> +
>> + mutex_init(&ispif->cmdq_mutex); /* used for cmdq access */
>> + mutex_init(&ispif->bufq_mutex); /* used for bufq access */
>> + mutex_init(&ispif->isp4if_mutex); /* used for commands sent to ispfw */
>> +
>> + INIT_LIST_HEAD(&ispif->cmdq);
>> + INIT_LIST_HEAD(&ispif->bufq);
>> +
>> + return 0;
>> +}
>> diff --git a/drivers/media/platform/amd/isp4/isp4_interface.h b/drivers/media/platform/amd/isp4/isp4_interface.h
>> new file mode 100644
>> index 000000000000..5b94985cdc44
>> --- /dev/null
>> +++ b/drivers/media/platform/amd/isp4/isp4_interface.h
>> @@ -0,0 +1,149 @@
>> +/* SPDX-License-Identifier: GPL-2.0+ */
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#ifndef _ISP4_INTERFACE_
>> +#define _ISP4_INTERFACE_
>
> It seems strange that isp4_interface.h has so many include dependencies and yet
> doesn't #include anything on its own. Maybe the includes needed by
> isp4_interface.h should be moved into isp4_interface.h?
>
Yes, will do the include optimization in the next version, e.g. mutex.h
is included by isp4_subdev.c, isp4_subdev.h, isp4_video.h, while they
all include isp4_interface.h and isp4_interface.h also uses mutex, so
will move mutex.h to isp4_interface.h and remove mutex.h from
isp4_subdev.c, isp4_subdev.h and isp4_video.h, same situation happens to
platform_device.h, drm/amd/isp.h
>> +
>> +#define ISP4IF_RB_MAX (25)
>> +#define ISP4IF_RESP_CHAN_TO_RB_OFFSET (9)
>> +#define ISP4IF_RB_PMBMAP_MEM_SIZE (16 * 1024 * 1024 - 1)
>> +#define ISP4IF_RB_PMBMAP_MEM_CHUNK (ISP4IF_RB_PMBMAP_MEM_SIZE \
>> + / (ISP4IF_RB_MAX - 1))
>> +#define ISP4IF_HOST2FW_COMMAND_SIZE (sizeof(struct isp4fw_cmd))
>> +#define ISP4IF_FW_CMD_BUF_COUNT 4
>> +#define ISP4IF_FW_RESP_BUF_COUNT 4
>> +#define ISP4IF_MAX_NUM_HOST2FW_COMMAND (40)
>> +#define ISP4IF_FW_CMD_BUF_SIZE (ISP4IF_MAX_NUM_HOST2FW_COMMAND \
>> + * ISP4IF_HOST2FW_COMMAND_SIZE)
>> +#define ISP4IF_MAX_SLEEP_COUNT (10)
>> +#define ISP4IF_MAX_SLEEP_TIME (33)
>> +
>> +#define ISP4IF_META_INFO_BUF_SIZE ALIGN(sizeof(struct isp4fw_meta_info), 0x8000)
>> +#define ISP4IF_MAX_STREAM_BUF_COUNT 8
>> +
>> +#define ISP4IF_FW_LOG_RINGBUF_SIZE (2 * 1024 * 1024)
>> +
>> +#define ISP4IF_MAX_CMD_RESPONSE_BUF_SIZE (4 * 1024)
>> +
>> +#define GET_ISP4IF_REG_BASE(ispif) (((ispif))->mmio)
>> +
>> +enum isp4if_stream_id {
>> + ISP4IF_STREAM_ID_GLOBAL = 0,
>> + ISP4IF_STREAM_ID_1 = 1,
>> + ISP4IF_STREAM_ID_MAX = 4
>> +};
>> +
>> +enum isp4if_status {
>> + ISP4IF_STATUS_PWR_OFF,
>> + ISP4IF_STATUS_PWR_ON,
>> + ISP4IF_STATUS_FW_RUNNING,
>> + ISP4IF_FSM_STATUS_MAX
>> +};
>> +
>> +struct isp4if_gpu_mem_info {
>> + u32 mem_domain;
>> + u64 mem_size;
>> + u32 mem_align;
>
> mem_domain, mem_size, and mem_align are all unused. Remove them.
Will remove mem_domain and mem_align in the next version. Because
mem_size will be used in following patch, so will keep it.
>
>> + u64 gpu_mc_addr;
>> + void *sys_addr;
>> + void *mem_handle;
>> +};
>> +
>> +struct isp4if_img_buf_info {
>> + struct {
>> + void *sys_addr;
>> + u64 mc_addr;
>> + u32 len;
>> + } planes[3];
>> +};
>> +
>> +struct isp4if_img_buf_node {
>> + struct list_head node;
>> + struct isp4if_img_buf_info buf_info;
>> +};
>> +
>> +struct isp4if_cmd_element {
>> + struct list_head list;
>> + u32 seq_num;
>> + u32 cmd_id;
>> + enum isp4if_stream_id stream;
>> + u64 mc_addr;
>
> stream and mc_addr are not used for anything, remove them.
>
Sure, will remove them in the next version
>> + wait_queue_head_t *wq;
>> + u32 *wq_cond;
>> + struct isp4if_gpu_mem_info *gpu_pkg;
>
> gpu_pkg is not used for anything, remove it.
>
Sure, will remove then in the next version.
>> +};
>> +
>> +struct isp4_interface {
>> + struct device *dev;
>> + void __iomem *mmio;
>> +
>> + struct mutex cmdq_mutex; /* used for cmdq access */
>> + struct mutex bufq_mutex; /* used for bufq access */
>
> It makes more sense for cmdq_mutex and bufq_mutex to be spin locks since they
> are only held briefly for list traversal.
>
I'd like to keep them as mutex, because 1.no sync with hard/soft
interrupt is needed, 2.Not very time critical 3.Make guard mutex
optimization possible. What do you think?
>> + struct mutex isp4if_mutex; /* used to send fw cmd and read fw log */
>> +
>> + struct list_head cmdq; /* commands sent to fw */
>> + struct list_head bufq; /* buffers sent to fw */
>> +
>> + enum isp4if_status status;
>> + u32 host2fw_seq_num;
>> +
>> + /* FW ring buffer configs */
>> + u32 cmd_rb_base_index;
>> + u32 resp_rb_base_index;
>> + u32 aligned_rb_chunk_size;
>> +
>> + /* ISP fw buffers */
>> + struct isp4if_gpu_mem_info *fw_log_buf;
>> + struct isp4if_gpu_mem_info *fw_cmd_resp_buf;
>> + struct isp4if_gpu_mem_info *fw_mem_pool;
>> + struct isp4if_gpu_mem_info *
>> + metainfo_buf_pool[ISP4IF_MAX_STREAM_BUF_COUNT];
>> +};
>
> [snip]
>
>> +
>> +#endif
>
> Add /* _ISP4_INTERFACE_ */
>
Sure, will add it in the next version and check all header files. BTW,
will change the macro name to _ISP4_INTERFACE_H_ to align with others
>> --
>> 2.34.1
>>
>
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface
2025-09-23 9:24 ` Du, Bin
@ 2025-09-24 7:09 ` Sultan Alsawaf
2025-09-25 3:56 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-09-24 7:09 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On Tue, Sep 23, 2025 at 05:24:11PM +0800, Du, Bin wrote:
> On 9/22/2025 5:55 AM, Sultan Alsawaf wrote:
> > On Thu, Sep 11, 2025 at 06:08:43PM +0800, Bin Du wrote:
> > > + if (!mem_info)
> > > + return NULL;
> > > +
> > > + mem_info->mem_size = mem_size;
> >
> > mem_info->mem_size is not used anywhere, remove it.
> >
>
> Actually, mem_size will be used in following patches in isp4_subdev.c, so,
> i'd like to keep it.
Ah, I didn't notice, my apologies.
> > > + for (i = 0; i < buf_size / sizeof(u32); i++)
> > > + checksum += buffer[i];
> > > +
> > > + surplus_ptr = (u8 *)&buffer[i];
> >
> > Change cast to `(const u32 *)`
> >
>
> Sure, will modify in the next version. I guess you mean `(const u8 *)`
Yes, it should be `(const u8 *)`, apologies for the typo.
> > > +
> > > + guard(mutex)(&ispif->cmdq_mutex);
> > > +
> > > + list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
> > > + list_del(&buf_node->list);
> > > + kfree(buf_node);
> > > + }
> >
> > Move the whole list to a local LIST_HEAD(free_list) variable and then release
> > the lock. Then you can list_for_each_entry_safe() without needing to do a
> > list_del() every time, and you won't need to hold the lock the whole time.
> >
>
> Thanks for the suggestion, seems that will make code complicated, e.g. this
> is the suggested implementation in my mind if i don't get you wrong,
>
> void isp4if_clear_cmdq(struct isp4_interface *ispif)
> {
> struct isp4if_cmd_element *buf_node;
> struct isp4if_cmd_element *tmp_node;
> LIST_HEAD(free_list);
>
> {
> guard(mutex)(&ispif->cmdq_mutex);
> if (list_empty(&ispif->cmdq))
> return;
> free_list = ispif->cmdq;
> INIT_LIST_HEAD(&ispif->cmdq);
> }
>
> list_for_each_entry_safe(buf_node, tmp_node, &free_list, list) {
> bool quit = false;
>
> if (buf_node->list.next == &ispif->cmdq)
> quit = true;
> kfree(buf_node);
> if (quit)
> return;
> }
> }
> So, I'd like to keep previous implementation by removing list_del and adding
> INIT_LIST_HEAD, so this will be the code after refine,
>
> void isp4if_clear_cmdq(struct isp4_interface *ispif)
> {
> struct isp4if_cmd_element *buf_node;
> struct isp4if_cmd_element *tmp_node;
>
> guard(mutex)(&ispif->cmdq_mutex);
>
> list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list)
> kfree(buf_node);
> INIT_LIST_HEAD(&ispif->cmdq);
> }
> It's much simpler, and based on our test, for command and buffer queue, the
> elements in the queue won't exceed 5 when run to here, so the lock time will
> be very short. What do you think?
This is what I am thinking (with cmdq_mutex replaced with a spin lock):
void isp4if_clear_cmdq(struct isp4_interface *ispif)
{
struct isp4if_cmd_element *buf_node, *tmp_node;
LIST_HEAD(free_list);
scoped_guard(spinlock, &ispif->cmdq_lock)
list_splice_init(&ispif->cmdq, &free_list);
list_for_each_entry_safe(buf_node, tmp_node, &free_list, list)
kfree(buf_node);
}
> > > + struct isp4if_cmd_element *cmd_ele = NULL;
> > > + struct isp4if_rb_config *rb_config;
> > > + struct device *dev = ispif->dev;
> > > + struct isp4fw_cmd cmd = {};
> >
> > Use memset() to guarantee padding bits of cmd are zeroed, since this may not
> > guarantee it on all compilers.
> >
>
> Sure, will do it in the next version. Just a question, padding bits seem
> never to be used, will it cause any problem if they are not zeroed?
Padding bits, if there are any, are used by isp4if_compute_check_sum() and are
also sent to the firmware.
> > > +
> > > + ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
> > > + if (ret) {
> > > + dev_err(dev, "fail for insert_isp_fw_cmd camId (0x%08x)\n", cmd_id);
> > > + if (cmd_ele) {
> > > + isp4if_rm_cmd_from_cmdq(ispif, cmd_ele->seq_num, cmd_ele->cmd_id);
> >
> > Using isp4if_rm_cmd_from_cmdq() sort of implies that there is a risk that
> > cmd_ele may have been removed from the list somehow, even though the fw cmd
> > insertion failed? In any case, either make it truly safe by assuming that it's
> > unsafe to dereference cmd_ele right now, or just delete cmd_ele directly from
> > the list under the lock.
> >
>
> Good consideration, so will change it to following in the next version,
> ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
> if (ret) {
> dev_err(dev, "fail for insert_isp_fw_cmd camId %s(0x%08x)\n",
> isp4dbg_get_cmd_str(cmd_id), cmd_id);
> if (cmd_ele) {
> cmd_ele = isp4if_rm_cmd_from_cmdq(ispif, seq_num, cmd_id);
> kfree(cmd_ele);
> }
> }
> The final cmd_ele to be freed will come from cmdq which will be protected by
> mutex, so it will be safe.
Looks good to me!
> > > +static int isp4if_send_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_info *buf_info)
> > > +{
> > > + struct isp4fw_cmd_send_buffer cmd = {};
> >
> > Use memset() to guarantee padding bits are zeroed, since this may not guarantee
> > it on all compilers.
> >
>
> Sure, will do it in the next version. as mentioned above, padding bits seem
> never to be used, will it cause any problem if they are not zeroed?
Padding bits, if there are any, are used by isp4if_compute_check_sum() and are
also sent to the firmware.
> > > +
> > > + guard(mutex)(&ispif->bufq_mutex);
> > > +
> > > + list_for_each_entry_safe(buf_node, tmp_node, &ispif->bufq, node) {
> > > + list_del(&buf_node->node);
> > > + kfree(buf_node);
> > > + }
> >
> > Move the whole list to a local LIST_HEAD(free_list) variable and then release
> > the lock. Then you can list_for_each_entry_safe() without needing to do a
> > list_del() every time, and you won't need to hold the lock the whole time.
> >
>
> Same comments as above cmdq
This is what I am thinking (with bufq_mutex replaced with a spin lock):
void isp4if_clear_bufq(struct isp4_interface *ispif)
{
struct isp4if_img_buf_node *buf_node, *tmp_node;
LIST_HEAD(free_list);
scoped_guard(spinlock, &ispif->bufq_lock)
list_splice_init(&ispif->bufq, &free_list);
list_for_each_entry_safe(buf_node, tmp_node, &free_list, node)
kfree(buf_node);
}
> > > +struct isp4_interface {
> > > + struct device *dev;
> > > + void __iomem *mmio;
> > > +
> > > + struct mutex cmdq_mutex; /* used for cmdq access */
> > > + struct mutex bufq_mutex; /* used for bufq access */
> >
> > It makes more sense for cmdq_mutex and bufq_mutex to be spin locks since they
> > are only held briefly for list traversal.
> >
>
> I'd like to keep them as mutex, because 1.no sync with hard/soft interrupt
> is needed, 2.Not very time critical 3.Make guard mutex optimization
> possible. What do you think?
For very quick/short critical sections that don't sleep, it makes more sense to
use a spin lock. A mutex lock is heavy when it needs to sleep on contention,
which isn't worth it if the critical section has very few instructions.
Also, guard and scoped_guard are available for spin locks too, as shown in my
replies above.
> > > +
> > > +#endif
> >
> > Add /* _ISP4_INTERFACE_ */
> >
>
> Sure, will add it in the next version and check all header files. BTW, will
> change the macro name to _ISP4_INTERFACE_H_ to align with others
Good catch, sounds good.
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface
2025-09-24 7:09 ` Sultan Alsawaf
@ 2025-09-25 3:56 ` Du, Bin
2025-09-25 7:20 ` Sultan Alsawaf
0 siblings, 1 reply; 35+ messages in thread
From: Du, Bin @ 2025-09-25 3:56 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Thanks Sultan.
On 9/24/2025 3:09 PM, Sultan Alsawaf wrote:
> On Tue, Sep 23, 2025 at 05:24:11PM +0800, Du, Bin wrote:
>> On 9/22/2025 5:55 AM, Sultan Alsawaf wrote:
>>> On Thu, Sep 11, 2025 at 06:08:43PM +0800, Bin Du wrote:
>>>> + if (!mem_info)
>>>> + return NULL;
>>>> +
>>>> + mem_info->mem_size = mem_size;
>>>
>>> mem_info->mem_size is not used anywhere, remove it.
>>>
>>
>> Actually, mem_size will be used in following patches in isp4_subdev.c, so,
>> i'd like to keep it.
>
> Ah, I didn't notice, my apologies.
>
It's okay. Actually, it is not used in this patch, so I don't think you
are wrong :)
>>>> + for (i = 0; i < buf_size / sizeof(u32); i++)
>>>> + checksum += buffer[i];
>>>> +
>>>> + surplus_ptr = (u8 *)&buffer[i];
>>>
>>> Change cast to `(const u32 *)`
>>>
>>
>> Sure, will modify in the next version. I guess you mean `(const u8 *)`
>
> Yes, it should be `(const u8 *)`, apologies for the typo.
>
Never mind, just coding details, what's most important is the idea,
thanks for that.
>>>> +
>>>> + guard(mutex)(&ispif->cmdq_mutex);
>>>> +
>>>> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list) {
>>>> + list_del(&buf_node->list);
>>>> + kfree(buf_node);
>>>> + }
>>>
>>> Move the whole list to a local LIST_HEAD(free_list) variable and then release
>>> the lock. Then you can list_for_each_entry_safe() without needing to do a
>>> list_del() every time, and you won't need to hold the lock the whole time.
>>>
>>
>> Thanks for the suggestion, seems that will make code complicated, e.g. this
>> is the suggested implementation in my mind if i don't get you wrong,
>>
>> void isp4if_clear_cmdq(struct isp4_interface *ispif)
>> {
>> struct isp4if_cmd_element *buf_node;
>> struct isp4if_cmd_element *tmp_node;
>> LIST_HEAD(free_list);
>>
>> {
>> guard(mutex)(&ispif->cmdq_mutex);
>> if (list_empty(&ispif->cmdq))
>> return;
>> free_list = ispif->cmdq;
>> INIT_LIST_HEAD(&ispif->cmdq);
>> }
>>
>> list_for_each_entry_safe(buf_node, tmp_node, &free_list, list) {
>> bool quit = false;
>>
>> if (buf_node->list.next == &ispif->cmdq)
>> quit = true;
>> kfree(buf_node);
>> if (quit)
>> return;
>> }
>> }
>> So, I'd like to keep previous implementation by removing list_del and adding
>> INIT_LIST_HEAD, so this will be the code after refine,
>>
>> void isp4if_clear_cmdq(struct isp4_interface *ispif)
>> {
>> struct isp4if_cmd_element *buf_node;
>> struct isp4if_cmd_element *tmp_node;
>>
>> guard(mutex)(&ispif->cmdq_mutex);
>>
>> list_for_each_entry_safe(buf_node, tmp_node, &ispif->cmdq, list)
>> kfree(buf_node);
>> INIT_LIST_HEAD(&ispif->cmdq);
>> }
>> It's much simpler, and based on our test, for command and buffer queue, the
>> elements in the queue won't exceed 5 when run to here, so the lock time will
>> be very short. What do you think?
>
> This is what I am thinking (with cmdq_mutex replaced with a spin lock):
>
> void isp4if_clear_cmdq(struct isp4_interface *ispif)
> {
> struct isp4if_cmd_element *buf_node, *tmp_node;
> LIST_HEAD(free_list);
>
> scoped_guard(spinlock, &ispif->cmdq_lock)
> list_splice_init(&ispif->cmdq, &free_list);
>
> list_for_each_entry_safe(buf_node, tmp_node, &free_list, list)
> kfree(buf_node);
> }
>
Many thanks for the reference code, it's concise and easy to understand,
will incorporate it into the next version.
>>>> + struct isp4if_cmd_element *cmd_ele = NULL;
>>>> + struct isp4if_rb_config *rb_config;
>>>> + struct device *dev = ispif->dev;
>>>> + struct isp4fw_cmd cmd = {};
>>>
>>> Use memset() to guarantee padding bits of cmd are zeroed, since this may not
>>> guarantee it on all compilers.
>>>
>>
>> Sure, will do it in the next version. Just a question, padding bits seem
>> never to be used, will it cause any problem if they are not zeroed?
>
> Padding bits, if there are any, are used by isp4if_compute_check_sum() and are
> also sent to the firmware.
>
Yes, this will impact the checksum value. However, based on my
understanding, it will not affect the error detection outcome, since the
firmware uses the same padding bits during both checksum calculation and
comparison. I apologize for the minor disagreement—I just want to avoid
introducing redundant code, especially given that similar scenarios
appear a lot. Originally, we used memset in the initial version, but
switched to { } initialization in subsequent versions based on review
feedback. Please feel free to share your ideas, if you believe it is
still necessary, we will add them.
>>>> +
>>>> + ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
>>>> + if (ret) {
>>>> + dev_err(dev, "fail for insert_isp_fw_cmd camId (0x%08x)\n", cmd_id);
>>>> + if (cmd_ele) {
>>>> + isp4if_rm_cmd_from_cmdq(ispif, cmd_ele->seq_num, cmd_ele->cmd_id);
>>>
>>> Using isp4if_rm_cmd_from_cmdq() sort of implies that there is a risk that
>>> cmd_ele may have been removed from the list somehow, even though the fw cmd
>>> insertion failed? In any case, either make it truly safe by assuming that it's
>>> unsafe to dereference cmd_ele right now, or just delete cmd_ele directly from
>>> the list under the lock.
>>>
>>
>> Good consideration, so will change it to following in the next version,
>> ret = isp4if_insert_isp_fw_cmd(ispif, stream, &cmd);
>> if (ret) {
>> dev_err(dev, "fail for insert_isp_fw_cmd camId %s(0x%08x)\n",
>> isp4dbg_get_cmd_str(cmd_id), cmd_id);
>> if (cmd_ele) {
>> cmd_ele = isp4if_rm_cmd_from_cmdq(ispif, seq_num, cmd_id);
>> kfree(cmd_ele);
>> }
>> }
>> The final cmd_ele to be freed will come from cmdq which will be protected by
>> mutex, so it will be safe.
>
> Looks good to me!
>
Thanks for the confirmation.
>>>> +static int isp4if_send_buffer(struct isp4_interface *ispif, struct isp4if_img_buf_info *buf_info)
>>>> +{
>>>> + struct isp4fw_cmd_send_buffer cmd = {};
>>>
>>> Use memset() to guarantee padding bits are zeroed, since this may not guarantee
>>> it on all compilers.
>>>
>>
>> Sure, will do it in the next version. as mentioned above, padding bits seem
>> never to be used, will it cause any problem if they are not zeroed?
>
> Padding bits, if there are any, are used by isp4if_compute_check_sum() and are
> also sent to the firmware.
>
Same comments as above
>>>> +
>>>> + guard(mutex)(&ispif->bufq_mutex);
>>>> +
>>>> + list_for_each_entry_safe(buf_node, tmp_node, &ispif->bufq, node) {
>>>> + list_del(&buf_node->node);
>>>> + kfree(buf_node);
>>>> + }
>>>
>>> Move the whole list to a local LIST_HEAD(free_list) variable and then release
>>> the lock. Then you can list_for_each_entry_safe() without needing to do a
>>> list_del() every time, and you won't need to hold the lock the whole time.
>>>
>>
>> Same comments as above cmdq
>
> This is what I am thinking (with bufq_mutex replaced with a spin lock):
>
> void isp4if_clear_bufq(struct isp4_interface *ispif)
> {
> struct isp4if_img_buf_node *buf_node, *tmp_node;
> LIST_HEAD(free_list);
>
> scoped_guard(spinlock, &ispif->bufq_lock)
> list_splice_init(&ispif->bufq, &free_list);
>
> list_for_each_entry_safe(buf_node, tmp_node, &free_list, node)
> kfree(buf_node);
> }
>
Very good reference, will update in the next version.
>>>> +struct isp4_interface {
>>>> + struct device *dev;
>>>> + void __iomem *mmio;
>>>> +
>>>> + struct mutex cmdq_mutex; /* used for cmdq access */
>>>> + struct mutex bufq_mutex; /* used for bufq access */
>>>
>>> It makes more sense for cmdq_mutex and bufq_mutex to be spin locks since they
>>> are only held briefly for list traversal.
>>>
>>
>> I'd like to keep them as mutex, because 1.no sync with hard/soft interrupt
>> is needed, 2.Not very time critical 3.Make guard mutex optimization
>> possible. What do you think?
>
> For very quick/short critical sections that don't sleep, it makes more sense to
> use a spin lock. A mutex lock is heavy when it needs to sleep on contention,
> which isn't worth it if the critical section has very few instructions.
>
> Also, guard and scoped_guard are available for spin locks too, as shown in my
> replies above.
>
Thank you for the detailed explanation. Really appreciate the insights
and will make sure to include these updates in the next version.
>>>> +
>>>> +#endif
>>>
>>> Add /* _ISP4_INTERFACE_ */
>>>
>>
>> Sure, will add it in the next version and check all header files. BTW, will
>> change the macro name to _ISP4_INTERFACE_H_ to align with others
>
> Good catch, sounds good.
>
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface
2025-09-25 3:56 ` Du, Bin
@ 2025-09-25 7:20 ` Sultan Alsawaf
2025-09-25 9:42 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-09-25 7:20 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On Thu, Sep 25, 2025 at 11:56:13AM +0800, Du, Bin wrote:
> On 9/24/2025 3:09 PM, Sultan Alsawaf wrote:
> > On Tue, Sep 23, 2025 at 05:24:11PM +0800, Du, Bin wrote:
> > > On 9/22/2025 5:55 AM, Sultan Alsawaf wrote:
> > > > On Thu, Sep 11, 2025 at 06:08:43PM +0800, Bin Du wrote:
> > > > > + struct isp4if_cmd_element *cmd_ele = NULL;
> > > > > + struct isp4if_rb_config *rb_config;
> > > > > + struct device *dev = ispif->dev;
> > > > > + struct isp4fw_cmd cmd = {};
> > > >
> > > > Use memset() to guarantee padding bits of cmd are zeroed, since this may not
> > > > guarantee it on all compilers.
> > > >
> > >
> > > Sure, will do it in the next version. Just a question, padding bits seem
> > > never to be used, will it cause any problem if they are not zeroed?
> >
> > Padding bits, if there are any, are used by isp4if_compute_check_sum() and are
> > also sent to the firmware.
> >
>
> Yes, this will impact the checksum value. However, based on my
> understanding, it will not affect the error detection outcome, since the
> firmware uses the same padding bits during both checksum calculation and
> comparison. I apologize for the minor disagreement—I just want to avoid
> introducing redundant code, especially given that similar scenarios appear a
> lot. Originally, we used memset in the initial version, but switched to { }
> initialization in subsequent versions based on review feedback. Please feel
> free to share your ideas, if you believe it is still necessary, we will add
> them.
Ah, I see Sakari suggested that during a prior review [1].
Whenever a struct is sent outside of the kernel, padding bits should be zeroed
for a few reasons:
1. Uninitialized padding bits can expose sensitive information from kernel
memory, which can be a security concern.
2. There is no guarantee that the recipient will always behave the same way with
different values for the padding bits. In this case for example, I cannot
look at the ISP source code and say for sure that the padding bits don't
affect its operation. And even if I could, that may always change with a new
firmware version.
3. You can ensure more reliable testing results by guaranteeing that the padding
bits are the same value (zero) for everyone. For example, if the padding bits
accidentally affected the firmware, some users with different padding bits
values could experience bugs that you cannot reproduce in your lab or dev
environment.
The only way to ensure padding bits are zeroed on all compilers is to use
memset; using { } won't do this on every compiler or every compiler version or
even every compiler optimization level [2].
So I still believe it is necessary to use memset for those structs which are
sent outside of the kernel, in this case for the structs sent to firmware. For
structs which are used _only inside_ the kernel, it is preferred to use { }.
[1] https://lore.kernel.org/all/aIclcwRep3F_z7PF@kekkonen.localdomain/
[2] https://interrupt.memfault.com/blog/c-struct-padding-initialization#strategy-4---gcc-extension
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface
2025-09-25 7:20 ` Sultan Alsawaf
@ 2025-09-25 9:42 ` Du, Bin
0 siblings, 0 replies; 35+ messages in thread
From: Du, Bin @ 2025-09-25 9:42 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Thanks Sultan.
On 9/25/2025 3:20 PM, Sultan Alsawaf wrote:
> On Thu, Sep 25, 2025 at 11:56:13AM +0800, Du, Bin wrote:
>> On 9/24/2025 3:09 PM, Sultan Alsawaf wrote:
>>> On Tue, Sep 23, 2025 at 05:24:11PM +0800, Du, Bin wrote:
>>>> On 9/22/2025 5:55 AM, Sultan Alsawaf wrote:
>>>>> On Thu, Sep 11, 2025 at 06:08:43PM +0800, Bin Du wrote:
>>>>>> + struct isp4if_cmd_element *cmd_ele = NULL;
>>>>>> + struct isp4if_rb_config *rb_config;
>>>>>> + struct device *dev = ispif->dev;
>>>>>> + struct isp4fw_cmd cmd = {};
>>>>>
>>>>> Use memset() to guarantee padding bits of cmd are zeroed, since this may not
>>>>> guarantee it on all compilers.
>>>>>
>>>>
>>>> Sure, will do it in the next version. Just a question, padding bits seem
>>>> never to be used, will it cause any problem if they are not zeroed?
>>>
>>> Padding bits, if there are any, are used by isp4if_compute_check_sum() and are
>>> also sent to the firmware.
>>>
>>
>> Yes, this will impact the checksum value. However, based on my
>> understanding, it will not affect the error detection outcome, since the
>> firmware uses the same padding bits during both checksum calculation and
>> comparison. I apologize for the minor disagreement—I just want to avoid
>> introducing redundant code, especially given that similar scenarios appear a
>> lot. Originally, we used memset in the initial version, but switched to { }
>> initialization in subsequent versions based on review feedback. Please feel
>> free to share your ideas, if you believe it is still necessary, we will add
>> them.
>
> Ah, I see Sakari suggested that during a prior review [1].
>
> Whenever a struct is sent outside of the kernel, padding bits should be zeroed
> for a few reasons:
>
> 1. Uninitialized padding bits can expose sensitive information from kernel
> memory, which can be a security concern.
>
> 2. There is no guarantee that the recipient will always behave the same way with
> different values for the padding bits. In this case for example, I cannot
> look at the ISP source code and say for sure that the padding bits don't
> affect its operation. And even if I could, that may always change with a new
> firmware version.
>
> 3. You can ensure more reliable testing results by guaranteeing that the padding
> bits are the same value (zero) for everyone. For example, if the padding bits
> accidentally affected the firmware, some users with different padding bits
> values could experience bugs that you cannot reproduce in your lab or dev
> environment.
>
> The only way to ensure padding bits are zeroed on all compilers is to use
> memset; using { } won't do this on every compiler or every compiler version or
> even every compiler optimization level [2].
>
> So I still believe it is necessary to use memset for those structs which are
> sent outside of the kernel, in this case for the structs sent to firmware. For
> structs which are used _only inside_ the kernel, it is preferred to use { }.
>
> [1] https://lore.kernel.org/all/aIclcwRep3F_z7PF@kekkonen.localdomain/
> [2] https://interrupt.memfault.com/blog/c-struct-padding-initialization#strategy-4---gcc-extension
>
Thank you for the detailed explanation. Your reasoning is both
professional and persuasive. Will switch to using memset instead of { }
initialization for structures that are shared with firmware.
Additionally, will include comments before the memset call to clarify
that it is used to ensure all padding bits are properly zeroed.
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added
2025-09-23 7:23 ` Sultan Alsawaf
@ 2025-09-30 7:30 ` Du, Bin
2025-10-01 7:24 ` Sultan Alsawaf
0 siblings, 1 reply; 35+ messages in thread
From: Du, Bin @ 2025-09-30 7:30 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Hi Sultan,
Thank you very much for your thorough review — We truly appreciate it.
On 9/23/2025 3:23 PM, Sultan Alsawaf wrote:
> Hi Bin,
>
> On Thu, Sep 11, 2025 at 06:08:44PM +0800, Bin Du wrote:
>> Isp4 sub-device is implementing v4l2 sub-device interface. It has one
>> capture video node, and supports only preview stream. It manages firmware
>> states, stream configuration. Add interrupt handling and notification for
>> isp firmware to isp-subdevice.
>>
>> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Bin Du <Bin.Du@amd.com>
>> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/isp4.c
>> @@ -5,13 +5,19 @@
>>
>> #include <linux/pm_runtime.h>
>> #include <linux/vmalloc.h>
>> +
>> +#include <media/v4l2-fwnode.h>
>> #include <media/v4l2-ioctl.h>
>>
>> #include "isp4.h"
>> -
>> -#define VIDEO_BUF_NUM 5
>> +#include "isp4_hw_reg.h"
>>
>> #define ISP4_DRV_NAME "amd_isp_capture"
>> +#define ISP4_FW_RESP_RB_IRQ_STATUS_MASK \
>> + (ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK | \
>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK | \
>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK | \
>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK)
>>
>> /* interrupt num */
>> static const u32 isp4_ringbuf_interrupt_num[] = {
>> @@ -21,19 +27,95 @@ static const u32 isp4_ringbuf_interrupt_num[] = {
>> 4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
>> };
>>
>> -#define to_isp4_device(dev) \
>> - ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
>> +#define to_isp4_device(dev) container_of(dev, struct isp4_device, v4l2_dev)
>> +
>> +static void isp4_wake_up_resp_thread(struct isp4_subdev *isp, u32 index)
>> +{
>> + if (isp && index < ISP4SD_MAX_FW_RESP_STREAM_NUM) {
>> + struct isp4sd_thread_handler *thread_ctx =
>> + &isp->fw_resp_thread[index];
>> +
>> + thread_ctx->wq_cond = 1;
>> + wake_up_interruptible(&thread_ctx->waitq);
>> + }
>> +}
>> +
>> +static void isp4_resp_interrupt_notify(struct isp4_subdev *isp, u32 intr_status)
>> +{
>> + bool wake = (isp->ispif.status == ISP4IF_STATUS_FW_RUNNING);
>> +
>> + u32 intr_ack = 0;
>> +
>> + /* global response */
>> + if (intr_status &
>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK) {
>> + if (wake)
>> + isp4_wake_up_resp_thread(isp, 0);
>> +
>> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT12_ACK_MASK;
>
> The INT_MASKs and ACK_MASKs are the same; perhaps the ACK_MASKs can just be
> removed so you can just write intr_status to ISP_SYS_INT0_ACK instead?
>
These macro definitions are automatically generated from the IP design
by the hardware team. INT_MASK and ACK_MASK represent specific bits in
different registers—the status and acknowledgment registers,
respectively. While their values are currently the same, they could
differ depending on the IP design. I prefer to keep both definitions to
maintain clarity.
>> + }
>> +
>> + /* stream 1 response */
>> + if (intr_status &
>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK) {
>> + if (wake)
>> + isp4_wake_up_resp_thread(isp, 1);
>> +
>> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT9_ACK_MASK;
>> + }
>> +
>> + /* stream 2 response */
>> + if (intr_status &
>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK) {
>> + if (wake)
>> + isp4_wake_up_resp_thread(isp, 2);
>> +
>> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT10_ACK_MASK;
>> + }
>> +
>> + /* stream 3 response */
>> + if (intr_status &
>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK) {
>> + if (wake)
>> + isp4_wake_up_resp_thread(isp, 3);
>> +
>> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT11_ACK_MASK;
>> + }
>
> I think it'd be cleaner to put these masks into an array and loop over them in
> here instead of writing them all out by hand.
>
Yes, that will make the code concise. Will update in the next version
>> +
>> + /* clear ISP_SYS interrupts */
>> + isp4hw_wreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_ACK, intr_ack);
>> +}
>>
>> static irqreturn_t isp4_irq_handler(int irq, void *arg)
>> {
>> + struct isp4_device *isp_dev = dev_get_drvdata(arg);
>
> This is technically a data race because setting drvdata and reading drvdata do
> not use WRITE_ONCE() and READ_ONCE(), respectively. And enabling the IRQ before
> the handler is allowed to do anything is why that `if (!isp_dev)` check exists,
> because that is another race.
>
> Instead, pass the isp_dev pointer through the private pointer of
> devm_request_irq() and add IRQ_NOAUTOEN so the IRQ is enabled by default. Then,
> when it is safe for the IRQ to run, enable it with enable_irq().
>
> That way you can delete the `if (!isp_dev)` check and resolve the two races.
>
Good deep insight, suppose you mean use IRQ_NOAUTOEN to make irq default
disabled. Sure, will add support to dynamically enable/disable IRQ
during camera open/close and remove unnecessary check.
>> + struct isp4_subdev *isp = NULL;
>> + u32 isp_sys_irq_status = 0x0;
>
> Remove unnecessary initialization of `isp` and `isp_sys_irq_status` variables.
>
Sure, will update in the next version.
>> + u32 r1;
>> +
>> + if (!isp_dev)
>> + goto error_drv_data;
>> +
>> + isp = &isp_dev->isp_sdev;
>> + /* check ISP_SYS interrupts status */
>> + r1 = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_STATUS);
>> +
>> + isp_sys_irq_status = r1 & ISP4_FW_RESP_RB_IRQ_STATUS_MASK;
>
> There are four IRQs (one for each stream) but any one of the IRQs can result in
> a notification for _all_ streams. Each IRQ should only do the work of its own
> stream.
>
> You can do this by passing devm_request_irq() a private pointer to indicate the
> mapping between a stream and its IRQ, so that isp4_irq_handler() can know which
> stream it should look at.
>
Will do optimization to remove unused IRQs and keep only necessary ones
(reduce from 4 to 2), actually an IRQ won't result in notification to
all streams, please check the implementation of
isp4_resp_interrupt_notify, it will only wake up IRQ corresponding
stream processing thread.
>> +
>> + isp4_resp_interrupt_notify(isp, isp_sys_irq_status);
>> +
>> +error_drv_data:
>> return IRQ_HANDLED;
>> }
>>
>> static int isp4_capture_probe(struct platform_device *pdev)
>> {
>> struct device *dev = &pdev->dev;
>> + struct isp4_subdev *isp_sdev;
>> struct isp4_device *isp_dev;
>> - int i, irq, ret;
>> + size_t i;
>> + int irq;
>> + int ret;
>>
>> isp_dev = devm_kzalloc(&pdev->dev, sizeof(*isp_dev), GFP_KERNEL);
>> if (!isp_dev)
>> @@ -42,6 +124,12 @@ static int isp4_capture_probe(struct platform_device *pdev)
>> isp_dev->pdev = pdev;
>> dev->init_name = ISP4_DRV_NAME;
>>
>> + isp_sdev = &isp_dev->isp_sdev;
>> + isp_sdev->mmio = devm_platform_ioremap_resource(pdev, 0);
>> + if (IS_ERR(isp_sdev->mmio))
>> + return dev_err_probe(dev, PTR_ERR(isp_sdev->mmio),
>> + "isp ioremap fail\n");
>> +
>> for (i = 0; i < ARRAY_SIZE(isp4_ringbuf_interrupt_num); i++) {
>> irq = platform_get_irq(pdev, isp4_ringbuf_interrupt_num[i]);
>> if (irq < 0)
>> @@ -55,6 +143,8 @@ static int isp4_capture_probe(struct platform_device *pdev)
>> irq);
>> }
>>
>> + isp_dev->pltf_data = pdev->dev.platform_data;
>> +
>> /* Link the media device within the v4l2_device */
>> isp_dev->v4l2_dev.mdev = &isp_dev->mdev;
>>
>> @@ -66,6 +156,8 @@ static int isp4_capture_probe(struct platform_device *pdev)
>> isp_dev->mdev.dev = &pdev->dev;
>> media_device_init(&isp_dev->mdev);
>>
>> + pm_runtime_set_suspended(dev);
>> + pm_runtime_enable(dev);
>> /* register v4l2 device */
>> snprintf(isp_dev->v4l2_dev.name, sizeof(isp_dev->v4l2_dev.name),
>> "AMD-V4L2-ROOT");
>> @@ -74,19 +166,24 @@ static int isp4_capture_probe(struct platform_device *pdev)
>> return dev_err_probe(dev, ret,
>> "fail register v4l2 device\n");
>>
>> + ret = isp4sd_init(&isp_dev->isp_sdev, &isp_dev->v4l2_dev);
>> + if (ret) {
>> + dev_err(dev, "fail init isp4 sub dev %d\n", ret);
>> + goto err_unreg_v4l2;
>> + }
>> +
>> ret = media_device_register(&isp_dev->mdev);
>> if (ret) {
>> dev_err(dev, "fail to register media device %d\n", ret);
>> - goto err_unreg_v4l2;
>> + goto err_isp4_deinit;
>> }
>>
>> platform_set_drvdata(pdev, isp_dev);
>>
>> - pm_runtime_set_suspended(dev);
>> - pm_runtime_enable(dev);
>> -
>> return 0;
>>
>> +err_isp4_deinit:
>> + isp4sd_deinit(&isp_dev->isp_sdev);
>> err_unreg_v4l2:
>> v4l2_device_unregister(&isp_dev->v4l2_dev);
>>
>> @@ -97,8 +194,13 @@ static void isp4_capture_remove(struct platform_device *pdev)
>> {
>> struct isp4_device *isp_dev = platform_get_drvdata(pdev);
>>
>> + v4l2_device_unregister_subdev(&isp_dev->isp_sdev.sdev);
>> +
>> media_device_unregister(&isp_dev->mdev);
>> + media_entity_cleanup(&isp_dev->isp_sdev.sdev.entity);
>
> Why is isp4_capture_remove() handling the cleanup responsibility of
> isp4sd_deinit()? The v4l2_device_unregister_subdev() and media_entity_cleanup()
> on the subdev should only be done by isp4sd_deinit().
>
> Since v4l2_device_unregister_subdev() is not called by isp4sd_deinit(), this
> results in missing cleanup on error from isp4_capture_probe() when
> isp4sd_deinit() is called.
>
Yes, thanks for catching this, it's an issue, will move
v4l2_device_unregister_subdev() and media_entity_cleanup() on the subdev
to isp4sd_deinit()
>> v4l2_device_unregister(&isp_dev->v4l2_dev);
>> +
>> + isp4sd_deinit(&isp_dev->isp_sdev);
>> }
>>
>> static struct platform_driver isp4_capture_drv = {
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
>> @@ -0,0 +1,1095 @@
>> +// SPDX-License-Identifier: GPL-2.0+
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#include <linux/mutex.h>
>> +#include <linux/pm_domain.h>
>> +#include <linux/pm_runtime.h>
>> +
>> +#include "isp4_fw_cmd_resp.h"
>> +#include "isp4_interface.h"
>> +#include "isp4_subdev.h"
>> +#include <linux/units.h>
>> +
>> +#define ISP4SD_MAX_CMD_RESP_BUF_SIZE (4 * 1024)
>> +#define ISP4SD_MIN_BUF_CNT_BEF_START_STREAM 4
>> +
>> +#define ISP4SD_PERFORMANCE_STATE_LOW 0
>> +#define ISP4SD_PERFORMANCE_STATE_HIGH 1
>> +
>> +#define ISP4SD_FW_CMD_TIMEOUT_IN_MS 500
>> +#define ISP4SD_WAIT_RESP_IRQ_TIMEOUT 5 /* ms */
>> +/* align 32KB */
>> +#define ISP4SD_META_BUF_SIZE ALIGN(sizeof(struct isp4fw_meta_info), 0x8000)
>> +
>> +#define to_isp4_subdev(v4l2_sdev) \
>> + container_of(v4l2_sdev, struct isp4_subdev, sdev)
>> +
>> +static const char *isp4sd_entity_name = "amd isp4";
>> +
>> +static void isp4sd_module_enable(struct isp4_subdev *isp_subdev, bool enable)
>> +{
>> + if (isp_subdev->enable_gpio) {
>> + gpiod_set_value(isp_subdev->enable_gpio, enable ? 1 : 0);
>> + dev_dbg(isp_subdev->dev, "%s isp_subdev module\n",
>> + enable ? "enable" : "disable");
>> + }
>> +}
>> +
>> +static int isp4sd_setup_fw_mem_pool(struct isp4_subdev *isp_subdev)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4fw_cmd_send_buffer buf_type = {};
>
> Use memset to guarantee zeroing of padding bits.
>
Sure, will do that and add comments
>> + struct device *dev = isp_subdev->dev;
>> + int ret;
>> +
>> + if (!ispif->fw_mem_pool) {
>> + dev_err(dev, "fail to alloc mem pool\n");
>> + return -ENOMEM;
>> + }
>> +
>> + buf_type.buffer_type = BUFFER_TYPE_MEM_POOL;
>> + buf_type.buffer.buf_tags = 0;
>> + buf_type.buffer.vmid_space.bit.vmid = 0;
>> + buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
>> + isp4if_split_addr64(ispif->fw_mem_pool->gpu_mc_addr,
>> + &buf_type.buffer.buf_base_a_lo,
>> + &buf_type.buffer.buf_base_a_hi);
>> + buf_type.buffer.buf_size_a = (u32)ispif->fw_mem_pool->mem_size;
>> +
>> + ret = isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
>> + &buf_type, sizeof(buf_type));
>> + if (ret) {
>> + dev_err(dev, "send fw mem pool 0x%llx(%u) fail %d\n",
>> + ispif->fw_mem_pool->gpu_mc_addr,
>> + buf_type.buffer.buf_size_a,
>> + ret);
>> + return ret;
>> + }
>> +
>> + dev_dbg(dev, "send fw mem pool 0x%llx(%u) suc\n",
>> + ispif->fw_mem_pool->gpu_mc_addr,
>> + buf_type.buffer.buf_size_a);
>> +
>> + return 0;
>> +};
>> +
>> +static int isp4sd_set_stream_path(struct isp4_subdev *isp_subdev)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4fw_cmd_set_stream_cfg cmd = {};
>
> Use memset to guarantee zeroing of padding bits.
>
Sure, will do that and add comments
>> + struct device *dev = isp_subdev->dev;
>> +
>> + cmd.stream_cfg.mipi_pipe_path_cfg.isp4fw_sensor_id = SENSOR_ID_ON_MIPI0;
>> + cmd.stream_cfg.mipi_pipe_path_cfg.b_enable = true;
>> + cmd.stream_cfg.isp_pipe_path_cfg.isp_pipe_id = MIPI0_ISP_PIPELINE_ID;
>> +
>> + cmd.stream_cfg.b_enable_tnr = true;
>> + dev_dbg(dev, "isp4fw_sensor_id %d, pipeId 0x%x EnableTnr %u\n",
>> + cmd.stream_cfg.mipi_pipe_path_cfg.isp4fw_sensor_id,
>> + cmd.stream_cfg.isp_pipe_path_cfg.isp_pipe_id,
>> + cmd.stream_cfg.b_enable_tnr);
>> +
>> + return isp4if_send_command(ispif, CMD_ID_SET_STREAM_CONFIG,
>> + &cmd, sizeof(cmd));
>> +}
>> +
>> +static int isp4sd_send_meta_buf(struct isp4_subdev *isp_subdev)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4fw_cmd_send_buffer buf_type = {};
>
> Use memset to guarantee zeroing of padding bits.
>
Sure, will do that and add comments
>> + struct isp4sd_sensor_info *sensor_info;
>> + struct device *dev = isp_subdev->dev;
>> + u32 i;
>> +
>> + sensor_info = &isp_subdev->sensor_info;
>> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
>> + int ret;
>> +
>> + if (!sensor_info->meta_info_buf[i]) {
>> + dev_err(dev, "fail for no meta info buf(%u)\n", i);
>> + return -ENOMEM;
>> + }
>> + buf_type.buffer_type = BUFFER_TYPE_META_INFO;
>> + buf_type.buffer.buf_tags = 0;
>> + buf_type.buffer.vmid_space.bit.vmid = 0;
>> + buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
>> + isp4if_split_addr64(sensor_info->meta_info_buf[i]->gpu_mc_addr,
>> + &buf_type.buffer.buf_base_a_lo,
>> + &buf_type.buffer.buf_base_a_hi);
>> + buf_type.buffer.buf_size_a =
>> + (u32)sensor_info->meta_info_buf[i]->mem_size;
>> + ret = isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
>> + &buf_type,
>> + sizeof(buf_type));
>> + if (ret) {
>> + dev_err(dev, "send meta info(%u) fail\n", i);
>> + return ret;
>> + }
>> + }
>> +
>> + dev_dbg(dev, "send meta info suc\n");
>> + return 0;
>> +}
>
> [snip]
>
>> +static int isp4sd_setup_output(struct isp4_subdev *isp_subdev,
>> + struct v4l2_subdev_state *state, u32 pad)
>> +{
>> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
>> + struct isp4sd_output_info *output_info =
>> + &isp_subdev->sensor_info.output_info;
>> + struct isp4fw_cmd_set_out_ch_prop cmd_ch_prop = {};
>
> Use memset to guarantee zeroing of padding bits.
Sure, will do that and add comments
>
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4fw_cmd_enable_out_ch cmd_ch_en = {};
>
> Use memset to guarantee zeroing of padding bits.
>
Sure, will do that and add comments
>> + struct device *dev = isp_subdev->dev;
>> + struct isp4fw_image_prop *out_prop;
>> + int ret;
>> +
>> + if (output_info->start_status == ISP4SD_START_STATUS_STARTED)
>> + return 0;
>> +
>> + if (output_info->start_status == ISP4SD_START_STATUS_START_FAIL) {
>> + dev_err(dev, "fail for previous start fail\n");
>> + return -EINVAL;
>> + }
>> +
>> + out_prop = &cmd_ch_prop.image_prop;
>> + cmd_ch_prop.ch = ISP_PIPE_OUT_CH_PREVIEW;
>> + cmd_ch_en.ch = ISP_PIPE_OUT_CH_PREVIEW;
>> + cmd_ch_en.is_enable = true;
>> +
>> + if (!isp4sd_get_str_out_prop(isp_subdev, out_prop, state, pad)) {
>> + dev_err(dev, "fail to get out prop\n");
>> + return -EINVAL;
>> + }
>> +
>> + dev_dbg(dev, "channel: w:h=%u:%u,lp:%u,cp%u\n",
>> + cmd_ch_prop.image_prop.width, cmd_ch_prop.image_prop.height,
>> + cmd_ch_prop.image_prop.luma_pitch,
>> + cmd_ch_prop.image_prop.chroma_pitch);
>> +
>> + ret = isp4if_send_command(ispif, CMD_ID_SET_OUT_CHAN_PROP,
>> + &cmd_ch_prop,
>> + sizeof(cmd_ch_prop));
>> + if (ret) {
>> + output_info->start_status = ISP4SD_START_STATUS_START_FAIL;
>> + dev_err(dev, "fail to set out prop\n");
>> + return ret;
>> + };
>> +
>> + ret = isp4if_send_command(ispif, CMD_ID_ENABLE_OUT_CHAN,
>> + &cmd_ch_en, sizeof(cmd_ch_en));
>> +
>> + if (ret) {
>> + output_info->start_status = ISP4SD_START_STATUS_START_FAIL;
>> + dev_err(dev, "fail to enable channel\n");
>> + return ret;
>> + }
>> +
>> + if (!sensor_info->start_stream_cmd_sent) {
>> + ret = isp4sd_kickoff_stream(isp_subdev, out_prop->width,
>> + out_prop->height);
>> + if (ret) {
>> + dev_err(dev, "kickoff stream fail %d\n", ret);
>> + return ret;
>> + }
>> + /*
>> + * sensor_info->start_stream_cmd_sent will be set to true
>> + * 1. in isp4sd_kickoff_stream, if app first send buffer then
>> + * start stream
>> + * 2. in isp_set_stream_buf, if app first start stream, then
>> + * send buffer
>> + * because ISP FW has the requirement, host needs to send buffer
>> + * before send start stream cmd
>> + */
>> + if (sensor_info->start_stream_cmd_sent) {
>> + sensor_info->status = ISP4SD_START_STATUS_STARTED;
>> + output_info->start_status = ISP4SD_START_STATUS_STARTED;
>> + dev_dbg(dev, "kickoff stream suc,start cmd sent\n");
>> + }
>> + } else {
>> + dev_dbg(dev, "stream running, no need kickoff\n");
>> + output_info->start_status = ISP4SD_START_STATUS_STARTED;
>> + }
>> +
>> + dev_dbg(dev, "setup output suc\n");
>> + return 0;
>> +}
>> +
>> +static int isp4sd_init_meta_buf(struct isp4_subdev *isp_subdev)
>> +{
>> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct device *dev = isp_subdev->dev;
>> + u32 i;
>> +
>> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
>> + if (!sensor_info->meta_info_buf[i]) {
>> + sensor_info->meta_info_buf[i] = ispif->metainfo_buf_pool[i];
>> + if (!sensor_info->meta_info_buf[i]) {
>> + dev_err(dev, "invalid %u meta_info_buf fail\n", i);
>> + return -ENOMEM;
>> + }
>> + }
>> + }
>> +
>> + return 0;
>> +}
>
> What is the point of metainfo_buf_pool? Especially since metainfo_buf_pool[i] is
> not set to NULL after this "allocation" occurs.
>
> I think isp4sd_init_meta_buf() and metainfo_buf_pool are unnecessary and can be
> factored out.
>
I suppose you mean meta_info_buf, will remove it together with
isp4sd_init_meta_buf() and use metainfo_buf_pool from ispif directly
which is vital for ISP FW to carry response info.
>> +
>> +static int isp4sd_init_stream(struct isp4_subdev *isp_subdev)
>> +{
>> + struct device *dev = isp_subdev->dev;
>> + int ret;
>> +
>> + ret = isp4sd_setup_fw_mem_pool(isp_subdev);
>> + if (ret) {
>> + dev_err(dev, "fail to setup fw mem pool\n");
>> + return ret;
>> + }
>> +
>> + ret = isp4sd_init_meta_buf(isp_subdev);
>> + if (ret) {
>> + dev_err(dev, "fail to alloc fw driver shared buf\n");
>> + return ret;
>> + }
>> +
>> + ret = isp4sd_set_stream_path(isp_subdev);
>> + if (ret) {
>> + dev_err(dev, "fail to setup stream path\n");
>> + return ret;
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static void isp4sd_reset_stream_info(struct isp4_subdev *isp_subdev,
>> + struct v4l2_subdev_state *state, u32 pad)
>> +{
>> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
>> + struct v4l2_mbus_framefmt *format = NULL;
>
> Remove unnecessary initialization of `format`.
>
Sure, will do that
>> + struct isp4sd_output_info *str_info;
>> + int i;
>> +
>> + format = v4l2_subdev_state_get_format(state, pad, 0);
>> +
>> + if (!format) {
>> + dev_err(isp_subdev->dev, "fail to setup stream path\n");
>> + } else {
>> + memset(format, 0, sizeof(*format));
>> + format->code = MEDIA_BUS_FMT_YUYV8_1_5X8;
>> + }
>> +
>> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++)
>> + sensor_info->meta_info_buf[i] = NULL;
>> +
>> + str_info = &sensor_info->output_info;
>> + str_info->start_status = ISP4SD_START_STATUS_NOT_START;
>> +}
>
> [snip]
>
>> +static struct isp4fw_meta_info *
>> +isp4sd_get_meta_by_mc(struct isp4_subdev *isp_subdev,
>> + u64 mc)
>> +{
>> + u32 i;
>
> Change u32 to int for `i` to match other ISP4IF_MAX_STREAM_BUF_COUNT loops.
>
Sure, will do that
>> +
>> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
>> + struct isp4if_gpu_mem_info *meta_info_buf =
>> + isp_subdev->sensor_info.meta_info_buf[i];
>> +
>> + if (meta_info_buf) {
>> + if (mc == meta_info_buf->gpu_mc_addr)
>> + return meta_info_buf->sys_addr;
>> + }
>
> meta_info_buf is never NULL. Also it's easier to read with the constant operand
> on the right side of the comparison. Change to:
>
> if (meta_info_buf->gpu_mc_addr == mc)
> return meta_info_buf->sys_addr;
>
Sure, will remove unnecessary check of meta_info_buf and put mc at the
right of the comparison.
>> + }
>> + return NULL;
>> +};
>
> Remove unnecessary ; after }.
>
Sure, will do that
>> +
>> +static struct isp4if_img_buf_node *
>> +isp4sd_preview_done(struct isp4_subdev *isp_subdev,
>> + struct isp4fw_meta_info *meta)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4if_img_buf_node *prev = NULL;
>> + struct device *dev = isp_subdev->dev;
>> +
>> + if (meta->preview.enabled &&
>> + (meta->preview.status == BUFFER_STATUS_SKIPPED ||
>> + meta->preview.status == BUFFER_STATUS_DONE ||
>> + meta->preview.status == BUFFER_STATUS_DIRTY)) {
>> + prev = isp4if_dequeue_buffer(ispif);
>> + if (!prev)
>> + dev_err(dev, "fail null prev buf\n");
>> +
>> + } else if (meta->preview.enabled) {
>> + dev_err(dev, "fail bad preview status %u\n",
>> + meta->preview.status);
>> + }
>> +
>> + return prev;
>> +}
>> +
>> +static void isp4sd_send_meta_info(struct isp4_subdev *isp_subdev,
>> + u64 meta_info_mc)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4fw_cmd_send_buffer buf_type = {};
>
> Use memset to guarantee zeroing of padding bits.
Sure, will do that and add comment
>
>> + struct device *dev = isp_subdev->dev;
>> +
>> + if (isp_subdev->sensor_info.status != ISP4SD_START_STATUS_STARTED) {
>> + dev_warn(dev, "not working status %i, meta_info 0x%llx\n",
>> + isp_subdev->sensor_info.status, meta_info_mc);
>> + return;
>> + }
>> +
>> + if (meta_info_mc) {
>> + buf_type.buffer_type = BUFFER_TYPE_META_INFO;
>> + buf_type.buffer.buf_tags = 0;
>> + buf_type.buffer.vmid_space.bit.vmid = 0;
>> + buf_type.buffer.vmid_space.bit.space = ADDR_SPACE_TYPE_GPU_VA;
>> + isp4if_split_addr64(meta_info_mc,
>> + &buf_type.buffer.buf_base_a_lo,
>> + &buf_type.buffer.buf_base_a_hi);
>> +
>> + buf_type.buffer.buf_size_a = ISP4SD_META_BUF_SIZE;
>> + if (isp4if_send_command(ispif, CMD_ID_SEND_BUFFER,
>> + &buf_type, sizeof(buf_type))) {
>> + dev_err(dev, "fail send meta_info 0x%llx\n",
>> + meta_info_mc);
>> + } else {
>> + dev_dbg(dev, "resend meta_info 0x%llx\n", meta_info_mc);
>> + }
>> + }
>> +}
>> +
>> +static void isp4sd_fw_resp_frame_done(struct isp4_subdev *isp_subdev,
>> + enum isp4if_stream_id stream_id,
>> + struct isp4fw_resp_param_package *para)
>> +{
>> + struct isp4if_img_buf_node *prev = NULL;
>> + struct device *dev = isp_subdev->dev;
>> + struct isp4fw_meta_info *meta;
>> + u64 mc = 0;
>> +
>> + mc = isp4if_join_addr64(para->package_addr_lo, para->package_addr_hi);
>> + meta = isp4sd_get_meta_by_mc(isp_subdev, mc);
>> + if (mc == 0 || !meta) {
>
> If `mc == 0` is always an error then why pass it to isp4sd_get_meta_by_mc()?
> Change it to skip isp4sd_get_meta_by_mc() when mc is 0, or make
> isp4sd_get_meta_by_mc() return NULL when mc is 0 and then you can remove the
> `mc == 0` check from here.
>
Will remove redundant mc == 0 check, yes, isp4sd_get_meta_by_mc will
return NULL given mc input as 0
>> + dev_err(dev, "fail to get meta from mc %llx\n", mc);
>> + return;
>> + }
>> +
>> + dev_dbg(dev, "ts:%llu,streamId:%d,poc:%u,preview_en:%u,(%i)\n",
>> + ktime_get_ns(), stream_id, meta->poc,
>> + meta->preview.enabled,
>> + meta->preview.status);
>> +
>> + prev = isp4sd_preview_done(isp_subdev, meta);
>> +
>> + isp4if_dealloc_buffer_node(prev);
>> +
>> + if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)
>> + isp4sd_send_meta_info(isp_subdev, mc);
>> +
>> + dev_dbg(dev, "stream_id:%d, status:%d\n", stream_id,
>> + isp_subdev->sensor_info.status);
>> +}
>> +
>> +static void isp4sd_fw_resp_func(struct isp4_subdev *isp_subdev,
>> + enum isp4if_stream_id stream_id)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct device *dev = isp_subdev->dev;
>> + struct isp4fw_resp resp;
>> +
>> + if (ispif->status < ISP4IF_STATUS_FW_RUNNING)
>> + return;
>
> If the lifetime of the kthread is managed correctly, this check shouldn't be
> needed.
>
The original idea is to make sure isp4sd_fw_resp_func only does real
work when ISP FW is running, will refine the start/stop sequecece to
quarentee this is always true so to remove the check.
>> +
>> + while (true) {
>> + s32 ret;
>
> Change from s32 to int to match isp4if_f2h_resp().
>
Sure, will do that.
>> +
>> + ret = isp4if_f2h_resp(ispif, stream_id, &resp);
>> + if (ret)
>> + break;
>
> This loop just parses responses from firmware until the ringbuffer has nothing
> new left, which kind of makes the IRQ useless when this scenario occurs, meaning
> that you can mask the IRQ from the IRQ handler and then unmask it when the
> resp kthread goes back to sleep. So the IRQ doesn't fire needlessly.
>
Very good idea, will add IRQ mask as you suggested.
>> +
>> + switch (resp.resp_id) {
>> + case RESP_ID_CMD_DONE:
>> + isp4sd_fw_resp_cmd_done(isp_subdev, stream_id,
>> + &resp.param.cmd_done);
>> + break;
>> + case RESP_ID_NOTI_FRAME_DONE:
>> + isp4sd_fw_resp_frame_done(isp_subdev, stream_id,
>> + &resp.param.frame_done);
>> + break;
>> + default:
>> + dev_err(dev, "-><- fail respid (0x%x)\n",
>> + resp.resp_id);
>> + break;
>> + }
>> + }
>> +}
>> +
>> +static s32 isp4sd_fw_resp_thread_wrapper(void *context)
>> +{
>> + struct isp4_subdev_thread_param *para = context;
>> + struct isp4sd_thread_handler *thread_ctx;
>> + enum isp4if_stream_id stream_id;
>> +
>> + struct isp4_subdev *isp_subdev;
>> + struct device *dev;
>> + u64 timeout;
>> +
>> + if (!para)
>> + return -EINVAL;
>> +
>> + isp_subdev = para->isp_subdev;
>> + dev = isp_subdev->dev;
>> +
>> + switch (para->idx) {
>> + case 0:
>> + stream_id = ISP4IF_STREAM_ID_GLOBAL;
>> + break;
>> + case 1:
>> + stream_id = ISP4IF_STREAM_ID_1;
>> + break;
>> + default:
>> + dev_err(dev, "fail invalid %d\n", para->idx);
>> + return -EINVAL;
>> + }
>> +
>> + thread_ctx = &isp_subdev->fw_resp_thread[para->idx];
>> +
>> + thread_ctx->wq_cond = 0;
>> + mutex_init(&thread_ctx->mutex);
>
> This mutex doesn't protect anything. Remove it.
>
yes, will remove it.
>> + init_waitqueue_head(&thread_ctx->waitq);
>> + timeout = msecs_to_jiffies(ISP4SD_WAIT_RESP_IRQ_TIMEOUT);
>> +
>> + dev_dbg(dev, "[%u] started\n", para->idx);
>> +
>> + while (true) {
>> + wait_event_interruptible_timeout(thread_ctx->waitq,
>> + thread_ctx->wq_cond != 0,
>> + timeout);
>
> Why is there a timeout? What does the timeout even do since the return value of
> wait_event_interruptible_timeout() is not checked? Doesn't that mean that once
> the timeout is hit, isp4sd_fw_resp_func() will be called for nothing?
>
> I observe that most of the time spent by these kthreads is due to the constant
> wake-ups from the very short 5 ms timeout. This is bad for energy efficiency and
> creates needless overhead.
>
Good catch, previouly before IRQ is really enabled, this is to make sure
ISP can work normally even for 120fps sensor, since now IRQ is enabled,
we can increase the timeout value to like 200ms to avoid the unwanted
timeout caused wake-ups.
>> + thread_ctx->wq_cond = 0;
>> +
>> + if (kthread_should_stop()) {
>> + dev_dbg(dev, "[%u] quit\n", para->idx);
>> + break;
>> + }
>> +
>> + guard(mutex)(&thread_ctx->mutex);
>> + isp4sd_fw_resp_func(isp_subdev, stream_id);
>> + }
>> +
>> + mutex_destroy(&thread_ctx->mutex);
>> +
>> + return 0;
>> +}
>> +
>> +static int isp4sd_start_resp_proc_threads(struct isp4_subdev *isp_subdev)
>> +{
>> + struct device *dev = isp_subdev->dev;
>> + int i;
>> +
>> + for (i = 0; i < ISP4SD_MAX_FW_RESP_STREAM_NUM; i++) {
>> + struct isp4sd_thread_handler *thread_ctx =
>> + &isp_subdev->fw_resp_thread[i];
>> +
>> + isp_subdev->isp_resp_para[i].idx = i;
>> + isp_subdev->isp_resp_para[i].isp_subdev = isp_subdev;
>> +
>> + thread_ctx->thread = kthread_run(isp4sd_fw_resp_thread_wrapper,
>> + &isp_subdev->isp_resp_para[i],
>> + "amd_isp4_thread");
>
> The kthread name and also the IRQ name can be made more informative by appending
> the index number.
>
Yes, will use distinct names to differentiate multiple kthreads and IRQs
>> + if (IS_ERR(thread_ctx->thread)) {
>> + dev_err(dev, "create thread [%d] fail\n", i);
>> + return -EINVAL;
>> + }
>> + }
>> +
>> + return 0;
>> +}
>
> [snip]
>
>> +static u32 isp4sd_get_started_stream_count(struct isp4_subdev *isp_subdev)
>> +{
>> + u32 cnt = 0;
>> +
>> + if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)
>> + cnt++;
>> + return cnt;
>> +}
>
> isp4sd_get_started_stream_count() is unnecessary, remove it and just use
> `if (isp_subdev->sensor_info.status == ISP4SD_START_STATUS_STARTED)` instead.
>
Sure, will do optimization as you suggested
>> +
>> +static int isp4sd_pwroff_and_deinit(struct isp4_subdev *isp_subdev)
>> +{
>> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
>> + unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_LOW;
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> +
>> + struct device *dev = isp_subdev->dev;
>> + u32 cnt;
>> + int ret;
>> +
>> + mutex_lock(&isp_subdev->ops_mutex);
>> +
>> + if (sensor_info->status == ISP4SD_START_STATUS_STARTED) {
>> + dev_err(dev, "fail for stream still running\n");
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> + return -EINVAL;
>> + }
>> +
>> + sensor_info->status = ISP4SD_START_STATUS_NOT_START;
>> + cnt = isp4sd_get_started_stream_count(isp_subdev);
>> + if (cnt > 0) {
>> + dev_dbg(dev, "no need power off isp_subdev\n");
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> + return 0;
>> + }
>> +
>> + isp4if_stop(ispif);
>> +
>> + ret = dev_pm_genpd_set_performance_state(dev, perf_state);
>> + if (ret)
>> + dev_err(dev,
>> + "fail to set isp_subdev performance state %u,ret %d\n",
>> + perf_state, ret);
>> + isp4sd_stop_resp_proc_threads(isp_subdev);
>> + dev_dbg(dev, "isp_subdev stop resp proc streads suc");
>> + /* hold ccpu reset */
>> + isp4hw_wreg(isp_subdev->mmio, ISP_SOFT_RESET, 0x0);
>> + isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0);
>> + ret = pm_runtime_put_sync(dev);
>> + if (ret)
>> + dev_err(dev, "power off isp_subdev fail %d\n", ret);
>> + else
>> + dev_dbg(dev, "power off isp_subdev suc\n");
>> +
>> + ispif->status = ISP4IF_STATUS_PWR_OFF;
>> + isp4if_clear_cmdq(ispif);
>> + isp4sd_module_enable(isp_subdev, false);
>> +
>> + msleep(20);
>
> What is this msleep for?
>
This is the HW requirement, at least 20ms is needed for the possible
quickly open followed.
>> +
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> +
>> + return 0;
>> +}
>> +
>> +static int isp4sd_pwron_and_init(struct isp4_subdev *isp_subdev)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct device *dev = isp_subdev->dev;
>> + int ret;
>> +
>> + if (ispif->status == ISP4IF_STATUS_FW_RUNNING) {
>
> `ispif->status` is checked under ops_mutex elsewhere but not in this function?
>
As you suggested below, will use guard(mutex)(&ispsd->ops_mutex) to
protect isp4sd_set_power, so this won't be a problem any more.
>> + dev_dbg(dev, "camera already opened, do nothing\n");
>> + return 0;
>> + }
>> +
>> + mutex_lock(&isp_subdev->ops_mutex);
>> +
>> + isp4sd_module_enable(isp_subdev, true);
>> +
>> + isp_subdev->sensor_info.start_stream_cmd_sent = false;
>> + isp_subdev->sensor_info.buf_sent_cnt = 0;
>> +
>> + if (ispif->status < ISP4IF_STATUS_PWR_ON) {
>> + unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_HIGH;
>> +
>> + ret = pm_runtime_resume_and_get(dev);
>> + if (ret) {
>> + dev_err(dev, "fail to power on isp_subdev ret %d\n",
>> + ret);
>> + goto err_unlock_and_close;
>> + }
>> +
>> + /* ISPPG ISP Power Status */
>> + isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0x7FF);
>> + ret = dev_pm_genpd_set_performance_state(dev, perf_state);
>> + if (ret) {
>> + dev_err(dev,
>> + "fail to set performance state %u, ret %d\n",
>> + perf_state, ret);
>> + goto err_unlock_and_close;
>> + }
>> +
>> + ispif->status = ISP4IF_STATUS_PWR_ON;
>> +
>> + if (isp4sd_start_resp_proc_threads(isp_subdev)) {
>> + dev_err(dev, "isp_start_resp_proc_threads fail");
>> + goto err_unlock_and_close;
>> + } else {
>> + dev_dbg(dev, "create resp threads ok");
>> + }
>> + }
>> +
>> + isp_subdev->sensor_info.start_stream_cmd_sent = false;
>> + isp_subdev->sensor_info.buf_sent_cnt = 0;
>> +
>> + ret = isp4if_start(ispif);
>> + if (ret) {
>> + dev_err(dev, "fail to start isp_subdev interface\n");
>> + goto err_unlock_and_close;
>> + }
>> +
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> + return 0;
>> +err_unlock_and_close:
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> + isp4sd_pwroff_and_deinit(isp_subdev);
>> + return -EINVAL;
>> +}
>
> [snip]
>
>> +static int isp4sd_set_power(struct v4l2_subdev *sd, int on)
>> +{
>> + struct isp4_subdev *ispsd = to_isp4_subdev(sd);
>
> `ispsd` is the variable name used here and in a couple other functions but
> `isp_subdev` is the name used everywhere else. Make the variable name consistent
> for `struct isp4_subdev *`.
>
Yes, will unify the naming
>> +
>
> Add `guard(mutex)(&ispsd->ops_mutex);` here and remove all uses of ops_mutex
> from isp4sd_pwron_and_init() and isp4sd_pwroff_and_deinit(). This simplifies the
> locking and also ensures that the lock is not released and reacquired when
> isp4sd_pwron_and_init() needs to call isp4sd_pwroff_and_deinit() on error.
>
Very good suggestion, it will make code simple, will do that.
>> + if (on)
>> + return isp4sd_pwron_and_init(ispsd);
>> + else
>> + return isp4sd_pwroff_and_deinit(ispsd);
>> +};
>
> Remove unnecessary ; after }.
>
Sure.
>> +
>> +static const struct v4l2_subdev_core_ops isp4sd_core_ops = {
>> + .s_power = isp4sd_set_power,
>> +};
>
> [snip]
>
>> +int isp4sd_init(struct isp4_subdev *isp_subdev,
>> + struct v4l2_device *v4l2_dev)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4sd_sensor_info *sensor_info;
>> + struct device *dev = v4l2_dev->dev;
>> + int ret;
>> +
>> + isp_subdev->dev = dev;
>> + v4l2_subdev_init(&isp_subdev->sdev, &isp4sd_subdev_ops);
>> + isp_subdev->sdev.owner = THIS_MODULE;
>> + isp_subdev->sdev.dev = dev;
>> + snprintf(isp_subdev->sdev.name, sizeof(isp_subdev->sdev.name), "%s",
>> + dev_name(dev));
>> +
>> + isp_subdev->sdev.entity.name = isp4sd_entity_name;
>> + isp_subdev->sdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
>> + isp_subdev->sdev.entity.ops = &isp4sd_sdev_ent_ops;
>> + isp_subdev->sdev_pad.flags = MEDIA_PAD_FL_SOURCE;
>> + ret = media_entity_pads_init(&isp_subdev->sdev.entity, 1,
>> + &isp_subdev->sdev_pad);
>> + if (ret) {
>> + dev_err(dev, "fail to init isp4 subdev entity pad %d\n", ret);
>> + return ret;
>> + }
>> + ret = v4l2_subdev_init_finalize(&isp_subdev->sdev);
>> + if (ret < 0) {
>> + dev_err(dev, "fail to init finalize isp4 subdev %d\n",
>> + ret);
>> + return ret;
>> + }
>> + ret = v4l2_device_register_subdev(v4l2_dev, &isp_subdev->sdev);
>> + if (ret) {
>> + dev_err(dev, "fail to register isp4 subdev to V4L2 device %d\n",
>> + ret);
>> + goto err_media_clean_up;
>
> Missing error handling: v4l2_subdev_cleanup() is not called.
>
Good catching, will add it.
>> + }
>> +
>> + sensor_info = &isp_subdev->sensor_info;
>> +
>> + isp4if_init(ispif, dev, isp_subdev->mmio);
>> +
>> + mutex_init(&isp_subdev->ops_mutex);
>> + sensor_info->start_stream_cmd_sent = false;
>> + sensor_info->status = ISP4SD_START_STATUS_NOT_START;
>> +
>> + /* create ISP enable gpio control */
>> + isp_subdev->enable_gpio = devm_gpiod_get(isp_subdev->dev,
>> + "enable_isp",
>> + GPIOD_OUT_LOW);
>> + if (IS_ERR(isp_subdev->enable_gpio)) {
>> + dev_err(dev, "fail to get gpiod %d\n", ret);
>
> This prints ret instead of the actual error, PTR_ERR(isp_subdev->enable_gpio).
>
> Instead, add `ret = PTR_ERR(isp_subdev->enable_gpio);` before the dev_err().
>
Yes, will modify as you suggested.
>> + media_entity_cleanup(&isp_subdev->sdev.entity);
>> + return PTR_ERR(isp_subdev->enable_gpio);
>
> Missing error handling: v4l2_device_unregister_subdev() is not called.
>
> Add another goto label and use that instead of returning here.
>
Sure, will modify as you suggested.
>> + }
>> +
>> + isp_subdev->host2fw_seq_num = 1;
>> + ispif->status = ISP4IF_STATUS_PWR_OFF;
>> +
>> + if (ret)
>> + goto err_media_clean_up;
>> + return ret;
>
> ret is always zero at this point. Remove this `if (ret) ...` and change the
> return to `return 0`.
>
Yes, that's ture, will modify it.
>> +
>> +err_media_clean_up:
>> + media_entity_cleanup(&isp_subdev->sdev.entity);
>> + return ret;
>> +}
>> +
>> +void isp4sd_deinit(struct isp4_subdev *isp_subdev)
>> +{
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> +
>> + media_entity_cleanup(&isp_subdev->sdev.entity);
>> + isp4if_deinit(ispif);
>> + isp4sd_module_enable(isp_subdev, false);
>> +
>> + ispif->status = ISP4IF_STATUS_PWR_OFF;
>> +}
>> diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.h b/drivers/media/platform/amd/isp4/isp4_subdev.h
>> new file mode 100644
>> index 000000000000..524a8de5e18d
>> --- /dev/null
>> +++ b/drivers/media/platform/amd/isp4/isp4_subdev.h
>> @@ -0,0 +1,131 @@
>> +/* SPDX-License-Identifier: GPL-2.0+ */
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#ifndef _ISP4_CONTEXT_H_
>> +#define _ISP4_CONTEXT_H_
>> +
>> +#include <linux/delay.h>
>> +#include <linux/firmware.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/uaccess.h>
>> +#include <linux/types.h>
>> +#include <linux/debugfs.h>
>> +#include <media/v4l2-device.h>
>> +
>> +#include "isp4_fw_cmd_resp.h"
>> +#include "isp4_hw_reg.h"
>> +#include "isp4_interface.h"
>> +
>> +/*
>> + * one is for none sesnor specefic response which is not used now
>> + * another is for sensor specific response
>> + */
>> +#define ISP4SD_MAX_FW_RESP_STREAM_NUM 2
>
> Only two out of four possible streams are used, yet IRQs are requested for all
> four streams in isp4_capture_probe(). ISP4SD_MAX_FW_RESP_STREAM_NUM should be
> checked and used to limit the number of IRQs requested in isp4_capture_probe().
>
Yes, and only two IRQs are used, we reduce IRQs from 4 to 2 and request
the used ones in the isp4_capture_probe
>> +
>> +/*
>> + * cmd used to register frame done callback, parameter is
>> + * struct isp4sd_register_framedone_cb_param *
>> + * when a image buffer is filled by ISP, ISP will call the registered callback.
>> + * callback func prototype is isp4sd_framedone_cb, cb_ctx can be anything
>> + * provided by caller which will be provided back as the first parameter of the
>> + * callback function.
>> + * both cb_func and cb_ctx are provide by caller, set cb_func to NULL to
>> + * unregister the callback
>> + */
>
> [snip]
>
>> +void isp4sd_deinit(struct isp4_subdev *isp_subdev);
>> +
>> +#endif
>
> Add /* _ISP4_CONTEXT_H_ */
>
Yes, will update and check all other header files, sorry for the bad
macro name, will change to _ISP4_SUBDEV_H_
>> --
>> 2.34.1
>>
>
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-09-11 10:08 ` [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers " Bin Du
@ 2025-10-01 6:53 ` Sultan Alsawaf
2025-10-11 9:30 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-10-01 6:53 UTC (permalink / raw)
To: Bin Du
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Svetoslav Stoilov, Alexey Zagorodnikov
Hi Bin,
On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
> Isp video implements v4l2 video interface and supports NV12 and YUYV. It
> manages buffers, pipeline power and state. Cherry-picked Sultan's DMA
> buffer related fix from branch v6.16-drm-tip-isp4-for-amd on
> https://github.com/kerneltoast/kernel_x86_laptop.git
>
> Co-developed-by: Sultan Alsawaf <sultan@kerneltoast.com>
> Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> Signed-off-by: Bin Du <Bin.Du@amd.com>
> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
[snip]
> +++ b/drivers/media/platform/amd/isp4/isp4.c
> @@ -178,6 +178,16 @@ static int isp4_capture_probe(struct platform_device *pdev)
> goto err_isp4_deinit;
> }
>
> + ret = media_create_pad_link(&isp_dev->isp_sdev.sdev.entity,
> + 0, &isp_dev->isp_sdev.isp_vdev.vdev.entity,
> + 0,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> + if (ret) {
> + dev_err(dev, "fail to create pad link %d\n", ret);
> + goto err_isp4_deinit;
> + }
> +
Two problems with this hunk:
1. According to the comment in include/media/media-device.h [1],
media_create_pad_link() should be called before media_device_register():
* So drivers need to first initialize the media device, register any entity
* within the media device, create pad to pad links and then finally register
* the media device by calling media_device_register() as a final step.
2. Missing call to media_device_unregister() on error when
media_create_pad_link() fails.
Since the media_create_pad_link() will be moved before media_device_register(),
we will need to clean up media_create_pad_link() when media_device_register()
fails.
The clean-up function for media_create_pad_link() is media_device_unregister().
According to the comment for media_device_unregister() [2], it is safe to call
media_device_unregister() on an unregistered media device that is initialized
(through media_device_init()).
In addition, this made me realize that there's no call to media_device_cleanup()
in the entire driver too. This is the cleanup function for media_device_init(),
so it should be called on error and on module unload.
To summarize, make the following changes:
1. Move the media_create_pad_link() up, right before media_device_register().
2. When media_device_register() fails, clean up media_create_pad_link() by
calling media_device_unregister().
3. Add a missing call to media_device_cleanup() on error and module unload to
clean up media_device_init().
> platform_set_drvdata(pdev, isp_dev);
>
> return 0;
> diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.c b/drivers/media/platform/amd/isp4/isp4_subdev.c
> index a9cb14de04ca..7d3339c915eb 100644
> --- a/drivers/media/platform/amd/isp4/isp4_subdev.c
> +++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
[snip]
> +static int isp4sd_ioc_send_img_buf(struct v4l2_subdev *sd,
> + struct isp4if_img_buf_info *buf_info)
> +{
> + struct isp4_subdev *isp_subdev = to_isp4_subdev(sd);
> + struct isp4_interface *ispif = &isp_subdev->ispif;
> + struct isp4if_img_buf_node *buf_node = NULL;
Remove unnecessary initialization of `buf_node`.
> + struct device *dev = isp_subdev->dev;
> + int ret = -EINVAL;
Remove unnecessary initialization of `ret`.
> +
> + mutex_lock(&isp_subdev->ops_mutex);
Use guard() for this mutex and remove the unlock_and_return label.
> + /* TODO: remove isp_status */
> + if (ispif->status != ISP4IF_STATUS_FW_RUNNING) {
> + dev_err(dev, "fail send img buf for bad fsm %d\n",
> + ispif->status);
> + mutex_unlock(&isp_subdev->ops_mutex);
> + return -EINVAL;
> + }
> +
> + buf_node = isp4if_alloc_buffer_node(buf_info);
> + if (!buf_node) {
> + dev_err(dev, "fail alloc sys img buf info node\n");
> + ret = -ENOMEM;
> + goto unlock_and_return;
> + }
> +
> + ret = isp4if_queue_buffer(ispif, buf_node);
> + if (ret) {
> + dev_err(dev, "fail to queue image buf, %d\n", ret);
> + goto error_release_buf_node;
> + }
> +
> + if (!isp_subdev->sensor_info.start_stream_cmd_sent) {
> + isp_subdev->sensor_info.buf_sent_cnt++;
> +
> + if (isp_subdev->sensor_info.buf_sent_cnt >=
> + ISP4SD_MIN_BUF_CNT_BEF_START_STREAM) {
> + ret = isp4if_send_command(ispif, CMD_ID_START_STREAM,
> + NULL, 0);
> +
> + if (ret) {
> + dev_err(dev, "fail to START_STREAM");
> + goto error_release_buf_node;
> + }
> + isp_subdev->sensor_info.start_stream_cmd_sent = true;
> + isp_subdev->sensor_info.output_info.start_status =
> + ISP4SD_START_STATUS_STARTED;
> + isp_subdev->sensor_info.status =
> + ISP4SD_START_STATUS_STARTED;
> + } else {
> + dev_dbg(dev,
> + "no send start,required %u,buf sent %u\n",
Add a space after each comma in this string.
> + ISP4SD_MIN_BUF_CNT_BEF_START_STREAM,
> + isp_subdev->sensor_info.buf_sent_cnt);
> + }
> + }
> +
> + mutex_unlock(&isp_subdev->ops_mutex);
> +
> + return 0;
> +
> +error_release_buf_node:
> + isp4if_dealloc_buffer_node(buf_node);
> +
> +unlock_and_return:
> + mutex_unlock(&isp_subdev->ops_mutex);
> +
> + return ret;
> +}
[snip]
> +++ b/drivers/media/platform/amd/isp4/isp4_video.c
> @@ -0,0 +1,1207 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
> + */
> +
> +#include <drm/amd/isp.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/vmalloc.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +
> +#include "isp4_interface.h"
> +#include "isp4_subdev.h"
> +#include "isp4_video.h"
> +
> +#define ISP4VID_ISP_DRV_NAME "amd_isp_capture"
> +#define ISP4VID_MAX_PREVIEW_FPS 30
> +#define ISP4VID_DEFAULT_FMT isp4vid_formats[0]
> +
> +#define ISP4VID_PAD_VIDEO_OUTPUT 0
> +
> +/* timeperframe default */
> +#define ISP4VID_ISP_TPF_DEFAULT isp4vid_tpfs[0]
> +
> +/* amdisp buffer for vb2 operations */
> +struct isp4vid_vb2_buf {
> + struct device *dev;
> + void *vaddr;
> + unsigned long size;
> + refcount_t refcount;
> + struct dma_buf *dbuf;
> + void *bo;
> + u64 gpu_addr;
> + struct vb2_vmarea_handler handler;
> +};
> +
> +static int isp4vid_vb2_mmap(void *buf_priv, struct vm_area_struct *vma);
Don't need the isp4vid_vb2_mmap() prototype here anymore, remove it.
> +
> +static void isp4vid_vb2_put(void *buf_priv);
> +
> +static const char *isp4vid_video_dev_name = "Preview";
Turn this into `static const char *const isp4vid_video_dev_name = "Preview";`
which makes the `isp4vid_video_dev_name` variable itself const, so that you
cannot change `isp4vid_video_dev_name` to something else.
> +
> +/* Sizes must be in increasing order */
> +static const struct v4l2_frmsize_discrete isp4vid_frmsize[] = {
> + {640, 360},
> + {640, 480},
> + {1280, 720},
> + {1280, 960},
> + {1920, 1080},
> + {1920, 1440},
> + {2560, 1440},
> + {2880, 1620},
> + {2880, 1624},
> + {2888, 1808},
> +};
> +
> +static const u32 isp4vid_formats[] = {
> + V4L2_PIX_FMT_NV12,
> + V4L2_PIX_FMT_YUYV
> +};
> +
> +/* timeperframe list */
> +static const struct v4l2_fract isp4vid_tpfs[] = {
> + {.numerator = 1, .denominator = ISP4VID_MAX_PREVIEW_FPS}
Add a space after { and a space before }.
Also, it is more common to see v4l2_fract initialized without specifying the
struct member names.
To summarize, change to `{ 1, ISP4VID_MAX_PREVIEW_FPS }`
> +};
> +
> +static void isp4vid_handle_frame_done(struct isp4vid_dev *isp_vdev,
> + const struct isp4if_img_buf_info *img_buf,
> + bool done_suc)
> +{
> + struct isp4vid_capture_buffer *isp4vid_buf;
Rename isp4vid_buf to isp_buf like in isp4vid_qops_start_streaming().
> + enum vb2_buffer_state state;
> + void *vbuf;
> +
> + mutex_lock(&isp_vdev->buf_list_lock);
> +
> + /* Get the first entry of the list */
> + isp4vid_buf = list_first_entry_or_null(&isp_vdev->buf_list, typeof(*isp4vid_buf), list);
> + if (!isp4vid_buf) {
> + mutex_unlock(&isp_vdev->buf_list_lock);
> + return;
> + }
> +
> + vbuf = vb2_plane_vaddr(&isp4vid_buf->vb2.vb2_buf, 0);
> +
> + if (vbuf != img_buf->planes[0].sys_addr) {
> + dev_err(isp_vdev->dev, "Invalid vbuf");
> + mutex_unlock(&isp_vdev->buf_list_lock);
> + return;
> + }
> +
> + /* Remove this entry from the list */
> + list_del(&isp4vid_buf->list);
> +
> + mutex_unlock(&isp_vdev->buf_list_lock);
Change to this starting from the mutex_lock():
scoped_guard(mutex, &isp_vdev->buf_list_lock) {
/* Get the first entry of the list */
isp_buf = list_first_entry_or_null(&isp_vdev->buf_list,
typeof(*isp_buf), list);
if (!isp_buf)
return;
vbuf = vb2_plane_vaddr(&isp_buf->vb2.vb2_buf, 0);
if (vbuf != img_buf->planes[0].sys_addr) {
dev_err(isp_vdev->dev, "Invalid vbuf");
return;
}
/* Remove this entry from the list */
list_del(&isp_buf->list);
}
> +
> + /* Fill the buffer */
> + isp4vid_buf->vb2.vb2_buf.timestamp = ktime_get_ns();
> + isp4vid_buf->vb2.sequence = isp_vdev->sequence++;
> + isp4vid_buf->vb2.field = V4L2_FIELD_ANY;
> +
> + /* at most 2 planes */
> + vb2_set_plane_payload(&isp4vid_buf->vb2.vb2_buf,
> + 0, isp_vdev->format.sizeimage);
> +
> + state = done_suc ? VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
> + vb2_buffer_done(&isp4vid_buf->vb2.vb2_buf, state);
> +
> + dev_dbg(isp_vdev->dev, "call vb2_buffer_done(size=%u)\n",
> + isp_vdev->format.sizeimage);
> +}
[snip]
> +static void *isp4vid_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
> + struct dma_buf *dbuf,
> + unsigned long size)
> +{
> + struct isp4vid_vb2_buf *buf;
> +
> + if (dbuf->size < size) {
> + dev_err(dev, "Invalid dmabuf size %zu %lu", dbuf->size, size);
> + return ERR_PTR(-EFAULT);
> + }
> +
> + buf = kzalloc(sizeof(*buf), GFP_KERNEL);
> + if (!buf)
> + return ERR_PTR(-ENOMEM);
> +
> + struct isp4vid_vb2_buf *dbg_buf = dbuf->priv;
Move dbg_buf declaration to the top of the function.
> +
> + buf->dev = dev;
> + buf->dbuf = dbuf;
> + buf->size = size;
> +
> + dev_dbg(dev, "attach dmabuf of isp user bo 0x%llx size %ld",
> + dbg_buf->gpu_addr, dbg_buf->size);
> +
> + return buf;
> +}
> +
> +static void isp4vid_vb2_unmap_dmabuf(void *mem_priv)
> +{
> + struct isp4vid_vb2_buf *buf = mem_priv;
> + struct iosys_map map = IOSYS_MAP_INIT_VADDR(buf->vaddr);
> +
> + dev_dbg(buf->dev, "unmap dmabuf of isp user bo 0x%llx size %ld",
> + buf->gpu_addr, buf->size);
> +
> + dma_buf_vunmap_unlocked(buf->dbuf, &map);
> + buf->vaddr = NULL;
> +}
> +
> +static int isp4vid_vb2_map_dmabuf(void *mem_priv)
> +{
> + struct isp4vid_vb2_buf *mmap_buf = NULL;
Remove unnecessary initialization of `mmap_buf`, and combine it onto one line
with `buf`:
struct isp4vid_vb2_buf *buf = mem_priv, *mmap_buf;
> + struct isp4vid_vb2_buf *buf = mem_priv;
> + struct iosys_map map = {};
Remove unnecessary initialization of `map`, it is initialized inside
dma_buf_vmap_unlocked() at the very beginning.
> + int ret;
> +
> + ret = dma_buf_vmap_unlocked(buf->dbuf, &map);
> + if (ret) {
> + dev_err(buf->dev, "vmap_unlocked fail");
> + return -EFAULT;
> + }
> + buf->vaddr = map.vaddr;
> +
> + mmap_buf = buf->dbuf->priv;
> + buf->gpu_addr = mmap_buf->gpu_addr;
> +
> + dev_dbg(buf->dev, "map dmabuf of isp user bo 0x%llx size %ld",
> + buf->gpu_addr, buf->size);
> +
> + return 0;
> +}
[snip]
> +static const struct v4l2_pix_format isp4vid_fmt_default = {
> + .width = 1920,
> + .height = 1080,
> + .pixelformat = V4L2_PIX_FMT_NV12,
Set .pixelformat to ISP4VID_DEFAULT_FMT.
> + .field = V4L2_FIELD_NONE,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> +};
> +
> +static void isp4vid_capture_return_all_buffers(struct isp4vid_dev *isp_vdev,
> + enum vb2_buffer_state state)
> +{
> + struct isp4vid_capture_buffer *vbuf, *node;
> +
> + mutex_lock(&isp_vdev->buf_list_lock);
> +
> + list_for_each_entry_safe(vbuf, node, &isp_vdev->buf_list, list) {
> + list_del(&vbuf->list);
> + vb2_buffer_done(&vbuf->vb2.vb2_buf, state);
> + }
> + mutex_unlock(&isp_vdev->buf_list_lock);
Change to this starting from the mutex_lock():
scoped_guard(mutex, &isp_vdev->buf_list_lock) {
list_for_each_entry_safe(vbuf, node, &isp_vdev->buf_list, list)
vb2_buffer_done(&vbuf->vb2.vb2_buf, state);
INIT_LIST_HEAD(&isp_vdev->buf_list);
}
> +
> + dev_dbg(isp_vdev->dev, "call vb2_buffer_done(%d)\n", state);
> +}
> +
> +static int isp4vid_vdev_link_validate(struct media_link *link)
> +{
> + return 0;
> +}
> +
> +static const struct media_entity_operations isp4vid_vdev_ent_ops = {
> + .link_validate = isp4vid_vdev_link_validate,
> +};
> +
> +static const struct v4l2_file_operations isp4vid_vdev_fops = {
> + .owner = THIS_MODULE,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .read = vb2_fop_read,
> + .poll = vb2_fop_poll,
> + .unlocked_ioctl = video_ioctl2,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +static int isp4vid_ioctl_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
> +{
> + struct isp4vid_dev *isp_vdev = video_drvdata(file);
> +
> + strscpy(cap->driver, ISP4VID_ISP_DRV_NAME, sizeof(cap->driver));
> + snprintf(cap->card, sizeof(cap->card), "%s", ISP4VID_ISP_DRV_NAME);
> + cap->capabilities |= V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
> +
> + dev_dbg(isp_vdev->dev, "%s|capabilities=0x%X", isp_vdev->vdev.name, cap->capabilities);
> +
> + return 0;
> +}
> +
> +static int isp4vid_g_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f)
> +{
> + struct isp4vid_dev *isp_vdev = video_drvdata(file);
> +
> + f->fmt.pix = isp_vdev->format;
> +
> + return 0;
> +}
> +
> +static int isp4vid_try_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f)
> +{
> + struct isp4vid_dev *isp_vdev = video_drvdata(file);
> + struct v4l2_pix_format *format = &f->fmt.pix;
> + unsigned int i;
Change to `int i;` for consistency.
> +
> + if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> + return -EINVAL;
> +
> + /*
> + * Check if the hardware supports the requested format, use the default
> + * format otherwise.
> + */
> + for (i = 0; i < ARRAY_SIZE(isp4vid_formats); i++)
> + if (isp4vid_formats[i] == format->pixelformat)
> + break;
> +
> + if (i == ARRAY_SIZE(isp4vid_formats))
> + format->pixelformat = ISP4VID_DEFAULT_FMT;
> +
> + switch (format->pixelformat) {
> + case V4L2_PIX_FMT_NV12: {
> + const struct v4l2_frmsize_discrete *fsz =
> + v4l2_find_nearest_size(isp4vid_frmsize,
> + ARRAY_SIZE(isp4vid_frmsize),
> + width, height,
> + format->width, format->height);
> +
> + format->width = fsz->width;
> + format->height = fsz->height;
> +
> + format->bytesperline = format->width;
> + format->sizeimage = format->bytesperline *
> + format->height * 3 / 2;
> + }
> + break;
> + case V4L2_PIX_FMT_YUYV: {
> + const struct v4l2_frmsize_discrete *fsz =
> + v4l2_find_nearest_size(isp4vid_frmsize,
> + ARRAY_SIZE(isp4vid_frmsize),
> + width, height,
> + format->width, format->height);
> +
> + format->width = fsz->width;
> + format->height = fsz->height;
> +
> + format->bytesperline = format->width * 2;
> + format->sizeimage = format->bytesperline * format->height;
> + }
> + break;
> + default:
> + dev_err(isp_vdev->dev, "%s|unsupported fmt=%u",
> + isp_vdev->vdev.name, format->pixelformat);
> + return -EINVAL;
> + }
Create a variable declaration `const struct v4l2_frmsize_discrete *fsz;` at the
top of the function and change everything starting from the switch to this:
fsz = v4l2_find_nearest_size(isp4vid_frmsize,
ARRAY_SIZE(isp4vid_frmsize), width, height,
format->width, format->height);
format->width = fsz->width;
format->height = fsz->height;
isp4vid_fill_buffer_size(format);
And this will go with a complementary change to isp4vid_fill_buffer_size() to
make it possible to reuse isp4vid_fill_buffer_size() here to remove code
duplication.
> +
> + if (format->field == V4L2_FIELD_ANY)
> + format->field = isp4vid_fmt_default.field;
> +
> + if (format->colorspace == V4L2_COLORSPACE_DEFAULT)
> + format->colorspace = isp4vid_fmt_default.colorspace;
> +
> + return 0;
> +}
[snip]
> +static int isp4vid_fill_buffer_size(struct isp4vid_dev *isp_vdev)
> +{
> + int ret = 0;
> +
> + switch (isp_vdev->format.pixelformat) {
> + case V4L2_PIX_FMT_NV12:
> + isp_vdev->format.bytesperline = isp_vdev->format.width;
> + isp_vdev->format.sizeimage = isp_vdev->format.bytesperline *
> + isp_vdev->format.height * 3 / 2;
> + break;
> + case V4L2_PIX_FMT_YUYV:
> + isp_vdev->format.bytesperline = isp_vdev->format.width;
> + isp_vdev->format.sizeimage = isp_vdev->format.bytesperline *
> + isp_vdev->format.height * 2;
> + break;
> + default:
> + dev_err(isp_vdev->dev, "fail for invalid default format 0x%x",
> + isp_vdev->format.pixelformat);
> + ret = -EINVAL;
> + break;
> + }
> +
> + return ret;
> +}
Looks like isp_vdev->format.bytesperline is set wrong here for YUYV, it should
be width * 2.
Move isp4vid_fill_buffer_size() definition above isp4vid_try_fmt_vid_cap() so it
can be used there and change isp4vid_fill_buffer_size() to this:
static int isp4vid_fill_buffer_size(struct v4l2_pix_format *fmt)
{
switch (fmt->pixelformat) {
case V4L2_PIX_FMT_NV12: {
fmt->bytesperline = fmt->width;
fmt->sizeimage = fmt->bytesperline * fmt->height * 3 / 2;
break;
case V4L2_PIX_FMT_YUYV:
fmt->bytesperline = fmt->width * 2;
fmt->sizeimage = fmt->bytesperline * fmt->height;
break;
default:
return -EINVAL;
}
return 0;
}
This fixes the isp_vdev->format.bytesperline issue too.
> +
> +static const struct vb2_ops isp4vid_qops = {
> + .queue_setup = isp4vid_qops_queue_setup,
> + .buf_queue = isp4vid_qops_buffer_queue,
> + .start_streaming = isp4vid_qops_start_streaming,
> + .stop_streaming = isp4vid_qops_stop_streaming,
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> +};
> +
> +int isp4vid_dev_init(struct isp4vid_dev *isp_vdev, struct v4l2_subdev *isp_sdev,
> + const struct isp4vid_ops *ops)
> +{
> + const char *vdev_name = isp4vid_video_dev_name;
> + struct v4l2_device *v4l2_dev;
> + struct video_device *vdev;
> + struct vb2_queue *q;
> + int ret;
> +
> + if (!isp_vdev || !isp_sdev || !isp_sdev->v4l2_dev)
> + return -EINVAL;
> +
> + v4l2_dev = isp_sdev->v4l2_dev;
> + vdev = &isp_vdev->vdev;
> +
> + isp_vdev->isp_sdev = isp_sdev;
> + isp_vdev->dev = v4l2_dev->dev;
> + isp_vdev->ops = ops;
> +
> + /* Initialize the vb2_queue struct */
> + mutex_init(&isp_vdev->vbq_lock);
> + q = &isp_vdev->vbq;
> + q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + q->io_modes = VB2_MMAP | VB2_DMABUF;
> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + q->buf_struct_size = sizeof(struct isp4vid_capture_buffer);
> + q->min_queued_buffers = 2;
> + q->ops = &isp4vid_qops;
> + q->drv_priv = isp_vdev;
> + q->mem_ops = &isp4vid_vb2_memops;
> + q->lock = &isp_vdev->vbq_lock;
> + q->dev = v4l2_dev->dev;
> + ret = vb2_queue_init(q);
> + if (ret) {
> + dev_err(v4l2_dev->dev, "vb2_queue_init error:%d", ret);
> + return ret;
> + }
> + /* Initialize buffer list and its lock */
> + mutex_init(&isp_vdev->buf_list_lock);
> + INIT_LIST_HEAD(&isp_vdev->buf_list);
> +
> + /* Set default frame format */
> + isp_vdev->format = isp4vid_fmt_default;
> + isp_vdev->timeperframe = ISP4VID_ISP_TPF_DEFAULT;
> + v4l2_simplify_fraction(&isp_vdev->timeperframe.numerator,
> + &isp_vdev->timeperframe.denominator, 8, 333);
> +
> + ret = isp4vid_fill_buffer_size(isp_vdev);
Change to `ret = isp4vid_fill_buffer_size(&isp_vdev->format);`
> + if (ret) {
> + dev_err(v4l2_dev->dev, "fail to fill buffer size: %d\n", ret);
> + return ret;
> + }
> +
> + ret = isp4vid_set_fmt_2_isp(isp_sdev, &isp_vdev->format);
> + if (ret) {
> + dev_err(v4l2_dev->dev, "fail init format :%d\n", ret);
> + return ret;
> + }
> +
> + /* Initialize the video_device struct */
> + isp_vdev->vdev.entity.name = vdev_name;
> + isp_vdev->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
> + isp_vdev->vdev_pad.flags = MEDIA_PAD_FL_SINK;
> + ret = media_entity_pads_init(&isp_vdev->vdev.entity, 1,
> + &isp_vdev->vdev_pad);
> +
> + if (ret) {
> + dev_err(v4l2_dev->dev, "init media entity pad fail:%d\n", ret);
> + return ret;
> + }
> +
> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE |
> + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> + vdev->entity.ops = &isp4vid_vdev_ent_ops;
> + vdev->release = video_device_release_empty;
> + vdev->fops = &isp4vid_vdev_fops;
> + vdev->ioctl_ops = &isp4vid_vdev_ioctl_ops;
> + vdev->lock = NULL;
> + vdev->queue = q;
> + vdev->v4l2_dev = v4l2_dev;
> + vdev->vfl_dir = VFL_DIR_RX;
> + strscpy(vdev->name, vdev_name, sizeof(vdev->name));
> + video_set_drvdata(vdev, isp_vdev);
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret)
> + dev_err(v4l2_dev->dev, "register video device fail:%d\n", ret);
No error handling?
> +
> + return ret;
> +}
> +
> +void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev)
> +{
> + video_unregister_device(&isp_vdev->vdev);
> +}
> diff --git a/drivers/media/platform/amd/isp4/isp4_video.h b/drivers/media/platform/amd/isp4/isp4_video.h
> new file mode 100644
> index 000000000000..fae7dbdedd18
> --- /dev/null
> +++ b/drivers/media/platform/amd/isp4/isp4_video.h
[snip]
> +
> +struct isp4vid_capture_buffer {
> + /*
> + * struct vb2_v4l2_buffer must be the first element
> + * the videobuf2 framework will allocate this struct based on
> + * buf_struct_size and use the first sizeof(struct vb2_buffer) bytes of
> + * memory as a vb2_buffer
> + */
> + struct vb2_v4l2_buffer vb2;
> + struct isp4if_img_buf_info img_buf;
> + struct list_head list;
> +};
> +
> +struct isp4vid_dev;
Unnecessary isp4vid_dev forward declaration, remove it.
> +
> +struct isp4vid_ops {
> + int (*send_buffer)(struct v4l2_subdev *sd,
> + struct isp4if_img_buf_info *img_buf);
> +};
> +
> +struct isp4vid_dev {
> + struct video_device vdev;
> + struct media_pad vdev_pad;
> + struct v4l2_pix_format format;
> +
> + /* mutex that protects vbq */
> + struct mutex vbq_lock;
> + struct vb2_queue vbq;
> +
> + /* mutex that protects buf_list */
> + struct mutex buf_list_lock;
> + struct list_head buf_list;
> +
> + u32 sequence;
> + bool stream_started;
> + struct task_struct *kthread;
Remove unused `kthread` struct member.
> +
> + struct media_pipeline pipe;
> + struct device *dev;
> + struct v4l2_subdev *isp_sdev;
> + struct v4l2_fract timeperframe;
> +
> + /* Callback operations */
> + const struct isp4vid_ops *ops;
> +};
> +
> +int isp4vid_dev_init(struct isp4vid_dev *isp_vdev,
> + struct v4l2_subdev *isp_sdev,
> + const struct isp4vid_ops *ops);
> +
> +void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev);
> +
> +s32 isp4vid_notify(void *cb_ctx, struct isp4vid_framedone_param *evt_param);
> +
> +#endif
> --
> 2.34.1
>
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/media/media-device.h?h=v6.17-rc7#n204
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/media/media-device.h?h=v6.17-rc7#n289
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added
2025-09-30 7:30 ` Du, Bin
@ 2025-10-01 7:24 ` Sultan Alsawaf
2025-10-10 10:25 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-10-01 7:24 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On Tue, Sep 30, 2025 at 03:30:49PM +0800, Du, Bin wrote:
> On 9/23/2025 3:23 PM, Sultan Alsawaf wrote:
> > On Thu, Sep 11, 2025 at 06:08:44PM +0800, Bin Du wrote:
> > > Isp4 sub-device is implementing v4l2 sub-device interface. It has one
> > > capture video node, and supports only preview stream. It manages firmware
> > > states, stream configuration. Add interrupt handling and notification for
> > > isp firmware to isp-subdevice.
> > >
> > > Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> > > Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
> > > Signed-off-by: Bin Du <Bin.Du@amd.com>
> > > Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
> >
> > [snip]
> >
> > > +++ b/drivers/media/platform/amd/isp4/isp4.c
> > > @@ -5,13 +5,19 @@
> > > #include <linux/pm_runtime.h>
> > > #include <linux/vmalloc.h>
> > > +
> > > +#include <media/v4l2-fwnode.h>
> > > #include <media/v4l2-ioctl.h>
> > > #include "isp4.h"
> > > -
> > > -#define VIDEO_BUF_NUM 5
> > > +#include "isp4_hw_reg.h"
> > > #define ISP4_DRV_NAME "amd_isp_capture"
> > > +#define ISP4_FW_RESP_RB_IRQ_STATUS_MASK \
> > > + (ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK | \
> > > + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK | \
> > > + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK | \
> > > + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK)
> > > /* interrupt num */
> > > static const u32 isp4_ringbuf_interrupt_num[] = {
> > > @@ -21,19 +27,95 @@ static const u32 isp4_ringbuf_interrupt_num[] = {
> > > 4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
> > > };
> > > -#define to_isp4_device(dev) \
> > > - ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
> > > +#define to_isp4_device(dev) container_of(dev, struct isp4_device, v4l2_dev)
> > > +
> > > +static void isp4_wake_up_resp_thread(struct isp4_subdev *isp, u32 index)
> > > +{
> > > + if (isp && index < ISP4SD_MAX_FW_RESP_STREAM_NUM) {
> > > + struct isp4sd_thread_handler *thread_ctx =
> > > + &isp->fw_resp_thread[index];
> > > +
> > > + thread_ctx->wq_cond = 1;
> > > + wake_up_interruptible(&thread_ctx->waitq);
> > > + }
> > > +}
> > > +
> > > +static void isp4_resp_interrupt_notify(struct isp4_subdev *isp, u32 intr_status)
> > > +{
> > > + bool wake = (isp->ispif.status == ISP4IF_STATUS_FW_RUNNING);
> > > +
> > > + u32 intr_ack = 0;
> > > +
> > > + /* global response */
> > > + if (intr_status &
> > > + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK) {
> > > + if (wake)
> > > + isp4_wake_up_resp_thread(isp, 0);
> > > +
> > > + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT12_ACK_MASK;
> >
> > The INT_MASKs and ACK_MASKs are the same; perhaps the ACK_MASKs can just be
> > removed so you can just write intr_status to ISP_SYS_INT0_ACK instead?
> >
>
> These macro definitions are automatically generated from the IP design by
> the hardware team. INT_MASK and ACK_MASK represent specific bits in
> different registers—the status and acknowledgment registers, respectively.
> While their values are currently the same, they could differ depending on
> the IP design. I prefer to keep both definitions to maintain clarity.
Sure, no problem.
> > > +
> > > + /* clear ISP_SYS interrupts */
> > > + isp4hw_wreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_ACK, intr_ack);
> > > +}
> > > static irqreturn_t isp4_irq_handler(int irq, void *arg)
> > > {
> > > + struct isp4_device *isp_dev = dev_get_drvdata(arg);
> >
> > This is technically a data race because setting drvdata and reading drvdata do
> > not use WRITE_ONCE() and READ_ONCE(), respectively. And enabling the IRQ before
> > the handler is allowed to do anything is why that `if (!isp_dev)` check exists,
> > because that is another race.
> >
> > Instead, pass the isp_dev pointer through the private pointer of
> > devm_request_irq() and add IRQ_NOAUTOEN so the IRQ is enabled by default. Then,
> > when it is safe for the IRQ to run, enable it with enable_irq().
> >
> > That way you can delete the `if (!isp_dev)` check and resolve the two races.
> >
>
> Good deep insight, suppose you mean use IRQ_NOAUTOEN to make irq default
> disabled. Sure, will add support to dynamically enable/disable IRQ during
> camera open/close and remove unnecessary check.
Sorry for the typo, meant to say default disabled indeed. :)
> > > + u32 r1;
> > > +
> > > + if (!isp_dev)
> > > + goto error_drv_data;
> > > +
> > > + isp = &isp_dev->isp_sdev;
> > > + /* check ISP_SYS interrupts status */
> > > + r1 = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_STATUS);
> > > +
> > > + isp_sys_irq_status = r1 & ISP4_FW_RESP_RB_IRQ_STATUS_MASK;
> >
> > There are four IRQs (one for each stream) but any one of the IRQs can result in
> > a notification for _all_ streams. Each IRQ should only do the work of its own
> > stream.
> >
> > You can do this by passing devm_request_irq() a private pointer to indicate the
> > mapping between a stream and its IRQ, so that isp4_irq_handler() can know which
> > stream it should look at.
> >
>
> Will do optimization to remove unused IRQs and keep only necessary ones
> (reduce from 4 to 2), actually an IRQ won't result in notification to all
> streams, please check the implementation of isp4_resp_interrupt_notify, it
> will only wake up IRQ corresponding stream processing thread.
What I mean is that the IRQ for one stream can wake a different stream if it is
also ready at the same time according to the interrupt status register.
Assume we have ISP_IRQ 0 and 1 for streams 1 (WPT9) and 2 (WPT10), respectively.
Consider the following sequence of events:
ISP_IRQ0 (WPT9) ISP_IRQ1 (WPT10)
--------------- ----------------
<interrupt fires> <interrupt fires>
isp4_irq_handler() isp4_irq_handler()
isp_sys_irq_status = WPT9|WPT10 isp_sys_irq_status = WPT9|WPT10
isp4_wake_up_resp_thread(isp, 1) isp4_wake_up_resp_thread(isp, 1)
// ^ woke up WPT9 from WPT10 IRQ!
isp4_wake_up_resp_thread(isp, 2) isp4_wake_up_resp_thread(isp, 2)
// ^ woke up WPT10 from WPT9 IRQ!
The problem is that isp4_irq_handler() doesn't know which IRQ triggered the call
into isp4_irq_handler().
> > > +static int isp4sd_init_meta_buf(struct isp4_subdev *isp_subdev)
> > > +{
> > > + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
> > > + struct isp4_interface *ispif = &isp_subdev->ispif;
> > > + struct device *dev = isp_subdev->dev;
> > > + u32 i;
> > > +
> > > + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
> > > + if (!sensor_info->meta_info_buf[i]) {
> > > + sensor_info->meta_info_buf[i] = ispif->metainfo_buf_pool[i];
> > > + if (!sensor_info->meta_info_buf[i]) {
> > > + dev_err(dev, "invalid %u meta_info_buf fail\n", i);
> > > + return -ENOMEM;
> > > + }
> > > + }
> > > + }
> > > +
> > > + return 0;
> > > +}
> >
> > What is the point of metainfo_buf_pool? Especially since metainfo_buf_pool[i] is
> > not set to NULL after this "allocation" occurs.
> >
> > I think isp4sd_init_meta_buf() and metainfo_buf_pool are unnecessary and can be
> > factored out.
> >
>
> I suppose you mean meta_info_buf, will remove it together with
> isp4sd_init_meta_buf() and use metainfo_buf_pool from ispif directly which
> is vital for ISP FW to carry response info.
I was thinking that metainfo_buf_pool could be renamed to meta_info_buf and then
the old meta_info_buf could be deleted. Same result either way. :)
> > > + init_waitqueue_head(&thread_ctx->waitq);
> > > + timeout = msecs_to_jiffies(ISP4SD_WAIT_RESP_IRQ_TIMEOUT);
> > > +
> > > + dev_dbg(dev, "[%u] started\n", para->idx);
> > > +
> > > + while (true) {
> > > + wait_event_interruptible_timeout(thread_ctx->waitq,
> > > + thread_ctx->wq_cond != 0,
> > > + timeout);
> >
> > Why is there a timeout? What does the timeout even do since the return value of
> > wait_event_interruptible_timeout() is not checked? Doesn't that mean that once
> > the timeout is hit, isp4sd_fw_resp_func() will be called for nothing?
> >
> > I observe that most of the time spent by these kthreads is due to the constant
> > wake-ups from the very short 5 ms timeout. This is bad for energy efficiency and
> > creates needless overhead.
> >
>
> Good catch, previouly before IRQ is really enabled, this is to make sure ISP
> can work normally even for 120fps sensor, since now IRQ is enabled, we can
> increase the timeout value to like 200ms to avoid the unwanted timeout
> caused wake-ups.
What should the kthread do when there is a timeout though? Is the timeout
necessary to detect when FW is no longer responding? If so, shouldn't there be
error handling?
If the timeout isn't used to check for error then I think it should be removed.
> > > + thread_ctx->wq_cond = 0;
> > > +
> > > + if (kthread_should_stop()) {
> > > + dev_dbg(dev, "[%u] quit\n", para->idx);
> > > + break;
> > > + }
> > > +
> > > + guard(mutex)(&thread_ctx->mutex);
> > > + isp4sd_fw_resp_func(isp_subdev, stream_id);
> > > + }
> > > +
> > > + mutex_destroy(&thread_ctx->mutex);
> > > +
> > > + return 0;
> > > +}
[snip]
> > > +
> > > +static int isp4sd_pwroff_and_deinit(struct isp4_subdev *isp_subdev)
> > > +{
> > > + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
> > > + unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_LOW;
> > > + struct isp4_interface *ispif = &isp_subdev->ispif;
> > > +
> > > + struct device *dev = isp_subdev->dev;
> > > + u32 cnt;
> > > + int ret;
> > > +
> > > + mutex_lock(&isp_subdev->ops_mutex);
> > > +
> > > + if (sensor_info->status == ISP4SD_START_STATUS_STARTED) {
> > > + dev_err(dev, "fail for stream still running\n");
> > > + mutex_unlock(&isp_subdev->ops_mutex);
> > > + return -EINVAL;
> > > + }
> > > +
> > > + sensor_info->status = ISP4SD_START_STATUS_NOT_START;
> > > + cnt = isp4sd_get_started_stream_count(isp_subdev);
> > > + if (cnt > 0) {
> > > + dev_dbg(dev, "no need power off isp_subdev\n");
> > > + mutex_unlock(&isp_subdev->ops_mutex);
> > > + return 0;
> > > + }
> > > +
> > > + isp4if_stop(ispif);
> > > +
> > > + ret = dev_pm_genpd_set_performance_state(dev, perf_state);
> > > + if (ret)
> > > + dev_err(dev,
> > > + "fail to set isp_subdev performance state %u,ret %d\n",
> > > + perf_state, ret);
> > > + isp4sd_stop_resp_proc_threads(isp_subdev);
> > > + dev_dbg(dev, "isp_subdev stop resp proc streads suc");
> > > + /* hold ccpu reset */
> > > + isp4hw_wreg(isp_subdev->mmio, ISP_SOFT_RESET, 0x0);
> > > + isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0);
> > > + ret = pm_runtime_put_sync(dev);
> > > + if (ret)
> > > + dev_err(dev, "power off isp_subdev fail %d\n", ret);
> > > + else
> > > + dev_dbg(dev, "power off isp_subdev suc\n");
> > > +
> > > + ispif->status = ISP4IF_STATUS_PWR_OFF;
> > > + isp4if_clear_cmdq(ispif);
> > > + isp4sd_module_enable(isp_subdev, false);
> > > +
> > > + msleep(20);
> >
> > What is this msleep for?
> >
>
> This is the HW requirement, at least 20ms is needed for the possible quickly
> open followed.
Add a comment explaining the HW requirement for this msleep.
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added
2025-10-01 7:24 ` Sultan Alsawaf
@ 2025-10-10 10:25 ` Du, Bin
2025-10-11 7:18 ` Sultan Alsawaf
0 siblings, 1 reply; 35+ messages in thread
From: Du, Bin @ 2025-10-10 10:25 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
Thanks, Sultan. sorry for the delayed response due to the long public
holiday here.
On 10/1/2025 3:24 PM, Sultan Alsawaf wrote:
> On Tue, Sep 30, 2025 at 03:30:49PM +0800, Du, Bin wrote:
>> On 9/23/2025 3:23 PM, Sultan Alsawaf wrote:
>>> On Thu, Sep 11, 2025 at 06:08:44PM +0800, Bin Du wrote:
>>>> Isp4 sub-device is implementing v4l2 sub-device interface. It has one
>>>> capture video node, and supports only preview stream. It manages firmware
>>>> states, stream configuration. Add interrupt handling and notification for
>>>> isp firmware to isp-subdevice.
>>>>
>>>> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>>>> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>>>> Signed-off-by: Bin Du <Bin.Du@amd.com>
>>>> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
>>>
>>> [snip]
>>>
>>>> +++ b/drivers/media/platform/amd/isp4/isp4.c
>>>> @@ -5,13 +5,19 @@
>>>> #include <linux/pm_runtime.h>
>>>> #include <linux/vmalloc.h>
>>>> +
>>>> +#include <media/v4l2-fwnode.h>
>>>> #include <media/v4l2-ioctl.h>
>>>> #include "isp4.h"
>>>> -
>>>> -#define VIDEO_BUF_NUM 5
>>>> +#include "isp4_hw_reg.h"
>>>> #define ISP4_DRV_NAME "amd_isp_capture"
>>>> +#define ISP4_FW_RESP_RB_IRQ_STATUS_MASK \
>>>> + (ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT9_INT_MASK | \
>>>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT10_INT_MASK | \
>>>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT11_INT_MASK | \
>>>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK)
>>>> /* interrupt num */
>>>> static const u32 isp4_ringbuf_interrupt_num[] = {
>>>> @@ -21,19 +27,95 @@ static const u32 isp4_ringbuf_interrupt_num[] = {
>>>> 4, /* ISP_4_1__SRCID__ISP_RINGBUFFER_WPT12 */
>>>> };
>>>> -#define to_isp4_device(dev) \
>>>> - ((struct isp4_device *)container_of(dev, struct isp4_device, v4l2_dev))
>>>> +#define to_isp4_device(dev) container_of(dev, struct isp4_device, v4l2_dev)
>>>> +
>>>> +static void isp4_wake_up_resp_thread(struct isp4_subdev *isp, u32 index)
>>>> +{
>>>> + if (isp && index < ISP4SD_MAX_FW_RESP_STREAM_NUM) {
>>>> + struct isp4sd_thread_handler *thread_ctx =
>>>> + &isp->fw_resp_thread[index];
>>>> +
>>>> + thread_ctx->wq_cond = 1;
>>>> + wake_up_interruptible(&thread_ctx->waitq);
>>>> + }
>>>> +}
>>>> +
>>>> +static void isp4_resp_interrupt_notify(struct isp4_subdev *isp, u32 intr_status)
>>>> +{
>>>> + bool wake = (isp->ispif.status == ISP4IF_STATUS_FW_RUNNING);
>>>> +
>>>> + u32 intr_ack = 0;
>>>> +
>>>> + /* global response */
>>>> + if (intr_status &
>>>> + ISP_SYS_INT0_STATUS__SYS_INT_RINGBUFFER_WPT12_INT_MASK) {
>>>> + if (wake)
>>>> + isp4_wake_up_resp_thread(isp, 0);
>>>> +
>>>> + intr_ack |= ISP_SYS_INT0_ACK__SYS_INT_RINGBUFFER_WPT12_ACK_MASK;
>>>
>>> The INT_MASKs and ACK_MASKs are the same; perhaps the ACK_MASKs can just be
>>> removed so you can just write intr_status to ISP_SYS_INT0_ACK instead?
>>>
>>
>> These macro definitions are automatically generated from the IP design by
>> the hardware team. INT_MASK and ACK_MASK represent specific bits in
>> different registers—the status and acknowledgment registers, respectively.
>> While their values are currently the same, they could differ depending on
>> the IP design. I prefer to keep both definitions to maintain clarity.
>
> Sure, no problem.
>
>>>> +
>>>> + /* clear ISP_SYS interrupts */
>>>> + isp4hw_wreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_ACK, intr_ack);
>>>> +}
>>>> static irqreturn_t isp4_irq_handler(int irq, void *arg)
>>>> {
>>>> + struct isp4_device *isp_dev = dev_get_drvdata(arg);
>>>
>>> This is technically a data race because setting drvdata and reading drvdata do
>>> not use WRITE_ONCE() and READ_ONCE(), respectively. And enabling the IRQ before
>>> the handler is allowed to do anything is why that `if (!isp_dev)` check exists,
>>> because that is another race.
>>>
>>> Instead, pass the isp_dev pointer through the private pointer of
>>> devm_request_irq() and add IRQ_NOAUTOEN so the IRQ is enabled by default. Then,
>>> when it is safe for the IRQ to run, enable it with enable_irq().
>>>
>>> That way you can delete the `if (!isp_dev)` check and resolve the two races.
>>>
>>
>> Good deep insight, suppose you mean use IRQ_NOAUTOEN to make irq default
>> disabled. Sure, will add support to dynamically enable/disable IRQ during
>> camera open/close and remove unnecessary check.
>
> Sorry for the typo, meant to say default disabled indeed. :)
>
No worries, got your point :)
>>>> + u32 r1;
>>>> +
>>>> + if (!isp_dev)
>>>> + goto error_drv_data;
>>>> +
>>>> + isp = &isp_dev->isp_sdev;
>>>> + /* check ISP_SYS interrupts status */
>>>> + r1 = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_STATUS);
>>>> +
>>>> + isp_sys_irq_status = r1 & ISP4_FW_RESP_RB_IRQ_STATUS_MASK;
>>>
>>> There are four IRQs (one for each stream) but any one of the IRQs can result in
>>> a notification for _all_ streams. Each IRQ should only do the work of its own
>>> stream.
>>>
>>> You can do this by passing devm_request_irq() a private pointer to indicate the
>>> mapping between a stream and its IRQ, so that isp4_irq_handler() can know which
>>> stream it should look at.
>>>
>>
>> Will do optimization to remove unused IRQs and keep only necessary ones
>> (reduce from 4 to 2), actually an IRQ won't result in notification to all
>> streams, please check the implementation of isp4_resp_interrupt_notify, it
>> will only wake up IRQ corresponding stream processing thread.
>
> What I mean is that the IRQ for one stream can wake a different stream if it is
> also ready at the same time according to the interrupt status register.
>
Yes, you are correct, besides its own stream, the IRQ may wake a
different stream if it is ready too in the IRQ status register. But i
believe the shared irq handler can improve the performance without
negative effects. The peseudo code of isp4_irq_handler works like this
(take your below example)
irqreturn_t isp4_irq_handler(...)
{
status = read_irq_status();
if(status & WPT9)
isp4_wake_up_resp_thread(isp, 1);
if(status & WPT10)
isp4_wake_up_resp_thread(isp, 2)
ack_irq_status(status);
return IRQ_HANDLED;
}
Which means the first isp4_irq_handler can process all IRQs at that
time. For the second isp4_irq_handler, because the irq status is cleared
by the first isp4_irq_handler, it just does nothing and quit. So even if
isp4_irq_handler doen't know exactly which IRQ triggers it, there's no
harm as far as I can tell, not sure if I missed something.
> Assume we have ISP_IRQ 0 and 1 for streams 1 (WPT9) and 2 (WPT10), respectively.
> Consider the following sequence of events:
>
> ISP_IRQ0 (WPT9) ISP_IRQ1 (WPT10)
> --------------- ----------------
> <interrupt fires> <interrupt fires>
> isp4_irq_handler() isp4_irq_handler()
> isp_sys_irq_status = WPT9|WPT10 isp_sys_irq_status = WPT9|WPT10
>
> isp4_wake_up_resp_thread(isp, 1) isp4_wake_up_resp_thread(isp, 1)
> // ^ woke up WPT9 from WPT10 IRQ!
>
> isp4_wake_up_resp_thread(isp, 2) isp4_wake_up_resp_thread(isp, 2)
> // ^ woke up WPT10 from WPT9 IRQ!
>
> The problem is that isp4_irq_handler() doesn't know which IRQ triggered the call
> into isp4_irq_handler().
>
>>>> +static int isp4sd_init_meta_buf(struct isp4_subdev *isp_subdev)
>>>> +{
>>>> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
>>>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>>>> + struct device *dev = isp_subdev->dev;
>>>> + u32 i;
>>>> +
>>>> + for (i = 0; i < ISP4IF_MAX_STREAM_BUF_COUNT; i++) {
>>>> + if (!sensor_info->meta_info_buf[i]) {
>>>> + sensor_info->meta_info_buf[i] = ispif->metainfo_buf_pool[i];
>>>> + if (!sensor_info->meta_info_buf[i]) {
>>>> + dev_err(dev, "invalid %u meta_info_buf fail\n", i);
>>>> + return -ENOMEM;
>>>> + }
>>>> + }
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>
>>> What is the point of metainfo_buf_pool? Especially since metainfo_buf_pool[i] is
>>> not set to NULL after this "allocation" occurs.
>>>
>>> I think isp4sd_init_meta_buf() and metainfo_buf_pool are unnecessary and can be
>>> factored out.
>>>
>>
>> I suppose you mean meta_info_buf, will remove it together with
>> isp4sd_init_meta_buf() and use metainfo_buf_pool from ispif directly which
>> is vital for ISP FW to carry response info.
>
> I was thinking that metainfo_buf_pool could be renamed to meta_info_buf and then
> the old meta_info_buf could be deleted. Same result either way. :)
>
Sure, will do that in the next version.
>>>> + init_waitqueue_head(&thread_ctx->waitq);
>>>> + timeout = msecs_to_jiffies(ISP4SD_WAIT_RESP_IRQ_TIMEOUT);
>>>> +
>>>> + dev_dbg(dev, "[%u] started\n", para->idx);
>>>> +
>>>> + while (true) {
>>>> + wait_event_interruptible_timeout(thread_ctx->waitq,
>>>> + thread_ctx->wq_cond != 0,
>>>> + timeout);
>>>
>>> Why is there a timeout? What does the timeout even do since the return value of
>>> wait_event_interruptible_timeout() is not checked? Doesn't that mean that once
>>> the timeout is hit, isp4sd_fw_resp_func() will be called for nothing?
>>>
>>> I observe that most of the time spent by these kthreads is due to the constant
>>> wake-ups from the very short 5 ms timeout. This is bad for energy efficiency and
>>> creates needless overhead.
>>>
>>
>> Good catch, previouly before IRQ is really enabled, this is to make sure ISP
>> can work normally even for 120fps sensor, since now IRQ is enabled, we can
>> increase the timeout value to like 200ms to avoid the unwanted timeout
>> caused wake-ups.
>
> What should the kthread do when there is a timeout though? Is the timeout
> necessary to detect when FW is no longer responding? If so, shouldn't there be
> error handling?
>
> If the timeout isn't used to check for error then I think it should be removed.
>
Yes, good suggestion, will remove the timeout in the next version
>>>> + thread_ctx->wq_cond = 0;
>>>> +
>>>> + if (kthread_should_stop()) {
>>>> + dev_dbg(dev, "[%u] quit\n", para->idx);
>>>> + break;
>>>> + }
>>>> +
>>>> + guard(mutex)(&thread_ctx->mutex);
>>>> + isp4sd_fw_resp_func(isp_subdev, stream_id);
>>>> + }
>>>> +
>>>> + mutex_destroy(&thread_ctx->mutex);
>>>> +
>>>> + return 0;
>>>> +}
>
> [snip]
>
>>>> +
>>>> +static int isp4sd_pwroff_and_deinit(struct isp4_subdev *isp_subdev)
>>>> +{
>>>> + struct isp4sd_sensor_info *sensor_info = &isp_subdev->sensor_info;
>>>> + unsigned int perf_state = ISP4SD_PERFORMANCE_STATE_LOW;
>>>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>>>> +
>>>> + struct device *dev = isp_subdev->dev;
>>>> + u32 cnt;
>>>> + int ret;
>>>> +
>>>> + mutex_lock(&isp_subdev->ops_mutex);
>>>> +
>>>> + if (sensor_info->status == ISP4SD_START_STATUS_STARTED) {
>>>> + dev_err(dev, "fail for stream still running\n");
>>>> + mutex_unlock(&isp_subdev->ops_mutex);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + sensor_info->status = ISP4SD_START_STATUS_NOT_START;
>>>> + cnt = isp4sd_get_started_stream_count(isp_subdev);
>>>> + if (cnt > 0) {
>>>> + dev_dbg(dev, "no need power off isp_subdev\n");
>>>> + mutex_unlock(&isp_subdev->ops_mutex);
>>>> + return 0;
>>>> + }
>>>> +
>>>> + isp4if_stop(ispif);
>>>> +
>>>> + ret = dev_pm_genpd_set_performance_state(dev, perf_state);
>>>> + if (ret)
>>>> + dev_err(dev,
>>>> + "fail to set isp_subdev performance state %u,ret %d\n",
>>>> + perf_state, ret);
>>>> + isp4sd_stop_resp_proc_threads(isp_subdev);
>>>> + dev_dbg(dev, "isp_subdev stop resp proc streads suc");
>>>> + /* hold ccpu reset */
>>>> + isp4hw_wreg(isp_subdev->mmio, ISP_SOFT_RESET, 0x0);
>>>> + isp4hw_wreg(isp_subdev->mmio, ISP_POWER_STATUS, 0);
>>>> + ret = pm_runtime_put_sync(dev);
>>>> + if (ret)
>>>> + dev_err(dev, "power off isp_subdev fail %d\n", ret);
>>>> + else
>>>> + dev_dbg(dev, "power off isp_subdev suc\n");
>>>> +
>>>> + ispif->status = ISP4IF_STATUS_PWR_OFF;
>>>> + isp4if_clear_cmdq(ispif);
>>>> + isp4sd_module_enable(isp_subdev, false);
>>>> +
>>>> + msleep(20);
>>>
>>> What is this msleep for?
>>>
>>
>> This is the HW requirement, at least 20ms is needed for the possible quickly
>> open followed.
>
> Add a comment explaining the HW requirement for this msleep.
>
Sure, will add comments to make it clear
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added
2025-10-10 10:25 ` Du, Bin
@ 2025-10-11 7:18 ` Sultan Alsawaf
2025-10-11 8:27 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-10-11 7:18 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On Fri, Oct 10, 2025 at 06:25:48PM +0800, Du, Bin wrote:
> Thanks, Sultan. sorry for the delayed response due to the long public
> holiday here.
No worries, hope you had a good holiday. :)
> On 10/1/2025 3:24 PM, Sultan Alsawaf wrote:
> > On Tue, Sep 30, 2025 at 03:30:49PM +0800, Du, Bin wrote:
> > > On 9/23/2025 3:23 PM, Sultan Alsawaf wrote:
> > > > On Thu, Sep 11, 2025 at 06:08:44PM +0800, Bin Du wrote:
> > > > > + u32 r1;
> > > > > +
> > > > > + if (!isp_dev)
> > > > > + goto error_drv_data;
> > > > > +
> > > > > + isp = &isp_dev->isp_sdev;
> > > > > + /* check ISP_SYS interrupts status */
> > > > > + r1 = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_STATUS);
> > > > > +
> > > > > + isp_sys_irq_status = r1 & ISP4_FW_RESP_RB_IRQ_STATUS_MASK;
> > > >
> > > > There are four IRQs (one for each stream) but any one of the IRQs can result in
> > > > a notification for _all_ streams. Each IRQ should only do the work of its own
> > > > stream.
> > > >
> > > > You can do this by passing devm_request_irq() a private pointer to indicate the
> > > > mapping between a stream and its IRQ, so that isp4_irq_handler() can know which
> > > > stream it should look at.
> > > >
> > >
> > > Will do optimization to remove unused IRQs and keep only necessary ones
> > > (reduce from 4 to 2), actually an IRQ won't result in notification to all
> > > streams, please check the implementation of isp4_resp_interrupt_notify, it
> > > will only wake up IRQ corresponding stream processing thread.
> >
> > What I mean is that the IRQ for one stream can wake a different stream if it is
> > also ready at the same time according to the interrupt status register.
> >
>
> Yes, you are correct, besides its own stream, the IRQ may wake a different
> stream if it is ready too in the IRQ status register. But i believe the
> shared irq handler can improve the performance without negative effects. The
> peseudo code of isp4_irq_handler works like this (take your below example)
> irqreturn_t isp4_irq_handler(...)
> {
> status = read_irq_status();
> if(status & WPT9)
> isp4_wake_up_resp_thread(isp, 1);
> if(status & WPT10)
> isp4_wake_up_resp_thread(isp, 2)
> ack_irq_status(status);
> return IRQ_HANDLED;
> }
> Which means the first isp4_irq_handler can process all IRQs at that time.
> For the second isp4_irq_handler, because the irq status is cleared by the
> first isp4_irq_handler, it just does nothing and quit. So even if
> isp4_irq_handler doen't know exactly which IRQ triggers it, there's no harm
> as far as I can tell, not sure if I missed something.
My thinking was that it's possible for duplicate wakeups to occur when the
stream IRQs are affined to different CPUs and they fire around the same time in
parallel.
But now that I see how the ISP interrupts are actually GPU interrupts, it means
that the stream IRQs will always be affined to the same CPU as each other.
So my concern does not apply here, and you should disregard it. :)
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added
2025-10-11 7:18 ` Sultan Alsawaf
@ 2025-10-11 8:27 ` Du, Bin
0 siblings, 0 replies; 35+ messages in thread
From: Du, Bin @ 2025-10-11 8:27 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On 10/11/2025 3:18 PM, Sultan Alsawaf wrote:
> On Fri, Oct 10, 2025 at 06:25:48PM +0800, Du, Bin wrote:
>> Thanks, Sultan. sorry for the delayed response due to the long public
>> holiday here.
>
> No worries, hope you had a good holiday. :)
>
Thank you!
>> On 10/1/2025 3:24 PM, Sultan Alsawaf wrote:
>>> On Tue, Sep 30, 2025 at 03:30:49PM +0800, Du, Bin wrote:
>>>> On 9/23/2025 3:23 PM, Sultan Alsawaf wrote:
>>>>> On Thu, Sep 11, 2025 at 06:08:44PM +0800, Bin Du wrote:
>>>>>> + u32 r1;
>>>>>> +
>>>>>> + if (!isp_dev)
>>>>>> + goto error_drv_data;
>>>>>> +
>>>>>> + isp = &isp_dev->isp_sdev;
>>>>>> + /* check ISP_SYS interrupts status */
>>>>>> + r1 = isp4hw_rreg(ISP4_GET_ISP_REG_BASE(isp), ISP_SYS_INT0_STATUS);
>>>>>> +
>>>>>> + isp_sys_irq_status = r1 & ISP4_FW_RESP_RB_IRQ_STATUS_MASK;
>>>>>
>>>>> There are four IRQs (one for each stream) but any one of the IRQs can result in
>>>>> a notification for _all_ streams. Each IRQ should only do the work of its own
>>>>> stream.
>>>>>
>>>>> You can do this by passing devm_request_irq() a private pointer to indicate the
>>>>> mapping between a stream and its IRQ, so that isp4_irq_handler() can know which
>>>>> stream it should look at.
>>>>>
>>>>
>>>> Will do optimization to remove unused IRQs and keep only necessary ones
>>>> (reduce from 4 to 2), actually an IRQ won't result in notification to all
>>>> streams, please check the implementation of isp4_resp_interrupt_notify, it
>>>> will only wake up IRQ corresponding stream processing thread.
>>>
>>> What I mean is that the IRQ for one stream can wake a different stream if it is
>>> also ready at the same time according to the interrupt status register.
>>>
>>
>> Yes, you are correct, besides its own stream, the IRQ may wake a different
>> stream if it is ready too in the IRQ status register. But i believe the
>> shared irq handler can improve the performance without negative effects. The
>> peseudo code of isp4_irq_handler works like this (take your below example)
>> irqreturn_t isp4_irq_handler(...)
>> {
>> status = read_irq_status();
>> if(status & WPT9)
>> isp4_wake_up_resp_thread(isp, 1);
>> if(status & WPT10)
>> isp4_wake_up_resp_thread(isp, 2)
>> ack_irq_status(status);
>> return IRQ_HANDLED;
>> }
>> Which means the first isp4_irq_handler can process all IRQs at that time.
>> For the second isp4_irq_handler, because the irq status is cleared by the
>> first isp4_irq_handler, it just does nothing and quit. So even if
>> isp4_irq_handler doen't know exactly which IRQ triggers it, there's no harm
>> as far as I can tell, not sure if I missed something.
>
> My thinking was that it's possible for duplicate wakeups to occur when the
> stream IRQs are affined to different CPUs and they fire around the same time in
> parallel.
>
> But now that I see how the ISP interrupts are actually GPU interrupts, it means
> that the stream IRQs will always be affined to the same CPU as each other.
>
> So my concern does not apply here, and you should disregard it. :)
>
> Sultan
Thanks for the further clarification.
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-10-01 6:53 ` Sultan Alsawaf
@ 2025-10-11 9:30 ` Du, Bin
2025-10-12 6:08 ` Sultan Alsawaf
2025-10-16 8:13 ` Du, Bin
0 siblings, 2 replies; 35+ messages in thread
From: Du, Bin @ 2025-10-11 9:30 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Svetoslav Stoilov, Alexey Zagorodnikov
Many thanks Sultan for your review, that's really helpful.
On 10/1/2025 2:53 PM, Sultan Alsawaf wrote:
> Hi Bin,
>
> On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
>> Isp video implements v4l2 video interface and supports NV12 and YUYV. It
>> manages buffers, pipeline power and state. Cherry-picked Sultan's DMA
>> buffer related fix from branch v6.16-drm-tip-isp4-for-amd on
>> https://github.com/kerneltoast/kernel_x86_laptop.git
>>
>> Co-developed-by: Sultan Alsawaf <sultan@kerneltoast.com>
>> Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
>> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>> Signed-off-by: Bin Du <Bin.Du@amd.com>
>> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/isp4.c
>> @@ -178,6 +178,16 @@ static int isp4_capture_probe(struct platform_device *pdev)
>> goto err_isp4_deinit;
>> }
>>
>> + ret = media_create_pad_link(&isp_dev->isp_sdev.sdev.entity,
>> + 0, &isp_dev->isp_sdev.isp_vdev.vdev.entity,
>> + 0,
>> + MEDIA_LNK_FL_ENABLED |
>> + MEDIA_LNK_FL_IMMUTABLE);
>> + if (ret) {
>> + dev_err(dev, "fail to create pad link %d\n", ret);
>> + goto err_isp4_deinit;
>> + }
>> +
>
> Two problems with this hunk:
>
> 1. According to the comment in include/media/media-device.h [1],
> media_create_pad_link() should be called before media_device_register():
>
> * So drivers need to first initialize the media device, register any entity
> * within the media device, create pad to pad links and then finally register
> * the media device by calling media_device_register() as a final step.
>
> 2. Missing call to media_device_unregister() on error when
> media_create_pad_link() fails.
>
> Since the media_create_pad_link() will be moved before media_device_register(),
> we will need to clean up media_create_pad_link() when media_device_register()
> fails.
>
> The clean-up function for media_create_pad_link() is media_device_unregister().
> According to the comment for media_device_unregister() [2], it is safe to call
> media_device_unregister() on an unregistered media device that is initialized
> (through media_device_init()).
>
> In addition, this made me realize that there's no call to media_device_cleanup()
> in the entire driver too. This is the cleanup function for media_device_init(),
> so it should be called on error and on module unload.
>
> To summarize, make the following changes:
>
> 1. Move the media_create_pad_link() up, right before media_device_register().
>
> 2. When media_device_register() fails, clean up media_create_pad_link() by
> calling media_device_unregister().
>
> 3. Add a missing call to media_device_cleanup() on error and module unload to
> clean up media_device_init().
>
Very clear guide, will follow your advice.
>> platform_set_drvdata(pdev, isp_dev);
>>
>> return 0;
>> diff --git a/drivers/media/platform/amd/isp4/isp4_subdev.c b/drivers/media/platform/amd/isp4/isp4_subdev.c
>> index a9cb14de04ca..7d3339c915eb 100644
>> --- a/drivers/media/platform/amd/isp4/isp4_subdev.c
>> +++ b/drivers/media/platform/amd/isp4/isp4_subdev.c
>
> [snip]
>
>> +static int isp4sd_ioc_send_img_buf(struct v4l2_subdev *sd,
>> + struct isp4if_img_buf_info *buf_info)
>> +{
>> + struct isp4_subdev *isp_subdev = to_isp4_subdev(sd);
>> + struct isp4_interface *ispif = &isp_subdev->ispif;
>> + struct isp4if_img_buf_node *buf_node = NULL;
>
> Remove unnecessary initialization of `buf_node`.
>
Sure, will do that.
>> + struct device *dev = isp_subdev->dev;
>> + int ret = -EINVAL;
>
> Remove unnecessary initialization of `ret`.
>
Sure, will do that.
>> +
>> + mutex_lock(&isp_subdev->ops_mutex);
>
> Use guard() for this mutex and remove the unlock_and_return label.
>
Sure, will do that.
>> + /* TODO: remove isp_status */
>> + if (ispif->status != ISP4IF_STATUS_FW_RUNNING) {
>> + dev_err(dev, "fail send img buf for bad fsm %d\n",
>> + ispif->status);
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> + return -EINVAL;
>> + }
>> +
>> + buf_node = isp4if_alloc_buffer_node(buf_info);
>> + if (!buf_node) {
>> + dev_err(dev, "fail alloc sys img buf info node\n");
>> + ret = -ENOMEM;
>> + goto unlock_and_return;
>> + }
>> +
>> + ret = isp4if_queue_buffer(ispif, buf_node);
>> + if (ret) {
>> + dev_err(dev, "fail to queue image buf, %d\n", ret);
>> + goto error_release_buf_node;
>> + }
>> +
>> + if (!isp_subdev->sensor_info.start_stream_cmd_sent) {
>> + isp_subdev->sensor_info.buf_sent_cnt++;
>> +
>> + if (isp_subdev->sensor_info.buf_sent_cnt >=
>> + ISP4SD_MIN_BUF_CNT_BEF_START_STREAM) {
>> + ret = isp4if_send_command(ispif, CMD_ID_START_STREAM,
>> + NULL, 0);
>> +
>> + if (ret) {
>> + dev_err(dev, "fail to START_STREAM");
>> + goto error_release_buf_node;
>> + }
>> + isp_subdev->sensor_info.start_stream_cmd_sent = true;
>> + isp_subdev->sensor_info.output_info.start_status =
>> + ISP4SD_START_STATUS_STARTED;
>> + isp_subdev->sensor_info.status =
>> + ISP4SD_START_STATUS_STARTED;
>> + } else {
>> + dev_dbg(dev,
>> + "no send start,required %u,buf sent %u\n",
>
> Add a space after each comma in this string.
>
Sure, will add.
>> + ISP4SD_MIN_BUF_CNT_BEF_START_STREAM,
>> + isp_subdev->sensor_info.buf_sent_cnt);
>> + }
>> + }
>> +
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> +
>> + return 0;
>> +
>> +error_release_buf_node:
>> + isp4if_dealloc_buffer_node(buf_node);
>> +
>> +unlock_and_return:
>> + mutex_unlock(&isp_subdev->ops_mutex);
>> +
>> + return ret;
>> +}
>
> [snip]
>
>> +++ b/drivers/media/platform/amd/isp4/isp4_video.c
>> @@ -0,0 +1,1207 @@
>> +// SPDX-License-Identifier: GPL-2.0+
>> +/*
>> + * Copyright (C) 2025 Advanced Micro Devices, Inc.
>> + */
>> +
>> +#include <drm/amd/isp.h>
>> +#include <linux/pm_runtime.h>
>> +#include <linux/vmalloc.h>
>> +#include <media/v4l2-ioctl.h>
>> +#include <media/v4l2-mc.h>
>> +
>> +#include "isp4_interface.h"
>> +#include "isp4_subdev.h"
>> +#include "isp4_video.h"
>> +
>> +#define ISP4VID_ISP_DRV_NAME "amd_isp_capture"
>> +#define ISP4VID_MAX_PREVIEW_FPS 30
>> +#define ISP4VID_DEFAULT_FMT isp4vid_formats[0]
>> +
>> +#define ISP4VID_PAD_VIDEO_OUTPUT 0
>> +
>> +/* timeperframe default */
>> +#define ISP4VID_ISP_TPF_DEFAULT isp4vid_tpfs[0]
>> +
>> +/* amdisp buffer for vb2 operations */
>> +struct isp4vid_vb2_buf {
>> + struct device *dev;
>> + void *vaddr;
>> + unsigned long size;
>> + refcount_t refcount;
>> + struct dma_buf *dbuf;
>> + void *bo;
>> + u64 gpu_addr;
>> + struct vb2_vmarea_handler handler;
>> +};
>> +
>> +static int isp4vid_vb2_mmap(void *buf_priv, struct vm_area_struct *vma);
>
> Don't need the isp4vid_vb2_mmap() prototype here anymore, remove it.
>
Yes, it's not necessary, will remove it.
>> +
>> +static void isp4vid_vb2_put(void *buf_priv);
>> +
>> +static const char *isp4vid_video_dev_name = "Preview";
>
> Turn this into `static const char *const isp4vid_video_dev_name = "Preview";`
> which makes the `isp4vid_video_dev_name` variable itself const, so that you
> cannot change `isp4vid_video_dev_name` to something else.
>
Sure, will add as you suggested.
>> +
>> +/* Sizes must be in increasing order */
>> +static const struct v4l2_frmsize_discrete isp4vid_frmsize[] = {
>> + {640, 360},
>> + {640, 480},
>> + {1280, 720},
>> + {1280, 960},
>> + {1920, 1080},
>> + {1920, 1440},
>> + {2560, 1440},
>> + {2880, 1620},
>> + {2880, 1624},
>> + {2888, 1808},
>> +};
>> +
>> +static const u32 isp4vid_formats[] = {
>> + V4L2_PIX_FMT_NV12,
>> + V4L2_PIX_FMT_YUYV
>> +};
>> +
>> +/* timeperframe list */
>> +static const struct v4l2_fract isp4vid_tpfs[] = {
>> + {.numerator = 1, .denominator = ISP4VID_MAX_PREVIEW_FPS}
>
> Add a space after { and a space before }.
>
> Also, it is more common to see v4l2_fract initialized without specifying the
> struct member names.
>
> To summarize, change to `{ 1, ISP4VID_MAX_PREVIEW_FPS }`
>
Will follow your advice. Seems no explicit description about this in
"Linux kernel coding style" and both spacing options after { are common
in the current kernel code.
>> +};
>> +
>> +static void isp4vid_handle_frame_done(struct isp4vid_dev *isp_vdev,
>> + const struct isp4if_img_buf_info *img_buf,
>> + bool done_suc)
>> +{
>> + struct isp4vid_capture_buffer *isp4vid_buf;
>
> Rename isp4vid_buf to isp_buf like in isp4vid_qops_start_streaming().
>
Seems isp4vid_buf appears to be more descriptive, plan to rename isp_buf
in isp4vid_qops_start_streaming to isp4vid_buf, what do you think?
>> + enum vb2_buffer_state state;
>> + void *vbuf;
>> +
>> + mutex_lock(&isp_vdev->buf_list_lock);
>> +
>> + /* Get the first entry of the list */
>> + isp4vid_buf = list_first_entry_or_null(&isp_vdev->buf_list, typeof(*isp4vid_buf), list);
>> + if (!isp4vid_buf) {
>> + mutex_unlock(&isp_vdev->buf_list_lock);
>> + return;
>> + }
>> +
>> + vbuf = vb2_plane_vaddr(&isp4vid_buf->vb2.vb2_buf, 0);
>> +
>> + if (vbuf != img_buf->planes[0].sys_addr) {
>> + dev_err(isp_vdev->dev, "Invalid vbuf");
>> + mutex_unlock(&isp_vdev->buf_list_lock);
>> + return;
>> + }
>> +
>> + /* Remove this entry from the list */
>> + list_del(&isp4vid_buf->list);
>> +
>> + mutex_unlock(&isp_vdev->buf_list_lock);
>
> Change to this starting from the mutex_lock():
>
> scoped_guard(mutex, &isp_vdev->buf_list_lock) {
> /* Get the first entry of the list */
> isp_buf = list_first_entry_or_null(&isp_vdev->buf_list,
> typeof(*isp_buf), list);
> if (!isp_buf)
> return;
>
> vbuf = vb2_plane_vaddr(&isp_buf->vb2.vb2_buf, 0);
> if (vbuf != img_buf->planes[0].sys_addr) {
> dev_err(isp_vdev->dev, "Invalid vbuf");
> return;
> }
>
> /* Remove this entry from the list */
> list_del(&isp_buf->list);
> }
>
Sure, good optimization.
>> +
>> + /* Fill the buffer */
>> + isp4vid_buf->vb2.vb2_buf.timestamp = ktime_get_ns();
>> + isp4vid_buf->vb2.sequence = isp_vdev->sequence++;
>> + isp4vid_buf->vb2.field = V4L2_FIELD_ANY;
>> +
>> + /* at most 2 planes */
>> + vb2_set_plane_payload(&isp4vid_buf->vb2.vb2_buf,
>> + 0, isp_vdev->format.sizeimage);
>> +
>> + state = done_suc ? VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
>> + vb2_buffer_done(&isp4vid_buf->vb2.vb2_buf, state);
>> +
>> + dev_dbg(isp_vdev->dev, "call vb2_buffer_done(size=%u)\n",
>> + isp_vdev->format.sizeimage);
>> +}
>
> [snip]
>
>> +static void *isp4vid_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev,
>> + struct dma_buf *dbuf,
>> + unsigned long size)
>> +{
>> + struct isp4vid_vb2_buf *buf;
>> +
>> + if (dbuf->size < size) {
>> + dev_err(dev, "Invalid dmabuf size %zu %lu", dbuf->size, size);
>> + return ERR_PTR(-EFAULT);
>> + }
>> +
>> + buf = kzalloc(sizeof(*buf), GFP_KERNEL);
>> + if (!buf)
>> + return ERR_PTR(-ENOMEM);
>> +
>> + struct isp4vid_vb2_buf *dbg_buf = dbuf->priv;
>
> Move dbg_buf declaration to the top of the function.
>
Sure, will do that.
>> +
>> + buf->dev = dev;
>> + buf->dbuf = dbuf;
>> + buf->size = size;
>> +
>> + dev_dbg(dev, "attach dmabuf of isp user bo 0x%llx size %ld",
>> + dbg_buf->gpu_addr, dbg_buf->size);
>> +
>> + return buf;
>> +}
>> +
>> +static void isp4vid_vb2_unmap_dmabuf(void *mem_priv)
>> +{
>> + struct isp4vid_vb2_buf *buf = mem_priv;
>> + struct iosys_map map = IOSYS_MAP_INIT_VADDR(buf->vaddr);
>> +
>> + dev_dbg(buf->dev, "unmap dmabuf of isp user bo 0x%llx size %ld",
>> + buf->gpu_addr, buf->size);
>> +
>> + dma_buf_vunmap_unlocked(buf->dbuf, &map);
>> + buf->vaddr = NULL;
>> +}
>> +
>> +static int isp4vid_vb2_map_dmabuf(void *mem_priv)
>> +{
>> + struct isp4vid_vb2_buf *mmap_buf = NULL;
>
> Remove unnecessary initialization of `mmap_buf`, and combine it onto one line
> with `buf`:
>
Sure, will remove unnecessary initialization of `mmap_buf, but i'd
rather not combine because it is mentioned in "Linux kernel coding
style" that "to this end, use just one data declaration per line (no
commas for multiple data declarations). This leaves you room for a small
comment on each item, explaining its use.", what do you think?
> struct isp4vid_vb2_buf *buf = mem_priv, *mmap_buf;
>
>> + struct isp4vid_vb2_buf *buf = mem_priv;
>> + struct iosys_map map = {};
>
> Remove unnecessary initialization of `map`, it is initialized inside
> dma_buf_vmap_unlocked() at the very beginning.
>
Sure, will remove it. Yes, map is initialized inside
dma_buf_vmap_unlocked by iosys_map_clear()
>> + int ret;
>> +
>> + ret = dma_buf_vmap_unlocked(buf->dbuf, &map);
>> + if (ret) {
>> + dev_err(buf->dev, "vmap_unlocked fail");
>> + return -EFAULT;
>> + }
>> + buf->vaddr = map.vaddr;
>> +
>> + mmap_buf = buf->dbuf->priv;
>> + buf->gpu_addr = mmap_buf->gpu_addr;
>> +
>> + dev_dbg(buf->dev, "map dmabuf of isp user bo 0x%llx size %ld",
>> + buf->gpu_addr, buf->size);
>> +
>> + return 0;
>> +}
>
> [snip]
>
>> +static const struct v4l2_pix_format isp4vid_fmt_default = {
>> + .width = 1920,
>> + .height = 1080,
>> + .pixelformat = V4L2_PIX_FMT_NV12,
>
> Set .pixelformat to ISP4VID_DEFAULT_FMT.
>
Sure, will modify.
>> + .field = V4L2_FIELD_NONE,
>> + .colorspace = V4L2_COLORSPACE_SRGB,
>> +};
>> +
>> +static void isp4vid_capture_return_all_buffers(struct isp4vid_dev *isp_vdev,
>> + enum vb2_buffer_state state)
>> +{
>> + struct isp4vid_capture_buffer *vbuf, *node;
>> +
>> + mutex_lock(&isp_vdev->buf_list_lock);
>> +
>> + list_for_each_entry_safe(vbuf, node, &isp_vdev->buf_list, list) {
>> + list_del(&vbuf->list);
>> + vb2_buffer_done(&vbuf->vb2.vb2_buf, state);
>> + }
>> + mutex_unlock(&isp_vdev->buf_list_lock);
>
> Change to this starting from the mutex_lock():
>
> scoped_guard(mutex, &isp_vdev->buf_list_lock) {
> list_for_each_entry_safe(vbuf, node, &isp_vdev->buf_list, list)
> vb2_buffer_done(&vbuf->vb2.vb2_buf, state);
> INIT_LIST_HEAD(&isp_vdev->buf_list);
> }
>
Sure, will do optimization as you advised.
>> +
>> + dev_dbg(isp_vdev->dev, "call vb2_buffer_done(%d)\n", state);
>> +}
>> +
>> +static int isp4vid_vdev_link_validate(struct media_link *link)
>> +{
>> + return 0;
>> +}
>> +
>> +static const struct media_entity_operations isp4vid_vdev_ent_ops = {
>> + .link_validate = isp4vid_vdev_link_validate,
>> +};
>> +
>> +static const struct v4l2_file_operations isp4vid_vdev_fops = {
>> + .owner = THIS_MODULE,
>> + .open = v4l2_fh_open,
>> + .release = vb2_fop_release,
>> + .read = vb2_fop_read,
>> + .poll = vb2_fop_poll,
>> + .unlocked_ioctl = video_ioctl2,
>> + .mmap = vb2_fop_mmap,
>> +};
>> +
>> +static int isp4vid_ioctl_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
>> +{
>> + struct isp4vid_dev *isp_vdev = video_drvdata(file);
>> +
>> + strscpy(cap->driver, ISP4VID_ISP_DRV_NAME, sizeof(cap->driver));
>> + snprintf(cap->card, sizeof(cap->card), "%s", ISP4VID_ISP_DRV_NAME);
>> + cap->capabilities |= V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE;
>> +
>> + dev_dbg(isp_vdev->dev, "%s|capabilities=0x%X", isp_vdev->vdev.name, cap->capabilities);
>> +
>> + return 0;
>> +}
>> +
>> +static int isp4vid_g_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f)
>> +{
>> + struct isp4vid_dev *isp_vdev = video_drvdata(file);
>> +
>> + f->fmt.pix = isp_vdev->format;
>> +
>> + return 0;
>> +}
>> +
>> +static int isp4vid_try_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f)
>> +{
>> + struct isp4vid_dev *isp_vdev = video_drvdata(file);
>> + struct v4l2_pix_format *format = &f->fmt.pix;
>> + unsigned int i;
>
> Change to `int i;` for consistency.
>
Sure, will change it.
>> +
>> + if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
>> + return -EINVAL;
>> +
>> + /*
>> + * Check if the hardware supports the requested format, use the default
>> + * format otherwise.
>> + */
>> + for (i = 0; i < ARRAY_SIZE(isp4vid_formats); i++)
>> + if (isp4vid_formats[i] == format->pixelformat)
>> + break;
>> +
>> + if (i == ARRAY_SIZE(isp4vid_formats))
>> + format->pixelformat = ISP4VID_DEFAULT_FMT;
>> +
>> + switch (format->pixelformat) {
>> + case V4L2_PIX_FMT_NV12: {
>> + const struct v4l2_frmsize_discrete *fsz =
>> + v4l2_find_nearest_size(isp4vid_frmsize,
>> + ARRAY_SIZE(isp4vid_frmsize),
>> + width, height,
>> + format->width, format->height);
>> +
>> + format->width = fsz->width;
>> + format->height = fsz->height;
>> +
>> + format->bytesperline = format->width;
>> + format->sizeimage = format->bytesperline *
>> + format->height * 3 / 2;
>> + }
>> + break;
>> + case V4L2_PIX_FMT_YUYV: {
>> + const struct v4l2_frmsize_discrete *fsz =
>> + v4l2_find_nearest_size(isp4vid_frmsize,
>> + ARRAY_SIZE(isp4vid_frmsize),
>> + width, height,
>> + format->width, format->height);
>> +
>> + format->width = fsz->width;
>> + format->height = fsz->height;
>> +
>> + format->bytesperline = format->width * 2;
>> + format->sizeimage = format->bytesperline * format->height;
>> + }
>> + break;
>> + default:
>> + dev_err(isp_vdev->dev, "%s|unsupported fmt=%u",
>> + isp_vdev->vdev.name, format->pixelformat);
>> + return -EINVAL;
>> + }
>
> Create a variable declaration `const struct v4l2_frmsize_discrete *fsz;` at the
> top of the function and change everything starting from the switch to this:
>
> fsz = v4l2_find_nearest_size(isp4vid_frmsize,
> ARRAY_SIZE(isp4vid_frmsize), width, height,
> format->width, format->height);
> format->width = fsz->width;
> format->height = fsz->height;
> isp4vid_fill_buffer_size(format);
>
> And this will go with a complementary change to isp4vid_fill_buffer_size() to
> make it possible to reuse isp4vid_fill_buffer_size() here to remove code
> duplication.
>
Very good refinement, will do it.
>> +
>> + if (format->field == V4L2_FIELD_ANY)
>> + format->field = isp4vid_fmt_default.field;
>> +
>> + if (format->colorspace == V4L2_COLORSPACE_DEFAULT)
>> + format->colorspace = isp4vid_fmt_default.colorspace;
>> +
>> + return 0;
>> +}
>
> [snip]
>
>> +static int isp4vid_fill_buffer_size(struct isp4vid_dev *isp_vdev)
>> +{
>> + int ret = 0;
>> +
>> + switch (isp_vdev->format.pixelformat) {
>> + case V4L2_PIX_FMT_NV12:
>> + isp_vdev->format.bytesperline = isp_vdev->format.width;
>> + isp_vdev->format.sizeimage = isp_vdev->format.bytesperline *
>> + isp_vdev->format.height * 3 / 2;
>> + break;
>> + case V4L2_PIX_FMT_YUYV:
>> + isp_vdev->format.bytesperline = isp_vdev->format.width;
>> + isp_vdev->format.sizeimage = isp_vdev->format.bytesperline *
>> + isp_vdev->format.height * 2;
>> + break;
>> + default:
>> + dev_err(isp_vdev->dev, "fail for invalid default format 0x%x",
>> + isp_vdev->format.pixelformat);
>> + ret = -EINVAL;
>> + break;
>> + }
>> +
>> + return ret;
>> +}
>
> Looks like isp_vdev->format.bytesperline is set wrong here for YUYV, it should
> be width * 2.
>
Thanks for catching this.
> Move isp4vid_fill_buffer_size() definition above isp4vid_try_fmt_vid_cap() so it
> can be used there and change isp4vid_fill_buffer_size() to this:
>
> static int isp4vid_fill_buffer_size(struct v4l2_pix_format *fmt)
> {
> switch (fmt->pixelformat) {
> case V4L2_PIX_FMT_NV12: {
> fmt->bytesperline = fmt->width;
> fmt->sizeimage = fmt->bytesperline * fmt->height * 3 / 2;
> break;
> case V4L2_PIX_FMT_YUYV:
> fmt->bytesperline = fmt->width * 2;
> fmt->sizeimage = fmt->bytesperline * fmt->height;
> break;
> default:
> return -EINVAL;
> }
>
> return 0;
> }
>
> This fixes the isp_vdev->format.bytesperline issue too.
>
Yes, will modify in the next version.
>> +
>> +static const struct vb2_ops isp4vid_qops = {
>> + .queue_setup = isp4vid_qops_queue_setup,
>> + .buf_queue = isp4vid_qops_buffer_queue,
>> + .start_streaming = isp4vid_qops_start_streaming,
>> + .stop_streaming = isp4vid_qops_stop_streaming,
>> + .wait_prepare = vb2_ops_wait_prepare,
>> + .wait_finish = vb2_ops_wait_finish,
>> +};
>> +
>> +int isp4vid_dev_init(struct isp4vid_dev *isp_vdev, struct v4l2_subdev *isp_sdev,
>> + const struct isp4vid_ops *ops)
>> +{
>> + const char *vdev_name = isp4vid_video_dev_name;
>> + struct v4l2_device *v4l2_dev;
>> + struct video_device *vdev;
>> + struct vb2_queue *q;
>> + int ret;
>> +
>> + if (!isp_vdev || !isp_sdev || !isp_sdev->v4l2_dev)
>> + return -EINVAL;
>> +
>> + v4l2_dev = isp_sdev->v4l2_dev;
>> + vdev = &isp_vdev->vdev;
>> +
>> + isp_vdev->isp_sdev = isp_sdev;
>> + isp_vdev->dev = v4l2_dev->dev;
>> + isp_vdev->ops = ops;
>> +
>> + /* Initialize the vb2_queue struct */
>> + mutex_init(&isp_vdev->vbq_lock);
>> + q = &isp_vdev->vbq;
>> + q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>> + q->io_modes = VB2_MMAP | VB2_DMABUF;
>> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
>> + q->buf_struct_size = sizeof(struct isp4vid_capture_buffer);
>> + q->min_queued_buffers = 2;
>> + q->ops = &isp4vid_qops;
>> + q->drv_priv = isp_vdev;
>> + q->mem_ops = &isp4vid_vb2_memops;
>> + q->lock = &isp_vdev->vbq_lock;
>> + q->dev = v4l2_dev->dev;
>> + ret = vb2_queue_init(q);
>> + if (ret) {
>> + dev_err(v4l2_dev->dev, "vb2_queue_init error:%d", ret);
>> + return ret;
>> + }
>> + /* Initialize buffer list and its lock */
>> + mutex_init(&isp_vdev->buf_list_lock);
>> + INIT_LIST_HEAD(&isp_vdev->buf_list);
>> +
>> + /* Set default frame format */
>> + isp_vdev->format = isp4vid_fmt_default;
>> + isp_vdev->timeperframe = ISP4VID_ISP_TPF_DEFAULT;
>> + v4l2_simplify_fraction(&isp_vdev->timeperframe.numerator,
>> + &isp_vdev->timeperframe.denominator, 8, 333);
>> +
>> + ret = isp4vid_fill_buffer_size(isp_vdev);
>
> Change to `ret = isp4vid_fill_buffer_size(&isp_vdev->format);`
>
Yes, will modify together with above changes.
>> + if (ret) {
>> + dev_err(v4l2_dev->dev, "fail to fill buffer size: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + ret = isp4vid_set_fmt_2_isp(isp_sdev, &isp_vdev->format);
>> + if (ret) {
>> + dev_err(v4l2_dev->dev, "fail init format :%d\n", ret);
>> + return ret;
>> + }
>> +
>> + /* Initialize the video_device struct */
>> + isp_vdev->vdev.entity.name = vdev_name;
>> + isp_vdev->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
>> + isp_vdev->vdev_pad.flags = MEDIA_PAD_FL_SINK;
>> + ret = media_entity_pads_init(&isp_vdev->vdev.entity, 1,
>> + &isp_vdev->vdev_pad);
>> +
>> + if (ret) {
>> + dev_err(v4l2_dev->dev, "init media entity pad fail:%d\n", ret);
>> + return ret;
>> + }
>> +
>> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE |
>> + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
>> + vdev->entity.ops = &isp4vid_vdev_ent_ops;
>> + vdev->release = video_device_release_empty;
>> + vdev->fops = &isp4vid_vdev_fops;
>> + vdev->ioctl_ops = &isp4vid_vdev_ioctl_ops;
>> + vdev->lock = NULL;
>> + vdev->queue = q;
>> + vdev->v4l2_dev = v4l2_dev;
>> + vdev->vfl_dir = VFL_DIR_RX;
>> + strscpy(vdev->name, vdev_name, sizeof(vdev->name));
>> + video_set_drvdata(vdev, isp_vdev);
>> +
>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>> + if (ret)
>> + dev_err(v4l2_dev->dev, "register video device fail:%d\n", ret);
>
> No error handling?
>
Not necessary here because the caller isp4sd_init() will handle this.
>> +
>> + return ret;
>> +}
>> +
>> +void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev)
>> +{
>> + video_unregister_device(&isp_vdev->vdev);
>> +}
>> diff --git a/drivers/media/platform/amd/isp4/isp4_video.h b/drivers/media/platform/amd/isp4/isp4_video.h
>> new file mode 100644
>> index 000000000000..fae7dbdedd18
>> --- /dev/null
>> +++ b/drivers/media/platform/amd/isp4/isp4_video.h
>
> [snip]
>
>> +
>> +struct isp4vid_capture_buffer {
>> + /*
>> + * struct vb2_v4l2_buffer must be the first element
>> + * the videobuf2 framework will allocate this struct based on
>> + * buf_struct_size and use the first sizeof(struct vb2_buffer) bytes of
>> + * memory as a vb2_buffer
>> + */
>> + struct vb2_v4l2_buffer vb2;
>> + struct isp4if_img_buf_info img_buf;
>> + struct list_head list;
>> +};
>> +
>> +struct isp4vid_dev;
>
> Unnecessary isp4vid_dev forward declaration, remove it.
>
yes, will remove it.
>> +
>> +struct isp4vid_ops {
>> + int (*send_buffer)(struct v4l2_subdev *sd,
>> + struct isp4if_img_buf_info *img_buf);
>> +};
>> +
>> +struct isp4vid_dev {
>> + struct video_device vdev;
>> + struct media_pad vdev_pad;
>> + struct v4l2_pix_format format;
>> +
>> + /* mutex that protects vbq */
>> + struct mutex vbq_lock;
>> + struct vb2_queue vbq;
>> +
>> + /* mutex that protects buf_list */
>> + struct mutex buf_list_lock;
>> + struct list_head buf_list;
>> +
>> + u32 sequence;
>> + bool stream_started;
>> + struct task_struct *kthread;
>
> Remove unused `kthread` struct member.
>
Yes, not used, will remove it.
>> +
>> + struct media_pipeline pipe;
>> + struct device *dev;
>> + struct v4l2_subdev *isp_sdev;
>> + struct v4l2_fract timeperframe;
>> +
>> + /* Callback operations */
>> + const struct isp4vid_ops *ops;
>> +};
>> +
>> +int isp4vid_dev_init(struct isp4vid_dev *isp_vdev,
>> + struct v4l2_subdev *isp_sdev,
>> + const struct isp4vid_ops *ops);
>> +
>> +void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev);
>> +
>> +s32 isp4vid_notify(void *cb_ctx, struct isp4vid_framedone_param *evt_param);
>> +
>> +#endif
>> --
>> 2.34.1
>>
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/media/media-device.h?h=v6.17-rc7#n204
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/media/media-device.h?h=v6.17-rc7#n289
>
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-10-11 9:30 ` Du, Bin
@ 2025-10-12 6:08 ` Sultan Alsawaf
2025-10-13 9:55 ` Du, Bin
2025-10-16 8:13 ` Du, Bin
1 sibling, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-10-12 6:08 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Svetoslav Stoilov, Alexey Zagorodnikov
On Sat, Oct 11, 2025 at 05:30:43PM +0800, Du, Bin wrote:
> On 10/1/2025 2:53 PM, Sultan Alsawaf wrote:
> > On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
> > > +
> > > +/* Sizes must be in increasing order */
> > > +static const struct v4l2_frmsize_discrete isp4vid_frmsize[] = {
> > > + {640, 360},
> > > + {640, 480},
> > > + {1280, 720},
> > > + {1280, 960},
> > > + {1920, 1080},
> > > + {1920, 1440},
> > > + {2560, 1440},
> > > + {2880, 1620},
> > > + {2880, 1624},
> > > + {2888, 1808},
> > > +};
> > > +
> > > +static const u32 isp4vid_formats[] = {
> > > + V4L2_PIX_FMT_NV12,
> > > + V4L2_PIX_FMT_YUYV
> > > +};
> > > +
> > > +/* timeperframe list */
> > > +static const struct v4l2_fract isp4vid_tpfs[] = {
> > > + {.numerator = 1, .denominator = ISP4VID_MAX_PREVIEW_FPS}
> >
> > Add a space after { and a space before }.
> >
> > Also, it is more common to see v4l2_fract initialized without specifying the
> > struct member names.
> >
> > To summarize, change to `{ 1, ISP4VID_MAX_PREVIEW_FPS }`
> >
>
> Will follow your advice. Seems no explicit description about this in "Linux
> kernel coding style" and both spacing options after { are common in the
> current kernel code.
True, it's not explicitly stated in "Linux kernel coding style", but it _is_
specified in the .clang-format [1][2] via `Cpp11BracedListStyle: false`. And in
my experience, it's much more common to see spaces padded inside braced lists.
> > > +};
> > > +
> > > +static void isp4vid_handle_frame_done(struct isp4vid_dev *isp_vdev,
> > > + const struct isp4if_img_buf_info *img_buf,
> > > + bool done_suc)
> > > +{
> > > + struct isp4vid_capture_buffer *isp4vid_buf;
> >
> > Rename isp4vid_buf to isp_buf like in isp4vid_qops_start_streaming().
> >
>
> Seems isp4vid_buf appears to be more descriptive, plan to rename isp_buf in
> isp4vid_qops_start_streaming to isp4vid_buf, what do you think?
Either way is fine so long as it is consistent. It's just my own personal
preference to use shorter variable names, especially for pointers, which is why
I suggested isp_buf. :)
> > > +
> > > + buf->dev = dev;
> > > + buf->dbuf = dbuf;
> > > + buf->size = size;
> > > +
> > > + dev_dbg(dev, "attach dmabuf of isp user bo 0x%llx size %ld",
> > > + dbg_buf->gpu_addr, dbg_buf->size);
> > > +
> > > + return buf;
> > > +}
> > > +
> > > +static void isp4vid_vb2_unmap_dmabuf(void *mem_priv)
> > > +{
> > > + struct isp4vid_vb2_buf *buf = mem_priv;
> > > + struct iosys_map map = IOSYS_MAP_INIT_VADDR(buf->vaddr);
> > > +
> > > + dev_dbg(buf->dev, "unmap dmabuf of isp user bo 0x%llx size %ld",
> > > + buf->gpu_addr, buf->size);
> > > +
> > > + dma_buf_vunmap_unlocked(buf->dbuf, &map);
> > > + buf->vaddr = NULL;
> > > +}
> > > +
> > > +static int isp4vid_vb2_map_dmabuf(void *mem_priv)
> > > +{
> > > + struct isp4vid_vb2_buf *mmap_buf = NULL;
> >
> > Remove unnecessary initialization of `mmap_buf`, and combine it onto one line
> > with `buf`:
> >
>
> Sure, will remove unnecessary initialization of `mmap_buf, but i'd rather
> not combine because it is mentioned in "Linux kernel coding style" that "to
> this end, use just one data declaration per line (no commas for multiple
> data declarations). This leaves you room for a small comment on each item,
> explaining its use.", what do you think?
Huh, that quote is odd, as it's quite common in the kernel to declare multiple
local variables of the same type on one line. In fact, Linus has done this
himself [3][4].
I think it's better to put `mmap_buf` on the same line because the type name is
quite long, so declaring `buf` and `mmap_buf` on the same line makes it easier
to see that they are exactly the same type.
Also, creating a new line for every local variable declaration would really
bloat a lot of code around the kernel and hurt readability. That quote from
"Linux kernel coding style" was added almost *20 years* ago (commit b3fc9941fbc6
"[PATCH] CodingStyle updates"), so maybe it is just obsolete.
> > > + if (ret) {
> > > + dev_err(v4l2_dev->dev, "fail to fill buffer size: %d\n", ret);
> > > + return ret;
> > > + }
> > > +
> > > + ret = isp4vid_set_fmt_2_isp(isp_sdev, &isp_vdev->format);
> > > + if (ret) {
> > > + dev_err(v4l2_dev->dev, "fail init format :%d\n", ret);
> > > + return ret;
> > > + }
> > > +
> > > + /* Initialize the video_device struct */
> > > + isp_vdev->vdev.entity.name = vdev_name;
> > > + isp_vdev->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
> > > + isp_vdev->vdev_pad.flags = MEDIA_PAD_FL_SINK;
> > > + ret = media_entity_pads_init(&isp_vdev->vdev.entity, 1,
> > > + &isp_vdev->vdev_pad);
> > > +
> > > + if (ret) {
> > > + dev_err(v4l2_dev->dev, "init media entity pad fail:%d\n", ret);
> > > + return ret;
> > > + }
> > > +
> > > + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE |
> > > + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> > > + vdev->entity.ops = &isp4vid_vdev_ent_ops;
> > > + vdev->release = video_device_release_empty;
> > > + vdev->fops = &isp4vid_vdev_fops;
> > > + vdev->ioctl_ops = &isp4vid_vdev_ioctl_ops;
> > > + vdev->lock = NULL;
> > > + vdev->queue = q;
> > > + vdev->v4l2_dev = v4l2_dev;
> > > + vdev->vfl_dir = VFL_DIR_RX;
> > > + strscpy(vdev->name, vdev_name, sizeof(vdev->name));
> > > + video_set_drvdata(vdev, isp_vdev);
> > > +
> > > + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> > > + if (ret)
> > > + dev_err(v4l2_dev->dev, "register video device fail:%d\n", ret);
> >
> > No error handling?
> >
>
> Not necessary here because the caller isp4sd_init() will handle this.
This doesn't seem to be the case; in fact, isp4sd_init() doesn't seem to handle
any of the error cleanup for isp4vid_dev_init().
I started looking deeper at it and saw that vb2_queue_release() is never called
to clean up after vb2_queue_init(). See my next comment below about changing
video_unregister_device() to vb2_video_unregister_device(), which calls
vb2_queue_release().
And isp4sd_init() calls media_entity_cleanup() for the subdev, not the vdev.
So you need to add `media_entity_cleanup(&isp_vdev->vdev.entity)`.
To summarize, make the following changes to isp4vid_dev_init():
1. On error starting from isp4vid_fill_buffer_size(), call vb2_queue_release()
to do cleanup for vb2_queue_init().
2. When video_register_device() fails, do cleanup for media_entity_pads_init()
by adding `media_entity_cleanup(&isp_vdev->vdev.entity)`.
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev)
> > > +{
> > > + video_unregister_device(&isp_vdev->vdev);
> > > +}
I just noticed: isp4vid_dev_deinit() should call vb2_video_unregister_device()
instead of video_unregister_device().
See the comment in include/media/videobuf2-v4l2.h [5]:
* If the driver uses vb2_fop_release()/_vb2_fop_release(), then it should use
* vb2_video_unregister_device() instead of video_unregister_device().
Since vb2_fop_release() is used, vb2_video_unregister_device() should be used.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/.clang-format?h=v6.17#n61
[2] https://clang.llvm.org/docs/ClangFormatStyleOptions.html#cpp11bracedliststyle
[3] https://github.com/torvalds/linux/commit/fe673d3f5bf1fc50cdc4b754831db91a2ec10126#diff-b7746cf0f2ab471c30d08fe3391b7d3ba45adbe7616e4fae0504b29f40b49dd5L1747-R1747
[4] https://github.com/torvalds/linux/commit/d7fe75cbc23c7d225eee2ef04def239b6603dce7#diff-b8c8d3c5770869f743e155faac7cccc97082918c3e092ffdf592efa796725c79L2019-R2019
[5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/media/videobuf2-v4l2.h?h=v6.17#n360
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-10-12 6:08 ` Sultan Alsawaf
@ 2025-10-13 9:55 ` Du, Bin
0 siblings, 0 replies; 35+ messages in thread
From: Du, Bin @ 2025-10-13 9:55 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Svetoslav Stoilov, Alexey Zagorodnikov
Thanks Sultan.
On 10/12/2025 2:08 PM, Sultan Alsawaf wrote:
> On Sat, Oct 11, 2025 at 05:30:43PM +0800, Du, Bin wrote:
>> On 10/1/2025 2:53 PM, Sultan Alsawaf wrote:
>>> On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
>>>> +
>>>> +/* Sizes must be in increasing order */
>>>> +static const struct v4l2_frmsize_discrete isp4vid_frmsize[] = {
>>>> + {640, 360},
>>>> + {640, 480},
>>>> + {1280, 720},
>>>> + {1280, 960},
>>>> + {1920, 1080},
>>>> + {1920, 1440},
>>>> + {2560, 1440},
>>>> + {2880, 1620},
>>>> + {2880, 1624},
>>>> + {2888, 1808},
>>>> +};
>>>> +
>>>> +static const u32 isp4vid_formats[] = {
>>>> + V4L2_PIX_FMT_NV12,
>>>> + V4L2_PIX_FMT_YUYV
>>>> +};
>>>> +
>>>> +/* timeperframe list */
>>>> +static const struct v4l2_fract isp4vid_tpfs[] = {
>>>> + {.numerator = 1, .denominator = ISP4VID_MAX_PREVIEW_FPS}
>>>
>>> Add a space after { and a space before }.
>>>
>>> Also, it is more common to see v4l2_fract initialized without specifying the
>>> struct member names.
>>>
>>> To summarize, change to `{ 1, ISP4VID_MAX_PREVIEW_FPS }`
>>>
>>
>> Will follow your advice. Seems no explicit description about this in "Linux
>> kernel coding style" and both spacing options after { are common in the
>> current kernel code.
>
> True, it's not explicitly stated in "Linux kernel coding style", but it _is_
> specified in the .clang-format [1][2] via `Cpp11BracedListStyle: false`. And in
> my experience, it's much more common to see spaces padded inside braced lists.
>
Thanks for the reference, yes, will add space pad.
>>>> +};
>>>> +
>>>> +static void isp4vid_handle_frame_done(struct isp4vid_dev *isp_vdev,
>>>> + const struct isp4if_img_buf_info *img_buf,
>>>> + bool done_suc)
>>>> +{
>>>> + struct isp4vid_capture_buffer *isp4vid_buf;
>>>
>>> Rename isp4vid_buf to isp_buf like in isp4vid_qops_start_streaming().
>>>
>>
>> Seems isp4vid_buf appears to be more descriptive, plan to rename isp_buf in
>> isp4vid_qops_start_streaming to isp4vid_buf, what do you think?
>
> Either way is fine so long as it is consistent. It's just my own personal
> preference to use shorter variable names, especially for pointers, which is why
> I suggested isp_buf. :)
>
Yes, will make them consistent.
>>>> +
>>>> + buf->dev = dev;
>>>> + buf->dbuf = dbuf;
>>>> + buf->size = size;
>>>> +
>>>> + dev_dbg(dev, "attach dmabuf of isp user bo 0x%llx size %ld",
>>>> + dbg_buf->gpu_addr, dbg_buf->size);
>>>> +
>>>> + return buf;
>>>> +}
>>>> +
>>>> +static void isp4vid_vb2_unmap_dmabuf(void *mem_priv)
>>>> +{
>>>> + struct isp4vid_vb2_buf *buf = mem_priv;
>>>> + struct iosys_map map = IOSYS_MAP_INIT_VADDR(buf->vaddr);
>>>> +
>>>> + dev_dbg(buf->dev, "unmap dmabuf of isp user bo 0x%llx size %ld",
>>>> + buf->gpu_addr, buf->size);
>>>> +
>>>> + dma_buf_vunmap_unlocked(buf->dbuf, &map);
>>>> + buf->vaddr = NULL;
>>>> +}
>>>> +
>>>> +static int isp4vid_vb2_map_dmabuf(void *mem_priv)
>>>> +{
>>>> + struct isp4vid_vb2_buf *mmap_buf = NULL;
>>>
>>> Remove unnecessary initialization of `mmap_buf`, and combine it onto one line
>>> with `buf`:
>>>
>>
>> Sure, will remove unnecessary initialization of `mmap_buf, but i'd rather
>> not combine because it is mentioned in "Linux kernel coding style" that "to
>> this end, use just one data declaration per line (no commas for multiple
>> data declarations). This leaves you room for a small comment on each item,
>> explaining its use.", what do you think?
>
> Huh, that quote is odd, as it's quite common in the kernel to declare multiple
> local variables of the same type on one line. In fact, Linus has done this
> himself [3][4].
>
> I think it's better to put `mmap_buf` on the same line because the type name is
> quite long, so declaring `buf` and `mmap_buf` on the same line makes it easier
> to see that they are exactly the same type.
>
> Also, creating a new line for every local variable declaration would really
> bloat a lot of code around the kernel and hurt readability. That quote from
> "Linux kernel coding style" was added almost *20 years* ago (commit b3fc9941fbc6
> "[PATCH] CodingStyle updates"), so maybe it is just obsolete.
>
Will combine them, thanks for the clear explanation.
>>>> + if (ret) {
>>>> + dev_err(v4l2_dev->dev, "fail to fill buffer size: %d\n", ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + ret = isp4vid_set_fmt_2_isp(isp_sdev, &isp_vdev->format);
>>>> + if (ret) {
>>>> + dev_err(v4l2_dev->dev, "fail init format :%d\n", ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + /* Initialize the video_device struct */
>>>> + isp_vdev->vdev.entity.name = vdev_name;
>>>> + isp_vdev->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
>>>> + isp_vdev->vdev_pad.flags = MEDIA_PAD_FL_SINK;
>>>> + ret = media_entity_pads_init(&isp_vdev->vdev.entity, 1,
>>>> + &isp_vdev->vdev_pad);
>>>> +
>>>> + if (ret) {
>>>> + dev_err(v4l2_dev->dev, "init media entity pad fail:%d\n", ret);
>>>> + return ret;
>>>> + }
>>>> +
>>>> + vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE |
>>>> + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
>>>> + vdev->entity.ops = &isp4vid_vdev_ent_ops;
>>>> + vdev->release = video_device_release_empty;
>>>> + vdev->fops = &isp4vid_vdev_fops;
>>>> + vdev->ioctl_ops = &isp4vid_vdev_ioctl_ops;
>>>> + vdev->lock = NULL;
>>>> + vdev->queue = q;
>>>> + vdev->v4l2_dev = v4l2_dev;
>>>> + vdev->vfl_dir = VFL_DIR_RX;
>>>> + strscpy(vdev->name, vdev_name, sizeof(vdev->name));
>>>> + video_set_drvdata(vdev, isp_vdev);
>>>> +
>>>> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
>>>> + if (ret)
>>>> + dev_err(v4l2_dev->dev, "register video device fail:%d\n", ret);
>>>
>>> No error handling?
>>>
>>
>> Not necessary here because the caller isp4sd_init() will handle this.
>
> This doesn't seem to be the case; in fact, isp4sd_init() doesn't seem to handle
> any of the error cleanup for isp4vid_dev_init().
>
> I started looking deeper at it and saw that vb2_queue_release() is never called
> to clean up after vb2_queue_init(). See my next comment below about changing
> video_unregister_device() to vb2_video_unregister_device(), which calls
> vb2_queue_release().
>
> And isp4sd_init() calls media_entity_cleanup() for the subdev, not the vdev.
> So you need to add `media_entity_cleanup(&isp_vdev->vdev.entity)`.
>
> To summarize, make the following changes to isp4vid_dev_init():
>
> 1. On error starting from isp4vid_fill_buffer_size(), call vb2_queue_release()
> to do cleanup for vb2_queue_init().
>
> 2. When video_register_device() fails, do cleanup for media_entity_pads_init()
> by adding `media_entity_cleanup(&isp_vdev->vdev.entity)`.
>
Thank you for identifying the missed vb2 queue release and providing the
modification guide, will follow it.
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +void isp4vid_dev_deinit(struct isp4vid_dev *isp_vdev)
>>>> +{
>>>> + video_unregister_device(&isp_vdev->vdev);
>>>> +}
>
> I just noticed: isp4vid_dev_deinit() should call vb2_video_unregister_device()
> instead of video_unregister_device().
>
> See the comment in include/media/videobuf2-v4l2.h [5]:
>
> * If the driver uses vb2_fop_release()/_vb2_fop_release(), then it should use
> * vb2_video_unregister_device() instead of video_unregister_device().
>
> Since vb2_fop_release() is used, vb2_video_unregister_device() should be used.
>
Yes, vb2_video_unregister_device() should be used in this case, will
follow your advice.
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/.clang-format?h=v6.17#n61
> [2] https://clang.llvm.org/docs/ClangFormatStyleOptions.html#cpp11bracedliststyle
> [3] https://github.com/torvalds/linux/commit/fe673d3f5bf1fc50cdc4b754831db91a2ec10126#diff-b7746cf0f2ab471c30d08fe3391b7d3ba45adbe7616e4fae0504b29f40b49dd5L1747-R1747
> [4] https://github.com/torvalds/linux/commit/d7fe75cbc23c7d225eee2ef04def239b6603dce7#diff-b8c8d3c5770869f743e155faac7cccc97082918c3e092ffdf592efa796725c79L2019-R2019
> [5] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/media/videobuf2-v4l2.h?h=v6.17#n360
>
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-10-11 9:30 ` Du, Bin
2025-10-12 6:08 ` Sultan Alsawaf
@ 2025-10-16 8:13 ` Du, Bin
2025-10-17 8:34 ` Sultan Alsawaf
1 sibling, 1 reply; 35+ messages in thread
From: Du, Bin @ 2025-10-16 8:13 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Svetoslav Stoilov, Alexey Zagorodnikov
Hi Sultan,
On 10/11/2025 5:30 PM, Du, Bin wrote:
> Many thanks Sultan for your review, that's really helpful.
>
> On 10/1/2025 2:53 PM, Sultan Alsawaf wrote:
>> Hi Bin,
>>
>> On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
>>> Isp video implements v4l2 video interface and supports NV12 and YUYV. It
>>> manages buffers, pipeline power and state. Cherry-picked Sultan's DMA
>>> buffer related fix from branch v6.16-drm-tip-isp4-for-amd on
>>> https://github.com/kerneltoast/kernel_x86_laptop.git
>>>
>>> Co-developed-by: Sultan Alsawaf <sultan@kerneltoast.com>
>>> Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>
>>> Co-developed-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>>> Signed-off-by: Svetoslav Stoilov <Svetoslav.Stoilov@amd.com>
>>> Signed-off-by: Bin Du <Bin.Du@amd.com>
>>> Tested-by: Alexey Zagorodnikov <xglooom@gmail.com>
>>
>> [snip]
>>
>>> +++ b/drivers/media/platform/amd/isp4/isp4.c
>>> @@ -178,6 +178,16 @@ static int isp4_capture_probe(struct
>>> platform_device *pdev)
>>> goto err_isp4_deinit;
>>> }
>>> + ret = media_create_pad_link(&isp_dev->isp_sdev.sdev.entity,
>>> + 0, &isp_dev->isp_sdev.isp_vdev.vdev.entity,
>>> + 0,
>>> + MEDIA_LNK_FL_ENABLED |
>>> + MEDIA_LNK_FL_IMMUTABLE);
>>> + if (ret) {
>>> + dev_err(dev, "fail to create pad link %d\n", ret);
>>> + goto err_isp4_deinit;
>>> + }
>>> +
>>
>> Two problems with this hunk:
>>
>> 1. According to the comment in include/media/media-device.h [1],
>> media_create_pad_link() should be called before
>> media_device_register():
>>
>> * So drivers need to first initialize the media device, register
>> any entity
>> * within the media device, create pad to pad links and then
>> finally register
>> * the media device by calling media_device_register() as a final
>> step.
>>
>> 2. Missing call to media_device_unregister() on error when
>> media_create_pad_link() fails.
>>
>> Since the media_create_pad_link() will be moved before
>> media_device_register(),
>> we will need to clean up media_create_pad_link() when
>> media_device_register()
>> fails.
>>
>> The clean-up function for media_create_pad_link() is
>> media_device_unregister().
>> According to the comment for media_device_unregister() [2], it is safe
>> to call
>> media_device_unregister() on an unregistered media device that is
>> initialized
>> (through media_device_init()).
>>
>> In addition, this made me realize that there's no call to
>> media_device_cleanup()
>> in the entire driver too. This is the cleanup function for
>> media_device_init(),
>> so it should be called on error and on module unload.
>>
>> To summarize, make the following changes:
>>
>> 1. Move the media_create_pad_link() up, right before
>> media_device_register().
>>
>> 2. When media_device_register() fails, clean up
>> media_create_pad_link() by
>> calling media_device_unregister().
>>
>> 3. Add a missing call to media_device_cleanup() on error and module
>> unload to
>> clean up media_device_init().
>>
>
> Very clear guide, will follow your advice.
>
>>> platform_set_drvdata(pdev, isp_dev);
>>> return 0;
For 2, we found when media_device_register() fails, calling
media_device_unregister() won't clean up media_create_pad_link() because
it simply returns without doing anything(see
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/mc/mc-device.c?h=v6.17-rc7#n797).
Therefore like other kernel drivers, we'd rather not call
media_device_unregister() in this scenario, it doesn't cause issues, but
it's not logically correct. Cleanup for media_create_pad_link() occurs
during error handling via
isp4sd_deinit()->isp4vid_dev_deinit()->vb2_video_unregister_device()->...->media_entity_remove_link().
What do you think?
[snip]
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-10-16 8:13 ` Du, Bin
@ 2025-10-17 8:34 ` Sultan Alsawaf
2025-10-17 9:53 ` Du, Bin
0 siblings, 1 reply; 35+ messages in thread
From: Sultan Alsawaf @ 2025-10-17 8:34 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On Thu, Oct 16, 2025 at 04:13:47PM +0800, Du, Bin wrote:
> On 10/11/2025 5:30 PM, Du, Bin wrote:
> > On 10/1/2025 2:53 PM, Sultan Alsawaf wrote:
> > > On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
> > > > +++ b/drivers/media/platform/amd/isp4/isp4.c
> > > > @@ -178,6 +178,16 @@ static int isp4_capture_probe(struct
> > > > platform_device *pdev)
> > > > goto err_isp4_deinit;
> > > > }
> > > > + ret = media_create_pad_link(&isp_dev->isp_sdev.sdev.entity,
> > > > + 0, &isp_dev->isp_sdev.isp_vdev.vdev.entity,
> > > > + 0,
> > > > + MEDIA_LNK_FL_ENABLED |
> > > > + MEDIA_LNK_FL_IMMUTABLE);
> > > > + if (ret) {
> > > > + dev_err(dev, "fail to create pad link %d\n", ret);
> > > > + goto err_isp4_deinit;
> > > > + }
> > > > +
> > >
> > > Two problems with this hunk:
> > >
> > > 1. According to the comment in include/media/media-device.h [1],
> > > media_create_pad_link() should be called before
> > > media_device_register():
> > >
> > > * So drivers need to first initialize the media device,
> > > register any entity
> > > * within the media device, create pad to pad links and then
> > > finally register
> > > * the media device by calling media_device_register() as a
> > > final step.
> > >
> > > 2. Missing call to media_device_unregister() on error when
> > > media_create_pad_link() fails.
> > >
> > > Since the media_create_pad_link() will be moved before
> > > media_device_register(),
> > > we will need to clean up media_create_pad_link() when
> > > media_device_register()
> > > fails.
> > >
> > > The clean-up function for media_create_pad_link() is
> > > media_device_unregister().
> > > According to the comment for media_device_unregister() [2], it is
> > > safe to call
> > > media_device_unregister() on an unregistered media device that is
> > > initialized
> > > (through media_device_init()).
> > >
> > > In addition, this made me realize that there's no call to
> > > media_device_cleanup()
> > > in the entire driver too. This is the cleanup function for
> > > media_device_init(),
> > > so it should be called on error and on module unload.
> > >
> > > To summarize, make the following changes:
> > >
> > > 1. Move the media_create_pad_link() up, right before
> > > media_device_register().
> > >
> > > 2. When media_device_register() fails, clean up
> > > media_create_pad_link() by
> > > calling media_device_unregister().
> > >
> > > 3. Add a missing call to media_device_cleanup() on error and module
> > > unload to
> > > clean up media_device_init().
> > >
> >
> > Very clear guide, will follow your advice.
> >
> > > > platform_set_drvdata(pdev, isp_dev);
> > > > return 0;
>
> For 2, we found when media_device_register() fails, calling
> media_device_unregister() won't clean up media_create_pad_link() because it
> simply returns without doing anything(see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/mc/mc-device.c?h=v6.17-rc7#n797).
> Therefore like other kernel drivers, we'd rather not call
> media_device_unregister() in this scenario, it doesn't cause issues, but
> it's not logically correct. Cleanup for media_create_pad_link() occurs
> during error handling via isp4sd_deinit()->isp4vid_dev_deinit()->vb2_video_unregister_device()->...->media_entity_remove_link().
> What do you think?
Oh, good catch! You are right about media_device_unregister() not cleaning up
media_create_pad_link().
But I don't see how vb2_video_unregister_device() ends up calling
media_entity_remove_links().
It looks like media_create_pad_link() is actually cleaned up via
v4l2_device_unregister_subdev()->media_device_unregister_entity()->__media_device_unregister_entity()->__media_entity_remove_links()
And I mentioned before to add a missing call to v4l2_device_unregister_subdev()
on error, so it looks like that will cover the media_create_pad_link() cleanup
and therefore you don't need to call media_device_unregister() in this scenario.
Does that look correct?
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-10-17 8:34 ` Sultan Alsawaf
@ 2025-10-17 9:53 ` Du, Bin
2025-10-19 22:11 ` Sultan Alsawaf
0 siblings, 1 reply; 35+ messages in thread
From: Du, Bin @ 2025-10-17 9:53 UTC (permalink / raw)
To: Sultan Alsawaf
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On 10/17/2025 4:34 PM, Sultan Alsawaf wrote:
> On Thu, Oct 16, 2025 at 04:13:47PM +0800, Du, Bin wrote:
>> On 10/11/2025 5:30 PM, Du, Bin wrote:
>>> On 10/1/2025 2:53 PM, Sultan Alsawaf wrote:
>>>> On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
>>>>> +++ b/drivers/media/platform/amd/isp4/isp4.c
>>>>> @@ -178,6 +178,16 @@ static int isp4_capture_probe(struct
>>>>> platform_device *pdev)
>>>>> goto err_isp4_deinit;
>>>>> }
>>>>> + ret = media_create_pad_link(&isp_dev->isp_sdev.sdev.entity,
>>>>> + 0, &isp_dev->isp_sdev.isp_vdev.vdev.entity,
>>>>> + 0,
>>>>> + MEDIA_LNK_FL_ENABLED |
>>>>> + MEDIA_LNK_FL_IMMUTABLE);
>>>>> + if (ret) {
>>>>> + dev_err(dev, "fail to create pad link %d\n", ret);
>>>>> + goto err_isp4_deinit;
>>>>> + }
>>>>> +
>>>>
>>>> Two problems with this hunk:
>>>>
>>>> 1. According to the comment in include/media/media-device.h [1],
>>>> media_create_pad_link() should be called before
>>>> media_device_register():
>>>>
>>>> * So drivers need to first initialize the media device,
>>>> register any entity
>>>> * within the media device, create pad to pad links and then
>>>> finally register
>>>> * the media device by calling media_device_register() as a
>>>> final step.
>>>>
>>>> 2. Missing call to media_device_unregister() on error when
>>>> media_create_pad_link() fails.
>>>>
>>>> Since the media_create_pad_link() will be moved before
>>>> media_device_register(),
>>>> we will need to clean up media_create_pad_link() when
>>>> media_device_register()
>>>> fails.
>>>>
>>>> The clean-up function for media_create_pad_link() is
>>>> media_device_unregister().
>>>> According to the comment for media_device_unregister() [2], it is
>>>> safe to call
>>>> media_device_unregister() on an unregistered media device that is
>>>> initialized
>>>> (through media_device_init()).
>>>>
>>>> In addition, this made me realize that there's no call to
>>>> media_device_cleanup()
>>>> in the entire driver too. This is the cleanup function for
>>>> media_device_init(),
>>>> so it should be called on error and on module unload.
>>>>
>>>> To summarize, make the following changes:
>>>>
>>>> 1. Move the media_create_pad_link() up, right before
>>>> media_device_register().
>>>>
>>>> 2. When media_device_register() fails, clean up
>>>> media_create_pad_link() by
>>>> calling media_device_unregister().
>>>>
>>>> 3. Add a missing call to media_device_cleanup() on error and module
>>>> unload to
>>>> clean up media_device_init().
>>>>
>>>
>>> Very clear guide, will follow your advice.
>>>
>>>>> platform_set_drvdata(pdev, isp_dev);
>>>>> return 0;
>>
>> For 2, we found when media_device_register() fails, calling
>> media_device_unregister() won't clean up media_create_pad_link() because it
>> simply returns without doing anything(see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/mc/mc-device.c?h=v6.17-rc7#n797).
>> Therefore like other kernel drivers, we'd rather not call
>> media_device_unregister() in this scenario, it doesn't cause issues, but
>> it's not logically correct. Cleanup for media_create_pad_link() occurs
>> during error handling via isp4sd_deinit()->isp4vid_dev_deinit()->vb2_video_unregister_device()->...->media_entity_remove_link().
>> What do you think?
>
> Oh, good catch! You are right about media_device_unregister() not cleaning up
> media_create_pad_link().
>
> But I don't see how vb2_video_unregister_device() ends up calling
> media_entity_remove_links().
>
> It looks like media_create_pad_link() is actually cleaned up via
> v4l2_device_unregister_subdev()->media_device_unregister_entity()->__media_device_unregister_entity()->__media_entity_remove_links()
>
> And I mentioned before to add a missing call to v4l2_device_unregister_subdev()
> on error, so it looks like that will cover the media_create_pad_link() cleanup
> and therefore you don't need to call media_device_unregister() in this scenario.
>
> Does that look correct?
>
Yes, Sultan, we moved v4l2_device_unregister_subdev to isp4sd_deinit as
you suggested, so current isp4sd_deinit() looks like this
void isp4sd_deinit(struct isp4_subdev *isp_subdev)
{
struct isp4_interface *ispif = &isp_subdev->ispif;
isp4vid_dev_deinit(&isp_subdev->isp_vdev);
v4l2_device_unregister_subdev(&isp_subdev->sdev);
media_entity_cleanup(&isp_subdev->sdev.entity);
isp4if_deinit(ispif);
isp4sd_module_enable(isp_subdev, false);
ispif->status = ISP4IF_STATUS_PWR_OFF;
}
You are correct and I believe both isp4vid_dev_deinit and
v4l2_device_unregister_subdev can cause media_create_pad_link() being
cleaned up. Because isp4vid_dev_deinit is called first, so the link will
be cleaned by it, here is the call stack FYI, does it make sense?
[ 5.198328] Call Trace:
[ 5.198329] <TASK>
[ 5.198331] dump_stack_lvl+0x76/0xa0
[ 5.198336] dump_stack+0x10/0x20
[ 5.198338] __media_entity_remove_link+0xdf/0x1f0 [mc]
[ 5.198342] __media_entity_remove_links+0x31/0x70 [mc]
[ 5.198344] __media_device_unregister_entity+0x93/0xf0 [mc]
[ 5.198346] media_device_unregister_entity+0x2f/0x50 [mc]
[ 5.198348] v4l2_device_release+0x112/0x190 [videodev]
[ 5.198355] device_release+0x38/0xa0
[ 5.198358] kobject_put+0x9e/0x200
[ 5.198359] put_device+0x13/0x30
[ 5.198359] vb2_video_unregister_device+0x8e/0xd0 [videobuf2_v4l2]
[ 5.198362] isp4vid_dev_deinit+0xe/0x20 [amd_capture]
[ 5.198364] isp4sd_deinit+0x25/0x80 [amd_capture]
[ 5.198366] isp4_capture_probe+0x1ec/0x2f0 [amd_capture]
[ 5.198368] platform_probe+0x3f/0xb0
[ 5.198370] really_probe+0xf4/0x3b0
> Sultan
--
Regards,
Bin
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers handling added
2025-10-17 9:53 ` Du, Bin
@ 2025-10-19 22:11 ` Sultan Alsawaf
0 siblings, 0 replies; 35+ messages in thread
From: Sultan Alsawaf @ 2025-10-19 22:11 UTC (permalink / raw)
To: Du, Bin
Cc: mchehab, hverkuil, laurent.pinchart+renesas, bryan.odonoghue,
sakari.ailus, prabhakar.mahadev-lad.rj, linux-media, linux-kernel,
pratap.nirujogi, benjamin.chan, king.li, gjorgji.rosikopulos,
Phil.Jawich, Dominic.Antony, mario.limonciello, richard.gong,
anson.tsao, Alexey Zagorodnikov
On Fri, Oct 17, 2025 at 05:53:27PM +0800, Du, Bin wrote:
> On 10/17/2025 4:34 PM, Sultan Alsawaf wrote:
> > On Thu, Oct 16, 2025 at 04:13:47PM +0800, Du, Bin wrote:
> > > On 10/11/2025 5:30 PM, Du, Bin wrote:
> > > > On 10/1/2025 2:53 PM, Sultan Alsawaf wrote:
> > > > > On Thu, Sep 11, 2025 at 06:08:45PM +0800, Bin Du wrote:
> > > > > > +++ b/drivers/media/platform/amd/isp4/isp4.c
> > > > > > @@ -178,6 +178,16 @@ static int isp4_capture_probe(struct
> > > > > > platform_device *pdev)
> > > > > > goto err_isp4_deinit;
> > > > > > }
> > > > > > + ret = media_create_pad_link(&isp_dev->isp_sdev.sdev.entity,
> > > > > > + 0, &isp_dev->isp_sdev.isp_vdev.vdev.entity,
> > > > > > + 0,
> > > > > > + MEDIA_LNK_FL_ENABLED |
> > > > > > + MEDIA_LNK_FL_IMMUTABLE);
> > > > > > + if (ret) {
> > > > > > + dev_err(dev, "fail to create pad link %d\n", ret);
> > > > > > + goto err_isp4_deinit;
> > > > > > + }
> > > > > > +
> > > > >
> > > > > Two problems with this hunk:
> > > > >
> > > > > 1. According to the comment in include/media/media-device.h [1],
> > > > > media_create_pad_link() should be called before
> > > > > media_device_register():
> > > > >
> > > > > * So drivers need to first initialize the media device,
> > > > > register any entity
> > > > > * within the media device, create pad to pad links and then
> > > > > finally register
> > > > > * the media device by calling media_device_register() as a
> > > > > final step.
> > > > >
> > > > > 2. Missing call to media_device_unregister() on error when
> > > > > media_create_pad_link() fails.
> > > > >
> > > > > Since the media_create_pad_link() will be moved before
> > > > > media_device_register(),
> > > > > we will need to clean up media_create_pad_link() when
> > > > > media_device_register()
> > > > > fails.
> > > > >
> > > > > The clean-up function for media_create_pad_link() is
> > > > > media_device_unregister().
> > > > > According to the comment for media_device_unregister() [2], it is
> > > > > safe to call
> > > > > media_device_unregister() on an unregistered media device that is
> > > > > initialized
> > > > > (through media_device_init()).
> > > > >
> > > > > In addition, this made me realize that there's no call to
> > > > > media_device_cleanup()
> > > > > in the entire driver too. This is the cleanup function for
> > > > > media_device_init(),
> > > > > so it should be called on error and on module unload.
> > > > >
> > > > > To summarize, make the following changes:
> > > > >
> > > > > 1. Move the media_create_pad_link() up, right before
> > > > > media_device_register().
> > > > >
> > > > > 2. When media_device_register() fails, clean up
> > > > > media_create_pad_link() by
> > > > > calling media_device_unregister().
> > > > >
> > > > > 3. Add a missing call to media_device_cleanup() on error and module
> > > > > unload to
> > > > > clean up media_device_init().
> > > > >
> > > >
> > > > Very clear guide, will follow your advice.
> > > >
> > > > > > platform_set_drvdata(pdev, isp_dev);
> > > > > > return 0;
> > >
> > > For 2, we found when media_device_register() fails, calling
> > > media_device_unregister() won't clean up media_create_pad_link() because it
> > > simply returns without doing anything(see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/mc/mc-device.c?h=v6.17-rc7#n797).
> > > Therefore like other kernel drivers, we'd rather not call
> > > media_device_unregister() in this scenario, it doesn't cause issues, but
> > > it's not logically correct. Cleanup for media_create_pad_link() occurs
> > > during error handling via isp4sd_deinit()->isp4vid_dev_deinit()->vb2_video_unregister_device()->...->media_entity_remove_link().
> > > What do you think?
> >
> > Oh, good catch! You are right about media_device_unregister() not cleaning up
> > media_create_pad_link().
> >
> > But I don't see how vb2_video_unregister_device() ends up calling
> > media_entity_remove_links().
> >
> > It looks like media_create_pad_link() is actually cleaned up via
> > v4l2_device_unregister_subdev()->media_device_unregister_entity()->__media_device_unregister_entity()->__media_entity_remove_links()
> >
> > And I mentioned before to add a missing call to v4l2_device_unregister_subdev()
> > on error, so it looks like that will cover the media_create_pad_link() cleanup
> > and therefore you don't need to call media_device_unregister() in this scenario.
> >
> > Does that look correct?
> >
>
> Yes, Sultan, we moved v4l2_device_unregister_subdev to isp4sd_deinit as you
> suggested, so current isp4sd_deinit() looks like this
> void isp4sd_deinit(struct isp4_subdev *isp_subdev)
> {
> struct isp4_interface *ispif = &isp_subdev->ispif;
>
> isp4vid_dev_deinit(&isp_subdev->isp_vdev);
> v4l2_device_unregister_subdev(&isp_subdev->sdev);
> media_entity_cleanup(&isp_subdev->sdev.entity);
> isp4if_deinit(ispif);
> isp4sd_module_enable(isp_subdev, false);
>
> ispif->status = ISP4IF_STATUS_PWR_OFF;
> }
Right, and v4l2_device_unregister_subdev() is *also* needed inside isp4sd_init()
to handle cleanup on error when isp4sd_init() doesn't complete successfully.
That's what I meant in our last few emails, since isp4sd_deinit() won't be
called in that scenario. Sorry that wasn't clear.
>
> You are correct and I believe both isp4vid_dev_deinit and
> v4l2_device_unregister_subdev can cause media_create_pad_link() being
> cleaned up. Because isp4vid_dev_deinit is called first, so the link will be
> cleaned by it, here is the call stack FYI, does it make sense?
> [ 5.198328] Call Trace:
> [ 5.198329] <TASK>
> [ 5.198331] dump_stack_lvl+0x76/0xa0
> [ 5.198336] dump_stack+0x10/0x20
> [ 5.198338] __media_entity_remove_link+0xdf/0x1f0 [mc]
> [ 5.198342] __media_entity_remove_links+0x31/0x70 [mc]
> [ 5.198344] __media_device_unregister_entity+0x93/0xf0 [mc]
> [ 5.198346] media_device_unregister_entity+0x2f/0x50 [mc]
> [ 5.198348] v4l2_device_release+0x112/0x190 [videodev]
> [ 5.198355] device_release+0x38/0xa0
> [ 5.198358] kobject_put+0x9e/0x200
> [ 5.198359] put_device+0x13/0x30
> [ 5.198359] vb2_video_unregister_device+0x8e/0xd0 [videobuf2_v4l2]
> [ 5.198362] isp4vid_dev_deinit+0xe/0x20 [amd_capture]
> [ 5.198364] isp4sd_deinit+0x25/0x80 [amd_capture]
> [ 5.198366] isp4_capture_probe+0x1ec/0x2f0 [amd_capture]
> [ 5.198368] platform_probe+0x3f/0xb0
> [ 5.198370] really_probe+0xf4/0x3b0
Thanks for clarifying! I was wondering where vdev.entity gets cleaned up, and it
looks like that is where it happens. And the subdev entity, sdev.entity, gets
cleaned up by v4l2_device_unregister_subdev().
To summarize: Assuming all of my other advice is implemented, the cleanup for
media_create_pad_link() will already be handled on error and device removal. So
the only action you should take is to disregard this advice from me:
"2. When media_device_register() fails, clean up media_create_pad_link() by
calling media_device_unregister()."
And that is all. :)
Sultan
^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2025-10-19 22:11 UTC | newest]
Thread overview: 35+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-11 10:08 [PATCH v4 0/7] Add AMD ISP4 driver Bin Du
2025-09-11 10:08 ` [PATCH v4 1/7] media: platform: amd: Introduce amd isp4 capture driver Bin Du
2025-09-21 20:23 ` Sultan Alsawaf
2025-09-23 7:56 ` Du, Bin
2025-09-11 10:08 ` [PATCH v4 2/7] media: platform: amd: low level support for isp4 firmware Bin Du
2025-09-21 20:31 ` Sultan Alsawaf
2025-09-23 8:05 ` Du, Bin
2025-09-11 10:08 ` [PATCH v4 3/7] media: platform: amd: Add isp4 fw and hw interface Bin Du
2025-09-21 21:55 ` Sultan Alsawaf
2025-09-23 9:24 ` Du, Bin
2025-09-24 7:09 ` Sultan Alsawaf
2025-09-25 3:56 ` Du, Bin
2025-09-25 7:20 ` Sultan Alsawaf
2025-09-25 9:42 ` Du, Bin
2025-09-11 10:08 ` [PATCH v4 4/7] media: platform: amd: isp4 subdev and firmware loading handling added Bin Du
2025-09-23 7:23 ` Sultan Alsawaf
2025-09-30 7:30 ` Du, Bin
2025-10-01 7:24 ` Sultan Alsawaf
2025-10-10 10:25 ` Du, Bin
2025-10-11 7:18 ` Sultan Alsawaf
2025-10-11 8:27 ` Du, Bin
2025-09-11 10:08 ` [PATCH v4 5/7] media: platform: amd: isp4 video node and buffers " Bin Du
2025-10-01 6:53 ` Sultan Alsawaf
2025-10-11 9:30 ` Du, Bin
2025-10-12 6:08 ` Sultan Alsawaf
2025-10-13 9:55 ` Du, Bin
2025-10-16 8:13 ` Du, Bin
2025-10-17 8:34 ` Sultan Alsawaf
2025-10-17 9:53 ` Du, Bin
2025-10-19 22:11 ` Sultan Alsawaf
2025-09-11 10:08 ` [PATCH v4 6/7] media: platform: amd: isp4 debug fs logging and more descriptive errors Bin Du
2025-09-11 10:08 ` [PATCH v4 7/7] Documentation: add documentation of AMD isp 4 driver Bin Du
2025-09-19 3:24 ` [PATCH v4 0/7] Add AMD ISP4 driver Du, Bin
2025-09-19 12:23 ` Laurent Pinchart
2025-09-22 2:50 ` Du, Bin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox