Devicetree
 help / color / mirror / Atom feed
* [PATCH v3 0/8] media: Add iMX95 neoisp driver
@ 2026-06-12 13:20 Antoine Bouyer
  2026-06-12 13:20 ` [PATCH v3 1/8] dt-bindings: media: Add nxp neoisp support Antoine Bouyer
                   ` (7 more replies)
  0 siblings, 8 replies; 18+ messages in thread
From: Antoine Bouyer @ 2026-06-12 13:20 UTC (permalink / raw)
  To: julien.vuillaumier, alexi.birlinger, daniel.baluta, peng.fan,
	frank.li, jacopo.mondi, laurent.pinchart, mchehab, robh, krzk+dt,
	conor+dt, michael.riesch, anthony.mcgivern
  Cc: linux-media, linux-kernel, devicetree, imx, ai.luthra, paul.elder,
	geert, sakari.ailus, hverkuil+cisco, Antoine Bouyer

Hello

This patch series introduces the NXP Neo Image Signal Processor (ISP)
driver, used in the NXP i.MX95 SoC and future devices in the i.MX9 family.

---
Dependencies
------------
This series is based on v1 of:

  https://lore.kernel.org/linux-media/20260505-extensible-stats-v1-0-e16f326b8dad@ideasonboard.com


The reference tag I rebased all on is:

  next-20260610

---
Description
-----------

The Neo ISP processes one or more camera streams, converting RAW formats
into YUV or RGB outputs. Its architecture is largely influenced by the
PISP driver. To limit the number of v4l2 devices, the driver supports only
one context, with three sink pads (main input, second input for HDR, and
parameter buffers) and three source pads (RGB output, IR output, and
statistics metadata).

The driver uses the generic extensible v4l2-isp framework for parameters,
similar to rkisp1 and mali-c55. The same mechanism is reused for
statistics buffers by relying on the extensible stats introduced in
Jacopo's series. This version does not use a common generic meta format
for the params and stats nodes, as originally planned. Instead, it still
defines meta formats dedicated to neoisp. I preferred to wait for a
consensus on Jacopo's thread first and limit the impact on the neoisp
changes compared to the previous version.

The driver currently supports M2M operation; inline mode (CSI-to-ISP
streaming) is still under evaluation.

A few checkpatch warnings in v4l2-ioctl.c are intentionally kept to match
the existing coding style in that file. Another checkpatch warning
related to job pointer initialization is also kept, as it matches the
approach used in the pispbe driver.

Testing was performed on the i.MX95 EVK with neoisp configured in
standalone mode, with a vivid (Virtual Video Device) instance and a
libcamera neo pipeline handler. An engineering version of the libcamera
pipeline handler is under preparation, and can be shared as dedicated
branch if needed.

End-to-end camera-to-ISP capture, using a single media graph, has been
validated using the downstream NXP kernel.

Thanks,
Antoine

---
Changes in v3:
 * Use single mutex for all nodes' queues handling to avoid list_del
   errors when playing from multiple threads.
 * Use FIELD_xxx macros as suggested by Geert.
 * Add "Reviewed-by" tag from Rob on bindings patch.
 * Force using 32-bits R/W accesses to registers as specified in
   reference manual.
 * Create dedicated accessors for both register and ISP memory accesses:
   register region use readl and writel functions to force 32-bits
   accesses, while memcpy is used for local memories.
 * Fix error code check when initializing v4l2_isp_block_header.
 * Fix some errors reported by sashiko bot:
   - Rework Kconfig and align with imx8-isi
   - Fix NEOISP_COLORSPACE macro logical AND usage
   - Use WC ioremap for statistics region and use memcpy
   - Fix some error cases handling when creating/deleting elements
 * Link to v2: https://lore.kernel.org/linux-media/20260511132629.1300868-1-antoine.bouyer@nxp.com/

Changes in v2:
 * Fix dt_binding_check errors reported by Rob's bot.
 * Remove extensible stats introduction, and use v4l2_isp patches
   from Jacopo's series instead.
 * Use the common v4l2_isp definitions for stats and params in neoisp.
   Replace all occurences of v4l2_isp_params_* and v4l2_isp_stats_* by
   the common structs and helpers.
 * Use the new v4l2_isp helpers for statistics buffer handling.
 * Apply comment from Geert in Kconfig.
 * Fix some typo in the neoisp documentation.
 * Remove `neoisp_feat_ctrl_s` from uapi (used only in legacy format
   which is not supported anymore).
 * Link to v1: https://lore.kernel.org/linux-media/20260413160331.2611829-1-antoine.bouyer@nxp.com/

Changes in v1 (compared to the v0-like-version wrongly called RFC)
 * Integrate Krzysztof's comments in neoisp bindings document: use
   maxItems for clocks, remove unused configs, and rename filename to
   match compatible name used in imx95 SoC.
 * Provide a `neoisp_core_media_register` API to let neoisp register
   itself into an existing media graph, instead of creating its own. The
   goal is to prepare for supporting inline mode alongside M2M mode, and
   to allow userspace to select between these modes at runtime (the 2
   modes cannot run together because of hardware constraints).
 * Use only one Neo ISP context, whereas the RFC version prepared 8
   contexts.
 * Add a module parameter to support a standalone mode. When enabled,
   neoisp registers its own media graph. The goal is to allow testing the
   Neo ISP IP without a camera or other subdevice drivers, such as ISI,
   pixel formatter, etc.
 * Remove support of the legacy mode using fixed-size buffers for parameters
   and statistics; only the generic extensible framework is supported.
 * Remove support of the hardware version 1.
 * Use job scheduling like pispbe driver, to limit impact on interrupt
   handler.
 * Link to RFC: https://lore.kernel.org/linux-media/20260123080938.3367348-1-antoine.bouyer@nxp.com/

---
Antoine Bouyer (8):
  dt-bindings: media: Add nxp neoisp support
  media: v4l2-ctrls: Add user control base for NXP neoisp controls
  media: Add meta formats supported by NXP neoisp driver
  media: uapi: Add NXP NEOISP user interface header file
  media: Documentation: Add NXP neoisp driver documentation
  media: platform: Add NXP Neoisp Image Signal Processor
  media: platform: neoisp: Add debugfs support
  arm64: dts: freescale: imx95: Add NXP neoisp device tree node

 .../admin-guide/media/nxp-neoisp-diagram.dot  |   24 +
 .../admin-guide/media/nxp-neoisp.dot          |   18 +
 .../admin-guide/media/nxp-neoisp.rst          |  179 ++
 .../admin-guide/media/v4l-drivers.rst         |    1 +
 .../bindings/media/nxp,imx95-neoisp.yaml      |   62 +
 .../userspace-api/media/v4l/meta-formats.rst  |    1 +
 .../media/v4l/metafmt-nxp-neoisp.rst          |   70 +
 MAINTAINERS                                   |    9 +
 .../boot/dts/freescale/imx95-19x19-evk.dts    |    4 +
 arch/arm64/boot/dts/freescale/imx95.dtsi      |   11 +
 drivers/media/platform/nxp/Kconfig            |    1 +
 drivers/media/platform/nxp/Makefile           |    1 +
 drivers/media/platform/nxp/neoisp/Kconfig     |   17 +
 drivers/media/platform/nxp/neoisp/Makefile    |    8 +
 drivers/media/platform/nxp/neoisp/neoisp.h    |  265 ++
 .../media/platform/nxp/neoisp/neoisp_core.h   |   30 +
 .../media/platform/nxp/neoisp/neoisp_ctx.c    | 2635 +++++++++++++++++
 .../media/platform/nxp/neoisp/neoisp_ctx.h    |   77 +
 .../platform/nxp/neoisp/neoisp_debugfs.c      |  495 ++++
 .../media/platform/nxp/neoisp/neoisp_fmt.h    |  495 ++++
 drivers/media/platform/nxp/neoisp/neoisp_hw.h |  557 ++++
 .../media/platform/nxp/neoisp/neoisp_main.c   | 1926 ++++++++++++
 .../media/platform/nxp/neoisp/neoisp_nodes.h  |   54 +
 .../media/platform/nxp/neoisp/neoisp_regs.h   | 1465 +++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |    2 +
 include/uapi/linux/media/nxp/nxp_neoisp.h     | 1694 +++++++++++
 include/uapi/linux/v4l2-controls.h            |    6 +
 include/uapi/linux/videodev2.h                |    4 +
 28 files changed, 10111 insertions(+)
 create mode 100644 Documentation/admin-guide/media/nxp-neoisp-diagram.dot
 create mode 100644 Documentation/admin-guide/media/nxp-neoisp.dot
 create mode 100644 Documentation/admin-guide/media/nxp-neoisp.rst
 create mode 100644 Documentation/devicetree/bindings/media/nxp,imx95-neoisp.yaml
 create mode 100644 Documentation/userspace-api/media/v4l/metafmt-nxp-neoisp.rst
 create mode 100644 drivers/media/platform/nxp/neoisp/Kconfig
 create mode 100644 drivers/media/platform/nxp/neoisp/Makefile
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp.h
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_core.h
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_ctx.c
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_ctx.h
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_debugfs.c
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_fmt.h
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_hw.h
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_main.c
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_nodes.h
 create mode 100644 drivers/media/platform/nxp/neoisp/neoisp_regs.h
 create mode 100644 include/uapi/linux/media/nxp/nxp_neoisp.h


base-commit: abe651837cb394f76d738a7a747322fca3bf17ba
prerequisite-patch-id: 2939dff7f477209138725aa8d86318d6580f4ea3
prerequisite-patch-id: f77370d8fc480b2e972a773f7d56f33ff1995eef
prerequisite-patch-id: bd3b231bc86129d25fca4ae34408a61db43bf883
prerequisite-patch-id: 691322bcfe69b3c51a886441fb241067355b9bdc
prerequisite-patch-id: a5f27213eba078f7ed5420d90df3ecd45151219e
prerequisite-patch-id: 9b2fbe610fb17689030c6da6c529f7db91b86d9c
-- 
2.53.0


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2026-06-12 19:25 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-12 13:20 [PATCH v3 0/8] media: Add iMX95 neoisp driver Antoine Bouyer
2026-06-12 13:20 ` [PATCH v3 1/8] dt-bindings: media: Add nxp neoisp support Antoine Bouyer
2026-06-12 15:39   ` Frank Li
2026-06-12 13:20 ` [PATCH v3 2/8] media: v4l2-ctrls: Add user control base for NXP neoisp controls Antoine Bouyer
2026-06-12 16:52   ` Frank Li
2026-06-12 13:20 ` [PATCH v3 3/8] media: Add meta formats supported by NXP neoisp driver Antoine Bouyer
2026-06-12 16:55   ` Frank Li
2026-06-12 13:20 ` [PATCH v3 4/8] media: uapi: Add NXP NEOISP user interface header file Antoine Bouyer
2026-06-12 13:34   ` sashiko-bot
2026-06-12 18:31   ` Frank Li
2026-06-12 13:20 ` [PATCH v3 5/8] media: Documentation: Add NXP neoisp driver documentation Antoine Bouyer
2026-06-12 13:20 ` [PATCH v3 6/8] media: platform: Add NXP Neoisp Image Signal Processor Antoine Bouyer
2026-06-12 13:41   ` sashiko-bot
2026-06-12 19:25   ` Frank Li
2026-06-12 13:20 ` [PATCH v3 7/8] media: platform: neoisp: Add debugfs support Antoine Bouyer
2026-06-12 13:34   ` sashiko-bot
2026-06-12 16:49   ` Frank Li
2026-06-12 13:20 ` [PATCH v3 8/8] arm64: dts: freescale: imx95: Add NXP neoisp device tree node Antoine Bouyer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox