public inbox for linux-media@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH v4 00/27] media: platform: rga: Add RGA3 support
@ 2026-03-25 14:50 Sven Püschel
  2026-03-25 14:50 ` [PATCH v4 01/27] media: dt-bindings: media: rockchip-rga: add rockchip,rk3588-rga3 Sven Püschel
                   ` (26 more replies)
  0 siblings, 27 replies; 28+ messages in thread
From: Sven Püschel @ 2026-03-25 14:50 UTC (permalink / raw)
  To: Jacob Chen, Ezequiel Garcia, Mauro Carvalho Chehab,
	Heiko Stuebner, Rob Herring, Krzysztof Kozlowski, Conor Dooley
  Cc: linux-media, linux-rockchip, linux-arm-kernel, linux-kernel,
	devicetree, kernel, nicolas, sebastian.reichel, Sven Püschel,
	Nicolas Dufresne, Michael Olbrich

This series adds support for the Raster Graphic Acceleration 3 (RGA3)
peripheral, which is included in the RK3588 SoC. The RK3588
contains one RGA2-Enhanced core (which is already implemented by the
rockchip rga driver) and two independent RGA3 cores. They feature
a similar functionality of scaling, cropping and rotating of up to two input
images into one output image. Key differences of the RGA3 are:

- supports 10bit YUV output formats
- supports 8x8 tiles and FBCD as inputs and outputs
- supports BT2020 color space conversion
- max output resolution of (8192-64)x(8192-64)
- MMU can map up to 32G DDR RAM
- fully planar formats (3 planes) are not supported
- max scale up/down factor of 8 (RGA2 allows up to 16)

This patch set adds support for one RGA3 core in the existing
rga m2m driver. The feature set of the PR is limited to scaling,
format and color space conversions between common 8bit RGB/YUV formats.
This already allows a practical usage of the RGA3.

During testing it has been noted that the scaling of the hardware is
slightly incorrect. A test conversion of 128x128 RGBA to 256x256 RGBA
causes a slight shift to the bottom right. The shift is suddle, as it seems
that the image is shifted by about 2px down and right and then cropped to
it's final size (probably caused by the source sampling).
The same behavior has been observed when using the vendor driver
with the librga library.

Furthermore comparing the RGA3 conversion with the GStreamer
videoconvertscale element, the chroma-site is different. A quick testing
didn't reveal a chroma-site that creates the same image with the
GStreamer Element. Also when converting from YUV to RGB the RGB values
differ by 1 or 2. This doesn't seem to be a colorspace conversion issue
but rather a slightly different precision on the calculation.

This was tested on a Radxa Rock 5T. Around 80 fps were measured when
scaling and converting from RGBA 480x360 to NV12 3840x2160 in a single
gstreamer pipeline. Format conversions were tested with a single
gstreamer pipeline converting a fixed input to a given input format.
Afterwards it's piped through the RGA3 and the result is converted back
to rgba and compared against a given hash value (generated after
comparing the output manually to the input).

The patchset also fixes the failing v4l2-compliance tests due to the
missing colorimetry propagation from output to capture:

  v4l2-compliance 1.32.0, 64 bits, 64-bit time_t
  ...
  	Card type        : rga2
  ...
  Total for rockchip-rga device /dev/video0: 48, Succeeded: 48, Failed: 0, Warnings: 0

  v4l2-compliance 1.32.0, 64 bits, 64-bit time_t
  ...
	Card type        : rga3
  ...
  Total for rockchip-rga device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0

To distinguish the RGA2 core from the RGA3 cores the Card type is set
accordingly. Scheduling operations between both RGA3 cores to double
the possible frame rate might be a future improvement. Until then
additional RGA3 cores are disabled to only provide one video device to
the user space. This prevents a potential ABI breakage when multi core
support is implemented.

The DTS change at the end is just as a preview, as this series targets
media/next. After it's merged the DTS change will be sent as a new
patch not targeting media.

Patch 1 updates the dtb bindings doc to support the RGA3
Patch 2-5 extend v4l2 common functionality
Patch 6-9 are general cleanups
Patch 10-25 prepare the rga driver for the RGA3
Patch 26 adds RGA3 support to the rga driver
Patch 27 dtsi additions for the RGA3

Signed-off-by: Sven Püschel <s.pueschel@pengutronix.de>
---
Changes in v4:
- Add Nicolas to Cc for potential reviews and Sebastian for the nice
  RK3588 mainline status table
- Improved single memory plane y stride alignment adjustments
- Adjusted scaling inaccuracy description
- Dropped required iommu property from the binding yaml
- Fixed binding yaml indentation
- Link to v3: https://lore.kernel.org/r/20260127-spu-rga3-v3-0-77b273067beb@pengutronix.de

Changes in v3:
- Add iommus property to the dtb bindings documentation
- Drop interrupt name from the dtsi
- Added v4l2_format_info for missing 2 byte RGB formats
- Fixed incorrect dt node reference in the binding patch commit message
- Removed now unused depth member of rga_frame
- Replaced RGA3 semi planar bool with v4l2_format_info check
- Calculated x_div/y_div variables instead of storing them
- Limited width/height to even values for YUV formats
- Support all 4 CSC modes: BT601L, BT601F, BT709L, BT2020L
- Note slightly incorrect scaling by the hardware
- Fix stride alignment to bytes
- Use early returns in rga-buf init/cleanup
- Fix incorrect devm_clk_bulk_get with devm_clk_bulk_get_all
- Don't enforce max scaling factor in try_fmt (only in s_fmt)
- Merge single register editing RGA3 functions into the other functions
- Link to v2: https://lore.kernel.org/r/20251203-spu-rga3-v2-0-989a67947f71@pengutronix.de

Changes in v2:
- Removed overclocking (assigning higher clock speeds in the dts)
- Disable the second RGA3 core
- Improved RGA3 feature documentation and code comments
- Don't write the whole command buffer in each frame
- Don't announce CIDs for the RGA3 and error out on s_selection
- Check the max scaling factor of 16 (RGA2) and 8 (RGA3)
- Move stride alignment and alpha checking to v4l2 common
- Register the interrupt as shared for an external IOMMU
- Add IOMMU patch as dependency to fix sporadic hangups
- Link to v1: https://lore.kernel.org/r/20251007-spu-rga3-v1-0-36ad85570402@pengutronix.de

---
Michael Olbrich (1):
      media: rockchip: rga: share the interrupt when an external iommu is used

Sven Püschel (26):
      media: dt-bindings: media: rockchip-rga: add rockchip,rk3588-rga3
      media: v4l2-common: sort RGB formats in v4l2_format_info
      media: v4l2-common: add missing 1 and 2 byte RGB formats to v4l2_format_info
      media: v4l2-common: add has_alpha to v4l2_format_info
      media: v4l2-common: add v4l2_fill_pixfmt_mp_aligned helper
      media: rockchip: rga: use clk_bulk api
      media: rockchip: rga: use stride for offset calculation
      media: rockchip: rga: remove redundant rga_frame variables
      media: rockchip: rga: announce and sync colorimetry
      media: rockchip: rga: move hw specific parts to a dedicated struct
      media: rockchip: rga: avoid odd frame sizes for YUV formats
      media: rockchip: rga: calculate x_div/y_div using v4l2_format_info
      media: rockchip: rga: move cmdbuf to rga_ctx
      media: rockchip: rga: align stride to 4 bytes
      media: rockchip: rga: prepare cmdbuf on streamon
      media: rockchip: rga: check scaling factor
      media: rockchip: rga: use card type to specify rga type
      media: rockchip: rga: change offset to dma_addresses
      media: rockchip: rga: support external iommus
      media: rockchip: rga: remove size from rga_frame
      media: rockchip: rga: remove stride from rga_frame
      media: rockchip: rga: move rga_fmt to rga-hw.h
      media: rockchip: rga: add feature flags
      media: rockchip: rga: disable multi-core support
      media: rockchip: rga: add rga3 support
      arm64: dts: rockchip: add rga3 dt nodes

 .../devicetree/bindings/media/rockchip-rga.yaml    |  10 +-
 arch/arm64/boot/dts/rockchip/rk3588-base.dtsi      |  44 ++
 drivers/media/platform/rockchip/rga/Makefile       |   2 +-
 drivers/media/platform/rockchip/rga/rga-buf.c      |  61 ++-
 drivers/media/platform/rockchip/rga/rga-hw.c       | 358 +++++++++----
 drivers/media/platform/rockchip/rga/rga-hw.h       |  14 +-
 drivers/media/platform/rockchip/rga/rga.c          | 577 ++++++++++-----------
 drivers/media/platform/rockchip/rga/rga.h          |  85 +--
 drivers/media/platform/rockchip/rga/rga3-hw.c      | 507 ++++++++++++++++++
 drivers/media/platform/rockchip/rga/rga3-hw.h      | 192 +++++++
 drivers/media/v4l2-core/v4l2-common.c              | 128 +++--
 include/media/v4l2-common.h                        |   6 +
 12 files changed, 1480 insertions(+), 504 deletions(-)
---
base-commit: 0e2c4117c3512cf6b8f54c2c3d37564bfa3ccd67
change-id: 20251001-spu-rga3-8a00e018b120
prerequisite-change-id: 20251126-spu-iommudtefix-cd0c5244c74a:v1
prerequisite-patch-id: 10c6c977c0f71400931941b42da73adcaf63e810

Best regards,
--  
Sven Püschel <s.pueschel@pengutronix.de>


^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2026-03-25 14:51 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-03-25 14:50 [PATCH v4 00/27] media: platform: rga: Add RGA3 support Sven Püschel
2026-03-25 14:50 ` [PATCH v4 01/27] media: dt-bindings: media: rockchip-rga: add rockchip,rk3588-rga3 Sven Püschel
2026-03-25 14:50 ` [PATCH v4 02/27] media: v4l2-common: sort RGB formats in v4l2_format_info Sven Püschel
2026-03-25 14:50 ` [PATCH v4 03/27] media: v4l2-common: add missing 1 and 2 byte RGB formats to v4l2_format_info Sven Püschel
2026-03-25 14:50 ` [PATCH v4 04/27] media: v4l2-common: add has_alpha " Sven Püschel
2026-03-25 14:50 ` [PATCH v4 05/27] media: v4l2-common: add v4l2_fill_pixfmt_mp_aligned helper Sven Püschel
2026-03-25 14:50 ` [PATCH v4 06/27] media: rockchip: rga: use clk_bulk api Sven Püschel
2026-03-25 14:50 ` [PATCH v4 07/27] media: rockchip: rga: use stride for offset calculation Sven Püschel
2026-03-25 14:50 ` [PATCH v4 08/27] media: rockchip: rga: remove redundant rga_frame variables Sven Püschel
2026-03-25 14:50 ` [PATCH v4 09/27] media: rockchip: rga: announce and sync colorimetry Sven Püschel
2026-03-25 14:50 ` [PATCH v4 10/27] media: rockchip: rga: move hw specific parts to a dedicated struct Sven Püschel
2026-03-25 14:50 ` [PATCH v4 11/27] media: rockchip: rga: avoid odd frame sizes for YUV formats Sven Püschel
2026-03-25 14:50 ` [PATCH v4 12/27] media: rockchip: rga: calculate x_div/y_div using v4l2_format_info Sven Püschel
2026-03-25 14:50 ` [PATCH v4 13/27] media: rockchip: rga: move cmdbuf to rga_ctx Sven Püschel
2026-03-25 14:50 ` [PATCH v4 14/27] media: rockchip: rga: align stride to 4 bytes Sven Püschel
2026-03-25 14:50 ` [PATCH v4 15/27] media: rockchip: rga: prepare cmdbuf on streamon Sven Püschel
2026-03-25 14:50 ` [PATCH v4 16/27] media: rockchip: rga: check scaling factor Sven Püschel
2026-03-25 14:50 ` [PATCH v4 17/27] media: rockchip: rga: use card type to specify rga type Sven Püschel
2026-03-25 14:50 ` [PATCH v4 18/27] media: rockchip: rga: change offset to dma_addresses Sven Püschel
2026-03-25 14:50 ` [PATCH v4 19/27] media: rockchip: rga: support external iommus Sven Püschel
2026-03-25 14:50 ` [PATCH v4 20/27] media: rockchip: rga: share the interrupt when an external iommu is used Sven Püschel
2026-03-25 14:50 ` [PATCH v4 21/27] media: rockchip: rga: remove size from rga_frame Sven Püschel
2026-03-25 14:50 ` [PATCH v4 22/27] media: rockchip: rga: remove stride " Sven Püschel
2026-03-25 14:50 ` [PATCH v4 23/27] media: rockchip: rga: move rga_fmt to rga-hw.h Sven Püschel
2026-03-25 14:50 ` [PATCH v4 24/27] media: rockchip: rga: add feature flags Sven Püschel
2026-03-25 14:50 ` [PATCH v4 25/27] media: rockchip: rga: disable multi-core support Sven Püschel
2026-03-25 14:50 ` [PATCH v4 26/27] media: rockchip: rga: add rga3 support Sven Püschel
2026-03-25 14:50 ` [PATCH v4 27/27] arm64: dts: rockchip: add rga3 dt nodes Sven Püschel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox