* [RFC PATCH 0/6] media: rockchip: RKVENC v1 H.264/HEVC encoder for the RV1126
@ 2026-11-14 14:54 ` Daniel Golle
0 siblings, 0 replies; 17+ messages in thread
From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw)
To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd,
Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus,
Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker,
linux-media, linux-rockchip, devicetree, linux-arm-kernel,
linux-kernel, linux-clk
Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel
This RFC adds a driver for RKVENC v1, the H.264 and HEVC video encoder
of the Rockchip RV1109 and RV1126 (Rockchip calls this generation
"vepu541"; the RK3566 and RK3568 carry the "vepu540" variant). It is a
stateful V4L2 memory-to-memory encoder: NV12 or NV12M in, H.264 or HEVC
Annex B out, low-delay IDR plus P pictures with a single reference, no
B frames. The driver writes the parameter sets (SPS, PPS and VPS), the
hardware writes the slice headers and data. Rate control is frame
level, in the driver: CBR with one QP per picture, the hardware row
rate control stays off. The core clock follows the load through
devfreq and the encoder registers as a cooling device. No new uAPI is
added; the driver uses the existing stateful encoder controls and
formats only.
The series:
1 binding (with MAINTAINERS entry)
2 RV1126 clock tree of the VEPU power domain
3 the driver
4 frequency scaling and cooling
5 RV1126 device tree node, IOMMU, power domain and QoS nodes
6 RV1126 operating points
It is an RFC because of the questions below, which decide where and
in which shape this can go.
Hardware and testing:
I have tested H.264 and H.265 encoding extensively with this driver on
Linux 6.18 in an RV1126 IP camera (Herospeed board, Sony IMX415
sensor) behind the RV1126 ISP, including 3856x2192 and 1280x720 H.265
at 30 fps streamed continuously over RTSP. The driver has been part of
the OpenWrt rockchip/armv7 target since 2026-10-17
(target/linux/rockchip/patches-6.18/183-*), so that testing can be
reproduced by anyone with the hardware.
The port to linux-next in this series is compile-tested only: gcc 16.2
and clang 23, both with W=1, ARM multi_v7_defconfig plus the RKVENC
options; dt_binding_check on the binding and dtbs_check on the RV1109
and RV1126 boards in tree, all without warnings (dtschema 2026.9).
Against the 6.18 code the driver differs in the cooling device
registration signature of the base, kzalloc_obj(), one min_t(), a few
named constants and its comments.
Design questions:
There is no stateless encoder framework upstream yet. The driver
implements a stateful front end only, and keeps the hardware back end
apart from it so that it can move to stateless infrastructure once
that exists:
- Hardware back end: rkvenc.c (job start, interrupt, recovery,
watchdog, power), rkvenc-vepu541*.c (the register image, built from
struct rkvenc_frame, which carries no V4L2 types), the tuning tables,
rkvenc-ref.c (reconstruction and reference buffers) and
rkvenc-dvfs.c.
- Codec front end: rkvenc-gop.c (IDR cadence, forced key frames, which
slot is the reference), rkvenc-h264.c and rkvenc-hevc.c (frame_num,
POC, idr_pic_id and NAL type of each picture), rkvenc-nal.c (the
parameter set writer) and rkvenc-rc.c (the picture QP).
- V4L2 binding: rkvenc-v4l2.c.
Under the H.264 stateless encode series Paul posted in May, userspace
decides the frame type, frame_num, POC and references and passes them
in V4L2_CID_STATELESS_H264_ENCODE_PARAMS, while header writing and rate
control stay in the kernel as shared helpers. Applied to this driver,
rkvenc-gop.c and the picture fields of rkvenc-h264.c become userspace
input, rkvenc-nal.c and rkvenc-rc.c give way to the shared helpers,
and the back end stays as it is: struct rkvenc_frame is already the
per-picture state such a core hands to a driver. The single reference
with sliding window marking in that series is what the two-slot
reference handling here implements. HEVC has no posted stateless
encode uAPI at the moment. Still to be done for a conversion: the GOP,
reference and rate control code works on struct rkvenc_ctx, which also
carries the V4L2 file handle and control state, and the parameter set
code reads V4L2 control enums; both need a codec state of their own.
Questions:
1. Conformance of a stateful encoder.
The maintainer entry profile requires v4l2-compliance to pass, and
dev-encoder.rst defines the interface, but there is no fluster
equivalent for encoders, and Nicolas noted in January 2025 that we
have no metric of how good or bad the in-tree encoders are. The plan
for the next version is: v4l2-compliance -s with streaming from a
file, plus every stream decoded by FFmpeg and by the H.264 JM and
HEVC HM reference decoders, the decoded pictures required to be
bit-identical between the decoders, PSNR reported per picture
against the source, a static-scene sequence to catch
encoder/decoder drift, and the bitrate of CBR runs reported against
the target. Is that what you expect to see for a new encoder, is
there a test corpus or tool you would rather have used, and should
the results go into the cover letter or somewhere else?
2. Staging or drivers/media/platform/rockchip.
hantro and rkvdec were staged because the stateless uAPI was not
stable yet, see commit fbb6c848dd89 ("media: destage Hantro VPU
driver"), and allegro left staging once the stateful encoder API
was final, see commit d74d4e2359ec ("media: allegro: move driver
out of staging"). This driver uses only the stable stateful encoder
uAPI and adds no controls, formats or ioctls of its own, so it is
posted for drivers/media/platform/rockchip/rkvenc. The in-kernel
rate control and the driver-written parameter sets are the parts
that differ from the firmware-based stateful encoders in tree. Do
either of them, or anything else, argue for staging in your view?
3. Stateful now, and the back end for the stateless work?
dev-encoder.rst strongly discourages header generation in the
driver for a stateful encoder and advises the stateless encoder
interface instead, and in the VEPU510 thread Nicolas asked not to
offer two interfaces for the same hardware. RKVENC v1 is the
generation before VEPU510/VEPU580, its main users are 32-bit camera
SoCs whose userspace is GStreamer and v4l2 RTSP servers, and no
other interface for it has been posted. Nicolas also asked for a
second implementation of the stateless encoder spec, from another
vendor than the VC8000E. Would you take the stateful driver now,
with its back end offered as that second implementation and the
driver converted once the spec is merged, or should RKVENC v1 only
come as a stateless driver?
4. Naming. The binding is rockchip,rv1126-rkvenc.yaml with the
compatibles rockchip,rv1126-rkvenc and rockchip,rk3568-rkvenc, and
the driver directory rockchip/rkvenc. The RK3576 and RK3588 RFCs
below used rockchip/rkvenc for a different generation of the
block. Should RKVENC v1 keep that directory and those names, or
take generation specific ones (for example vepu541)?
5. RK3566/RK3568. The driver matches rockchip,rk3568-rkvenc with the
vepu540 variant data, but I don't have hardware to actuall test
that. Keep the compatible for a later RK356x series, or drop it
until someone with the hardware has done some testing?
Related postings:
- Jiaxing Hu, "[RFC PATCH 0/3] media: rockchip: VEPU510 H.264 encoder
for RK3576", stateful, I frames only; the author moved to the Vulkan
Video direction in the thread:
https://lore.kernel.org/all/20260722073417.2064667-1-gahing@gahingwoo.com/
- Arun Iyengar, "[PATCH RFC 00/12] media: rockchip: Add a VEPU580
stateless H.264/HEVC encoder driver for the RK3588", withdrawn by
the author on 2026-11-13:
https://lore.kernel.org/all/20261110-rkvenc-v1-0-ea8b47122d8c@gmail.com/
- Paul Kocialkowski, "[PATCH 00/14] media: Add V4L2 H.264 stateless
encode and VC8000E support":
https://lore.kernel.org/all/20260522101653.2565125-1-paulk@sys-base.io/
Not in this series:
- Binding the encoder to the RV1126 NPU thermal zone, the zone that
covers the VEPU power domain. It needs RV1126 TSADC support, which is
not in mainline yet, and follows with it.
- venc-supply on the RV1126 boards in tree. Their encoder shares the
vdd_npu_vepu rail with the NPU.
Provenance:
There is no public documentation of this block. The driver was written
by reverse engineering the hardware, with the Rockchip MPP userspace
library as a behavioural reference. A spec was derived by an LLM
agent, and that spec was then used to create the driver
Tool use:
Most of this series was written with multiple LLM coding assistants:
The spec was derived by Claude Sonnet 5.5, handed to a locally run
Qwen3.8-37B which carried out the initial implementation. Claude
Opus 5.5 then did the initial testing and bug fixing, and pre-submission
polishment under my micro-management.
The full test coverage, and long-term testing on the off-the-shelf camera
was done by me.
Daniel Golle (6):
dt-bindings: media: Add Rockchip RKVENC v1 video encoder
clk: rockchip: rv1126: Add PD_VEPU clock tree
media: rockchip: Add RKVENC v1 video encoder driver
media: rockchip: rkvenc: Add frequency scaling and cooling
ARM: dts: rockchip: Add RKVENC encoder to RV1126
ARM: dts: rockchip: Add RKVENC operating points to RV1126
.../media/rockchip,rv1126-rkvenc.yaml | 104 ++
MAINTAINERS | 8 +
arch/arm/boot/dts/rockchip/rv1126.dtsi | 80 ++
drivers/clk/rockchip/clk-rv1126.c | 21 +
drivers/media/platform/rockchip/Kconfig | 1 +
drivers/media/platform/rockchip/Makefile | 1 +
.../media/platform/rockchip/rkvenc/Kconfig | 20 +
.../media/platform/rockchip/rkvenc/Makefile | 16 +
.../platform/rockchip/rkvenc/rkvenc-dvfs.c | 364 ++++++
.../platform/rockchip/rkvenc/rkvenc-gop.c | 72 ++
.../platform/rockchip/rkvenc/rkvenc-h264.c | 82 ++
.../platform/rockchip/rkvenc/rkvenc-hevc.c | 63 +
.../platform/rockchip/rkvenc/rkvenc-nal.c | 351 ++++++
.../platform/rockchip/rkvenc/rkvenc-nal.h | 12 +
.../platform/rockchip/rkvenc/rkvenc-rc.c | 214 ++++
.../platform/rockchip/rkvenc/rkvenc-ref.c | 114 ++
.../platform/rockchip/rkvenc/rkvenc-v4l2.c | 1085 +++++++++++++++++
.../rockchip/rkvenc/rkvenc-vepu541-h264.c | 206 ++++
.../rockchip/rkvenc/rkvenc-vepu541-hevc.c | 182 +++
.../rockchip/rkvenc/rkvenc-vepu541-regs.h | 283 +++++
.../rockchip/rkvenc/rkvenc-vepu541-tables.c | 256 ++++
.../platform/rockchip/rkvenc/rkvenc-vepu541.c | 215 ++++
.../platform/rockchip/rkvenc/rkvenc-vepu541.h | 201 +++
.../media/platform/rockchip/rkvenc/rkvenc.c | 498 ++++++++
.../media/platform/rockchip/rkvenc/rkvenc.h | 263 ++++
25 files changed, 4712 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml
create mode 100644 drivers/media/platform/rockchip/rkvenc/Kconfig
create mode 100644 drivers/media/platform/rockchip/rkvenc/Makefile
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.c
create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.h
base-commit: 84abb955b1015477aa11a96e2f3ac1b938b77f48
--
2.56.0
^ permalink raw reply [flat|nested] 17+ messages in thread* [RFC PATCH 0/6] media: rockchip: RKVENC v1 H.264/HEVC encoder for the RV1126 @ 2026-11-14 14:54 ` Daniel Golle 0 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel This RFC adds a driver for RKVENC v1, the H.264 and HEVC video encoder of the Rockchip RV1109 and RV1126 (Rockchip calls this generation "vepu541"; the RK3566 and RK3568 carry the "vepu540" variant). It is a stateful V4L2 memory-to-memory encoder: NV12 or NV12M in, H.264 or HEVC Annex B out, low-delay IDR plus P pictures with a single reference, no B frames. The driver writes the parameter sets (SPS, PPS and VPS), the hardware writes the slice headers and data. Rate control is frame level, in the driver: CBR with one QP per picture, the hardware row rate control stays off. The core clock follows the load through devfreq and the encoder registers as a cooling device. No new uAPI is added; the driver uses the existing stateful encoder controls and formats only. The series: 1 binding (with MAINTAINERS entry) 2 RV1126 clock tree of the VEPU power domain 3 the driver 4 frequency scaling and cooling 5 RV1126 device tree node, IOMMU, power domain and QoS nodes 6 RV1126 operating points It is an RFC because of the questions below, which decide where and in which shape this can go. Hardware and testing: I have tested H.264 and H.265 encoding extensively with this driver on Linux 6.18 in an RV1126 IP camera (Herospeed board, Sony IMX415 sensor) behind the RV1126 ISP, including 3856x2192 and 1280x720 H.265 at 30 fps streamed continuously over RTSP. The driver has been part of the OpenWrt rockchip/armv7 target since 2026-10-17 (target/linux/rockchip/patches-6.18/183-*), so that testing can be reproduced by anyone with the hardware. The port to linux-next in this series is compile-tested only: gcc 16.2 and clang 23, both with W=1, ARM multi_v7_defconfig plus the RKVENC options; dt_binding_check on the binding and dtbs_check on the RV1109 and RV1126 boards in tree, all without warnings (dtschema 2026.9). Against the 6.18 code the driver differs in the cooling device registration signature of the base, kzalloc_obj(), one min_t(), a few named constants and its comments. Design questions: There is no stateless encoder framework upstream yet. The driver implements a stateful front end only, and keeps the hardware back end apart from it so that it can move to stateless infrastructure once that exists: - Hardware back end: rkvenc.c (job start, interrupt, recovery, watchdog, power), rkvenc-vepu541*.c (the register image, built from struct rkvenc_frame, which carries no V4L2 types), the tuning tables, rkvenc-ref.c (reconstruction and reference buffers) and rkvenc-dvfs.c. - Codec front end: rkvenc-gop.c (IDR cadence, forced key frames, which slot is the reference), rkvenc-h264.c and rkvenc-hevc.c (frame_num, POC, idr_pic_id and NAL type of each picture), rkvenc-nal.c (the parameter set writer) and rkvenc-rc.c (the picture QP). - V4L2 binding: rkvenc-v4l2.c. Under the H.264 stateless encode series Paul posted in May, userspace decides the frame type, frame_num, POC and references and passes them in V4L2_CID_STATELESS_H264_ENCODE_PARAMS, while header writing and rate control stay in the kernel as shared helpers. Applied to this driver, rkvenc-gop.c and the picture fields of rkvenc-h264.c become userspace input, rkvenc-nal.c and rkvenc-rc.c give way to the shared helpers, and the back end stays as it is: struct rkvenc_frame is already the per-picture state such a core hands to a driver. The single reference with sliding window marking in that series is what the two-slot reference handling here implements. HEVC has no posted stateless encode uAPI at the moment. Still to be done for a conversion: the GOP, reference and rate control code works on struct rkvenc_ctx, which also carries the V4L2 file handle and control state, and the parameter set code reads V4L2 control enums; both need a codec state of their own. Questions: 1. Conformance of a stateful encoder. The maintainer entry profile requires v4l2-compliance to pass, and dev-encoder.rst defines the interface, but there is no fluster equivalent for encoders, and Nicolas noted in January 2025 that we have no metric of how good or bad the in-tree encoders are. The plan for the next version is: v4l2-compliance -s with streaming from a file, plus every stream decoded by FFmpeg and by the H.264 JM and HEVC HM reference decoders, the decoded pictures required to be bit-identical between the decoders, PSNR reported per picture against the source, a static-scene sequence to catch encoder/decoder drift, and the bitrate of CBR runs reported against the target. Is that what you expect to see for a new encoder, is there a test corpus or tool you would rather have used, and should the results go into the cover letter or somewhere else? 2. Staging or drivers/media/platform/rockchip. hantro and rkvdec were staged because the stateless uAPI was not stable yet, see commit fbb6c848dd89 ("media: destage Hantro VPU driver"), and allegro left staging once the stateful encoder API was final, see commit d74d4e2359ec ("media: allegro: move driver out of staging"). This driver uses only the stable stateful encoder uAPI and adds no controls, formats or ioctls of its own, so it is posted for drivers/media/platform/rockchip/rkvenc. The in-kernel rate control and the driver-written parameter sets are the parts that differ from the firmware-based stateful encoders in tree. Do either of them, or anything else, argue for staging in your view? 3. Stateful now, and the back end for the stateless work? dev-encoder.rst strongly discourages header generation in the driver for a stateful encoder and advises the stateless encoder interface instead, and in the VEPU510 thread Nicolas asked not to offer two interfaces for the same hardware. RKVENC v1 is the generation before VEPU510/VEPU580, its main users are 32-bit camera SoCs whose userspace is GStreamer and v4l2 RTSP servers, and no other interface for it has been posted. Nicolas also asked for a second implementation of the stateless encoder spec, from another vendor than the VC8000E. Would you take the stateful driver now, with its back end offered as that second implementation and the driver converted once the spec is merged, or should RKVENC v1 only come as a stateless driver? 4. Naming. The binding is rockchip,rv1126-rkvenc.yaml with the compatibles rockchip,rv1126-rkvenc and rockchip,rk3568-rkvenc, and the driver directory rockchip/rkvenc. The RK3576 and RK3588 RFCs below used rockchip/rkvenc for a different generation of the block. Should RKVENC v1 keep that directory and those names, or take generation specific ones (for example vepu541)? 5. RK3566/RK3568. The driver matches rockchip,rk3568-rkvenc with the vepu540 variant data, but I don't have hardware to actuall test that. Keep the compatible for a later RK356x series, or drop it until someone with the hardware has done some testing? Related postings: - Jiaxing Hu, "[RFC PATCH 0/3] media: rockchip: VEPU510 H.264 encoder for RK3576", stateful, I frames only; the author moved to the Vulkan Video direction in the thread: https://lore.kernel.org/all/20260722073417.2064667-1-gahing@gahingwoo.com/ - Arun Iyengar, "[PATCH RFC 00/12] media: rockchip: Add a VEPU580 stateless H.264/HEVC encoder driver for the RK3588", withdrawn by the author on 2026-11-13: https://lore.kernel.org/all/20261110-rkvenc-v1-0-ea8b47122d8c@gmail.com/ - Paul Kocialkowski, "[PATCH 00/14] media: Add V4L2 H.264 stateless encode and VC8000E support": https://lore.kernel.org/all/20260522101653.2565125-1-paulk@sys-base.io/ Not in this series: - Binding the encoder to the RV1126 NPU thermal zone, the zone that covers the VEPU power domain. It needs RV1126 TSADC support, which is not in mainline yet, and follows with it. - venc-supply on the RV1126 boards in tree. Their encoder shares the vdd_npu_vepu rail with the NPU. Provenance: There is no public documentation of this block. The driver was written by reverse engineering the hardware, with the Rockchip MPP userspace library as a behavioural reference. A spec was derived by an LLM agent, and that spec was then used to create the driver Tool use: Most of this series was written with multiple LLM coding assistants: The spec was derived by Claude Sonnet 5.5, handed to a locally run Qwen3.8-37B which carried out the initial implementation. Claude Opus 5.5 then did the initial testing and bug fixing, and pre-submission polishment under my micro-management. The full test coverage, and long-term testing on the off-the-shelf camera was done by me. Daniel Golle (6): dt-bindings: media: Add Rockchip RKVENC v1 video encoder clk: rockchip: rv1126: Add PD_VEPU clock tree media: rockchip: Add RKVENC v1 video encoder driver media: rockchip: rkvenc: Add frequency scaling and cooling ARM: dts: rockchip: Add RKVENC encoder to RV1126 ARM: dts: rockchip: Add RKVENC operating points to RV1126 .../media/rockchip,rv1126-rkvenc.yaml | 104 ++ MAINTAINERS | 8 + arch/arm/boot/dts/rockchip/rv1126.dtsi | 80 ++ drivers/clk/rockchip/clk-rv1126.c | 21 + drivers/media/platform/rockchip/Kconfig | 1 + drivers/media/platform/rockchip/Makefile | 1 + .../media/platform/rockchip/rkvenc/Kconfig | 20 + .../media/platform/rockchip/rkvenc/Makefile | 16 + .../platform/rockchip/rkvenc/rkvenc-dvfs.c | 364 ++++++ .../platform/rockchip/rkvenc/rkvenc-gop.c | 72 ++ .../platform/rockchip/rkvenc/rkvenc-h264.c | 82 ++ .../platform/rockchip/rkvenc/rkvenc-hevc.c | 63 + .../platform/rockchip/rkvenc/rkvenc-nal.c | 351 ++++++ .../platform/rockchip/rkvenc/rkvenc-nal.h | 12 + .../platform/rockchip/rkvenc/rkvenc-rc.c | 214 ++++ .../platform/rockchip/rkvenc/rkvenc-ref.c | 114 ++ .../platform/rockchip/rkvenc/rkvenc-v4l2.c | 1085 +++++++++++++++++ .../rockchip/rkvenc/rkvenc-vepu541-h264.c | 206 ++++ .../rockchip/rkvenc/rkvenc-vepu541-hevc.c | 182 +++ .../rockchip/rkvenc/rkvenc-vepu541-regs.h | 283 +++++ .../rockchip/rkvenc/rkvenc-vepu541-tables.c | 256 ++++ .../platform/rockchip/rkvenc/rkvenc-vepu541.c | 215 ++++ .../platform/rockchip/rkvenc/rkvenc-vepu541.h | 201 +++ .../media/platform/rockchip/rkvenc/rkvenc.c | 498 ++++++++ .../media/platform/rockchip/rkvenc/rkvenc.h | 263 ++++ 25 files changed, 4712 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml create mode 100644 drivers/media/platform/rockchip/rkvenc/Kconfig create mode 100644 drivers/media/platform/rockchip/rkvenc/Makefile create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.h base-commit: 84abb955b1015477aa11a96e2f3ac1b938b77f48 -- 2.56.0 _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip ^ permalink raw reply [flat|nested] 17+ messages in thread
* [RFC PATCH 1/6] dt-bindings: media: Add Rockchip RKVENC v1 video encoder 2026-11-14 14:54 ` Daniel Golle @ 2026-11-14 14:54 ` Daniel Golle -1 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add the binding of the RKVENC v1 video encoder found in the RV1109 and RV1126 ("vepu541") and in the RK3566 and RK3568 ("vepu540"). The block encodes H.264 and HEVC from raw frames and reaches memory through an IOMMU. Describe the register space, the interrupt, the AXI, AHB and core clocks and resets, the power domain and the IOMMU. The core clock is scaled between operating points with an optional supply, and the encoder is a cooling device of the thermal zones. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- .../media/rockchip,rv1126-rkvenc.yaml | 104 ++++++++++++++++++ MAINTAINERS | 7 ++ 2 files changed, 111 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml diff --git a/Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml b/Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml new file mode 100644 index 0000000000000..9a83ce1499493 --- /dev/null +++ b/Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml @@ -0,0 +1,104 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/rockchip,rv1126-rkvenc.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Rockchip RKVENC v1 H.264/HEVC video encoder + +maintainers: + - Daniel Golle <daniel@makrotopia.org> + +description: + The RKVENC v1 encoder found in the RV1109, RV1126, RK3566 and RK3568 + encodes H.264 and HEVC from raw YUV or RGB frames. It reads and writes + memory through an IOMMU with two MMU instances. + +properties: + compatible: + enum: + - rockchip,rk3568-rkvenc + - rockchip,rv1126-rkvenc + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: AXI master clock + - description: AHB register interface clock + - description: Encoder core clock + + clock-names: + items: + - const: axi + - const: ahb + - const: core + + assigned-clocks: true + + assigned-clock-rates: true + + operating-points-v2: true + + venc-supply: + description: Supply of the encoder core, scaled with the operating points. + + "#cooling-cells": + const: 2 + + resets: + items: + - description: AXI reset + - description: AHB reset + - description: Encoder core reset + + reset-names: + items: + - const: axi + - const: ahb + - const: core + + power-domains: + maxItems: 1 + + iommus: + maxItems: 1 + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - resets + - reset-names + - power-domains + - iommus + +additionalProperties: false + +examples: + - | + #include <dt-bindings/clock/rockchip,rv1126-cru.h> + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/power/rockchip,rv1126-power.h> + + video-codec@ffbb0000 { + compatible = "rockchip,rv1126-rkvenc"; + reg = <0xffbb0000 0xf00>; + interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_VENC>, <&cru HCLK_VENC>, <&cru CLK_VENC_CORE>; + clock-names = "axi", "ahb", "core"; + assigned-clocks = <&cru ACLK_VENC>, <&cru CLK_VENC_CORE>; + assigned-clock-rates = <297000000>, <396000000>; + resets = <&cru SRST_VENC_A>, <&cru SRST_VENC_H>, + <&cru SRST_VENC_CORE>; + reset-names = "axi", "ahb", "core"; + iommus = <&venc_mmu>; + power-domains = <&power RV1126_PD_VEPU>; + }; +... diff --git a/MAINTAINERS b/MAINTAINERS index 9cd098f0d91f3..26991dd3a3569 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -24544,6 +24544,13 @@ S: Maintained F: Documentation/devicetree/bindings/media/rockchip,vdec.yaml F: drivers/media/platform/rockchip/rkvdec/ +ROCKCHIP RKVENC VIDEO ENCODER DRIVER +M: Daniel Golle <daniel@makrotopia.org> +L: linux-media@vger.kernel.org +L: linux-rockchip@lists.infradead.org +S: Maintained +F: Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml + ROCKCHIP RK3308 INTERNAL AUDIO CODEC M: Luca Ceresoli <luca.ceresoli@bootlin.com> S: Maintained -- 2.56.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 1/6] dt-bindings: media: Add Rockchip RKVENC v1 video encoder @ 2026-11-14 14:54 ` Daniel Golle 0 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add the binding of the RKVENC v1 video encoder found in the RV1109 and RV1126 ("vepu541") and in the RK3566 and RK3568 ("vepu540"). The block encodes H.264 and HEVC from raw frames and reaches memory through an IOMMU. Describe the register space, the interrupt, the AXI, AHB and core clocks and resets, the power domain and the IOMMU. The core clock is scaled between operating points with an optional supply, and the encoder is a cooling device of the thermal zones. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- .../media/rockchip,rv1126-rkvenc.yaml | 104 ++++++++++++++++++ MAINTAINERS | 7 ++ 2 files changed, 111 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml diff --git a/Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml b/Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml new file mode 100644 index 0000000000000..9a83ce1499493 --- /dev/null +++ b/Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml @@ -0,0 +1,104 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/rockchip,rv1126-rkvenc.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Rockchip RKVENC v1 H.264/HEVC video encoder + +maintainers: + - Daniel Golle <daniel@makrotopia.org> + +description: + The RKVENC v1 encoder found in the RV1109, RV1126, RK3566 and RK3568 + encodes H.264 and HEVC from raw YUV or RGB frames. It reads and writes + memory through an IOMMU with two MMU instances. + +properties: + compatible: + enum: + - rockchip,rk3568-rkvenc + - rockchip,rv1126-rkvenc + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: AXI master clock + - description: AHB register interface clock + - description: Encoder core clock + + clock-names: + items: + - const: axi + - const: ahb + - const: core + + assigned-clocks: true + + assigned-clock-rates: true + + operating-points-v2: true + + venc-supply: + description: Supply of the encoder core, scaled with the operating points. + + "#cooling-cells": + const: 2 + + resets: + items: + - description: AXI reset + - description: AHB reset + - description: Encoder core reset + + reset-names: + items: + - const: axi + - const: ahb + - const: core + + power-domains: + maxItems: 1 + + iommus: + maxItems: 1 + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - resets + - reset-names + - power-domains + - iommus + +additionalProperties: false + +examples: + - | + #include <dt-bindings/clock/rockchip,rv1126-cru.h> + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/power/rockchip,rv1126-power.h> + + video-codec@ffbb0000 { + compatible = "rockchip,rv1126-rkvenc"; + reg = <0xffbb0000 0xf00>; + interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_VENC>, <&cru HCLK_VENC>, <&cru CLK_VENC_CORE>; + clock-names = "axi", "ahb", "core"; + assigned-clocks = <&cru ACLK_VENC>, <&cru CLK_VENC_CORE>; + assigned-clock-rates = <297000000>, <396000000>; + resets = <&cru SRST_VENC_A>, <&cru SRST_VENC_H>, + <&cru SRST_VENC_CORE>; + reset-names = "axi", "ahb", "core"; + iommus = <&venc_mmu>; + power-domains = <&power RV1126_PD_VEPU>; + }; +... diff --git a/MAINTAINERS b/MAINTAINERS index 9cd098f0d91f3..26991dd3a3569 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -24544,6 +24544,13 @@ S: Maintained F: Documentation/devicetree/bindings/media/rockchip,vdec.yaml F: drivers/media/platform/rockchip/rkvdec/ +ROCKCHIP RKVENC VIDEO ENCODER DRIVER +M: Daniel Golle <daniel@makrotopia.org> +L: linux-media@vger.kernel.org +L: linux-rockchip@lists.infradead.org +S: Maintained +F: Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml + ROCKCHIP RK3308 INTERNAL AUDIO CODEC M: Luca Ceresoli <luca.ceresoli@bootlin.com> S: Maintained -- 2.56.0 _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 2/6] clk: rockchip: rv1126: Add PD_VEPU clock tree 2026-11-14 14:54 ` Daniel Golle @ 2026-11-14 14:54 ` Daniel Golle -1 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel The video encoder power domain of the RV1126 contains the RKVENC encoder. Add its AXI and AHB bus clocks, the AXI and AHB clocks of the encoder and the encoder core clock, together with the two interconnect gates that stay enabled and the two parent selections of the bus and core clocks. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- drivers/clk/rockchip/clk-rv1126.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/clk/rockchip/clk-rv1126.c b/drivers/clk/rockchip/clk-rv1126.c index f35bc9670c274..111595c9d990d 100644 --- a/drivers/clk/rockchip/clk-rv1126.c +++ b/drivers/clk/rockchip/clk-rv1126.c @@ -176,6 +176,8 @@ PNAME(mux_i2s2_out2io_p) = { "mclk_i2s2", "xin12m" }; PNAME(mux_gpll_cpll_xin24m_p) = { "gpll", "cpll", "xin24m" }; PNAME(mux_audpwm_p) = { "sclk_audpwm_div", "sclk_audpwm_fracdiv", "xin24m" }; PNAME(mux_dclk_vop_p) = { "dclk_vop_div", "dclk_vop_fracdiv", "xin24m" }; +PNAME(mux_cpll_gpll_hpll_p) = { "cpll", "gpll", "hpll" }; +PNAME(mux_cpll_hpll_gpll_p) = { "cpll", "hpll", "gpll" }; PNAME(mux_usb480m_gpll_p) = { "usb480m", "gpll" }; PNAME(clk_gmac_src_m0_p) = { "clk_gmac_div", "clk_gmac_rgmii_m0" }; PNAME(clk_gmac_src_m1_p) = { "clk_gmac_div", "clk_gmac_rgmii_m1" }; @@ -719,6 +721,25 @@ static struct rockchip_clk_branch rv1126_clk_branches[] __initdata = { RV1126_CLKSEL_CON(72), 8, 1, MFLAGS, 0, 7, DFLAGS, RV1126_CLKGATE_CON(11), 1, GFLAGS), + /* PD_VEPU */ + COMPOSITE(ACLK_PDVEPU, "aclk_pdvepu", mux_cpll_hpll_gpll_p, 0, + RV1126_CLKSEL_CON(40), 6, 2, MFLAGS, 0, 5, DFLAGS, + RV1126_CLKGATE_CON(12), 0, GFLAGS), + COMPOSITE_NOMUX(HCLK_PDVEPU, "hclk_pdvepu", "aclk_pdvepu", 0, + RV1126_CLKSEL_CON(41), 0, 5, DFLAGS, + RV1126_CLKGATE_CON(12), 2, GFLAGS), + GATE(ACLK_VENC, "aclk_venc", "aclk_pdvepu", 0, + RV1126_CLKGATE_CON(12), 5, GFLAGS), + GATE(HCLK_VENC, "hclk_venc", "hclk_pdvepu", 0, + RV1126_CLKGATE_CON(12), 6, GFLAGS), + COMPOSITE(CLK_VENC_CORE, "clk_venc_core", mux_cpll_gpll_hpll_p, 0, + RV1126_CLKSEL_CON(40), 14, 2, MFLAGS, 8, 5, DFLAGS, + RV1126_CLKGATE_CON(12), 1, GFLAGS), + GATE(0, "aclk_pdvepu_niu", "aclk_pdvepu", CLK_IGNORE_UNUSED, + RV1126_CLKGATE_CON(12), 3, GFLAGS), + GATE(0, "hclk_pdvepu_niu", "hclk_pdvepu", CLK_IGNORE_UNUSED, + RV1126_CLKGATE_CON(12), 4, GFLAGS), + /* * Clock-Architecture Diagram 9 */ -- 2.56.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 2/6] clk: rockchip: rv1126: Add PD_VEPU clock tree @ 2026-11-14 14:54 ` Daniel Golle 0 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel The video encoder power domain of the RV1126 contains the RKVENC encoder. Add its AXI and AHB bus clocks, the AXI and AHB clocks of the encoder and the encoder core clock, together with the two interconnect gates that stay enabled and the two parent selections of the bus and core clocks. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- drivers/clk/rockchip/clk-rv1126.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/clk/rockchip/clk-rv1126.c b/drivers/clk/rockchip/clk-rv1126.c index f35bc9670c274..111595c9d990d 100644 --- a/drivers/clk/rockchip/clk-rv1126.c +++ b/drivers/clk/rockchip/clk-rv1126.c @@ -176,6 +176,8 @@ PNAME(mux_i2s2_out2io_p) = { "mclk_i2s2", "xin12m" }; PNAME(mux_gpll_cpll_xin24m_p) = { "gpll", "cpll", "xin24m" }; PNAME(mux_audpwm_p) = { "sclk_audpwm_div", "sclk_audpwm_fracdiv", "xin24m" }; PNAME(mux_dclk_vop_p) = { "dclk_vop_div", "dclk_vop_fracdiv", "xin24m" }; +PNAME(mux_cpll_gpll_hpll_p) = { "cpll", "gpll", "hpll" }; +PNAME(mux_cpll_hpll_gpll_p) = { "cpll", "hpll", "gpll" }; PNAME(mux_usb480m_gpll_p) = { "usb480m", "gpll" }; PNAME(clk_gmac_src_m0_p) = { "clk_gmac_div", "clk_gmac_rgmii_m0" }; PNAME(clk_gmac_src_m1_p) = { "clk_gmac_div", "clk_gmac_rgmii_m1" }; @@ -719,6 +721,25 @@ static struct rockchip_clk_branch rv1126_clk_branches[] __initdata = { RV1126_CLKSEL_CON(72), 8, 1, MFLAGS, 0, 7, DFLAGS, RV1126_CLKGATE_CON(11), 1, GFLAGS), + /* PD_VEPU */ + COMPOSITE(ACLK_PDVEPU, "aclk_pdvepu", mux_cpll_hpll_gpll_p, 0, + RV1126_CLKSEL_CON(40), 6, 2, MFLAGS, 0, 5, DFLAGS, + RV1126_CLKGATE_CON(12), 0, GFLAGS), + COMPOSITE_NOMUX(HCLK_PDVEPU, "hclk_pdvepu", "aclk_pdvepu", 0, + RV1126_CLKSEL_CON(41), 0, 5, DFLAGS, + RV1126_CLKGATE_CON(12), 2, GFLAGS), + GATE(ACLK_VENC, "aclk_venc", "aclk_pdvepu", 0, + RV1126_CLKGATE_CON(12), 5, GFLAGS), + GATE(HCLK_VENC, "hclk_venc", "hclk_pdvepu", 0, + RV1126_CLKGATE_CON(12), 6, GFLAGS), + COMPOSITE(CLK_VENC_CORE, "clk_venc_core", mux_cpll_gpll_hpll_p, 0, + RV1126_CLKSEL_CON(40), 14, 2, MFLAGS, 8, 5, DFLAGS, + RV1126_CLKGATE_CON(12), 1, GFLAGS), + GATE(0, "aclk_pdvepu_niu", "aclk_pdvepu", CLK_IGNORE_UNUSED, + RV1126_CLKGATE_CON(12), 3, GFLAGS), + GATE(0, "hclk_pdvepu_niu", "hclk_pdvepu", CLK_IGNORE_UNUSED, + RV1126_CLKGATE_CON(12), 4, GFLAGS), + /* * Clock-Architecture Diagram 9 */ -- 2.56.0 _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 3/6] media: rockchip: Add RKVENC v1 video encoder driver 2026-11-14 14:54 ` Daniel Golle @ 2026-11-14 14:54 ` Daniel Golle -1 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add a stateful V4L2 memory-to-memory encoder driver for the RKVENC v1 H.264 and HEVC encoder of the RV1109/RV1126 ("vepu541") and the RK3566/RK3568 ("vepu540"). The driver takes NV12 or NV12M and produces low-delay streams of IDR and P pictures with one short-term reference, POC type 0 and no B frames. It writes the SPS, PPS and VPS itself, the hardware writes the slice headers and data. The SEPARATE and JOINED_WITH_1ST_FRAME header modes are supported, the latter also with PREPEND_SPSPPS_TO_IDR. Pictures too small for P frames are coded as IDR pictures only. The register image builder takes a struct rkvenc_frame with no V4L2 types, so that a stateless front end can drive the same hardware back end; rkvenc-v4l2.c is the stateful front end on the mem2mem framework. Rate control is frame level CBR: the hardware row rate control stays off and every picture is coded at one QP. An I picture takes the mean P QP of the previous GOP lowered by 4 for HEVC and 2 for H.264, the P QP moves by at most one per picture, and the deviation from the plan is taken at GOP boundaries and spread over the next GOP. A bitstream overflow is fed to the rate control and the picture is encoded once more. A failed job is stopped with a force clear, and with a safe clear and a block reset only when that does not stop it. Source frames are imported without a kernel mapping and a capture buffer is sized to half the pixel count, so that 4K streams fit the vmalloc space of 32-bit systems. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- MAINTAINERS | 1 + drivers/media/platform/rockchip/Kconfig | 1 + drivers/media/platform/rockchip/Makefile | 1 + .../media/platform/rockchip/rkvenc/Kconfig | 17 + .../media/platform/rockchip/rkvenc/Makefile | 15 + .../platform/rockchip/rkvenc/rkvenc-gop.c | 72 ++ .../platform/rockchip/rkvenc/rkvenc-h264.c | 82 ++ .../platform/rockchip/rkvenc/rkvenc-hevc.c | 63 + .../platform/rockchip/rkvenc/rkvenc-nal.c | 351 ++++++ .../platform/rockchip/rkvenc/rkvenc-nal.h | 12 + .../platform/rockchip/rkvenc/rkvenc-rc.c | 214 ++++ .../platform/rockchip/rkvenc/rkvenc-ref.c | 114 ++ .../platform/rockchip/rkvenc/rkvenc-v4l2.c | 1069 +++++++++++++++++ .../rockchip/rkvenc/rkvenc-vepu541-h264.c | 206 ++++ .../rockchip/rkvenc/rkvenc-vepu541-hevc.c | 182 +++ .../rockchip/rkvenc/rkvenc-vepu541-regs.h | 283 +++++ .../rockchip/rkvenc/rkvenc-vepu541-tables.c | 256 ++++ .../platform/rockchip/rkvenc/rkvenc-vepu541.c | 215 ++++ .../platform/rockchip/rkvenc/rkvenc-vepu541.h | 201 ++++ .../media/platform/rockchip/rkvenc/rkvenc.c | 489 ++++++++ .../media/platform/rockchip/rkvenc/rkvenc.h | 216 ++++ 21 files changed, 4060 insertions(+) create mode 100644 drivers/media/platform/rockchip/rkvenc/Kconfig create mode 100644 drivers/media/platform/rockchip/rkvenc/Makefile create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.h diff --git a/MAINTAINERS b/MAINTAINERS index 26991dd3a3569..a70ee0cf2655b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -24550,6 +24550,7 @@ L: linux-media@vger.kernel.org L: linux-rockchip@lists.infradead.org S: Maintained F: Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml +F: drivers/media/platform/rockchip/rkvenc/ ROCKCHIP RK3308 INTERNAL AUDIO CODEC M: Luca Ceresoli <luca.ceresoli@bootlin.com> diff --git a/drivers/media/platform/rockchip/Kconfig b/drivers/media/platform/rockchip/Kconfig index ba401d32f01ba..e6afa60923f1b 100644 --- a/drivers/media/platform/rockchip/Kconfig +++ b/drivers/media/platform/rockchip/Kconfig @@ -6,3 +6,4 @@ source "drivers/media/platform/rockchip/rga/Kconfig" source "drivers/media/platform/rockchip/rkcif/Kconfig" source "drivers/media/platform/rockchip/rkisp1/Kconfig" source "drivers/media/platform/rockchip/rkvdec/Kconfig" +source "drivers/media/platform/rockchip/rkvenc/Kconfig" diff --git a/drivers/media/platform/rockchip/Makefile b/drivers/media/platform/rockchip/Makefile index 0e0b2cbbd4bd2..6a1d847a10a43 100644 --- a/drivers/media/platform/rockchip/Makefile +++ b/drivers/media/platform/rockchip/Makefile @@ -3,3 +3,4 @@ obj-y += rga/ obj-y += rkcif/ obj-y += rkisp1/ obj-y += rkvdec/ +obj-y += rkvenc/ diff --git a/drivers/media/platform/rockchip/rkvenc/Kconfig b/drivers/media/platform/rockchip/rkvenc/Kconfig new file mode 100644 index 0000000000000..1d76314a2ea6d --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/Kconfig @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0-only +config VIDEO_ROCKCHIP_RKVENC + tristate "Rockchip RKVENC (vepu540/vepu541) H.264/HEVC encoder" + depends on ARCH_ROCKCHIP || COMPILE_TEST + depends on VIDEO_DEV + select MEDIA_CONTROLLER + select VIDEOBUF2_DMA_CONTIG + select V4L2_MEM2MEM_DEV + help + Support for the Rockchip RKVENC v1 hardware video encoder found + in the RV1109/RV1126 (vepu541) and RK3566/RK3568 (vepu540) SoCs. + + It encodes NV12 raw frames into H.264 (AVC) or HEVC (H.265) + bitstreams using a stateful memory-to-memory V4L2 interface. + + To compile this driver as a module, choose M here: the module + will be called rockchip-rkvenc. diff --git a/drivers/media/platform/rockchip/rkvenc/Makefile b/drivers/media/platform/rockchip/rkvenc/Makefile new file mode 100644 index 0000000000000..026c81aec492c --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/Makefile @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +rockchip-rkvenc-y += rkvenc.o \ + rkvenc-v4l2.o \ + rkvenc-gop.o \ + rkvenc-ref.o \ + rkvenc-rc.o \ + rkvenc-h264.o \ + rkvenc-hevc.o \ + rkvenc-nal.o \ + rkvenc-vepu541.o \ + rkvenc-vepu541-h264.o \ + rkvenc-vepu541-hevc.o \ + rkvenc-vepu541-tables.o + +obj-$(CONFIG_VIDEO_ROCKCHIP_RKVENC) += rockchip-rkvenc.o diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c new file mode 100644 index 0000000000000..208bb122a3f87 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Low-delay IDR + P structure of the RKVENC v1 encoder: one short-term + * reference, no B frames. A failed frame never commits, so rollback is + * free. + */ + +#include "rkvenc.h" + +void rkvenc_gop_init(struct rkvenc_ctx *ctx) +{ + ctx->gop.frames_since_idr = 0; + ctx->gop.idr_pic_id = 0; + ctx->cur = 0; + ctx->last = 0; + ctx->last_good_valid = false; + ctx->consecutive_errors = 0; +} + +/* The hardware codes P frames of these sizes wrongly */ +static bool rkvenc_gop_intra_only(struct rkvenc_ctx *ctx) +{ + u32 w16 = ALIGN(ctx->width, 16) / 16; + u32 h16 = ALIGN(ctx->height, 16) / 16; + + if (ctx->codec == RKVENC_CODEC_HEVC) + return ctx->height <= 64; + + return !((w16 >= 3 && h16 >= 2) || (w16 == 1 && h16 == 1)); +} + +bool rkvenc_gop_is_idr(struct rkvenc_ctx *ctx) +{ + struct rkvenc_gop *g = &ctx->gop; + + if (g->frames_since_idr == 0 || rkvenc_gop_intra_only(ctx)) + return true; + if (ctx->c.gop_size && g->frames_since_idr >= ctx->c.gop_size) + return true; + if (ctx->c.force_keyframe) + return true; + if (!ctx->last_good_valid) + return true; + return false; +} + +void rkvenc_gop_commit(struct rkvenc_ctx *ctx) +{ + struct rkvenc_gop *g = &ctx->gop; + + if (ctx->cur_intra) { + g->idr_pic_id++; + ctx->last_good = ctx->cur; + ctx->last_good_valid = true; + g->frames_since_idr = 1; + } else { + g->frames_since_idr++; + } + ctx->last = ctx->cur; + ctx->cur ^= 1; + ctx->c.force_keyframe = false; +} + +void rkvenc_gop_rollback(struct rkvenc_ctx *ctx) +{ + ctx->c.force_keyframe = false; +} + +void rkvenc_gop_force_keyframe(struct rkvenc_ctx *ctx) +{ + ctx->c.force_keyframe = true; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c new file mode 100644 index 0000000000000..391755a42dd3b --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * H.264 parameter sets of the RKVENC v1 encoder. One struct + * rkvenc_h264_seq feeds both the slice header registers and the SPS/PPS + * writer, so the two cannot disagree. + */ + +#include <linux/string.h> + +#include "rkvenc.h" +#include "rkvenc-nal.h" + +static const u8 rkvenc_h264_level_idc[] = { + 10, 11, 11, 12, 13, 20, 21, 22, 30, 31, 32, 40, + 41, 42, 50, 51, 52, 60, 61, 62, +}; + +static u8 rkvenc_h264_level(u32 ctrl) +{ + if (ctrl >= ARRAY_SIZE(rkvenc_h264_level_idc)) + return 62; + return rkvenc_h264_level_idc[ctrl]; +} + +static u8 rkvenc_h264_profile_idc(u32 ctrl) +{ + switch (ctrl) { + case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE: + case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE: + return 66; + case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN: + return 77; + case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH: + default: + return 100; + } +} + +int rkvenc_h264_setup(struct rkvenc_ctx *ctx) +{ + struct rkvenc_h264_seq *seq = &ctx->h264_seq; + + memset(seq, 0, sizeof(*seq)); + seq->width = ctx->width; + seq->height = ctx->height; + seq->profile_idc = rkvenc_h264_profile_idc(ctx->c.profile); + seq->level_idc = rkvenc_h264_level(ctx->c.level); + seq->log2_max_frame_num_m4 = 4; + seq->log2_max_poc_lsb_m4 = 5; + seq->pic_init_qp = ctx->c.p_frame_qp; + seq->chroma_qp_offset = ctx->c.chroma_qp_offset; + seq->cabac = ctx->c.cabac && seq->profile_idc != 66; + if (ctx->c.profile == V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) + seq->transform_8x8 = ctx->c.transform_8x8; + seq->dblk_idc = ctx->c.lf_mode; + seq->alpha_offset = ctx->c.lf_alpha; + seq->beta_offset = ctx->c.lf_beta; + seq->timing = true; + seq->fps = ctx->fps ?: 30; + + return rkvenc_h264_headers_write(seq, ctx->hdr, RKVENC_HDR_MAX); +} + +int rkvenc_h264_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size) +{ + if (ctx->hdr_len > size) + return -ENOSPC; + memcpy(buf, ctx->hdr, ctx->hdr_len); + return ctx->hdr_len; +} + +void rkvenc_h264_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_h264_pic *pic, bool idr) +{ + struct rkvenc_gop *g = &ctx->gop; + + pic->seq = &ctx->h264_seq; + pic->nal_ref_idc = idr ? 3 : 2; + pic->frame_num = g->frames_since_idr & 0xff; + pic->poc_lsb = (2 * g->frames_since_idr) & 0x1ff; + pic->idr_pic_id = g->idr_pic_id & 0xffff; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c new file mode 100644 index 0000000000000..49bf3cccdf8a0 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * HEVC parameter sets of the RKVENC v1 encoder. One struct + * rkvenc_hevc_seq feeds both the slice header registers and the + * VPS/SPS/PPS writer. + */ + +#include <linux/string.h> + +#include "rkvenc.h" +#include "rkvenc-nal.h" + +#define HEVC_NAL_TRAIL_R 1 +#define HEVC_NAL_IDR_W_RADL 19 + +static const u16 rkvenc_hevc_level_idc[] = { + 30, 60, 63, 90, 93, 120, 123, 150, 153, 156, 180, 183, 186, +}; + +static u16 rkvenc_hevc_level(u32 ctrl) +{ + if (ctrl >= ARRAY_SIZE(rkvenc_hevc_level_idc)) + return 186; + return rkvenc_hevc_level_idc[ctrl]; +} + +int rkvenc_hevc_setup(struct rkvenc_ctx *ctx) +{ + struct rkvenc_hevc_seq *seq = &ctx->hevc_seq; + + memset(seq, 0, sizeof(*seq)); + seq->width = ctx->width; + seq->height = ctx->height; + seq->level_idc = rkvenc_hevc_level(ctx->c.level); + seq->log2_max_poc_lsb_m4 = 4; + seq->pic_init_qp = ctx->c.p_frame_qp; + seq->max_merge_cand = ctx->c.hevc_max_merge ?: 2; + seq->sao = ctx->c.hevc_sao; + seq->deblock_disabled = ctx->c.lf_mode == + V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_DISABLED; + seq->beta_offset = ctx->c.hevc_beta_offset; + seq->tc_offset = ctx->c.hevc_tc_offset; + seq->strong_intra_smoothing = ctx->c.hevc_strong_intra; + seq->cabac_init_present = ctx->c.hevc_cabac_init; + + return rkvenc_hevc_headers_write(seq, ctx->hdr, RKVENC_HDR_MAX); +} + +int rkvenc_hevc_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size) +{ + if (ctx->hdr_len > size) + return -ENOSPC; + memcpy(buf, ctx->hdr, ctx->hdr_len); + return ctx->hdr_len; +} + +void rkvenc_hevc_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_hevc_pic *pic, bool idr) +{ + pic->seq = &ctx->hevc_seq; + pic->nal_unit_type = idr ? HEVC_NAL_IDR_W_RADL : HEVC_NAL_TRAIL_R; + pic->poc_lsb = ctx->gop.frames_since_idr & 0xff; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c new file mode 100644 index 0000000000000..b5a3f01852c0b --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c @@ -0,0 +1,351 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * H.264 SPS/PPS (ITU-T H.264 7.3.2) and HEVC VPS/SPS/PPS (ITU-T H.265 7.3.2) + * writer for the RKVENC v1 encoder. + */ + +#include "rkvenc-nal.h" + +#define NAL_RBSP_MAX 256 + +#define H264_NAL_SPS 7 +#define H264_NAL_PPS 8 +#define H264_PROFILE_HIGH 100 +#define HEVC_NAL_VPS 32 +#define HEVC_NAL_SPS 33 +#define HEVC_NAL_PPS 34 + +struct nal_bw { + u8 buf[NAL_RBSP_MAX]; + u32 bits; + bool overflow; +}; + +static void bw_put(struct nal_bw *bw, u32 val, u32 bits) +{ + u32 i, bit; + + for (i = bits; i > 0; i--) { + if (bw->bits >= 8 * NAL_RBSP_MAX) { + bw->overflow = true; + return; + } + bit = (val >> (i - 1)) & 1; + if (bit) + bw->buf[bw->bits / 8] |= 0x80 >> (bw->bits % 8); + bw->bits++; + } +} + +static void bw_ue(struct nal_bw *bw, u32 val) +{ + u32 len = 32 - __builtin_clz(val + 1); + + bw_put(bw, 0, len - 1); + bw_put(bw, val + 1, len); +} + +static void bw_se(struct nal_bw *bw, s32 val) +{ + bw_ue(bw, val > 0 ? 2 * val - 1 : -2 * val); +} + +static void bw_trailing(struct nal_bw *bw) +{ + bw_put(bw, 1, 1); + while (bw->bits % 8) + bw_put(bw, 0, 1); +} + +static int nal_emit(const struct nal_bw *bw, const u8 *hdr, u32 hdr_len, + u8 *out, u32 size) +{ + u32 len = bw->bits / 8, pos = 0, zeros = 0, i; + + if (bw->overflow || size < 4 + hdr_len) + return -ENOSPC; + + out[pos++] = 0; + out[pos++] = 0; + out[pos++] = 0; + out[pos++] = 1; + for (i = 0; i < hdr_len; i++) + out[pos++] = hdr[i]; + + /* Emulation prevention, H.264 7.4.1 */ + for (i = 0; i < len; i++) { + if (pos + 2 > size) + return -ENOSPC; + if (zeros == 2 && bw->buf[i] <= 3) { + out[pos++] = 3; + zeros = 0; + } + out[pos++] = bw->buf[i]; + zeros = bw->buf[i] ? 0 : zeros + 1; + } + + return pos; +} + +static void h264_vui_write(struct nal_bw *bw, const struct rkvenc_h264_seq *seq) +{ + bw_put(bw, 0, 4); + bw_put(bw, seq->timing, 1); + if (seq->timing) { + bw_put(bw, 1, 32); + bw_put(bw, 2 * seq->fps, 32); + bw_put(bw, 1, 1); + } + bw_put(bw, 0, 3); + bw_put(bw, 1, 1); /* bitstream_restriction_flag */ + bw_put(bw, 1, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, 16); + bw_ue(bw, 16); + bw_ue(bw, 0); /* max_num_reorder_frames */ + bw_ue(bw, 1); /* max_dec_frame_buffering */ +} + +static void h264_sps_write(struct nal_bw *bw, const struct rkvenc_h264_seq *seq) +{ + u32 w16 = ALIGN(seq->width, 16), h16 = ALIGN(seq->height, 16); + bool crop = w16 != seq->width || h16 != seq->height; + u32 constraints; + + if (seq->profile_idc == H264_PROFILE_HIGH) + constraints = 0; + else if (seq->profile_idc == 66) + constraints = 0xc0; + else + constraints = 0x40; + + bw_put(bw, seq->profile_idc, 8); + bw_put(bw, constraints, 8); + bw_put(bw, seq->level_idc, 8); + bw_ue(bw, 0); + if (seq->profile_idc == H264_PROFILE_HIGH) { + bw_ue(bw, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 2); + } + bw_ue(bw, seq->log2_max_frame_num_m4); + bw_ue(bw, 0); + bw_ue(bw, seq->log2_max_poc_lsb_m4); + bw_ue(bw, 1); + bw_put(bw, 0, 1); + bw_ue(bw, w16 / 16 - 1); + bw_ue(bw, h16 / 16 - 1); + bw_put(bw, 1, 1); + bw_put(bw, 1, 1); + bw_put(bw, crop, 1); + if (crop) { + bw_ue(bw, 0); + bw_ue(bw, (w16 - seq->width) / 2); + bw_ue(bw, 0); + bw_ue(bw, (h16 - seq->height) / 2); + } + bw_put(bw, 1, 1); + h264_vui_write(bw, seq); + bw_trailing(bw); +} + +static void h264_pps_write(struct nal_bw *bw, const struct rkvenc_h264_seq *seq) +{ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, seq->cabac, 1); + bw_put(bw, 0, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 3); + bw_se(bw, seq->pic_init_qp - 26); + bw_se(bw, 0); + bw_se(bw, seq->chroma_qp_offset); + bw_put(bw, 1, 1); + bw_put(bw, 0, 2); + if (seq->transform_8x8) { + bw_put(bw, 1, 1); + bw_put(bw, 0, 1); + bw_se(bw, seq->chroma_qp_offset); + } + bw_trailing(bw); +} + +int rkvenc_h264_headers_write(const struct rkvenc_h264_seq *seq, u8 *buf, + u32 size) +{ + u8 hdr; + struct nal_bw bw; + int len, pos; + + memset(&bw, 0, sizeof(bw)); + h264_sps_write(&bw, seq); + hdr = 3 << 5 | H264_NAL_SPS; + len = nal_emit(&bw, &hdr, 1, buf, size); + if (len < 0) + return len; + pos = len; + + memset(&bw, 0, sizeof(bw)); + h264_pps_write(&bw, seq); + hdr = 3 << 5 | H264_NAL_PPS; + len = nal_emit(&bw, &hdr, 1, buf + pos, size - pos); + if (len < 0) + return len; + + return pos + len; +} + +static void hevc_ptl_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + bw_put(bw, 0, 2); + bw_put(bw, 0, 1); + bw_put(bw, 1, 5); /* general_profile_idc: Main */ + bw_put(bw, 0x60000000, 32); /* compatible with profiles 1, 2 */ + bw_put(bw, 1, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 1, 1); + bw_put(bw, 0, 32); + bw_put(bw, 0, 12); + bw_put(bw, seq->level_idc, 8); +} + +static void hevc_vps_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + bw_put(bw, 0, 4); + bw_put(bw, 1, 1); + bw_put(bw, 1, 1); + bw_put(bw, 0, 6); + bw_put(bw, 0, 3); + bw_put(bw, 1, 1); + bw_put(bw, 0xffff, 16); + hevc_ptl_write(bw, seq); + bw_put(bw, 1, 1); + bw_ue(bw, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 6); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_trailing(bw); +} + +static void hevc_sps_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + u32 w8 = ALIGN(seq->width, 8), h8 = ALIGN(seq->height, 8); + bool crop = w8 != seq->width || h8 != seq->height; + + bw_put(bw, 0, 4); + bw_put(bw, 0, 3); + bw_put(bw, 1, 1); + hevc_ptl_write(bw, seq); + bw_ue(bw, 0); + bw_ue(bw, 1); /* chroma_format_idc 4:2:0 */ + bw_ue(bw, w8); + bw_ue(bw, h8); + bw_put(bw, crop, 1); + if (crop) { + bw_ue(bw, 0); + bw_ue(bw, (w8 - seq->width) / 2); + bw_ue(bw, 0); + bw_ue(bw, (h8 - seq->height) / 2); + } + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, seq->log2_max_poc_lsb_m4); + bw_put(bw, 1, 1); + bw_ue(bw, 1); /* sps_max_dec_pic_buffering_minus1 */ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, 0); /* min CU 8x8 */ + bw_ue(bw, 3); /* CTU 64x64 */ + bw_ue(bw, 0); /* min TU 4x4 */ + bw_ue(bw, 3); /* max TU 32x32 */ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, seq->sao, 1); + bw_put(bw, 0, 1); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, seq->strong_intra_smoothing, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_trailing(bw); +} + +static void hevc_pps_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 3); + bw_put(bw, 0, 1); + bw_put(bw, seq->cabac_init_present, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_se(bw, seq->pic_init_qp - 26); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 1, 1); /* cu_qp_delta_enabled_flag */ + bw_ue(bw, 0); + bw_se(bw, 0); + bw_se(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 1, 1); + bw_put(bw, 0, 1); + bw_put(bw, seq->deblock_disabled, 1); + if (!seq->deblock_disabled) { + bw_se(bw, seq->beta_offset); + bw_se(bw, seq->tc_offset); + } + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_trailing(bw); +} + +int rkvenc_hevc_headers_write(const struct rkvenc_hevc_seq *seq, u8 *buf, + u32 size) +{ + static const u8 types[] = { HEVC_NAL_VPS, HEVC_NAL_SPS, HEVC_NAL_PPS }; + struct nal_bw bw; + int len, pos; + u8 hdr[2]; + u32 i; + + pos = 0; + for (i = 0; i < ARRAY_SIZE(types); i++) { + memset(&bw, 0, sizeof(bw)); + if (types[i] == HEVC_NAL_VPS) + hevc_vps_write(&bw, seq); + else if (types[i] == HEVC_NAL_SPS) + hevc_sps_write(&bw, seq); + else + hevc_pps_write(&bw, seq); + hdr[0] = types[i] << 1; + hdr[1] = 1; + len = nal_emit(&bw, hdr, 2, buf + pos, size - pos); + if (len < 0) + return len; + pos += len; + } + + return pos; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h new file mode 100644 index 0000000000000..58b30d2ac6124 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef RKVENC_NAL_H +#define RKVENC_NAL_H + +#include "rkvenc-vepu541.h" + +int rkvenc_h264_headers_write(const struct rkvenc_h264_seq *seq, u8 *buf, + u32 size); +int rkvenc_hevc_headers_write(const struct rkvenc_hevc_seq *seq, u8 *buf, + u32 size); + +#endif diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c new file mode 100644 index 0000000000000..b7ab57e490cf0 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c @@ -0,0 +1,214 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Frame-level rate control of the RKVENC v1 encoder. The hardware row-level + * rate control is off, every picture is coded at one QP picked here. + */ + +#include <linux/math.h> +#include <linux/math64.h> + +#include "rkvenc.h" + +/* qs[q] = round(256 * 2^((q-4)/6)) in Q8, q = 0..51. */ +#define RKVENC_QS_BASE { 256, 287, 323, 362, 406, 456 } + +#define RKVENC_RC_DEFAULT_FPS 30u + +/* I/P complexity ratio cap, Q8; static scenes reach 37 */ +#define RKVENC_RC_W_CLAMP (64u * 256) +/* A larger step makes the P QP alternate between two values */ +#define RKVENC_RC_QP_STEP 1 +/* Q8 weight of the newest P frame in the P complexity */ +#define RKVENC_RC_X_WEIGHT 32 + +/* I frame QP offset below the mean P frame QP */ +#define RKVENC_RC_QP_DELTA_IP_HEVC 4 +#define RKVENC_RC_QP_DELTA_IP_H264 2 + +static u32 rkvenc_qs[RKVENC_QP_MAX + 1]; + +static void rkvenc_qs_build(void) +{ + static const u32 base[6] = RKVENC_QS_BASE; + unsigned int q; + int k, sh, m; + u64 v; + + for (q = 0; q <= RKVENC_QP_MAX; q++) { + k = (int)q - 4; + /* floor(k / 6), correct for negative k. */ + sh = (k - (k < 0 ? 5 : 0)) / 6; + m = k - 6 * sh; + if (sh >= 0) + v = (u64)base[m] << sh; + else + v = (u64)base[m] >> (-sh); + rkvenc_qs[q] = (u32)v; + } +} + +static u32 rkvenc_rc_capacity(struct rkvenc_ctx *ctx) +{ + if (ctx->c.vbv_size) + return ctx->c.vbv_size; + return ctx->rc_tavg * max_t(u32, ctx->c.gop_size, 1); +} + +/* The plan budgets the I frame, so the bucket is on plan at GOP starts */ +static u64 rkvenc_rc_correct(struct rkvenc_ctx *ctx, u64 t_t, u32 n) +{ + s64 c = div_s64(ctx->rc_buf_gop, n); + + if (c < 0) + return t_t + (u64)-c; + if ((u64)c >= t_t) + return 1; + return t_t - (u64)c; +} + +static u32 rkvenc_rc_complexity(struct rkvenc_ctx *ctx, unsigned int t, u64 s, + u32 qp_avg) +{ + u32 x = (u32)div_u64(s * rkvenc_qs[min(qp_avg, RKVENC_QP_MAX)], 256); + + /* a P frame below the skip threshold alone would swing the model */ + if (!t || ctx->rc_first[1]) + return x; + return (u32)div_u64((u64)ctx->rc_x[1] * (256 - RKVENC_RC_X_WEIGHT) + + (u64)x * RKVENC_RC_X_WEIGHT, + 256); +} + +static int rkvenc_rc_p_qp_mean(struct rkvenc_ctx *ctx) +{ + int q = ctx->rc_qp[1]; + + if (ctx->rc_p_qp_cnt) + q = DIV_ROUND_CLOSEST(ctx->rc_p_qp_sum, ctx->rc_p_qp_cnt); + ctx->rc_p_qp_sum = 0; + ctx->rc_p_qp_cnt = 0; + return q; +} + +void rkvenc_rc_init(struct rkvenc_ctx *ctx) +{ + u32 bitrate = ctx->c.bitrate ?: 4000000; + u32 fps = ctx->fps ?: RKVENC_RC_DEFAULT_FPS; + + rkvenc_qs_build(); + + ctx->rc_tavg = div_u64((u64)bitrate, fps); + if (!ctx->rc_tavg) + ctx->rc_tavg = 1; + ctx->rc_buf = 0; + ctx->rc_buf_gop = 0; + ctx->rc_p_qp_sum = 0; + ctx->rc_p_qp_cnt = 0; + ctx->rc_x[0] = ctx->rc_x[1] = 0; + ctx->rc_qp[0] = ctx->c.i_frame_qp; + ctx->rc_qp[1] = ctx->c.p_frame_qp; + ctx->rc_first[0] = true; + ctx->rc_first[1] = true; +} + +void rkvenc_rc_pick(struct rkvenc_ctx *ctx, u8 *qp, u8 frame_type) +{ + unsigned int t = frame_type ? 1 : 0; + u8 lo = ctx->c.min_qp; + u8 hi = ctx->c.max_qp; + s32 prev = ctx->rc_qp[t]; + u32 target; + int q; + + if (!ctx->rc_active) { + *qp = frame_type ? ctx->c.p_frame_qp : ctx->c.i_frame_qp; + return; + } + + if (ctx->rc_first[t]) { + *qp = frame_type ? ctx->c.p_frame_qp : ctx->c.i_frame_qp; + return; + } + + if (!t) + ctx->rc_buf_gop = ctx->rc_buf; + + /* An I frame coarser than the P frames of the GOP shows as a pulse */ + if (!t && !ctx->rc_first[1]) { + q = rkvenc_rc_p_qp_mean(ctx); + if (ctx->codec == RKVENC_CODEC_HEVC) + q -= RKVENC_RC_QP_DELTA_IP_HEVC; + else + q -= RKVENC_RC_QP_DELTA_IP_H264; + q = clamp_t(int, q, lo, hi); + ctx->rc_qp[0] = (s8)q; + *qp = q; + return; + } + + if (ctx->rc_x[0] && ctx->rc_x[1]) { + u64 w = div_u64((u64)ctx->rc_x[0] * 256, + ctx->rc_x[1] ?: 1); + u32 n = max_t(u32, ctx->c.gop_size, 1); + u64 t_p, t_t; + + w = min_t(u64, w, RKVENC_RC_W_CLAMP); + t_p = div64_u64((u64)ctx->rc_tavg * n * 256, + w + 256 * (n - 1)); + t_t = t ? t_p : t_p * w / 256; + t_t = rkvenc_rc_correct(ctx, t_t, n); + + target = (u32)div_u64((u64)ctx->rc_x[t] * 256, t_t); + + for (q = 0; q <= RKVENC_QP_MAX; q++) + if (rkvenc_qs[q] >= target) + break; + if (q > RKVENC_QP_MAX) + q = RKVENC_QP_MAX; + } else { + q = prev; + } + + q = clamp_t(int, q, lo, hi); + if (q > prev + RKVENC_RC_QP_STEP) + q = prev + RKVENC_RC_QP_STEP; + if (q < prev - RKVENC_RC_QP_STEP) + q = prev - RKVENC_RC_QP_STEP; + q = clamp_t(int, q, lo, hi); + + ctx->rc_qp[t] = (s8)q; + if (t) { + ctx->rc_p_qp_sum += q; + ctx->rc_p_qp_cnt++; + } + *qp = q; +} + +void rkvenc_rc_update(struct rkvenc_ctx *ctx, u32 bytes, u32 qp_avg, + u8 frame_type) +{ + unsigned int t = frame_type ? 1 : 0; + u64 s = (u64)bytes * 8; + s64 cap = max_t(u64, rkvenc_rc_capacity(ctx), 1); + + if (!ctx->rc_active) + return; + + ctx->rc_x[t] = rkvenc_rc_complexity(ctx, t, s, qp_avg); + + ctx->rc_buf += (s32)s - (s32)ctx->rc_tavg; + ctx->rc_buf = clamp_t(s64, ctx->rc_buf, -cap, cap); + + ctx->rc_first[t] = false; +} + +/* The step limit alone would need frames that the overflow keeps dropping */ +void rkvenc_rc_overflow(struct rkvenc_ctx *ctx, u32 bytes, u8 frame_type) +{ + s8 start = frame_type ? ctx->c.p_frame_qp : ctx->c.i_frame_qp; + unsigned int t = frame_type ? 1 : 0; + + rkvenc_rc_update(ctx, bytes, ctx->cur_qp, frame_type); + if (ctx->rc_qp[t] < start) + ctx->rc_qp[t] = start; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c new file mode 100644 index 0000000000000..b0f15d31f1e11 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c @@ -0,0 +1,114 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Reconstruction and reference buffers of the RKVENC v1 encoder: two slots + * alternate as the single short-term reference. + */ + +#include <linux/dma-mapping.h> +#include <linux/slab.h> + +#include "rkvenc.h" + +enum { + RKVENC_BUF_HDR = 0, + RKVENC_BUF_BODY, + RKVENC_BUF_DSP, + RKVENC_BUF_CMV, +}; + +static void rkvenc_ref_free_slot(struct rkvenc_ctx *ctx, unsigned int i) +{ + struct device *dev = ctx->dev->dev; + size_t sizes[RKVENC_SLOT_BUFS]; + u32 handles[RKVENC_SLOT_BUFS]; + unsigned int b; + + sizes[RKVENC_BUF_HDR] = ctx->slot_sz.rfp_hdr; + sizes[RKVENC_BUF_BODY] = ctx->slot_sz.rfp_body; + sizes[RKVENC_BUF_DSP] = ctx->slot_sz.dsp; + sizes[RKVENC_BUF_CMV] = ctx->slot_sz.cmv; + handles[RKVENC_BUF_HDR] = ctx->slot[i].rfp_hdr; + handles[RKVENC_BUF_BODY] = ctx->slot[i].rfp_body; + handles[RKVENC_BUF_DSP] = ctx->slot[i].dsp; + handles[RKVENC_BUF_CMV] = ctx->slot[i].cmv; + + for (b = 0; b < RKVENC_SLOT_BUFS; b++) { + if (ctx->slot_cpu[i][b]) + dma_free_attrs(dev, sizes[b], ctx->slot_cpu[i][b], + handles[b], DMA_ATTR_NO_KERNEL_MAPPING); + ctx->slot_cpu[i][b] = NULL; + } + memset(&ctx->slot[i], 0, sizeof(ctx->slot[i])); +} + +int rkvenc_ref_alloc(struct rkvenc_ctx *ctx) +{ + struct device *dev = ctx->dev->dev; + struct rkvenc_slot_sizes *sz = &ctx->slot_sz; + dma_addr_t dma; + void *cpu; + unsigned int i; + int ret = 0; + + if (ctx->slots_alloc) + return 0; + + rkvenc_vepu541_slot_sizes(ctx->codec, ctx->width, ctx->height, sz); + + for (i = 0; i < RKVENC_SLOTS; i++) { + cpu = dma_alloc_attrs(dev, sz->rfp_hdr, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_HDR] = cpu; + ctx->slot[i].rfp_hdr = lower_32_bits(dma); + + cpu = dma_alloc_attrs(dev, sz->rfp_body, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_BODY] = cpu; + ctx->slot[i].rfp_body = lower_32_bits(dma); + + cpu = dma_alloc_attrs(dev, sz->dsp, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_DSP] = cpu; + ctx->slot[i].dsp = lower_32_bits(dma); + + if (ctx->codec == RKVENC_CODEC_HEVC && sz->cmv) { + cpu = dma_alloc_attrs(dev, sz->cmv, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_CMV] = cpu; + ctx->slot[i].cmv = lower_32_bits(dma); + } else { + ctx->slot[i].cmv = 0; + } + } + + ctx->slots_alloc = true; + return 0; + +err: + for (i = 0; i < RKVENC_SLOTS; i++) + rkvenc_ref_free_slot(ctx, i); + ret = -ENOMEM; + return ret; +} + +void rkvenc_ref_free(struct rkvenc_ctx *ctx) +{ + unsigned int i; + + if (!ctx->slots_alloc) + return; + + for (i = 0; i < RKVENC_SLOTS; i++) + rkvenc_ref_free_slot(ctx, i); + ctx->slots_alloc = false; + ctx->cur = 0; + ctx->last = 0; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c new file mode 100644 index 0000000000000..3be496f5cf0c1 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c @@ -0,0 +1,1069 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * V4L2 stateful encoder front end of the Rockchip RKVENC v1 encoder. + */ + +#include <linux/dma-mapping.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> +#include <media/v4l2-ctrls.h> +#include <media/v4l2-event.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-mem2mem.h> +#include <media/videobuf2-dma-contig.h> + +#include "rkvenc.h" + +static inline struct rkvenc_ctx *fh_to_rkvenc_ctx(void *priv) +{ + return container_of(priv, struct rkvenc_ctx, fh); +} + +struct rkvenc_fmt { + u32 fourcc; + u8 num_planes; +}; + +static const struct rkvenc_fmt rkvenc_output_fmts[] = { + { V4L2_PIX_FMT_NV12, 1 }, + { V4L2_PIX_FMT_NV12M, 2 }, +}; + +static const struct rkvenc_fmt rkvenc_capture_fmts[] = { + { V4L2_PIX_FMT_H264, 1 }, + { V4L2_PIX_FMT_HEVC, 1 }, +}; + +static const struct rkvenc_fmt * +rkvenc_find_fmt(const struct rkvenc_fmt *fmts, unsigned int n, u32 fourcc) +{ + unsigned int i; + + for (i = 0; i < n; i++) + if (fmts[i].fourcc == fourcc) + return &fmts[i]; + return NULL; +} + +static bool rkvenc_is_h264(u32 fourcc) +{ + return fourcc == V4L2_PIX_FMT_H264; +} + +static u32 rkvenc_ctrl_val(struct rkvenc_ctx *ctx, u32 id) +{ + struct v4l2_ctrl *ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, id); + + if (!ctrl) + return 0; + return ctrl->cur.val; +} + +/* Runs for every frame, so that control changes apply to the next one */ +static void rkvenc_apply_ctrls(struct rkvenc_ctx *ctx) +{ + struct rkvenc_ctrls *c = &ctx->c; + bool h264 = ctx->codec == RKVENC_CODEC_H264; + + c->bitrate_mode = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_BITRATE_MODE); + c->bitrate = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_BITRATE); + c->vbv_size = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_VBV_SIZE); + c->frame_rc_enable = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE); + c->gop_size = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_GOP_SIZE); + c->i_period = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_PERIOD); + c->header_mode = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEADER_MODE); + c->prepend_spspps = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR); + + if (h264) { + c->profile = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_PROFILE); + c->level = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_LEVEL); + c->cabac = rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE) == + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC; + c->transform_8x8 = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM); + c->lf_mode = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE); + c->lf_alpha = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA); + c->lf_beta = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA); + c->chroma_qp_offset = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET); + c->min_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_MIN_QP); + c->max_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_MAX_QP); + c->i_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP); + c->p_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP); + if (rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_PERIOD) > + c->gop_size) + c->gop_size = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_I_PERIOD); + } else { + c->level = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_LEVEL); + c->tier = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_TIER); + c->lf_mode = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE); + c->hevc_beta_offset = rkvenc_ctrl_val( + ctx, V4L2_CID_MPEG_VIDEO_HEVC_LF_BETA_OFFSET_DIV2); + c->hevc_tc_offset = rkvenc_ctrl_val( + ctx, V4L2_CID_MPEG_VIDEO_HEVC_LF_TC_OFFSET_DIV2); + c->min_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP); + c->max_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP); + c->i_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP); + c->p_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP); + c->hevc_max_merge = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_MAX_NUM_MERGE_MV_MINUS1); + /* There is no SAO control, SAO follows the loop filter mode */ + c->hevc_sao = + c->lf_mode != V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_DISABLED; + c->hevc_strong_intra = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_STRONG_SMOOTHING); + c->hevc_tmv = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_TMV_PREDICTION); + c->hevc_cabac_init = 0; + if (rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD) > + c->gop_size) + c->gop_size = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD); + } + + c->framerate = ctx->fps ?: 30; + ctx->rc_active = c->frame_rc_enable; +} + +static void rkvenc_refresh_header(struct rkvenc_ctx *ctx) +{ + rkvenc_apply_ctrls(ctx); + ctx->hdr_len = ctx->codec == RKVENC_CODEC_H264 ? + rkvenc_h264_setup(ctx) : rkvenc_hevc_setup(ctx); +} + +static void rkvenc_ensure_header(struct rkvenc_ctx *ctx) +{ + if (ctx->hdr_len == 0) + rkvenc_refresh_header(ctx); +} + +static int rkvenc_copy_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size) +{ + return ctx->codec == RKVENC_CODEC_H264 ? + rkvenc_h264_write_headers(ctx, buf, size) : + rkvenc_hevc_write_headers(ctx, buf, size); +} + +static void rkvenc_update_blocks(struct rkvenc_ctx *ctx) +{ + if (ctx->codec == RKVENC_CODEC_H264) { + ctx->blocks = (ALIGN(ctx->width, 16) / 16) * + (ALIGN(ctx->height, 16) / 16); + ctx->qp_blocks = ctx->blocks; + } else { + ctx->blocks = (ALIGN(ctx->width, 64) / 64) * + (ALIGN(ctx->height, 64) / 64); + ctx->qp_blocks = (ALIGN(ctx->width, 8) / 8) * + (ALIGN(ctx->height, 8) / 8); + } +} + +static u32 rkvenc_output_plane_size(struct rkvenc_ctx *ctx, unsigned int plane) +{ + u32 bpl = ctx->bytesperline; + u32 h = ctx->height; + + if (ctx->src_num_planes == 1) + return bpl * h * 3 / 2; + return plane == 0 ? bpl * h : bpl * h / 2; +} + +static u32 rkvenc_capture_size(struct rkvenc_ctx *ctx) +{ + if (ctx->codec == RKVENC_CODEC_H264) { + u32 mbw = ALIGN(ctx->width, 16) / 16; + u32 mbh = ALIGN(ctx->height, 16) / 16; + + return min(mbw * mbh * 400, ctx->width * ctx->height / 2) + + RKVENC_HDR_MAX; + } + return ctx->width * ctx->height / 2 + RKVENC_HDR_MAX; +} + +static int rkvenc_queue_setup(struct vb2_queue *vq, unsigned int *num_buffers, + unsigned int *num_planes, unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq); + const struct rkvenc_fmt *fmt; + unsigned int i; + u32 size; + + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { + fmt = rkvenc_find_fmt(rkvenc_output_fmts, + ARRAY_SIZE(rkvenc_output_fmts), + ctx->src_fourcc); + if (!fmt) + return -EINVAL; + *num_planes = fmt->num_planes; + for (i = 0; i < *num_planes; i++) + sizes[i] = rkvenc_output_plane_size(ctx, i); + return 0; + } + + fmt = rkvenc_find_fmt(rkvenc_capture_fmts, + ARRAY_SIZE(rkvenc_capture_fmts), + ctx->coded_fourcc); + if (!fmt) + return -EINVAL; + size = rkvenc_capture_size(ctx); + *num_planes = 1; + if (*num_buffers) + sizes[0] = max(sizes[0], size); + else + sizes[0] = size; + return 0; +} + +static int rkvenc_buf_prepare(struct vb2_buffer *vb) +{ + struct vb2_queue *vq = vb->vb2_queue; + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq); + + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { + if (vb2_plane_size(vb, 0) < rkvenc_output_plane_size(ctx, 0)) + return -EINVAL; + } else { + if (vb2_plane_size(vb, 0) < rkvenc_capture_size(ctx)) + return -EINVAL; + } + return 0; +} + +static void rkvenc_buf_queue(struct vb2_buffer *vb) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + + if (V4L2_TYPE_IS_CAPTURE(vb->vb2_queue->type)) { + if (vb2_is_streaming(vb->vb2_queue) && + v4l2_m2m_dst_buf_is_last(ctx->m2m_ctx)) { + vb2_set_plane_payload(vb, 0, 0); + vbuf->field = V4L2_FIELD_NONE; + v4l2_m2m_last_buffer_done(ctx->m2m_ctx, vbuf); + return; + } + + /* The parameter sets take a CAPTURE buffer of their own */ + if (ctx->header_pending && + ctx->c.header_mode == + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) { + u8 *vaddr = vb2_plane_vaddr(vb, 0); + + if (vaddr) { + rkvenc_ensure_header(ctx); + if (rkvenc_copy_headers(ctx, vaddr, + vb2_plane_size(vb, 0)) < 0) + vb2_set_plane_payload(vb, 0, 0); + else + vb2_set_plane_payload(vb, 0, + ctx->hdr_len); + } else { + vb2_set_plane_payload(vb, 0, 0); + } + vbuf->field = V4L2_FIELD_NONE; + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); + ctx->header_pending = false; + return; + } + } + + v4l2_m2m_buf_queue(ctx->m2m_ctx, vbuf); +} + +static int rkvenc_buf_out_validate(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + + vbuf->field = V4L2_FIELD_NONE; + return 0; +} + +static void rkvenc_return_dst_bufs(struct rkvenc_ctx *ctx) +{ + struct vb2_v4l2_buffer *vbuf; + + while ((vbuf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_QUEUED); +} + +static int rkvenc_start_streaming(struct vb2_queue *q, unsigned int count) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(q); + int ret; + + if (V4L2_TYPE_IS_OUTPUT(q->type)) + return 0; + + if (!ctx->coded_fourcc || !ctx->src_fourcc) { + rkvenc_return_dst_bufs(ctx); + return -EINVAL; + } + + /* Hardware bug: H.264 at most 2 MBs wide and over 1 MB high hangs */ + if (ctx->codec == RKVENC_CODEC_H264 && ALIGN(ctx->width, 16) <= 32 && + ALIGN(ctx->height, 16) > 16) { + rkvenc_return_dst_bufs(ctx); + return -EINVAL; + } + + rkvenc_update_blocks(ctx); + rkvenc_refresh_header(ctx); + + ret = rkvenc_ref_alloc(ctx); + if (ret) { + rkvenc_return_dst_bufs(ctx); + return ret; + } + + rkvenc_rc_init(ctx); + rkvenc_gop_init(ctx); + ctx->joined_header_done = false; + return 0; +} + +static void rkvenc_stop_streaming(struct vb2_queue *q) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(q); + struct vb2_v4l2_buffer *vbuf; + + v4l2_m2m_update_stop_streaming_state(ctx->m2m_ctx, q); + + if (V4L2_TYPE_IS_OUTPUT(q->type)) { + while ((vbuf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx))) + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + } else { + while ((vbuf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + + rkvenc_ref_free(ctx); + } +} + +static const struct vb2_ops rkvenc_queue_ops = { + .queue_setup = rkvenc_queue_setup, + .buf_prepare = rkvenc_buf_prepare, + .buf_queue = rkvenc_buf_queue, + .buf_out_validate = rkvenc_buf_out_validate, + .start_streaming = rkvenc_start_streaming, + .stop_streaming = rkvenc_stop_streaming, +}; + +static void rkvenc_device_run(void *priv) +{ + struct rkvenc_ctx *ctx = priv; + struct rkvenc_dev *dev = ctx->dev; + struct vb2_v4l2_buffer *src, *dst; + struct rkvenc_frame f = {}; + u8 qp; + bool idr; + u32 hdr_len = 0; + u8 *vaddr; + int ret; + + dev->job_ctx = ctx; + dev->job_running = true; + + src = v4l2_m2m_next_src_buf(ctx->m2m_ctx); + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + if (!src || !dst) + goto finish_error; + + idr = rkvenc_gop_is_idr(ctx); + + if (idr) + ctx->gop.frames_since_idr = 0; + + rkvenc_apply_ctrls(ctx); + rkvenc_rc_pick(ctx, &qp, idr ? 0 : 1); + ctx->cur_qp = qp; + ctx->cur_intra = idr; + + if (idr) + rkvenc_refresh_header(ctx); + + if (ctx->c.header_mode == + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME && + idr && (!ctx->joined_header_done || ctx->c.prepend_spspps)) { + vaddr = vb2_plane_vaddr(&dst->vb2_buf, 0); + if (vaddr) { + if (rkvenc_copy_headers(ctx, vaddr, + vb2_plane_size(&dst->vb2_buf, 0)) >= 0) { + hdr_len = ctx->hdr_len; + ctx->joined_header_done = true; + } + } else { + dev_warn_once(dev->dev, + "JOINED header needs an MMAP capture buffer\n"); + } + } + ctx->cur_prepend = hdr_len; + + f.codec = ctx->codec; + f.intra = idr; + f.referenced = true; + f.src.width = ctx->width; + f.src.height = ctx->height; + f.src.stride[0] = ctx->bytesperline; + f.src.stride[1] = ctx->bytesperline; + f.src.plane[0] = lower_32_bits( + vb2_dma_contig_plane_dma_addr(&src->vb2_buf, 0)); + if (ctx->src_num_planes == 1) + f.src.plane[1] = f.src.plane[0] + ctx->bytesperline * ctx->height; + else + f.src.plane[1] = lower_32_bits( + vb2_dma_contig_plane_dma_addr(&src->vb2_buf, 1) + + src->vb2_buf.planes[1].data_offset); + + f.bs.base = lower_32_bits( + vb2_dma_contig_plane_dma_addr(&dst->vb2_buf, 0)); + f.bs.size = vb2_plane_size(&dst->vb2_buf, 0); + f.bs.offset = hdr_len; + + f.recon = ctx->slot[ctx->cur]; + f.ref = idr ? ctx->slot[ctx->cur] : ctx->slot[ctx->last]; + + f.qp = qp; + f.qp_min = ctx->c.min_qp; + f.qp_max = ctx->c.max_qp; + f.rc = NULL; + f.wdg_vs_load = 0; + + if (f.codec == RKVENC_CODEC_H264) + rkvenc_h264_build_pic(ctx, &f.h264, idr); + else + rkvenc_hevc_build_pic(ctx, &f.hevc, idr); + + ret = pm_runtime_resume_and_get(dev->dev); + if (ret) + goto finish_error; + + ret = rkvenc_hw_run(dev, &f); + if (ret) { + pm_runtime_put(dev->dev); + goto finish_error; + } + return; + +finish_error: + dev->job_running = false; + dev->job_ctx = NULL; + if (src && dst) + v4l2_m2m_buf_done_and_job_finish(dev->m2m_dev, ctx->m2m_ctx, + VB2_BUF_STATE_ERROR); + else + v4l2_m2m_job_finish(dev->m2m_dev, ctx->m2m_ctx); +} + +const struct v4l2_m2m_ops rkvenc_m2m_ops = { + .device_run = rkvenc_device_run, +}; + +static int rkvenc_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct rkvenc_ctx *ctx = + container_of(ctrl->handler, struct rkvenc_ctx, ctrl_hdl); + + switch (ctrl->id) { + case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: + rkvenc_gop_force_keyframe(ctx); + break; + case V4L2_CID_MPEG_VIDEO_HEADER_MODE: + if (ctrl->val == V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) + ctx->header_pending = true; + break; + default: + break; + } + return 0; +} + +static const struct v4l2_ctrl_ops rkvenc_ctrl_ops = { + .s_ctrl = rkvenc_s_ctrl, +}; + +static int rkvenc_init_ctrls(struct rkvenc_ctx *ctx) +{ + struct v4l2_ctrl_handler *hdl = &ctx->ctrl_hdl; + + v4l2_ctrl_handler_init(hdl, 48); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_BITRATE_MODE, + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, + ~(1 << V4L2_MPEG_VIDEO_BITRATE_MODE_CBR), + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_BITRATE, + 25600, 40 * 1000 * 1000, 1, 4 * 1000 * 1000); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE, 0, 1, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_VBV_SIZE, + 0, 1 << 24, 1, 0); + + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_GOP_SIZE, + 0, 300, 1, 30); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_I_PERIOD, 0, 300, 1, 30); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD, 0, 300, 1, 30); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_TYPE, + V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR, + ~(1 << V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR), + V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME, 0, 0, 0, 0); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEADER_MODE, + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME, + 0, + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR, + 0, 1, 1, 0); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_PROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + ~((1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH)), + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LEVEL, + V4L2_MPEG_VIDEO_H264_LEVEL_5_2, + 0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC, 0, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM, + 0, 1, 1, 1); + v4l2_ctrl_new_std_menu( + hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE, + V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY, + 0, V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_ENABLED); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET, + -12, 12, 1, 0); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_PROFILE, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, 0, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, + V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2, + 0, V4L2_MPEG_VIDEO_HEVC_LEVEL_4); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_TIER, + V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, + ~(1 << V4L2_MPEG_VIDEO_HEVC_TIER_MAIN), + V4L2_MPEG_VIDEO_HEVC_TIER_MAIN); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE, + V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_ENABLED, + 0, + V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_ENABLED); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LF_BETA_OFFSET_DIV2, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LF_TC_OFFSET_DIV2, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_STRONG_SMOOTHING, + 0, 1, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_TMV_PREDICTION, + 0, 1, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_MAX_NUM_MERGE_MV_MINUS1, + 0, 4, 1, 2); + + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_MIN_QP, + 1, 51, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_MAX_QP, + 1, 51, 1, 51); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, 1, 51, 1, 26); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, 1, 51, 1, 28); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP, + 1, 51, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP, + 1, 51, 1, 51); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP, 1, 51, 1, 26); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP, 1, 51, 1, 28); + + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_AVERAGE_QP, 0, 51, 1, 0); + + if (hdl->error) + return hdl->error; + + v4l2_ctrl_handler_setup(hdl); + + ctx->header_pending = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEADER_MODE) == + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE; + + ctx->fh.ctrl_handler = hdl; + return 0; +} + +static int rkvenc_querycap(struct file *file, void *priv, + struct v4l2_capability *cap) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct rkvenc_dev *dev = ctx->dev; + + strscpy(cap->driver, "rkvenc", sizeof(cap->driver)); + strscpy(cap->card, "rockchip-rkvenc", sizeof(cap->card)); + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s", + dev_name(dev->dev)); + + return 0; +} + +static int rkvenc_enum_fmt(struct file *file, void *priv, + struct v4l2_fmtdesc *f, bool capture) +{ + const struct rkvenc_fmt *fmts = capture ? rkvenc_capture_fmts + : rkvenc_output_fmts; + unsigned int n = capture ? ARRAY_SIZE(rkvenc_capture_fmts) + : ARRAY_SIZE(rkvenc_output_fmts); + + if (f->index >= n) + return -EINVAL; + f->pixelformat = fmts[f->index].fourcc; + f->flags = 0; + return 0; +} + +static int rkvenc_enum_fmt_vid_out(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + return rkvenc_enum_fmt(file, priv, f, false); +} + +static int rkvenc_enum_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + return rkvenc_enum_fmt(file, priv, f, true); +} + +static void rkvenc_try_fmt_vid_out(struct rkvenc_ctx *ctx, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; + const struct rkvenc_fmt *fmt; + u32 bpl; + + fmt = rkvenc_find_fmt(rkvenc_output_fmts, + ARRAY_SIZE(rkvenc_output_fmts), + pix->pixelformat); + if (!fmt) { + fmt = &rkvenc_output_fmts[0]; + pix->pixelformat = fmt->fourcc; + } + + pix->field = V4L2_FIELD_NONE; + pix->width = clamp(round_up(pix->width, 2), 16, ctx->dev->max_width); + pix->height = clamp(round_up(pix->height, 2), 16, ctx->dev->max_height); + pix->num_planes = fmt->num_planes; + + bpl = ALIGN(pix->width, 16); + pix->plane_fmt[0].bytesperline = bpl; + if (fmt->num_planes == 2) { + pix->plane_fmt[0].sizeimage = bpl * pix->height; + pix->plane_fmt[1].bytesperline = bpl; + pix->plane_fmt[1].sizeimage = bpl * pix->height / 2; + } else { + pix->plane_fmt[0].sizeimage = bpl * pix->height * 3 / 2; + } +} + +static void rkvenc_try_fmt_vid_cap(struct rkvenc_ctx *ctx, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; + const struct rkvenc_fmt *fmt; + + fmt = rkvenc_find_fmt(rkvenc_capture_fmts, + ARRAY_SIZE(rkvenc_capture_fmts), + pix->pixelformat); + if (!fmt) { + fmt = &rkvenc_capture_fmts[0]; + pix->pixelformat = fmt->fourcc; + } + + pix->field = V4L2_FIELD_NONE; + pix->width = clamp(round_up(pix->width, 2), 16, ctx->dev->max_width); + pix->height = clamp(round_up(pix->height, 2), 16, ctx->dev->max_height); + pix->num_planes = 1; + pix->plane_fmt[0].bytesperline = 0; + + if (fmt->fourcc == V4L2_PIX_FMT_H264) { + u32 mbw = ALIGN(pix->width, 16) / 16; + u32 mbh = ALIGN(pix->height, 16) / 16; + + pix->plane_fmt[0].sizeimage = + min(mbw * mbh * 400, pix->width * pix->height / 2) + + RKVENC_HDR_MAX; + } else { + pix->plane_fmt[0].sizeimage = + pix->width * pix->height / 2 + RKVENC_HDR_MAX; + } +} + +static int rkvenc_try_fmt_vid_out_ioctl(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + + rkvenc_try_fmt_vid_out(ctx, f); + return 0; +} + +static int rkvenc_try_fmt_vid_cap_ioctl(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + + rkvenc_try_fmt_vid_cap(ctx, f); + return 0; +} + +static int rkvenc_s_fmt_vid_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct vb2_queue *vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type); + + if (vb2_is_busy(vq)) + return -EBUSY; + + rkvenc_try_fmt_vid_out(ctx, f); + ctx->src_fourcc = f->fmt.pix_mp.pixelformat; + ctx->src_num_planes = f->fmt.pix_mp.num_planes; + ctx->width = f->fmt.pix_mp.width; + ctx->height = f->fmt.pix_mp.height; + ctx->bytesperline = f->fmt.pix_mp.plane_fmt[0].bytesperline; + return 0; +} + +static int rkvenc_s_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct vb2_queue *vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type); + + if (vb2_is_busy(vq)) + return -EBUSY; + + rkvenc_try_fmt_vid_cap(ctx, f); + ctx->coded_fourcc = f->fmt.pix_mp.pixelformat; + ctx->codec = rkvenc_is_h264(f->fmt.pix_mp.pixelformat) ? + RKVENC_CODEC_H264 : RKVENC_CODEC_HEVC; + return 0; +} + +static int rkvenc_g_fmt(struct file *file, void *priv, struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + pix->pixelformat = ctx->src_fourcc; + pix->num_planes = ctx->src_num_planes; + } else { + pix->pixelformat = ctx->coded_fourcc; + pix->num_planes = 1; + } + pix->width = ctx->width; + pix->height = ctx->height; + pix->field = V4L2_FIELD_NONE; + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + pix->plane_fmt[0].bytesperline = ctx->bytesperline; + pix->plane_fmt[0].sizeimage = + rkvenc_output_plane_size(ctx, 0); + if (ctx->src_num_planes == 2) { + pix->plane_fmt[1].bytesperline = ctx->bytesperline; + pix->plane_fmt[1].sizeimage = + rkvenc_output_plane_size(ctx, 1); + } + } else { + pix->plane_fmt[0].sizeimage = rkvenc_capture_size(ctx); + } + return 0; +} + +static int rkvenc_s_parm(struct file *file, void *priv, + struct v4l2_streamparm *a) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct v4l2_fract *tpf = &a->parm.output.timeperframe; + unsigned int fps; + + if (!V4L2_TYPE_IS_MULTIPLANAR(a->type)) + return -EINVAL; + + if (tpf->numerator == 0 || tpf->denominator == 0) { + tpf->numerator = 1; + tpf->denominator = 30; + } + fps = tpf->denominator / tpf->numerator; + if (fps < 1) + fps = 1; + ctx->fps = fps; + a->parm.output.capability = V4L2_CAP_TIMEPERFRAME; + return 0; +} + +static int rkvenc_g_parm(struct file *file, void *priv, + struct v4l2_streamparm *a) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + + if (!V4L2_TYPE_IS_MULTIPLANAR(a->type)) + return -EINVAL; + + a->parm.output.capability = V4L2_CAP_TIMEPERFRAME; + a->parm.output.timeperframe.numerator = 1; + a->parm.output.timeperframe.denominator = ctx->fps ?: 30; + return 0; +} + +static int rkvenc_vidioc_encoder_cmd(struct file *file, void *priv, + struct v4l2_encoder_cmd *cmd) +{ + if (cmd->cmd != V4L2_ENC_CMD_STOP && cmd->cmd != V4L2_ENC_CMD_START) + return -EINVAL; + return v4l2_m2m_ioctl_encoder_cmd(file, priv, cmd); +} + +static int rkvenc_queue_init(void *priv, struct vb2_queue *src_vq, + struct vb2_queue *dst_vq) +{ + struct rkvenc_ctx *ctx = priv; + struct rkvenc_dev *dev = ctx->dev; + int ret; + + src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + src_vq->io_modes = VB2_MMAP | VB2_DMABUF; + src_vq->drv_priv = ctx; + src_vq->ops = &rkvenc_queue_ops; + src_vq->mem_ops = &vb2_dma_contig_memops; + src_vq->dma_attrs = DMA_ATTR_NO_KERNEL_MAPPING; + src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + src_vq->lock = &dev->vdev_lock; + src_vq->dev = dev->v4l2_dev.dev; + ret = vb2_queue_init(src_vq); + if (ret) + return ret; + + dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; + dst_vq->drv_priv = ctx; + dst_vq->ops = &rkvenc_queue_ops; + dst_vq->mem_ops = &vb2_dma_contig_memops; + dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + dst_vq->lock = &dev->vdev_lock; + dst_vq->dev = dev->v4l2_dev.dev; + return vb2_queue_init(dst_vq); +} + +static int rkvenc_open(struct file *filp) +{ + struct rkvenc_dev *dev = video_drvdata(filp); + struct rkvenc_ctx *ctx; + int ret; + + ctx = kzalloc_obj(*ctx); + if (!ctx) + return -ENOMEM; + + ctx->dev = dev; + ctx->codec = RKVENC_CODEC_H264; + ctx->coded_fourcc = V4L2_PIX_FMT_H264; + ctx->src_fourcc = V4L2_PIX_FMT_NV12; + ctx->src_num_planes = 1; + ctx->width = 1920; + ctx->height = 1080; + ctx->bytesperline = ALIGN(1920, 16); + ctx->fps = 30; + mutex_init(&ctx->vlock); + + v4l2_fh_init(&ctx->fh, video_devdata(filp)); + filp->private_data = &ctx->fh; + + ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx, rkvenc_queue_init); + if (IS_ERR(ctx->m2m_ctx)) { + ret = PTR_ERR(ctx->m2m_ctx); + goto err_fh; + } + ctx->fh.m2m_ctx = ctx->m2m_ctx; + + ret = rkvenc_init_ctrls(ctx); + if (ret) + goto err_m2m; + + v4l2_fh_add(&ctx->fh, filp); + return 0; + +err_m2m: + v4l2_m2m_ctx_release(ctx->m2m_ctx); +err_fh: + v4l2_fh_exit(&ctx->fh); + kfree(ctx); + return ret; +} + +static int rkvenc_release(struct file *filp) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(filp->private_data); + + v4l2_m2m_ctx_release(ctx->m2m_ctx); + v4l2_ctrl_handler_free(&ctx->ctrl_hdl); + v4l2_fh_del(&ctx->fh, filp); + v4l2_fh_exit(&ctx->fh); + kfree(ctx); + return 0; +} + +static const struct v4l2_ioctl_ops rkvenc_v4l2_ioctl_ops = { + .vidioc_querycap = rkvenc_querycap, + + .vidioc_enum_fmt_vid_cap = rkvenc_enum_fmt_vid_cap, + .vidioc_enum_fmt_vid_out = rkvenc_enum_fmt_vid_out, + .vidioc_try_fmt_vid_cap_mplane = rkvenc_try_fmt_vid_cap_ioctl, + .vidioc_try_fmt_vid_out_mplane = rkvenc_try_fmt_vid_out_ioctl, + .vidioc_s_fmt_vid_cap_mplane = rkvenc_s_fmt_vid_cap, + .vidioc_s_fmt_vid_out_mplane = rkvenc_s_fmt_vid_out, + .vidioc_g_fmt_vid_cap_mplane = rkvenc_g_fmt, + .vidioc_g_fmt_vid_out_mplane = rkvenc_g_fmt, + + .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querybuf = v4l2_m2m_ioctl_querybuf, + .vidioc_qbuf = v4l2_m2m_ioctl_qbuf, + .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, + .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, + .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, + .vidioc_streamon = v4l2_m2m_ioctl_streamon, + .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, + + .vidioc_s_parm = rkvenc_s_parm, + .vidioc_g_parm = rkvenc_g_parm, + + .vidioc_encoder_cmd = rkvenc_vidioc_encoder_cmd, + .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd, + + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, +}; + +static const struct v4l2_file_operations rkvenc_v4l2_fops = { + .owner = THIS_MODULE, + .open = rkvenc_open, + .release = rkvenc_release, + .poll = v4l2_m2m_fop_poll, + .unlocked_ioctl = video_ioctl2, + .mmap = v4l2_m2m_fop_mmap, +}; + +int rkvenc_v4l2_init(struct rkvenc_dev *dev) +{ + struct video_device *vdev = &dev->vdev; + int ret; + + dev->mdev.dev = dev->dev; + strscpy(dev->mdev.model, "rkvenc", sizeof(dev->mdev.model)); + strscpy(dev->mdev.bus_info, "platform:rkvenc", + sizeof(dev->mdev.bus_info)); + media_device_init(&dev->mdev); + dev->v4l2_dev.mdev = &dev->mdev; + + vdev->lock = &dev->vdev_lock; + vdev->v4l2_dev = &dev->v4l2_dev; + vdev->fops = &rkvenc_v4l2_fops; + vdev->release = video_device_release_empty; + vdev->vfl_dir = VFL_DIR_M2M; + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE; + vdev->ioctl_ops = &rkvenc_v4l2_ioctl_ops; + video_set_drvdata(vdev, dev); + strscpy(vdev->name, "rkvenc", sizeof(vdev->name)); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register video device\n"); + goto err_cleanup_mc; + } + + ret = v4l2_m2m_register_media_controller(dev->m2m_dev, vdev, + MEDIA_ENT_F_PROC_VIDEO_ENCODER); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to init M2M media controller\n"); + goto err_unregister_vdev; + } + + ret = media_device_register(&dev->mdev); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register media device\n"); + goto err_unregister_mc; + } + + return 0; + +err_unregister_mc: + v4l2_m2m_unregister_media_controller(dev->m2m_dev); +err_unregister_vdev: + video_unregister_device(vdev); +err_cleanup_mc: + media_device_cleanup(&dev->mdev); + return ret; +} + +void rkvenc_v4l2_cleanup(struct rkvenc_dev *dev) +{ + media_device_unregister(&dev->mdev); + v4l2_m2m_unregister_media_controller(dev->m2m_dev); + video_unregister_device(&dev->vdev); + media_device_cleanup(&dev->mdev); +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c new file mode 100644 index 0000000000000..202391672f2c7 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c @@ -0,0 +1,206 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RKVENC v1 register image builder, H.264 part. + */ + +#include "rkvenc-vepu541.h" + +#define H264_PROFILE_MAIN 77 +#define H264_LEVEL_1 10 +#define H264_LEVEL_3 30 +#define H264_NAL_SLICE 1 +#define H264_NAL_IDR 5 +#define H264_SLICE_P 0 +#define H264_SLICE_I 2 + +/* Coarse-ME RAM height by MB-aligned width */ +static const u16 h264_rama_h_width[] = { + 3584, 3136, 2816, 2560, 2368, 2176, 2048, 1856, 1792, +}; + +static void h264_me_build(const struct rkvenc_frame *f, struct rkvenc_img *img) +{ + const struct rkvenc_h264_seq *seq = f->h264.seq; + u32 w16 = ALIGN(f->src.width, 16), h16 = ALIGN(f->src.height, 16); + u32 mbw = w16 / 16, w64 = DIV_ROUND_UP(w16, 64); + u32 srch_h, srch_v, rama_h, v, hc, i; + u32 ah64 = ALIGN(h16, 64); + + srch_h = mbw < 22 ? mbw / 2 : 11; + srch_v = seq->level_idc <= H264_LEVEL_1 ? 3 : 7; + if (ah64 <= 64) + srch_v = 0; + else if ((ah64 == 128 || ah64 == 192) && seq->level_idc > H264_LEVEL_1) + srch_v = 4; + + rkvenc_img_l1(img, RKVENC_ME_RNGE, + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_H, srch_h) | + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_V, srch_v) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_H, 7) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_V, 5)); + + rkvenc_img_l1(img, RKVENC_ME_CNST, + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_H, f->intra ? 0 : 5) | + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_V, f->intra ? 0 : 5) | + FIELD_PREP(RKVENC_ME_CNST_MV_LIMIT, 2) | + FIELD_PREP(RKVENC_ME_CNST_PMV_NUM, 2)); + + rama_h = 17; + for (i = 0; i < ARRAY_SIZE(h264_rama_h_width); i++) { + if (w16 > h264_rama_h_width[i]) { + rama_h = 8 + i; + break; + } + } + v = 2 * srch_v + 1; + hc = (srch_h * 4 + 15) / 16 * 2 + 1; + rkvenc_img_l1(img, RKVENC_ME_RAM, + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_MAX, + (min(v, rama_h) - 1) * w64 + hc) | + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_H, rama_h)); +} + +static void h264_syntax_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + const struct rkvenc_h264_seq *seq = f->h264.seq; + const struct rkvenc_h264_pic *pic = &f->h264; + u32 rdo, sli2; + + rdo = RKVENC_H264_RDO_INTER_4X4 | RKVENC_H264_RDO_CHRM_SPCL | + RKVENC_H264_RDO_CCWA_E | RKVENC_H264_RDO_ATR_E; + if (seq->profile_idc < H264_PROFILE_MAIN && seq->level_idc <= H264_LEVEL_3) + rdo |= RKVENC_H264_RDO_RECT_SIZE; + if (seq->profile_idc < H264_PROFILE_MAIN && !seq->cabac) + rdo |= RKVENC_H264_RDO_VLC_LMT; + rkvenc_img_l1(img, RKVENC_RDO_CFG, rdo); + rkvenc_img_l1(img, RKVENC_IPRD_CSTS, + FIELD_PREP(RKVENC_IPRD_CSTS_VTHD_Y, 2501) | + FIELD_PREP(RKVENC_IPRD_CSTS_VTHD_C, 2501)); + + rkvenc_img_l1(img, RKVENC_SYNT_NAL, + FIELD_PREP(RKVENC_H264_NAL_REF_IDC, pic->nal_ref_idc) | + FIELD_PREP(RKVENC_H264_NAL_UNIT_TYPE, + f->intra ? H264_NAL_IDR : H264_NAL_SLICE)); + rkvenc_img_l1(img, RKVENC_SYNT_SPS, + FIELD_PREP(RKVENC_H264_SPS_MAX_FNUM, + seq->log2_max_frame_num_m4) | + RKVENC_H264_SPS_DRCT_8X8 | + FIELD_PREP(RKVENC_H264_SPS_MPOC_LM4, + seq->log2_max_poc_lsb_m4)); + rkvenc_img_l1( + img, RKVENC_SYNT_PPS, + (seq->cabac ? RKVENC_H264_PPS_ETPY_MODE : 0) | + (seq->transform_8x8 ? RKVENC_H264_PPS_TRNS_8X8 : 0) | + FIELD_PREP(RKVENC_PPS_PIC_INIT_QP, seq->pic_init_qp) | + FIELD_PREP(RKVENC_H264_PPS_CB_OFST, + rkvenc_s2u(seq->chroma_qp_offset, 5)) | + FIELD_PREP(RKVENC_H264_PPS_CR_OFST, + rkvenc_s2u(seq->chroma_qp_offset, 5)) | + RKVENC_H264_PPS_DBF_CP_FLG); + rkvenc_img_l1(img, RKVENC_SYNT_SLI0, + FIELD_PREP(RKVENC_H264_SLI0_SLI_TYPE, + f->intra ? H264_SLICE_I : H264_SLICE_P) | + FIELD_PREP(RKVENC_H264_SLI0_FRM_NUM, pic->frame_num)); + rkvenc_img_l1(img, RKVENC_SYNT_SLI1, + FIELD_PREP(RKVENC_H264_SLI1_IDR_PIC_ID, + f->intra ? pic->idr_pic_id : 0) | + FIELD_PREP(RKVENC_H264_SLI1_POC_LSB, pic->poc_lsb)); + + sli2 = FIELD_PREP(RKVENC_H264_SLI2_DIS_DBLK_IDC, seq->dblk_idc); + sli2 |= FIELD_PREP(RKVENC_H264_SLI2_ALPH_OFST, + rkvenc_s2u(seq->alpha_offset, 4)); + sli2 |= FIELD_PREP(RKVENC_H264_SLI2_BETA_OFST, + rkvenc_s2u(seq->beta_offset, 4)); + rkvenc_img_l1(img, RKVENC_SYNT_SLI2, sli2); +} + +static void h264_l2_build(const struct rkvenc_frame *f, struct rkvenc_img *img) +{ + u32 i; + + for (i = 0; i < 2; i++) { + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHDY4(i), + i ? rkvenc_pair(9, 36) : rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHDC8(i), + i ? rkvenc_pair(9, 36) : rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHDY8(i), + i ? rkvenc_pair(9, 36) : rkvenc_pair(1, 4)); + } + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHD_UL, 2501); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTY8, rkvenc_bytes(48, 60, 40, 48)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTY4, rkvenc_bytes(48, 60, 40, 48)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTY16, rkvenc_bytes(48, 60, 40, 48)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTC8, rkvenc_bytes(36, 42, 28, 32)); + + /* Rounding offsets in 1/2048 units */ + rkvenc_img_l2(img, RKVENC_L2_QNT_BIAS, + FIELD_PREP(RKVENC_L2_QNT_BIAS_I, f->intra ? 683 : 583) | + FIELD_PREP(RKVENC_L2_QNT_BIAS_P, 171)); + + rkvenc_img_l2(img, RKVENC_L2_ATR_THD0, rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_ATR_THD1, + rkvenc_pair(f->intra ? 36 : 81, 45)); + rkvenc_img_l2(img, RKVENC_L2_ATR_WGT16, f->intra ? + rkvenc_bytes(16, 16, 16, 0) : rkvenc_bytes(18, 17, 16, 0)); + rkvenc_img_l2(img, RKVENC_L2_ATR_WGT8, rkvenc_bytes(20, 20, 20, 0)); + rkvenc_img_l2(img, RKVENC_L2_ATR_WGT4, f->intra ? + rkvenc_bytes(20, 18, 16, 0) : rkvenc_bytes(18, 17, 16, 0)); + + rkvenc_img_l2(img, RKVENC_L2_ATF_TTHD(0), rkvenc_pair(0, 64)); + rkvenc_img_l2(img, RKVENC_L2_ATF_TTHD(1), rkvenc_pair(144, 2500)); + rkvenc_img_l2(img, RKVENC_L2_ATF_STHD(0), rkvenc_pair(80, 280)); + rkvenc_img_l2(img, RKVENC_L2_ATF_STHD(1), rkvenc_pair(144, 192)); + rkvenc_img_l2(img, RKVENC_L2_ATF_WGT(0), rkvenc_pair(26, 24)); + rkvenc_img_l2(img, RKVENC_L2_ATF_WGT(1), rkvenc_pair(19, 22)); + rkvenc_img_l2(img, RKVENC_L2_ATF_WGT(2), rkvenc_pair(19, 19)); + rkvenc_img_l2(img, RKVENC_L2_ATF_OFST(0), rkvenc_pair(3500, 3500)); + rkvenc_img_l2(img, RKVENC_L2_ATF_OFST(1), rkvenc_pair(0, 3500)); + rkvenc_img_l2(img, RKVENC_L2_ATF_OFST(2), rkvenc_pair(1000, 0)); + + for (i = 0; i <= RKVENC_QP_MAX; i++) { + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTA_QP(i), + rkvenc_h264_lambda[i + 1]); + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTB_QP(i), + rkvenc_h264_lambda[i]); + } + + rkvenc_img_l2_u8(img, RKVENC_L2_AQ_TTHD(0), rkvenc_h264_aq_thd, 16); + rkvenc_img_l2_s6(img, RKVENC_L2_AQ_STP(0), f->intra ? + rkvenc_h264_aq_step_i : rkvenc_h264_aq_step_p, 16); + + rkvenc_img_l2(img, RKVENC_L2_RME_MVD_PNSH, 16000 << 6); + rkvenc_img_l2(img, RKVENC_L2_ATR1_THD0, rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_ATR1_THD1, 49); + img->l2_last = RKVENC_L2_H264_LAST; +} + +void rkvenc_vepu541_h264_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + u32 w16 = ALIGN(f->src.width, 16), h16 = ALIGN(f->src.height, 16); + u32 pic, i; + + rkvenc_img_l1(img, RKVENC_ENC_RSL, + FIELD_PREP(RKVENC_ENC_RSL_WD8_M1, w16 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_WFILL, w16 - f->src.width) | + FIELD_PREP(RKVENC_ENC_RSL_HD8_M1, h16 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_HFILL, h16 - f->src.height)); + + pic = RKVENC_ENC_PIC_BS_SCP | FIELD_PREP(RKVENC_ENC_PIC_QP, f->qp); + if (f->referenced) + pic |= RKVENC_ENC_PIC_CUR_FRM_REF; + if (!f->intra) + pic |= RKVENC_ENC_PIC_LAMB_MOD_SEL; + rkvenc_img_l1(img, RKVENC_ENC_PIC, pic); + + rkvenc_img_l1(img, RKVENC_DTRNS_CFG, RKVENC_DTRNS_CFG_DSPR_OTSD); + + rkvenc_img_l1(img, RKVENC_KLUT_OFST, f->intra ? 0 : 3); + for (i = 0; i < RKVENC_KLUT_WORDS; i++) + rkvenc_img_l1(img, RKVENC_KLUT_WGT(i), rkvenc_h264_klut[i]); + + h264_me_build(f, img); + h264_syntax_build(f, img); + h264_l2_build(f, img); +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c new file mode 100644 index 0000000000000..0a6dea9062a54 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c @@ -0,0 +1,182 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RKVENC v1 register image builder, HEVC part. + */ + +#include "rkvenc-vepu541.h" + +#define HEVC_SLICE_P 1 +#define HEVC_SLICE_I 2 + +static void hevc_me_build(const struct rkvenc_frame *f, struct rkvenc_img *img) +{ + u32 w = ALIGN(f->src.width, 8), h = ALIGN(f->src.height, 8); + u32 wc = DIV_ROUND_UP(w, 64); + u32 sx, sy, srch_h, srch_v, rama_h, s, m, l2map; + + sx = w >= 384 + 60 ? 384 : 2 * (min(384U, w) / 4); + sy = h >= 256 + 60 ? 256 : 2 * (min(256U, h) / 4); + srch_h = (sx / 2 - 1) / 16; + srch_v = (sy / 2 - 1) / 16; + + rkvenc_img_l1(img, RKVENC_ME_RNGE, + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_H, srch_h) | + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_V, srch_v) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_H, 7) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_V, 5) | + FIELD_PREP(RKVENC_ME_RNGE_DLT_FRM_NUM, 1)); + rkvenc_img_l1(img, RKVENC_ME_CNST, + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_H, 5) | + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_V, 5) | + FIELD_PREP(RKVENC_ME_CNST_PMV_NUM, 2)); + + rama_h = w > 2688 ? 12 : w > 2048 ? 16 : 20; + s = 2 * ((srch_h + 4) / 4) + 1; + m = min(2 * ((srch_v + 3) / 4) + 1, rama_h / 4); + if (wc * 64 <= 512) + l2map = 0; + else if (wc * 64 <= 1024) + l2map = 1; + else if (wc * 64 <= 2048) + l2map = 2; + else + l2map = 3; + rkvenc_img_l1(img, RKVENC_ME_RAM, + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_MAX, + wc * (m - 1) + (wc >= s ? s : 2 * wc)) | + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_H, rama_h) | + FIELD_PREP(RKVENC_ME_RAM_CACH_L2_MAP, l2map)); +} + +static void hevc_syntax_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + const struct rkvenc_hevc_seq *seq = f->hevc.seq; + const struct rkvenc_hevc_pic *pic = &f->hevc; + u32 sli0, sli1; + + rkvenc_img_l1(img, RKVENC_RDO_CFG, + RKVENC_HEVC_RDO_CHRM_SPECIAL | + FIELD_PREP(RKVENC_HEVC_RDO_CU_INTER_EN, 0xf) | + FIELD_PREP(RKVENC_HEVC_RDO_CU_INTRA_EN, 0xf) | + RKVENC_HEVC_RDO_CHRM_KLUT_EN); + + rkvenc_img_l1(img, RKVENC_SYNT_NAL, + FIELD_PREP(RKVENC_HEVC_NAL_UNIT_TYPE, pic->nal_unit_type)); + rkvenc_img_l1(img, RKVENC_SYNT_SPS, + (seq->sao ? RKVENC_HEVC_SPS_SAO_EN : 0) | + FIELD_PREP(RKVENC_HEVC_SPS_LOG2_MAX_POC_LSB, + seq->log2_max_poc_lsb_m4) | + (seq->strong_intra_smoothing ? + RKVENC_HEVC_SPS_STRG_INTRA_SMTH : 0)); + rkvenc_img_l1(img, RKVENC_SYNT_PPS, + (seq->cabac_init_present ? + RKVENC_HEVC_PPS_CBC_INIT_PRSNT : 0) | + FIELD_PREP(RKVENC_PPS_PIC_INIT_QP, seq->pic_init_qp) | + RKVENC_HEVC_PPS_CU_QP_DLT_EN); + + sli0 = RKVENC_HEVC_SLI0_MERGE_UP | RKVENC_HEVC_SLI0_MERGE_LEFT | + RKVENC_HEVC_SLI0_PIC_OUT_FLG | + FIELD_PREP(RKVENC_HEVC_SLI0_SLI_TYPE, + f->intra ? HEVC_SLICE_I : HEVC_SLICE_P); + if (seq->cabac_init_present) + sli0 |= RKVENC_HEVC_SLI0_CBC_INIT_FLG; + if (seq->sao) + sli0 |= RKVENC_HEVC_SLI0_SAO_LUMA | RKVENC_HEVC_SLI0_SAO_CHRM; + rkvenc_img_l1(img, RKVENC_SYNT_SLI0, sli0); + + /* The header QP must match the QP the picture is quantised with */ + sli1 = FIELD_PREP(RKVENC_HEVC_SLI1_SLI_QP, f->qp) | + FIELD_PREP(RKVENC_HEVC_SLI1_MAX_MRG_CND, seq->max_merge_cand) | + RKVENC_HEVC_SLI1_COL_FRM_L0; + sli1 |= FIELD_PREP(RKVENC_HEVC_SLI1_TC_OFST, + rkvenc_s2u(seq->tc_offset, 4)); + sli1 |= FIELD_PREP(RKVENC_HEVC_SLI1_BETA_OFST, + rkvenc_s2u(seq->beta_offset, 4)); + if (seq->deblock_disabled) + sli1 |= RKVENC_HEVC_SLI1_DBLK_DIS; + rkvenc_img_l1(img, RKVENC_SYNT_SLI1, sli1); + rkvenc_img_l1(img, RKVENC_SYNT_SLI2, + FIELD_PREP(RKVENC_HEVC_SLI2_POC_LSB, pic->poc_lsb)); + + if (!f->intra) + rkvenc_img_l1(img, RKVENC_SYNT_REFM1, + FIELD_PREP(RKVENC_HEVC_REFM1_USED_BY_S0, 1) | + FIELD_PREP(RKVENC_HEVC_REFM1_NUM_NEG_PIC, 1)); +} + +static void hevc_l2_build(struct rkvenc_img *img) +{ + const u16 *thd = rkvenc_hevc_intra_thd; + u32 i; + + for (i = 0; i < 2; i++) { + rkvenc_img_l2(img, RKVENC_L2_HEVC_INTRA_THD32(i), + thd[2 * i] | thd[2 * i + 1] << 16); + rkvenc_img_l2(img, RKVENC_L2_HEVC_INTRA_THD16(i), + thd[2 * i] | thd[2 * i + 1] << 16); + } + rkvenc_img_l2_u8(img, RKVENC_L2_HEVC_INTRA_WGT32(0), + rkvenc_hevc_intra_wgt32, 7); + rkvenc_img_l2_u8(img, RKVENC_L2_HEVC_INTRA_WGT16(0), + rkvenc_hevc_intra_wgt16, 7); + + /* Rounding offsets in 1/512 units */ + rkvenc_img_l2(img, RKVENC_L2_QNT_BIAS, + FIELD_PREP(RKVENC_L2_QNT_BIAS_I, 171) | + FIELD_PREP(RKVENC_L2_QNT_BIAS_P, 85)); + + for (i = 0; i < 5; i++) + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_THD(i), rkvenc_pair(0, 63)); + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_WGT(0), rkvenc_pair(16, 16)); + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_WGT(1), rkvenc_pair(16, 16)); + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_WGT(2), 16); + + for (i = 0; i <= RKVENC_QP_MAX; i++) { + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGT_QP(i), + rkvenc_hevc_lambda_satd[i]); + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTA_QP(i), + rkvenc_hevc_lambda_a[i]); + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTB_QP(i), + rkvenc_hevc_lambda_b[i]); + } + + rkvenc_img_l2_u8(img, RKVENC_L2_AQ_TTHD(0), rkvenc_hevc_aq_thd, 16); + rkvenc_img_l2_s6(img, RKVENC_L2_AQ_STP(0), rkvenc_hevc_aq_step, 16); + img->l2_last = RKVENC_L2_HEVC_LAST; +} + +void rkvenc_vepu541_hevc_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + u32 w8 = ALIGN(f->src.width, 8), h8 = ALIGN(f->src.height, 8); + u32 ctus = DIV_ROUND_UP(w8, 64) * DIV_ROUND_UP(h8, 64); + u32 pic, i; + + rkvenc_img_l1(img, RKVENC_ENC_RSL, + FIELD_PREP(RKVENC_ENC_RSL_WD8_M1, w8 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_WFILL, w8 - f->src.width) | + FIELD_PREP(RKVENC_ENC_RSL_HD8_M1, h8 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_HFILL, h8 - f->src.height)); + + pic = RKVENC_ENC_PIC_STND_HEVC | RKVENC_ENC_PIC_BS_SCP | + FIELD_PREP(RKVENC_ENC_PIC_QP, f->qp) | + FIELD_PREP(RKVENC_ENC_PIC_TOT_POC_NUM, f->intra ? 0 : 1) | + FIELD_PREP(RKVENC_ENC_PIC_LOG2_CTU_NUM, order_base_2(ctus)); + if (f->referenced) + pic |= RKVENC_ENC_PIC_CUR_FRM_REF; + if (!f->intra) + pic |= RKVENC_ENC_PIC_LAMB_MOD_SEL; + rkvenc_img_l1(img, RKVENC_ENC_PIC, pic); + + rkvenc_img_l1(img, RKVENC_DTRNS_CFG, + f->intra ? 0 : RKVENC_DTRNS_CFG_DSPR_OTSD); + + rkvenc_img_l1(img, RKVENC_KLUT_OFST, f->intra ? 0 : 3); + for (i = 0; i < RKVENC_KLUT_WORDS; i++) + rkvenc_img_l1(img, RKVENC_KLUT_WGT(i), rkvenc_hevc_klut[i]); + + hevc_me_build(f, img); + hevc_syntax_build(f, img); + hevc_l2_build(img); +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h new file mode 100644 index 0000000000000..9abe6628a5831 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h @@ -0,0 +1,283 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef RKVENC_VEPU541_REGS_H +#define RKVENC_VEPU541_REGS_H + +#include <linux/bits.h> +#include <linux/bitfield.h> + +#define RKVENC_VERSION 0x000 + +#define RKVENC_ENC_STRT 0x004 +#define RKVENC_ENC_STRT_CMD GENMASK(9, 8) +#define RKVENC_ENC_STRT_CMD_FRAME 1 +#define RKVENC_ENC_STRT_CLK_GATE_EN BIT(16) +#define RKVENC_ENC_STRT_DONE_TMVP_EN BIT(18) + +#define RKVENC_ENC_CLR 0x008 +#define RKVENC_ENC_CLR_SAFE BIT(0) +#define RKVENC_ENC_CLR_FORCE BIT(1) + +#define RKVENC_LKT_ADDR 0x00c +#define RKVENC_INT_EN 0x010 +#define RKVENC_INT_MSK 0x014 +#define RKVENC_INT_CLR 0x018 +#define RKVENC_INT_STA 0x01c +#define RKVENC_INT_ENC_DONE BIT(0) +#define RKVENC_INT_LKT_DONE BIT(1) +#define RKVENC_INT_SCLR_DONE BIT(2) +#define RKVENC_INT_SLICE_DONE BIT(3) +#define RKVENC_INT_OFLW BIT(4) +#define RKVENC_INT_BRSP BIT(5) +#define RKVENC_INT_BERR BIT(6) +#define RKVENC_INT_RERR BIT(7) +#define RKVENC_INT_WDG BIT(8) +#define RKVENC_INT_ALL GENMASK(8, 0) +#define RKVENC_INT_ERRORS GENMASK(8, 4) + +#define RKVENC_ENC_RSL 0x030 +#define RKVENC_ENC_RSL_WD8_M1 GENMASK(8, 0) +#define RKVENC_ENC_RSL_WFILL GENMASK(15, 10) +#define RKVENC_ENC_RSL_HD8_M1 GENMASK(24, 16) +#define RKVENC_ENC_RSL_HFILL GENMASK(31, 26) + +#define RKVENC_ENC_PIC 0x034 +#define RKVENC_ENC_PIC_STND_HEVC BIT(0) +#define RKVENC_ENC_PIC_ROI_ENC BIT(1) +#define RKVENC_ENC_PIC_CUR_FRM_REF BIT(2) +#define RKVENC_ENC_PIC_BS_SCP BIT(4) +#define RKVENC_ENC_PIC_LAMB_MOD_SEL BIT(5) +#define RKVENC_ENC_PIC_QP GENMASK(13, 8) +#define RKVENC_ENC_PIC_TOT_POC_NUM GENMASK(18, 14) +#define RKVENC_ENC_PIC_LOG2_CTU_NUM GENMASK(22, 19) + +#define RKVENC_ENC_WDG 0x038 +#define RKVENC_ENC_WDG_VS_LOAD_THD GENMASK(23, 0) + +#define RKVENC_DTRNS_MAP 0x03c +#define RKVENC_DTRNS_MAP_BSW_BUS_EDIN GENMASK(14, 12) + +#define RKVENC_DTRNS_CFG 0x040 +#define RKVENC_DTRNS_CFG_DSPR_OTSD BIT(7) + +#define RKVENC_SRC_FMT 0x044 +#define RKVENC_SRC_FMT_CFMT GENMASK(5, 2) +#define RKVENC_SRC_FMT_CFMT_NV12 6 +#define RKVENC_SRC_FMT_RANGE BIT(6) + +#define RKVENC_SRC_PROC 0x058 +#define RKVENC_SRC_PROC_TXA_EN BIT(29) + +#define RKVENC_KLUT_OFST 0x064 +#define RKVENC_KLUT_WGT(n) (0x068 + 4 * (n)) +#define RKVENC_KLUT_WORDS 24 +#define RKVENC_KLUT_ENTRIES 35 + +#define RKVENC_RC_CFG 0x0c8 +#define RKVENC_RC_CFG_RC_EN BIT(0) +#define RKVENC_RC_CFG_AQ_EN BIT(1) +#define RKVENC_RC_CFG_AQ_MODE BIT(2) +#define RKVENC_RC_CFG_CTU_NUM GENMASK(31, 16) + +#define RKVENC_RC_QP 0x0cc +#define RKVENC_RC_QP_RANGE GENMASK(19, 16) +#define RKVENC_RC_QP_MAX GENMASK(25, 20) +#define RKVENC_RC_QP_MIN GENMASK(31, 26) + +#define RKVENC_RC_TGT 0x0d0 +#define RKVENC_RC_TGT_CTU_EBIT GENMASK(19, 0) + +#define RKVENC_RC_ADJ0 0x0d4 +#define RKVENC_RC_ADJ1 0x0d8 +#define RKVENC_RC_DTHD(n) (0x0dc + 4 * (n)) +#define RKVENC_RC_STEPS 9 + +#define RKVENC_ROI_QTHD(n) (0x100 + 4 * (n)) +#define RKVENC_PIC_OFST 0x110 + +#define RKVENC_SRC_STRID 0x114 +#define RKVENC_SRC_STRID_STRD0 GENMASK(15, 0) +#define RKVENC_SRC_STRID_STRD1 GENMASK(31, 16) + +#define RKVENC_ADR_SRC0 0x118 +#define RKVENC_ADR_SRC1 0x11c +#define RKVENC_ADR_SRC2 0x120 +#define RKVENC_ADR_ROI 0x124 +#define RKVENC_ADR_RFPW_H 0x128 +#define RKVENC_ADR_RFPW_B 0x12c +#define RKVENC_ADR_RFPR_H 0x130 +#define RKVENC_ADR_RFPR_B 0x134 +#define RKVENC_ADR_CMVW 0x138 +#define RKVENC_ADR_CMVR 0x13c +#define RKVENC_ADR_DSPW 0x140 +#define RKVENC_ADR_DSPR 0x144 +#define RKVENC_ADR_MEIW 0x148 +#define RKVENC_ADR_BSBT 0x14c +#define RKVENC_ADR_BSBB 0x150 +#define RKVENC_ADR_BSBR 0x154 +#define RKVENC_ADR_BSBS 0x158 + +#define RKVENC_SLI_SPLT 0x15c +#define RKVENC_SLI_BYTE 0x160 + +#define RKVENC_ME_RNGE 0x164 +#define RKVENC_ME_RNGE_CME_SRCH_H GENMASK(3, 0) +#define RKVENC_ME_RNGE_CME_SRCH_V GENMASK(7, 4) +#define RKVENC_ME_RNGE_RME_SRCH_H GENMASK(10, 8) +#define RKVENC_ME_RNGE_RME_SRCH_V GENMASK(13, 11) +#define RKVENC_ME_RNGE_DLT_FRM_NUM GENMASK(31, 16) + +#define RKVENC_ME_CNST 0x168 +#define RKVENC_ME_CNST_PMV_MDST_H GENMASK(7, 0) +#define RKVENC_ME_CNST_PMV_MDST_V GENMASK(15, 8) +#define RKVENC_ME_CNST_MV_LIMIT GENMASK(17, 16) +#define RKVENC_ME_CNST_PMV_NUM GENMASK(19, 18) + +#define RKVENC_ME_RAM 0x16c +#define RKVENC_ME_RAM_CME_RAMA_MAX GENMASK(10, 0) +#define RKVENC_ME_RAM_CME_RAMA_H GENMASK(15, 11) +#define RKVENC_ME_RAM_CACH_L2_MAP GENMASK(17, 16) + +#define RKVENC_IPRD_CSTS 0x194 +#define RKVENC_IPRD_CSTS_VTHD_Y GENMASK(11, 0) +#define RKVENC_IPRD_CSTS_VTHD_C GENMASK(27, 16) + +#define RKVENC_RDO_CFG 0x198 +#define RKVENC_H264_RDO_RECT_SIZE BIT(0) +#define RKVENC_H264_RDO_INTER_4X4 BIT(1) +#define RKVENC_H264_RDO_VLC_LMT BIT(3) +#define RKVENC_H264_RDO_CHRM_SPCL BIT(4) +#define RKVENC_H264_RDO_CCWA_E BIT(13) +#define RKVENC_H264_RDO_SCL_LST_SEL BIT(14) +#define RKVENC_H264_RDO_ATR_E BIT(15) +#define RKVENC_HEVC_RDO_CHRM_SPECIAL BIT(2) +#define RKVENC_HEVC_RDO_CU_INTER_EN GENMASK(9, 6) +#define RKVENC_HEVC_RDO_CU_INTRA_EN GENMASK(22, 19) +#define RKVENC_HEVC_RDO_CHRM_KLUT_EN BIT(23) + +#define RKVENC_SYNT_NAL 0x19c +#define RKVENC_H264_NAL_REF_IDC GENMASK(1, 0) +#define RKVENC_H264_NAL_UNIT_TYPE GENMASK(6, 2) +#define RKVENC_HEVC_NAL_UNIT_TYPE GENMASK(5, 0) + +#define RKVENC_SYNT_SPS 0x1a0 +#define RKVENC_H264_SPS_MAX_FNUM GENMASK(3, 0) +#define RKVENC_H264_SPS_DRCT_8X8 BIT(4) +#define RKVENC_H264_SPS_MPOC_LM4 GENMASK(8, 5) +#define RKVENC_HEVC_SPS_SAO_EN BIT(0) +#define RKVENC_HEVC_SPS_TMVP_EN BIT(15) +#define RKVENC_HEVC_SPS_LOG2_MAX_POC_LSB GENMASK(19, 16) +#define RKVENC_HEVC_SPS_STRG_INTRA_SMTH BIT(20) + +#define RKVENC_SYNT_PPS 0x1a4 +#define RKVENC_PPS_PIC_INIT_QP GENMASK(12, 7) +#define RKVENC_H264_PPS_ETPY_MODE BIT(0) +#define RKVENC_H264_PPS_TRNS_8X8 BIT(1) +#define RKVENC_H264_PPS_CB_OFST GENMASK(17, 13) +#define RKVENC_H264_PPS_CR_OFST GENMASK(22, 18) +#define RKVENC_H264_PPS_DBF_CP_FLG BIT(24) +#define RKVENC_HEVC_PPS_CBC_INIT_PRSNT BIT(6) +#define RKVENC_HEVC_PPS_CU_QP_DLT_EN BIT(13) + +#define RKVENC_SYNT_SLI0 0x1a8 +#define RKVENC_H264_SLI0_SLI_TYPE GENMASK(1, 0) +#define RKVENC_H264_SLI0_PPS_ID GENMASK(9, 2) +#define RKVENC_H264_SLI0_CBC_INIT_IDC GENMASK(13, 12) +#define RKVENC_H264_SLI0_FRM_NUM GENMASK(31, 16) +#define RKVENC_HEVC_SLI0_CBC_INIT_FLG BIT(0) +#define RKVENC_HEVC_SLI0_MERGE_UP BIT(2) +#define RKVENC_HEVC_SLI0_MERGE_LEFT BIT(3) +#define RKVENC_HEVC_SLI0_SAO_CHRM BIT(11) +#define RKVENC_HEVC_SLI0_SAO_LUMA BIT(12) +#define RKVENC_HEVC_SLI0_PIC_OUT_FLG BIT(14) +#define RKVENC_HEVC_SLI0_SLI_TYPE GENMASK(16, 15) + +#define RKVENC_SYNT_SLI1 0x1ac +#define RKVENC_H264_SLI1_IDR_PIC_ID GENMASK(15, 0) +#define RKVENC_H264_SLI1_POC_LSB GENMASK(31, 16) +#define RKVENC_HEVC_SLI1_TC_OFST GENMASK(3, 0) +#define RKVENC_HEVC_SLI1_BETA_OFST GENMASK(7, 4) +#define RKVENC_HEVC_SLI1_DBLK_DIS BIT(9) +#define RKVENC_HEVC_SLI1_SLI_QP GENMASK(21, 16) +#define RKVENC_HEVC_SLI1_MAX_MRG_CND GENMASK(24, 22) +#define RKVENC_HEVC_SLI1_COL_FRM_L0 BIT(26) + +#define RKVENC_SYNT_SLI2 0x1b0 +#define RKVENC_H264_SLI2_BETA_OFST GENMASK(6, 3) +#define RKVENC_H264_SLI2_ALPH_OFST GENMASK(10, 7) +#define RKVENC_H264_SLI2_DIS_DBLK_IDC GENMASK(12, 11) +#define RKVENC_HEVC_SLI2_POC_LSB GENMASK(15, 0) + +#define RKVENC_SYNT_REFM0 0x1b4 +#define RKVENC_SYNT_REFM1 0x1b8 +#define RKVENC_HEVC_REFM1_USED_BY_S0 GENMASK(3, 0) +#define RKVENC_HEVC_REFM1_NUM_NEG_PIC GENMASK(9, 5) +#define RKVENC_OSD_CFG 0x1c0 +#define RKVENC_SYNT_REFM2 0x1c8 +#define RKVENC_SYNT_REFM3 0x1cc + +#define RKVENC_L1_CFG_END 0x210 + +#define RKVENC_ST_BSL 0x210 +#define RKVENC_ST_BSL_LEN GENMASK(26, 0) +#define RKVENC_ST_SSE_L32 0x214 +#define RKVENC_ST_SSE_QP 0x218 +#define RKVENC_ST_ENC 0x228 +#define RKVENC_ST_BSB 0x234 +#define RKVENC_ST_BUS 0x238 +#define RKVENC_ST_PNUM(n) (0x244 + 4 * (n)) +#define RKVENC_ST_B8_QP(n) (0x264 + 4 * (n)) +#define RKVENC_ST_CPLX_TMP 0x334 +#define RKVENC_ST_BNUM_CME 0x338 +#define RKVENC_ST_CPLX_SPT 0x33c +#define RKVENC_ST_BNUM_B16 0x340 + +#define RKVENC_L2CFG_ADDR 0x3f0 +#define RKVENC_L2CFG_ADDR_AUTO_INC BIT(0) +#define RKVENC_L2CFG_WDATA 0x3f4 +#define RKVENC_L2CFG_RDATA 0x3f8 + +#define RKVENC_L2_FIRST 0x004 +#define RKVENC_L2_H264_LAST 0x30c +#define RKVENC_L2_HEVC_LAST 0x300 +#define RKVENC_L2_WORDS (RKVENC_L2_H264_LAST / 4 + 1) + +#define RKVENC_L2_IPRD_TTHDY4(n) (0x004 + 4 * (n)) +#define RKVENC_L2_IPRD_TTHDC8(n) (0x00c + 4 * (n)) +#define RKVENC_L2_IPRD_TTHDY8(n) (0x014 + 4 * (n)) +#define RKVENC_L2_IPRD_TTHD_UL 0x01c +#define RKVENC_L2_IPRD_WGTY8 0x020 +#define RKVENC_L2_IPRD_WGTY4 0x024 +#define RKVENC_L2_IPRD_WGTY16 0x028 +#define RKVENC_L2_IPRD_WGTC8 0x02c +#define RKVENC_L2_QNT_BIAS 0x030 +#define RKVENC_L2_QNT_BIAS_I GENMASK(9, 0) +#define RKVENC_L2_QNT_BIAS_P GENMASK(19, 10) +#define RKVENC_L2_ATR_THD0 0x034 +#define RKVENC_L2_ATR_THD1 0x038 +#define RKVENC_L2_ATR_WGT16 0x03c +#define RKVENC_L2_ATR_WGT8 0x040 +#define RKVENC_L2_ATR_WGT4 0x044 +#define RKVENC_L2_ATF_TTHD(n) (0x048 + 4 * (n)) +#define RKVENC_L2_ATF_STHD(n) (0x050 + 4 * (n)) +#define RKVENC_L2_ATF_WGT(n) (0x058 + 4 * (n)) +#define RKVENC_L2_ATF_OFST(n) (0x064 + 4 * (n)) +#define RKVENC_L2_IPRD_WGT_QP(q) (0x070 + 4 * (q)) +#define RKVENC_L2_RDO_WGTA_QP(q) (0x140 + 4 * (q)) +#define RKVENC_L2_RDO_WGTB_QP(q) (0x210 + 4 * (q)) +#define RKVENC_L2_MADI_CFG 0x2e0 +#define RKVENC_L2_AQ_TTHD(n) (0x2e4 + 4 * (n)) +#define RKVENC_L2_AQ_STP(n) (0x2f4 + 4 * (n)) +#define RKVENC_L2_RME_MVD_PNSH 0x304 +#define RKVENC_L2_ATR1_THD0 0x308 +#define RKVENC_L2_ATR1_THD1 0x30c + +#define RKVENC_L2_HEVC_INTRA_THD32(n) (0x004 + 4 * (n)) +#define RKVENC_L2_HEVC_INTRA_THD16(n) (0x00c + 4 * (n)) +#define RKVENC_L2_HEVC_INTRA_WGT32(n) (0x020 + 4 * (n)) +#define RKVENC_L2_HEVC_INTRA_WGT16(n) (0x028 + 4 * (n)) +#define RKVENC_L2_HEVC_ATF_THD(n) (0x048 + 4 * (n)) +#define RKVENC_L2_HEVC_ATF_WGT(n) (0x05c + 4 * (n)) + +#define RKVENC_QP_MAX 51 + +#endif diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c new file mode 100644 index 0000000000000..da4794b82bb12 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c @@ -0,0 +1,256 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Tuning tables of the RKVENC v1 encoder. Tables with a closed form are + * computed once at module load in integer fixed-point arithmetic. + */ + +#include <linux/build_bug.h> +#include <linux/cache.h> +#include <linux/math64.h> + +#include "rkvenc-vepu541.h" + +/* Fraction bits of the fixed-point numbers; the largest value is below 2^8 */ +#define RKVENC_FX_BITS 56 +#define RKVENC_FX_ONE (1ULL << RKVENC_FX_BITS) +#define RKVENC_FX_TWO (2ULL << RKVENC_FX_BITS) + +/* Weights and lambdas grow by 2^(1/3) per step, sixths give square roots */ +#define RKVENC_STEPS_PER_OCTAVE 3 +#define RKVENC_SIXTHS_PER_OCTAVE 6 +#define RKVENC_SIXTHS_PER_STEP 2 + +/* H.264 weight k = floor(16 * 2^(k / 3)); 16 is the neutral weight */ +#define RKVENC_H264_KLUT_NEUTRAL 16 + +/* H.264 lambda j = floor(3.6 * 2^((j + 5) / 3)), the factor as 18 / 5 */ +#define RKVENC_H264_LAMBDA_FACTOR_NUM 18 +#define RKVENC_H264_LAMBDA_FACTOR_DEN 5 +#define RKVENC_H264_LAMBDA_J_OFFSET 5 + +/* HEVC lambda 0.57 * 2^((q - 12) / 3), 11 fraction bits */ +#define RKVENC_HEVC_LAMBDA_PCT 57 +#define RKVENC_HEVC_LAMBDA_QP_REF 12 +#define RKVENC_HEVC_LAMBDA_FRAC 11 +#define RKVENC_PERCENT 100 + +/* HEVC weight i = S[i mod 3] * 2^(i div 3), S about 2^(7 + n / 3) */ +#define RKVENC_HEVC_KLUT_S0 128 +#define RKVENC_HEVC_KLUT_S1 161 +#define RKVENC_HEVC_KLUT_S2 204 +#define RKVENC_HEVC_LAMBDA_B_S0 112 +#define RKVENC_HEVC_LAMBDA_B_S1 137 +#define RKVENC_HEVC_LAMBDA_B_S2 176 +#define RKVENC_KLUT_ENTRY_BITS 18 + +/* Three KLUT weights per word pair, the middle one split across both */ +#define RKVENC_KLUT_PAIRS (RKVENC_KLUT_WORDS / 2) +#define RKVENC_KLUT_EVEN_WORD(n) (2 * (n)) +#define RKVENC_KLUT_ODD_WORD(n) (2 * (n) + 1) +#define RKVENC_KLUT_SPLIT_BITS 9 +#define RKVENC_KLUT_SPLIT_EVEN_SHIFT 23 +#define RKVENC_KLUT_LAST_ODD_SHIFT 14 + +static_assert(RKVENC_SIXTHS_PER_STEP * RKVENC_STEPS_PER_OCTAVE == + RKVENC_SIXTHS_PER_OCTAVE); +static_assert(RKVENC_HEVC_LAMBDA_QP_REF % RKVENC_SIXTHS_PER_OCTAVE == 0); + +u32 rkvenc_h264_klut[RKVENC_KLUT_WORDS] __ro_after_init; +u32 rkvenc_hevc_klut[RKVENC_KLUT_WORDS] __ro_after_init; +u32 rkvenc_h264_lambda[RKVENC_QP_MAX + 2] __ro_after_init; +u32 rkvenc_hevc_lambda_satd[RKVENC_QP_MAX + 1] __ro_after_init; +u32 rkvenc_hevc_lambda_a[RKVENC_QP_MAX + 1] __ro_after_init; +u32 rkvenc_hevc_lambda_b[RKVENC_QP_MAX + 1] __ro_after_init; + +typedef u64 (*rkvenc_fx_pow_t)(u64 x); + +static u64 fx_mul(u64 a, u64 b) +{ + return mul_u64_u64_shr(a, b, RKVENC_FX_BITS); +} + +static u64 fx_pow2(u64 x) +{ + return fx_mul(x, x); +} + +static u64 fx_pow6(u64 x) +{ + u64 x3 = fx_mul(fx_pow2(x), x); + + return fx_mul(x3, x3); +} + +/* Largest x below 2.0 whose power does not exceed target, by bisection */ +static u64 fx_root(rkvenc_fx_pow_t pow, u64 target) +{ + u64 lo = 0, hi = RKVENC_FX_TWO, mid; + + while (lo < hi) { + mid = (lo + hi + 1) / 2; + if (pow(mid) <= target) + lo = mid; + else + hi = mid - 1; + } + + return lo; +} + +/* x * 2^exp, rounded down or to nearest */ +static u32 fx_floor(u64 x, int exp) +{ + return x >> (RKVENC_FX_BITS - exp); +} + +static u32 fx_round(u64 x, int exp) +{ + unsigned int s = RKVENC_FX_BITS - exp; + + return (x + (1ULL << (s - 1))) >> s; +} + +static void klut_pack(u32 *words, const u32 *weights) +{ + u32 n, e0, e1, e2, low, i = 0; + + for (n = 0; n < RKVENC_KLUT_PAIRS; n++) { + e0 = weights[i++]; + e1 = i < RKVENC_KLUT_ENTRIES ? weights[i++] : 0; + e2 = i < RKVENC_KLUT_ENTRIES ? weights[i++] : 0; + low = e1 & GENMASK(RKVENC_KLUT_SPLIT_BITS - 1, 0); + words[RKVENC_KLUT_EVEN_WORD(n)] = + e0 | low << RKVENC_KLUT_SPLIT_EVEN_SHIFT; + words[RKVENC_KLUT_ODD_WORD(n)] = + e1 >> RKVENC_KLUT_SPLIT_BITS | + e2 << RKVENC_KLUT_LAST_ODD_SHIFT; + } +} + +static void h264_klut_init(const u64 *step_frac) +{ + u32 weights[RKVENC_KLUT_ENTRIES]; + u64 x; + u32 k; + + for (k = 0; k < RKVENC_KLUT_ENTRIES; k++) { + x = RKVENC_H264_KLUT_NEUTRAL * + step_frac[k % RKVENC_STEPS_PER_OCTAVE]; + weights[k] = fx_floor(x, k / RKVENC_STEPS_PER_OCTAVE); + } + + klut_pack(rkvenc_h264_klut, weights); +} + +static void hevc_klut_init(void) +{ + static const u16 s[RKVENC_STEPS_PER_OCTAVE] = { RKVENC_HEVC_KLUT_S0, + RKVENC_HEVC_KLUT_S1, + RKVENC_HEVC_KLUT_S2 }; + u32 weights[RKVENC_KLUT_ENTRIES]; + u64 v; + u32 i; + + for (i = 0; i < RKVENC_KLUT_ENTRIES; i++) { + v = (u64)s[i % RKVENC_STEPS_PER_OCTAVE] + << (i / RKVENC_STEPS_PER_OCTAVE); + weights[i] = + min_t(u64, v, GENMASK(RKVENC_KLUT_ENTRY_BITS - 1, 0)); + } + + klut_pack(rkvenc_hevc_klut, weights); +} + +static void h264_lambda_init(const u64 *step_frac) +{ + u32 j, m; + u64 x; + + for (j = 0; j < ARRAY_SIZE(rkvenc_h264_lambda); j++) { + m = j + RKVENC_H264_LAMBDA_J_OFFSET; + x = mul_u64_u32_div(step_frac[m % RKVENC_STEPS_PER_OCTAVE], + RKVENC_H264_LAMBDA_FACTOR_NUM, + RKVENC_H264_LAMBDA_FACTOR_DEN); + rkvenc_h264_lambda[j] = + fx_floor(x, m / RKVENC_STEPS_PER_OCTAVE); + } +} + +/* The SATD lambda is the square root, sqrt(0.57) * 2^((q - 12) / 6) */ +static void hevc_lambda_init(const u64 *sixth, u64 sqrt_factor) +{ + u32 pct = (RKVENC_HEVC_LAMBDA_PCT << RKVENC_HEVC_LAMBDA_FRAC) >> + (RKVENC_HEVC_LAMBDA_QP_REF / RKVENC_STEPS_PER_OCTAVE); + int exp = RKVENC_HEVC_LAMBDA_FRAC - + RKVENC_HEVC_LAMBDA_QP_REF / RKVENC_SIXTHS_PER_OCTAVE; + static const u16 b[RKVENC_STEPS_PER_OCTAVE] = { + RKVENC_HEVC_LAMBDA_B_S0, RKVENC_HEVC_LAMBDA_B_S1, + RKVENC_HEVC_LAMBDA_B_S2 + }; + u32 q, r; + u64 x; + + for (q = 0; q <= RKVENC_QP_MAX; q++) { + r = q % RKVENC_STEPS_PER_OCTAVE; + x = mul_u64_u32_div(sixth[RKVENC_SIXTHS_PER_STEP * r], pct, + RKVENC_PERCENT); + rkvenc_hevc_lambda_a[q] = + fx_round(x, q / RKVENC_STEPS_PER_OCTAVE); + + x = fx_mul(sqrt_factor, sixth[q % RKVENC_SIXTHS_PER_OCTAVE]); + rkvenc_hevc_lambda_satd[q] = + fx_round(x, q / RKVENC_SIXTHS_PER_OCTAVE + exp); + + rkvenc_hevc_lambda_b[q] = b[r] << (q / RKVENC_STEPS_PER_OCTAVE); + } +} + +void __init rkvenc_tables_init(void) +{ + u64 sixth[RKVENC_SIXTHS_PER_OCTAVE]; + u64 step_frac[RKVENC_STEPS_PER_OCTAVE]; + u64 factor, c, sqrt_factor; + u32 r; + + c = fx_root(fx_pow6, RKVENC_FX_TWO); + sixth[0] = RKVENC_FX_ONE; + for (r = 1; r < ARRAY_SIZE(sixth); r++) + sixth[r] = fx_mul(sixth[r - 1], c); + + for (r = 0; r < ARRAY_SIZE(step_frac); r++) + step_frac[r] = sixth[RKVENC_SIXTHS_PER_STEP * r]; + + factor = + div_u64(RKVENC_HEVC_LAMBDA_PCT * RKVENC_FX_ONE, RKVENC_PERCENT); + sqrt_factor = fx_root(fx_pow2, factor); + + h264_klut_init(step_frac); + hevc_klut_init(); + h264_lambda_init(step_frac); + hevc_lambda_init(sixth, sqrt_factor); +} + +/* Values without a closed form */ +const u8 rkvenc_h264_aq_thd[16] = { + 0, 0, 0, 0, 3, 3, 5, 5, 8, 8, 8, 15, 15, 20, 25, 25, +}; + +const s8 rkvenc_h264_aq_step_i[16] = { + -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 3, 3, 4, 5, 8, 8, +}; + +const s8 rkvenc_h264_aq_step_p[16] = { + -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 7, 8, +}; + +const u8 rkvenc_hevc_aq_thd[16] = { + 0, 0, 0, 0, 3, 3, 5, 5, 8, 8, 8, 15, 15, 20, 25, 35, +}; + +const s8 rkvenc_hevc_aq_step[16] = { + -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 7, 8, +}; + +const u16 rkvenc_hevc_intra_thd[4] = { 2, 6, 16, 36 }; +const u8 rkvenc_hevc_intra_wgt32[7] = { 23, 22, 21, 20, 22, 24, 26 }; +const u8 rkvenc_hevc_intra_wgt16[7] = { 17, 17, 17, 18, 17, 18, 18 }; diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c new file mode 100644 index 0000000000000..8bca4f6527347 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c @@ -0,0 +1,215 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RKVENC v1 register image builder, codec independent part. + */ + +#include <linux/math64.h> + +#include "rkvenc-vepu541.h" + +#define RKVENC_MIN_SIZE 8 +#define RKVENC_MAX_SIZE 4096 + +static const u16 rkvenc_l1_gaps[][2] = { + { RKVENC_INT_CLR, 0x02c }, + { 0x184, 0x190 }, + { 0x1bc, 0x1bc }, +}; + +bool rkvenc_vepu541_l1_written(u32 reg) +{ + u32 i; + + if (reg < RKVENC_ENC_CLR || reg >= RKVENC_L1_CFG_END) + return false; + + for (i = 0; i < ARRAY_SIZE(rkvenc_l1_gaps); i++) + if (reg >= rkvenc_l1_gaps[i][0] && reg <= rkvenc_l1_gaps[i][1]) + return false; + + return true; +} + +void rkvenc_vepu541_slot_sizes(enum rkvenc_codec codec, u32 width, u32 height, + struct rkvenc_slot_sizes *sz) +{ + u32 aw, ah, wc, hx; + + if (codec == RKVENC_CODEC_H264) { + aw = ALIGN(width, 64); + ah = ALIGN(height, 64) + 16; + sz->rfp_hdr = ALIGN(aw * ah / 64, 8192); + sz->rfp_body = aw * ah * 3 / 2; + sz->dsp = ALIGN(aw / 64 * ah / 64 * 256, 8192); + sz->cmv = 0; + return; + } + + wc = DIV_ROUND_UP(width, 64); + hx = DIV_ROUND_UP(height, 64) + 1; + sz->rfp_hdr = ALIGN(wc * hx * 64, 8192); + sz->rfp_body = wc * hx * 4096 * 3 / 2; + sz->dsp = wc * hx * 256; + sz->cmv = ALIGN(wc * hx * 64, 256); +} + +static void rkvenc_rc_build(const struct rkvenc_frame *f, u32 blocks, + u32 row_blocks, struct rkvenc_img *img) +{ + const struct rkvenc_rc *rc = f->rc; + u32 cfg, ebit, d, adj0, adj1, i; + s32 thd[RKVENC_RC_STEPS]; + u64 ebit64; + + cfg = 0; + if (f->aq) + cfg |= RKVENC_RC_CFG_AQ_EN; + if (f->aq && f->codec == RKVENC_CODEC_HEVC) + cfg |= RKVENC_RC_CFG_AQ_MODE; + + rkvenc_img_l1(img, RKVENC_RC_QP, + FIELD_PREP(RKVENC_RC_QP_RANGE, rc ? rc->qp_range : 0) | + FIELD_PREP(RKVENC_RC_QP_MAX, f->qp_max) | + FIELD_PREP(RKVENC_RC_QP_MIN, f->qp_min)); + + if (!rc) { + rkvenc_img_l1(img, RKVENC_RC_CFG, cfg); + return; + } + + cfg |= RKVENC_RC_CFG_RC_EN | + FIELD_PREP(RKVENC_RC_CFG_CTU_NUM, row_blocks); + rkvenc_img_l1(img, RKVENC_RC_CFG, cfg); + + ebit64 = div_u64(16ULL * rc->target_bits, blocks); + ebit = min_t(u64, ebit64, RKVENC_RC_TGT_CTU_EBIT); + rkvenc_img_l1(img, RKVENC_RC_TGT, ebit); + + /* The row rate control measures its deviation per row of blocks */ + d = ebit * row_blocks / 16 * 5 / 16; + thd[0] = -2 * (s32)d; + thd[1] = -(s32)d; + thd[2] = d; + thd[3] = 2 * d; + for (i = 4; i < RKVENC_RC_STEPS; i++) + thd[i] = 0x7fffffff; + + adj0 = 0; + adj1 = 0; + for (i = 0; i < RKVENC_RC_STEPS; i++) { + rkvenc_img_l1(img, RKVENC_RC_DTHD(i), (u32)thd[i]); + if (i < 5) + adj0 |= rkvenc_s2u(rc->step[i], 5) << (5 * i); + else + adj1 |= rkvenc_s2u(rc->step[i], 5) << (5 * (i - 5)); + } + rkvenc_img_l1(img, RKVENC_RC_ADJ0, adj0); + rkvenc_img_l1(img, RKVENC_RC_ADJ1, adj1); +} + +static void rkvenc_buffers_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + const struct rkvenc_src *src = &f->src; + const struct rkvenc_bs *bs = &f->bs; + + rkvenc_img_l1(img, RKVENC_SRC_FMT, + FIELD_PREP(RKVENC_SRC_FMT_CFMT, + RKVENC_SRC_FMT_CFMT_NV12)); + rkvenc_img_l1(img, RKVENC_SRC_PROC, RKVENC_SRC_PROC_TXA_EN); + rkvenc_img_l1(img, RKVENC_SRC_STRID, + FIELD_PREP(RKVENC_SRC_STRID_STRD0, src->stride[0]) | + FIELD_PREP(RKVENC_SRC_STRID_STRD1, src->stride[1])); + rkvenc_img_l1(img, RKVENC_ADR_SRC0, src->plane[0]); + rkvenc_img_l1(img, RKVENC_ADR_SRC1, src->plane[1]); + rkvenc_img_l1(img, RKVENC_ADR_SRC2, src->plane[2]); + + if (f->referenced) { + rkvenc_img_l1(img, RKVENC_ADR_RFPW_H, f->recon.rfp_hdr); + rkvenc_img_l1(img, RKVENC_ADR_RFPW_B, f->recon.rfp_body); + } + rkvenc_img_l1(img, RKVENC_ADR_DSPW, f->recon.dsp); + rkvenc_img_l1(img, RKVENC_ADR_RFPR_H, f->ref.rfp_hdr); + rkvenc_img_l1(img, RKVENC_ADR_RFPR_B, f->ref.rfp_body); + rkvenc_img_l1(img, RKVENC_ADR_DSPR, f->ref.dsp); + if (f->codec == RKVENC_CODEC_HEVC) { + rkvenc_img_l1(img, RKVENC_ADR_CMVW, f->recon.cmv); + rkvenc_img_l1(img, RKVENC_ADR_CMVR, f->ref.cmv); + } + + rkvenc_img_l1(img, RKVENC_ADR_BSBB, bs->base); + rkvenc_img_l1(img, RKVENC_ADR_BSBR, bs->base); + rkvenc_img_l1(img, RKVENC_ADR_BSBT, bs->base + bs->size - 1); + rkvenc_img_l1(img, RKVENC_ADR_BSBS, bs->base + bs->offset); + rkvenc_img_l1(img, RKVENC_DTRNS_MAP, + FIELD_PREP(RKVENC_DTRNS_MAP_BSW_BUS_EDIN, 7)); +} + +static int rkvenc_frame_check(const struct rkvenc_frame *f) +{ + u32 w = f->src.width, h = f->src.height; + + if (w < RKVENC_MIN_SIZE || h < RKVENC_MIN_SIZE || + w > RKVENC_MAX_SIZE || h > RKVENC_MAX_SIZE) + return -EINVAL; + + if (f->qp > RKVENC_QP_MAX || f->qp_min > f->qp_max || + f->qp_max > RKVENC_QP_MAX) + return -EINVAL; + + /* Hardware bug: H.264 at most 2 MBs wide and over 1 MB high hangs */ + if (f->codec == RKVENC_CODEC_H264 && ALIGN(w, 16) <= 32 && + ALIGN(h, 16) > 16) + return -EINVAL; + + /* Hardware bug: HEVC P pictures one CTU row high decode wrongly */ + if (f->codec == RKVENC_CODEC_HEVC && !f->intra && h <= 64) + return -EINVAL; + + if (f->bs.size < 256 || f->bs.offset >= f->bs.size) + return -EINVAL; + + return 0; +} + +int rkvenc_vepu541_img_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + u32 blocks, row_blocks, int_en; + int ret; + + ret = rkvenc_frame_check(f); + if (ret) + return ret; + + memset(img, 0, sizeof(*img)); + + int_en = RKVENC_INT_ENC_DONE | RKVENC_INT_SCLR_DONE | RKVENC_INT_OFLW | + RKVENC_INT_BRSP | RKVENC_INT_BERR | RKVENC_INT_RERR; + if (f->wdg_vs_load) + int_en |= RKVENC_INT_WDG; + rkvenc_img_l1(img, RKVENC_INT_EN, int_en); + rkvenc_img_l1(img, RKVENC_ENC_WDG, + FIELD_PREP(RKVENC_ENC_WDG_VS_LOAD_THD, f->wdg_vs_load)); + + rkvenc_buffers_build(f, img); + + if (f->codec == RKVENC_CODEC_H264) { + row_blocks = DIV_ROUND_UP(f->src.width, 16); + blocks = row_blocks * DIV_ROUND_UP(f->src.height, 16); + rkvenc_vepu541_h264_build(f, img); + } else { + row_blocks = DIV_ROUND_UP(f->src.width, 64); + blocks = row_blocks * DIV_ROUND_UP(f->src.height, 64); + rkvenc_vepu541_hevc_build(f, img); + } + + rkvenc_rc_build(f, blocks, row_blocks, img); + + img->enc_strt = FIELD_PREP(RKVENC_ENC_STRT_CMD, + RKVENC_ENC_STRT_CMD_FRAME) | + RKVENC_ENC_STRT_CLK_GATE_EN | + RKVENC_ENC_STRT_DONE_TMVP_EN; + + return 0; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h new file mode 100644 index 0000000000000..33a13402d0ade --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef RKVENC_VEPU541_H +#define RKVENC_VEPU541_H + +#include <linux/errno.h> +#include <linux/string.h> +#include <linux/types.h> +#include <linux/bits.h> +#include <linux/bitfield.h> +#include <linux/kernel.h> +#include <linux/minmax.h> +#include <linux/math.h> +#include <linux/log2.h> + +#include "rkvenc-vepu541-regs.h" + +enum rkvenc_codec { + RKVENC_CODEC_H264, + RKVENC_CODEC_HEVC, +}; + +struct rkvenc_src { + u32 plane[3]; + u32 stride[2]; + u32 width; + u32 height; +}; + +struct rkvenc_bs { + u32 base; + u32 size; + u32 offset; +}; + +struct rkvenc_slot { + u32 rfp_hdr; + u32 rfp_body; + u32 dsp; + u32 cmv; +}; + +struct rkvenc_slot_sizes { + u32 rfp_hdr; + u32 rfp_body; + u32 dsp; + u32 cmv; +}; + +struct rkvenc_h264_seq { + u32 width; + u32 height; + u8 profile_idc; + u8 level_idc; + u8 log2_max_frame_num_m4; + u8 log2_max_poc_lsb_m4; + u8 pic_init_qp; + s8 chroma_qp_offset; + bool cabac; + bool transform_8x8; + u8 dblk_idc; + s8 alpha_offset; + s8 beta_offset; + bool timing; + u32 fps; +}; + +struct rkvenc_h264_pic { + const struct rkvenc_h264_seq *seq; + u16 frame_num; + u16 poc_lsb; + u16 idr_pic_id; + u8 nal_ref_idc; +}; + +struct rkvenc_hevc_seq { + u32 width; + u32 height; + u8 level_idc; + u8 log2_max_poc_lsb_m4; + u8 pic_init_qp; + u8 max_merge_cand; + bool sao; + bool deblock_disabled; + s8 beta_offset; + s8 tc_offset; + bool strong_intra_smoothing; + bool cabac_init_present; +}; + +struct rkvenc_hevc_pic { + const struct rkvenc_hevc_seq *seq; + u16 poc_lsb; + u8 nal_unit_type; +}; + +struct rkvenc_rc { + u32 target_bits; + u8 qp_range; + s8 step[RKVENC_RC_STEPS]; +}; + +struct rkvenc_frame { + enum rkvenc_codec codec; + bool intra; + bool referenced; + struct rkvenc_src src; + struct rkvenc_bs bs; + struct rkvenc_slot recon; + struct rkvenc_slot ref; + u8 qp; + u8 qp_min; + u8 qp_max; + bool aq; + const struct rkvenc_rc *rc; + u32 wdg_vs_load; + union { + struct rkvenc_h264_pic h264; + struct rkvenc_hevc_pic hevc; + }; +}; + +#define RKVENC_L1_CFG_WORDS (RKVENC_L1_CFG_END / 4) + +struct rkvenc_img { + u32 l1[RKVENC_L1_CFG_WORDS]; + u32 l2[RKVENC_L2_WORDS]; + u32 l2_last; + u32 enc_strt; +}; + +static inline void rkvenc_img_l1(struct rkvenc_img *img, u32 reg, u32 val) +{ + img->l1[reg / 4] = val; +} + +static inline void rkvenc_img_l2(struct rkvenc_img *img, u32 reg, u32 val) +{ + img->l2[reg / 4] = val; +} + +static inline u32 rkvenc_s2u(s32 val, u32 bits) +{ + return (u32)val & GENMASK(bits - 1, 0); +} + +static inline u32 rkvenc_pair(u32 lo, u32 hi) +{ + return lo | hi << 16; +} + +static inline u32 rkvenc_bytes(u8 b0, u8 b1, u8 b2, u8 b3) +{ + return b0 | b1 << 8 | b2 << 16 | (u32)b3 << 24; +} + +static inline void rkvenc_img_l2_u8(struct rkvenc_img *img, u32 reg, + const u8 *v, u32 n) +{ + u32 i; + + for (i = 0; i < n; i++) + img->l2[reg / 4 + i / 4] |= (u32)v[i] << (8 * (i % 4)); +} + +static inline void rkvenc_img_l2_s6(struct rkvenc_img *img, u32 reg, + const s8 *v, u32 n) +{ + u32 i; + + for (i = 0; i < n; i++) + img->l2[reg / 4 + i / 4] |= rkvenc_s2u(v[i], 6) << (8 * (i % 4)); +} + +bool rkvenc_vepu541_l1_written(u32 reg); +void rkvenc_vepu541_slot_sizes(enum rkvenc_codec codec, u32 width, u32 height, + struct rkvenc_slot_sizes *sz); +int rkvenc_vepu541_img_build(const struct rkvenc_frame *f, + struct rkvenc_img *img); + +void rkvenc_vepu541_h264_build(const struct rkvenc_frame *f, + struct rkvenc_img *img); +void rkvenc_vepu541_hevc_build(const struct rkvenc_frame *f, + struct rkvenc_img *img); + +void rkvenc_tables_init(void); +extern u32 rkvenc_h264_klut[RKVENC_KLUT_WORDS]; +extern u32 rkvenc_hevc_klut[RKVENC_KLUT_WORDS]; +extern u32 rkvenc_h264_lambda[RKVENC_QP_MAX + 2]; +extern u32 rkvenc_hevc_lambda_satd[RKVENC_QP_MAX + 1]; +extern u32 rkvenc_hevc_lambda_a[RKVENC_QP_MAX + 1]; +extern u32 rkvenc_hevc_lambda_b[RKVENC_QP_MAX + 1]; +extern const u8 rkvenc_h264_aq_thd[16]; +extern const s8 rkvenc_h264_aq_step_i[16]; +extern const s8 rkvenc_h264_aq_step_p[16]; +extern const u8 rkvenc_hevc_aq_thd[16]; +extern const s8 rkvenc_hevc_aq_step[16]; +extern const u16 rkvenc_hevc_intra_thd[4]; +extern const u8 rkvenc_hevc_intra_wgt32[7]; +extern const u8 rkvenc_hevc_intra_wgt16[7]; + +#endif diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c new file mode 100644 index 0000000000000..00778e97686d7 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c @@ -0,0 +1,489 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Rockchip RKVENC v1 H.264/HEVC encoder: platform driver, interrupt and + * job handling. + */ + +#include <linux/clk.h> +#include <linux/completion.h> +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/module.h> +#include <linux/platform_device.h> +#include <linux/pm_runtime.h> +#include <linux/reset.h> + +#include <media/v4l2-mem2mem.h> +#include <media/videobuf2-dma-contig.h> + +#include "rkvenc-vepu541-regs.h" +#include "rkvenc.h" + +#define RKVENC_WATCHDOG_MS 1000 +#define RKVENC_SAFE_CLEAR_MS 100 +#define RKVENC_RESET_PULSE_US 5 + +#define RKVENC_L1_CFG_START 0x00c + +static const struct rkvenc_variant rkvenc_variant_rv1126 = { + .vepu540 = false, +}; + +static const struct rkvenc_variant rkvenc_variant_rk3568 = { + .vepu540 = true, +}; + +int rkvenc_hw_run(struct rkvenc_dev *dev, const struct rkvenc_frame *f) +{ + struct rkvenc_img *img = &dev->img; + void __iomem *regs = dev->regs; + unsigned int a; + int ret; + + ret = rkvenc_vepu541_img_build(f, img); + if (WARN_ON(ret)) + return ret; + + /* INT_STA keeps its bits across a start */ + writel(RKVENC_INT_ALL, regs + RKVENC_INT_CLR); + + writel(RKVENC_L2_FIRST | RKVENC_L2CFG_ADDR_AUTO_INC, + regs + RKVENC_L2CFG_ADDR); + for (a = RKVENC_L2_FIRST; a <= img->l2_last; a += 4) + writel(img->l2[a / 4], regs + RKVENC_L2CFG_WDATA); + + /* A write to ENC_CLR would clear the job, so L1 starts above it */ + for (a = RKVENC_L1_CFG_END - 4; a >= RKVENC_L1_CFG_START; a -= 4) + if (rkvenc_vepu541_l1_written(a)) + writel(img->l1[a / 4], regs + a); + + /* A timed out job must not be judged by the previous status. */ + dev->irq_sta = 0; + writel(img->enc_strt, regs + RKVENC_ENC_STRT); + + schedule_delayed_work(&dev->watchdog, + msecs_to_jiffies(RKVENC_WATCHDOG_MS)); + return 0; +} + +void rkvenc_hw_status_read(struct rkvenc_dev *dev, u32 *int_sta, u32 *bytes, + u32 *qp_sum) +{ + void __iomem *regs = dev->regs; + + *int_sta = dev->irq_sta; + *bytes = FIELD_GET(RKVENC_ST_BSL_LEN, readl(regs + RKVENC_ST_BSL)); + *qp_sum = readl(regs + RKVENC_ST_SSE_QP) & GENMASK(23, 0); +} + +static bool rkvenc_hw_force_clear(struct rkvenc_dev *dev) +{ + void __iomem *regs = dev->regs; + + writel(RKVENC_INT_ALL, regs + RKVENC_INT_MSK); + writel(RKVENC_ENC_CLR_FORCE, regs + RKVENC_ENC_CLR); + writel(0, regs + RKVENC_ENC_CLR); + writel(RKVENC_INT_ALL, regs + RKVENC_INT_CLR); + readl(regs + RKVENC_INT_STA); + writel(0, regs + RKVENC_INT_MSK); + + return !(readl(regs + RKVENC_ST_ENC) & GENMASK(1, 0)); +} + +void rkvenc_hw_recover(struct rkvenc_dev *dev) +{ + void __iomem *regs = dev->regs; + + if (rkvenc_hw_force_clear(dev)) + return; + + dev_err(dev->dev, "job did not stop on force clear, resetting\n"); + + writel(RKVENC_INT_ALL & ~RKVENC_INT_SCLR_DONE, regs + RKVENC_INT_MSK); + writel(RKVENC_INT_ALL, regs + RKVENC_INT_CLR); + + reinit_completion(&dev->sclr_done); + writel(RKVENC_ENC_CLR_SAFE, regs + RKVENC_ENC_CLR); + wait_for_completion_timeout(&dev->sclr_done, + msecs_to_jiffies(RKVENC_SAFE_CLEAR_MS)); + + if (dev->num_resets) { + reset_control_bulk_assert(dev->num_resets, dev->resets); + udelay(RKVENC_RESET_PULSE_US); + reset_control_bulk_deassert(dev->num_resets, dev->resets); + } + + writel(0, regs + RKVENC_INT_MSK); +} + +static irqreturn_t rkvenc_irq(int irq, void *priv) +{ + struct rkvenc_dev *dev = priv; + u32 sta; + + sta = readl(dev->regs + RKVENC_INT_STA); + if (!sta) + return IRQ_NONE; + + writel(sta, dev->regs + RKVENC_INT_CLR); + /* Flush the posted clear before the interrupt line is sampled again */ + readl(dev->regs + RKVENC_INT_STA); + + if (sta & RKVENC_INT_SCLR_DONE) + complete(&dev->sclr_done); + + dev->irq_sta = sta; + return IRQ_WAKE_THREAD; +} + +static void rkvenc_finish_job(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx, + enum vb2_buffer_state state) +{ + struct vb2_v4l2_buffer *src, *dst; + u32 payload, int_sta, bytes, qp_sum; + u8 frame_type, qp_avg; + struct v4l2_ctrl *avg_qp; + + rkvenc_hw_status_read(dev, &int_sta, &bytes, &qp_sum); + + if (state == VB2_BUF_STATE_DONE) { + src = v4l2_m2m_next_src_buf(ctx->m2m_ctx); + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + + payload = ctx->cur_prepend + bytes; + frame_type = ctx->cur_intra ? 0 : 1; + if (ctx->qp_blocks) + qp_avg = (u8)(qp_sum / ctx->qp_blocks); + else + qp_avg = ctx->cur_qp; + + if (dst) { + vb2_set_plane_payload(&dst->vb2_buf, 0, payload); + if (src) + v4l2_m2m_buf_copy_metadata(src, dst); + dst->flags |= ctx->cur_intra ? V4L2_BUF_FLAG_KEYFRAME + : V4L2_BUF_FLAG_PFRAME; + } + + rkvenc_rc_update(ctx, payload, qp_avg, frame_type); + rkvenc_gop_commit(ctx); + ctx->consecutive_errors = 0; + ctx->overflow_retried = false; + + avg_qp = v4l2_ctrl_find(&ctx->ctrl_hdl, + V4L2_CID_MPEG_VIDEO_AVERAGE_QP); + if (avg_qp) + v4l2_ctrl_s_ctrl(avg_qp, qp_avg); + } else if (ctx->rc_active && !ctx->overflow_retried && + (int_sta & RKVENC_INT_ERRORS) == RKVENC_INT_OFLW) { + /* The job stays running for the framework: the frame is retried */ + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + rkvenc_hw_recover(dev); + rkvenc_rc_overflow(ctx, + dst ? vb2_plane_size(&dst->vb2_buf, 0) : 0, + ctx->cur_intra ? 0 : 1); + ctx->overflow_retried = true; + if (ctx->cur_prepend) + ctx->joined_header_done = false; + dev->job_ctx = NULL; + pm_runtime_put(dev->dev); + rkvenc_m2m_ops.device_run(ctx); + return; + } else { + rkvenc_hw_recover(dev); + rkvenc_gop_rollback(ctx); + ctx->overflow_retried = false; + ctx->consecutive_errors++; + if (ctx->consecutive_errors >= RKVENC_MAX_CONSEC_ERRORS) { + vb2_queue_error(&ctx->m2m_ctx->out_q_ctx.q); + vb2_queue_error(&ctx->m2m_ctx->cap_q_ctx.q); + } + } + + src = v4l2_m2m_next_src_buf(ctx->m2m_ctx); + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + if (src && dst && + v4l2_m2m_is_last_draining_src_buf(ctx->m2m_ctx, src)) { + dst->flags |= V4L2_BUF_FLAG_LAST; + v4l2_m2m_mark_stopped(ctx->m2m_ctx); + } + + dev->job_ctx = NULL; + pm_runtime_put(dev->dev); + + v4l2_m2m_buf_done_and_job_finish(dev->m2m_dev, ctx->m2m_ctx, state); +} + +static irqreturn_t rkvenc_irq_thread(int irq, void *priv) +{ + struct rkvenc_dev *dev = priv; + struct rkvenc_ctx *ctx; + enum vb2_buffer_state state; + u32 int_sta, bytes, qp_sum; + unsigned long flags; + + /* Whoever clears job_running first, IRQ or watchdog, finishes the job */ + spin_lock_irqsave(&dev->irq_lock, flags); + if (!dev->job_running) { + spin_unlock_irqrestore(&dev->irq_lock, flags); + return IRQ_HANDLED; + } + ctx = dev->job_ctx; + dev->job_running = false; + spin_unlock_irqrestore(&dev->irq_lock, flags); + + cancel_delayed_work(&dev->watchdog); + + rkvenc_hw_status_read(dev, &int_sta, &bytes, &qp_sum); + + if ((int_sta & RKVENC_INT_ENC_DONE) && + !(int_sta & RKVENC_INT_ERRORS)) + state = VB2_BUF_STATE_DONE; + else + state = VB2_BUF_STATE_ERROR; + + rkvenc_finish_job(dev, ctx, state); + return IRQ_HANDLED; +} + +static void rkvenc_watchdog_work(struct work_struct *work) +{ + struct rkvenc_dev *dev = container_of(to_delayed_work(work), + struct rkvenc_dev, watchdog); + struct rkvenc_ctx *ctx; + unsigned long flags; + + spin_lock_irqsave(&dev->irq_lock, flags); + if (!dev->job_running) { + spin_unlock_irqrestore(&dev->irq_lock, flags); + return; + } + ctx = dev->job_ctx; + dev->job_running = false; + spin_unlock_irqrestore(&dev->irq_lock, flags); + + dev_err(dev->dev, "RKVENC encode timed out\n"); + rkvenc_finish_job(dev, ctx, VB2_BUF_STATE_ERROR); +} + +static int rkvenc_runtime_resume(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + return clk_bulk_prepare_enable(rkvenc->num_clocks, rkvenc->clocks); +} + +static int rkvenc_runtime_suspend(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + clk_bulk_disable_unprepare(rkvenc->num_clocks, rkvenc->clocks); + return 0; +} + +static int rkvenc_suspend(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + v4l2_m2m_suspend(rkvenc->m2m_dev); + return pm_runtime_force_suspend(dev); +} + +static int rkvenc_resume(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + int ret; + + ret = pm_runtime_force_resume(dev); + if (ret) + return ret; + /* Jobs rewrite every register and the references stay in memory */ + v4l2_m2m_resume(rkvenc->m2m_dev); + return 0; +} + +static void rkvenc_init_clocks_resets(struct rkvenc_dev *rkvenc) +{ + static const char *const clk_names[] = { "axi", "ahb", "core" }; + static const char *const rst_names[] = { "axi", "ahb", "core" }; + unsigned int i; + + for (i = 0; i < RKVENC_NUM_CLKS; i++) + rkvenc->clocks[i].id = clk_names[i]; + for (i = 0; i < RKVENC_NUM_RESETS; i++) + rkvenc->resets[i].id = rst_names[i]; + rkvenc->num_clocks = RKVENC_NUM_CLKS; + rkvenc->num_resets = RKVENC_NUM_RESETS; +} + +static int rkvenc_probe(struct platform_device *pdev) +{ + const struct rkvenc_variant *variant; + struct rkvenc_dev *rkvenc; + struct device *dev = &pdev->dev; + u32 ver, pic_size; + int ret, irq; + + variant = of_device_get_match_data(dev); + if (!variant) + return -EINVAL; + + rkvenc = devm_kzalloc(dev, sizeof(*rkvenc), GFP_KERNEL); + if (!rkvenc) + return -ENOMEM; + + platform_set_drvdata(pdev, rkvenc); + rkvenc->dev = dev; + rkvenc->variant = variant; + + spin_lock_init(&rkvenc->irq_lock); + mutex_init(&rkvenc->vdev_lock); + init_completion(&rkvenc->sclr_done); + INIT_DELAYED_WORK(&rkvenc->watchdog, rkvenc_watchdog_work); + + rkvenc->regs = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(rkvenc->regs)) + return PTR_ERR(rkvenc->regs); + + rkvenc_init_clocks_resets(rkvenc); + + ret = devm_clk_bulk_get(dev, rkvenc->num_clocks, rkvenc->clocks); + if (ret) + return ret; + + ret = devm_reset_control_bulk_get_exclusive(dev, rkvenc->num_resets, + rkvenc->resets); + if (ret) + return ret; + + ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); + if (ret) { + dev_err(dev, "Could not set DMA mask\n"); + return ret; + } + + ret = vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32)); + if (ret) + return ret; + + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return irq; + + ret = devm_request_threaded_irq(dev, irq, rkvenc_irq, + rkvenc_irq_thread, IRQF_ONESHOT, + dev_name(dev), rkvenc); + if (ret) { + dev_err(dev, "Could not request encoder IRQ\n"); + return ret; + } + + pm_runtime_enable(dev); + + ret = pm_runtime_resume_and_get(dev); + if (ret) + return ret; + + /* VERSION tells the 4K encoder from the 1080p one */ + ver = readl(rkvenc->regs + RKVENC_VERSION); + pic_size = (ver >> 12) & 0xf; + switch (pic_size) { + case 0: + rkvenc->max_width = 4096; + rkvenc->max_height = 2304; + break; + case 1: + rkvenc->max_width = 1920; + rkvenc->max_height = 1088; + break; + default: + dev_warn(dev, "unknown VERSION pic_size %u (0x%08x)\n", + pic_size, ver); + rkvenc->max_width = 4096; + rkvenc->max_height = 2304; + break; + } + if (pic_size != 0) + dev_info(dev, "RKVENC VERSION 0x%08x (pic_size %u)\n", + ver, pic_size); + + pm_runtime_put(dev); + + ret = v4l2_device_register(dev, &rkvenc->v4l2_dev); + if (ret) { + dev_err(dev, "Failed to register V4L2 device\n"); + goto err_pm_disable; + } + + rkvenc->m2m_dev = v4l2_m2m_init(&rkvenc_m2m_ops); + if (IS_ERR(rkvenc->m2m_dev)) { + v4l2_err(&rkvenc->v4l2_dev, "Failed to init mem2mem device\n"); + ret = PTR_ERR(rkvenc->m2m_dev); + goto err_unreg_v4l2; + } + + ret = rkvenc_v4l2_init(rkvenc); + if (ret) + goto err_release_m2m; + + return 0; + +err_release_m2m: + v4l2_m2m_release(rkvenc->m2m_dev); +err_unreg_v4l2: + v4l2_device_unregister(&rkvenc->v4l2_dev); +err_pm_disable: + pm_runtime_disable(dev); + return ret; +} + +static void rkvenc_remove(struct platform_device *pdev) +{ + struct rkvenc_dev *rkvenc = platform_get_drvdata(pdev); + + cancel_delayed_work_sync(&rkvenc->watchdog); + rkvenc_v4l2_cleanup(rkvenc); + v4l2_m2m_release(rkvenc->m2m_dev); + v4l2_device_unregister(&rkvenc->v4l2_dev); + pm_runtime_disable(&pdev->dev); +} + +static const struct of_device_id rkvenc_dt_match[] = { + { .compatible = "rockchip,rv1126-rkvenc", .data = &rkvenc_variant_rv1126 }, + { .compatible = "rockchip,rk3568-rkvenc", .data = &rkvenc_variant_rk3568 }, + { /* sentinel */ }, +}; +MODULE_DEVICE_TABLE(of, rkvenc_dt_match); + +static const struct dev_pm_ops rkvenc_pm_ops = { + RUNTIME_PM_OPS(rkvenc_runtime_suspend, rkvenc_runtime_resume, NULL) + SYSTEM_SLEEP_PM_OPS(rkvenc_suspend, rkvenc_resume) +}; + +static struct platform_driver rkvenc_driver = { + .probe = rkvenc_probe, + .remove = rkvenc_remove, + .driver = { + .name = "rockchip-rkvenc", + .of_match_table = rkvenc_dt_match, + .pm = pm_ptr(&rkvenc_pm_ops), + }, +}; + +static int __init rkvenc_init(void) +{ + rkvenc_tables_init(); + + return platform_driver_register(&rkvenc_driver); +} +module_init(rkvenc_init); + +static void __exit rkvenc_exit(void) +{ + platform_driver_unregister(&rkvenc_driver); +} +module_exit(rkvenc_exit); + +MODULE_AUTHOR("Daniel Golle <daniel@makrotopia.org>"); +MODULE_DESCRIPTION("Rockchip RKVENC v1 H.264/HEVC encoder driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h new file mode 100644 index 0000000000000..df5905a329107 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h @@ -0,0 +1,216 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Rockchip RKVENC v1 H.264 / HEVC encoder driver. + * + * struct rkvenc_frame carries no V4L2 types, so that a stateless front end + * can drive the same hardware back end. + */ + +#ifndef RKVENC_H +#define RKVENC_H + +#include <linux/clk.h> +#include <linux/completion.h> +#include <linux/delay.h> +#include <linux/mutex.h> +#include <linux/spinlock.h> +#include <linux/workqueue.h> +#include <linux/reset.h> + +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/v4l2-mem2mem.h> + +#include "rkvenc-vepu541.h" + +#define RKVENC_HDR_MAX 1024 + +#define RKVENC_SLOTS 2 +#define RKVENC_SLOT_BUFS 4 + +#define RKVENC_NUM_CLKS 3 +#define RKVENC_NUM_RESETS 3 + +#define RKVENC_MAX_CONSEC_ERRORS 2 + +struct rkvenc_dev; + +/** + * struct rkvenc_variant - per-SoC integration differences. + * @vepu540: RK3566/RK3568 "vepu540" rather than RV1109/RV1126 "vepu541". + */ +struct rkvenc_variant { + bool vepu540; +}; + +struct rkvenc_dev { + struct v4l2_device v4l2_dev; + struct media_device mdev; + struct video_device vdev; + struct v4l2_m2m_dev *m2m_dev; + struct device *dev; + void __iomem *regs; + + struct clk_bulk_data clocks[RKVENC_NUM_CLKS]; + int num_clocks; + struct reset_control_bulk_data resets[RKVENC_NUM_RESETS]; + int num_resets; + + const struct rkvenc_variant *variant; + + struct rkvenc_img img; + + struct mutex vdev_lock; /* serialises the video node ioctls */ + spinlock_t irq_lock; + struct completion sclr_done; + u32 irq_sta; + struct delayed_work watchdog; + bool job_running; + struct rkvenc_ctx *job_ctx; + u32 max_width; + u32 max_height; +}; + +struct rkvenc_ctrls { + u32 bitrate_mode; + u32 bitrate; + u32 vbv_size; + bool frame_rc_enable; + u32 gop_size; + u32 i_period; + u32 header_mode; + bool prepend_spspps; + bool force_keyframe; + + u8 profile; + u8 level; + u8 tier; + bool cabac; + bool transform_8x8; + u8 lf_mode; + s8 lf_alpha; + s8 lf_beta; + s8 hevc_beta_offset; + s8 hevc_tc_offset; + s8 chroma_qp_offset; + u8 min_qp; + u8 max_qp; + u8 i_frame_qp; + u8 p_frame_qp; + + u8 hevc_max_merge; + bool hevc_sao; + bool hevc_strong_intra; + bool hevc_tmv; + bool hevc_cabac_init; + + u32 framerate; +}; + +struct rkvenc_gop { + u32 frames_since_idr; + u32 idr_pic_id; + u16 frame_num; + u16 poc_lsb; +}; + +struct rkvenc_ctx { + struct v4l2_fh fh; + struct rkvenc_dev *dev; + struct v4l2_m2m_ctx *m2m_ctx; + struct v4l2_ctrl_handler ctrl_hdl; + struct mutex vlock; + + u32 src_fourcc; + u32 coded_fourcc; + enum rkvenc_codec codec; + u32 width; + u32 height; + u32 bytesperline; + u32 src_cfmt; + + struct rkvenc_ctrls c; + + struct rkvenc_gop gop; + struct rkvenc_slot slot[RKVENC_SLOTS]; + struct rkvenc_slot_sizes slot_sz; + void *slot_cpu[RKVENC_SLOTS][RKVENC_SLOT_BUFS]; + bool slots_alloc; + int cur; + int last; + bool last_good_valid; + int last_good; + u32 consecutive_errors; + bool overflow_retried; + + u32 src_num_planes; + + bool joined_header_done; + + s32 rc_buf; /* in bits */ + u32 rc_tavg; + s8 rc_qp[2]; /* indexed by frame type, 0 = I, 1 = P */ + u32 rc_x[2]; + s32 rc_buf_gop; + u32 rc_p_qp_sum; + u32 rc_p_qp_cnt; + bool rc_active; + bool rc_first[2]; + + bool header_pending; + u8 hdr[RKVENC_HDR_MAX]; + u32 hdr_len; + + u32 fps; + u32 blocks; + u32 qp_blocks; + + bool cur_intra; + u8 cur_qp; + u32 cur_prepend; + + struct rkvenc_h264_seq h264_seq; + struct rkvenc_hevc_seq hevc_seq; +}; + +/* rkvenc.c */ +int rkvenc_hw_run(struct rkvenc_dev *dev, const struct rkvenc_frame *f); +void rkvenc_hw_status_read(struct rkvenc_dev *dev, u32 *int_sta, u32 *bytes, + u32 *qp_sum); +void rkvenc_hw_recover(struct rkvenc_dev *dev); +int rkvenc_v4l2_init(struct rkvenc_dev *dev); +void rkvenc_v4l2_cleanup(struct rkvenc_dev *dev); + +/* rkvenc-v4l2.c */ +extern const struct v4l2_m2m_ops rkvenc_m2m_ops; + +/* rkvenc-gop.c */ +void rkvenc_gop_init(struct rkvenc_ctx *ctx); +bool rkvenc_gop_is_idr(struct rkvenc_ctx *ctx); +void rkvenc_gop_commit(struct rkvenc_ctx *ctx); +void rkvenc_gop_rollback(struct rkvenc_ctx *ctx); +void rkvenc_gop_force_keyframe(struct rkvenc_ctx *ctx); + +/* rkvenc-ref.c */ +int rkvenc_ref_alloc(struct rkvenc_ctx *ctx); +void rkvenc_ref_free(struct rkvenc_ctx *ctx); + +/* rkvenc-rc.c */ +void rkvenc_rc_init(struct rkvenc_ctx *ctx); +void rkvenc_rc_pick(struct rkvenc_ctx *ctx, u8 *qp, u8 frame_type); +void rkvenc_rc_update(struct rkvenc_ctx *ctx, u32 bytes, u32 qp_avg, + u8 frame_type); +void rkvenc_rc_overflow(struct rkvenc_ctx *ctx, u32 bytes, u8 frame_type); + +/* rkvenc-h264.c / rkvenc-hevc.c */ +int rkvenc_h264_setup(struct rkvenc_ctx *ctx); +int rkvenc_h264_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size); +void rkvenc_h264_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_h264_pic *pic, bool idr); + +int rkvenc_hevc_setup(struct rkvenc_ctx *ctx); +int rkvenc_hevc_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size); +void rkvenc_hevc_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_hevc_pic *pic, bool idr); + +#endif /* RKVENC_H */ -- 2.56.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 3/6] media: rockchip: Add RKVENC v1 video encoder driver @ 2026-11-14 14:54 ` Daniel Golle 0 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:54 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add a stateful V4L2 memory-to-memory encoder driver for the RKVENC v1 H.264 and HEVC encoder of the RV1109/RV1126 ("vepu541") and the RK3566/RK3568 ("vepu540"). The driver takes NV12 or NV12M and produces low-delay streams of IDR and P pictures with one short-term reference, POC type 0 and no B frames. It writes the SPS, PPS and VPS itself, the hardware writes the slice headers and data. The SEPARATE and JOINED_WITH_1ST_FRAME header modes are supported, the latter also with PREPEND_SPSPPS_TO_IDR. Pictures too small for P frames are coded as IDR pictures only. The register image builder takes a struct rkvenc_frame with no V4L2 types, so that a stateless front end can drive the same hardware back end; rkvenc-v4l2.c is the stateful front end on the mem2mem framework. Rate control is frame level CBR: the hardware row rate control stays off and every picture is coded at one QP. An I picture takes the mean P QP of the previous GOP lowered by 4 for HEVC and 2 for H.264, the P QP moves by at most one per picture, and the deviation from the plan is taken at GOP boundaries and spread over the next GOP. A bitstream overflow is fed to the rate control and the picture is encoded once more. A failed job is stopped with a force clear, and with a safe clear and a block reset only when that does not stop it. Source frames are imported without a kernel mapping and a capture buffer is sized to half the pixel count, so that 4K streams fit the vmalloc space of 32-bit systems. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- MAINTAINERS | 1 + drivers/media/platform/rockchip/Kconfig | 1 + drivers/media/platform/rockchip/Makefile | 1 + .../media/platform/rockchip/rkvenc/Kconfig | 17 + .../media/platform/rockchip/rkvenc/Makefile | 15 + .../platform/rockchip/rkvenc/rkvenc-gop.c | 72 ++ .../platform/rockchip/rkvenc/rkvenc-h264.c | 82 ++ .../platform/rockchip/rkvenc/rkvenc-hevc.c | 63 + .../platform/rockchip/rkvenc/rkvenc-nal.c | 351 ++++++ .../platform/rockchip/rkvenc/rkvenc-nal.h | 12 + .../platform/rockchip/rkvenc/rkvenc-rc.c | 214 ++++ .../platform/rockchip/rkvenc/rkvenc-ref.c | 114 ++ .../platform/rockchip/rkvenc/rkvenc-v4l2.c | 1069 +++++++++++++++++ .../rockchip/rkvenc/rkvenc-vepu541-h264.c | 206 ++++ .../rockchip/rkvenc/rkvenc-vepu541-hevc.c | 182 +++ .../rockchip/rkvenc/rkvenc-vepu541-regs.h | 283 +++++ .../rockchip/rkvenc/rkvenc-vepu541-tables.c | 256 ++++ .../platform/rockchip/rkvenc/rkvenc-vepu541.c | 215 ++++ .../platform/rockchip/rkvenc/rkvenc-vepu541.h | 201 ++++ .../media/platform/rockchip/rkvenc/rkvenc.c | 489 ++++++++ .../media/platform/rockchip/rkvenc/rkvenc.h | 216 ++++ 21 files changed, 4060 insertions(+) create mode 100644 drivers/media/platform/rockchip/rkvenc/Kconfig create mode 100644 drivers/media/platform/rockchip/rkvenc/Makefile create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.c create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc.h diff --git a/MAINTAINERS b/MAINTAINERS index 26991dd3a3569..a70ee0cf2655b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -24550,6 +24550,7 @@ L: linux-media@vger.kernel.org L: linux-rockchip@lists.infradead.org S: Maintained F: Documentation/devicetree/bindings/media/rockchip,rv1126-rkvenc.yaml +F: drivers/media/platform/rockchip/rkvenc/ ROCKCHIP RK3308 INTERNAL AUDIO CODEC M: Luca Ceresoli <luca.ceresoli@bootlin.com> diff --git a/drivers/media/platform/rockchip/Kconfig b/drivers/media/platform/rockchip/Kconfig index ba401d32f01ba..e6afa60923f1b 100644 --- a/drivers/media/platform/rockchip/Kconfig +++ b/drivers/media/platform/rockchip/Kconfig @@ -6,3 +6,4 @@ source "drivers/media/platform/rockchip/rga/Kconfig" source "drivers/media/platform/rockchip/rkcif/Kconfig" source "drivers/media/platform/rockchip/rkisp1/Kconfig" source "drivers/media/platform/rockchip/rkvdec/Kconfig" +source "drivers/media/platform/rockchip/rkvenc/Kconfig" diff --git a/drivers/media/platform/rockchip/Makefile b/drivers/media/platform/rockchip/Makefile index 0e0b2cbbd4bd2..6a1d847a10a43 100644 --- a/drivers/media/platform/rockchip/Makefile +++ b/drivers/media/platform/rockchip/Makefile @@ -3,3 +3,4 @@ obj-y += rga/ obj-y += rkcif/ obj-y += rkisp1/ obj-y += rkvdec/ +obj-y += rkvenc/ diff --git a/drivers/media/platform/rockchip/rkvenc/Kconfig b/drivers/media/platform/rockchip/rkvenc/Kconfig new file mode 100644 index 0000000000000..1d76314a2ea6d --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/Kconfig @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0-only +config VIDEO_ROCKCHIP_RKVENC + tristate "Rockchip RKVENC (vepu540/vepu541) H.264/HEVC encoder" + depends on ARCH_ROCKCHIP || COMPILE_TEST + depends on VIDEO_DEV + select MEDIA_CONTROLLER + select VIDEOBUF2_DMA_CONTIG + select V4L2_MEM2MEM_DEV + help + Support for the Rockchip RKVENC v1 hardware video encoder found + in the RV1109/RV1126 (vepu541) and RK3566/RK3568 (vepu540) SoCs. + + It encodes NV12 raw frames into H.264 (AVC) or HEVC (H.265) + bitstreams using a stateful memory-to-memory V4L2 interface. + + To compile this driver as a module, choose M here: the module + will be called rockchip-rkvenc. diff --git a/drivers/media/platform/rockchip/rkvenc/Makefile b/drivers/media/platform/rockchip/rkvenc/Makefile new file mode 100644 index 0000000000000..026c81aec492c --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/Makefile @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +rockchip-rkvenc-y += rkvenc.o \ + rkvenc-v4l2.o \ + rkvenc-gop.o \ + rkvenc-ref.o \ + rkvenc-rc.o \ + rkvenc-h264.o \ + rkvenc-hevc.o \ + rkvenc-nal.o \ + rkvenc-vepu541.o \ + rkvenc-vepu541-h264.o \ + rkvenc-vepu541-hevc.o \ + rkvenc-vepu541-tables.o + +obj-$(CONFIG_VIDEO_ROCKCHIP_RKVENC) += rockchip-rkvenc.o diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c new file mode 100644 index 0000000000000..208bb122a3f87 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Low-delay IDR + P structure of the RKVENC v1 encoder: one short-term + * reference, no B frames. A failed frame never commits, so rollback is + * free. + */ + +#include "rkvenc.h" + +void rkvenc_gop_init(struct rkvenc_ctx *ctx) +{ + ctx->gop.frames_since_idr = 0; + ctx->gop.idr_pic_id = 0; + ctx->cur = 0; + ctx->last = 0; + ctx->last_good_valid = false; + ctx->consecutive_errors = 0; +} + +/* The hardware codes P frames of these sizes wrongly */ +static bool rkvenc_gop_intra_only(struct rkvenc_ctx *ctx) +{ + u32 w16 = ALIGN(ctx->width, 16) / 16; + u32 h16 = ALIGN(ctx->height, 16) / 16; + + if (ctx->codec == RKVENC_CODEC_HEVC) + return ctx->height <= 64; + + return !((w16 >= 3 && h16 >= 2) || (w16 == 1 && h16 == 1)); +} + +bool rkvenc_gop_is_idr(struct rkvenc_ctx *ctx) +{ + struct rkvenc_gop *g = &ctx->gop; + + if (g->frames_since_idr == 0 || rkvenc_gop_intra_only(ctx)) + return true; + if (ctx->c.gop_size && g->frames_since_idr >= ctx->c.gop_size) + return true; + if (ctx->c.force_keyframe) + return true; + if (!ctx->last_good_valid) + return true; + return false; +} + +void rkvenc_gop_commit(struct rkvenc_ctx *ctx) +{ + struct rkvenc_gop *g = &ctx->gop; + + if (ctx->cur_intra) { + g->idr_pic_id++; + ctx->last_good = ctx->cur; + ctx->last_good_valid = true; + g->frames_since_idr = 1; + } else { + g->frames_since_idr++; + } + ctx->last = ctx->cur; + ctx->cur ^= 1; + ctx->c.force_keyframe = false; +} + +void rkvenc_gop_rollback(struct rkvenc_ctx *ctx) +{ + ctx->c.force_keyframe = false; +} + +void rkvenc_gop_force_keyframe(struct rkvenc_ctx *ctx) +{ + ctx->c.force_keyframe = true; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c new file mode 100644 index 0000000000000..391755a42dd3b --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-h264.c @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * H.264 parameter sets of the RKVENC v1 encoder. One struct + * rkvenc_h264_seq feeds both the slice header registers and the SPS/PPS + * writer, so the two cannot disagree. + */ + +#include <linux/string.h> + +#include "rkvenc.h" +#include "rkvenc-nal.h" + +static const u8 rkvenc_h264_level_idc[] = { + 10, 11, 11, 12, 13, 20, 21, 22, 30, 31, 32, 40, + 41, 42, 50, 51, 52, 60, 61, 62, +}; + +static u8 rkvenc_h264_level(u32 ctrl) +{ + if (ctrl >= ARRAY_SIZE(rkvenc_h264_level_idc)) + return 62; + return rkvenc_h264_level_idc[ctrl]; +} + +static u8 rkvenc_h264_profile_idc(u32 ctrl) +{ + switch (ctrl) { + case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE: + case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE: + return 66; + case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN: + return 77; + case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH: + default: + return 100; + } +} + +int rkvenc_h264_setup(struct rkvenc_ctx *ctx) +{ + struct rkvenc_h264_seq *seq = &ctx->h264_seq; + + memset(seq, 0, sizeof(*seq)); + seq->width = ctx->width; + seq->height = ctx->height; + seq->profile_idc = rkvenc_h264_profile_idc(ctx->c.profile); + seq->level_idc = rkvenc_h264_level(ctx->c.level); + seq->log2_max_frame_num_m4 = 4; + seq->log2_max_poc_lsb_m4 = 5; + seq->pic_init_qp = ctx->c.p_frame_qp; + seq->chroma_qp_offset = ctx->c.chroma_qp_offset; + seq->cabac = ctx->c.cabac && seq->profile_idc != 66; + if (ctx->c.profile == V4L2_MPEG_VIDEO_H264_PROFILE_HIGH) + seq->transform_8x8 = ctx->c.transform_8x8; + seq->dblk_idc = ctx->c.lf_mode; + seq->alpha_offset = ctx->c.lf_alpha; + seq->beta_offset = ctx->c.lf_beta; + seq->timing = true; + seq->fps = ctx->fps ?: 30; + + return rkvenc_h264_headers_write(seq, ctx->hdr, RKVENC_HDR_MAX); +} + +int rkvenc_h264_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size) +{ + if (ctx->hdr_len > size) + return -ENOSPC; + memcpy(buf, ctx->hdr, ctx->hdr_len); + return ctx->hdr_len; +} + +void rkvenc_h264_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_h264_pic *pic, bool idr) +{ + struct rkvenc_gop *g = &ctx->gop; + + pic->seq = &ctx->h264_seq; + pic->nal_ref_idc = idr ? 3 : 2; + pic->frame_num = g->frames_since_idr & 0xff; + pic->poc_lsb = (2 * g->frames_since_idr) & 0x1ff; + pic->idr_pic_id = g->idr_pic_id & 0xffff; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c new file mode 100644 index 0000000000000..49bf3cccdf8a0 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c @@ -0,0 +1,63 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * HEVC parameter sets of the RKVENC v1 encoder. One struct + * rkvenc_hevc_seq feeds both the slice header registers and the + * VPS/SPS/PPS writer. + */ + +#include <linux/string.h> + +#include "rkvenc.h" +#include "rkvenc-nal.h" + +#define HEVC_NAL_TRAIL_R 1 +#define HEVC_NAL_IDR_W_RADL 19 + +static const u16 rkvenc_hevc_level_idc[] = { + 30, 60, 63, 90, 93, 120, 123, 150, 153, 156, 180, 183, 186, +}; + +static u16 rkvenc_hevc_level(u32 ctrl) +{ + if (ctrl >= ARRAY_SIZE(rkvenc_hevc_level_idc)) + return 186; + return rkvenc_hevc_level_idc[ctrl]; +} + +int rkvenc_hevc_setup(struct rkvenc_ctx *ctx) +{ + struct rkvenc_hevc_seq *seq = &ctx->hevc_seq; + + memset(seq, 0, sizeof(*seq)); + seq->width = ctx->width; + seq->height = ctx->height; + seq->level_idc = rkvenc_hevc_level(ctx->c.level); + seq->log2_max_poc_lsb_m4 = 4; + seq->pic_init_qp = ctx->c.p_frame_qp; + seq->max_merge_cand = ctx->c.hevc_max_merge ?: 2; + seq->sao = ctx->c.hevc_sao; + seq->deblock_disabled = ctx->c.lf_mode == + V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_DISABLED; + seq->beta_offset = ctx->c.hevc_beta_offset; + seq->tc_offset = ctx->c.hevc_tc_offset; + seq->strong_intra_smoothing = ctx->c.hevc_strong_intra; + seq->cabac_init_present = ctx->c.hevc_cabac_init; + + return rkvenc_hevc_headers_write(seq, ctx->hdr, RKVENC_HDR_MAX); +} + +int rkvenc_hevc_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size) +{ + if (ctx->hdr_len > size) + return -ENOSPC; + memcpy(buf, ctx->hdr, ctx->hdr_len); + return ctx->hdr_len; +} + +void rkvenc_hevc_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_hevc_pic *pic, bool idr) +{ + pic->seq = &ctx->hevc_seq; + pic->nal_unit_type = idr ? HEVC_NAL_IDR_W_RADL : HEVC_NAL_TRAIL_R; + pic->poc_lsb = ctx->gop.frames_since_idr & 0xff; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c new file mode 100644 index 0000000000000..b5a3f01852c0b --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.c @@ -0,0 +1,351 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * H.264 SPS/PPS (ITU-T H.264 7.3.2) and HEVC VPS/SPS/PPS (ITU-T H.265 7.3.2) + * writer for the RKVENC v1 encoder. + */ + +#include "rkvenc-nal.h" + +#define NAL_RBSP_MAX 256 + +#define H264_NAL_SPS 7 +#define H264_NAL_PPS 8 +#define H264_PROFILE_HIGH 100 +#define HEVC_NAL_VPS 32 +#define HEVC_NAL_SPS 33 +#define HEVC_NAL_PPS 34 + +struct nal_bw { + u8 buf[NAL_RBSP_MAX]; + u32 bits; + bool overflow; +}; + +static void bw_put(struct nal_bw *bw, u32 val, u32 bits) +{ + u32 i, bit; + + for (i = bits; i > 0; i--) { + if (bw->bits >= 8 * NAL_RBSP_MAX) { + bw->overflow = true; + return; + } + bit = (val >> (i - 1)) & 1; + if (bit) + bw->buf[bw->bits / 8] |= 0x80 >> (bw->bits % 8); + bw->bits++; + } +} + +static void bw_ue(struct nal_bw *bw, u32 val) +{ + u32 len = 32 - __builtin_clz(val + 1); + + bw_put(bw, 0, len - 1); + bw_put(bw, val + 1, len); +} + +static void bw_se(struct nal_bw *bw, s32 val) +{ + bw_ue(bw, val > 0 ? 2 * val - 1 : -2 * val); +} + +static void bw_trailing(struct nal_bw *bw) +{ + bw_put(bw, 1, 1); + while (bw->bits % 8) + bw_put(bw, 0, 1); +} + +static int nal_emit(const struct nal_bw *bw, const u8 *hdr, u32 hdr_len, + u8 *out, u32 size) +{ + u32 len = bw->bits / 8, pos = 0, zeros = 0, i; + + if (bw->overflow || size < 4 + hdr_len) + return -ENOSPC; + + out[pos++] = 0; + out[pos++] = 0; + out[pos++] = 0; + out[pos++] = 1; + for (i = 0; i < hdr_len; i++) + out[pos++] = hdr[i]; + + /* Emulation prevention, H.264 7.4.1 */ + for (i = 0; i < len; i++) { + if (pos + 2 > size) + return -ENOSPC; + if (zeros == 2 && bw->buf[i] <= 3) { + out[pos++] = 3; + zeros = 0; + } + out[pos++] = bw->buf[i]; + zeros = bw->buf[i] ? 0 : zeros + 1; + } + + return pos; +} + +static void h264_vui_write(struct nal_bw *bw, const struct rkvenc_h264_seq *seq) +{ + bw_put(bw, 0, 4); + bw_put(bw, seq->timing, 1); + if (seq->timing) { + bw_put(bw, 1, 32); + bw_put(bw, 2 * seq->fps, 32); + bw_put(bw, 1, 1); + } + bw_put(bw, 0, 3); + bw_put(bw, 1, 1); /* bitstream_restriction_flag */ + bw_put(bw, 1, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, 16); + bw_ue(bw, 16); + bw_ue(bw, 0); /* max_num_reorder_frames */ + bw_ue(bw, 1); /* max_dec_frame_buffering */ +} + +static void h264_sps_write(struct nal_bw *bw, const struct rkvenc_h264_seq *seq) +{ + u32 w16 = ALIGN(seq->width, 16), h16 = ALIGN(seq->height, 16); + bool crop = w16 != seq->width || h16 != seq->height; + u32 constraints; + + if (seq->profile_idc == H264_PROFILE_HIGH) + constraints = 0; + else if (seq->profile_idc == 66) + constraints = 0xc0; + else + constraints = 0x40; + + bw_put(bw, seq->profile_idc, 8); + bw_put(bw, constraints, 8); + bw_put(bw, seq->level_idc, 8); + bw_ue(bw, 0); + if (seq->profile_idc == H264_PROFILE_HIGH) { + bw_ue(bw, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 2); + } + bw_ue(bw, seq->log2_max_frame_num_m4); + bw_ue(bw, 0); + bw_ue(bw, seq->log2_max_poc_lsb_m4); + bw_ue(bw, 1); + bw_put(bw, 0, 1); + bw_ue(bw, w16 / 16 - 1); + bw_ue(bw, h16 / 16 - 1); + bw_put(bw, 1, 1); + bw_put(bw, 1, 1); + bw_put(bw, crop, 1); + if (crop) { + bw_ue(bw, 0); + bw_ue(bw, (w16 - seq->width) / 2); + bw_ue(bw, 0); + bw_ue(bw, (h16 - seq->height) / 2); + } + bw_put(bw, 1, 1); + h264_vui_write(bw, seq); + bw_trailing(bw); +} + +static void h264_pps_write(struct nal_bw *bw, const struct rkvenc_h264_seq *seq) +{ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, seq->cabac, 1); + bw_put(bw, 0, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 3); + bw_se(bw, seq->pic_init_qp - 26); + bw_se(bw, 0); + bw_se(bw, seq->chroma_qp_offset); + bw_put(bw, 1, 1); + bw_put(bw, 0, 2); + if (seq->transform_8x8) { + bw_put(bw, 1, 1); + bw_put(bw, 0, 1); + bw_se(bw, seq->chroma_qp_offset); + } + bw_trailing(bw); +} + +int rkvenc_h264_headers_write(const struct rkvenc_h264_seq *seq, u8 *buf, + u32 size) +{ + u8 hdr; + struct nal_bw bw; + int len, pos; + + memset(&bw, 0, sizeof(bw)); + h264_sps_write(&bw, seq); + hdr = 3 << 5 | H264_NAL_SPS; + len = nal_emit(&bw, &hdr, 1, buf, size); + if (len < 0) + return len; + pos = len; + + memset(&bw, 0, sizeof(bw)); + h264_pps_write(&bw, seq); + hdr = 3 << 5 | H264_NAL_PPS; + len = nal_emit(&bw, &hdr, 1, buf + pos, size - pos); + if (len < 0) + return len; + + return pos + len; +} + +static void hevc_ptl_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + bw_put(bw, 0, 2); + bw_put(bw, 0, 1); + bw_put(bw, 1, 5); /* general_profile_idc: Main */ + bw_put(bw, 0x60000000, 32); /* compatible with profiles 1, 2 */ + bw_put(bw, 1, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 1, 1); + bw_put(bw, 0, 32); + bw_put(bw, 0, 12); + bw_put(bw, seq->level_idc, 8); +} + +static void hevc_vps_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + bw_put(bw, 0, 4); + bw_put(bw, 1, 1); + bw_put(bw, 1, 1); + bw_put(bw, 0, 6); + bw_put(bw, 0, 3); + bw_put(bw, 1, 1); + bw_put(bw, 0xffff, 16); + hevc_ptl_write(bw, seq); + bw_put(bw, 1, 1); + bw_ue(bw, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 6); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_trailing(bw); +} + +static void hevc_sps_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + u32 w8 = ALIGN(seq->width, 8), h8 = ALIGN(seq->height, 8); + bool crop = w8 != seq->width || h8 != seq->height; + + bw_put(bw, 0, 4); + bw_put(bw, 0, 3); + bw_put(bw, 1, 1); + hevc_ptl_write(bw, seq); + bw_ue(bw, 0); + bw_ue(bw, 1); /* chroma_format_idc 4:2:0 */ + bw_ue(bw, w8); + bw_ue(bw, h8); + bw_put(bw, crop, 1); + if (crop) { + bw_ue(bw, 0); + bw_ue(bw, (w8 - seq->width) / 2); + bw_ue(bw, 0); + bw_ue(bw, (h8 - seq->height) / 2); + } + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, seq->log2_max_poc_lsb_m4); + bw_put(bw, 1, 1); + bw_ue(bw, 1); /* sps_max_dec_pic_buffering_minus1 */ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_ue(bw, 0); /* min CU 8x8 */ + bw_ue(bw, 3); /* CTU 64x64 */ + bw_ue(bw, 0); /* min TU 4x4 */ + bw_ue(bw, 3); /* max TU 32x32 */ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, seq->sao, 1); + bw_put(bw, 0, 1); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, seq->strong_intra_smoothing, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_trailing(bw); +} + +static void hevc_pps_write(struct nal_bw *bw, const struct rkvenc_hevc_seq *seq) +{ + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 3); + bw_put(bw, 0, 1); + bw_put(bw, seq->cabac_init_present, 1); + bw_ue(bw, 0); + bw_ue(bw, 0); + bw_se(bw, seq->pic_init_qp - 26); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 1, 1); /* cu_qp_delta_enabled_flag */ + bw_ue(bw, 0); + bw_se(bw, 0); + bw_se(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_put(bw, 1, 1); + bw_put(bw, 0, 1); + bw_put(bw, seq->deblock_disabled, 1); + if (!seq->deblock_disabled) { + bw_se(bw, seq->beta_offset); + bw_se(bw, seq->tc_offset); + } + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_ue(bw, 0); + bw_put(bw, 0, 1); + bw_put(bw, 0, 1); + bw_trailing(bw); +} + +int rkvenc_hevc_headers_write(const struct rkvenc_hevc_seq *seq, u8 *buf, + u32 size) +{ + static const u8 types[] = { HEVC_NAL_VPS, HEVC_NAL_SPS, HEVC_NAL_PPS }; + struct nal_bw bw; + int len, pos; + u8 hdr[2]; + u32 i; + + pos = 0; + for (i = 0; i < ARRAY_SIZE(types); i++) { + memset(&bw, 0, sizeof(bw)); + if (types[i] == HEVC_NAL_VPS) + hevc_vps_write(&bw, seq); + else if (types[i] == HEVC_NAL_SPS) + hevc_sps_write(&bw, seq); + else + hevc_pps_write(&bw, seq); + hdr[0] = types[i] << 1; + hdr[1] = 1; + len = nal_emit(&bw, hdr, 2, buf + pos, size - pos); + if (len < 0) + return len; + pos += len; + } + + return pos; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h new file mode 100644 index 0000000000000..58b30d2ac6124 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-nal.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef RKVENC_NAL_H +#define RKVENC_NAL_H + +#include "rkvenc-vepu541.h" + +int rkvenc_h264_headers_write(const struct rkvenc_h264_seq *seq, u8 *buf, + u32 size); +int rkvenc_hevc_headers_write(const struct rkvenc_hevc_seq *seq, u8 *buf, + u32 size); + +#endif diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c new file mode 100644 index 0000000000000..b7ab57e490cf0 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c @@ -0,0 +1,214 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Frame-level rate control of the RKVENC v1 encoder. The hardware row-level + * rate control is off, every picture is coded at one QP picked here. + */ + +#include <linux/math.h> +#include <linux/math64.h> + +#include "rkvenc.h" + +/* qs[q] = round(256 * 2^((q-4)/6)) in Q8, q = 0..51. */ +#define RKVENC_QS_BASE { 256, 287, 323, 362, 406, 456 } + +#define RKVENC_RC_DEFAULT_FPS 30u + +/* I/P complexity ratio cap, Q8; static scenes reach 37 */ +#define RKVENC_RC_W_CLAMP (64u * 256) +/* A larger step makes the P QP alternate between two values */ +#define RKVENC_RC_QP_STEP 1 +/* Q8 weight of the newest P frame in the P complexity */ +#define RKVENC_RC_X_WEIGHT 32 + +/* I frame QP offset below the mean P frame QP */ +#define RKVENC_RC_QP_DELTA_IP_HEVC 4 +#define RKVENC_RC_QP_DELTA_IP_H264 2 + +static u32 rkvenc_qs[RKVENC_QP_MAX + 1]; + +static void rkvenc_qs_build(void) +{ + static const u32 base[6] = RKVENC_QS_BASE; + unsigned int q; + int k, sh, m; + u64 v; + + for (q = 0; q <= RKVENC_QP_MAX; q++) { + k = (int)q - 4; + /* floor(k / 6), correct for negative k. */ + sh = (k - (k < 0 ? 5 : 0)) / 6; + m = k - 6 * sh; + if (sh >= 0) + v = (u64)base[m] << sh; + else + v = (u64)base[m] >> (-sh); + rkvenc_qs[q] = (u32)v; + } +} + +static u32 rkvenc_rc_capacity(struct rkvenc_ctx *ctx) +{ + if (ctx->c.vbv_size) + return ctx->c.vbv_size; + return ctx->rc_tavg * max_t(u32, ctx->c.gop_size, 1); +} + +/* The plan budgets the I frame, so the bucket is on plan at GOP starts */ +static u64 rkvenc_rc_correct(struct rkvenc_ctx *ctx, u64 t_t, u32 n) +{ + s64 c = div_s64(ctx->rc_buf_gop, n); + + if (c < 0) + return t_t + (u64)-c; + if ((u64)c >= t_t) + return 1; + return t_t - (u64)c; +} + +static u32 rkvenc_rc_complexity(struct rkvenc_ctx *ctx, unsigned int t, u64 s, + u32 qp_avg) +{ + u32 x = (u32)div_u64(s * rkvenc_qs[min(qp_avg, RKVENC_QP_MAX)], 256); + + /* a P frame below the skip threshold alone would swing the model */ + if (!t || ctx->rc_first[1]) + return x; + return (u32)div_u64((u64)ctx->rc_x[1] * (256 - RKVENC_RC_X_WEIGHT) + + (u64)x * RKVENC_RC_X_WEIGHT, + 256); +} + +static int rkvenc_rc_p_qp_mean(struct rkvenc_ctx *ctx) +{ + int q = ctx->rc_qp[1]; + + if (ctx->rc_p_qp_cnt) + q = DIV_ROUND_CLOSEST(ctx->rc_p_qp_sum, ctx->rc_p_qp_cnt); + ctx->rc_p_qp_sum = 0; + ctx->rc_p_qp_cnt = 0; + return q; +} + +void rkvenc_rc_init(struct rkvenc_ctx *ctx) +{ + u32 bitrate = ctx->c.bitrate ?: 4000000; + u32 fps = ctx->fps ?: RKVENC_RC_DEFAULT_FPS; + + rkvenc_qs_build(); + + ctx->rc_tavg = div_u64((u64)bitrate, fps); + if (!ctx->rc_tavg) + ctx->rc_tavg = 1; + ctx->rc_buf = 0; + ctx->rc_buf_gop = 0; + ctx->rc_p_qp_sum = 0; + ctx->rc_p_qp_cnt = 0; + ctx->rc_x[0] = ctx->rc_x[1] = 0; + ctx->rc_qp[0] = ctx->c.i_frame_qp; + ctx->rc_qp[1] = ctx->c.p_frame_qp; + ctx->rc_first[0] = true; + ctx->rc_first[1] = true; +} + +void rkvenc_rc_pick(struct rkvenc_ctx *ctx, u8 *qp, u8 frame_type) +{ + unsigned int t = frame_type ? 1 : 0; + u8 lo = ctx->c.min_qp; + u8 hi = ctx->c.max_qp; + s32 prev = ctx->rc_qp[t]; + u32 target; + int q; + + if (!ctx->rc_active) { + *qp = frame_type ? ctx->c.p_frame_qp : ctx->c.i_frame_qp; + return; + } + + if (ctx->rc_first[t]) { + *qp = frame_type ? ctx->c.p_frame_qp : ctx->c.i_frame_qp; + return; + } + + if (!t) + ctx->rc_buf_gop = ctx->rc_buf; + + /* An I frame coarser than the P frames of the GOP shows as a pulse */ + if (!t && !ctx->rc_first[1]) { + q = rkvenc_rc_p_qp_mean(ctx); + if (ctx->codec == RKVENC_CODEC_HEVC) + q -= RKVENC_RC_QP_DELTA_IP_HEVC; + else + q -= RKVENC_RC_QP_DELTA_IP_H264; + q = clamp_t(int, q, lo, hi); + ctx->rc_qp[0] = (s8)q; + *qp = q; + return; + } + + if (ctx->rc_x[0] && ctx->rc_x[1]) { + u64 w = div_u64((u64)ctx->rc_x[0] * 256, + ctx->rc_x[1] ?: 1); + u32 n = max_t(u32, ctx->c.gop_size, 1); + u64 t_p, t_t; + + w = min_t(u64, w, RKVENC_RC_W_CLAMP); + t_p = div64_u64((u64)ctx->rc_tavg * n * 256, + w + 256 * (n - 1)); + t_t = t ? t_p : t_p * w / 256; + t_t = rkvenc_rc_correct(ctx, t_t, n); + + target = (u32)div_u64((u64)ctx->rc_x[t] * 256, t_t); + + for (q = 0; q <= RKVENC_QP_MAX; q++) + if (rkvenc_qs[q] >= target) + break; + if (q > RKVENC_QP_MAX) + q = RKVENC_QP_MAX; + } else { + q = prev; + } + + q = clamp_t(int, q, lo, hi); + if (q > prev + RKVENC_RC_QP_STEP) + q = prev + RKVENC_RC_QP_STEP; + if (q < prev - RKVENC_RC_QP_STEP) + q = prev - RKVENC_RC_QP_STEP; + q = clamp_t(int, q, lo, hi); + + ctx->rc_qp[t] = (s8)q; + if (t) { + ctx->rc_p_qp_sum += q; + ctx->rc_p_qp_cnt++; + } + *qp = q; +} + +void rkvenc_rc_update(struct rkvenc_ctx *ctx, u32 bytes, u32 qp_avg, + u8 frame_type) +{ + unsigned int t = frame_type ? 1 : 0; + u64 s = (u64)bytes * 8; + s64 cap = max_t(u64, rkvenc_rc_capacity(ctx), 1); + + if (!ctx->rc_active) + return; + + ctx->rc_x[t] = rkvenc_rc_complexity(ctx, t, s, qp_avg); + + ctx->rc_buf += (s32)s - (s32)ctx->rc_tavg; + ctx->rc_buf = clamp_t(s64, ctx->rc_buf, -cap, cap); + + ctx->rc_first[t] = false; +} + +/* The step limit alone would need frames that the overflow keeps dropping */ +void rkvenc_rc_overflow(struct rkvenc_ctx *ctx, u32 bytes, u8 frame_type) +{ + s8 start = frame_type ? ctx->c.p_frame_qp : ctx->c.i_frame_qp; + unsigned int t = frame_type ? 1 : 0; + + rkvenc_rc_update(ctx, bytes, ctx->cur_qp, frame_type); + if (ctx->rc_qp[t] < start) + ctx->rc_qp[t] = start; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c new file mode 100644 index 0000000000000..b0f15d31f1e11 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-ref.c @@ -0,0 +1,114 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Reconstruction and reference buffers of the RKVENC v1 encoder: two slots + * alternate as the single short-term reference. + */ + +#include <linux/dma-mapping.h> +#include <linux/slab.h> + +#include "rkvenc.h" + +enum { + RKVENC_BUF_HDR = 0, + RKVENC_BUF_BODY, + RKVENC_BUF_DSP, + RKVENC_BUF_CMV, +}; + +static void rkvenc_ref_free_slot(struct rkvenc_ctx *ctx, unsigned int i) +{ + struct device *dev = ctx->dev->dev; + size_t sizes[RKVENC_SLOT_BUFS]; + u32 handles[RKVENC_SLOT_BUFS]; + unsigned int b; + + sizes[RKVENC_BUF_HDR] = ctx->slot_sz.rfp_hdr; + sizes[RKVENC_BUF_BODY] = ctx->slot_sz.rfp_body; + sizes[RKVENC_BUF_DSP] = ctx->slot_sz.dsp; + sizes[RKVENC_BUF_CMV] = ctx->slot_sz.cmv; + handles[RKVENC_BUF_HDR] = ctx->slot[i].rfp_hdr; + handles[RKVENC_BUF_BODY] = ctx->slot[i].rfp_body; + handles[RKVENC_BUF_DSP] = ctx->slot[i].dsp; + handles[RKVENC_BUF_CMV] = ctx->slot[i].cmv; + + for (b = 0; b < RKVENC_SLOT_BUFS; b++) { + if (ctx->slot_cpu[i][b]) + dma_free_attrs(dev, sizes[b], ctx->slot_cpu[i][b], + handles[b], DMA_ATTR_NO_KERNEL_MAPPING); + ctx->slot_cpu[i][b] = NULL; + } + memset(&ctx->slot[i], 0, sizeof(ctx->slot[i])); +} + +int rkvenc_ref_alloc(struct rkvenc_ctx *ctx) +{ + struct device *dev = ctx->dev->dev; + struct rkvenc_slot_sizes *sz = &ctx->slot_sz; + dma_addr_t dma; + void *cpu; + unsigned int i; + int ret = 0; + + if (ctx->slots_alloc) + return 0; + + rkvenc_vepu541_slot_sizes(ctx->codec, ctx->width, ctx->height, sz); + + for (i = 0; i < RKVENC_SLOTS; i++) { + cpu = dma_alloc_attrs(dev, sz->rfp_hdr, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_HDR] = cpu; + ctx->slot[i].rfp_hdr = lower_32_bits(dma); + + cpu = dma_alloc_attrs(dev, sz->rfp_body, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_BODY] = cpu; + ctx->slot[i].rfp_body = lower_32_bits(dma); + + cpu = dma_alloc_attrs(dev, sz->dsp, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_DSP] = cpu; + ctx->slot[i].dsp = lower_32_bits(dma); + + if (ctx->codec == RKVENC_CODEC_HEVC && sz->cmv) { + cpu = dma_alloc_attrs(dev, sz->cmv, &dma, GFP_KERNEL, + DMA_ATTR_NO_KERNEL_MAPPING); + if (!cpu) + goto err; + ctx->slot_cpu[i][RKVENC_BUF_CMV] = cpu; + ctx->slot[i].cmv = lower_32_bits(dma); + } else { + ctx->slot[i].cmv = 0; + } + } + + ctx->slots_alloc = true; + return 0; + +err: + for (i = 0; i < RKVENC_SLOTS; i++) + rkvenc_ref_free_slot(ctx, i); + ret = -ENOMEM; + return ret; +} + +void rkvenc_ref_free(struct rkvenc_ctx *ctx) +{ + unsigned int i; + + if (!ctx->slots_alloc) + return; + + for (i = 0; i < RKVENC_SLOTS; i++) + rkvenc_ref_free_slot(ctx, i); + ctx->slots_alloc = false; + ctx->cur = 0; + ctx->last = 0; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c new file mode 100644 index 0000000000000..3be496f5cf0c1 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c @@ -0,0 +1,1069 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * V4L2 stateful encoder front end of the Rockchip RKVENC v1 encoder. + */ + +#include <linux/dma-mapping.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> +#include <media/v4l2-ctrls.h> +#include <media/v4l2-event.h> +#include <media/v4l2-ioctl.h> +#include <media/v4l2-mem2mem.h> +#include <media/videobuf2-dma-contig.h> + +#include "rkvenc.h" + +static inline struct rkvenc_ctx *fh_to_rkvenc_ctx(void *priv) +{ + return container_of(priv, struct rkvenc_ctx, fh); +} + +struct rkvenc_fmt { + u32 fourcc; + u8 num_planes; +}; + +static const struct rkvenc_fmt rkvenc_output_fmts[] = { + { V4L2_PIX_FMT_NV12, 1 }, + { V4L2_PIX_FMT_NV12M, 2 }, +}; + +static const struct rkvenc_fmt rkvenc_capture_fmts[] = { + { V4L2_PIX_FMT_H264, 1 }, + { V4L2_PIX_FMT_HEVC, 1 }, +}; + +static const struct rkvenc_fmt * +rkvenc_find_fmt(const struct rkvenc_fmt *fmts, unsigned int n, u32 fourcc) +{ + unsigned int i; + + for (i = 0; i < n; i++) + if (fmts[i].fourcc == fourcc) + return &fmts[i]; + return NULL; +} + +static bool rkvenc_is_h264(u32 fourcc) +{ + return fourcc == V4L2_PIX_FMT_H264; +} + +static u32 rkvenc_ctrl_val(struct rkvenc_ctx *ctx, u32 id) +{ + struct v4l2_ctrl *ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, id); + + if (!ctrl) + return 0; + return ctrl->cur.val; +} + +/* Runs for every frame, so that control changes apply to the next one */ +static void rkvenc_apply_ctrls(struct rkvenc_ctx *ctx) +{ + struct rkvenc_ctrls *c = &ctx->c; + bool h264 = ctx->codec == RKVENC_CODEC_H264; + + c->bitrate_mode = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_BITRATE_MODE); + c->bitrate = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_BITRATE); + c->vbv_size = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_VBV_SIZE); + c->frame_rc_enable = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE); + c->gop_size = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_GOP_SIZE); + c->i_period = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_PERIOD); + c->header_mode = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEADER_MODE); + c->prepend_spspps = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR); + + if (h264) { + c->profile = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_PROFILE); + c->level = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_LEVEL); + c->cabac = rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE) == + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC; + c->transform_8x8 = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM); + c->lf_mode = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE); + c->lf_alpha = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA); + c->lf_beta = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA); + c->chroma_qp_offset = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET); + c->min_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_MIN_QP); + c->max_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_MAX_QP); + c->i_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP); + c->p_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP); + if (rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_PERIOD) > + c->gop_size) + c->gop_size = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_H264_I_PERIOD); + } else { + c->level = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_LEVEL); + c->tier = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_TIER); + c->lf_mode = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE); + c->hevc_beta_offset = rkvenc_ctrl_val( + ctx, V4L2_CID_MPEG_VIDEO_HEVC_LF_BETA_OFFSET_DIV2); + c->hevc_tc_offset = rkvenc_ctrl_val( + ctx, V4L2_CID_MPEG_VIDEO_HEVC_LF_TC_OFFSET_DIV2); + c->min_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP); + c->max_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP); + c->i_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP); + c->p_frame_qp = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP); + c->hevc_max_merge = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_MAX_NUM_MERGE_MV_MINUS1); + /* There is no SAO control, SAO follows the loop filter mode */ + c->hevc_sao = + c->lf_mode != V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_DISABLED; + c->hevc_strong_intra = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_STRONG_SMOOTHING); + c->hevc_tmv = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_TMV_PREDICTION); + c->hevc_cabac_init = 0; + if (rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD) > + c->gop_size) + c->gop_size = + rkvenc_ctrl_val(ctx, + V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD); + } + + c->framerate = ctx->fps ?: 30; + ctx->rc_active = c->frame_rc_enable; +} + +static void rkvenc_refresh_header(struct rkvenc_ctx *ctx) +{ + rkvenc_apply_ctrls(ctx); + ctx->hdr_len = ctx->codec == RKVENC_CODEC_H264 ? + rkvenc_h264_setup(ctx) : rkvenc_hevc_setup(ctx); +} + +static void rkvenc_ensure_header(struct rkvenc_ctx *ctx) +{ + if (ctx->hdr_len == 0) + rkvenc_refresh_header(ctx); +} + +static int rkvenc_copy_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size) +{ + return ctx->codec == RKVENC_CODEC_H264 ? + rkvenc_h264_write_headers(ctx, buf, size) : + rkvenc_hevc_write_headers(ctx, buf, size); +} + +static void rkvenc_update_blocks(struct rkvenc_ctx *ctx) +{ + if (ctx->codec == RKVENC_CODEC_H264) { + ctx->blocks = (ALIGN(ctx->width, 16) / 16) * + (ALIGN(ctx->height, 16) / 16); + ctx->qp_blocks = ctx->blocks; + } else { + ctx->blocks = (ALIGN(ctx->width, 64) / 64) * + (ALIGN(ctx->height, 64) / 64); + ctx->qp_blocks = (ALIGN(ctx->width, 8) / 8) * + (ALIGN(ctx->height, 8) / 8); + } +} + +static u32 rkvenc_output_plane_size(struct rkvenc_ctx *ctx, unsigned int plane) +{ + u32 bpl = ctx->bytesperline; + u32 h = ctx->height; + + if (ctx->src_num_planes == 1) + return bpl * h * 3 / 2; + return plane == 0 ? bpl * h : bpl * h / 2; +} + +static u32 rkvenc_capture_size(struct rkvenc_ctx *ctx) +{ + if (ctx->codec == RKVENC_CODEC_H264) { + u32 mbw = ALIGN(ctx->width, 16) / 16; + u32 mbh = ALIGN(ctx->height, 16) / 16; + + return min(mbw * mbh * 400, ctx->width * ctx->height / 2) + + RKVENC_HDR_MAX; + } + return ctx->width * ctx->height / 2 + RKVENC_HDR_MAX; +} + +static int rkvenc_queue_setup(struct vb2_queue *vq, unsigned int *num_buffers, + unsigned int *num_planes, unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq); + const struct rkvenc_fmt *fmt; + unsigned int i; + u32 size; + + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { + fmt = rkvenc_find_fmt(rkvenc_output_fmts, + ARRAY_SIZE(rkvenc_output_fmts), + ctx->src_fourcc); + if (!fmt) + return -EINVAL; + *num_planes = fmt->num_planes; + for (i = 0; i < *num_planes; i++) + sizes[i] = rkvenc_output_plane_size(ctx, i); + return 0; + } + + fmt = rkvenc_find_fmt(rkvenc_capture_fmts, + ARRAY_SIZE(rkvenc_capture_fmts), + ctx->coded_fourcc); + if (!fmt) + return -EINVAL; + size = rkvenc_capture_size(ctx); + *num_planes = 1; + if (*num_buffers) + sizes[0] = max(sizes[0], size); + else + sizes[0] = size; + return 0; +} + +static int rkvenc_buf_prepare(struct vb2_buffer *vb) +{ + struct vb2_queue *vq = vb->vb2_queue; + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq); + + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { + if (vb2_plane_size(vb, 0) < rkvenc_output_plane_size(ctx, 0)) + return -EINVAL; + } else { + if (vb2_plane_size(vb, 0) < rkvenc_capture_size(ctx)) + return -EINVAL; + } + return 0; +} + +static void rkvenc_buf_queue(struct vb2_buffer *vb) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + + if (V4L2_TYPE_IS_CAPTURE(vb->vb2_queue->type)) { + if (vb2_is_streaming(vb->vb2_queue) && + v4l2_m2m_dst_buf_is_last(ctx->m2m_ctx)) { + vb2_set_plane_payload(vb, 0, 0); + vbuf->field = V4L2_FIELD_NONE; + v4l2_m2m_last_buffer_done(ctx->m2m_ctx, vbuf); + return; + } + + /* The parameter sets take a CAPTURE buffer of their own */ + if (ctx->header_pending && + ctx->c.header_mode == + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) { + u8 *vaddr = vb2_plane_vaddr(vb, 0); + + if (vaddr) { + rkvenc_ensure_header(ctx); + if (rkvenc_copy_headers(ctx, vaddr, + vb2_plane_size(vb, 0)) < 0) + vb2_set_plane_payload(vb, 0, 0); + else + vb2_set_plane_payload(vb, 0, + ctx->hdr_len); + } else { + vb2_set_plane_payload(vb, 0, 0); + } + vbuf->field = V4L2_FIELD_NONE; + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); + ctx->header_pending = false; + return; + } + } + + v4l2_m2m_buf_queue(ctx->m2m_ctx, vbuf); +} + +static int rkvenc_buf_out_validate(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + + vbuf->field = V4L2_FIELD_NONE; + return 0; +} + +static void rkvenc_return_dst_bufs(struct rkvenc_ctx *ctx) +{ + struct vb2_v4l2_buffer *vbuf; + + while ((vbuf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_QUEUED); +} + +static int rkvenc_start_streaming(struct vb2_queue *q, unsigned int count) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(q); + int ret; + + if (V4L2_TYPE_IS_OUTPUT(q->type)) + return 0; + + if (!ctx->coded_fourcc || !ctx->src_fourcc) { + rkvenc_return_dst_bufs(ctx); + return -EINVAL; + } + + /* Hardware bug: H.264 at most 2 MBs wide and over 1 MB high hangs */ + if (ctx->codec == RKVENC_CODEC_H264 && ALIGN(ctx->width, 16) <= 32 && + ALIGN(ctx->height, 16) > 16) { + rkvenc_return_dst_bufs(ctx); + return -EINVAL; + } + + rkvenc_update_blocks(ctx); + rkvenc_refresh_header(ctx); + + ret = rkvenc_ref_alloc(ctx); + if (ret) { + rkvenc_return_dst_bufs(ctx); + return ret; + } + + rkvenc_rc_init(ctx); + rkvenc_gop_init(ctx); + ctx->joined_header_done = false; + return 0; +} + +static void rkvenc_stop_streaming(struct vb2_queue *q) +{ + struct rkvenc_ctx *ctx = vb2_get_drv_priv(q); + struct vb2_v4l2_buffer *vbuf; + + v4l2_m2m_update_stop_streaming_state(ctx->m2m_ctx, q); + + if (V4L2_TYPE_IS_OUTPUT(q->type)) { + while ((vbuf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx))) + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + } else { + while ((vbuf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + + rkvenc_ref_free(ctx); + } +} + +static const struct vb2_ops rkvenc_queue_ops = { + .queue_setup = rkvenc_queue_setup, + .buf_prepare = rkvenc_buf_prepare, + .buf_queue = rkvenc_buf_queue, + .buf_out_validate = rkvenc_buf_out_validate, + .start_streaming = rkvenc_start_streaming, + .stop_streaming = rkvenc_stop_streaming, +}; + +static void rkvenc_device_run(void *priv) +{ + struct rkvenc_ctx *ctx = priv; + struct rkvenc_dev *dev = ctx->dev; + struct vb2_v4l2_buffer *src, *dst; + struct rkvenc_frame f = {}; + u8 qp; + bool idr; + u32 hdr_len = 0; + u8 *vaddr; + int ret; + + dev->job_ctx = ctx; + dev->job_running = true; + + src = v4l2_m2m_next_src_buf(ctx->m2m_ctx); + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + if (!src || !dst) + goto finish_error; + + idr = rkvenc_gop_is_idr(ctx); + + if (idr) + ctx->gop.frames_since_idr = 0; + + rkvenc_apply_ctrls(ctx); + rkvenc_rc_pick(ctx, &qp, idr ? 0 : 1); + ctx->cur_qp = qp; + ctx->cur_intra = idr; + + if (idr) + rkvenc_refresh_header(ctx); + + if (ctx->c.header_mode == + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME && + idr && (!ctx->joined_header_done || ctx->c.prepend_spspps)) { + vaddr = vb2_plane_vaddr(&dst->vb2_buf, 0); + if (vaddr) { + if (rkvenc_copy_headers(ctx, vaddr, + vb2_plane_size(&dst->vb2_buf, 0)) >= 0) { + hdr_len = ctx->hdr_len; + ctx->joined_header_done = true; + } + } else { + dev_warn_once(dev->dev, + "JOINED header needs an MMAP capture buffer\n"); + } + } + ctx->cur_prepend = hdr_len; + + f.codec = ctx->codec; + f.intra = idr; + f.referenced = true; + f.src.width = ctx->width; + f.src.height = ctx->height; + f.src.stride[0] = ctx->bytesperline; + f.src.stride[1] = ctx->bytesperline; + f.src.plane[0] = lower_32_bits( + vb2_dma_contig_plane_dma_addr(&src->vb2_buf, 0)); + if (ctx->src_num_planes == 1) + f.src.plane[1] = f.src.plane[0] + ctx->bytesperline * ctx->height; + else + f.src.plane[1] = lower_32_bits( + vb2_dma_contig_plane_dma_addr(&src->vb2_buf, 1) + + src->vb2_buf.planes[1].data_offset); + + f.bs.base = lower_32_bits( + vb2_dma_contig_plane_dma_addr(&dst->vb2_buf, 0)); + f.bs.size = vb2_plane_size(&dst->vb2_buf, 0); + f.bs.offset = hdr_len; + + f.recon = ctx->slot[ctx->cur]; + f.ref = idr ? ctx->slot[ctx->cur] : ctx->slot[ctx->last]; + + f.qp = qp; + f.qp_min = ctx->c.min_qp; + f.qp_max = ctx->c.max_qp; + f.rc = NULL; + f.wdg_vs_load = 0; + + if (f.codec == RKVENC_CODEC_H264) + rkvenc_h264_build_pic(ctx, &f.h264, idr); + else + rkvenc_hevc_build_pic(ctx, &f.hevc, idr); + + ret = pm_runtime_resume_and_get(dev->dev); + if (ret) + goto finish_error; + + ret = rkvenc_hw_run(dev, &f); + if (ret) { + pm_runtime_put(dev->dev); + goto finish_error; + } + return; + +finish_error: + dev->job_running = false; + dev->job_ctx = NULL; + if (src && dst) + v4l2_m2m_buf_done_and_job_finish(dev->m2m_dev, ctx->m2m_ctx, + VB2_BUF_STATE_ERROR); + else + v4l2_m2m_job_finish(dev->m2m_dev, ctx->m2m_ctx); +} + +const struct v4l2_m2m_ops rkvenc_m2m_ops = { + .device_run = rkvenc_device_run, +}; + +static int rkvenc_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct rkvenc_ctx *ctx = + container_of(ctrl->handler, struct rkvenc_ctx, ctrl_hdl); + + switch (ctrl->id) { + case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: + rkvenc_gop_force_keyframe(ctx); + break; + case V4L2_CID_MPEG_VIDEO_HEADER_MODE: + if (ctrl->val == V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) + ctx->header_pending = true; + break; + default: + break; + } + return 0; +} + +static const struct v4l2_ctrl_ops rkvenc_ctrl_ops = { + .s_ctrl = rkvenc_s_ctrl, +}; + +static int rkvenc_init_ctrls(struct rkvenc_ctx *ctx) +{ + struct v4l2_ctrl_handler *hdl = &ctx->ctrl_hdl; + + v4l2_ctrl_handler_init(hdl, 48); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_BITRATE_MODE, + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, + ~(1 << V4L2_MPEG_VIDEO_BITRATE_MODE_CBR), + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_BITRATE, + 25600, 40 * 1000 * 1000, 1, 4 * 1000 * 1000); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE, 0, 1, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_VBV_SIZE, + 0, 1 << 24, 1, 0); + + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_GOP_SIZE, + 0, 300, 1, 30); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_I_PERIOD, 0, 300, 1, 30); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD, 0, 300, 1, 30); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_TYPE, + V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR, + ~(1 << V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR), + V4L2_MPEG_VIDEO_HEVC_REFRESH_IDR); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME, 0, 0, 0, 0); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEADER_MODE, + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME, + 0, + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR, + 0, 1, 1, 0); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_PROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + ~((1 << V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + (1 << V4L2_MPEG_VIDEO_H264_PROFILE_HIGH)), + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LEVEL, + V4L2_MPEG_VIDEO_H264_LEVEL_5_2, + 0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC, 0, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM, + 0, 1, 1, 1); + v4l2_ctrl_new_std_menu( + hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE, + V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED_AT_SLICE_BOUNDARY, + 0, V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_ENABLED); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_CHROMA_QP_INDEX_OFFSET, + -12, 12, 1, 0); + + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_PROFILE, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, 0, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, + V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2, + 0, V4L2_MPEG_VIDEO_HEVC_LEVEL_4); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_TIER, + V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, + ~(1 << V4L2_MPEG_VIDEO_HEVC_TIER_MAIN), + V4L2_MPEG_VIDEO_HEVC_TIER_MAIN); + v4l2_ctrl_new_std_menu(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE, + V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_ENABLED, + 0, + V4L2_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE_ENABLED); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LF_BETA_OFFSET_DIV2, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LF_TC_OFFSET_DIV2, + -6, 6, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_STRONG_SMOOTHING, + 0, 1, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_TMV_PREDICTION, + 0, 1, 1, 0); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_MAX_NUM_MERGE_MV_MINUS1, + 0, 4, 1, 2); + + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_MIN_QP, + 1, 51, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_H264_MAX_QP, + 1, 51, 1, 51); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, 1, 51, 1, 26); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, 1, 51, 1, 28); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP, + 1, 51, 1, 1); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP, + 1, 51, 1, 51); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP, 1, 51, 1, 26); + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP, 1, 51, 1, 28); + + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_AVERAGE_QP, 0, 51, 1, 0); + + if (hdl->error) + return hdl->error; + + v4l2_ctrl_handler_setup(hdl); + + ctx->header_pending = + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEADER_MODE) == + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE; + + ctx->fh.ctrl_handler = hdl; + return 0; +} + +static int rkvenc_querycap(struct file *file, void *priv, + struct v4l2_capability *cap) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct rkvenc_dev *dev = ctx->dev; + + strscpy(cap->driver, "rkvenc", sizeof(cap->driver)); + strscpy(cap->card, "rockchip-rkvenc", sizeof(cap->card)); + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s", + dev_name(dev->dev)); + + return 0; +} + +static int rkvenc_enum_fmt(struct file *file, void *priv, + struct v4l2_fmtdesc *f, bool capture) +{ + const struct rkvenc_fmt *fmts = capture ? rkvenc_capture_fmts + : rkvenc_output_fmts; + unsigned int n = capture ? ARRAY_SIZE(rkvenc_capture_fmts) + : ARRAY_SIZE(rkvenc_output_fmts); + + if (f->index >= n) + return -EINVAL; + f->pixelformat = fmts[f->index].fourcc; + f->flags = 0; + return 0; +} + +static int rkvenc_enum_fmt_vid_out(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + return rkvenc_enum_fmt(file, priv, f, false); +} + +static int rkvenc_enum_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_fmtdesc *f) +{ + return rkvenc_enum_fmt(file, priv, f, true); +} + +static void rkvenc_try_fmt_vid_out(struct rkvenc_ctx *ctx, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; + const struct rkvenc_fmt *fmt; + u32 bpl; + + fmt = rkvenc_find_fmt(rkvenc_output_fmts, + ARRAY_SIZE(rkvenc_output_fmts), + pix->pixelformat); + if (!fmt) { + fmt = &rkvenc_output_fmts[0]; + pix->pixelformat = fmt->fourcc; + } + + pix->field = V4L2_FIELD_NONE; + pix->width = clamp(round_up(pix->width, 2), 16, ctx->dev->max_width); + pix->height = clamp(round_up(pix->height, 2), 16, ctx->dev->max_height); + pix->num_planes = fmt->num_planes; + + bpl = ALIGN(pix->width, 16); + pix->plane_fmt[0].bytesperline = bpl; + if (fmt->num_planes == 2) { + pix->plane_fmt[0].sizeimage = bpl * pix->height; + pix->plane_fmt[1].bytesperline = bpl; + pix->plane_fmt[1].sizeimage = bpl * pix->height / 2; + } else { + pix->plane_fmt[0].sizeimage = bpl * pix->height * 3 / 2; + } +} + +static void rkvenc_try_fmt_vid_cap(struct rkvenc_ctx *ctx, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; + const struct rkvenc_fmt *fmt; + + fmt = rkvenc_find_fmt(rkvenc_capture_fmts, + ARRAY_SIZE(rkvenc_capture_fmts), + pix->pixelformat); + if (!fmt) { + fmt = &rkvenc_capture_fmts[0]; + pix->pixelformat = fmt->fourcc; + } + + pix->field = V4L2_FIELD_NONE; + pix->width = clamp(round_up(pix->width, 2), 16, ctx->dev->max_width); + pix->height = clamp(round_up(pix->height, 2), 16, ctx->dev->max_height); + pix->num_planes = 1; + pix->plane_fmt[0].bytesperline = 0; + + if (fmt->fourcc == V4L2_PIX_FMT_H264) { + u32 mbw = ALIGN(pix->width, 16) / 16; + u32 mbh = ALIGN(pix->height, 16) / 16; + + pix->plane_fmt[0].sizeimage = + min(mbw * mbh * 400, pix->width * pix->height / 2) + + RKVENC_HDR_MAX; + } else { + pix->plane_fmt[0].sizeimage = + pix->width * pix->height / 2 + RKVENC_HDR_MAX; + } +} + +static int rkvenc_try_fmt_vid_out_ioctl(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + + rkvenc_try_fmt_vid_out(ctx, f); + return 0; +} + +static int rkvenc_try_fmt_vid_cap_ioctl(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + + rkvenc_try_fmt_vid_cap(ctx, f); + return 0; +} + +static int rkvenc_s_fmt_vid_out(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct vb2_queue *vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type); + + if (vb2_is_busy(vq)) + return -EBUSY; + + rkvenc_try_fmt_vid_out(ctx, f); + ctx->src_fourcc = f->fmt.pix_mp.pixelformat; + ctx->src_num_planes = f->fmt.pix_mp.num_planes; + ctx->width = f->fmt.pix_mp.width; + ctx->height = f->fmt.pix_mp.height; + ctx->bytesperline = f->fmt.pix_mp.plane_fmt[0].bytesperline; + return 0; +} + +static int rkvenc_s_fmt_vid_cap(struct file *file, void *priv, + struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct vb2_queue *vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type); + + if (vb2_is_busy(vq)) + return -EBUSY; + + rkvenc_try_fmt_vid_cap(ctx, f); + ctx->coded_fourcc = f->fmt.pix_mp.pixelformat; + ctx->codec = rkvenc_is_h264(f->fmt.pix_mp.pixelformat) ? + RKVENC_CODEC_H264 : RKVENC_CODEC_HEVC; + return 0; +} + +static int rkvenc_g_fmt(struct file *file, void *priv, struct v4l2_format *f) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + pix->pixelformat = ctx->src_fourcc; + pix->num_planes = ctx->src_num_planes; + } else { + pix->pixelformat = ctx->coded_fourcc; + pix->num_planes = 1; + } + pix->width = ctx->width; + pix->height = ctx->height; + pix->field = V4L2_FIELD_NONE; + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + pix->plane_fmt[0].bytesperline = ctx->bytesperline; + pix->plane_fmt[0].sizeimage = + rkvenc_output_plane_size(ctx, 0); + if (ctx->src_num_planes == 2) { + pix->plane_fmt[1].bytesperline = ctx->bytesperline; + pix->plane_fmt[1].sizeimage = + rkvenc_output_plane_size(ctx, 1); + } + } else { + pix->plane_fmt[0].sizeimage = rkvenc_capture_size(ctx); + } + return 0; +} + +static int rkvenc_s_parm(struct file *file, void *priv, + struct v4l2_streamparm *a) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + struct v4l2_fract *tpf = &a->parm.output.timeperframe; + unsigned int fps; + + if (!V4L2_TYPE_IS_MULTIPLANAR(a->type)) + return -EINVAL; + + if (tpf->numerator == 0 || tpf->denominator == 0) { + tpf->numerator = 1; + tpf->denominator = 30; + } + fps = tpf->denominator / tpf->numerator; + if (fps < 1) + fps = 1; + ctx->fps = fps; + a->parm.output.capability = V4L2_CAP_TIMEPERFRAME; + return 0; +} + +static int rkvenc_g_parm(struct file *file, void *priv, + struct v4l2_streamparm *a) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); + + if (!V4L2_TYPE_IS_MULTIPLANAR(a->type)) + return -EINVAL; + + a->parm.output.capability = V4L2_CAP_TIMEPERFRAME; + a->parm.output.timeperframe.numerator = 1; + a->parm.output.timeperframe.denominator = ctx->fps ?: 30; + return 0; +} + +static int rkvenc_vidioc_encoder_cmd(struct file *file, void *priv, + struct v4l2_encoder_cmd *cmd) +{ + if (cmd->cmd != V4L2_ENC_CMD_STOP && cmd->cmd != V4L2_ENC_CMD_START) + return -EINVAL; + return v4l2_m2m_ioctl_encoder_cmd(file, priv, cmd); +} + +static int rkvenc_queue_init(void *priv, struct vb2_queue *src_vq, + struct vb2_queue *dst_vq) +{ + struct rkvenc_ctx *ctx = priv; + struct rkvenc_dev *dev = ctx->dev; + int ret; + + src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + src_vq->io_modes = VB2_MMAP | VB2_DMABUF; + src_vq->drv_priv = ctx; + src_vq->ops = &rkvenc_queue_ops; + src_vq->mem_ops = &vb2_dma_contig_memops; + src_vq->dma_attrs = DMA_ATTR_NO_KERNEL_MAPPING; + src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + src_vq->lock = &dev->vdev_lock; + src_vq->dev = dev->v4l2_dev.dev; + ret = vb2_queue_init(src_vq); + if (ret) + return ret; + + dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; + dst_vq->drv_priv = ctx; + dst_vq->ops = &rkvenc_queue_ops; + dst_vq->mem_ops = &vb2_dma_contig_memops; + dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer); + dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + dst_vq->lock = &dev->vdev_lock; + dst_vq->dev = dev->v4l2_dev.dev; + return vb2_queue_init(dst_vq); +} + +static int rkvenc_open(struct file *filp) +{ + struct rkvenc_dev *dev = video_drvdata(filp); + struct rkvenc_ctx *ctx; + int ret; + + ctx = kzalloc_obj(*ctx); + if (!ctx) + return -ENOMEM; + + ctx->dev = dev; + ctx->codec = RKVENC_CODEC_H264; + ctx->coded_fourcc = V4L2_PIX_FMT_H264; + ctx->src_fourcc = V4L2_PIX_FMT_NV12; + ctx->src_num_planes = 1; + ctx->width = 1920; + ctx->height = 1080; + ctx->bytesperline = ALIGN(1920, 16); + ctx->fps = 30; + mutex_init(&ctx->vlock); + + v4l2_fh_init(&ctx->fh, video_devdata(filp)); + filp->private_data = &ctx->fh; + + ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx, rkvenc_queue_init); + if (IS_ERR(ctx->m2m_ctx)) { + ret = PTR_ERR(ctx->m2m_ctx); + goto err_fh; + } + ctx->fh.m2m_ctx = ctx->m2m_ctx; + + ret = rkvenc_init_ctrls(ctx); + if (ret) + goto err_m2m; + + v4l2_fh_add(&ctx->fh, filp); + return 0; + +err_m2m: + v4l2_m2m_ctx_release(ctx->m2m_ctx); +err_fh: + v4l2_fh_exit(&ctx->fh); + kfree(ctx); + return ret; +} + +static int rkvenc_release(struct file *filp) +{ + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(filp->private_data); + + v4l2_m2m_ctx_release(ctx->m2m_ctx); + v4l2_ctrl_handler_free(&ctx->ctrl_hdl); + v4l2_fh_del(&ctx->fh, filp); + v4l2_fh_exit(&ctx->fh); + kfree(ctx); + return 0; +} + +static const struct v4l2_ioctl_ops rkvenc_v4l2_ioctl_ops = { + .vidioc_querycap = rkvenc_querycap, + + .vidioc_enum_fmt_vid_cap = rkvenc_enum_fmt_vid_cap, + .vidioc_enum_fmt_vid_out = rkvenc_enum_fmt_vid_out, + .vidioc_try_fmt_vid_cap_mplane = rkvenc_try_fmt_vid_cap_ioctl, + .vidioc_try_fmt_vid_out_mplane = rkvenc_try_fmt_vid_out_ioctl, + .vidioc_s_fmt_vid_cap_mplane = rkvenc_s_fmt_vid_cap, + .vidioc_s_fmt_vid_out_mplane = rkvenc_s_fmt_vid_out, + .vidioc_g_fmt_vid_cap_mplane = rkvenc_g_fmt, + .vidioc_g_fmt_vid_out_mplane = rkvenc_g_fmt, + + .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querybuf = v4l2_m2m_ioctl_querybuf, + .vidioc_qbuf = v4l2_m2m_ioctl_qbuf, + .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, + .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, + .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, + .vidioc_streamon = v4l2_m2m_ioctl_streamon, + .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, + + .vidioc_s_parm = rkvenc_s_parm, + .vidioc_g_parm = rkvenc_g_parm, + + .vidioc_encoder_cmd = rkvenc_vidioc_encoder_cmd, + .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd, + + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, +}; + +static const struct v4l2_file_operations rkvenc_v4l2_fops = { + .owner = THIS_MODULE, + .open = rkvenc_open, + .release = rkvenc_release, + .poll = v4l2_m2m_fop_poll, + .unlocked_ioctl = video_ioctl2, + .mmap = v4l2_m2m_fop_mmap, +}; + +int rkvenc_v4l2_init(struct rkvenc_dev *dev) +{ + struct video_device *vdev = &dev->vdev; + int ret; + + dev->mdev.dev = dev->dev; + strscpy(dev->mdev.model, "rkvenc", sizeof(dev->mdev.model)); + strscpy(dev->mdev.bus_info, "platform:rkvenc", + sizeof(dev->mdev.bus_info)); + media_device_init(&dev->mdev); + dev->v4l2_dev.mdev = &dev->mdev; + + vdev->lock = &dev->vdev_lock; + vdev->v4l2_dev = &dev->v4l2_dev; + vdev->fops = &rkvenc_v4l2_fops; + vdev->release = video_device_release_empty; + vdev->vfl_dir = VFL_DIR_M2M; + vdev->device_caps = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_M2M_MPLANE; + vdev->ioctl_ops = &rkvenc_v4l2_ioctl_ops; + video_set_drvdata(vdev, dev); + strscpy(vdev->name, "rkvenc", sizeof(vdev->name)); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register video device\n"); + goto err_cleanup_mc; + } + + ret = v4l2_m2m_register_media_controller(dev->m2m_dev, vdev, + MEDIA_ENT_F_PROC_VIDEO_ENCODER); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to init M2M media controller\n"); + goto err_unregister_vdev; + } + + ret = media_device_register(&dev->mdev); + if (ret) { + v4l2_err(&dev->v4l2_dev, "Failed to register media device\n"); + goto err_unregister_mc; + } + + return 0; + +err_unregister_mc: + v4l2_m2m_unregister_media_controller(dev->m2m_dev); +err_unregister_vdev: + video_unregister_device(vdev); +err_cleanup_mc: + media_device_cleanup(&dev->mdev); + return ret; +} + +void rkvenc_v4l2_cleanup(struct rkvenc_dev *dev) +{ + media_device_unregister(&dev->mdev); + v4l2_m2m_unregister_media_controller(dev->m2m_dev); + video_unregister_device(&dev->vdev); + media_device_cleanup(&dev->mdev); +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c new file mode 100644 index 0000000000000..202391672f2c7 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-h264.c @@ -0,0 +1,206 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RKVENC v1 register image builder, H.264 part. + */ + +#include "rkvenc-vepu541.h" + +#define H264_PROFILE_MAIN 77 +#define H264_LEVEL_1 10 +#define H264_LEVEL_3 30 +#define H264_NAL_SLICE 1 +#define H264_NAL_IDR 5 +#define H264_SLICE_P 0 +#define H264_SLICE_I 2 + +/* Coarse-ME RAM height by MB-aligned width */ +static const u16 h264_rama_h_width[] = { + 3584, 3136, 2816, 2560, 2368, 2176, 2048, 1856, 1792, +}; + +static void h264_me_build(const struct rkvenc_frame *f, struct rkvenc_img *img) +{ + const struct rkvenc_h264_seq *seq = f->h264.seq; + u32 w16 = ALIGN(f->src.width, 16), h16 = ALIGN(f->src.height, 16); + u32 mbw = w16 / 16, w64 = DIV_ROUND_UP(w16, 64); + u32 srch_h, srch_v, rama_h, v, hc, i; + u32 ah64 = ALIGN(h16, 64); + + srch_h = mbw < 22 ? mbw / 2 : 11; + srch_v = seq->level_idc <= H264_LEVEL_1 ? 3 : 7; + if (ah64 <= 64) + srch_v = 0; + else if ((ah64 == 128 || ah64 == 192) && seq->level_idc > H264_LEVEL_1) + srch_v = 4; + + rkvenc_img_l1(img, RKVENC_ME_RNGE, + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_H, srch_h) | + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_V, srch_v) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_H, 7) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_V, 5)); + + rkvenc_img_l1(img, RKVENC_ME_CNST, + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_H, f->intra ? 0 : 5) | + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_V, f->intra ? 0 : 5) | + FIELD_PREP(RKVENC_ME_CNST_MV_LIMIT, 2) | + FIELD_PREP(RKVENC_ME_CNST_PMV_NUM, 2)); + + rama_h = 17; + for (i = 0; i < ARRAY_SIZE(h264_rama_h_width); i++) { + if (w16 > h264_rama_h_width[i]) { + rama_h = 8 + i; + break; + } + } + v = 2 * srch_v + 1; + hc = (srch_h * 4 + 15) / 16 * 2 + 1; + rkvenc_img_l1(img, RKVENC_ME_RAM, + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_MAX, + (min(v, rama_h) - 1) * w64 + hc) | + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_H, rama_h)); +} + +static void h264_syntax_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + const struct rkvenc_h264_seq *seq = f->h264.seq; + const struct rkvenc_h264_pic *pic = &f->h264; + u32 rdo, sli2; + + rdo = RKVENC_H264_RDO_INTER_4X4 | RKVENC_H264_RDO_CHRM_SPCL | + RKVENC_H264_RDO_CCWA_E | RKVENC_H264_RDO_ATR_E; + if (seq->profile_idc < H264_PROFILE_MAIN && seq->level_idc <= H264_LEVEL_3) + rdo |= RKVENC_H264_RDO_RECT_SIZE; + if (seq->profile_idc < H264_PROFILE_MAIN && !seq->cabac) + rdo |= RKVENC_H264_RDO_VLC_LMT; + rkvenc_img_l1(img, RKVENC_RDO_CFG, rdo); + rkvenc_img_l1(img, RKVENC_IPRD_CSTS, + FIELD_PREP(RKVENC_IPRD_CSTS_VTHD_Y, 2501) | + FIELD_PREP(RKVENC_IPRD_CSTS_VTHD_C, 2501)); + + rkvenc_img_l1(img, RKVENC_SYNT_NAL, + FIELD_PREP(RKVENC_H264_NAL_REF_IDC, pic->nal_ref_idc) | + FIELD_PREP(RKVENC_H264_NAL_UNIT_TYPE, + f->intra ? H264_NAL_IDR : H264_NAL_SLICE)); + rkvenc_img_l1(img, RKVENC_SYNT_SPS, + FIELD_PREP(RKVENC_H264_SPS_MAX_FNUM, + seq->log2_max_frame_num_m4) | + RKVENC_H264_SPS_DRCT_8X8 | + FIELD_PREP(RKVENC_H264_SPS_MPOC_LM4, + seq->log2_max_poc_lsb_m4)); + rkvenc_img_l1( + img, RKVENC_SYNT_PPS, + (seq->cabac ? RKVENC_H264_PPS_ETPY_MODE : 0) | + (seq->transform_8x8 ? RKVENC_H264_PPS_TRNS_8X8 : 0) | + FIELD_PREP(RKVENC_PPS_PIC_INIT_QP, seq->pic_init_qp) | + FIELD_PREP(RKVENC_H264_PPS_CB_OFST, + rkvenc_s2u(seq->chroma_qp_offset, 5)) | + FIELD_PREP(RKVENC_H264_PPS_CR_OFST, + rkvenc_s2u(seq->chroma_qp_offset, 5)) | + RKVENC_H264_PPS_DBF_CP_FLG); + rkvenc_img_l1(img, RKVENC_SYNT_SLI0, + FIELD_PREP(RKVENC_H264_SLI0_SLI_TYPE, + f->intra ? H264_SLICE_I : H264_SLICE_P) | + FIELD_PREP(RKVENC_H264_SLI0_FRM_NUM, pic->frame_num)); + rkvenc_img_l1(img, RKVENC_SYNT_SLI1, + FIELD_PREP(RKVENC_H264_SLI1_IDR_PIC_ID, + f->intra ? pic->idr_pic_id : 0) | + FIELD_PREP(RKVENC_H264_SLI1_POC_LSB, pic->poc_lsb)); + + sli2 = FIELD_PREP(RKVENC_H264_SLI2_DIS_DBLK_IDC, seq->dblk_idc); + sli2 |= FIELD_PREP(RKVENC_H264_SLI2_ALPH_OFST, + rkvenc_s2u(seq->alpha_offset, 4)); + sli2 |= FIELD_PREP(RKVENC_H264_SLI2_BETA_OFST, + rkvenc_s2u(seq->beta_offset, 4)); + rkvenc_img_l1(img, RKVENC_SYNT_SLI2, sli2); +} + +static void h264_l2_build(const struct rkvenc_frame *f, struct rkvenc_img *img) +{ + u32 i; + + for (i = 0; i < 2; i++) { + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHDY4(i), + i ? rkvenc_pair(9, 36) : rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHDC8(i), + i ? rkvenc_pair(9, 36) : rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHDY8(i), + i ? rkvenc_pair(9, 36) : rkvenc_pair(1, 4)); + } + rkvenc_img_l2(img, RKVENC_L2_IPRD_TTHD_UL, 2501); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTY8, rkvenc_bytes(48, 60, 40, 48)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTY4, rkvenc_bytes(48, 60, 40, 48)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTY16, rkvenc_bytes(48, 60, 40, 48)); + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGTC8, rkvenc_bytes(36, 42, 28, 32)); + + /* Rounding offsets in 1/2048 units */ + rkvenc_img_l2(img, RKVENC_L2_QNT_BIAS, + FIELD_PREP(RKVENC_L2_QNT_BIAS_I, f->intra ? 683 : 583) | + FIELD_PREP(RKVENC_L2_QNT_BIAS_P, 171)); + + rkvenc_img_l2(img, RKVENC_L2_ATR_THD0, rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_ATR_THD1, + rkvenc_pair(f->intra ? 36 : 81, 45)); + rkvenc_img_l2(img, RKVENC_L2_ATR_WGT16, f->intra ? + rkvenc_bytes(16, 16, 16, 0) : rkvenc_bytes(18, 17, 16, 0)); + rkvenc_img_l2(img, RKVENC_L2_ATR_WGT8, rkvenc_bytes(20, 20, 20, 0)); + rkvenc_img_l2(img, RKVENC_L2_ATR_WGT4, f->intra ? + rkvenc_bytes(20, 18, 16, 0) : rkvenc_bytes(18, 17, 16, 0)); + + rkvenc_img_l2(img, RKVENC_L2_ATF_TTHD(0), rkvenc_pair(0, 64)); + rkvenc_img_l2(img, RKVENC_L2_ATF_TTHD(1), rkvenc_pair(144, 2500)); + rkvenc_img_l2(img, RKVENC_L2_ATF_STHD(0), rkvenc_pair(80, 280)); + rkvenc_img_l2(img, RKVENC_L2_ATF_STHD(1), rkvenc_pair(144, 192)); + rkvenc_img_l2(img, RKVENC_L2_ATF_WGT(0), rkvenc_pair(26, 24)); + rkvenc_img_l2(img, RKVENC_L2_ATF_WGT(1), rkvenc_pair(19, 22)); + rkvenc_img_l2(img, RKVENC_L2_ATF_WGT(2), rkvenc_pair(19, 19)); + rkvenc_img_l2(img, RKVENC_L2_ATF_OFST(0), rkvenc_pair(3500, 3500)); + rkvenc_img_l2(img, RKVENC_L2_ATF_OFST(1), rkvenc_pair(0, 3500)); + rkvenc_img_l2(img, RKVENC_L2_ATF_OFST(2), rkvenc_pair(1000, 0)); + + for (i = 0; i <= RKVENC_QP_MAX; i++) { + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTA_QP(i), + rkvenc_h264_lambda[i + 1]); + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTB_QP(i), + rkvenc_h264_lambda[i]); + } + + rkvenc_img_l2_u8(img, RKVENC_L2_AQ_TTHD(0), rkvenc_h264_aq_thd, 16); + rkvenc_img_l2_s6(img, RKVENC_L2_AQ_STP(0), f->intra ? + rkvenc_h264_aq_step_i : rkvenc_h264_aq_step_p, 16); + + rkvenc_img_l2(img, RKVENC_L2_RME_MVD_PNSH, 16000 << 6); + rkvenc_img_l2(img, RKVENC_L2_ATR1_THD0, rkvenc_pair(1, 4)); + rkvenc_img_l2(img, RKVENC_L2_ATR1_THD1, 49); + img->l2_last = RKVENC_L2_H264_LAST; +} + +void rkvenc_vepu541_h264_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + u32 w16 = ALIGN(f->src.width, 16), h16 = ALIGN(f->src.height, 16); + u32 pic, i; + + rkvenc_img_l1(img, RKVENC_ENC_RSL, + FIELD_PREP(RKVENC_ENC_RSL_WD8_M1, w16 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_WFILL, w16 - f->src.width) | + FIELD_PREP(RKVENC_ENC_RSL_HD8_M1, h16 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_HFILL, h16 - f->src.height)); + + pic = RKVENC_ENC_PIC_BS_SCP | FIELD_PREP(RKVENC_ENC_PIC_QP, f->qp); + if (f->referenced) + pic |= RKVENC_ENC_PIC_CUR_FRM_REF; + if (!f->intra) + pic |= RKVENC_ENC_PIC_LAMB_MOD_SEL; + rkvenc_img_l1(img, RKVENC_ENC_PIC, pic); + + rkvenc_img_l1(img, RKVENC_DTRNS_CFG, RKVENC_DTRNS_CFG_DSPR_OTSD); + + rkvenc_img_l1(img, RKVENC_KLUT_OFST, f->intra ? 0 : 3); + for (i = 0; i < RKVENC_KLUT_WORDS; i++) + rkvenc_img_l1(img, RKVENC_KLUT_WGT(i), rkvenc_h264_klut[i]); + + h264_me_build(f, img); + h264_syntax_build(f, img); + h264_l2_build(f, img); +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c new file mode 100644 index 0000000000000..0a6dea9062a54 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-hevc.c @@ -0,0 +1,182 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RKVENC v1 register image builder, HEVC part. + */ + +#include "rkvenc-vepu541.h" + +#define HEVC_SLICE_P 1 +#define HEVC_SLICE_I 2 + +static void hevc_me_build(const struct rkvenc_frame *f, struct rkvenc_img *img) +{ + u32 w = ALIGN(f->src.width, 8), h = ALIGN(f->src.height, 8); + u32 wc = DIV_ROUND_UP(w, 64); + u32 sx, sy, srch_h, srch_v, rama_h, s, m, l2map; + + sx = w >= 384 + 60 ? 384 : 2 * (min(384U, w) / 4); + sy = h >= 256 + 60 ? 256 : 2 * (min(256U, h) / 4); + srch_h = (sx / 2 - 1) / 16; + srch_v = (sy / 2 - 1) / 16; + + rkvenc_img_l1(img, RKVENC_ME_RNGE, + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_H, srch_h) | + FIELD_PREP(RKVENC_ME_RNGE_CME_SRCH_V, srch_v) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_H, 7) | + FIELD_PREP(RKVENC_ME_RNGE_RME_SRCH_V, 5) | + FIELD_PREP(RKVENC_ME_RNGE_DLT_FRM_NUM, 1)); + rkvenc_img_l1(img, RKVENC_ME_CNST, + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_H, 5) | + FIELD_PREP(RKVENC_ME_CNST_PMV_MDST_V, 5) | + FIELD_PREP(RKVENC_ME_CNST_PMV_NUM, 2)); + + rama_h = w > 2688 ? 12 : w > 2048 ? 16 : 20; + s = 2 * ((srch_h + 4) / 4) + 1; + m = min(2 * ((srch_v + 3) / 4) + 1, rama_h / 4); + if (wc * 64 <= 512) + l2map = 0; + else if (wc * 64 <= 1024) + l2map = 1; + else if (wc * 64 <= 2048) + l2map = 2; + else + l2map = 3; + rkvenc_img_l1(img, RKVENC_ME_RAM, + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_MAX, + wc * (m - 1) + (wc >= s ? s : 2 * wc)) | + FIELD_PREP(RKVENC_ME_RAM_CME_RAMA_H, rama_h) | + FIELD_PREP(RKVENC_ME_RAM_CACH_L2_MAP, l2map)); +} + +static void hevc_syntax_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + const struct rkvenc_hevc_seq *seq = f->hevc.seq; + const struct rkvenc_hevc_pic *pic = &f->hevc; + u32 sli0, sli1; + + rkvenc_img_l1(img, RKVENC_RDO_CFG, + RKVENC_HEVC_RDO_CHRM_SPECIAL | + FIELD_PREP(RKVENC_HEVC_RDO_CU_INTER_EN, 0xf) | + FIELD_PREP(RKVENC_HEVC_RDO_CU_INTRA_EN, 0xf) | + RKVENC_HEVC_RDO_CHRM_KLUT_EN); + + rkvenc_img_l1(img, RKVENC_SYNT_NAL, + FIELD_PREP(RKVENC_HEVC_NAL_UNIT_TYPE, pic->nal_unit_type)); + rkvenc_img_l1(img, RKVENC_SYNT_SPS, + (seq->sao ? RKVENC_HEVC_SPS_SAO_EN : 0) | + FIELD_PREP(RKVENC_HEVC_SPS_LOG2_MAX_POC_LSB, + seq->log2_max_poc_lsb_m4) | + (seq->strong_intra_smoothing ? + RKVENC_HEVC_SPS_STRG_INTRA_SMTH : 0)); + rkvenc_img_l1(img, RKVENC_SYNT_PPS, + (seq->cabac_init_present ? + RKVENC_HEVC_PPS_CBC_INIT_PRSNT : 0) | + FIELD_PREP(RKVENC_PPS_PIC_INIT_QP, seq->pic_init_qp) | + RKVENC_HEVC_PPS_CU_QP_DLT_EN); + + sli0 = RKVENC_HEVC_SLI0_MERGE_UP | RKVENC_HEVC_SLI0_MERGE_LEFT | + RKVENC_HEVC_SLI0_PIC_OUT_FLG | + FIELD_PREP(RKVENC_HEVC_SLI0_SLI_TYPE, + f->intra ? HEVC_SLICE_I : HEVC_SLICE_P); + if (seq->cabac_init_present) + sli0 |= RKVENC_HEVC_SLI0_CBC_INIT_FLG; + if (seq->sao) + sli0 |= RKVENC_HEVC_SLI0_SAO_LUMA | RKVENC_HEVC_SLI0_SAO_CHRM; + rkvenc_img_l1(img, RKVENC_SYNT_SLI0, sli0); + + /* The header QP must match the QP the picture is quantised with */ + sli1 = FIELD_PREP(RKVENC_HEVC_SLI1_SLI_QP, f->qp) | + FIELD_PREP(RKVENC_HEVC_SLI1_MAX_MRG_CND, seq->max_merge_cand) | + RKVENC_HEVC_SLI1_COL_FRM_L0; + sli1 |= FIELD_PREP(RKVENC_HEVC_SLI1_TC_OFST, + rkvenc_s2u(seq->tc_offset, 4)); + sli1 |= FIELD_PREP(RKVENC_HEVC_SLI1_BETA_OFST, + rkvenc_s2u(seq->beta_offset, 4)); + if (seq->deblock_disabled) + sli1 |= RKVENC_HEVC_SLI1_DBLK_DIS; + rkvenc_img_l1(img, RKVENC_SYNT_SLI1, sli1); + rkvenc_img_l1(img, RKVENC_SYNT_SLI2, + FIELD_PREP(RKVENC_HEVC_SLI2_POC_LSB, pic->poc_lsb)); + + if (!f->intra) + rkvenc_img_l1(img, RKVENC_SYNT_REFM1, + FIELD_PREP(RKVENC_HEVC_REFM1_USED_BY_S0, 1) | + FIELD_PREP(RKVENC_HEVC_REFM1_NUM_NEG_PIC, 1)); +} + +static void hevc_l2_build(struct rkvenc_img *img) +{ + const u16 *thd = rkvenc_hevc_intra_thd; + u32 i; + + for (i = 0; i < 2; i++) { + rkvenc_img_l2(img, RKVENC_L2_HEVC_INTRA_THD32(i), + thd[2 * i] | thd[2 * i + 1] << 16); + rkvenc_img_l2(img, RKVENC_L2_HEVC_INTRA_THD16(i), + thd[2 * i] | thd[2 * i + 1] << 16); + } + rkvenc_img_l2_u8(img, RKVENC_L2_HEVC_INTRA_WGT32(0), + rkvenc_hevc_intra_wgt32, 7); + rkvenc_img_l2_u8(img, RKVENC_L2_HEVC_INTRA_WGT16(0), + rkvenc_hevc_intra_wgt16, 7); + + /* Rounding offsets in 1/512 units */ + rkvenc_img_l2(img, RKVENC_L2_QNT_BIAS, + FIELD_PREP(RKVENC_L2_QNT_BIAS_I, 171) | + FIELD_PREP(RKVENC_L2_QNT_BIAS_P, 85)); + + for (i = 0; i < 5; i++) + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_THD(i), rkvenc_pair(0, 63)); + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_WGT(0), rkvenc_pair(16, 16)); + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_WGT(1), rkvenc_pair(16, 16)); + rkvenc_img_l2(img, RKVENC_L2_HEVC_ATF_WGT(2), 16); + + for (i = 0; i <= RKVENC_QP_MAX; i++) { + rkvenc_img_l2(img, RKVENC_L2_IPRD_WGT_QP(i), + rkvenc_hevc_lambda_satd[i]); + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTA_QP(i), + rkvenc_hevc_lambda_a[i]); + rkvenc_img_l2(img, RKVENC_L2_RDO_WGTB_QP(i), + rkvenc_hevc_lambda_b[i]); + } + + rkvenc_img_l2_u8(img, RKVENC_L2_AQ_TTHD(0), rkvenc_hevc_aq_thd, 16); + rkvenc_img_l2_s6(img, RKVENC_L2_AQ_STP(0), rkvenc_hevc_aq_step, 16); + img->l2_last = RKVENC_L2_HEVC_LAST; +} + +void rkvenc_vepu541_hevc_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + u32 w8 = ALIGN(f->src.width, 8), h8 = ALIGN(f->src.height, 8); + u32 ctus = DIV_ROUND_UP(w8, 64) * DIV_ROUND_UP(h8, 64); + u32 pic, i; + + rkvenc_img_l1(img, RKVENC_ENC_RSL, + FIELD_PREP(RKVENC_ENC_RSL_WD8_M1, w8 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_WFILL, w8 - f->src.width) | + FIELD_PREP(RKVENC_ENC_RSL_HD8_M1, h8 / 8 - 1) | + FIELD_PREP(RKVENC_ENC_RSL_HFILL, h8 - f->src.height)); + + pic = RKVENC_ENC_PIC_STND_HEVC | RKVENC_ENC_PIC_BS_SCP | + FIELD_PREP(RKVENC_ENC_PIC_QP, f->qp) | + FIELD_PREP(RKVENC_ENC_PIC_TOT_POC_NUM, f->intra ? 0 : 1) | + FIELD_PREP(RKVENC_ENC_PIC_LOG2_CTU_NUM, order_base_2(ctus)); + if (f->referenced) + pic |= RKVENC_ENC_PIC_CUR_FRM_REF; + if (!f->intra) + pic |= RKVENC_ENC_PIC_LAMB_MOD_SEL; + rkvenc_img_l1(img, RKVENC_ENC_PIC, pic); + + rkvenc_img_l1(img, RKVENC_DTRNS_CFG, + f->intra ? 0 : RKVENC_DTRNS_CFG_DSPR_OTSD); + + rkvenc_img_l1(img, RKVENC_KLUT_OFST, f->intra ? 0 : 3); + for (i = 0; i < RKVENC_KLUT_WORDS; i++) + rkvenc_img_l1(img, RKVENC_KLUT_WGT(i), rkvenc_hevc_klut[i]); + + hevc_me_build(f, img); + hevc_syntax_build(f, img); + hevc_l2_build(img); +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h new file mode 100644 index 0000000000000..9abe6628a5831 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-regs.h @@ -0,0 +1,283 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef RKVENC_VEPU541_REGS_H +#define RKVENC_VEPU541_REGS_H + +#include <linux/bits.h> +#include <linux/bitfield.h> + +#define RKVENC_VERSION 0x000 + +#define RKVENC_ENC_STRT 0x004 +#define RKVENC_ENC_STRT_CMD GENMASK(9, 8) +#define RKVENC_ENC_STRT_CMD_FRAME 1 +#define RKVENC_ENC_STRT_CLK_GATE_EN BIT(16) +#define RKVENC_ENC_STRT_DONE_TMVP_EN BIT(18) + +#define RKVENC_ENC_CLR 0x008 +#define RKVENC_ENC_CLR_SAFE BIT(0) +#define RKVENC_ENC_CLR_FORCE BIT(1) + +#define RKVENC_LKT_ADDR 0x00c +#define RKVENC_INT_EN 0x010 +#define RKVENC_INT_MSK 0x014 +#define RKVENC_INT_CLR 0x018 +#define RKVENC_INT_STA 0x01c +#define RKVENC_INT_ENC_DONE BIT(0) +#define RKVENC_INT_LKT_DONE BIT(1) +#define RKVENC_INT_SCLR_DONE BIT(2) +#define RKVENC_INT_SLICE_DONE BIT(3) +#define RKVENC_INT_OFLW BIT(4) +#define RKVENC_INT_BRSP BIT(5) +#define RKVENC_INT_BERR BIT(6) +#define RKVENC_INT_RERR BIT(7) +#define RKVENC_INT_WDG BIT(8) +#define RKVENC_INT_ALL GENMASK(8, 0) +#define RKVENC_INT_ERRORS GENMASK(8, 4) + +#define RKVENC_ENC_RSL 0x030 +#define RKVENC_ENC_RSL_WD8_M1 GENMASK(8, 0) +#define RKVENC_ENC_RSL_WFILL GENMASK(15, 10) +#define RKVENC_ENC_RSL_HD8_M1 GENMASK(24, 16) +#define RKVENC_ENC_RSL_HFILL GENMASK(31, 26) + +#define RKVENC_ENC_PIC 0x034 +#define RKVENC_ENC_PIC_STND_HEVC BIT(0) +#define RKVENC_ENC_PIC_ROI_ENC BIT(1) +#define RKVENC_ENC_PIC_CUR_FRM_REF BIT(2) +#define RKVENC_ENC_PIC_BS_SCP BIT(4) +#define RKVENC_ENC_PIC_LAMB_MOD_SEL BIT(5) +#define RKVENC_ENC_PIC_QP GENMASK(13, 8) +#define RKVENC_ENC_PIC_TOT_POC_NUM GENMASK(18, 14) +#define RKVENC_ENC_PIC_LOG2_CTU_NUM GENMASK(22, 19) + +#define RKVENC_ENC_WDG 0x038 +#define RKVENC_ENC_WDG_VS_LOAD_THD GENMASK(23, 0) + +#define RKVENC_DTRNS_MAP 0x03c +#define RKVENC_DTRNS_MAP_BSW_BUS_EDIN GENMASK(14, 12) + +#define RKVENC_DTRNS_CFG 0x040 +#define RKVENC_DTRNS_CFG_DSPR_OTSD BIT(7) + +#define RKVENC_SRC_FMT 0x044 +#define RKVENC_SRC_FMT_CFMT GENMASK(5, 2) +#define RKVENC_SRC_FMT_CFMT_NV12 6 +#define RKVENC_SRC_FMT_RANGE BIT(6) + +#define RKVENC_SRC_PROC 0x058 +#define RKVENC_SRC_PROC_TXA_EN BIT(29) + +#define RKVENC_KLUT_OFST 0x064 +#define RKVENC_KLUT_WGT(n) (0x068 + 4 * (n)) +#define RKVENC_KLUT_WORDS 24 +#define RKVENC_KLUT_ENTRIES 35 + +#define RKVENC_RC_CFG 0x0c8 +#define RKVENC_RC_CFG_RC_EN BIT(0) +#define RKVENC_RC_CFG_AQ_EN BIT(1) +#define RKVENC_RC_CFG_AQ_MODE BIT(2) +#define RKVENC_RC_CFG_CTU_NUM GENMASK(31, 16) + +#define RKVENC_RC_QP 0x0cc +#define RKVENC_RC_QP_RANGE GENMASK(19, 16) +#define RKVENC_RC_QP_MAX GENMASK(25, 20) +#define RKVENC_RC_QP_MIN GENMASK(31, 26) + +#define RKVENC_RC_TGT 0x0d0 +#define RKVENC_RC_TGT_CTU_EBIT GENMASK(19, 0) + +#define RKVENC_RC_ADJ0 0x0d4 +#define RKVENC_RC_ADJ1 0x0d8 +#define RKVENC_RC_DTHD(n) (0x0dc + 4 * (n)) +#define RKVENC_RC_STEPS 9 + +#define RKVENC_ROI_QTHD(n) (0x100 + 4 * (n)) +#define RKVENC_PIC_OFST 0x110 + +#define RKVENC_SRC_STRID 0x114 +#define RKVENC_SRC_STRID_STRD0 GENMASK(15, 0) +#define RKVENC_SRC_STRID_STRD1 GENMASK(31, 16) + +#define RKVENC_ADR_SRC0 0x118 +#define RKVENC_ADR_SRC1 0x11c +#define RKVENC_ADR_SRC2 0x120 +#define RKVENC_ADR_ROI 0x124 +#define RKVENC_ADR_RFPW_H 0x128 +#define RKVENC_ADR_RFPW_B 0x12c +#define RKVENC_ADR_RFPR_H 0x130 +#define RKVENC_ADR_RFPR_B 0x134 +#define RKVENC_ADR_CMVW 0x138 +#define RKVENC_ADR_CMVR 0x13c +#define RKVENC_ADR_DSPW 0x140 +#define RKVENC_ADR_DSPR 0x144 +#define RKVENC_ADR_MEIW 0x148 +#define RKVENC_ADR_BSBT 0x14c +#define RKVENC_ADR_BSBB 0x150 +#define RKVENC_ADR_BSBR 0x154 +#define RKVENC_ADR_BSBS 0x158 + +#define RKVENC_SLI_SPLT 0x15c +#define RKVENC_SLI_BYTE 0x160 + +#define RKVENC_ME_RNGE 0x164 +#define RKVENC_ME_RNGE_CME_SRCH_H GENMASK(3, 0) +#define RKVENC_ME_RNGE_CME_SRCH_V GENMASK(7, 4) +#define RKVENC_ME_RNGE_RME_SRCH_H GENMASK(10, 8) +#define RKVENC_ME_RNGE_RME_SRCH_V GENMASK(13, 11) +#define RKVENC_ME_RNGE_DLT_FRM_NUM GENMASK(31, 16) + +#define RKVENC_ME_CNST 0x168 +#define RKVENC_ME_CNST_PMV_MDST_H GENMASK(7, 0) +#define RKVENC_ME_CNST_PMV_MDST_V GENMASK(15, 8) +#define RKVENC_ME_CNST_MV_LIMIT GENMASK(17, 16) +#define RKVENC_ME_CNST_PMV_NUM GENMASK(19, 18) + +#define RKVENC_ME_RAM 0x16c +#define RKVENC_ME_RAM_CME_RAMA_MAX GENMASK(10, 0) +#define RKVENC_ME_RAM_CME_RAMA_H GENMASK(15, 11) +#define RKVENC_ME_RAM_CACH_L2_MAP GENMASK(17, 16) + +#define RKVENC_IPRD_CSTS 0x194 +#define RKVENC_IPRD_CSTS_VTHD_Y GENMASK(11, 0) +#define RKVENC_IPRD_CSTS_VTHD_C GENMASK(27, 16) + +#define RKVENC_RDO_CFG 0x198 +#define RKVENC_H264_RDO_RECT_SIZE BIT(0) +#define RKVENC_H264_RDO_INTER_4X4 BIT(1) +#define RKVENC_H264_RDO_VLC_LMT BIT(3) +#define RKVENC_H264_RDO_CHRM_SPCL BIT(4) +#define RKVENC_H264_RDO_CCWA_E BIT(13) +#define RKVENC_H264_RDO_SCL_LST_SEL BIT(14) +#define RKVENC_H264_RDO_ATR_E BIT(15) +#define RKVENC_HEVC_RDO_CHRM_SPECIAL BIT(2) +#define RKVENC_HEVC_RDO_CU_INTER_EN GENMASK(9, 6) +#define RKVENC_HEVC_RDO_CU_INTRA_EN GENMASK(22, 19) +#define RKVENC_HEVC_RDO_CHRM_KLUT_EN BIT(23) + +#define RKVENC_SYNT_NAL 0x19c +#define RKVENC_H264_NAL_REF_IDC GENMASK(1, 0) +#define RKVENC_H264_NAL_UNIT_TYPE GENMASK(6, 2) +#define RKVENC_HEVC_NAL_UNIT_TYPE GENMASK(5, 0) + +#define RKVENC_SYNT_SPS 0x1a0 +#define RKVENC_H264_SPS_MAX_FNUM GENMASK(3, 0) +#define RKVENC_H264_SPS_DRCT_8X8 BIT(4) +#define RKVENC_H264_SPS_MPOC_LM4 GENMASK(8, 5) +#define RKVENC_HEVC_SPS_SAO_EN BIT(0) +#define RKVENC_HEVC_SPS_TMVP_EN BIT(15) +#define RKVENC_HEVC_SPS_LOG2_MAX_POC_LSB GENMASK(19, 16) +#define RKVENC_HEVC_SPS_STRG_INTRA_SMTH BIT(20) + +#define RKVENC_SYNT_PPS 0x1a4 +#define RKVENC_PPS_PIC_INIT_QP GENMASK(12, 7) +#define RKVENC_H264_PPS_ETPY_MODE BIT(0) +#define RKVENC_H264_PPS_TRNS_8X8 BIT(1) +#define RKVENC_H264_PPS_CB_OFST GENMASK(17, 13) +#define RKVENC_H264_PPS_CR_OFST GENMASK(22, 18) +#define RKVENC_H264_PPS_DBF_CP_FLG BIT(24) +#define RKVENC_HEVC_PPS_CBC_INIT_PRSNT BIT(6) +#define RKVENC_HEVC_PPS_CU_QP_DLT_EN BIT(13) + +#define RKVENC_SYNT_SLI0 0x1a8 +#define RKVENC_H264_SLI0_SLI_TYPE GENMASK(1, 0) +#define RKVENC_H264_SLI0_PPS_ID GENMASK(9, 2) +#define RKVENC_H264_SLI0_CBC_INIT_IDC GENMASK(13, 12) +#define RKVENC_H264_SLI0_FRM_NUM GENMASK(31, 16) +#define RKVENC_HEVC_SLI0_CBC_INIT_FLG BIT(0) +#define RKVENC_HEVC_SLI0_MERGE_UP BIT(2) +#define RKVENC_HEVC_SLI0_MERGE_LEFT BIT(3) +#define RKVENC_HEVC_SLI0_SAO_CHRM BIT(11) +#define RKVENC_HEVC_SLI0_SAO_LUMA BIT(12) +#define RKVENC_HEVC_SLI0_PIC_OUT_FLG BIT(14) +#define RKVENC_HEVC_SLI0_SLI_TYPE GENMASK(16, 15) + +#define RKVENC_SYNT_SLI1 0x1ac +#define RKVENC_H264_SLI1_IDR_PIC_ID GENMASK(15, 0) +#define RKVENC_H264_SLI1_POC_LSB GENMASK(31, 16) +#define RKVENC_HEVC_SLI1_TC_OFST GENMASK(3, 0) +#define RKVENC_HEVC_SLI1_BETA_OFST GENMASK(7, 4) +#define RKVENC_HEVC_SLI1_DBLK_DIS BIT(9) +#define RKVENC_HEVC_SLI1_SLI_QP GENMASK(21, 16) +#define RKVENC_HEVC_SLI1_MAX_MRG_CND GENMASK(24, 22) +#define RKVENC_HEVC_SLI1_COL_FRM_L0 BIT(26) + +#define RKVENC_SYNT_SLI2 0x1b0 +#define RKVENC_H264_SLI2_BETA_OFST GENMASK(6, 3) +#define RKVENC_H264_SLI2_ALPH_OFST GENMASK(10, 7) +#define RKVENC_H264_SLI2_DIS_DBLK_IDC GENMASK(12, 11) +#define RKVENC_HEVC_SLI2_POC_LSB GENMASK(15, 0) + +#define RKVENC_SYNT_REFM0 0x1b4 +#define RKVENC_SYNT_REFM1 0x1b8 +#define RKVENC_HEVC_REFM1_USED_BY_S0 GENMASK(3, 0) +#define RKVENC_HEVC_REFM1_NUM_NEG_PIC GENMASK(9, 5) +#define RKVENC_OSD_CFG 0x1c0 +#define RKVENC_SYNT_REFM2 0x1c8 +#define RKVENC_SYNT_REFM3 0x1cc + +#define RKVENC_L1_CFG_END 0x210 + +#define RKVENC_ST_BSL 0x210 +#define RKVENC_ST_BSL_LEN GENMASK(26, 0) +#define RKVENC_ST_SSE_L32 0x214 +#define RKVENC_ST_SSE_QP 0x218 +#define RKVENC_ST_ENC 0x228 +#define RKVENC_ST_BSB 0x234 +#define RKVENC_ST_BUS 0x238 +#define RKVENC_ST_PNUM(n) (0x244 + 4 * (n)) +#define RKVENC_ST_B8_QP(n) (0x264 + 4 * (n)) +#define RKVENC_ST_CPLX_TMP 0x334 +#define RKVENC_ST_BNUM_CME 0x338 +#define RKVENC_ST_CPLX_SPT 0x33c +#define RKVENC_ST_BNUM_B16 0x340 + +#define RKVENC_L2CFG_ADDR 0x3f0 +#define RKVENC_L2CFG_ADDR_AUTO_INC BIT(0) +#define RKVENC_L2CFG_WDATA 0x3f4 +#define RKVENC_L2CFG_RDATA 0x3f8 + +#define RKVENC_L2_FIRST 0x004 +#define RKVENC_L2_H264_LAST 0x30c +#define RKVENC_L2_HEVC_LAST 0x300 +#define RKVENC_L2_WORDS (RKVENC_L2_H264_LAST / 4 + 1) + +#define RKVENC_L2_IPRD_TTHDY4(n) (0x004 + 4 * (n)) +#define RKVENC_L2_IPRD_TTHDC8(n) (0x00c + 4 * (n)) +#define RKVENC_L2_IPRD_TTHDY8(n) (0x014 + 4 * (n)) +#define RKVENC_L2_IPRD_TTHD_UL 0x01c +#define RKVENC_L2_IPRD_WGTY8 0x020 +#define RKVENC_L2_IPRD_WGTY4 0x024 +#define RKVENC_L2_IPRD_WGTY16 0x028 +#define RKVENC_L2_IPRD_WGTC8 0x02c +#define RKVENC_L2_QNT_BIAS 0x030 +#define RKVENC_L2_QNT_BIAS_I GENMASK(9, 0) +#define RKVENC_L2_QNT_BIAS_P GENMASK(19, 10) +#define RKVENC_L2_ATR_THD0 0x034 +#define RKVENC_L2_ATR_THD1 0x038 +#define RKVENC_L2_ATR_WGT16 0x03c +#define RKVENC_L2_ATR_WGT8 0x040 +#define RKVENC_L2_ATR_WGT4 0x044 +#define RKVENC_L2_ATF_TTHD(n) (0x048 + 4 * (n)) +#define RKVENC_L2_ATF_STHD(n) (0x050 + 4 * (n)) +#define RKVENC_L2_ATF_WGT(n) (0x058 + 4 * (n)) +#define RKVENC_L2_ATF_OFST(n) (0x064 + 4 * (n)) +#define RKVENC_L2_IPRD_WGT_QP(q) (0x070 + 4 * (q)) +#define RKVENC_L2_RDO_WGTA_QP(q) (0x140 + 4 * (q)) +#define RKVENC_L2_RDO_WGTB_QP(q) (0x210 + 4 * (q)) +#define RKVENC_L2_MADI_CFG 0x2e0 +#define RKVENC_L2_AQ_TTHD(n) (0x2e4 + 4 * (n)) +#define RKVENC_L2_AQ_STP(n) (0x2f4 + 4 * (n)) +#define RKVENC_L2_RME_MVD_PNSH 0x304 +#define RKVENC_L2_ATR1_THD0 0x308 +#define RKVENC_L2_ATR1_THD1 0x30c + +#define RKVENC_L2_HEVC_INTRA_THD32(n) (0x004 + 4 * (n)) +#define RKVENC_L2_HEVC_INTRA_THD16(n) (0x00c + 4 * (n)) +#define RKVENC_L2_HEVC_INTRA_WGT32(n) (0x020 + 4 * (n)) +#define RKVENC_L2_HEVC_INTRA_WGT16(n) (0x028 + 4 * (n)) +#define RKVENC_L2_HEVC_ATF_THD(n) (0x048 + 4 * (n)) +#define RKVENC_L2_HEVC_ATF_WGT(n) (0x05c + 4 * (n)) + +#define RKVENC_QP_MAX 51 + +#endif diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c new file mode 100644 index 0000000000000..da4794b82bb12 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541-tables.c @@ -0,0 +1,256 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Tuning tables of the RKVENC v1 encoder. Tables with a closed form are + * computed once at module load in integer fixed-point arithmetic. + */ + +#include <linux/build_bug.h> +#include <linux/cache.h> +#include <linux/math64.h> + +#include "rkvenc-vepu541.h" + +/* Fraction bits of the fixed-point numbers; the largest value is below 2^8 */ +#define RKVENC_FX_BITS 56 +#define RKVENC_FX_ONE (1ULL << RKVENC_FX_BITS) +#define RKVENC_FX_TWO (2ULL << RKVENC_FX_BITS) + +/* Weights and lambdas grow by 2^(1/3) per step, sixths give square roots */ +#define RKVENC_STEPS_PER_OCTAVE 3 +#define RKVENC_SIXTHS_PER_OCTAVE 6 +#define RKVENC_SIXTHS_PER_STEP 2 + +/* H.264 weight k = floor(16 * 2^(k / 3)); 16 is the neutral weight */ +#define RKVENC_H264_KLUT_NEUTRAL 16 + +/* H.264 lambda j = floor(3.6 * 2^((j + 5) / 3)), the factor as 18 / 5 */ +#define RKVENC_H264_LAMBDA_FACTOR_NUM 18 +#define RKVENC_H264_LAMBDA_FACTOR_DEN 5 +#define RKVENC_H264_LAMBDA_J_OFFSET 5 + +/* HEVC lambda 0.57 * 2^((q - 12) / 3), 11 fraction bits */ +#define RKVENC_HEVC_LAMBDA_PCT 57 +#define RKVENC_HEVC_LAMBDA_QP_REF 12 +#define RKVENC_HEVC_LAMBDA_FRAC 11 +#define RKVENC_PERCENT 100 + +/* HEVC weight i = S[i mod 3] * 2^(i div 3), S about 2^(7 + n / 3) */ +#define RKVENC_HEVC_KLUT_S0 128 +#define RKVENC_HEVC_KLUT_S1 161 +#define RKVENC_HEVC_KLUT_S2 204 +#define RKVENC_HEVC_LAMBDA_B_S0 112 +#define RKVENC_HEVC_LAMBDA_B_S1 137 +#define RKVENC_HEVC_LAMBDA_B_S2 176 +#define RKVENC_KLUT_ENTRY_BITS 18 + +/* Three KLUT weights per word pair, the middle one split across both */ +#define RKVENC_KLUT_PAIRS (RKVENC_KLUT_WORDS / 2) +#define RKVENC_KLUT_EVEN_WORD(n) (2 * (n)) +#define RKVENC_KLUT_ODD_WORD(n) (2 * (n) + 1) +#define RKVENC_KLUT_SPLIT_BITS 9 +#define RKVENC_KLUT_SPLIT_EVEN_SHIFT 23 +#define RKVENC_KLUT_LAST_ODD_SHIFT 14 + +static_assert(RKVENC_SIXTHS_PER_STEP * RKVENC_STEPS_PER_OCTAVE == + RKVENC_SIXTHS_PER_OCTAVE); +static_assert(RKVENC_HEVC_LAMBDA_QP_REF % RKVENC_SIXTHS_PER_OCTAVE == 0); + +u32 rkvenc_h264_klut[RKVENC_KLUT_WORDS] __ro_after_init; +u32 rkvenc_hevc_klut[RKVENC_KLUT_WORDS] __ro_after_init; +u32 rkvenc_h264_lambda[RKVENC_QP_MAX + 2] __ro_after_init; +u32 rkvenc_hevc_lambda_satd[RKVENC_QP_MAX + 1] __ro_after_init; +u32 rkvenc_hevc_lambda_a[RKVENC_QP_MAX + 1] __ro_after_init; +u32 rkvenc_hevc_lambda_b[RKVENC_QP_MAX + 1] __ro_after_init; + +typedef u64 (*rkvenc_fx_pow_t)(u64 x); + +static u64 fx_mul(u64 a, u64 b) +{ + return mul_u64_u64_shr(a, b, RKVENC_FX_BITS); +} + +static u64 fx_pow2(u64 x) +{ + return fx_mul(x, x); +} + +static u64 fx_pow6(u64 x) +{ + u64 x3 = fx_mul(fx_pow2(x), x); + + return fx_mul(x3, x3); +} + +/* Largest x below 2.0 whose power does not exceed target, by bisection */ +static u64 fx_root(rkvenc_fx_pow_t pow, u64 target) +{ + u64 lo = 0, hi = RKVENC_FX_TWO, mid; + + while (lo < hi) { + mid = (lo + hi + 1) / 2; + if (pow(mid) <= target) + lo = mid; + else + hi = mid - 1; + } + + return lo; +} + +/* x * 2^exp, rounded down or to nearest */ +static u32 fx_floor(u64 x, int exp) +{ + return x >> (RKVENC_FX_BITS - exp); +} + +static u32 fx_round(u64 x, int exp) +{ + unsigned int s = RKVENC_FX_BITS - exp; + + return (x + (1ULL << (s - 1))) >> s; +} + +static void klut_pack(u32 *words, const u32 *weights) +{ + u32 n, e0, e1, e2, low, i = 0; + + for (n = 0; n < RKVENC_KLUT_PAIRS; n++) { + e0 = weights[i++]; + e1 = i < RKVENC_KLUT_ENTRIES ? weights[i++] : 0; + e2 = i < RKVENC_KLUT_ENTRIES ? weights[i++] : 0; + low = e1 & GENMASK(RKVENC_KLUT_SPLIT_BITS - 1, 0); + words[RKVENC_KLUT_EVEN_WORD(n)] = + e0 | low << RKVENC_KLUT_SPLIT_EVEN_SHIFT; + words[RKVENC_KLUT_ODD_WORD(n)] = + e1 >> RKVENC_KLUT_SPLIT_BITS | + e2 << RKVENC_KLUT_LAST_ODD_SHIFT; + } +} + +static void h264_klut_init(const u64 *step_frac) +{ + u32 weights[RKVENC_KLUT_ENTRIES]; + u64 x; + u32 k; + + for (k = 0; k < RKVENC_KLUT_ENTRIES; k++) { + x = RKVENC_H264_KLUT_NEUTRAL * + step_frac[k % RKVENC_STEPS_PER_OCTAVE]; + weights[k] = fx_floor(x, k / RKVENC_STEPS_PER_OCTAVE); + } + + klut_pack(rkvenc_h264_klut, weights); +} + +static void hevc_klut_init(void) +{ + static const u16 s[RKVENC_STEPS_PER_OCTAVE] = { RKVENC_HEVC_KLUT_S0, + RKVENC_HEVC_KLUT_S1, + RKVENC_HEVC_KLUT_S2 }; + u32 weights[RKVENC_KLUT_ENTRIES]; + u64 v; + u32 i; + + for (i = 0; i < RKVENC_KLUT_ENTRIES; i++) { + v = (u64)s[i % RKVENC_STEPS_PER_OCTAVE] + << (i / RKVENC_STEPS_PER_OCTAVE); + weights[i] = + min_t(u64, v, GENMASK(RKVENC_KLUT_ENTRY_BITS - 1, 0)); + } + + klut_pack(rkvenc_hevc_klut, weights); +} + +static void h264_lambda_init(const u64 *step_frac) +{ + u32 j, m; + u64 x; + + for (j = 0; j < ARRAY_SIZE(rkvenc_h264_lambda); j++) { + m = j + RKVENC_H264_LAMBDA_J_OFFSET; + x = mul_u64_u32_div(step_frac[m % RKVENC_STEPS_PER_OCTAVE], + RKVENC_H264_LAMBDA_FACTOR_NUM, + RKVENC_H264_LAMBDA_FACTOR_DEN); + rkvenc_h264_lambda[j] = + fx_floor(x, m / RKVENC_STEPS_PER_OCTAVE); + } +} + +/* The SATD lambda is the square root, sqrt(0.57) * 2^((q - 12) / 6) */ +static void hevc_lambda_init(const u64 *sixth, u64 sqrt_factor) +{ + u32 pct = (RKVENC_HEVC_LAMBDA_PCT << RKVENC_HEVC_LAMBDA_FRAC) >> + (RKVENC_HEVC_LAMBDA_QP_REF / RKVENC_STEPS_PER_OCTAVE); + int exp = RKVENC_HEVC_LAMBDA_FRAC - + RKVENC_HEVC_LAMBDA_QP_REF / RKVENC_SIXTHS_PER_OCTAVE; + static const u16 b[RKVENC_STEPS_PER_OCTAVE] = { + RKVENC_HEVC_LAMBDA_B_S0, RKVENC_HEVC_LAMBDA_B_S1, + RKVENC_HEVC_LAMBDA_B_S2 + }; + u32 q, r; + u64 x; + + for (q = 0; q <= RKVENC_QP_MAX; q++) { + r = q % RKVENC_STEPS_PER_OCTAVE; + x = mul_u64_u32_div(sixth[RKVENC_SIXTHS_PER_STEP * r], pct, + RKVENC_PERCENT); + rkvenc_hevc_lambda_a[q] = + fx_round(x, q / RKVENC_STEPS_PER_OCTAVE); + + x = fx_mul(sqrt_factor, sixth[q % RKVENC_SIXTHS_PER_OCTAVE]); + rkvenc_hevc_lambda_satd[q] = + fx_round(x, q / RKVENC_SIXTHS_PER_OCTAVE + exp); + + rkvenc_hevc_lambda_b[q] = b[r] << (q / RKVENC_STEPS_PER_OCTAVE); + } +} + +void __init rkvenc_tables_init(void) +{ + u64 sixth[RKVENC_SIXTHS_PER_OCTAVE]; + u64 step_frac[RKVENC_STEPS_PER_OCTAVE]; + u64 factor, c, sqrt_factor; + u32 r; + + c = fx_root(fx_pow6, RKVENC_FX_TWO); + sixth[0] = RKVENC_FX_ONE; + for (r = 1; r < ARRAY_SIZE(sixth); r++) + sixth[r] = fx_mul(sixth[r - 1], c); + + for (r = 0; r < ARRAY_SIZE(step_frac); r++) + step_frac[r] = sixth[RKVENC_SIXTHS_PER_STEP * r]; + + factor = + div_u64(RKVENC_HEVC_LAMBDA_PCT * RKVENC_FX_ONE, RKVENC_PERCENT); + sqrt_factor = fx_root(fx_pow2, factor); + + h264_klut_init(step_frac); + hevc_klut_init(); + h264_lambda_init(step_frac); + hevc_lambda_init(sixth, sqrt_factor); +} + +/* Values without a closed form */ +const u8 rkvenc_h264_aq_thd[16] = { + 0, 0, 0, 0, 3, 3, 5, 5, 8, 8, 8, 15, 15, 20, 25, 25, +}; + +const s8 rkvenc_h264_aq_step_i[16] = { + -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 3, 3, 4, 5, 8, 8, +}; + +const s8 rkvenc_h264_aq_step_p[16] = { + -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 7, 8, +}; + +const u8 rkvenc_hevc_aq_thd[16] = { + 0, 0, 0, 0, 3, 3, 5, 5, 8, 8, 8, 15, 15, 20, 25, 35, +}; + +const s8 rkvenc_hevc_aq_step[16] = { + -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 7, 8, +}; + +const u16 rkvenc_hevc_intra_thd[4] = { 2, 6, 16, 36 }; +const u8 rkvenc_hevc_intra_wgt32[7] = { 23, 22, 21, 20, 22, 24, 26 }; +const u8 rkvenc_hevc_intra_wgt16[7] = { 17, 17, 17, 18, 17, 18, 18 }; diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c new file mode 100644 index 0000000000000..8bca4f6527347 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.c @@ -0,0 +1,215 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RKVENC v1 register image builder, codec independent part. + */ + +#include <linux/math64.h> + +#include "rkvenc-vepu541.h" + +#define RKVENC_MIN_SIZE 8 +#define RKVENC_MAX_SIZE 4096 + +static const u16 rkvenc_l1_gaps[][2] = { + { RKVENC_INT_CLR, 0x02c }, + { 0x184, 0x190 }, + { 0x1bc, 0x1bc }, +}; + +bool rkvenc_vepu541_l1_written(u32 reg) +{ + u32 i; + + if (reg < RKVENC_ENC_CLR || reg >= RKVENC_L1_CFG_END) + return false; + + for (i = 0; i < ARRAY_SIZE(rkvenc_l1_gaps); i++) + if (reg >= rkvenc_l1_gaps[i][0] && reg <= rkvenc_l1_gaps[i][1]) + return false; + + return true; +} + +void rkvenc_vepu541_slot_sizes(enum rkvenc_codec codec, u32 width, u32 height, + struct rkvenc_slot_sizes *sz) +{ + u32 aw, ah, wc, hx; + + if (codec == RKVENC_CODEC_H264) { + aw = ALIGN(width, 64); + ah = ALIGN(height, 64) + 16; + sz->rfp_hdr = ALIGN(aw * ah / 64, 8192); + sz->rfp_body = aw * ah * 3 / 2; + sz->dsp = ALIGN(aw / 64 * ah / 64 * 256, 8192); + sz->cmv = 0; + return; + } + + wc = DIV_ROUND_UP(width, 64); + hx = DIV_ROUND_UP(height, 64) + 1; + sz->rfp_hdr = ALIGN(wc * hx * 64, 8192); + sz->rfp_body = wc * hx * 4096 * 3 / 2; + sz->dsp = wc * hx * 256; + sz->cmv = ALIGN(wc * hx * 64, 256); +} + +static void rkvenc_rc_build(const struct rkvenc_frame *f, u32 blocks, + u32 row_blocks, struct rkvenc_img *img) +{ + const struct rkvenc_rc *rc = f->rc; + u32 cfg, ebit, d, adj0, adj1, i; + s32 thd[RKVENC_RC_STEPS]; + u64 ebit64; + + cfg = 0; + if (f->aq) + cfg |= RKVENC_RC_CFG_AQ_EN; + if (f->aq && f->codec == RKVENC_CODEC_HEVC) + cfg |= RKVENC_RC_CFG_AQ_MODE; + + rkvenc_img_l1(img, RKVENC_RC_QP, + FIELD_PREP(RKVENC_RC_QP_RANGE, rc ? rc->qp_range : 0) | + FIELD_PREP(RKVENC_RC_QP_MAX, f->qp_max) | + FIELD_PREP(RKVENC_RC_QP_MIN, f->qp_min)); + + if (!rc) { + rkvenc_img_l1(img, RKVENC_RC_CFG, cfg); + return; + } + + cfg |= RKVENC_RC_CFG_RC_EN | + FIELD_PREP(RKVENC_RC_CFG_CTU_NUM, row_blocks); + rkvenc_img_l1(img, RKVENC_RC_CFG, cfg); + + ebit64 = div_u64(16ULL * rc->target_bits, blocks); + ebit = min_t(u64, ebit64, RKVENC_RC_TGT_CTU_EBIT); + rkvenc_img_l1(img, RKVENC_RC_TGT, ebit); + + /* The row rate control measures its deviation per row of blocks */ + d = ebit * row_blocks / 16 * 5 / 16; + thd[0] = -2 * (s32)d; + thd[1] = -(s32)d; + thd[2] = d; + thd[3] = 2 * d; + for (i = 4; i < RKVENC_RC_STEPS; i++) + thd[i] = 0x7fffffff; + + adj0 = 0; + adj1 = 0; + for (i = 0; i < RKVENC_RC_STEPS; i++) { + rkvenc_img_l1(img, RKVENC_RC_DTHD(i), (u32)thd[i]); + if (i < 5) + adj0 |= rkvenc_s2u(rc->step[i], 5) << (5 * i); + else + adj1 |= rkvenc_s2u(rc->step[i], 5) << (5 * (i - 5)); + } + rkvenc_img_l1(img, RKVENC_RC_ADJ0, adj0); + rkvenc_img_l1(img, RKVENC_RC_ADJ1, adj1); +} + +static void rkvenc_buffers_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + const struct rkvenc_src *src = &f->src; + const struct rkvenc_bs *bs = &f->bs; + + rkvenc_img_l1(img, RKVENC_SRC_FMT, + FIELD_PREP(RKVENC_SRC_FMT_CFMT, + RKVENC_SRC_FMT_CFMT_NV12)); + rkvenc_img_l1(img, RKVENC_SRC_PROC, RKVENC_SRC_PROC_TXA_EN); + rkvenc_img_l1(img, RKVENC_SRC_STRID, + FIELD_PREP(RKVENC_SRC_STRID_STRD0, src->stride[0]) | + FIELD_PREP(RKVENC_SRC_STRID_STRD1, src->stride[1])); + rkvenc_img_l1(img, RKVENC_ADR_SRC0, src->plane[0]); + rkvenc_img_l1(img, RKVENC_ADR_SRC1, src->plane[1]); + rkvenc_img_l1(img, RKVENC_ADR_SRC2, src->plane[2]); + + if (f->referenced) { + rkvenc_img_l1(img, RKVENC_ADR_RFPW_H, f->recon.rfp_hdr); + rkvenc_img_l1(img, RKVENC_ADR_RFPW_B, f->recon.rfp_body); + } + rkvenc_img_l1(img, RKVENC_ADR_DSPW, f->recon.dsp); + rkvenc_img_l1(img, RKVENC_ADR_RFPR_H, f->ref.rfp_hdr); + rkvenc_img_l1(img, RKVENC_ADR_RFPR_B, f->ref.rfp_body); + rkvenc_img_l1(img, RKVENC_ADR_DSPR, f->ref.dsp); + if (f->codec == RKVENC_CODEC_HEVC) { + rkvenc_img_l1(img, RKVENC_ADR_CMVW, f->recon.cmv); + rkvenc_img_l1(img, RKVENC_ADR_CMVR, f->ref.cmv); + } + + rkvenc_img_l1(img, RKVENC_ADR_BSBB, bs->base); + rkvenc_img_l1(img, RKVENC_ADR_BSBR, bs->base); + rkvenc_img_l1(img, RKVENC_ADR_BSBT, bs->base + bs->size - 1); + rkvenc_img_l1(img, RKVENC_ADR_BSBS, bs->base + bs->offset); + rkvenc_img_l1(img, RKVENC_DTRNS_MAP, + FIELD_PREP(RKVENC_DTRNS_MAP_BSW_BUS_EDIN, 7)); +} + +static int rkvenc_frame_check(const struct rkvenc_frame *f) +{ + u32 w = f->src.width, h = f->src.height; + + if (w < RKVENC_MIN_SIZE || h < RKVENC_MIN_SIZE || + w > RKVENC_MAX_SIZE || h > RKVENC_MAX_SIZE) + return -EINVAL; + + if (f->qp > RKVENC_QP_MAX || f->qp_min > f->qp_max || + f->qp_max > RKVENC_QP_MAX) + return -EINVAL; + + /* Hardware bug: H.264 at most 2 MBs wide and over 1 MB high hangs */ + if (f->codec == RKVENC_CODEC_H264 && ALIGN(w, 16) <= 32 && + ALIGN(h, 16) > 16) + return -EINVAL; + + /* Hardware bug: HEVC P pictures one CTU row high decode wrongly */ + if (f->codec == RKVENC_CODEC_HEVC && !f->intra && h <= 64) + return -EINVAL; + + if (f->bs.size < 256 || f->bs.offset >= f->bs.size) + return -EINVAL; + + return 0; +} + +int rkvenc_vepu541_img_build(const struct rkvenc_frame *f, + struct rkvenc_img *img) +{ + u32 blocks, row_blocks, int_en; + int ret; + + ret = rkvenc_frame_check(f); + if (ret) + return ret; + + memset(img, 0, sizeof(*img)); + + int_en = RKVENC_INT_ENC_DONE | RKVENC_INT_SCLR_DONE | RKVENC_INT_OFLW | + RKVENC_INT_BRSP | RKVENC_INT_BERR | RKVENC_INT_RERR; + if (f->wdg_vs_load) + int_en |= RKVENC_INT_WDG; + rkvenc_img_l1(img, RKVENC_INT_EN, int_en); + rkvenc_img_l1(img, RKVENC_ENC_WDG, + FIELD_PREP(RKVENC_ENC_WDG_VS_LOAD_THD, f->wdg_vs_load)); + + rkvenc_buffers_build(f, img); + + if (f->codec == RKVENC_CODEC_H264) { + row_blocks = DIV_ROUND_UP(f->src.width, 16); + blocks = row_blocks * DIV_ROUND_UP(f->src.height, 16); + rkvenc_vepu541_h264_build(f, img); + } else { + row_blocks = DIV_ROUND_UP(f->src.width, 64); + blocks = row_blocks * DIV_ROUND_UP(f->src.height, 64); + rkvenc_vepu541_hevc_build(f, img); + } + + rkvenc_rc_build(f, blocks, row_blocks, img); + + img->enc_strt = FIELD_PREP(RKVENC_ENC_STRT_CMD, + RKVENC_ENC_STRT_CMD_FRAME) | + RKVENC_ENC_STRT_CLK_GATE_EN | + RKVENC_ENC_STRT_DONE_TMVP_EN; + + return 0; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h new file mode 100644 index 0000000000000..33a13402d0ade --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-vepu541.h @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef RKVENC_VEPU541_H +#define RKVENC_VEPU541_H + +#include <linux/errno.h> +#include <linux/string.h> +#include <linux/types.h> +#include <linux/bits.h> +#include <linux/bitfield.h> +#include <linux/kernel.h> +#include <linux/minmax.h> +#include <linux/math.h> +#include <linux/log2.h> + +#include "rkvenc-vepu541-regs.h" + +enum rkvenc_codec { + RKVENC_CODEC_H264, + RKVENC_CODEC_HEVC, +}; + +struct rkvenc_src { + u32 plane[3]; + u32 stride[2]; + u32 width; + u32 height; +}; + +struct rkvenc_bs { + u32 base; + u32 size; + u32 offset; +}; + +struct rkvenc_slot { + u32 rfp_hdr; + u32 rfp_body; + u32 dsp; + u32 cmv; +}; + +struct rkvenc_slot_sizes { + u32 rfp_hdr; + u32 rfp_body; + u32 dsp; + u32 cmv; +}; + +struct rkvenc_h264_seq { + u32 width; + u32 height; + u8 profile_idc; + u8 level_idc; + u8 log2_max_frame_num_m4; + u8 log2_max_poc_lsb_m4; + u8 pic_init_qp; + s8 chroma_qp_offset; + bool cabac; + bool transform_8x8; + u8 dblk_idc; + s8 alpha_offset; + s8 beta_offset; + bool timing; + u32 fps; +}; + +struct rkvenc_h264_pic { + const struct rkvenc_h264_seq *seq; + u16 frame_num; + u16 poc_lsb; + u16 idr_pic_id; + u8 nal_ref_idc; +}; + +struct rkvenc_hevc_seq { + u32 width; + u32 height; + u8 level_idc; + u8 log2_max_poc_lsb_m4; + u8 pic_init_qp; + u8 max_merge_cand; + bool sao; + bool deblock_disabled; + s8 beta_offset; + s8 tc_offset; + bool strong_intra_smoothing; + bool cabac_init_present; +}; + +struct rkvenc_hevc_pic { + const struct rkvenc_hevc_seq *seq; + u16 poc_lsb; + u8 nal_unit_type; +}; + +struct rkvenc_rc { + u32 target_bits; + u8 qp_range; + s8 step[RKVENC_RC_STEPS]; +}; + +struct rkvenc_frame { + enum rkvenc_codec codec; + bool intra; + bool referenced; + struct rkvenc_src src; + struct rkvenc_bs bs; + struct rkvenc_slot recon; + struct rkvenc_slot ref; + u8 qp; + u8 qp_min; + u8 qp_max; + bool aq; + const struct rkvenc_rc *rc; + u32 wdg_vs_load; + union { + struct rkvenc_h264_pic h264; + struct rkvenc_hevc_pic hevc; + }; +}; + +#define RKVENC_L1_CFG_WORDS (RKVENC_L1_CFG_END / 4) + +struct rkvenc_img { + u32 l1[RKVENC_L1_CFG_WORDS]; + u32 l2[RKVENC_L2_WORDS]; + u32 l2_last; + u32 enc_strt; +}; + +static inline void rkvenc_img_l1(struct rkvenc_img *img, u32 reg, u32 val) +{ + img->l1[reg / 4] = val; +} + +static inline void rkvenc_img_l2(struct rkvenc_img *img, u32 reg, u32 val) +{ + img->l2[reg / 4] = val; +} + +static inline u32 rkvenc_s2u(s32 val, u32 bits) +{ + return (u32)val & GENMASK(bits - 1, 0); +} + +static inline u32 rkvenc_pair(u32 lo, u32 hi) +{ + return lo | hi << 16; +} + +static inline u32 rkvenc_bytes(u8 b0, u8 b1, u8 b2, u8 b3) +{ + return b0 | b1 << 8 | b2 << 16 | (u32)b3 << 24; +} + +static inline void rkvenc_img_l2_u8(struct rkvenc_img *img, u32 reg, + const u8 *v, u32 n) +{ + u32 i; + + for (i = 0; i < n; i++) + img->l2[reg / 4 + i / 4] |= (u32)v[i] << (8 * (i % 4)); +} + +static inline void rkvenc_img_l2_s6(struct rkvenc_img *img, u32 reg, + const s8 *v, u32 n) +{ + u32 i; + + for (i = 0; i < n; i++) + img->l2[reg / 4 + i / 4] |= rkvenc_s2u(v[i], 6) << (8 * (i % 4)); +} + +bool rkvenc_vepu541_l1_written(u32 reg); +void rkvenc_vepu541_slot_sizes(enum rkvenc_codec codec, u32 width, u32 height, + struct rkvenc_slot_sizes *sz); +int rkvenc_vepu541_img_build(const struct rkvenc_frame *f, + struct rkvenc_img *img); + +void rkvenc_vepu541_h264_build(const struct rkvenc_frame *f, + struct rkvenc_img *img); +void rkvenc_vepu541_hevc_build(const struct rkvenc_frame *f, + struct rkvenc_img *img); + +void rkvenc_tables_init(void); +extern u32 rkvenc_h264_klut[RKVENC_KLUT_WORDS]; +extern u32 rkvenc_hevc_klut[RKVENC_KLUT_WORDS]; +extern u32 rkvenc_h264_lambda[RKVENC_QP_MAX + 2]; +extern u32 rkvenc_hevc_lambda_satd[RKVENC_QP_MAX + 1]; +extern u32 rkvenc_hevc_lambda_a[RKVENC_QP_MAX + 1]; +extern u32 rkvenc_hevc_lambda_b[RKVENC_QP_MAX + 1]; +extern const u8 rkvenc_h264_aq_thd[16]; +extern const s8 rkvenc_h264_aq_step_i[16]; +extern const s8 rkvenc_h264_aq_step_p[16]; +extern const u8 rkvenc_hevc_aq_thd[16]; +extern const s8 rkvenc_hevc_aq_step[16]; +extern const u16 rkvenc_hevc_intra_thd[4]; +extern const u8 rkvenc_hevc_intra_wgt32[7]; +extern const u8 rkvenc_hevc_intra_wgt16[7]; + +#endif diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c new file mode 100644 index 0000000000000..00778e97686d7 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c @@ -0,0 +1,489 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Rockchip RKVENC v1 H.264/HEVC encoder: platform driver, interrupt and + * job handling. + */ + +#include <linux/clk.h> +#include <linux/completion.h> +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/module.h> +#include <linux/platform_device.h> +#include <linux/pm_runtime.h> +#include <linux/reset.h> + +#include <media/v4l2-mem2mem.h> +#include <media/videobuf2-dma-contig.h> + +#include "rkvenc-vepu541-regs.h" +#include "rkvenc.h" + +#define RKVENC_WATCHDOG_MS 1000 +#define RKVENC_SAFE_CLEAR_MS 100 +#define RKVENC_RESET_PULSE_US 5 + +#define RKVENC_L1_CFG_START 0x00c + +static const struct rkvenc_variant rkvenc_variant_rv1126 = { + .vepu540 = false, +}; + +static const struct rkvenc_variant rkvenc_variant_rk3568 = { + .vepu540 = true, +}; + +int rkvenc_hw_run(struct rkvenc_dev *dev, const struct rkvenc_frame *f) +{ + struct rkvenc_img *img = &dev->img; + void __iomem *regs = dev->regs; + unsigned int a; + int ret; + + ret = rkvenc_vepu541_img_build(f, img); + if (WARN_ON(ret)) + return ret; + + /* INT_STA keeps its bits across a start */ + writel(RKVENC_INT_ALL, regs + RKVENC_INT_CLR); + + writel(RKVENC_L2_FIRST | RKVENC_L2CFG_ADDR_AUTO_INC, + regs + RKVENC_L2CFG_ADDR); + for (a = RKVENC_L2_FIRST; a <= img->l2_last; a += 4) + writel(img->l2[a / 4], regs + RKVENC_L2CFG_WDATA); + + /* A write to ENC_CLR would clear the job, so L1 starts above it */ + for (a = RKVENC_L1_CFG_END - 4; a >= RKVENC_L1_CFG_START; a -= 4) + if (rkvenc_vepu541_l1_written(a)) + writel(img->l1[a / 4], regs + a); + + /* A timed out job must not be judged by the previous status. */ + dev->irq_sta = 0; + writel(img->enc_strt, regs + RKVENC_ENC_STRT); + + schedule_delayed_work(&dev->watchdog, + msecs_to_jiffies(RKVENC_WATCHDOG_MS)); + return 0; +} + +void rkvenc_hw_status_read(struct rkvenc_dev *dev, u32 *int_sta, u32 *bytes, + u32 *qp_sum) +{ + void __iomem *regs = dev->regs; + + *int_sta = dev->irq_sta; + *bytes = FIELD_GET(RKVENC_ST_BSL_LEN, readl(regs + RKVENC_ST_BSL)); + *qp_sum = readl(regs + RKVENC_ST_SSE_QP) & GENMASK(23, 0); +} + +static bool rkvenc_hw_force_clear(struct rkvenc_dev *dev) +{ + void __iomem *regs = dev->regs; + + writel(RKVENC_INT_ALL, regs + RKVENC_INT_MSK); + writel(RKVENC_ENC_CLR_FORCE, regs + RKVENC_ENC_CLR); + writel(0, regs + RKVENC_ENC_CLR); + writel(RKVENC_INT_ALL, regs + RKVENC_INT_CLR); + readl(regs + RKVENC_INT_STA); + writel(0, regs + RKVENC_INT_MSK); + + return !(readl(regs + RKVENC_ST_ENC) & GENMASK(1, 0)); +} + +void rkvenc_hw_recover(struct rkvenc_dev *dev) +{ + void __iomem *regs = dev->regs; + + if (rkvenc_hw_force_clear(dev)) + return; + + dev_err(dev->dev, "job did not stop on force clear, resetting\n"); + + writel(RKVENC_INT_ALL & ~RKVENC_INT_SCLR_DONE, regs + RKVENC_INT_MSK); + writel(RKVENC_INT_ALL, regs + RKVENC_INT_CLR); + + reinit_completion(&dev->sclr_done); + writel(RKVENC_ENC_CLR_SAFE, regs + RKVENC_ENC_CLR); + wait_for_completion_timeout(&dev->sclr_done, + msecs_to_jiffies(RKVENC_SAFE_CLEAR_MS)); + + if (dev->num_resets) { + reset_control_bulk_assert(dev->num_resets, dev->resets); + udelay(RKVENC_RESET_PULSE_US); + reset_control_bulk_deassert(dev->num_resets, dev->resets); + } + + writel(0, regs + RKVENC_INT_MSK); +} + +static irqreturn_t rkvenc_irq(int irq, void *priv) +{ + struct rkvenc_dev *dev = priv; + u32 sta; + + sta = readl(dev->regs + RKVENC_INT_STA); + if (!sta) + return IRQ_NONE; + + writel(sta, dev->regs + RKVENC_INT_CLR); + /* Flush the posted clear before the interrupt line is sampled again */ + readl(dev->regs + RKVENC_INT_STA); + + if (sta & RKVENC_INT_SCLR_DONE) + complete(&dev->sclr_done); + + dev->irq_sta = sta; + return IRQ_WAKE_THREAD; +} + +static void rkvenc_finish_job(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx, + enum vb2_buffer_state state) +{ + struct vb2_v4l2_buffer *src, *dst; + u32 payload, int_sta, bytes, qp_sum; + u8 frame_type, qp_avg; + struct v4l2_ctrl *avg_qp; + + rkvenc_hw_status_read(dev, &int_sta, &bytes, &qp_sum); + + if (state == VB2_BUF_STATE_DONE) { + src = v4l2_m2m_next_src_buf(ctx->m2m_ctx); + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + + payload = ctx->cur_prepend + bytes; + frame_type = ctx->cur_intra ? 0 : 1; + if (ctx->qp_blocks) + qp_avg = (u8)(qp_sum / ctx->qp_blocks); + else + qp_avg = ctx->cur_qp; + + if (dst) { + vb2_set_plane_payload(&dst->vb2_buf, 0, payload); + if (src) + v4l2_m2m_buf_copy_metadata(src, dst); + dst->flags |= ctx->cur_intra ? V4L2_BUF_FLAG_KEYFRAME + : V4L2_BUF_FLAG_PFRAME; + } + + rkvenc_rc_update(ctx, payload, qp_avg, frame_type); + rkvenc_gop_commit(ctx); + ctx->consecutive_errors = 0; + ctx->overflow_retried = false; + + avg_qp = v4l2_ctrl_find(&ctx->ctrl_hdl, + V4L2_CID_MPEG_VIDEO_AVERAGE_QP); + if (avg_qp) + v4l2_ctrl_s_ctrl(avg_qp, qp_avg); + } else if (ctx->rc_active && !ctx->overflow_retried && + (int_sta & RKVENC_INT_ERRORS) == RKVENC_INT_OFLW) { + /* The job stays running for the framework: the frame is retried */ + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + rkvenc_hw_recover(dev); + rkvenc_rc_overflow(ctx, + dst ? vb2_plane_size(&dst->vb2_buf, 0) : 0, + ctx->cur_intra ? 0 : 1); + ctx->overflow_retried = true; + if (ctx->cur_prepend) + ctx->joined_header_done = false; + dev->job_ctx = NULL; + pm_runtime_put(dev->dev); + rkvenc_m2m_ops.device_run(ctx); + return; + } else { + rkvenc_hw_recover(dev); + rkvenc_gop_rollback(ctx); + ctx->overflow_retried = false; + ctx->consecutive_errors++; + if (ctx->consecutive_errors >= RKVENC_MAX_CONSEC_ERRORS) { + vb2_queue_error(&ctx->m2m_ctx->out_q_ctx.q); + vb2_queue_error(&ctx->m2m_ctx->cap_q_ctx.q); + } + } + + src = v4l2_m2m_next_src_buf(ctx->m2m_ctx); + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); + if (src && dst && + v4l2_m2m_is_last_draining_src_buf(ctx->m2m_ctx, src)) { + dst->flags |= V4L2_BUF_FLAG_LAST; + v4l2_m2m_mark_stopped(ctx->m2m_ctx); + } + + dev->job_ctx = NULL; + pm_runtime_put(dev->dev); + + v4l2_m2m_buf_done_and_job_finish(dev->m2m_dev, ctx->m2m_ctx, state); +} + +static irqreturn_t rkvenc_irq_thread(int irq, void *priv) +{ + struct rkvenc_dev *dev = priv; + struct rkvenc_ctx *ctx; + enum vb2_buffer_state state; + u32 int_sta, bytes, qp_sum; + unsigned long flags; + + /* Whoever clears job_running first, IRQ or watchdog, finishes the job */ + spin_lock_irqsave(&dev->irq_lock, flags); + if (!dev->job_running) { + spin_unlock_irqrestore(&dev->irq_lock, flags); + return IRQ_HANDLED; + } + ctx = dev->job_ctx; + dev->job_running = false; + spin_unlock_irqrestore(&dev->irq_lock, flags); + + cancel_delayed_work(&dev->watchdog); + + rkvenc_hw_status_read(dev, &int_sta, &bytes, &qp_sum); + + if ((int_sta & RKVENC_INT_ENC_DONE) && + !(int_sta & RKVENC_INT_ERRORS)) + state = VB2_BUF_STATE_DONE; + else + state = VB2_BUF_STATE_ERROR; + + rkvenc_finish_job(dev, ctx, state); + return IRQ_HANDLED; +} + +static void rkvenc_watchdog_work(struct work_struct *work) +{ + struct rkvenc_dev *dev = container_of(to_delayed_work(work), + struct rkvenc_dev, watchdog); + struct rkvenc_ctx *ctx; + unsigned long flags; + + spin_lock_irqsave(&dev->irq_lock, flags); + if (!dev->job_running) { + spin_unlock_irqrestore(&dev->irq_lock, flags); + return; + } + ctx = dev->job_ctx; + dev->job_running = false; + spin_unlock_irqrestore(&dev->irq_lock, flags); + + dev_err(dev->dev, "RKVENC encode timed out\n"); + rkvenc_finish_job(dev, ctx, VB2_BUF_STATE_ERROR); +} + +static int rkvenc_runtime_resume(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + return clk_bulk_prepare_enable(rkvenc->num_clocks, rkvenc->clocks); +} + +static int rkvenc_runtime_suspend(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + clk_bulk_disable_unprepare(rkvenc->num_clocks, rkvenc->clocks); + return 0; +} + +static int rkvenc_suspend(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + v4l2_m2m_suspend(rkvenc->m2m_dev); + return pm_runtime_force_suspend(dev); +} + +static int rkvenc_resume(struct device *dev) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + int ret; + + ret = pm_runtime_force_resume(dev); + if (ret) + return ret; + /* Jobs rewrite every register and the references stay in memory */ + v4l2_m2m_resume(rkvenc->m2m_dev); + return 0; +} + +static void rkvenc_init_clocks_resets(struct rkvenc_dev *rkvenc) +{ + static const char *const clk_names[] = { "axi", "ahb", "core" }; + static const char *const rst_names[] = { "axi", "ahb", "core" }; + unsigned int i; + + for (i = 0; i < RKVENC_NUM_CLKS; i++) + rkvenc->clocks[i].id = clk_names[i]; + for (i = 0; i < RKVENC_NUM_RESETS; i++) + rkvenc->resets[i].id = rst_names[i]; + rkvenc->num_clocks = RKVENC_NUM_CLKS; + rkvenc->num_resets = RKVENC_NUM_RESETS; +} + +static int rkvenc_probe(struct platform_device *pdev) +{ + const struct rkvenc_variant *variant; + struct rkvenc_dev *rkvenc; + struct device *dev = &pdev->dev; + u32 ver, pic_size; + int ret, irq; + + variant = of_device_get_match_data(dev); + if (!variant) + return -EINVAL; + + rkvenc = devm_kzalloc(dev, sizeof(*rkvenc), GFP_KERNEL); + if (!rkvenc) + return -ENOMEM; + + platform_set_drvdata(pdev, rkvenc); + rkvenc->dev = dev; + rkvenc->variant = variant; + + spin_lock_init(&rkvenc->irq_lock); + mutex_init(&rkvenc->vdev_lock); + init_completion(&rkvenc->sclr_done); + INIT_DELAYED_WORK(&rkvenc->watchdog, rkvenc_watchdog_work); + + rkvenc->regs = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(rkvenc->regs)) + return PTR_ERR(rkvenc->regs); + + rkvenc_init_clocks_resets(rkvenc); + + ret = devm_clk_bulk_get(dev, rkvenc->num_clocks, rkvenc->clocks); + if (ret) + return ret; + + ret = devm_reset_control_bulk_get_exclusive(dev, rkvenc->num_resets, + rkvenc->resets); + if (ret) + return ret; + + ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32)); + if (ret) { + dev_err(dev, "Could not set DMA mask\n"); + return ret; + } + + ret = vb2_dma_contig_set_max_seg_size(dev, DMA_BIT_MASK(32)); + if (ret) + return ret; + + irq = platform_get_irq(pdev, 0); + if (irq < 0) + return irq; + + ret = devm_request_threaded_irq(dev, irq, rkvenc_irq, + rkvenc_irq_thread, IRQF_ONESHOT, + dev_name(dev), rkvenc); + if (ret) { + dev_err(dev, "Could not request encoder IRQ\n"); + return ret; + } + + pm_runtime_enable(dev); + + ret = pm_runtime_resume_and_get(dev); + if (ret) + return ret; + + /* VERSION tells the 4K encoder from the 1080p one */ + ver = readl(rkvenc->regs + RKVENC_VERSION); + pic_size = (ver >> 12) & 0xf; + switch (pic_size) { + case 0: + rkvenc->max_width = 4096; + rkvenc->max_height = 2304; + break; + case 1: + rkvenc->max_width = 1920; + rkvenc->max_height = 1088; + break; + default: + dev_warn(dev, "unknown VERSION pic_size %u (0x%08x)\n", + pic_size, ver); + rkvenc->max_width = 4096; + rkvenc->max_height = 2304; + break; + } + if (pic_size != 0) + dev_info(dev, "RKVENC VERSION 0x%08x (pic_size %u)\n", + ver, pic_size); + + pm_runtime_put(dev); + + ret = v4l2_device_register(dev, &rkvenc->v4l2_dev); + if (ret) { + dev_err(dev, "Failed to register V4L2 device\n"); + goto err_pm_disable; + } + + rkvenc->m2m_dev = v4l2_m2m_init(&rkvenc_m2m_ops); + if (IS_ERR(rkvenc->m2m_dev)) { + v4l2_err(&rkvenc->v4l2_dev, "Failed to init mem2mem device\n"); + ret = PTR_ERR(rkvenc->m2m_dev); + goto err_unreg_v4l2; + } + + ret = rkvenc_v4l2_init(rkvenc); + if (ret) + goto err_release_m2m; + + return 0; + +err_release_m2m: + v4l2_m2m_release(rkvenc->m2m_dev); +err_unreg_v4l2: + v4l2_device_unregister(&rkvenc->v4l2_dev); +err_pm_disable: + pm_runtime_disable(dev); + return ret; +} + +static void rkvenc_remove(struct platform_device *pdev) +{ + struct rkvenc_dev *rkvenc = platform_get_drvdata(pdev); + + cancel_delayed_work_sync(&rkvenc->watchdog); + rkvenc_v4l2_cleanup(rkvenc); + v4l2_m2m_release(rkvenc->m2m_dev); + v4l2_device_unregister(&rkvenc->v4l2_dev); + pm_runtime_disable(&pdev->dev); +} + +static const struct of_device_id rkvenc_dt_match[] = { + { .compatible = "rockchip,rv1126-rkvenc", .data = &rkvenc_variant_rv1126 }, + { .compatible = "rockchip,rk3568-rkvenc", .data = &rkvenc_variant_rk3568 }, + { /* sentinel */ }, +}; +MODULE_DEVICE_TABLE(of, rkvenc_dt_match); + +static const struct dev_pm_ops rkvenc_pm_ops = { + RUNTIME_PM_OPS(rkvenc_runtime_suspend, rkvenc_runtime_resume, NULL) + SYSTEM_SLEEP_PM_OPS(rkvenc_suspend, rkvenc_resume) +}; + +static struct platform_driver rkvenc_driver = { + .probe = rkvenc_probe, + .remove = rkvenc_remove, + .driver = { + .name = "rockchip-rkvenc", + .of_match_table = rkvenc_dt_match, + .pm = pm_ptr(&rkvenc_pm_ops), + }, +}; + +static int __init rkvenc_init(void) +{ + rkvenc_tables_init(); + + return platform_driver_register(&rkvenc_driver); +} +module_init(rkvenc_init); + +static void __exit rkvenc_exit(void) +{ + platform_driver_unregister(&rkvenc_driver); +} +module_exit(rkvenc_exit); + +MODULE_AUTHOR("Daniel Golle <daniel@makrotopia.org>"); +MODULE_DESCRIPTION("Rockchip RKVENC v1 H.264/HEVC encoder driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h new file mode 100644 index 0000000000000..df5905a329107 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h @@ -0,0 +1,216 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Rockchip RKVENC v1 H.264 / HEVC encoder driver. + * + * struct rkvenc_frame carries no V4L2 types, so that a stateless front end + * can drive the same hardware back end. + */ + +#ifndef RKVENC_H +#define RKVENC_H + +#include <linux/clk.h> +#include <linux/completion.h> +#include <linux/delay.h> +#include <linux/mutex.h> +#include <linux/spinlock.h> +#include <linux/workqueue.h> +#include <linux/reset.h> + +#include <media/v4l2-ctrls.h> +#include <media/v4l2-device.h> +#include <media/v4l2-mem2mem.h> + +#include "rkvenc-vepu541.h" + +#define RKVENC_HDR_MAX 1024 + +#define RKVENC_SLOTS 2 +#define RKVENC_SLOT_BUFS 4 + +#define RKVENC_NUM_CLKS 3 +#define RKVENC_NUM_RESETS 3 + +#define RKVENC_MAX_CONSEC_ERRORS 2 + +struct rkvenc_dev; + +/** + * struct rkvenc_variant - per-SoC integration differences. + * @vepu540: RK3566/RK3568 "vepu540" rather than RV1109/RV1126 "vepu541". + */ +struct rkvenc_variant { + bool vepu540; +}; + +struct rkvenc_dev { + struct v4l2_device v4l2_dev; + struct media_device mdev; + struct video_device vdev; + struct v4l2_m2m_dev *m2m_dev; + struct device *dev; + void __iomem *regs; + + struct clk_bulk_data clocks[RKVENC_NUM_CLKS]; + int num_clocks; + struct reset_control_bulk_data resets[RKVENC_NUM_RESETS]; + int num_resets; + + const struct rkvenc_variant *variant; + + struct rkvenc_img img; + + struct mutex vdev_lock; /* serialises the video node ioctls */ + spinlock_t irq_lock; + struct completion sclr_done; + u32 irq_sta; + struct delayed_work watchdog; + bool job_running; + struct rkvenc_ctx *job_ctx; + u32 max_width; + u32 max_height; +}; + +struct rkvenc_ctrls { + u32 bitrate_mode; + u32 bitrate; + u32 vbv_size; + bool frame_rc_enable; + u32 gop_size; + u32 i_period; + u32 header_mode; + bool prepend_spspps; + bool force_keyframe; + + u8 profile; + u8 level; + u8 tier; + bool cabac; + bool transform_8x8; + u8 lf_mode; + s8 lf_alpha; + s8 lf_beta; + s8 hevc_beta_offset; + s8 hevc_tc_offset; + s8 chroma_qp_offset; + u8 min_qp; + u8 max_qp; + u8 i_frame_qp; + u8 p_frame_qp; + + u8 hevc_max_merge; + bool hevc_sao; + bool hevc_strong_intra; + bool hevc_tmv; + bool hevc_cabac_init; + + u32 framerate; +}; + +struct rkvenc_gop { + u32 frames_since_idr; + u32 idr_pic_id; + u16 frame_num; + u16 poc_lsb; +}; + +struct rkvenc_ctx { + struct v4l2_fh fh; + struct rkvenc_dev *dev; + struct v4l2_m2m_ctx *m2m_ctx; + struct v4l2_ctrl_handler ctrl_hdl; + struct mutex vlock; + + u32 src_fourcc; + u32 coded_fourcc; + enum rkvenc_codec codec; + u32 width; + u32 height; + u32 bytesperline; + u32 src_cfmt; + + struct rkvenc_ctrls c; + + struct rkvenc_gop gop; + struct rkvenc_slot slot[RKVENC_SLOTS]; + struct rkvenc_slot_sizes slot_sz; + void *slot_cpu[RKVENC_SLOTS][RKVENC_SLOT_BUFS]; + bool slots_alloc; + int cur; + int last; + bool last_good_valid; + int last_good; + u32 consecutive_errors; + bool overflow_retried; + + u32 src_num_planes; + + bool joined_header_done; + + s32 rc_buf; /* in bits */ + u32 rc_tavg; + s8 rc_qp[2]; /* indexed by frame type, 0 = I, 1 = P */ + u32 rc_x[2]; + s32 rc_buf_gop; + u32 rc_p_qp_sum; + u32 rc_p_qp_cnt; + bool rc_active; + bool rc_first[2]; + + bool header_pending; + u8 hdr[RKVENC_HDR_MAX]; + u32 hdr_len; + + u32 fps; + u32 blocks; + u32 qp_blocks; + + bool cur_intra; + u8 cur_qp; + u32 cur_prepend; + + struct rkvenc_h264_seq h264_seq; + struct rkvenc_hevc_seq hevc_seq; +}; + +/* rkvenc.c */ +int rkvenc_hw_run(struct rkvenc_dev *dev, const struct rkvenc_frame *f); +void rkvenc_hw_status_read(struct rkvenc_dev *dev, u32 *int_sta, u32 *bytes, + u32 *qp_sum); +void rkvenc_hw_recover(struct rkvenc_dev *dev); +int rkvenc_v4l2_init(struct rkvenc_dev *dev); +void rkvenc_v4l2_cleanup(struct rkvenc_dev *dev); + +/* rkvenc-v4l2.c */ +extern const struct v4l2_m2m_ops rkvenc_m2m_ops; + +/* rkvenc-gop.c */ +void rkvenc_gop_init(struct rkvenc_ctx *ctx); +bool rkvenc_gop_is_idr(struct rkvenc_ctx *ctx); +void rkvenc_gop_commit(struct rkvenc_ctx *ctx); +void rkvenc_gop_rollback(struct rkvenc_ctx *ctx); +void rkvenc_gop_force_keyframe(struct rkvenc_ctx *ctx); + +/* rkvenc-ref.c */ +int rkvenc_ref_alloc(struct rkvenc_ctx *ctx); +void rkvenc_ref_free(struct rkvenc_ctx *ctx); + +/* rkvenc-rc.c */ +void rkvenc_rc_init(struct rkvenc_ctx *ctx); +void rkvenc_rc_pick(struct rkvenc_ctx *ctx, u8 *qp, u8 frame_type); +void rkvenc_rc_update(struct rkvenc_ctx *ctx, u32 bytes, u32 qp_avg, + u8 frame_type); +void rkvenc_rc_overflow(struct rkvenc_ctx *ctx, u32 bytes, u8 frame_type); + +/* rkvenc-h264.c / rkvenc-hevc.c */ +int rkvenc_h264_setup(struct rkvenc_ctx *ctx); +int rkvenc_h264_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size); +void rkvenc_h264_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_h264_pic *pic, bool idr); + +int rkvenc_hevc_setup(struct rkvenc_ctx *ctx); +int rkvenc_hevc_write_headers(struct rkvenc_ctx *ctx, u8 *buf, u32 size); +void rkvenc_hevc_build_pic(struct rkvenc_ctx *ctx, + struct rkvenc_hevc_pic *pic, bool idr); + +#endif /* RKVENC_H */ -- 2.56.0 _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RFC PATCH 3/6] media: rockchip: Add RKVENC v1 video encoder driver 2026-11-14 14:54 ` Daniel Golle (?) @ 2026-11-14 15:27 ` sashiko-bot -1 siblings, 0 replies; 17+ messages in thread From: sashiko-bot @ 2026-11-14 15:27 UTC (permalink / raw) To: Daniel Golle, devicetree, linux-clk; +Cc: robh, conor+dt, media-ci Thank you for your contribution! Sashiko AI review found 22 potential issues to consider: - [High] media: rkvenc: DMA buffer overflow via rkvenc_s_fmt_vid_out() while streaming - [High] media: rkvenc: missing NV12M plane 1 and data_offset check in rkvenc_buf_prepare() - [High] media: rkvenc: divide-by-zero in rkvenc_rc_pick() and 32-bit RC overflows - [High] media: rkvenc: missing plane 1 and data_offset checks in rkvenc_buf_prepare() - [High] media: rkvenc: premature watchdog cancel in rkvenc_remove() before cleanup - [High] media: rkvenc: SCLR_DONE masked by IRQF_ONESHOT in rkvenc_hw_recover() - [High] media: rkvenc: watchdog cancel race in rkvenc_irq_thread() and device_run() - [High] media: rockchip: rkvenc: job_running races in IRQ, watchdog, and device_run() - [Medium] media: rkvenc: unreachable I-frame RC model and broken IDR overflow retry - [Medium] media: rkvenc: VB2_BUF_STATE_DONE in rkvenc_buf_queue() and stale header_mode - [Medium] media: rkvenc: rkvenc_gop_commit() drops concurrent force_keyframe requests - [Medium] media: rkvenc: missing CREATE_BUFS validation in rkvenc_queue_setup() - [Medium] media: rkvenc: rkvenc_apply_ctrls() prevents setting gop_size below 30 - [Medium] media: rkvenc: control handler leak on error in rkvenc_init_ctrls() - [Medium] media: rkvenc: missing pm_runtime_disable() on error in rkvenc_probe() - [Medium] media: rkvenc: rkvenc_try_fmt_vid_cap() ignores active OUTPUT dimensions - [Medium] media: rockchip: rkvenc: userspace control/format values trigger WARN_ON() - [Medium] media: rockchip: rkvenc: broken control updates and IDR header refresh in v4l2 - [Medium] media: rockchip: rkvenc: missing pm_runtime_disable() on probe error path - [Medium] media: rockchip: rkvenc: watchdog canceled before V4L2/M2M teardown in remove() - [Low] media: rkvenc: HEVC merge candidate 0 overridden in rkvenc_hevc_setup() - [Low] media: rkvenc: CAPTURE format mismatch and round_up() overflow in try_fmt -- commit aafc130d19efda3692073c7ce74ebbd170ec8aca Author: Daniel Golle <daniel@makrotopia.org> media: rockchip: Add RKVENC v1 video encoder driver This commit adds a stateful V4L2 memory-to-memory H.264 and HEVC video encoder driver for the Rockchip RKVENC v1 IP on RV1109/RV1126 and RK3566/RK3568 SoCs. > diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c > new file mode 100644 > index 0000000000000..208bb122a3f87 > --- /dev/null > +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-gop.c > @@ -0,0 +1,72 @@ [ ... ] > +void rkvenc_gop_commit(struct rkvenc_ctx *ctx) > +{ > + struct rkvenc_gop *g = &ctx->gop; > + > + if (ctx->cur_intra) { > + g->idr_pic_id++; > + ctx->last_good = ctx->cur; > + ctx->last_good_valid = true; > + g->frames_since_idr = 1; > + } else { > + g->frames_since_idr++; > + } > + ctx->last = ctx->cur; > + ctx->cur ^= 1; > + ctx->c.force_keyframe = false; > +} > + > +void rkvenc_gop_rollback(struct rkvenc_ctx *ctx) > +{ > + ctx->c.force_keyframe = false; > +} > + > +void rkvenc_gop_force_keyframe(struct rkvenc_ctx *ctx) > +{ > + ctx->c.force_keyframe = true; > +} [Severity: Medium] Does clearing ctx->c.force_keyframe unconditionally in rkvenc_gop_commit() and rkvenc_gop_rollback() drop keyframe requests that arrive while a frame is in flight? CPU0 CPU1 rkvenc_device_run() idr = rkvenc_gop_is_idr(ctx) /* false: starts a P frame */ rkvenc_s_ctrl() rkvenc_gop_force_keyframe(ctx) ctx->c.force_keyframe = true rkvenc_irq_thread() rkvenc_finish_job() rkvenc_gop_commit(ctx) ctx->c.force_keyframe = false <-- request lost before next frame Consuming ctx->c.force_keyframe when rkvenc_device_run() decides idr = true (or only clearing it when committing an intra frame) would preserve requests made while a P frame is encoding. > diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c > new file mode 100644 > index 0000000000000..49bf3cccdf8a0 > --- /dev/null > +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-hevc.c > @@ -0,0 +1,63 @@ [ ... ] > +int rkvenc_hevc_setup(struct rkvenc_ctx *ctx) > +{ > + struct rkvenc_hevc_seq *seq = &ctx->hevc_seq; > + > + memset(seq, 0, sizeof(*seq)); > + seq->width = ctx->width; > + seq->height = ctx->height; > + seq->level_idc = rkvenc_hevc_level(ctx->c.level); > + seq->log2_max_poc_lsb_m4 = 4; > + seq->pic_init_qp = ctx->c.p_frame_qp; > + seq->max_merge_cand = ctx->c.hevc_max_merge ?: 2; [Severity: Low] Does using ?: 2 here prevent setting V4L2_CID_MPEG_VIDEO_HEVC_MAX_NUM_MERGE_MV_MINUS1 to 0? rkvenc_init_ctrls() registers that control with a minimum of 0, so a valid control value of 0 is overwritten with 2. Also, V4L2_CID_MPEG_VIDEO_HEVC_TMV_PREDICTION is registered and stored into ctx->c.hevc_tmv in rkvenc_apply_ctrls(), and ctx->vlock is initialized in rkvenc_open(), but neither is used anywhere in the driver. [ ... ] > diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c > new file mode 100644 > index 0000000000000..b7ab57e490cf0 > --- /dev/null > +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-rc.c > @@ -0,0 +1,214 @@ [ ... ] > +static u32 rkvenc_rc_capacity(struct rkvenc_ctx *ctx) > +{ > + if (ctx->c.vbv_size) > + return ctx->c.vbv_size; > + return ctx->rc_tavg * max_t(u32, ctx->c.gop_size, 1); > +} [ ... ] > + if (ctx->rc_x[0] && ctx->rc_x[1]) { > + u64 w = div_u64((u64)ctx->rc_x[0] * 256, > + ctx->rc_x[1] ?: 1); > + u32 n = max_t(u32, ctx->c.gop_size, 1); > + u64 t_p, t_t; > + > + w = min_t(u64, w, RKVENC_RC_W_CLAMP); > + t_p = div64_u64((u64)ctx->rc_tavg * n * 256, > + w + 256 * (n - 1)); > + t_t = t ? t_p : t_p * w / 256; > + t_t = rkvenc_rc_correct(ctx, t_t, n); > + > + target = (u32)div_u64((u64)ctx->rc_x[t] * 256, t_t); > + > + for (q = 0; q <= RKVENC_QP_MAX; q++) > + if (rkvenc_qs[q] >= target) > + break; > + if (q > RKVENC_QP_MAX) > + q = RKVENC_QP_MAX; > + } else { > + q = prev; > + } [ ... ] > +void rkvenc_rc_update(struct rkvenc_ctx *ctx, u32 bytes, u32 qp_avg, > + u8 frame_type) > +{ > + unsigned int t = frame_type ? 1 : 0; > + u64 s = (u64)bytes * 8; > + s64 cap = max_t(u64, rkvenc_rc_capacity(ctx), 1); > + > + if (!ctx->rc_active) > + return; > + > + ctx->rc_x[t] = rkvenc_rc_complexity(ctx, t, s, qp_avg); > + > + ctx->rc_buf += (s32)s - (s32)ctx->rc_tavg; > + ctx->rc_buf = clamp_t(s64, ctx->rc_buf, -cap, cap); > + > + ctx->rc_first[t] = false; > +} [Severity: High] Can div64_u64() in rkvenc_rc_pick() divide by zero when ctx->c.gop_size is 0? When ctx->c.gop_size is 0 (allowed by the 0..300 control range), n is 1. If a low-complexity, low-QP I frame is followed by a high-complexity, high-QP P frame such that ctx->rc_x[1] > (u64)ctx->rc_x[0] * 256, w truncates to 0 and the divisor w + 256 * (n - 1) becomes 0. In addition, ctx->rc_tavg * max_t(u32, ctx->c.gop_size, 1) in rkvenc_rc_capacity() can overflow u32 at high bitrates and low frame rates (e.g. 40 Mbps at 1 fps with gop_size = 300), and ctx->rc_buf += (s32)s - (s32)ctx->rc_tavg in rkvenc_rc_update() can overflow 32-bit signed arithmetic before being clamped to [-cap, cap]. > +/* The step limit alone would need frames that the overflow keeps dropping */ > +void rkvenc_rc_overflow(struct rkvenc_ctx *ctx, u32 bytes, u8 frame_type) > +{ > + s8 start = frame_type ? ctx->c.p_frame_qp : ctx->c.i_frame_qp; > + unsigned int t = frame_type ? 1 : 0; > + > + rkvenc_rc_update(ctx, bytes, ctx->cur_qp, frame_type); > + if (ctx->rc_qp[t] < start) > + ctx->rc_qp[t] = start; > +} [Severity: Medium] Does the I-frame rate control and overflow retry logic in rkvenc_rc_pick() and rkvenc_rc_overflow() ever adjust QP for IDR frames? In rkvenc_rc_pick(), when t == 0: - if !ctx->rc_first[1] is true, it always takes the rkvenc_rc_p_qp_mean() branch and returns early; - if ctx->rc_first[1] is true, ctx->rc_x[1] is still 0, so if (ctx->rc_x[0] && ctx->rc_x[1]) is false and it falls back to q = prev. As a result, the t == 0 complexity branch in rkvenc_rc_pick() is unreachable, all-Intra streams (gop_size == 1 or intra-only resolutions) stay at the initial i_frame_qp forever, and when an IDR frame overflows the bitstream buffer, the retry in rkvenc_finish_job() ignores the complexity update from rkvenc_rc_overflow() and re-encodes at the same failing QP (or an unadjusted P-frame mean QP). Also, ctx->rc_tavg is only computed in rkvenc_rc_init(), so runtime changes to V4L2_CID_MPEG_VIDEO_BITRATE have no effect. > diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c > new file mode 100644 > index 0000000000000..3be496f5cf0c1 > --- /dev/null > +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c > @@ -0,0 +1,1069 @@ [ ... ] > +/* Runs for every frame, so that control changes apply to the next one */ > +static void rkvenc_apply_ctrls(struct rkvenc_ctx *ctx) > +{ > + struct rkvenc_ctrls *c = &ctx->c; > + bool h264 = ctx->codec == RKVENC_CODEC_H264; > + > + c->bitrate_mode = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_BITRATE_MODE); > + c->bitrate = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_BITRATE); > + c->vbv_size = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_VBV_SIZE); > + c->frame_rc_enable = > + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE); > + c->gop_size = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_GOP_SIZE); > + c->i_period = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_PERIOD); > + c->header_mode = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEADER_MODE); > + c->prepend_spspps = > + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR); > + > + if (h264) { [ ... ] > + c->min_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_MIN_QP); > + c->max_qp = rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_MAX_QP); > + c->i_frame_qp = > + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP); > + c->p_frame_qp = > + rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP); > + if (rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_H264_I_PERIOD) > > + c->gop_size) > + c->gop_size = > + rkvenc_ctrl_val(ctx, > + V4L2_CID_MPEG_VIDEO_H264_I_PERIOD); > + } else { [ ... ] > + if (rkvenc_ctrl_val(ctx, V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD) > > + c->gop_size) > + c->gop_size = > + rkvenc_ctrl_val(ctx, > + V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD); > + } [Severity: Medium] Does taking the maximum of V4L2_CID_MPEG_VIDEO_GOP_SIZE and V4L2_CID_MPEG_VIDEO_H264_I_PERIOD (or V4L2_CID_MPEG_VIDEO_HEVC_REFRESH_PERIOD) prevent setting a GOP size below 30? All three controls default to 30 in rkvenc_init_ctrls(), so if an application lowers only V4L2_CID_MPEG_VIDEO_GOP_SIZE (for example to 1 for all-Intra or 0 for infinite GOP), the untouched codec-specific period control stays at 30 and overrides c->gop_size back to 30. [ ... ] > +static int rkvenc_queue_setup(struct vb2_queue *vq, unsigned int *num_buffers, > + unsigned int *num_planes, unsigned int sizes[], > + struct device *alloc_devs[]) > +{ > + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq); > + const struct rkvenc_fmt *fmt; > + unsigned int i; > + u32 size; > + > + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { > + fmt = rkvenc_find_fmt(rkvenc_output_fmts, > + ARRAY_SIZE(rkvenc_output_fmts), > + ctx->src_fourcc); > + if (!fmt) > + return -EINVAL; > + *num_planes = fmt->num_planes; > + for (i = 0; i < *num_planes; i++) > + sizes[i] = rkvenc_output_plane_size(ctx, i); > + return 0; > + } > + > + fmt = rkvenc_find_fmt(rkvenc_capture_fmts, > + ARRAY_SIZE(rkvenc_capture_fmts), > + ctx->coded_fourcc); > + if (!fmt) > + return -EINVAL; > + size = rkvenc_capture_size(ctx); > + *num_planes = 1; > + if (*num_buffers) > + sizes[0] = max(sizes[0], size); > + else > + sizes[0] = size; > + return 0; > +} [Severity: Medium] Does rkvenc_queue_setup() validate *num_planes and sizes[] when called from VIDIOC_CREATE_BUFS? When *num_planes != 0, the vb2 queue_setup callback is expected to return -EINVAL if *num_planes does not match the format or if any sizes[i] is smaller than the minimum plane size. Here, the OUTPUT branch unconditionally overwrites *num_planes and sizes[i], and the CAPTURE branch checks *num_buffers instead of *num_planes and clamps sizes[0] instead of rejecting invalid plane counts or undersized buffers. > +static int rkvenc_buf_prepare(struct vb2_buffer *vb) > +{ > + struct vb2_queue *vq = vb->vb2_queue; > + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vq); > + > + if (V4L2_TYPE_IS_OUTPUT(vq->type)) { > + if (vb2_plane_size(vb, 0) < rkvenc_output_plane_size(ctx, 0)) > + return -EINVAL; > + } else { > + if (vb2_plane_size(vb, 0) < rkvenc_capture_size(ctx)) > + return -EINVAL; > + } > + return 0; > +} [Severity: High] When V4L2_PIX_FMT_NV12M (ctx->src_num_planes == 2) is used, can a queued buffer with an undersized plane 1 or a large planes[1].data_offset cause an out-of-bounds DMA read? rkvenc_buf_prepare() only checks vb2_plane_size(vb, 0) and does not check plane 1 or account for planes[1].data_offset before rkvenc_device_run() adds data_offset to the plane 1 DMA address. > +static void rkvenc_buf_queue(struct vb2_buffer *vb) > +{ > + struct rkvenc_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue); > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); > + > + if (V4L2_TYPE_IS_CAPTURE(vb->vb2_queue->type)) { > + if (vb2_is_streaming(vb->vb2_queue) && > + v4l2_m2m_dst_buf_is_last(ctx->m2m_ctx)) { > + vb2_set_plane_payload(vb, 0, 0); > + vbuf->field = V4L2_FIELD_NONE; > + v4l2_m2m_last_buffer_done(ctx->m2m_ctx, vbuf); > + return; > + } > + > + /* The parameter sets take a CAPTURE buffer of their own */ > + if (ctx->header_pending && > + ctx->c.header_mode == > + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) { > + u8 *vaddr = vb2_plane_vaddr(vb, 0); > + > + if (vaddr) { > + rkvenc_ensure_header(ctx); > + if (rkvenc_copy_headers(ctx, vaddr, > + vb2_plane_size(vb, 0)) < 0) > + vb2_set_plane_payload(vb, 0, 0); > + else > + vb2_set_plane_payload(vb, 0, > + ctx->hdr_len); > + } else { > + vb2_set_plane_payload(vb, 0, 0); > + } > + vbuf->field = V4L2_FIELD_NONE; > + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); > + ctx->header_pending = false; > + return; > + } > + } > + > + v4l2_m2m_buf_queue(ctx->m2m_ctx, vbuf); > +} [Severity: Medium] Can completing the separate-header buffer with VB2_BUF_STATE_DONE inside rkvenc_buf_queue() trigger a vb2 warning if rkvenc_start_streaming() subsequently fails? For buffers queued before VIDIOC_STREAMON, vb2_start_streaming() invokes .buf_queue() before .start_streaming(). If rkvenc_buf_queue() finishes a buffer onto q->done_list and rkvenc_start_streaming() then returns an error, rkvenc_return_dst_bufs() cannot reclaim that buffer and vb2_start_streaming() hits WARN_ON(!list_empty(&q->done_list)). In addition, rkvenc_buf_queue() tests ctx->c.header_mode, which is only updated in rkvenc_apply_ctrls() (and starts at 0, i.e. SEPARATE, before the first stream), while rkvenc_s_ctrl() sets ctx->header_pending = true on SEPARATE without updating ctx->c.header_mode or clearing ctx->header_pending when switching back to JOINED_WITH_1ST_FRAME, and rkvenc_start_streaming() does not re-arm ctx->header_pending on stream restart. [ ... ] > +static void rkvenc_device_run(void *priv) > +{ > + struct rkvenc_ctx *ctx = priv; > + struct rkvenc_dev *dev = ctx->dev; > + struct vb2_v4l2_buffer *src, *dst; > + struct rkvenc_frame f = {}; > + u8 qp; > + bool idr; > + u32 hdr_len = 0; > + u8 *vaddr; > + int ret; > + > + dev->job_ctx = ctx; > + dev->job_running = true; [Severity: High] Can setting dev->job_ctx and dev->job_running = true here without holding dev->irq_lock race with an already-dispatched rkvenc_watchdog_work()? CPU0 (rkvenc_irq_thread) CPU1 (kworker) spin_lock(&dev->irq_lock) dev->job_running = false spin_unlock(&dev->irq_lock) cancel_delayed_work(&watchdog) rkvenc_watchdog_work() rkvenc_finish_job() rkvenc_device_run() dev->job_ctx = ctx dev->job_running = true spin_lock(&dev->irq_lock) /* sees job_running == true! */ dev->job_running = false spin_unlock(&dev->irq_lock) rkvenc_finish_job() <-- aborts new job pm_runtime_resume_and_get() Because rkvenc_irq_thread() uses non-synchronous cancel_delayed_work(), rkvenc_watchdog_work() can already be running on another CPU and mistake the newly starting job for the timed-out one before pm_runtime_resume_and_get() and rkvenc_hw_run() have even run, accessing MMIO registers while clocks are disabled and underflowing the runtime PM count. > + src = v4l2_m2m_next_src_buf(ctx->m2m_ctx); > + dst = v4l2_m2m_next_dst_buf(ctx->m2m_ctx); > + if (!src || !dst) > + goto finish_error; > + > + idr = rkvenc_gop_is_idr(ctx); > + > + if (idr) > + ctx->gop.frames_since_idr = 0; > + > + rkvenc_apply_ctrls(ctx); > + rkvenc_rc_pick(ctx, &qp, idr ? 0 : 1); > + ctx->cur_qp = qp; > + ctx->cur_intra = idr; > + > + if (idr) > + rkvenc_refresh_header(ctx); [Severity: Medium] Does calling rkvenc_refresh_header(ctx) on every IDR frame desynchronize the slice syntax registers from the bitstream parameter sets when new headers are not emitted? rkvenc_refresh_header() overwrites ctx->h264_seq or ctx->hevc_seq (which feed the hardware slice registers) from the current V4L2 controls on every IDR picture, even when V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE is used or when V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME is used without V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR. Also, as noted earlier, rkvenc_apply_ctrls() overrides gop_size < 30 with the default 30 of H264_I_PERIOD / HEVC_REFRESH_PERIOD, does not update ctx->rc_tavg when V4L2_CID_MPEG_VIDEO_BITRATE changes, and rkvenc_hevc_setup() replaces a hevc_max_merge value of 0 with 2. [ ... ] > + f.src.plane[0] = lower_32_bits( > + vb2_dma_contig_plane_dma_addr(&src->vb2_buf, 0)); > + if (ctx->src_num_planes == 1) > + f.src.plane[1] = f.src.plane[0] + ctx->bytesperline * ctx->height; > + else > + f.src.plane[1] = lower_32_bits( > + vb2_dma_contig_plane_dma_addr(&src->vb2_buf, 1) + > + src->vb2_buf.planes[1].data_offset); [Severity: High] Can src->vb2_buf.planes[1].data_offset push f.src.plane[1] past the end of the plane 1 DMA buffer? Because rkvenc_buf_prepare() never checks that vb2_plane_size(vb, 1) - vb->planes[1].data_offset is at least rkvenc_output_plane_size(ctx, 1), queueing a 2-plane V4L2_PIX_FMT_NV12M buffer with a non-zero data_offset or an undersized second plane programs RKVENC_ADR_SRC1 to DMA-read up to ctx->bytesperline * ctx->height / 2 bytes out of bounds. [ ... ] > + v4l2_ctrl_new_std(hdl, &rkvenc_ctrl_ops, > + V4L2_CID_MPEG_VIDEO_AVERAGE_QP, 0, 51, 1, 0); > + > + if (hdl->error) > + return hdl->error; [Severity: Medium] Does returning hdl->error here without calling v4l2_ctrl_handler_free(hdl) leak the control handler and any controls allocated before the failure? rkvenc_open()'s err_m2m path also does not free ctx->ctrl_hdl when rkvenc_init_ctrls() fails. [ ... ] > +static void rkvenc_try_fmt_vid_out(struct rkvenc_ctx *ctx, struct v4l2_format *f) > +{ > + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; > + const struct rkvenc_fmt *fmt; > + u32 bpl; > + > + fmt = rkvenc_find_fmt(rkvenc_output_fmts, > + ARRAY_SIZE(rkvenc_output_fmts), > + pix->pixelformat); > + if (!fmt) { > + fmt = &rkvenc_output_fmts[0]; > + pix->pixelformat = fmt->fourcc; > + } > + > + pix->field = V4L2_FIELD_NONE; > + pix->width = clamp(round_up(pix->width, 2), 16, ctx->dev->max_width); > + pix->height = clamp(round_up(pix->height, 2), 16, ctx->dev->max_height); [Severity: Low] Can round_up(pix->width, 2) overflow u32 before clamp() if userspace passes 0xffffffff? Wrapping to 0 clamps the dimension to 16 instead of ctx->dev->max_width (and likewise in rkvenc_try_fmt_vid_cap()). Also, rkvenc_start_streaming() does not call v4l2_m2m_update_start_streaming_state() to pair with v4l2_m2m_update_stop_streaming_state() in rkvenc_stop_streaming(). [ ... ] > +static void rkvenc_try_fmt_vid_cap(struct rkvenc_ctx *ctx, struct v4l2_format *f) > +{ > + struct v4l2_pix_format_mplane *pix = &f->fmt.pix_mp; > + const struct rkvenc_fmt *fmt; > + > + fmt = rkvenc_find_fmt(rkvenc_capture_fmts, > + ARRAY_SIZE(rkvenc_capture_fmts), > + pix->pixelformat); > + if (!fmt) { > + fmt = &rkvenc_capture_fmts[0]; > + pix->pixelformat = fmt->fourcc; > + } > + > + pix->field = V4L2_FIELD_NONE; > + pix->width = clamp(round_up(pix->width, 2), 16, ctx->dev->max_width); > + pix->height = clamp(round_up(pix->height, 2), 16, ctx->dev->max_height); > + pix->num_planes = 1; > + pix->plane_fmt[0].bytesperline = 0; > + > + if (fmt->fourcc == V4L2_PIX_FMT_H264) { > + u32 mbw = ALIGN(pix->width, 16) / 16; > + u32 mbh = ALIGN(pix->height, 16) / 16; > + > + pix->plane_fmt[0].sizeimage = > + min(mbw * mbh * 400, pix->width * pix->height / 2) + > + RKVENC_HDR_MAX; > + } else { > + pix->plane_fmt[0].sizeimage = > + pix->width * pix->height / 2 + RKVENC_HDR_MAX; > + } > +} [Severity: Medium] Should rkvenc_try_fmt_vid_cap() derive pix->width, pix->height, and pix->plane_fmt[0].sizeimage from ctx->width and ctx->height instead of the caller-supplied f->fmt.pix_mp? Because rkvenc_s_fmt_vid_cap() does not update ctx->width or ctx->height, calling VIDIOC_TRY_FMT or VIDIOC_S_FMT on the CAPTURE queue with different dimensions (such as 0x0 when selecting the coded format) returns a width, height, and sizeimage that disagree with VIDIOC_G_FMT(CAPTURE), rkvenc_queue_setup(), and rkvenc_buf_prepare(). [ ... ] > +static int rkvenc_s_fmt_vid_out(struct file *file, void *priv, > + struct v4l2_format *f) > +{ > + struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); > + struct vb2_queue *vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type); > + > + if (vb2_is_busy(vq)) > + return -EBUSY; > + > + rkvenc_try_fmt_vid_out(ctx, f); > + ctx->src_fourcc = f->fmt.pix_mp.pixelformat; > + ctx->src_num_planes = f->fmt.pix_mp.num_planes; > + ctx->width = f->fmt.pix_mp.width; > + ctx->height = f->fmt.pix_mp.height; > + ctx->bytesperline = f->fmt.pix_mp.plane_fmt[0].bytesperline; > + return 0; > +} [Severity: High] Can changing the OUTPUT format while the CAPTURE queue is already streaming overflow the reconstruction and reference DMA buffers in ctx->slot[]? rkvenc_start_streaming() only allocates ctx->slot[] (via rkvenc_ref_alloc()) and validates dimensions when the CAPTURE queue starts streaming, while returning 0 immediately for the OUTPUT queue. Because rkvenc_s_fmt_vid_out() only checks vb2_is_busy() on the OUTPUT queue, userspace can start CAPTURE streaming at a small resolution (e.g. 16x16), then call VIDIOC_S_FMT(OUTPUT) with a larger resolution (up to 4096x2304) and start OUTPUT streaming, causing the hardware encoder to DMA-write past the end of the undersized ctx->slot[] buffers. [ ... ] > diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c > new file mode 100644 > index 0000000000000..00778e97686d7 > --- /dev/null > +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c > @@ -0,0 +1,489 @@ [ ... ] > +int rkvenc_hw_run(struct rkvenc_dev *dev, const struct rkvenc_frame *f) > +{ > + struct rkvenc_img *img = &dev->img; > + void __iomem *regs = dev->regs; > + unsigned int a; > + int ret; > + > + ret = rkvenc_vepu541_img_build(f, img); > + if (WARN_ON(ret)) > + return ret; [Severity: Medium] Can userspace trigger this WARN_ON(ret) via standard V4L2 controls or format changes? V4L2_CID_MPEG_VIDEO_H264_MIN_QP / MAX_QP (and the HEVC equivalents) are independent controls with range 1..51 and no check enforcing min_qp <= max_qp. Setting min_qp > max_qp passes inverted bounds to clamp_t() in rkvenc_rc_pick() and causes rkvenc_frame_check() inside rkvenc_vepu541_img_build() to return -EINVAL and hit WARN_ON(ret) on every frame (as does setting a narrow H.264 resolution with width <= 32 and height > 16 on OUTPUT after CAPTURE is already streaming). [ ... ] > +void rkvenc_hw_recover(struct rkvenc_dev *dev) > +{ > + void __iomem *regs = dev->regs; > + > + if (rkvenc_hw_force_clear(dev)) > + return; > + > + dev_err(dev->dev, "job did not stop on force clear, resetting\n"); > + > + writel(RKVENC_INT_ALL & ~RKVENC_INT_SCLR_DONE, regs + RKVENC_INT_MSK); > + writel(RKVENC_INT_ALL, regs + RKVENC_INT_CLR); > + > + reinit_completion(&dev->sclr_done); > + writel(RKVENC_ENC_CLR_SAFE, regs + RKVENC_ENC_CLR); > + wait_for_completion_timeout(&dev->sclr_done, > + msecs_to_jiffies(RKVENC_SAFE_CLEAR_MS)); > + > + if (dev->num_resets) { > + reset_control_bulk_assert(dev->num_resets, dev->resets); > + udelay(RKVENC_RESET_PULSE_US); > + reset_control_bulk_deassert(dev->num_resets, dev->resets); > + } > + > + writel(0, regs + RKVENC_INT_MSK); > +} > + > +static irqreturn_t rkvenc_irq(int irq, void *priv) > +{ > + struct rkvenc_dev *dev = priv; > + u32 sta; > + > + sta = readl(dev->regs + RKVENC_INT_STA); > + if (!sta) > + return IRQ_NONE; > + > + writel(sta, dev->regs + RKVENC_INT_CLR); > + /* Flush the posted clear before the interrupt line is sampled again */ > + readl(dev->regs + RKVENC_INT_STA); > + > + if (sta & RKVENC_INT_SCLR_DONE) > + complete(&dev->sclr_done); > + > + dev->irq_sta = sta; > + return IRQ_WAKE_THREAD; > +} [Severity: High] Can dev->sclr_done ever be completed by rkvenc_irq() when rkvenc_hw_recover() is called from rkvenc_irq_thread()? Because the IRQ is requested with IRQF_ONESHOT, the interrupt line remains masked at the interrupt controller for the entire duration of rkvenc_irq_thread(). When rkvenc_irq_thread() -> rkvenc_finish_job() -> rkvenc_hw_recover() waits on dev->sclr_done, rkvenc_irq() cannot run until rkvenc_irq_thread() exits, so wait_for_completion_timeout() always stalls for the full 100 ms timeout. Furthermore, when RKVENC_INT_SCLR_DONE does fire in rkvenc_irq(), it overwrites dev->irq_sta = sta and returns IRQ_WAKE_THREAD instead of IRQ_HANDLED, queuing a spurious run of rkvenc_irq_thread(). [ ... ] > +static irqreturn_t rkvenc_irq_thread(int irq, void *priv) > +{ > + struct rkvenc_dev *dev = priv; > + struct rkvenc_ctx *ctx; > + enum vb2_buffer_state state; > + u32 int_sta, bytes, qp_sum; > + unsigned long flags; > + > + /* Whoever clears job_running first, IRQ or watchdog, finishes the job */ > + spin_lock_irqsave(&dev->irq_lock, flags); > + if (!dev->job_running) { > + spin_unlock_irqrestore(&dev->irq_lock, flags); > + return IRQ_HANDLED; > + } > + ctx = dev->job_ctx; > + dev->job_running = false; > + spin_unlock_irqrestore(&dev->irq_lock, flags); > + > + cancel_delayed_work(&dev->watchdog); [Severity: High] Can a spurious IRQ thread wake-up from RKVENC_INT_SCLR_DONE or a fast IRQ before schedule_delayed_work() corrupt the next job's state? <hardirq> rkvenc_irq_thread rkvenc_irq() (SCLR_DONE) dev->irq_sta = SCLR_DONE return IRQ_WAKE_THREAD rkvenc_finish_job() rkvenc_device_run() (retry / next job) dev->job_running = true /* next threaded IRQ run starts */ rkvenc_irq_thread() dev->job_running = false /* ENC_DONE not set in irq_sta */ rkvenc_finish_job(..., ERROR) <-- aborts In addition to the unlocked dev->job_running assignment in rkvenc_device_run() and the non-synchronous cancel_delayed_work() above, rkvenc_hw_run() writes RKVENC_ENC_STRT before calling schedule_delayed_work(&dev->watchdog, ...). If a small frame finishes and rkvenc_irq_thread() calls cancel_delayed_work() before rkvenc_hw_run() calls schedule_delayed_work(), the watchdog timer remains armed after the job completes. [ ... ] > + ret = devm_request_threaded_irq(dev, irq, rkvenc_irq, > + rkvenc_irq_thread, IRQF_ONESHOT, > + dev_name(dev), rkvenc); > + if (ret) { > + dev_err(dev, "Could not request encoder IRQ\n"); > + return ret; > + } [Severity: Medium] Is registering the IRQ with devm_request_threaded_irq() before pm_runtime_enable(dev) safe? Because devm_request_threaded_irq() is called before pm_runtime_enable(dev), devres will also free the IRQ after rkvenc_remove() has already called pm_runtime_disable(&pdev->dev), and an early error return after pm_runtime_enable(dev) leaves runtime PM enabled. > + > + pm_runtime_enable(dev); > + > + ret = pm_runtime_resume_and_get(dev); > + if (ret) > + return ret; [Severity: Medium] Does returning directly when pm_runtime_resume_and_get(dev) fails leave runtime PM enabled on the device? Jumping to err_pm_disable would balance pm_runtime_enable(dev). [ ... ] > +static void rkvenc_remove(struct platform_device *pdev) > +{ > + struct rkvenc_dev *rkvenc = platform_get_drvdata(pdev); > + > + cancel_delayed_work_sync(&rkvenc->watchdog); [Severity: High] Can rkvenc->watchdog be re-armed after cancel_delayed_work_sync() returns in rkvenc_remove()? CPU0 CPU1 rkvenc_remove() cancel_delayed_work_sync(&watchdog) rkvenc_device_run() / retry rkvenc_hw_run() schedule_delayed_work(&watchdog, ...) rkvenc_v4l2_cleanup() ... /* devm frees rkvenc */ rkvenc_watchdog_work() <-- use-after-free Because cancel_delayed_work_sync() runs before rkvenc_v4l2_cleanup() and v4l2_m2m_release(), a new job or an in-flight overflow retry in rkvenc_finish_job() can call rkvenc_hw_run() and re-schedule rkvenc->watchdog before the V4L2 and M2M devices are torn down. > + rkvenc_v4l2_cleanup(rkvenc); > + v4l2_m2m_release(rkvenc->m2m_dev); > + v4l2_device_unregister(&rkvenc->v4l2_dev); > + pm_runtime_disable(&pdev->dev); > +} [Severity: Medium] Should cancel_delayed_work_sync(&rkvenc->watchdog) (and stopping any active IRQ/retry activity) happen after rkvenc_v4l2_cleanup() and v4l2_m2m_release() have stopped new M2M jobs from starting, so that teardown cannot leave rkvenc->watchdog queued when devres frees struct rkvenc_dev? -- Sashiko AI review · https://sashiko.dev/#/patchset/cover.1794666330.git.daniel@makrotopia.org?part=3 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [RFC PATCH 4/6] media: rockchip: rkvenc: Add frequency scaling and cooling 2026-11-14 14:54 ` Daniel Golle @ 2026-11-14 14:55 ` Daniel Golle -1 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:55 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel The encoder core runs at the fixed 396 MHz of the clock tree setup, which is a little short of a 4K stream at 30 frames per second. Register the core clock with devfreq. The simple_ondemand governor takes the busy time of the jobs as load and runs once per round of the streaming contexts: a job end runs it when the context that opened the round completes again, so the load covers one job of every context and the clock settles at the lowest point that sustains all of them together. No polling interval is involved. The clock changes only while no job is on the hardware: a job start waits for a clock change in progress and a clock change waits for the running job. A venc-supply in the node is scaled with the operating points. Register a cooling device on top. Its first states lower the highest allowed operating point through a PM QoS frequency limit, one point per state. The remaining states stretch the period between jobs so that the busy time stays at 75, 50 and 25 percent of it, which limits the frame rate once the lowest point is reached. The cooling device, the QoS request and the devfreq device are removed from remove(), before the driver data goes away, since their callbacks use it. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- .../media/platform/rockchip/rkvenc/Kconfig | 3 + .../media/platform/rockchip/rkvenc/Makefile | 1 + .../platform/rockchip/rkvenc/rkvenc-dvfs.c | 364 ++++++++++++++++++ .../platform/rockchip/rkvenc/rkvenc-v4l2.c | 20 +- .../media/platform/rockchip/rkvenc/rkvenc.c | 11 +- .../media/platform/rockchip/rkvenc/rkvenc.h | 47 +++ 6 files changed, 443 insertions(+), 3 deletions(-) create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c diff --git a/drivers/media/platform/rockchip/rkvenc/Kconfig b/drivers/media/platform/rockchip/rkvenc/Kconfig index 1d76314a2ea6d..f4dd8d5dec37c 100644 --- a/drivers/media/platform/rockchip/rkvenc/Kconfig +++ b/drivers/media/platform/rockchip/rkvenc/Kconfig @@ -3,7 +3,10 @@ config VIDEO_ROCKCHIP_RKVENC tristate "Rockchip RKVENC (vepu540/vepu541) H.264/HEVC encoder" depends on ARCH_ROCKCHIP || COMPILE_TEST depends on VIDEO_DEV + select DEVFREQ_GOV_SIMPLE_ONDEMAND select MEDIA_CONTROLLER + select PM_DEVFREQ + select PM_OPP select VIDEOBUF2_DMA_CONTIG select V4L2_MEM2MEM_DEV help diff --git a/drivers/media/platform/rockchip/rkvenc/Makefile b/drivers/media/platform/rockchip/rkvenc/Makefile index 026c81aec492c..20987b293bded 100644 --- a/drivers/media/platform/rockchip/rkvenc/Makefile +++ b/drivers/media/platform/rockchip/rkvenc/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only rockchip-rkvenc-y += rkvenc.o \ + rkvenc-dvfs.o \ rkvenc-v4l2.o \ rkvenc-gop.o \ rkvenc-ref.o \ diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c new file mode 100644 index 0000000000000..8b59122d5d983 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c @@ -0,0 +1,364 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Frequency scaling and cooling of the RKVENC v1 encoder. devfreq is + * evaluated once per round of the streaming contexts, so no polling + * interval is involved. Below the lowest operating point, cooling + * stretches the period between jobs. + */ + +#include <linux/devfreq.h> +#include <linux/ktime.h> +#include <linux/math64.h> +#include <linux/pm_opp.h> +#include <linux/pm_qos.h> +#include <linux/thermal.h> +#include <linux/units.h> + +#include "rkvenc.h" + +#define RKVENC_THROTTLE_STEPS 3 +#define RKVENC_THROTTLE_STEP_PCT 25 + +static unsigned long rkvenc_dvfs_us(ktime_t t) +{ + return min_t(u64, ktime_to_us(t), ULONG_MAX); +} + +static int rkvenc_dvfs_target(struct device *dev, unsigned long *freq, + u32 flags) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct dev_pm_opp *opp; + int ret; + + opp = devfreq_recommended_opp(dev, freq, flags); + if (IS_ERR(opp)) + return PTR_ERR(opp); + + *freq = dev_pm_opp_get_freq(opp); + dev_pm_opp_put(opp); + + mutex_lock(&dvfs->freq_lock); + wait_event(dvfs->idle_wq, !READ_ONCE(dvfs->busy)); + ret = dev_pm_opp_set_rate(dev, *freq); + mutex_unlock(&dvfs->freq_lock); + + return ret; +} + +static int rkvenc_dvfs_cur_freq(struct device *dev, unsigned long *freq) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + *freq = clk_get_rate(rkvenc->clocks[RKVENC_CLK_CORE].clk); + return 0; +} + +static int rkvenc_dvfs_status(struct device *dev, + struct devfreq_dev_status *stat) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags; + ktime_t now, busy; + + spin_lock_irqsave(&dvfs->lock, flags); + now = ktime_get(); + busy = dvfs->busy_total; + if (dvfs->busy) { + busy = ktime_add(busy, ktime_sub(now, dvfs->busy_since)); + dvfs->busy_since = now; + } + stat->total_time = rkvenc_dvfs_us(ktime_sub(now, dvfs->last_eval)); + stat->busy_time = min(rkvenc_dvfs_us(busy), stat->total_time); + dvfs->busy_total = 0; + dvfs->last_eval = now; + spin_unlock_irqrestore(&dvfs->lock, flags); + + rkvenc_dvfs_cur_freq(dev, &stat->current_frequency); + return 0; +} + +void rkvenc_dvfs_job_begin(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags; + + mutex_lock(&dvfs->freq_lock); + spin_lock_irqsave(&dvfs->lock, flags); + dvfs->busy = true; + dvfs->job_start = ktime_get(); + dvfs->busy_since = dvfs->job_start; + spin_unlock_irqrestore(&dvfs->lock, flags); + mutex_unlock(&dvfs->freq_lock); +} + +bool rkvenc_dvfs_start_allowed(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags, delay; + s64 delay_us; + bool allowed; + + spin_lock_irqsave(&dvfs->lock, flags); + delay_us = ktime_to_us(ktime_sub(dvfs->next_start, ktime_get())); + allowed = dvfs->duty_pct >= 100 || delay_us <= 0; + delay = usecs_to_jiffies(clamp_t(s64, delay_us, 0, UINT_MAX)); + if (!allowed) + queue_delayed_work(system_wq, &dvfs->throttle_work, delay); + spin_unlock_irqrestore(&dvfs->lock, flags); + + return allowed; +} + +void rkvenc_dvfs_job_end(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct devfreq *devfreq = dvfs->devfreq; + unsigned long flags; + bool evaluate; + ktime_t now; + + spin_lock_irqsave(&dvfs->lock, flags); + now = ktime_get(); + dvfs->busy_total = + ktime_add(dvfs->busy_total, ktime_sub(now, dvfs->busy_since)); + dvfs->next_start = ktime_add_ns( + dvfs->job_start, + div_u64(ktime_to_ns(ktime_sub(now, dvfs->job_start)) * 100, + dvfs->duty_pct)); + dvfs->busy = false; + /* A round ends with the context that opened it, covering every other one */ + evaluate = dvfs->round_ctx == ctx; + if (!dvfs->round_ctx) + dvfs->round_ctx = ctx; + spin_unlock_irqrestore(&dvfs->lock, flags); + wake_up(&dvfs->idle_wq); + + if (!devfreq || !evaluate) + return; + + mutex_lock(&devfreq->lock); + update_devfreq(devfreq); + mutex_unlock(&devfreq->lock); +} + +static int rkvenc_dvfs_freqs_get(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct device *dev = rkvenc->dev; + struct dev_pm_opp *opp; + unsigned long freq = 0; + int num, i; + + num = dev_pm_opp_get_opp_count(dev); + if (num <= 0) + return num ? num : -ENODEV; + + dvfs->freqs = devm_kcalloc(dev, num, sizeof(*dvfs->freqs), GFP_KERNEL); + if (!dvfs->freqs) + return -ENOMEM; + + for (i = 0; i < num; i++, freq++) { + opp = dev_pm_opp_find_freq_ceil(dev, &freq); + if (IS_ERR(opp)) + return PTR_ERR(opp); + dvfs->freqs[i] = freq; + dev_pm_opp_put(opp); + } + + dvfs->num_freqs = num; + return 0; +} + +static int rkvenc_cdev_max_state(struct thermal_cooling_device *cdev, + unsigned long *state) +{ + struct rkvenc_dvfs *dvfs = cdev->devdata; + + *state = dvfs->num_freqs - 1 + RKVENC_THROTTLE_STEPS; + return 0; +} + +static int rkvenc_cdev_cur_state(struct thermal_cooling_device *cdev, + unsigned long *state) +{ + struct rkvenc_dvfs *dvfs = cdev->devdata; + + *state = dvfs->cooling_state; + return 0; +} + +static int rkvenc_cdev_set_state(struct thermal_cooling_device *cdev, + unsigned long state) +{ + struct rkvenc_dvfs *dvfs = cdev->devdata; + unsigned long idx = min_t(unsigned long, state, dvfs->num_freqs - 1); + unsigned long freq = dvfs->freqs[dvfs->num_freqs - 1 - idx]; + unsigned long flags; + int ret; + + ret = dev_pm_qos_update_request(&dvfs->max_freq_req, freq / HZ_PER_KHZ); + if (ret < 0) + return ret; + + spin_lock_irqsave(&dvfs->lock, flags); + dvfs->duty_pct = 100 - RKVENC_THROTTLE_STEP_PCT * (state - idx); + spin_unlock_irqrestore(&dvfs->lock, flags); + dvfs->cooling_state = state; + + return 0; +} + +static const struct thermal_cooling_device_ops rkvenc_cdev_ops = { + .get_max_state = rkvenc_cdev_max_state, + .get_cur_state = rkvenc_cdev_cur_state, + .set_cur_state = rkvenc_cdev_set_state, +}; + +static void rkvenc_dvfs_throttle_work(struct work_struct *work) +{ + struct rkvenc_dvfs *dvfs = container_of( + to_delayed_work(work), struct rkvenc_dvfs, throttle_work); + struct rkvenc_ctx *ctx; + + mutex_lock(&dvfs->ctx_lock); + list_for_each_entry(ctx, &dvfs->ctx_list, dvfs_node) + v4l2_m2m_try_schedule(ctx->m2m_ctx); + mutex_unlock(&dvfs->ctx_lock); +} + +void rkvenc_dvfs_ctx_add(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + + mutex_lock(&dvfs->ctx_lock); + list_add_tail(&ctx->dvfs_node, &dvfs->ctx_list); + mutex_unlock(&dvfs->ctx_lock); +} + +void rkvenc_dvfs_ctx_idle(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags; + + spin_lock_irqsave(&dvfs->lock, flags); + if (dvfs->round_ctx == ctx) + dvfs->round_ctx = NULL; + spin_unlock_irqrestore(&dvfs->lock, flags); +} + +void rkvenc_dvfs_ctx_del(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + + rkvenc_dvfs_ctx_idle(rkvenc, ctx); + mutex_lock(&dvfs->ctx_lock); + list_del(&ctx->dvfs_node); + mutex_unlock(&dvfs->ctx_lock); +} + +void rkvenc_dvfs_exit(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + + thermal_cooling_device_unregister(dvfs->cdev); + dvfs->cdev = NULL; + + if (dvfs->qos_added) + dev_pm_qos_remove_request(&dvfs->max_freq_req); + dvfs->qos_added = false; + + if (dvfs->devfreq) + devfreq_remove_device(dvfs->devfreq); + dvfs->devfreq = NULL; + + cancel_delayed_work_sync(&dvfs->throttle_work); +} + +static int rkvenc_dvfs_cooling_init(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct device *dev = rkvenc->dev; + int ret; + + if (!of_property_present(dev->of_node, "#cooling-cells")) + return 0; + + ret = dev_pm_qos_add_request(dev, &dvfs->max_freq_req, + DEV_PM_QOS_MAX_FREQUENCY, + PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE); + if (ret < 0) + return ret; + dvfs->qos_added = true; + + dvfs->cdev = thermal_of_cooling_device_register( + dev->of_node, 0, "rkvenc", dvfs, &rkvenc_cdev_ops); + if (IS_ERR(dvfs->cdev)) { + ret = PTR_ERR(dvfs->cdev); + dvfs->cdev = NULL; + return dev_err_probe(dev, ret, + "could not add the cooling device\n"); + } + + return 0; +} + +int rkvenc_dvfs_init(struct rkvenc_dev *rkvenc) +{ + static const char *const clk_names[] = { "core", NULL }; + static const char *const supply_names[] = { "venc", NULL }; + struct dev_pm_opp_config opp_config = { .clk_names = clk_names }; + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct device *dev = rkvenc->dev; + struct devfreq *devfreq; + int ret; + + spin_lock_init(&dvfs->lock); + mutex_init(&dvfs->freq_lock); + mutex_init(&dvfs->ctx_lock); + INIT_LIST_HEAD(&dvfs->ctx_list); + INIT_DELAYED_WORK(&dvfs->throttle_work, rkvenc_dvfs_throttle_work); + init_waitqueue_head(&dvfs->idle_wq); + dvfs->last_eval = ktime_get(); + dvfs->duty_pct = 100; + + if (!of_property_present(dev->of_node, "operating-points-v2")) + return 0; + + if (of_property_present(dev->of_node, "venc-supply")) + opp_config.regulator_names = supply_names; + + ret = devm_pm_opp_set_config(dev, &opp_config); + if (ret) + return ret; + + ret = devm_pm_opp_of_add_table(dev); + if (ret) + return ret; + + ret = rkvenc_dvfs_freqs_get(rkvenc); + if (ret) + return ret; + + dvfs->profile.target = rkvenc_dvfs_target; + dvfs->profile.get_dev_status = rkvenc_dvfs_status; + dvfs->profile.get_cur_freq = rkvenc_dvfs_cur_freq; + dvfs->profile.freq_table = dvfs->freqs; + dvfs->profile.max_state = dvfs->num_freqs; + rkvenc_dvfs_cur_freq(dev, &dvfs->profile.initial_freq); + + devfreq = devfreq_add_device(dev, &dvfs->profile, + DEVFREQ_GOV_SIMPLE_ONDEMAND, NULL); + if (IS_ERR(devfreq)) + return dev_err_probe(dev, PTR_ERR(devfreq), + "could not add the devfreq device\n"); + + dvfs->devfreq = devfreq; + ret = rkvenc_dvfs_cooling_init(rkvenc); + if (ret) + rkvenc_dvfs_exit(rkvenc); + + return ret; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c index 3be496f5cf0c1..8143f79baa551 100644 --- a/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c @@ -351,6 +351,7 @@ static void rkvenc_stop_streaming(struct vb2_queue *q) struct vb2_v4l2_buffer *vbuf; v4l2_m2m_update_stop_streaming_state(ctx->m2m_ctx, q); + rkvenc_dvfs_ctx_idle(ctx->dev, ctx); if (V4L2_TYPE_IS_OUTPUT(q->type)) { while ((vbuf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx))) @@ -457,17 +458,22 @@ static void rkvenc_device_run(void *priv) else rkvenc_hevc_build_pic(ctx, &f.hevc, idr); + rkvenc_dvfs_job_begin(dev); + ret = pm_runtime_resume_and_get(dev->dev); if (ret) - goto finish_error; + goto finish_error_idle; ret = rkvenc_hw_run(dev, &f); if (ret) { pm_runtime_put(dev->dev); - goto finish_error; + goto finish_error_idle; } return; +finish_error_idle: + rkvenc_dvfs_job_end(dev, ctx); + finish_error: dev->job_running = false; dev->job_ctx = NULL; @@ -478,8 +484,16 @@ static void rkvenc_device_run(void *priv) v4l2_m2m_job_finish(dev->m2m_dev, ctx->m2m_ctx); } +static int rkvenc_job_ready(void *priv) +{ + struct rkvenc_ctx *ctx = priv; + + return rkvenc_dvfs_start_allowed(ctx->dev); +} + const struct v4l2_m2m_ops rkvenc_m2m_ops = { .device_run = rkvenc_device_run, + .job_ready = rkvenc_job_ready, }; static int rkvenc_s_ctrl(struct v4l2_ctrl *ctrl) @@ -944,6 +958,7 @@ static int rkvenc_open(struct file *filp) if (ret) goto err_m2m; + rkvenc_dvfs_ctx_add(dev, ctx); v4l2_fh_add(&ctx->fh, filp); return 0; @@ -959,6 +974,7 @@ static int rkvenc_release(struct file *filp) { struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(filp->private_data); + rkvenc_dvfs_ctx_del(ctx->dev, ctx); v4l2_m2m_ctx_release(ctx->m2m_ctx); v4l2_ctrl_handler_free(&ctx->ctrl_hdl); v4l2_fh_del(&ctx->fh, filp); diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c index 00778e97686d7..f15a0922059f1 100644 --- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c @@ -187,6 +187,7 @@ static void rkvenc_finish_job(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx, ctx->joined_header_done = false; dev->job_ctx = NULL; pm_runtime_put(dev->dev); + rkvenc_dvfs_job_end(dev, ctx); rkvenc_m2m_ops.device_run(ctx); return; } else { @@ -210,6 +211,7 @@ static void rkvenc_finish_job(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx, dev->job_ctx = NULL; pm_runtime_put(dev->dev); + rkvenc_dvfs_job_end(dev, ctx); v4l2_m2m_buf_done_and_job_finish(dev->m2m_dev, ctx->m2m_ctx, state); } @@ -422,12 +424,18 @@ static int rkvenc_probe(struct platform_device *pdev) goto err_unreg_v4l2; } - ret = rkvenc_v4l2_init(rkvenc); + ret = rkvenc_dvfs_init(rkvenc); if (ret) goto err_release_m2m; + ret = rkvenc_v4l2_init(rkvenc); + if (ret) + goto err_dvfs; + return 0; +err_dvfs: + rkvenc_dvfs_exit(rkvenc); err_release_m2m: v4l2_m2m_release(rkvenc->m2m_dev); err_unreg_v4l2: @@ -442,6 +450,7 @@ static void rkvenc_remove(struct platform_device *pdev) struct rkvenc_dev *rkvenc = platform_get_drvdata(pdev); cancel_delayed_work_sync(&rkvenc->watchdog); + rkvenc_dvfs_exit(rkvenc); rkvenc_v4l2_cleanup(rkvenc); v4l2_m2m_release(rkvenc->m2m_dev); v4l2_device_unregister(&rkvenc->v4l2_dev); diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h index df5905a329107..01311c31bbd2a 100644 --- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h @@ -12,8 +12,14 @@ #include <linux/clk.h> #include <linux/completion.h> #include <linux/delay.h> +#include <linux/devfreq.h> +#include <linux/ktime.h> +#include <linux/list.h> #include <linux/mutex.h> +#include <linux/pm_qos.h> #include <linux/spinlock.h> +#include <linux/thermal.h> +#include <linux/wait.h> #include <linux/workqueue.h> #include <linux/reset.h> @@ -30,11 +36,40 @@ #define RKVENC_NUM_CLKS 3 #define RKVENC_NUM_RESETS 3 +#define RKVENC_CLK_CORE 2 #define RKVENC_MAX_CONSEC_ERRORS 2 struct rkvenc_dev; +struct rkvenc_dvfs { + struct devfreq *devfreq; + struct devfreq_dev_profile profile; + unsigned long *freqs; + unsigned int num_freqs; + + struct mutex freq_lock; + wait_queue_head_t idle_wq; + + struct thermal_cooling_device *cdev; + struct dev_pm_qos_request max_freq_req; + bool qos_added; + unsigned int cooling_state; + struct delayed_work throttle_work; + struct mutex ctx_lock; /* protects ctx_list */ + struct list_head ctx_list; + + spinlock_t lock; /* protects the fields below */ + bool busy; + ktime_t job_start; + ktime_t busy_since; + ktime_t busy_total; + ktime_t last_eval; + ktime_t next_start; + unsigned int duty_pct; + struct rkvenc_ctx *round_ctx; +}; + /** * struct rkvenc_variant - per-SoC integration differences. * @vepu540: RK3566/RK3568 "vepu540" rather than RV1109/RV1126 "vepu541". @@ -59,6 +94,7 @@ struct rkvenc_dev { const struct rkvenc_variant *variant; struct rkvenc_img img; + struct rkvenc_dvfs dvfs; struct mutex vdev_lock; /* serialises the video node ioctls */ spinlock_t irq_lock; @@ -120,6 +156,7 @@ struct rkvenc_ctx { struct v4l2_m2m_ctx *m2m_ctx; struct v4l2_ctrl_handler ctrl_hdl; struct mutex vlock; + struct list_head dvfs_node; u32 src_fourcc; u32 coded_fourcc; @@ -181,6 +218,16 @@ void rkvenc_hw_recover(struct rkvenc_dev *dev); int rkvenc_v4l2_init(struct rkvenc_dev *dev); void rkvenc_v4l2_cleanup(struct rkvenc_dev *dev); +/* rkvenc-dvfs.c */ +int rkvenc_dvfs_init(struct rkvenc_dev *dev); +void rkvenc_dvfs_exit(struct rkvenc_dev *dev); +void rkvenc_dvfs_job_begin(struct rkvenc_dev *dev); +void rkvenc_dvfs_job_end(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); +bool rkvenc_dvfs_start_allowed(struct rkvenc_dev *dev); +void rkvenc_dvfs_ctx_add(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); +void rkvenc_dvfs_ctx_idle(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); +void rkvenc_dvfs_ctx_del(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); + /* rkvenc-v4l2.c */ extern const struct v4l2_m2m_ops rkvenc_m2m_ops; -- 2.56.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 4/6] media: rockchip: rkvenc: Add frequency scaling and cooling @ 2026-11-14 14:55 ` Daniel Golle 0 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:55 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel The encoder core runs at the fixed 396 MHz of the clock tree setup, which is a little short of a 4K stream at 30 frames per second. Register the core clock with devfreq. The simple_ondemand governor takes the busy time of the jobs as load and runs once per round of the streaming contexts: a job end runs it when the context that opened the round completes again, so the load covers one job of every context and the clock settles at the lowest point that sustains all of them together. No polling interval is involved. The clock changes only while no job is on the hardware: a job start waits for a clock change in progress and a clock change waits for the running job. A venc-supply in the node is scaled with the operating points. Register a cooling device on top. Its first states lower the highest allowed operating point through a PM QoS frequency limit, one point per state. The remaining states stretch the period between jobs so that the busy time stays at 75, 50 and 25 percent of it, which limits the frame rate once the lowest point is reached. The cooling device, the QoS request and the devfreq device are removed from remove(), before the driver data goes away, since their callbacks use it. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- .../media/platform/rockchip/rkvenc/Kconfig | 3 + .../media/platform/rockchip/rkvenc/Makefile | 1 + .../platform/rockchip/rkvenc/rkvenc-dvfs.c | 364 ++++++++++++++++++ .../platform/rockchip/rkvenc/rkvenc-v4l2.c | 20 +- .../media/platform/rockchip/rkvenc/rkvenc.c | 11 +- .../media/platform/rockchip/rkvenc/rkvenc.h | 47 +++ 6 files changed, 443 insertions(+), 3 deletions(-) create mode 100644 drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c diff --git a/drivers/media/platform/rockchip/rkvenc/Kconfig b/drivers/media/platform/rockchip/rkvenc/Kconfig index 1d76314a2ea6d..f4dd8d5dec37c 100644 --- a/drivers/media/platform/rockchip/rkvenc/Kconfig +++ b/drivers/media/platform/rockchip/rkvenc/Kconfig @@ -3,7 +3,10 @@ config VIDEO_ROCKCHIP_RKVENC tristate "Rockchip RKVENC (vepu540/vepu541) H.264/HEVC encoder" depends on ARCH_ROCKCHIP || COMPILE_TEST depends on VIDEO_DEV + select DEVFREQ_GOV_SIMPLE_ONDEMAND select MEDIA_CONTROLLER + select PM_DEVFREQ + select PM_OPP select VIDEOBUF2_DMA_CONTIG select V4L2_MEM2MEM_DEV help diff --git a/drivers/media/platform/rockchip/rkvenc/Makefile b/drivers/media/platform/rockchip/rkvenc/Makefile index 026c81aec492c..20987b293bded 100644 --- a/drivers/media/platform/rockchip/rkvenc/Makefile +++ b/drivers/media/platform/rockchip/rkvenc/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only rockchip-rkvenc-y += rkvenc.o \ + rkvenc-dvfs.o \ rkvenc-v4l2.o \ rkvenc-gop.o \ rkvenc-ref.o \ diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c new file mode 100644 index 0000000000000..8b59122d5d983 --- /dev/null +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c @@ -0,0 +1,364 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Frequency scaling and cooling of the RKVENC v1 encoder. devfreq is + * evaluated once per round of the streaming contexts, so no polling + * interval is involved. Below the lowest operating point, cooling + * stretches the period between jobs. + */ + +#include <linux/devfreq.h> +#include <linux/ktime.h> +#include <linux/math64.h> +#include <linux/pm_opp.h> +#include <linux/pm_qos.h> +#include <linux/thermal.h> +#include <linux/units.h> + +#include "rkvenc.h" + +#define RKVENC_THROTTLE_STEPS 3 +#define RKVENC_THROTTLE_STEP_PCT 25 + +static unsigned long rkvenc_dvfs_us(ktime_t t) +{ + return min_t(u64, ktime_to_us(t), ULONG_MAX); +} + +static int rkvenc_dvfs_target(struct device *dev, unsigned long *freq, + u32 flags) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct dev_pm_opp *opp; + int ret; + + opp = devfreq_recommended_opp(dev, freq, flags); + if (IS_ERR(opp)) + return PTR_ERR(opp); + + *freq = dev_pm_opp_get_freq(opp); + dev_pm_opp_put(opp); + + mutex_lock(&dvfs->freq_lock); + wait_event(dvfs->idle_wq, !READ_ONCE(dvfs->busy)); + ret = dev_pm_opp_set_rate(dev, *freq); + mutex_unlock(&dvfs->freq_lock); + + return ret; +} + +static int rkvenc_dvfs_cur_freq(struct device *dev, unsigned long *freq) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + + *freq = clk_get_rate(rkvenc->clocks[RKVENC_CLK_CORE].clk); + return 0; +} + +static int rkvenc_dvfs_status(struct device *dev, + struct devfreq_dev_status *stat) +{ + struct rkvenc_dev *rkvenc = dev_get_drvdata(dev); + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags; + ktime_t now, busy; + + spin_lock_irqsave(&dvfs->lock, flags); + now = ktime_get(); + busy = dvfs->busy_total; + if (dvfs->busy) { + busy = ktime_add(busy, ktime_sub(now, dvfs->busy_since)); + dvfs->busy_since = now; + } + stat->total_time = rkvenc_dvfs_us(ktime_sub(now, dvfs->last_eval)); + stat->busy_time = min(rkvenc_dvfs_us(busy), stat->total_time); + dvfs->busy_total = 0; + dvfs->last_eval = now; + spin_unlock_irqrestore(&dvfs->lock, flags); + + rkvenc_dvfs_cur_freq(dev, &stat->current_frequency); + return 0; +} + +void rkvenc_dvfs_job_begin(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags; + + mutex_lock(&dvfs->freq_lock); + spin_lock_irqsave(&dvfs->lock, flags); + dvfs->busy = true; + dvfs->job_start = ktime_get(); + dvfs->busy_since = dvfs->job_start; + spin_unlock_irqrestore(&dvfs->lock, flags); + mutex_unlock(&dvfs->freq_lock); +} + +bool rkvenc_dvfs_start_allowed(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags, delay; + s64 delay_us; + bool allowed; + + spin_lock_irqsave(&dvfs->lock, flags); + delay_us = ktime_to_us(ktime_sub(dvfs->next_start, ktime_get())); + allowed = dvfs->duty_pct >= 100 || delay_us <= 0; + delay = usecs_to_jiffies(clamp_t(s64, delay_us, 0, UINT_MAX)); + if (!allowed) + queue_delayed_work(system_wq, &dvfs->throttle_work, delay); + spin_unlock_irqrestore(&dvfs->lock, flags); + + return allowed; +} + +void rkvenc_dvfs_job_end(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct devfreq *devfreq = dvfs->devfreq; + unsigned long flags; + bool evaluate; + ktime_t now; + + spin_lock_irqsave(&dvfs->lock, flags); + now = ktime_get(); + dvfs->busy_total = + ktime_add(dvfs->busy_total, ktime_sub(now, dvfs->busy_since)); + dvfs->next_start = ktime_add_ns( + dvfs->job_start, + div_u64(ktime_to_ns(ktime_sub(now, dvfs->job_start)) * 100, + dvfs->duty_pct)); + dvfs->busy = false; + /* A round ends with the context that opened it, covering every other one */ + evaluate = dvfs->round_ctx == ctx; + if (!dvfs->round_ctx) + dvfs->round_ctx = ctx; + spin_unlock_irqrestore(&dvfs->lock, flags); + wake_up(&dvfs->idle_wq); + + if (!devfreq || !evaluate) + return; + + mutex_lock(&devfreq->lock); + update_devfreq(devfreq); + mutex_unlock(&devfreq->lock); +} + +static int rkvenc_dvfs_freqs_get(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct device *dev = rkvenc->dev; + struct dev_pm_opp *opp; + unsigned long freq = 0; + int num, i; + + num = dev_pm_opp_get_opp_count(dev); + if (num <= 0) + return num ? num : -ENODEV; + + dvfs->freqs = devm_kcalloc(dev, num, sizeof(*dvfs->freqs), GFP_KERNEL); + if (!dvfs->freqs) + return -ENOMEM; + + for (i = 0; i < num; i++, freq++) { + opp = dev_pm_opp_find_freq_ceil(dev, &freq); + if (IS_ERR(opp)) + return PTR_ERR(opp); + dvfs->freqs[i] = freq; + dev_pm_opp_put(opp); + } + + dvfs->num_freqs = num; + return 0; +} + +static int rkvenc_cdev_max_state(struct thermal_cooling_device *cdev, + unsigned long *state) +{ + struct rkvenc_dvfs *dvfs = cdev->devdata; + + *state = dvfs->num_freqs - 1 + RKVENC_THROTTLE_STEPS; + return 0; +} + +static int rkvenc_cdev_cur_state(struct thermal_cooling_device *cdev, + unsigned long *state) +{ + struct rkvenc_dvfs *dvfs = cdev->devdata; + + *state = dvfs->cooling_state; + return 0; +} + +static int rkvenc_cdev_set_state(struct thermal_cooling_device *cdev, + unsigned long state) +{ + struct rkvenc_dvfs *dvfs = cdev->devdata; + unsigned long idx = min_t(unsigned long, state, dvfs->num_freqs - 1); + unsigned long freq = dvfs->freqs[dvfs->num_freqs - 1 - idx]; + unsigned long flags; + int ret; + + ret = dev_pm_qos_update_request(&dvfs->max_freq_req, freq / HZ_PER_KHZ); + if (ret < 0) + return ret; + + spin_lock_irqsave(&dvfs->lock, flags); + dvfs->duty_pct = 100 - RKVENC_THROTTLE_STEP_PCT * (state - idx); + spin_unlock_irqrestore(&dvfs->lock, flags); + dvfs->cooling_state = state; + + return 0; +} + +static const struct thermal_cooling_device_ops rkvenc_cdev_ops = { + .get_max_state = rkvenc_cdev_max_state, + .get_cur_state = rkvenc_cdev_cur_state, + .set_cur_state = rkvenc_cdev_set_state, +}; + +static void rkvenc_dvfs_throttle_work(struct work_struct *work) +{ + struct rkvenc_dvfs *dvfs = container_of( + to_delayed_work(work), struct rkvenc_dvfs, throttle_work); + struct rkvenc_ctx *ctx; + + mutex_lock(&dvfs->ctx_lock); + list_for_each_entry(ctx, &dvfs->ctx_list, dvfs_node) + v4l2_m2m_try_schedule(ctx->m2m_ctx); + mutex_unlock(&dvfs->ctx_lock); +} + +void rkvenc_dvfs_ctx_add(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + + mutex_lock(&dvfs->ctx_lock); + list_add_tail(&ctx->dvfs_node, &dvfs->ctx_list); + mutex_unlock(&dvfs->ctx_lock); +} + +void rkvenc_dvfs_ctx_idle(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + unsigned long flags; + + spin_lock_irqsave(&dvfs->lock, flags); + if (dvfs->round_ctx == ctx) + dvfs->round_ctx = NULL; + spin_unlock_irqrestore(&dvfs->lock, flags); +} + +void rkvenc_dvfs_ctx_del(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + + rkvenc_dvfs_ctx_idle(rkvenc, ctx); + mutex_lock(&dvfs->ctx_lock); + list_del(&ctx->dvfs_node); + mutex_unlock(&dvfs->ctx_lock); +} + +void rkvenc_dvfs_exit(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + + thermal_cooling_device_unregister(dvfs->cdev); + dvfs->cdev = NULL; + + if (dvfs->qos_added) + dev_pm_qos_remove_request(&dvfs->max_freq_req); + dvfs->qos_added = false; + + if (dvfs->devfreq) + devfreq_remove_device(dvfs->devfreq); + dvfs->devfreq = NULL; + + cancel_delayed_work_sync(&dvfs->throttle_work); +} + +static int rkvenc_dvfs_cooling_init(struct rkvenc_dev *rkvenc) +{ + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct device *dev = rkvenc->dev; + int ret; + + if (!of_property_present(dev->of_node, "#cooling-cells")) + return 0; + + ret = dev_pm_qos_add_request(dev, &dvfs->max_freq_req, + DEV_PM_QOS_MAX_FREQUENCY, + PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE); + if (ret < 0) + return ret; + dvfs->qos_added = true; + + dvfs->cdev = thermal_of_cooling_device_register( + dev->of_node, 0, "rkvenc", dvfs, &rkvenc_cdev_ops); + if (IS_ERR(dvfs->cdev)) { + ret = PTR_ERR(dvfs->cdev); + dvfs->cdev = NULL; + return dev_err_probe(dev, ret, + "could not add the cooling device\n"); + } + + return 0; +} + +int rkvenc_dvfs_init(struct rkvenc_dev *rkvenc) +{ + static const char *const clk_names[] = { "core", NULL }; + static const char *const supply_names[] = { "venc", NULL }; + struct dev_pm_opp_config opp_config = { .clk_names = clk_names }; + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; + struct device *dev = rkvenc->dev; + struct devfreq *devfreq; + int ret; + + spin_lock_init(&dvfs->lock); + mutex_init(&dvfs->freq_lock); + mutex_init(&dvfs->ctx_lock); + INIT_LIST_HEAD(&dvfs->ctx_list); + INIT_DELAYED_WORK(&dvfs->throttle_work, rkvenc_dvfs_throttle_work); + init_waitqueue_head(&dvfs->idle_wq); + dvfs->last_eval = ktime_get(); + dvfs->duty_pct = 100; + + if (!of_property_present(dev->of_node, "operating-points-v2")) + return 0; + + if (of_property_present(dev->of_node, "venc-supply")) + opp_config.regulator_names = supply_names; + + ret = devm_pm_opp_set_config(dev, &opp_config); + if (ret) + return ret; + + ret = devm_pm_opp_of_add_table(dev); + if (ret) + return ret; + + ret = rkvenc_dvfs_freqs_get(rkvenc); + if (ret) + return ret; + + dvfs->profile.target = rkvenc_dvfs_target; + dvfs->profile.get_dev_status = rkvenc_dvfs_status; + dvfs->profile.get_cur_freq = rkvenc_dvfs_cur_freq; + dvfs->profile.freq_table = dvfs->freqs; + dvfs->profile.max_state = dvfs->num_freqs; + rkvenc_dvfs_cur_freq(dev, &dvfs->profile.initial_freq); + + devfreq = devfreq_add_device(dev, &dvfs->profile, + DEVFREQ_GOV_SIMPLE_ONDEMAND, NULL); + if (IS_ERR(devfreq)) + return dev_err_probe(dev, PTR_ERR(devfreq), + "could not add the devfreq device\n"); + + dvfs->devfreq = devfreq; + ret = rkvenc_dvfs_cooling_init(rkvenc); + if (ret) + rkvenc_dvfs_exit(rkvenc); + + return ret; +} diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c index 3be496f5cf0c1..8143f79baa551 100644 --- a/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c @@ -351,6 +351,7 @@ static void rkvenc_stop_streaming(struct vb2_queue *q) struct vb2_v4l2_buffer *vbuf; v4l2_m2m_update_stop_streaming_state(ctx->m2m_ctx, q); + rkvenc_dvfs_ctx_idle(ctx->dev, ctx); if (V4L2_TYPE_IS_OUTPUT(q->type)) { while ((vbuf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx))) @@ -457,17 +458,22 @@ static void rkvenc_device_run(void *priv) else rkvenc_hevc_build_pic(ctx, &f.hevc, idr); + rkvenc_dvfs_job_begin(dev); + ret = pm_runtime_resume_and_get(dev->dev); if (ret) - goto finish_error; + goto finish_error_idle; ret = rkvenc_hw_run(dev, &f); if (ret) { pm_runtime_put(dev->dev); - goto finish_error; + goto finish_error_idle; } return; +finish_error_idle: + rkvenc_dvfs_job_end(dev, ctx); + finish_error: dev->job_running = false; dev->job_ctx = NULL; @@ -478,8 +484,16 @@ static void rkvenc_device_run(void *priv) v4l2_m2m_job_finish(dev->m2m_dev, ctx->m2m_ctx); } +static int rkvenc_job_ready(void *priv) +{ + struct rkvenc_ctx *ctx = priv; + + return rkvenc_dvfs_start_allowed(ctx->dev); +} + const struct v4l2_m2m_ops rkvenc_m2m_ops = { .device_run = rkvenc_device_run, + .job_ready = rkvenc_job_ready, }; static int rkvenc_s_ctrl(struct v4l2_ctrl *ctrl) @@ -944,6 +958,7 @@ static int rkvenc_open(struct file *filp) if (ret) goto err_m2m; + rkvenc_dvfs_ctx_add(dev, ctx); v4l2_fh_add(&ctx->fh, filp); return 0; @@ -959,6 +974,7 @@ static int rkvenc_release(struct file *filp) { struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(filp->private_data); + rkvenc_dvfs_ctx_del(ctx->dev, ctx); v4l2_m2m_ctx_release(ctx->m2m_ctx); v4l2_ctrl_handler_free(&ctx->ctrl_hdl); v4l2_fh_del(&ctx->fh, filp); diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c index 00778e97686d7..f15a0922059f1 100644 --- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c @@ -187,6 +187,7 @@ static void rkvenc_finish_job(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx, ctx->joined_header_done = false; dev->job_ctx = NULL; pm_runtime_put(dev->dev); + rkvenc_dvfs_job_end(dev, ctx); rkvenc_m2m_ops.device_run(ctx); return; } else { @@ -210,6 +211,7 @@ static void rkvenc_finish_job(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx, dev->job_ctx = NULL; pm_runtime_put(dev->dev); + rkvenc_dvfs_job_end(dev, ctx); v4l2_m2m_buf_done_and_job_finish(dev->m2m_dev, ctx->m2m_ctx, state); } @@ -422,12 +424,18 @@ static int rkvenc_probe(struct platform_device *pdev) goto err_unreg_v4l2; } - ret = rkvenc_v4l2_init(rkvenc); + ret = rkvenc_dvfs_init(rkvenc); if (ret) goto err_release_m2m; + ret = rkvenc_v4l2_init(rkvenc); + if (ret) + goto err_dvfs; + return 0; +err_dvfs: + rkvenc_dvfs_exit(rkvenc); err_release_m2m: v4l2_m2m_release(rkvenc->m2m_dev); err_unreg_v4l2: @@ -442,6 +450,7 @@ static void rkvenc_remove(struct platform_device *pdev) struct rkvenc_dev *rkvenc = platform_get_drvdata(pdev); cancel_delayed_work_sync(&rkvenc->watchdog); + rkvenc_dvfs_exit(rkvenc); rkvenc_v4l2_cleanup(rkvenc); v4l2_m2m_release(rkvenc->m2m_dev); v4l2_device_unregister(&rkvenc->v4l2_dev); diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.h b/drivers/media/platform/rockchip/rkvenc/rkvenc.h index df5905a329107..01311c31bbd2a 100644 --- a/drivers/media/platform/rockchip/rkvenc/rkvenc.h +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.h @@ -12,8 +12,14 @@ #include <linux/clk.h> #include <linux/completion.h> #include <linux/delay.h> +#include <linux/devfreq.h> +#include <linux/ktime.h> +#include <linux/list.h> #include <linux/mutex.h> +#include <linux/pm_qos.h> #include <linux/spinlock.h> +#include <linux/thermal.h> +#include <linux/wait.h> #include <linux/workqueue.h> #include <linux/reset.h> @@ -30,11 +36,40 @@ #define RKVENC_NUM_CLKS 3 #define RKVENC_NUM_RESETS 3 +#define RKVENC_CLK_CORE 2 #define RKVENC_MAX_CONSEC_ERRORS 2 struct rkvenc_dev; +struct rkvenc_dvfs { + struct devfreq *devfreq; + struct devfreq_dev_profile profile; + unsigned long *freqs; + unsigned int num_freqs; + + struct mutex freq_lock; + wait_queue_head_t idle_wq; + + struct thermal_cooling_device *cdev; + struct dev_pm_qos_request max_freq_req; + bool qos_added; + unsigned int cooling_state; + struct delayed_work throttle_work; + struct mutex ctx_lock; /* protects ctx_list */ + struct list_head ctx_list; + + spinlock_t lock; /* protects the fields below */ + bool busy; + ktime_t job_start; + ktime_t busy_since; + ktime_t busy_total; + ktime_t last_eval; + ktime_t next_start; + unsigned int duty_pct; + struct rkvenc_ctx *round_ctx; +}; + /** * struct rkvenc_variant - per-SoC integration differences. * @vepu540: RK3566/RK3568 "vepu540" rather than RV1109/RV1126 "vepu541". @@ -59,6 +94,7 @@ struct rkvenc_dev { const struct rkvenc_variant *variant; struct rkvenc_img img; + struct rkvenc_dvfs dvfs; struct mutex vdev_lock; /* serialises the video node ioctls */ spinlock_t irq_lock; @@ -120,6 +156,7 @@ struct rkvenc_ctx { struct v4l2_m2m_ctx *m2m_ctx; struct v4l2_ctrl_handler ctrl_hdl; struct mutex vlock; + struct list_head dvfs_node; u32 src_fourcc; u32 coded_fourcc; @@ -181,6 +218,16 @@ void rkvenc_hw_recover(struct rkvenc_dev *dev); int rkvenc_v4l2_init(struct rkvenc_dev *dev); void rkvenc_v4l2_cleanup(struct rkvenc_dev *dev); +/* rkvenc-dvfs.c */ +int rkvenc_dvfs_init(struct rkvenc_dev *dev); +void rkvenc_dvfs_exit(struct rkvenc_dev *dev); +void rkvenc_dvfs_job_begin(struct rkvenc_dev *dev); +void rkvenc_dvfs_job_end(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); +bool rkvenc_dvfs_start_allowed(struct rkvenc_dev *dev); +void rkvenc_dvfs_ctx_add(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); +void rkvenc_dvfs_ctx_idle(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); +void rkvenc_dvfs_ctx_del(struct rkvenc_dev *dev, struct rkvenc_ctx *ctx); + /* rkvenc-v4l2.c */ extern const struct v4l2_m2m_ops rkvenc_m2m_ops; -- 2.56.0 _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RFC PATCH 4/6] media: rockchip: rkvenc: Add frequency scaling and cooling 2026-11-14 14:55 ` Daniel Golle (?) @ 2026-11-14 15:21 ` sashiko-bot -1 siblings, 0 replies; 17+ messages in thread From: sashiko-bot @ 2026-11-14 15:21 UTC (permalink / raw) To: Daniel Golle, devicetree, linux-clk; +Cc: robh, conor+dt, media-ci Thank you for your contribution! Sashiko AI review found 7 potential issues to consider: - [High] media: rockchip: rkvenc: rkvenc_dvfs_init() leaves initial OPP and venc-supply unconfigured - [High] media: rkvenc: UAF and teardown race in rkvenc_remove() and rkvenc_dvfs_exit() - [Medium] media: rockchip: rkvenc: probe fails in rkvenc_dvfs_cooling_init() without THERMAL_OF - [Medium] media: rockchip: rkvenc: thermal duty-cycle throttling bypassed in rkvenc_dvfs_start_allowed() - [Medium] media: rockchip: rkvenc: devfreq updates stall in rkvenc_dvfs_job_end() when round_ctx idles - [Medium] media: rockchip: rkvenc: unreset last_eval in rkvenc_dvfs_job_end() skews first round load - [Low] media: rockchip: rkvenc: freq / HZ_PER_KHZ truncation in rkvenc_cdev_set_state() -- commit f718f2271f482c079204fa578dffff0091dcfc44 Author: Daniel Golle <daniel@makrotopia.org> media: rockchip: rkvenc: Add frequency scaling and cooling This commit adds devfreq-based frequency scaling for the RKVENC core clock andvenc-supply regulator, along with a thermal cooling device that caps the maximum frequency via PM QoS and throttles job execution duty cycle. > diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c b/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c > new file mode 100644 > index 0000000000000..8b59122d5d983 > --- /dev/null > +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c [ ... ] > @@ -0,0 +1,364 @@ [ ... ] > +bool rkvenc_dvfs_start_allowed(struct rkvenc_dev *rkvenc) > +{ > + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; > + unsigned long flags, delay; > + s64 delay_us; > + bool allowed; > + > + spin_lock_irqsave(&dvfs->lock, flags); > + delay_us = ktime_to_us(ktime_sub(dvfs->next_start, ktime_get())); > + allowed = dvfs->duty_pct >= 100 || delay_us <= 0; > + delay = usecs_to_jiffies(clamp_t(s64, delay_us, 0, UINT_MAX)); > + if (!allowed) > + queue_delayed_work(system_wq, &dvfs->throttle_work, delay); > + spin_unlock_irqrestore(&dvfs->lock, flags); > + > + return allowed; > +} [Severity: Medium] Can multiple contexts bypass duty-cycle throttling when dvfs->duty_pct < 100? v4l2_m2m only checks .job_ready() in __v4l2_m2m_try_queue() before placing a context onto m2m_dev->job_queue; once queued, v4l2_m2m_try_run() pops and runs contexts via .device_run() without calling .job_ready() again. Because dvfs->next_start is only updated when a job finishes in rkvenc_dvfs_job_end() and rkvenc_dvfs_start_allowed() does not check dvfs->busy, every context scheduled by rkvenc_dvfs_throttle_work() (or via VIDIOC_QBUF while the first job is running) sees delay_us <= 0 and enters m2m_dev->job_queue to run back-to-back without throttling. In addition, when rkvenc_finish_job() retries a frame on bitstream overflow, it calls rkvenc_dvfs_job_end() followed directly by rkvenc_m2m_ops.device_run(ctx), bypassing rkvenc_dvfs_start_allowed() completely. > + > +void rkvenc_dvfs_job_end(struct rkvenc_dev *rkvenc, struct rkvenc_ctx *ctx) > +{ > + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; > + struct devfreq *devfreq = dvfs->devfreq; > + unsigned long flags; > + bool evaluate; > + ktime_t now; > + > + spin_lock_irqsave(&dvfs->lock, flags); > + now = ktime_get(); > + dvfs->busy_total = > + ktime_add(dvfs->busy_total, ktime_sub(now, dvfs->busy_since)); > + dvfs->next_start = ktime_add_ns( > + dvfs->job_start, > + div_u64(ktime_to_ns(ktime_sub(now, dvfs->job_start)) * 100, > + dvfs->duty_pct)); > + dvfs->busy = false; > + /* A round ends with the context that opened it, covering every other one */ > + evaluate = dvfs->round_ctx == ctx; > + if (!dvfs->round_ctx) > + dvfs->round_ctx = ctx; [Severity: Medium] Does opening a new evaluation round when dvfs->round_ctx is NULL leave a stale dvfs->last_eval from before the encoder was idle? dvfs->last_eval is only initialized in rkvenc_dvfs_init() and updated in rkvenc_dvfs_status(). When the first job after boot or after VIDIOC_STREAMOFF sets dvfs->round_ctx = ctx, dvfs->last_eval and dvfs->busy_total are not reset, so on the next job completion rkvenc_dvfs_status() computes stat->total_time across the entire preceding idle period and drops the clock to the lowest OPP. > + spin_unlock_irqrestore(&dvfs->lock, flags); > + wake_up(&dvfs->idle_wq); > + > + if (!devfreq || !evaluate) > + return; > + > + mutex_lock(&devfreq->lock); > + update_devfreq(devfreq); > + mutex_unlock(&devfreq->lock); > +} [Severity: Medium] What happens to frequency scaling if the context stored in dvfs->round_ctx stops submitting jobs without calling VIDIOC_STREAMOFF or closing the fd? dvfs->round_ctx is only cleared in rkvenc_dvfs_ctx_idle() from rkvenc_stop_streaming() and rkvenc_release(). If that context finishes draining via v4l2_m2m_mark_stopped() or hits vb2_queue_error() in rkvenc_finish_job() (which then calls rkvenc_dvfs_job_end() and keeps or assigns dvfs->round_ctx = ctx), or if userspace simply pauses queueing frames on that context while other contexts keep streaming, evaluate will never be true again and update_devfreq() stops running. [ ... ] > +static int rkvenc_cdev_set_state(struct thermal_cooling_device *cdev, > + unsigned long state) > +{ > + struct rkvenc_dvfs *dvfs = cdev->devdata; > + unsigned long idx = min_t(unsigned long, state, dvfs->num_freqs - 1); > + unsigned long freq = dvfs->freqs[dvfs->num_freqs - 1 - idx]; > + unsigned long flags; > + int ret; > + > + ret = dev_pm_qos_update_request(&dvfs->max_freq_req, freq / HZ_PER_KHZ); > + if (ret < 0) > + return ret; > + > + spin_lock_irqsave(&dvfs->lock, flags); > + dvfs->duty_pct = 100 - RKVENC_THROTTLE_STEP_PCT * (state - idx); > + spin_unlock_irqrestore(&dvfs->lock, flags); > + dvfs->cooling_state = state; > + > + return 0; > +} [Severity: Low] Should freq / HZ_PER_KHZ use DIV_ROUND_UP(freq, HZ_PER_KHZ) here? devfreq converts the PM QoS maximum frequency back to Hz via HZ_PER_KHZ * qos_max_freq and looks up the target OPP with DEVFREQ_FLAG_LEAST_UPPER_BOUND (floor), so truncating an OPP frequency that is not an exact multiple of 1000 Hz drops the limit below that OPP and selects the next lower one. Also, state is not bounds-checked against dvfs->num_freqs - 1 + RKVENC_THROTTLE_STEPS before computing dvfs->duty_pct. [ ... ] > +static int rkvenc_dvfs_cooling_init(struct rkvenc_dev *rkvenc) > +{ > + struct rkvenc_dvfs *dvfs = &rkvenc->dvfs; > + struct device *dev = rkvenc->dev; > + int ret; > + > + if (!of_property_present(dev->of_node, "#cooling-cells")) > + return 0; > + > + ret = dev_pm_qos_add_request(dev, &dvfs->max_freq_req, > + DEV_PM_QOS_MAX_FREQUENCY, > + PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE); > + if (ret < 0) > + return ret; > + dvfs->qos_added = true; > + > + dvfs->cdev = thermal_of_cooling_device_register( > + dev->of_node, 0, "rkvenc", dvfs, &rkvenc_cdev_ops); > + if (IS_ERR(dvfs->cdev)) { > + ret = PTR_ERR(dvfs->cdev); > + dvfs->cdev = NULL; > + return dev_err_probe(dev, ret, > + "could not add the cooling device\n"); > + } > + > + return 0; > +} [Severity: Medium] Does rkvenc_probe() fail with -ENODEV when CONFIG_THERMAL_OF is disabled? VIDEO_ROCKCHIP_RKVENC does not depend on or select THERMAL or THERMAL_OF in Kconfig. When CONFIG_THERMAL_OF is disabled, the thermal_of_cooling_device_register() stub returns ERR_PTR(-ENODEV), which aborts probe on any board whose device tree node has #cooling-cells. > + > +int rkvenc_dvfs_init(struct rkvenc_dev *rkvenc) > +{ [ ... ] > + if (of_property_present(dev->of_node, "venc-supply")) > + opp_config.regulator_names = supply_names; > + > + ret = devm_pm_opp_set_config(dev, &opp_config); > + if (ret) > + return ret; > + > + ret = devm_pm_opp_of_add_table(dev); > + if (ret) > + return ret; > + > + ret = rkvenc_dvfs_freqs_get(rkvenc); > + if (ret) > + return ret; > + > + dvfs->profile.target = rkvenc_dvfs_target; > + dvfs->profile.get_dev_status = rkvenc_dvfs_status; > + dvfs->profile.get_cur_freq = rkvenc_dvfs_cur_freq; > + dvfs->profile.freq_table = dvfs->freqs; > + dvfs->profile.max_state = dvfs->num_freqs; > + rkvenc_dvfs_cur_freq(dev, &dvfs->profile.initial_freq); > + > + devfreq = devfreq_add_device(dev, &dvfs->profile, > + DEVFREQ_GOV_SIMPLE_ONDEMAND, NULL); [Severity: High] Does anything enable and configure venc-supply or set an initial OPP before the first round of encoding jobs completes? devm_pm_opp_set_config() only acquires venc-supply via regulator_get_optional(); the OPP core does not enable the regulator or configure its voltage until dev_pm_opp_set_rate() or dev_pm_opp_set_opp() is called. Because polling is disabled, devfreq_add_device() does not invoke rkvenc_dvfs_target() at registration time, so RKVENC_VERSION is read in rkvenc_probe() and the first round of jobs runs on the hardware before dev_pm_opp_set_rate() is ever called (and regulator_init_complete() can disable a boot-on venc-supply as unused). In addition, setting dvfs->profile.initial_freq directly from clk_get_rate() without rounding it to an OPP entry causes devfreq_update_status() to fail with -EINVAL on the first transition if the boot clock rate does not exactly match an OPP. > diff --git a/drivers/media/platform/rockchip/rkvenc/rkvenc.c b/drivers/media/platform/rockchip/rkvenc/rkvenc.c > index 00778e97686d7..f15a0922059f1 100644 > --- a/drivers/media/platform/rockchip/rkvenc/rkvenc.c > +++ b/drivers/media/platform/rockchip/rkvenc/rkvenc.c [ ... ] > @@ -442,6 +450,7 @@ static void rkvenc_remove(struct platform_device *pdev) > struct rkvenc_dev *rkvenc = platform_get_drvdata(pdev); > > cancel_delayed_work_sync(&rkvenc->watchdog); > + rkvenc_dvfs_exit(rkvenc); > rkvenc_v4l2_cleanup(rkvenc); > v4l2_m2m_release(rkvenc->m2m_dev); > v4l2_device_unregister(&rkvenc->v4l2_dev); [Severity: High] Can calling rkvenc_dvfs_exit() before rkvenc_v4l2_cleanup() and v4l2_m2m_release() cause a use-after-free during driver teardown? While V4L2 contexts and an in-flight job are still active, rkvenc_dvfs_exit() frees dvfs->devfreq before clearing the pointer and cancels dvfs->throttle_work while dvfs->duty_pct can still be below 100: CPU0 CPU1 rkvenc_remove() rkvenc_dvfs_exit() devfreq_remove_device(dvfs->devfreq) rkvenc_irq_thread() rkvenc_finish_job() rkvenc_dvfs_job_end() mutex_lock(&devfreq->lock) <-- use-after-free cancel_delayed_work_sync(&dvfs->throttle_work) v4l2_m2m_buf_done_and_job_finish() ... rkvenc_dvfs_start_allowed() queue_delayed_work(..., &dvfs->throttle_work, ...) ... (devm frees rkvenc) rkvenc_dvfs_throttle_work() <-- use-after-free on rkvenc Also, in rkvenc_dvfs_exit(), calling dev_pm_qos_remove_request() before devfreq_remove_device() fires devfreq's max-frequency QoS notifier and runs update_devfreq() -> rkvenc_dvfs_target() during teardown right before devfreq is removed. -- Sashiko AI review · https://sashiko.dev/#/patchset/cover.1794666330.git.daniel@makrotopia.org?part=4 ^ permalink raw reply [flat|nested] 17+ messages in thread
* [RFC PATCH 5/6] ARM: dts: rockchip: Add RKVENC encoder to RV1126 2026-11-14 14:54 ` Daniel Golle @ 2026-11-14 14:55 ` Daniel Golle -1 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:55 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add the encoder with its IOMMU, the VEPU power domain and the three QoS nodes of the encoder ports. The encoder AXI clock is set to 297 MHz and the core clock to 396 MHz. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- arch/arm/boot/dts/rockchip/rv1126.dtsi | 54 ++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/arch/arm/boot/dts/rockchip/rv1126.dtsi b/arch/arm/boot/dts/rockchip/rv1126.dtsi index 74f238e13533c..63705291dd3ce 100644 --- a/arch/arm/boot/dts/rockchip/rv1126.dtsi +++ b/arch/arm/boot/dts/rockchip/rv1126.dtsi @@ -139,6 +139,21 @@ qos_sdio: qos@fe86c000 { reg = <0xfe86c000 0x20>; }; + qos_venc_rd_m0: qos@fe870000 { + compatible = "rockchip,rv1126-qos", "syscon"; + reg = <0xfe870000 0x20>; + }; + + qos_venc_rd_m1: qos@fe870080 { + compatible = "rockchip,rv1126-qos", "syscon"; + reg = <0xfe870080 0x20>; + }; + + qos_venc_wr: qos@fe870100 { + compatible = "rockchip,rv1126-qos", "syscon"; + reg = <0xfe870100 0x20>; + }; + qos_iep: qos@fe8a0000 { compatible = "rockchip,rv1126-qos", "syscon"; reg = <0xfe8a0000 0x20>; @@ -223,6 +238,17 @@ power-domain@RV1126_PD_VO { <&qos_iep>; #power-domain-cells = <0>; }; + + power-domain@RV1126_PD_VEPU { + reg = <RV1126_PD_VEPU>; + clocks = <&cru ACLK_VENC>, + <&cru HCLK_VENC>, + <&cru CLK_VENC_CORE>; + pm_qos = <&qos_venc_rd_m0>, + <&qos_venc_rd_m1>, + <&qos_venc_wr>; + #power-domain-cells = <0>; + }; }; }; @@ -630,6 +656,34 @@ vop_mmu: iommu@ffb00f00 { status = "disabled"; }; + venc: video-codec@ffbb0000 { + compatible = "rockchip,rv1126-rkvenc"; + reg = <0xffbb0000 0xf00>; + interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_VENC>, <&cru HCLK_VENC>, + <&cru CLK_VENC_CORE>; + clock-names = "axi", "ahb", "core"; + assigned-clocks = <&cru ACLK_VENC>, <&cru CLK_VENC_CORE>; + assigned-clock-rates = <297000000>, <396000000>; + resets = <&cru SRST_VENC_A>, <&cru SRST_VENC_H>, + <&cru SRST_VENC_CORE>; + reset-names = "axi", "ahb", "core"; + iommus = <&venc_mmu>; + power-domains = <&power RV1126_PD_VEPU>; + }; + + venc_mmu: iommu@ffbb0f00 { + compatible = "rockchip,iommu"; + reg = <0xffbb0f00 0x40>, <0xffbb0f40 0x40>; + interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>, + <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_VENC>, <&cru HCLK_VENC>; + clock-names = "aclk", "iface"; + #iommu-cells = <0>; + power-domains = <&power RV1126_PD_VEPU>; + rockchip,disable-mmu-reset; + }; + gmac: ethernet@ffc40000 { compatible = "rockchip,rv1126-gmac", "snps,dwmac-4.20a"; reg = <0xffc40000 0x4000>; -- 2.56.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 5/6] ARM: dts: rockchip: Add RKVENC encoder to RV1126 @ 2026-11-14 14:55 ` Daniel Golle 0 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:55 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add the encoder with its IOMMU, the VEPU power domain and the three QoS nodes of the encoder ports. The encoder AXI clock is set to 297 MHz and the core clock to 396 MHz. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- arch/arm/boot/dts/rockchip/rv1126.dtsi | 54 ++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/arch/arm/boot/dts/rockchip/rv1126.dtsi b/arch/arm/boot/dts/rockchip/rv1126.dtsi index 74f238e13533c..63705291dd3ce 100644 --- a/arch/arm/boot/dts/rockchip/rv1126.dtsi +++ b/arch/arm/boot/dts/rockchip/rv1126.dtsi @@ -139,6 +139,21 @@ qos_sdio: qos@fe86c000 { reg = <0xfe86c000 0x20>; }; + qos_venc_rd_m0: qos@fe870000 { + compatible = "rockchip,rv1126-qos", "syscon"; + reg = <0xfe870000 0x20>; + }; + + qos_venc_rd_m1: qos@fe870080 { + compatible = "rockchip,rv1126-qos", "syscon"; + reg = <0xfe870080 0x20>; + }; + + qos_venc_wr: qos@fe870100 { + compatible = "rockchip,rv1126-qos", "syscon"; + reg = <0xfe870100 0x20>; + }; + qos_iep: qos@fe8a0000 { compatible = "rockchip,rv1126-qos", "syscon"; reg = <0xfe8a0000 0x20>; @@ -223,6 +238,17 @@ power-domain@RV1126_PD_VO { <&qos_iep>; #power-domain-cells = <0>; }; + + power-domain@RV1126_PD_VEPU { + reg = <RV1126_PD_VEPU>; + clocks = <&cru ACLK_VENC>, + <&cru HCLK_VENC>, + <&cru CLK_VENC_CORE>; + pm_qos = <&qos_venc_rd_m0>, + <&qos_venc_rd_m1>, + <&qos_venc_wr>; + #power-domain-cells = <0>; + }; }; }; @@ -630,6 +656,34 @@ vop_mmu: iommu@ffb00f00 { status = "disabled"; }; + venc: video-codec@ffbb0000 { + compatible = "rockchip,rv1126-rkvenc"; + reg = <0xffbb0000 0xf00>; + interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_VENC>, <&cru HCLK_VENC>, + <&cru CLK_VENC_CORE>; + clock-names = "axi", "ahb", "core"; + assigned-clocks = <&cru ACLK_VENC>, <&cru CLK_VENC_CORE>; + assigned-clock-rates = <297000000>, <396000000>; + resets = <&cru SRST_VENC_A>, <&cru SRST_VENC_H>, + <&cru SRST_VENC_CORE>; + reset-names = "axi", "ahb", "core"; + iommus = <&venc_mmu>; + power-domains = <&power RV1126_PD_VEPU>; + }; + + venc_mmu: iommu@ffbb0f00 { + compatible = "rockchip,iommu"; + reg = <0xffbb0f00 0x40>, <0xffbb0f40 0x40>; + interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>, + <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&cru ACLK_VENC>, <&cru HCLK_VENC>; + clock-names = "aclk", "iface"; + #iommu-cells = <0>; + power-domains = <&power RV1126_PD_VEPU>; + rockchip,disable-mmu-reset; + }; + gmac: ethernet@ffc40000 { compatible = "rockchip,rv1126-gmac", "snps,dwmac-4.20a"; reg = <0xffc40000 0x4000>; -- 2.56.0 _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 6/6] ARM: dts: rockchip: Add RKVENC operating points to RV1126 2026-11-14 14:54 ` Daniel Golle @ 2026-11-14 14:55 ` Daniel Golle -1 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:55 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add the four core clock rates of the encoder, 297, 396, 500 and 594 MHz, which the clock tree provides as GPLL/4, GPLL/3, CPLL and GPLL/2, with the voltages of 725, 725, 750 and 825 mV that the vendor device tree gives them. The voltages apply on boards that describe the supply of the encoder as venc-supply. Make the encoder a cooling device so that thermal zones can use it. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- arch/arm/boot/dts/rockchip/rv1126.dtsi | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/arm/boot/dts/rockchip/rv1126.dtsi b/arch/arm/boot/dts/rockchip/rv1126.dtsi index 63705291dd3ce..7e37fd89609a7 100644 --- a/arch/arm/boot/dts/rockchip/rv1126.dtsi +++ b/arch/arm/boot/dts/rockchip/rv1126.dtsi @@ -104,6 +104,30 @@ xin24m: oscillator { #clock-cells = <0>; }; + venc_opp_table: opp-table-venc { + compatible = "operating-points-v2"; + + opp-297000000 { + opp-hz = /bits/ 64 <297000000>; + opp-microvolt = <725000 725000 1000000>; + }; + + opp-396000000 { + opp-hz = /bits/ 64 <396000000>; + opp-microvolt = <725000 725000 1000000>; + }; + + opp-500000000 { + opp-hz = /bits/ 64 <500000000>; + opp-microvolt = <750000 750000 1000000>; + }; + + opp-594000000 { + opp-hz = /bits/ 64 <594000000>; + opp-microvolt = <825000 825000 1000000>; + }; + }; + grf: syscon@fe000000 { compatible = "rockchip,rv1126-grf", "syscon", "simple-mfd"; reg = <0xfe000000 0x20000>; @@ -665,6 +689,8 @@ venc: video-codec@ffbb0000 { clock-names = "axi", "ahb", "core"; assigned-clocks = <&cru ACLK_VENC>, <&cru CLK_VENC_CORE>; assigned-clock-rates = <297000000>, <396000000>; + operating-points-v2 = <&venc_opp_table>; + #cooling-cells = <2>; resets = <&cru SRST_VENC_A>, <&cru SRST_VENC_H>, <&cru SRST_VENC_CORE>; reset-names = "axi", "ahb", "core"; -- 2.56.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [RFC PATCH 6/6] ARM: dts: rockchip: Add RKVENC operating points to RV1126 @ 2026-11-14 14:55 ` Daniel Golle 0 siblings, 0 replies; 17+ messages in thread From: Daniel Golle @ 2026-11-14 14:55 UTC (permalink / raw) To: Daniel Golle, Mauro Carvalho Chehab, Rob Herring, Krzysztof Kozlowski, Conor Dooley, Heiko Stuebner, Stephen Boyd, Brian Masney, Jerome Brunet, Philipp Zabel, Sakari Ailus, Michael Riesch, Mehdi Djait, Hans Verkuil, Gerald Loacker, linux-media, linux-rockchip, devicetree, linux-arm-kernel, linux-kernel, linux-clk Cc: nicolas.dufresne, detlev.casanova, paulk, ezequiel Add the four core clock rates of the encoder, 297, 396, 500 and 594 MHz, which the clock tree provides as GPLL/4, GPLL/3, CPLL and GPLL/2, with the voltages of 725, 725, 750 and 825 mV that the vendor device tree gives them. The voltages apply on boards that describe the supply of the encoder as venc-supply. Make the encoder a cooling device so that thermal zones can use it. Signed-off-by: Daniel Golle <daniel@makrotopia.org> --- arch/arm/boot/dts/rockchip/rv1126.dtsi | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/arm/boot/dts/rockchip/rv1126.dtsi b/arch/arm/boot/dts/rockchip/rv1126.dtsi index 63705291dd3ce..7e37fd89609a7 100644 --- a/arch/arm/boot/dts/rockchip/rv1126.dtsi +++ b/arch/arm/boot/dts/rockchip/rv1126.dtsi @@ -104,6 +104,30 @@ xin24m: oscillator { #clock-cells = <0>; }; + venc_opp_table: opp-table-venc { + compatible = "operating-points-v2"; + + opp-297000000 { + opp-hz = /bits/ 64 <297000000>; + opp-microvolt = <725000 725000 1000000>; + }; + + opp-396000000 { + opp-hz = /bits/ 64 <396000000>; + opp-microvolt = <725000 725000 1000000>; + }; + + opp-500000000 { + opp-hz = /bits/ 64 <500000000>; + opp-microvolt = <750000 750000 1000000>; + }; + + opp-594000000 { + opp-hz = /bits/ 64 <594000000>; + opp-microvolt = <825000 825000 1000000>; + }; + }; + grf: syscon@fe000000 { compatible = "rockchip,rv1126-grf", "syscon", "simple-mfd"; reg = <0xfe000000 0x20000>; @@ -665,6 +689,8 @@ venc: video-codec@ffbb0000 { clock-names = "axi", "ahb", "core"; assigned-clocks = <&cru ACLK_VENC>, <&cru CLK_VENC_CORE>; assigned-clock-rates = <297000000>, <396000000>; + operating-points-v2 = <&venc_opp_table>; + #cooling-cells = <2>; resets = <&cru SRST_VENC_A>, <&cru SRST_VENC_H>, <&cru SRST_VENC_CORE>; reset-names = "axi", "ahb", "core"; -- 2.56.0 _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [RFC PATCH 6/6] ARM: dts: rockchip: Add RKVENC operating points to RV1126 2026-11-14 14:55 ` Daniel Golle (?) @ 2026-11-14 15:15 ` sashiko-bot -1 siblings, 0 replies; 17+ messages in thread From: sashiko-bot @ 2026-11-14 15:15 UTC (permalink / raw) To: Daniel Golle, devicetree, linux-clk; +Cc: robh, conor+dt, media-ci Thank you for your contribution! Sashiko AI review found 6 potential issues to consider: - [High] media: rkvenc: probe fails in rkvenc_dvfs_cooling_init() without THERMAL_OF - [High] media: rkvenc: teardown race in rkvenc_remove() and UAF in rkvenc_release() - [High] media: rkvenc: SCLR_DONE IRQ and unlocked job_running race in rkvenc - [High] media: rkvenc: slot DMA buffer overflow via rkvenc_s_fmt_vid_out() - [Medium] media: rkvenc: missing initial OPP and PM lifecycle for venc-supply - [Medium] media: rkvenc: unbalanced pm_runtime_enable() in rkvenc_probe() -- commit f41b6ddf3d36774f6ef1b61d948fc7c27e1896c8 Author: Daniel Golle <daniel@makrotopia.org> ARM: dts: rockchip: Add RKVENC operating points to RV1126 This commit adds the operating points table for the RKVENC video encoder on RV1126 and registers the encoder node as a cooling device. > diff --git a/arch/arm/boot/dts/rockchip/rv1126.dtsi b/arch/arm/boot/dts/rockchip/rv1126.dtsi > index 63705291dd3ce..7e37fd89609a7 100644 > --- a/arch/arm/boot/dts/rockchip/rv1126.dtsi > +++ b/arch/arm/boot/dts/rockchip/rv1126.dtsi > @@ -104,6 +104,30 @@ xin24m: oscillator { > #clock-cells = <0>; > }; > > + venc_opp_table: opp-table-venc { > + compatible = "operating-points-v2"; > + > + opp-297000000 { > + opp-hz = /bits/ 64 <297000000>; > + opp-microvolt = <725000 725000 1000000>; > + }; [Severity: Medium] On boards that define venc-supply, does rkvenc_dvfs_init() ever apply the initial OPP voltage or disable venc-supply across runtime suspend? In commit "media: rockchip: rkvenc: Add frequency scaling and cooling", devm_pm_opp_set_config() acquires venc-supply without enabling it or setting its voltage, and devfreq_add_device() does not run an initial update_devfreq() because polling_ms is 0: drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c:rkvenc_dvfs_init() { ... if (of_property_present(dev->of_node, "venc-supply")) opp_config.regulator_names = supply_names; ret = devm_pm_opp_set_config(dev, &opp_config); if (ret) return ret; ret = devm_pm_opp_of_add_table(dev); ... } As a result, the first batch of encode jobs runs at the 396 MHz assigned clock rate before dev_pm_opp_set_rate() has ever configured or enabled venc-supply. Once rkvenc_dvfs_job_end() later triggers dev_pm_opp_set_rate(), rkvenc_runtime_suspend() only disables clocks without calling dev_pm_opp_set_rate(dev, 0), leaving venc-supply enabled at the last active OPP voltage (up to 825 mV) while idle. [ ... ] > @@ -665,6 +689,8 @@ venc: video-codec@ffbb0000 { > clock-names = "axi", "ahb", "core"; > assigned-clocks = <&cru ACLK_VENC>, <&cru CLK_VENC_CORE>; > assigned-clock-rates = <297000000>, <396000000>; > + operating-points-v2 = <&venc_opp_table>; > + #cooling-cells = <2>; [Severity: High] Does adding #cooling-cells = <2> cause rkvenc_probe() to fail with -ENODEV when CONFIG_THERMAL_OF is disabled? VIDEO_ROCKCHIP_RKVENC does not depend on or select THERMAL or THERMAL_OF, and thermal_of_cooling_device_register() returns ERR_PTR(-ENODEV) when CONFIG_THERMAL_OF is disabled, which rkvenc_dvfs_cooling_init() treats as a fatal probe error: drivers/media/platform/rockchip/rkvenc/rkvenc-dvfs.c:rkvenc_dvfs_cooling_init() { if (!of_property_present(dev->of_node, "#cooling-cells")) return 0; ... dvfs->cdev = thermal_of_cooling_device_register( dev->of_node, 0, "rkvenc", dvfs, &rkvenc_cdev_ops); if (IS_ERR(dvfs->cdev)) { ret = PTR_ERR(dvfs->cdev); dvfs->cdev = NULL; return dev_err_probe(dev, ret, "could not add the cooling device\n"); } } [Severity: Medium] In commit "media: rockchip: Add RKVENC v1 video encoder driver", does rkvenc_probe() leave runtime PM enabled if pm_runtime_resume_and_get() fails? drivers/media/platform/rockchip/rkvenc/rkvenc.c:rkvenc_probe() { ... pm_runtime_enable(dev); ret = pm_runtime_resume_and_get(dev); if (ret) return ret; ... err_pm_disable: pm_runtime_disable(dev); return ret; } Returning directly instead of jumping to err_pm_disable leaves dev->power.disable_depth unbalanced. [Severity: High] In commits "media: rockchip: Add RKVENC v1 video encoder driver" and "media: rockchip: rkvenc: Add frequency scaling and cooling", can rkvenc_remove() and rkvenc_release() trigger a use-after-free on dvfs->devfreq, rkvenc_dev, and rkvenc->m2m_dev? drivers/media/platform/rockchip/rkvenc/rkvenc.c:rkvenc_remove() { struct rkvenc_dev *rkvenc = platform_get_drvdata(pdev); cancel_delayed_work_sync(&rkvenc->watchdog); rkvenc_dvfs_exit(rkvenc); rkvenc_v4l2_cleanup(rkvenc); v4l2_m2m_release(rkvenc->m2m_dev); v4l2_device_unregister(&rkvenc->v4l2_dev); pm_runtime_disable(&pdev->dev); } rkvenc_remove() cancels the watchdog and tears down DVFS before stopping V4L2/M2M: CPU0 CPU1 rkvenc_remove() rkvenc_dvfs_exit() rkvenc_irq_thread() rkvenc_finish_job() rkvenc_dvfs_job_end() devfreq = dvfs->devfreq devfreq_remove_device(devfreq) mutex_lock(&devfreq->lock) <-- use-after-free In addition, active jobs during rkvenc_dvfs_exit() can re-queue dvfs->throttle_work and rkvenc->watchdog after cancel_delayed_work_sync(). Finally, because struct rkvenc_dev is allocated with devm_kzalloc() and rkvenc->m2m_dev is freed in rkvenc_remove(), closing an open V4L2 file descriptor after unbind runs rkvenc_release() -> rkvenc_dvfs_ctx_del(ctx->dev, ctx) and v4l2_m2m_ctx_release(ctx->m2m_ctx) on freed ctx->dev and rkvenc->m2m_dev memory. [Severity: High] In commit "media: rockchip: Add RKVENC v1 video encoder driver", can rkvenc_hw_recover() and unlocked updates to dev->job_running race with rkvenc_irq_thread() and rkvenc_watchdog_work()? First, the IRQ is registered with IRQF_ONESHOT, which keeps the interrupt line masked while rkvenc_irq_thread() is running. When rkvenc_finish_job() -> rkvenc_hw_recover() waits on dev->sclr_done from inside rkvenc_irq_thread(), RKVENC_INT_SCLR_DONE cannot be delivered until rkvenc_irq_thread() returns, so wait_for_completion_timeout() always times out and the subsequent RKVENC_INT_SCLR_DONE interrupt wakes a spurious rkvenc_irq_thread(). Second, rkvenc_device_run() sets dev->job_ctx and dev->job_running = true without holding dev->irq_lock and before sleeping in rkvenc_dvfs_job_begin(), calling pm_runtime_resume_and_get(), and starting the hardware in rkvenc_hw_run(): CPU0 CPU1 rkvenc_device_run() dev->job_ctx = ctx dev->job_running = true rkvenc_dvfs_job_begin() rkvenc_irq_thread() / rkvenc_watchdog_work() spin_lock_irqsave(&dev->irq_lock) if (dev->job_running) dev->job_running = false spin_unlock_irqrestore(&dev->irq_lock) rkvenc_finish_job() pm_runtime_put() v4l2_m2m_buf_done_and_job_finish() pm_runtime_resume_and_get() rkvenc_hw_run() <-- starts DMA on completed vb2 buffers When the spurious rkvenc_irq_thread() or a racing rkvenc_watchdog_work() (since rkvenc_irq_thread() uses non-sync cancel_delayed_work()) runs in that window, rkvenc_finish_job() reads MMIO while the device may be suspended, underflows the runtime PM usage count, and returns the vb2 buffers to userspace before rkvenc_hw_run() starts DMA on them. [Severity: High] In commit "media: rockchip: Add RKVENC v1 video encoder driver", can changing the OUTPUT format while CAPTURE is streaming overflow the reference frame DMA buffers? rkvenc_start_streaming() only calls rkvenc_ref_alloc() when starting the CAPTURE queue, sizing ctx->slot[] from ctx->width and ctx->height once while ctx->slots_alloc is false. However, rkvenc_s_fmt_vid_out() only checks whether the OUTPUT queue is busy before updating ctx->width and ctx->height: drivers/media/platform/rockchip/rkvenc/rkvenc-v4l2.c:rkvenc_s_fmt_vid_out() { struct rkvenc_ctx *ctx = fh_to_rkvenc_ctx(file->private_data); struct vb2_queue *vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type); if (vb2_is_busy(vq)) return -EBUSY; rkvenc_try_fmt_vid_out(ctx, f); ... ctx->width = f->fmt.pix_mp.width; ctx->height = f->fmt.pix_mp.height; } If userspace starts streaming on CAPTURE first and then increases the OUTPUT dimensions with VIDIOC_S_FMT before starting OUTPUT, rkvenc_device_run() programs the encoder with the larger resolution while DMA-writing reconstructed frames into the smaller ctx->slot[] buffers. -- Sashiko AI review · https://sashiko.dev/#/patchset/cover.1794666330.git.daniel@makrotopia.org?part=6 ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2026-11-14 15:27 UTC | newest] Thread overview: 17+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-14 14:54 [RFC PATCH 0/6] media: rockchip: RKVENC v1 H.264/HEVC encoder for the RV1126 Daniel Golle 2026-11-14 14:54 ` Daniel Golle 2026-11-14 14:54 ` [RFC PATCH 1/6] dt-bindings: media: Add Rockchip RKVENC v1 video encoder Daniel Golle 2026-11-14 14:54 ` Daniel Golle 2026-11-14 14:54 ` [RFC PATCH 2/6] clk: rockchip: rv1126: Add PD_VEPU clock tree Daniel Golle 2026-11-14 14:54 ` Daniel Golle 2026-11-14 14:54 ` [RFC PATCH 3/6] media: rockchip: Add RKVENC v1 video encoder driver Daniel Golle 2026-11-14 14:54 ` Daniel Golle 2026-11-14 15:27 ` sashiko-bot 2026-11-14 14:55 ` [RFC PATCH 4/6] media: rockchip: rkvenc: Add frequency scaling and cooling Daniel Golle 2026-11-14 14:55 ` Daniel Golle 2026-11-14 15:21 ` sashiko-bot 2026-11-14 14:55 ` [RFC PATCH 5/6] ARM: dts: rockchip: Add RKVENC encoder to RV1126 Daniel Golle 2026-11-14 14:55 ` Daniel Golle 2026-11-14 14:55 ` [RFC PATCH 6/6] ARM: dts: rockchip: Add RKVENC operating points " Daniel Golle 2026-11-14 14:55 ` Daniel Golle 2026-11-14 15:15 ` sashiko-bot
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.