[PATCH 00/33] Qualcomm video decoder/encoder driver

Vikash Garodia
2023-07-28T13:24:54Z

This patch series introduces support for Qualcomm new video acceleration
hardware architecture, used for video stream decoding/encoding. This driver
is based on new communication protocol between video hardware and application
processor.

This driver comes with below capabilities:
- V4L2 complaint video driver with M2M and STREAMING capability.
- Supports H264, H265, VP9 decoders.
- Supports H264, H265 encoders.

This driver comes with below features:
- Centralized resource and memory management.
- Centralized management of core and instance states.
- Defines platform specific capabilities and features. As a results, it provides
a single point of control to enable/disable a given feature depending on
specific platform capabilities.
- Handles hardware interdependent configurations. For a given configuration from
client, the driver checks for hardware dependent configuration/s and updates
the same.
- Handles multiple complex video scenarios involving state transitions - DRC,
Drain, Seek, back to back DRC, DRC during Drain sequence, DRC during Seek
sequence.
- Introduces a flexible way for driver to subscribe for any property with
hardware. Hardware would inform driver with those subscribed property with any
change in value.
- Introduces performance (clock and bus) model based on new hardware
architecture.
- Introduces multi thread safe design to handle communication between client and
hardware.
- Adapts encoder quality improvements available in new hardware architecture.
- Implements asynchronous communication with hardware to achieve better
experience in low latency usecases.
- Supports multi stage hardware architecture for encode/decode.
- Output and capture planes are controlled independently. Thereby providing a
way to reconfigure individual plane.
- Hardware packetization layer supports synchronization between configuration
packet and data packet.
- Introduces a flexibility to receive a hardware response for a given command
packet.
- Native hardware support of LAST flag which is mandatory to align with port
reconfiguration and DRAIN sequence as per V4L guidelines.
- Native hardware support for drain sequence.

I think that the driver is in good shape for mainline kernel, and I hope the
review comments will help to improve it, so please do review, and make comments.

Dikshita Agarwal (17):
iris: vidc: add core functions
iris: add vidc wrapper file
iris: vidc: add vb2 ops
iris: vidc: add helpers for memory management
iris: vidc: add helper functions for resource management
iris: vidc: add helper functions for power management
iris: add helpers for media format
iris: vidc: add PIL functionality for video firmware
iris: platform: add platform files
iris: platform: sm8550: add capability file for sm8550
iris: variant: add helper functions for register handling
iris: variant: iris3: add iris3 specific ops
iris: variant: iris3: add helpers for buffer size calculations
iris: variant: iris3: add helper for bus and clock calculation
iris: variant: iris: implement the logic to compute bus bandwidth
iris: variant: iris3: implement logic to compute clock frequency
iris: enable building of iris video driver

Vikash Garodia (16):
MAINTAINERS: Add Qualcomm Iris video accelerator driver
iris: vidc: add v4l2 wrapper file
iris: vidc: define video core and instance context
iris: iris: add video encoder files
iris: vidc: add video decoder files
iris: vidc: add control files
iris: vidc: add helper functions
iris: vidc: add helpers for state management
iris: add vidc buffer files
iris: vidc: define various structures and enum
iris: vidc: hfi: add Host Firmware Interface (HFI)
iris: vidc: hfi: add Host Firmware Interface (HFI) response handling
iris: vidc: hfi: add helpers for handling shared queues
iris: vidc: hfi: Add packetization layer
iris: vidc: hfi: defines HFI properties and enums
iris: vidc: add debug files

MAINTAINERS | 10 +
drivers/media/platform/qcom/Kconfig | 1 +
drivers/media/platform/qcom/Makefile | 1 +
drivers/media/platform/qcom/iris/Kconfig | 15 +
drivers/media/platform/qcom/iris/Makefile | 46 +
.../iris/platform/common/inc/msm_vidc_platform.h | 305 ++
.../iris/platform/common/src/msm_vidc_platform.c | 2499 ++++++++++++
.../iris/platform/sm8550/inc/msm_vidc_sm8550.h | 14 +
.../iris/platform/sm8550/src/msm_vidc_sm8550.c | 1727 ++++++++
.../iris/variant/common/inc/msm_vidc_variant.h | 22 +
.../iris/variant/common/src/msm_vidc_variant.c | 163 +
.../qcom/iris/variant/iris3/inc/hfi_buffer_iris3.h | 1481 +++++++
.../iris/variant/iris3/inc/msm_vidc_buffer_iris3.h | 19 +
.../qcom/iris/variant/iris3/inc/msm_vidc_iris3.h | 15 +
.../iris/variant/iris3/inc/msm_vidc_power_iris3.h | 17 +
.../iris/variant/iris3/inc/perf_static_model.h | 229 ++
.../iris/variant/iris3/src/msm_vidc_buffer_iris3.c | 595 +++
.../iris/variant/iris3/src/msm_vidc_bus_iris3.c | 884 ++++
.../iris/variant/iris3/src/msm_vidc_clock_iris3.c | 627 +++
.../qcom/iris/variant/iris3/src/msm_vidc_iris3.c | 954 +++++
.../iris/variant/iris3/src/msm_vidc_power_iris3.c | 345 ++
.../media/platform/qcom/iris/vidc/inc/firmware.h | 18 +
.../platform/qcom/iris/vidc/inc/hfi_command.h | 190 +
.../media/platform/qcom/iris/vidc/inc/hfi_packet.h | 52 +
.../platform/qcom/iris/vidc/inc/hfi_property.h | 666 +++
.../platform/qcom/iris/vidc/inc/msm_media_info.h | 599 +++
.../media/platform/qcom/iris/vidc/inc/msm_vdec.h | 40 +
.../media/platform/qcom/iris/vidc/inc/msm_venc.h | 34 +
.../media/platform/qcom/iris/vidc/inc/msm_vidc.h | 60 +
.../platform/qcom/iris/vidc/inc/msm_vidc_buffer.h | 32 +
.../platform/qcom/iris/vidc/inc/msm_vidc_control.h | 26 +
.../platform/qcom/iris/vidc/inc/msm_vidc_core.h | 165 +
.../platform/qcom/iris/vidc/inc/msm_vidc_debug.h | 186 +
.../platform/qcom/iris/vidc/inc/msm_vidc_driver.h | 352 ++
.../platform/qcom/iris/vidc/inc/msm_vidc_inst.h | 207 +
.../qcom/iris/vidc/inc/msm_vidc_internal.h | 787 ++++
.../platform/qcom/iris/vidc/inc/msm_vidc_memory.h | 83 +
.../platform/qcom/iris/vidc/inc/msm_vidc_power.h | 94 +
.../platform/qcom/iris/vidc/inc/msm_vidc_state.h | 102 +
.../platform/qcom/iris/vidc/inc/msm_vidc_v4l2.h | 77 +
.../platform/qcom/iris/vidc/inc/msm_vidc_vb2.h | 39 +
.../media/platform/qcom/iris/vidc/inc/resources.h | 259 ++
.../media/platform/qcom/iris/vidc/inc/venus_hfi.h | 66 +
.../platform/qcom/iris/vidc/inc/venus_hfi_queue.h | 89 +
.../qcom/iris/vidc/inc/venus_hfi_response.h | 26 +
.../media/platform/qcom/iris/vidc/src/firmware.c | 294 ++
.../media/platform/qcom/iris/vidc/src/hfi_packet.c | 657 +++
.../media/platform/qcom/iris/vidc/src/msm_vdec.c | 2091 ++++++++++
.../media/platform/qcom/iris/vidc/src/msm_venc.c | 1484 +++++++
.../media/platform/qcom/iris/vidc/src/msm_vidc.c | 841 ++++
.../platform/qcom/iris/vidc/src/msm_vidc_buffer.c | 290 ++
.../platform/qcom/iris/vidc/src/msm_vidc_control.c | 824 ++++
.../platform/qcom/iris/vidc/src/msm_vidc_debug.c | 581 +++
.../platform/qcom/iris/vidc/src/msm_vidc_driver.c | 4276 ++++++++++++++++++++
.../platform/qcom/iris/vidc/src/msm_vidc_memory.c | 448 ++
.../platform/qcom/iris/vidc/src/msm_vidc_power.c | 560 +++
.../platform/qcom/iris/vidc/src/msm_vidc_probe.c | 660 +++
.../platform/qcom/iris/vidc/src/msm_vidc_state.c | 1607 ++++++++
.../platform/qcom/iris/vidc/src/msm_vidc_v4l2.c | 953 +++++
.../platform/qcom/iris/vidc/src/msm_vidc_vb2.c | 605 +++
.../media/platform/qcom/iris/vidc/src/resources.c | 1321 ++++++
.../media/platform/qcom/iris/vidc/src/venus_hfi.c | 1503 +++++++
.../platform/qcom/iris/vidc/src/venus_hfi_queue.c | 537 +++
.../qcom/iris/vidc/src/venus_hfi_response.c | 1607 ++++++++
64 files changed, 35357 insertions(+)
create mode 100644 drivers/media/platform/qcom/iris/Kconfig
create mode 100644 drivers/media/platform/qcom/iris/Makefile
create mode 100644 drivers/media/platform/qcom/iris/platform/common/inc/msm_vidc_platform.h
create mode 100644 drivers/media/platform/qcom/iris/platform/common/src/msm_vidc_platform.c
create mode 100644 drivers/media/platform/qcom/iris/platform/sm8550/inc/msm_vidc_sm8550.h
create mode 100644 drivers/media/platform/qcom/iris/platform/sm8550/src/msm_vidc_sm8550.c
create mode 100644 drivers/media/platform/qcom/iris/variant/common/inc/msm_vidc_variant.h
create mode 100644 drivers/media/platform/qcom/iris/variant/common/src/msm_vidc_variant.c
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/hfi_buffer_iris3.h
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/msm_vidc_buffer_iris3.h
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/msm_vidc_iris3.h
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/msm_vidc_power_iris3.h
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/inc/perf_static_model.h
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_buffer_iris3.c
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_bus_iris3.c
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_clock_iris3.c
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_iris3.c
create mode 100644 drivers/media/platform/qcom/iris/variant/iris3/src/msm_vidc_power_iris3.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/firmware.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/hfi_command.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/hfi_packet.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/hfi_property.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_media_info.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vdec.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_venc.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_buffer.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_control.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_core.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_debug.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_driver.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_inst.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_internal.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_memory.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_power.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_state.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_v4l2.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/msm_vidc_vb2.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/resources.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/venus_hfi.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/venus_hfi_queue.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/inc/venus_hfi_response.h
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/firmware.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/hfi_packet.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vdec.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_venc.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_buffer.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_control.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_debug.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_driver.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_memory.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_power.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_probe.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_state.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_v4l2.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/msm_vidc_vb2.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/resources.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/venus_hfi.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/venus_hfi_queue.c
create mode 100644 drivers/media/platform/qcom/iris/vidc/src/venus_hfi_response.c

--
2.7.4