* [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver
@ 2025-06-05 12:26 Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 1/4] media: allegro-dvt: Move the current driver to a subdirectory Yassine Ouaissa via B4 Relay
` (4 more replies)
0 siblings, 5 replies; 13+ messages in thread
From: Yassine Ouaissa via B4 Relay @ 2025-06-05 12:26 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Michael Tretter, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Yassine OUAISSA, Nicolas Dufresne
Cc: linux-kernel, linux-media, linux-arm-kernel, devicetree
This patch series introduces a new stateful decoder driver for the
allegrodvt GEN 3 IPs.
Changes in v2:
- patch 2: Fix dt-bindings reviews (Krzysztof Kozlowski) (Conor Dooley)
- patch 4: Fix code reviews (Nicolas Dufresne)
- MAINTAINERS: Remove my self from the "ALLEGRO DVT VIDEO IP CORE
DRIVER".
Link to v1: https://lore.kernel.org/all/20250523134207.68481-1-yassine.ouaissa@allegrodvt.com/
# V4L2 Video Decoder Driver System Description
** Hardware Architecture **
The system implements a heterogeneous computing architecture with two primary components:
- **Host Subsystem**: Linux-based CPU running the V4L2 framework and associated drivers
- **IP Subsystem**: Dedicated hardware containing an MCU and a hardware video CODEC
The communication between the two subsystems uses a shared DDR shared memory with bidirectional interrupt mechanism for synchronization.
The architecture is represented in the following diagram:
```
+---------------------+ +----------------------+
| Host | | IP |
| | | |
| +---------------+ | | +----------------+ |
| | | | DDR Shared | | | |
| | Linux Kernel |<-|----------------->|->| MCU | |
| | (CPU) | | Memory | | | |
| | +--------+ | | | +----------------+ |
| | | | | | IRQ when | ^ |
| | | V4L2 | |<-|----new message-->| | |
| | | Drivers| | | | | |
| | | | | | | | APB |
| | +--------+ | | | | |
| | | | | v |
| +---------------+ | | +----------------+ |
| | | | | |
| | | | CODEC | |
| | | | | |
| | | | | |
+---------------------+ +----------------------+
```
** Communication Protocol **
-- Current Implementation - Custom Mailbox --
The host CPU and MCU currently communicate through a custom mailbox protocol implemented over shared memory. The protocol operates as follows:
1. When the host has a new message for the MCU:
- The host writes data to a dedicated shared memory region
- The host triggers an interrupt to the MCU
- The MCU reads the shared memory to obtain the message type and data
2. Similarly, when the MCU has a message for the host:
- The MCU writes to the shared memory
- The MCU triggers an interrupt to the host
- The host reads the shared memory to process the message
-- Migration to RPMSG --
The custom mailbox implementation will be replaced by the standard Linux RPMSG framework.
** Driver Implementation **
This driver implements a V4L2-compliant stateful video decoder with the following characteristics:
-- Technical Specifications --
- **Codec Support**: AVC (H.264), HEVC (H.265), and JPEG
- **Resolution Support**: Up to 4K
- **Pixel Formats**:
- Currently supported: V4L2_PIX_FMT_NV12, V4L2_PIX_FMT_NV16, V4L2_PIX_FMT_P010
- Additional formats planned for future releases
- **Buffer Configuration**: Currently supports semi-planar format only; multiplanar support forthcoming
-- Initialization Sequence --
During probe, the driver performs the following operations:
1. Allocates memory for the MCU firmware
2. Loads the firmware into the allocated memory
3. Initializes the MCU by configuring internal registers (e.g BOOT_ADDR)
4. Establishes the shared memory communication interface (to be replaced by RPMSG)
5. Sets up interrupt handlers for MCU communication
-- Processing Model --
The driver implements a stateful decoding model with the following workflow:
-- Stream Initialization --
1. Upon `VIDIOC_STREAMON` on the OUTPUT queue:
- The driver sends a context creation request to the MCU
- This operation is blocking; the driver waits until the MCU responds with a context handler
- The context handler is stored in a driver-maintained list for subsequent operations
- Each context has its own unique handler to support multiple simultaneous streams
2. Initial stream analysis:
- The driver submits the first compressed buffer (OUTPUT queue)
- The MCU analyzes the stream and reports capability via:
- `resolution_found_event`: Stream is supported, includes stream parameters
- `error_event`: Stream format is unsupported or invalid
-- Decoding Pipeline --
1. After successful stream initialization and receiving the resolution_found_evt:
- The driver can begin normal decoding operations
- V4L2 framework can be informed of format requirements
2. For each compressed buffer (OUTPUT queue):
- The driver submits buffer to MCU with the appropriate context handler
- The MCU processes the buffer and sends `release_bitstream_evt` when complete
- This event signals that the input buffer can be returned to the application
3. For each decoded buffer (CAPTURE queue):
- The MCU fills the buffer with decoded frame data
- The MCU sends `frame_buffer_decode_evt` with important metadata including:
- Original source buffer timestamp
- Flags
- Timecode
- Actual payload size (bytes used in the decoded frame)
- This enables the driver to properly queue the filled buffer to the application
4. End-of-stream handling:
- The MCU sends an event with `eos_evt` when reaching the end of stream
- This allows proper handling of end-of-stream conditions
-- Multi-stream Support --
The driver architecture supports multiple simultaneous decoding contexts with the following characteristics:
1. Each context maintains separate state information
2. The driver manages multiple context handlers returned by the MCU
3. Buffer submissions include the appropriate context handler for routing
4. The system can decode multiple independent streams concurrently
-- Stream Termination --
When `VIDIOC_STREAMOFF` is called:
1. The driver sends a flush command to the MCU for the specific context
2. The driver issues a non-blocking destroy context message
3. All associated resources are released
4. The context handler is removed from the driver's context list
** Error Handling **
The driver implements comprehensive error handling including:
1. Firmware loading failures
2. MCU initialization errors
3. Context creation failures
4. Unsupported stream formats
5. Decoding errors reported by the MCU
6. Timeout handling for unresponsive hardware
** Memory Management **
The system uses the following memory management strategy:
1. Firmware memory is allocated during probe
2. Buffer memory is managed through the V4L2 buffer management interfaces
3. DMA-capable memory is used for buffer transfers between host and MCU
4. The driver properly synchronizes memory access to avoid coherency issues
** Future Enhancements **
Planned future enhancements include:
1. Migration from custom mailbox to RPMSG (in progress)
2. Support for additional pixel formats
3. Implementation of multiplanar buffer support
This comprehensive architecture enables efficient hardware-accelerated video decoding while adhering to standard V4L2 interfaces, making it suitable for upstream inclusion in the Linux kernel.
** Decoder Compliance Testing **
-- AVC and HEVC Fluster Report --
This section contains the compliance test results from Fluster framework for both AVC and HEVC decoders.
The reports validate the decoder's conformance to relevant standards and demonstrate compatibility with a wide range of video streams.
[FLUSTER REPORT FOR THE H.264]
-- JVT-AVC_V1
| Test | FFmpeg-H.264-v4l2m2m |
| ------------------------ | -------------------- |
| TOTAL | 79/135 |
| TOTAL TIME | 437.031s |
| - | - |
| AUD_MW_E | OK |
| BA1_FT_C | OK |
| BA1_Sony_D | OK |
| BA2_Sony_F | OK |
| BA3_SVA_C | OK |
| BA_MW_D | OK |
| BAMQ1_JVC_C | OK |
| BAMQ2_JVC_C | OK |
| BANM_MW_D | OK |
| BASQP1_Sony_C | OK |
| CABA1_Sony_D | OK |
| CABA1_SVA_B | OK |
| CABA2_Sony_E | OK |
| CABA2_SVA_B | OK |
| CABA3_Sony_C | OK |
| CABA3_SVA_B | OK |
| CABA3_TOSHIBA_E | OK |
| cabac_mot_fld0_full | ER |
| cabac_mot_frm0_full | OK |
| cabac_mot_mbaff0_full | ER |
| cabac_mot_picaff0_full | KO |
| CABACI3_Sony_B | OK |
| CABAST3_Sony_E | OK |
| CABASTBR3_Sony_B | OK |
| CABREF3_Sand_D | ER |
| CACQP3_Sony_D | OK |
| CAFI1_SVA_C | ER |
| CAMA1_Sony_C | ER |
| CAMA1_TOSHIBA_B | ER |
| cama1_vtc_c | ER |
| cama2_vtc_b | ER |
| CAMA3_Sand_E | ER |
| cama3_vtc_b | ER |
| CAMACI3_Sony_C | ER |
| CAMANL1_TOSHIBA_B | ER |
| CAMANL2_TOSHIBA_B | ER |
| CAMANL3_Sand_E | ER |
| CAMASL3_Sony_B | ER |
| CAMP_MOT_MBAFF_L30 | ER |
| CAMP_MOT_MBAFF_L31 | ER |
| CANL1_Sony_E | OK |
| CANL1_SVA_B | OK |
| CANL1_TOSHIBA_G | OK |
| CANL2_Sony_E | OK |
| CANL2_SVA_B | OK |
| CANL3_Sony_C | OK |
| CANL3_SVA_B | OK |
| CANL4_SVA_B | OK |
| CANLMA2_Sony_C | ER |
| CANLMA3_Sony_C | ER |
| CAPA1_TOSHIBA_B | ER |
| CAPAMA3_Sand_F | ER |
| CAPCM1_Sand_E | OK |
| CAPCMNL1_Sand_E | OK |
| CAPM3_Sony_D | OK |
| CAQP1_Sony_B | OK |
| cavlc_mot_fld0_full_B | ER |
| cavlc_mot_frm0_full_B | OK |
| cavlc_mot_mbaff0_full_B | ER |
| cavlc_mot_picaff0_full_B | KO |
| CAWP1_TOSHIBA_E | OK |
| CAWP5_TOSHIBA_E | OK |
| CI1_FT_B | OK |
| CI_MW_D | OK |
| CVBS3_Sony_C | OK |
| CVCANLMA2_Sony_C | ER |
| CVFC1_Sony_C | OK |
| CVFI1_Sony_D | ER |
| CVFI1_SVA_C | ER |
| CVFI2_Sony_H | ER |
| CVFI2_SVA_C | ER |
| CVMA1_Sony_D | ER |
| CVMA1_TOSHIBA_B | ER |
| CVMANL1_TOSHIBA_B | ER |
| CVMANL2_TOSHIBA_B | ER |
| CVMAPAQP3_Sony_E | ER |
| CVMAQP2_Sony_G | ER |
| CVMAQP3_Sony_D | ER |
| CVMP_MOT_FLD_L30_B | ER |
| CVMP_MOT_FRM_L31_B | ER |
| CVNLFI1_Sony_C | ER |
| CVNLFI2_Sony_H | ER |
| CVPA1_TOSHIBA_B | ER |
| CVPCMNL1_SVA_C | OK |
| CVPCMNL2_SVA_C | OK |
| CVSE2_Sony_B | OK |
| CVSE3_Sony_H | OK |
| CVSEFDFT3_Sony_E | OK |
| CVWP1_TOSHIBA_E | OK |
| CVWP2_TOSHIBA_E | OK |
| CVWP3_TOSHIBA_E | OK |
| CVWP5_TOSHIBA_E | OK |
| FI1_Sony_E | ER |
| FM1_BT_B | ER |
| FM1_FT_E | KO |
| FM2_SVA_C | ER |
| HCBP1_HHI_A | OK |
| HCBP2_HHI_A | OK |
| HCMP1_HHI_A | OK |
| LS_SVA_D | OK |
| MIDR_MW_D | OK |
| MPS_MW_A | OK |
| MR1_BT_A | OK |
| MR1_MW_A | OK |
| MR2_MW_A | OK |
| MR2_TANDBERG_E | OK |
| MR3_TANDBERG_B | OK |
| MR4_TANDBERG_C | OK |
| MR5_TANDBERG_C | OK |
| MR6_BT_B | ER |
| MR7_BT_B | OK |
| MR8_BT_B | ER |
| MR9_BT_B | ER |
| MV1_BRCM_D | OK |
| NL1_Sony_D | OK |
| NL2_Sony_H | OK |
| NL3_SVA_E | OK |
| NLMQ1_JVC_C | OK |
| NLMQ2_JVC_C | OK |
| NRF_MW_E | OK |
| Sharp_MP_Field_1_B | ER |
| Sharp_MP_Field_2_B | ER |
| Sharp_MP_Field_3_B | ER |
| Sharp_MP_PAFF_1r2 | ER |
| Sharp_MP_PAFF_2r | ER |
| SL1_SVA_B | OK |
| SP1_BT_A | ER |
| sp2_bt_b | ER |
| SVA_BA1_B | OK |
| SVA_BA2_D | OK |
| SVA_Base_B | OK |
| SVA_CL1_E | OK |
| SVA_FM1_E | OK |
| SVA_NL1_B | OK |
| SVA_NL2_E | OK |
| - | - |
| Test | FFmpeg-H.264-v4l2m2m |
| TOTAL | 79/135 |
| TOTAL TIME | 439.031s |
NOTE: The ER (ERROR) streams are not supported by the decoder.
The driver print error message "Unsupported stream"
- JVT-FR-EXT
| Test | FFmpeg-H.264-v4l2m2m |
| ------------------- | -------------------- |
| TOTAL | 23/69 |
| TOTAL TIME | 182.362s |
| - | - |
| alphaconformanceG | OK |
| brcm_freh10 | ER |
| brcm_freh11 | ER |
| brcm_freh3 | OK |
| brcm_freh4 | ER |
| brcm_freh5 | ER |
| brcm_freh6 | ER |
| brcm_freh8 | OK |
| brcm_freh9 | OK |
| FREH10-1 | ER |
| FREH10-2 | ER |
| freh12_b | OK |
| freh1_b | OK |
| freh2_b | OK |
| freh7_b | ER |
| FREXT01_JVC_D | ER |
| FREXT02_JVC_C | ER |
| FRExt1_Panasonic_D | OK |
| FREXT1_TANDBERG_A | ER |
| FRExt2_Panasonic_C | ER |
| FREXT2_TANDBERG_A | ER |
| FRExt3_Panasonic_E | OK |
| FREXT3_TANDBERG_A | ER |
| FRExt4_Panasonic_B | ER |
| FRExt_MMCO4_Sony_B | OK |
| HCAFF1_HHI_B | ER |
| HCAFR1_HHI_C | OK |
| HCAFR2_HHI_A | OK |
| HCAFR3_HHI_A | OK |
| HCAFR4_HHI_A | OK |
| HCAMFF1_HHI_B | ER |
| HCHP1_HHI_B | OK |
| HCHP2_HHI_A | OK |
| HCHP3_HHI_A | ER |
| Hi422FR10_SONY_A | ER |
| Hi422FR11_SONY_A | ER |
| Hi422FR12_SONY_A | ER |
| Hi422FR13_SONY_A | ER |
| Hi422FR14_SONY_A | ER |
| Hi422FR15_SONY_A | ER |
| Hi422FR1_SONY_A | ER |
| Hi422FR2_SONY_A | ER |
| Hi422FR3_SONY_A | ER |
| Hi422FR4_SONY_A | ER |
| Hi422FR6_SONY_A | ER |
| Hi422FR7_SONY_A | ER |
| Hi422FR8_SONY_A | ER |
| Hi422FR9_SONY_A | ER |
| Hi422FREXT16_SONY_A | ER |
| Hi422FREXT17_SONY_A | ER |
| Hi422FREXT18_SONY_A | ER |
| Hi422FREXT19_SONY_A | ER |
| HPCA_BRCM_C | OK |
| HPCADQ_BRCM_B | OK |
| HPCAFL_BRCM_C | ER |
| HPCAFLNL_BRCM_C | ER |
| HPCALQ_BRCM_B | OK |
| HPCAMAPALQ_BRCM_B | ER |
| HPCAMOLQ_BRCM_B | ER |
| HPCANL_BRCM_C | OK |
| HPCAQ2LQ_BRCM_B | OK |
| HPCV_BRCM_A | OK |
| HPCVFL_BRCM_A | ER |
| HPCVFLNL_BRCM_A | ER |
| HPCVMOLQ_BRCM_B | ER |
| HPCVNL_BRCM_A | OK |
| HVLCFI0_Sony_B | ER |
| HVLCMFF0_Sony_B | ER |
| HVLCPFF0_Sony_B | ER |
| - | - |
| Test | FFmpeg-H.264-v4l2m2m |
| TOTAL | 23/69 |
| TOTAL TIME | 182.362s |
NOTE: The ER (ERROR) streams are not supported by the decoder.
The driver print error message "Unsupported stream"
- JVT-MVC
| Test | FFmpeg-H.264-v4l2m2m |
| ---------- | -------------------- |
| TOTAL | 18/20 |
| TOTAL TIME | 147.076s |
| - | - |
| MVCDS-4 | OK |
| MVCDS-5 | OK |
| MVCDS-6 | OK |
| MVCDS1 | OK |
| MVCDS2 | OK |
| MVCDS3 | OK |
| MVCICT-1 | ER |
| MVCICT-2 | ER |
| MVCNV-2 | OK |
| MVCNV-3 | OK |
| MVCNV1 | OK |
| MVCNV4 | OK |
| MVCRP_1 | OK |
| MVCRP_2 | OK |
| MVCRP_3 | OK |
| MVCRP_4 | OK |
| MVCRP_5 | OK |
| MVCRP_6 | OK |
| MVCSPS-1 | OK |
| MVCSPS-2 | OK |
| - | - |
| Test | FFmpeg-H.264-v4l2m2m |
| TOTAL | 18/20 |
| TOTAL TIME | 147.076s |
- JVT-SVC
| Test | FFmpeg-H.264-v4l2m2m |
| --------------- | -------------------- |
| TOTAL | 75/185 |
| TOTAL TIME | 727.240s |
| - | - |
| SVCBC-1-L0 | OK |
| SVCBC-1-L1 | KO |
| SVCBCT-1-L0 | OK |
| SVCBCT-1-L1 | KO |
| SVCBCTS-1-r1-L0 | OK |
| SVCBCTS-1-r1-L1 | KO |
| SVCBCTS-1-r1-L2 | KO |
| SVCBCTS-2-r1-L0 | OK |
| SVCBCTS-2-r1-L1 | KO |
| SVCBCTS-2-r1-L2 | KO |
| SVCBCTS-3-L0 | OK |
| SVCBCTS-3-L1 | KO |
| SVCBCTS-3-L2 | KO |
| SVCBM-1-L0 | OK |
| SVCBM-1-L1 | KO |
| SVCBM-2-L0 | OK |
| SVCBM-2-L1 | KO |
| SVCBM-3-L0 | OK |
| SVCBM-3-L1 | KO |
| SVCBM-4-r1-L0 | OK |
| SVCBM-4-r1-L1 | KO |
| SVCBM-4-r1-L2 | KO |
| SVCBM-5-L0 | OK |
| SVCBM-5-L1 | KO |
| SVCBM-5-L2 | KO |
| SVCBM-5-L3 | KO |
| SVCBMST-1-L0 | OK |
| SVCBMST-1-L1 | KO |
| SVCBMST-1-L2 | KO |
| SVCBMST-2-L0 | OK |
| SVCBMST-2-L1 | KO |
| SVCBMST-2-L2 | KO |
| SVCBMST-3-L0 | OK |
| SVCBMST-3-L1 | KO |
| SVCBMST-3-L2 | KO |
| SVCBMT-1-L0 | OK |
| SVCBMT-1-L1 | KO |
| SVCBMT-10-L0 | OK |
| SVCBMT-10-L1 | KO |
| SVCBMT-11-L0 | OK |
| SVCBMT-11-L1 | KO |
| SVCBMT-12-L0 | OK |
| SVCBMT-12-L1 | KO |
| SVCBMT-13-L0 | OK |
| SVCBMT-13-L1 | KO |
| SVCBMT-13-L2 | KO |
| SVCBMT-2-L0 | OK |
| SVCBMT-2-L1 | KO |
| SVCBMT-3-L0 | OK |
| SVCBMT-3-L1 | KO |
| SVCBMT-4-L0 | OK |
| SVCBMT-4-L1 | KO |
| SVCBMT-5-L0 | OK |
| SVCBMT-5-L1 | KO |
| SVCBMT-6-L0 | OK |
| SVCBMT-6-L1 | KO |
| SVCBMT-7-L0 | OK |
| SVCBMT-7-L1 | KO |
| SVCBMT-8-L0 | OK |
| SVCBMT-8-L1 | KO |
| SVCBMT-9-L0 | OK |
| SVCBMT-9-L1 | KO |
| SVCBS-1-L0 | OK |
| SVCBS-1-L1 | KO |
| SVCBS-2-L0 | OK |
| SVCBS-2-L1 | KO |
| SVCBS-3-r1-L0 | OK |
| SVCBS-3-r1-L1 | KO |
| SVCBS-4-r1-L0 | OK |
| SVCBS-4-r1-L1 | KO |
| SVCBS-5-r1-L0 | OK |
| SVCBS-5-r1-L1 | KO |
| SVCBS-6-r1-L0 | OK |
| SVCBS-6-r1-L1 | KO |
| SVCBS-6-r1-L2 | KO |
| SVCBS-7-L0 | OK |
| SVCBS-7-L1 | KO |
| SVCBS-8-L0 | OK |
| SVCBS-8-L1 | KO |
| SVCBST-1-L0 | OK |
| SVCBST-1-L1 | KO |
| SVCBST-10-r1-L0 | OK |
| SVCBST-10-r1-L1 | KO |
| SVCBST-11-r1-L0 | OK |
| SVCBST-11-r1-L1 | KO |
| SVCBST-12-r1-L0 | OK |
| SVCBST-12-r1-L1 | KO |
| SVCBST-13-L0 | OK |
| SVCBST-13-L1 | KO |
| SVCBST-14-L0 | OK |
| SVCBST-14-L1 | KO |
| SVCBST-14-L2 | KO |
| SVCBST-15-L0 | OK |
| SVCBST-15-L1 | KO |
| SVCBST-15-L2 | KO |
| SVCBST-16-r1-L0 | OK |
| SVCBST-16-r1-L1 | KO |
| SVCBST-16-r1-L2 | KO |
| SVCBST-17-r1-L0 | OK |
| SVCBST-17-r1-L1 | KO |
| SVCBST-17-r1-L2 | KO |
| SVCBST-18-r1-L0 | OK |
| SVCBST-18-r1-L1 | KO |
| SVCBST-18-r1-L2 | KO |
| SVCBST-19-L0 | OK |
| SVCBST-19-L1 | KO |
| SVCBST-2-L0 | OK |
| SVCBST-2-L1 | KO |
| SVCBST-20-L0 | OK |
| SVCBST-20-L1 | KO |
| SVCBST-3-L0 | OK |
| SVCBST-3-L1 | KO |
| SVCBST-4-L0 | OK |
| SVCBST-4-L1 | KO |
| SVCBST-5-L0 | OK |
| SVCBST-5-L1 | KO |
| SVCBST-6-r1-L0 | OK |
| SVCBST-6-r1-L1 | KO |
| SVCBST-7-r1-L0 | OK |
| SVCBST-7-r1-L1 | KO |
| SVCBST-8-r1-L0 | OK |
| SVCBST-8-r1-L1 | KO |
| SVCBST-9-r1-L0 | OK |
| SVCBST-9-r1-L1 | KO |
| SVCBSTC-1-L0 | OK |
| SVCBSTC-1-L1 | KO |
| SVCBSTC-1-L2 | KO |
| SVCHCTS-1-r1-L0 | OK |
| SVCHCTS-1-r1-L1 | KO |
| SVCHCTS-1-r1-L2 | KO |
| SVCHCTS-1-r1-L3 | KO |
| SVCHCTS-1-r1-L4 | KO |
| SVCHCTS-1-r1-L5 | KO |
| SVCHCTS-1-r1-L6 | KO |
| SVCHCTS-1-r1-L7 | KO |
| SVCHICS-1-L0 | OK |
| SVCHICS-1-L1 | KO |
| SVCHICS-1-L2 | KO |
| SVCHICS-1-L3 | KO |
| SVCHIS-1-L0 | OK |
| SVCHIS-1-L1 | KO |
| SVCHIS-1-L2 | KO |
| SVCHIS-2-L0 | OK |
| SVCHIS-2-L1 | KO |
| SVCHIS-2-L2 | KO |
| SVCHIS-3-L0 | OK |
| SVCHIS-3-L1 | KO |
| SVCHIS-3-L2 | KO |
| SVCHM-1-L0 | OK |
| SVCHM-1-L1 | KO |
| SVCHM-1-L2 | KO |
| SVCHM-1-L3 | KO |
| SVCHM-2-L0 | OK |
| SVCHM-2-L1 | OK |
| SVCHM-3-L0 | OK |
| SVCHM-3-L1 | OK |
| SVCHM-4-L0 | OK |
| SVCHM-4-L1 | OK |
| SVCHM-4-L2 | OK |
| SVCHMTS-1-r1-L0 | OK |
| SVCHMTS-1-r1-L1 | KO |
| SVCHMTS-1-r1-L2 | KO |
| SVCHMTS-1-r1-L3 | KO |
| SVCHMTS-1-r1-L4 | KO |
| SVCHMTS-1-r1-L5 | KO |
| SVCHMTS-2-r1-L0 | OK |
| SVCHMTS-2-r1-L1 | KO |
| SVCHMTS-2-r1-L2 | KO |
| SVCHS-1-r1-L0 | OK |
| SVCHS-1-r1-L1 | KO |
| SVCHS-2-r1-L0 | OK |
| SVCHS-2-r1-L1 | KO |
| SVCHST-1-r1-L0 | OK |
| SVCHST-1-r1-L1 | KO |
| SVCHST-1-r1-L2 | KO |
| SVCHST-2-r1-L0 | OK |
| SVCHST-2-r1-L1 | KO |
| SVCHST-2-r1-L2 | KO |
| SVCHST-3-r1-L0 | ER |
| SVCHST-3-r1-L1 | ER |
| SVCHST-4-r1-L0 | ER |
| SVCHST-4-r1-L1 | ER |
| SVCHSTC-1-r1-L0 | OK |
| SVCHSTC-1-r1-L1 | KO |
| SVCHSTC-1-r1-L2 | KO |
| - | - |
| Test | FFmpeg-H.264-v4l2m2m |
| TOTAL | 75/185 |
| TOTAL TIME | 727.240s |
NOTE: The current implementation of the decoder only supports Layer 0 (base layer) processing.
When attempting to decode streams that contain multiple layers (such as scalable or multi-view content), the decoding operation fails.
This limitation means that enhanced features requiring layer-based processing beyond the base layer cannot be properly handled by the current decoder.
For successful decoding, input streams must be limited to single-layer content only.
[FLUSTER REPORT FOR THE H.265]
| - | - |
| AMP_A_Samsung_7 | OK |
| AMP_B_Samsung_7 | OK |
| AMP_D_Hisilicon_3 | OK |
| AMP_E_Hisilicon_3 | OK |
| AMP_F_Hisilicon_3 | ER |
| AMVP_A_MTK_4 | ER |
| AMVP_B_MTK_4 | OK |
| AMVP_C_Samsung_7 | ER |
| BUMPING_A_ericsson_1 | OK |
| CAINIT_A_SHARP_4 | OK |
| CAINIT_B_SHARP_4 | OK |
| CAINIT_C_SHARP_3 | OK |
| CAINIT_D_SHARP_3 | OK |
| CAINIT_E_SHARP_3 | OK |
| CAINIT_F_SHARP_3 | OK |
| CAINIT_G_SHARP_3 | OK |
| CAINIT_H_SHARP_3 | OK |
| CIP_A_Panasonic_3 | OK |
| cip_B_NEC_3 | OK |
| CIP_C_Panasonic_2 | OK |
| CONFWIN_A_Sony_1 | OK |
| DBLK_A_MAIN10_VIXS_4 | ER |
| DBLK_A_SONY_3 | OK |
| DBLK_B_SONY_3 | OK |
| DBLK_C_SONY_3 | OK |
| DBLK_D_VIXS_2 | OK |
| DBLK_E_VIXS_2 | OK |
| DBLK_F_VIXS_2 | OK |
| DBLK_G_VIXS_2 | OK |
| DELTAQP_A_BRCM_4 | OK |
| DELTAQP_B_SONY_3 | OK |
| DELTAQP_C_SONY_3 | OK |
| DSLICE_A_HHI_5 | OK |
| DSLICE_B_HHI_5 | OK |
| DSLICE_C_HHI_5 | OK |
| ENTP_A_QUALCOMM_1 | OK |
| ENTP_B_Qualcomm_1 | OK |
| ENTP_C_Qualcomm_1 | OK |
| EXT_A_ericsson_4 | OK |
| FILLER_A_Sony_1 | OK |
| HRD_A_Fujitsu_3 | OK |
| INITQP_A_Sony_1 | OK |
| INITQP_B_Main10_Sony_1 | ER |
| ipcm_A_NEC_3 | OK |
| ipcm_B_NEC_3 | OK |
| ipcm_C_NEC_3 | OK |
| ipcm_D_NEC_3 | OK |
| ipcm_E_NEC_2 | OK |
| IPRED_A_docomo_2 | OK |
| IPRED_B_Nokia_3 | OK |
| IPRED_C_Mitsubishi_3 | OK |
| LS_A_Orange_2 | OK |
| LS_B_Orange_4 | OK |
| LTRPSPS_A_Qualcomm_1 | KO |
| MAXBINS_A_TI_5 | OK |
| MAXBINS_B_TI_5 | OK |
| MAXBINS_C_TI_5 | OK |
| MERGE_A_TI_3 | OK |
| MERGE_B_TI_3 | OK |
| MERGE_C_TI_3 | OK |
| MERGE_D_TI_3 | OK |
| MERGE_E_TI_3 | OK |
| MERGE_F_MTK_4 | OK |
| MERGE_G_HHI_4 | OK |
| MVCLIP_A_qualcomm_3 | OK |
| MVDL1ZERO_A_docomo_4 | OK |
| MVEDGE_A_qualcomm_3 | OK |
| NoOutPrior_A_Qualcomm_1 | OK |
| NoOutPrior_B_Qualcomm_1 | OK |
| NUT_A_ericsson_5 | OK |
| OPFLAG_A_Qualcomm_1 | OK |
| OPFLAG_B_Qualcomm_1 | OK |
| OPFLAG_C_Qualcomm_1 | OK |
| PICSIZE_A_Bossen_1 | OK |
| PICSIZE_B_Bossen_1 | ER |
| PICSIZE_C_Bossen_1 | OK |
| PICSIZE_D_Bossen_1 | ER |
| PMERGE_A_TI_3 | OK |
| PMERGE_B_TI_3 | OK |
| PMERGE_C_TI_3 | OK |
| PMERGE_D_TI_3 | OK |
| PMERGE_E_TI_3 | OK |
| POC_A_Bossen_3 | OK |
| PPS_A_qualcomm_7 | OK |
| PS_B_VIDYO_3 | ER |
| RAP_A_docomo_6 | OK |
| RAP_B_Bossen_2 | OK |
| RPLM_A_qualcomm_4 | OK |
| RPLM_B_qualcomm_4 | OK |
| RPS_A_docomo_5 | OK |
| RPS_B_qualcomm_5 | OK |
| RPS_C_ericsson_5 | OK |
| RPS_D_ericsson_6 | OK |
| RPS_E_qualcomm_5 | OK |
| RPS_F_docomo_2 | OK |
| RQT_A_HHI_4 | OK |
| RQT_B_HHI_4 | OK |
| RQT_C_HHI_4 | OK |
| RQT_D_HHI_4 | OK |
| RQT_E_HHI_4 | OK |
| RQT_F_HHI_4 | OK |
| RQT_G_HHI_4 | OK |
| SAO_A_MediaTek_4 | OK |
| SAO_B_MediaTek_5 | OK |
| SAO_C_Samsung_5 | OK |
| SAO_D_Samsung_5 | OK |
| SAO_E_Canon_4 | OK |
| SAO_F_Canon_3 | OK |
| SAO_G_Canon_3 | OK |
| SAO_H_Parabola_1 | OK |
| SAODBLK_A_MainConcept_4 | OK |
| SAODBLK_B_MainConcept_4 | OK |
| SDH_A_Orange_4 | OK |
| SLICES_A_Rovi_3 | OK |
| SLIST_A_Sony_5 | OK |
| SLIST_B_Sony_9 | OK |
| SLIST_C_Sony_4 | OK |
| SLIST_D_Sony_9 | OK |
| SLPPLP_A_VIDYO_2 | ER |
| STRUCT_A_Samsung_7 | ER |
| STRUCT_B_Samsung_7 | ER |
| TILES_A_Cisco_2 | ER |
| TILES_B_Cisco_1 | ER |
| TMVP_A_MS_3 | OK |
| TSCL_A_VIDYO_5 | OK |
| TSCL_B_VIDYO_4 | ER |
| TSKIP_A_MS_3 | OK |
| TSUNEQBD_A_MAIN10_Technicolor_2 | ER |
| TUSIZE_A_Samsung_1 | OK |
| VPSID_A_VIDYO_2 | ER |
| VPSSPSPPS_A_MainConcept_1 | KO |
| WP_A_MAIN10_Toshiba_3 | ER |
| WP_A_Toshiba_3 | ER |
| WP_B_Toshiba_3 | OK |
| WP_MAIN10_B_Toshiba_3 | ER |
| WPP_A_ericsson_MAIN10_2 | ER |
| WPP_A_ericsson_MAIN_2 | OK |
| WPP_B_ericsson_MAIN10_2 | ER |
| WPP_B_ericsson_MAIN_2 | OK |
| WPP_C_ericsson_MAIN10_2 | ER |
| WPP_C_ericsson_MAIN_2 | OK |
| WPP_D_ericsson_MAIN10_2 | ER |
| WPP_D_ericsson_MAIN_2 | OK |
| WPP_E_ericsson_MAIN10_2 | ER |
| WPP_E_ericsson_MAIN_2 | OK |
| WPP_F_ericsson_MAIN10_2 | ER |
| WPP_F_ericsson_MAIN_2 | OK |
| - | - |
| Test | FFmpeg-H.265-v4l2m2m |
| TOTAL | 120/147 |
| TOTAL TIME | 12669.641s |
Failed streams :
- VPSSPSPPS_A_MainConcept_1 : Failed due to evolutive dynamic resolution increases. The decoder cannot properly handle upward resolution changes within the same stream.
- LTRPSPS_A_Qualcomm_1
Yassine Ouaissa (5):
media: allegro-dvt: Move the current driver to a subdirectory
dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP
MAINTAINERS: Add entry for allegrodvt Gen 3 drivers
dt-bindings: vendor-prefixes: Update the description of allegro prefix
media: allegro-dvt: Add Gen 3 IP stateful decoder driver
.../bindings/media/allegro,al300-vdec.yaml | 75 +
.../devicetree/bindings/vendor-prefixes.yaml | 2 +-
MAINTAINERS | 5 +-
drivers/media/platform/allegro-dvt/Kconfig | 17 +-
drivers/media/platform/allegro-dvt/Makefile | 6 +-
.../media/platform/allegro-dvt/al300/Kconfig | 23 +
.../media/platform/allegro-dvt/al300/Makefile | 6 +
.../allegro-dvt/al300/al_codec_common.c | 754 ++++++++
.../allegro-dvt/al300/al_codec_common.h | 247 +++
.../allegro-dvt/al300/al_codec_util.c | 177 ++
.../allegro-dvt/al300/al_codec_util.h | 185 ++
.../platform/allegro-dvt/al300/al_vdec_drv.c | 1530 +++++++++++++++++
.../platform/allegro-dvt/al300/al_vdec_drv.h | 94 +
.../media/platform/allegro-dvt/zynqmp/Kconfig | 17 +
.../platform/allegro-dvt/zynqmp/Makefile | 6 +
.../allegro-dvt/{ => zynqmp}/allegro-core.c | 0
.../allegro-dvt/{ => zynqmp}/allegro-mail.c | 0
.../allegro-dvt/{ => zynqmp}/allegro-mail.h | 0
.../allegro-dvt/{ => zynqmp}/nal-h264.c | 0
.../allegro-dvt/{ => zynqmp}/nal-h264.h | 0
.../allegro-dvt/{ => zynqmp}/nal-hevc.c | 0
.../allegro-dvt/{ => zynqmp}/nal-hevc.h | 0
.../allegro-dvt/{ => zynqmp}/nal-rbsp.c | 0
.../allegro-dvt/{ => zynqmp}/nal-rbsp.h | 0
24 files changed, 3123 insertions(+), 21 deletions(-)
create mode 100644 Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
create mode 100644 drivers/media/platform/allegro-dvt/al300/Kconfig
create mode 100644 drivers/media/platform/allegro-dvt/al300/Makefile
create mode 100644 drivers/media/platform/allegro-dvt/al300/al_codec_common.c
create mode 100644 drivers/media/platform/allegro-dvt/al300/al_codec_common.h
create mode 100644 drivers/media/platform/allegro-dvt/al300/al_codec_util.c
create mode 100644 drivers/media/platform/allegro-dvt/al300/al_codec_util.h
create mode 100644 drivers/media/platform/allegro-dvt/al300/al_vdec_drv.c
create mode 100644 drivers/media/platform/allegro-dvt/al300/al_vdec_drv.h
create mode 100644 drivers/media/platform/allegro-dvt/zynqmp/Kconfig
create mode 100644 drivers/media/platform/allegro-dvt/zynqmp/Makefile
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/allegro-core.c (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/allegro-mail.c (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/allegro-mail.h (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/nal-h264.c (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/nal-h264.h (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/nal-hevc.c (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/nal-hevc.h (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/nal-rbsp.c (100%)
rename drivers/media/platform/allegro-dvt/{ => zynqmp}/nal-rbsp.h (100%)
--
2.30.2
---
Yassine Ouaissa (4):
media: allegro-dvt: Move the current driver to a subdirectory
dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP
dt-bindings: vendor-prefixes: Update the description of allegro prefix
media: allegro-dvt: Add Gen 3 IP stateful decoder driver
.../bindings/media/allegro,al300-vdec.yaml | 75 +
.../devicetree/bindings/vendor-prefixes.yaml | 2 +-
MAINTAINERS | 4 +-
drivers/media/platform/allegro-dvt/Kconfig | 17 +-
drivers/media/platform/allegro-dvt/Makefile | 6 +-
drivers/media/platform/allegro-dvt/al300/Kconfig | 23 +
drivers/media/platform/allegro-dvt/al300/Makefile | 6 +
.../platform/allegro-dvt/al300/al_codec_common.c | 733 ++++++++++
.../platform/allegro-dvt/al300/al_codec_common.h | 248 ++++
.../platform/allegro-dvt/al300/al_codec_util.c | 174 +++
.../platform/allegro-dvt/al300/al_codec_util.h | 186 +++
.../media/platform/allegro-dvt/al300/al_vdec_drv.c | 1518 ++++++++++++++++++++
.../media/platform/allegro-dvt/al300/al_vdec_drv.h | 93 ++
drivers/media/platform/allegro-dvt/zynqmp/Kconfig | 17 +
drivers/media/platform/allegro-dvt/zynqmp/Makefile | 6 +
.../allegro-dvt/{ => zynqmp}/allegro-core.c | 0
.../allegro-dvt/{ => zynqmp}/allegro-mail.c | 0
.../allegro-dvt/{ => zynqmp}/allegro-mail.h | 0
.../platform/allegro-dvt/{ => zynqmp}/nal-h264.c | 0
.../platform/allegro-dvt/{ => zynqmp}/nal-h264.h | 0
.../platform/allegro-dvt/{ => zynqmp}/nal-hevc.c | 0
.../platform/allegro-dvt/{ => zynqmp}/nal-hevc.h | 0
.../platform/allegro-dvt/{ => zynqmp}/nal-rbsp.c | 0
.../platform/allegro-dvt/{ => zynqmp}/nal-rbsp.h | 0
24 files changed, 3087 insertions(+), 21 deletions(-)
---
base-commit: ec7714e4947909190ffb3041a03311a975350fe0
change-id: 20250527-allegro_dvt_al300_dec_driver-4e75a94af6dc
Best regards,
--
Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v2 1/4] media: allegro-dvt: Move the current driver to a subdirectory
2025-06-05 12:26 [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
@ 2025-06-05 12:26 ` Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP Yassine Ouaissa via B4 Relay
` (3 subsequent siblings)
4 siblings, 0 replies; 13+ messages in thread
From: Yassine Ouaissa via B4 Relay @ 2025-06-05 12:26 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Michael Tretter, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Yassine OUAISSA, Nicolas Dufresne
Cc: linux-kernel, linux-media, linux-arm-kernel, devicetree
From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
In preparation for the upcoming driver update, we need to relocate the
current driver.
This will help ensure a clean transition and avoid any potential
conflicts with the new driver.
This patch is crucial for keeping our directory organized and
facilitating a smooth integration of the new driver.
Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
---
MAINTAINERS | 2 +-
drivers/media/platform/allegro-dvt/Kconfig | 16 +---------------
drivers/media/platform/allegro-dvt/Makefile | 5 +----
drivers/media/platform/allegro-dvt/zynqmp/Kconfig | 17 +++++++++++++++++
drivers/media/platform/allegro-dvt/zynqmp/Makefile | 6 ++++++
.../platform/allegro-dvt/{ => zynqmp}/allegro-core.c | 0
.../platform/allegro-dvt/{ => zynqmp}/allegro-mail.c | 0
.../platform/allegro-dvt/{ => zynqmp}/allegro-mail.h | 0
.../media/platform/allegro-dvt/{ => zynqmp}/nal-h264.c | 0
.../media/platform/allegro-dvt/{ => zynqmp}/nal-h264.h | 0
.../media/platform/allegro-dvt/{ => zynqmp}/nal-hevc.c | 0
.../media/platform/allegro-dvt/{ => zynqmp}/nal-hevc.h | 0
.../media/platform/allegro-dvt/{ => zynqmp}/nal-rbsp.c | 0
.../media/platform/allegro-dvt/{ => zynqmp}/nal-rbsp.h | 0
14 files changed, 26 insertions(+), 20 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index eecc41c39a9cb467e91023307b92a181af6ee23d..abc6ba61048771303bc219102f2db602266b7c30 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -817,7 +817,7 @@ R: Pengutronix Kernel Team <kernel@pengutronix.de>
L: linux-media@vger.kernel.org
S: Maintained
F: Documentation/devicetree/bindings/media/allegro,al5e.yaml
-F: drivers/media/platform/allegro-dvt/
+F: drivers/media/platform/allegro-dvt/zynqmp
ALLIED VISION ALVIUM CAMERA DRIVER
M: Tommaso Merciai <tomm.merciai@gmail.com>
diff --git a/drivers/media/platform/allegro-dvt/Kconfig b/drivers/media/platform/allegro-dvt/Kconfig
index 2182e1277568a407f51a23ea437811c50b1183c8..e9008614c27b9490d1cd29fab887977a1918ede4 100644
--- a/drivers/media/platform/allegro-dvt/Kconfig
+++ b/drivers/media/platform/allegro-dvt/Kconfig
@@ -2,18 +2,4 @@
comment "Allegro DVT media platform drivers"
-config VIDEO_ALLEGRO_DVT
- tristate "Allegro DVT Video IP Core"
- depends on V4L_MEM2MEM_DRIVERS
- depends on VIDEO_DEV
- depends on ARCH_ZYNQMP || COMPILE_TEST
- select V4L2_MEM2MEM_DEV
- select VIDEOBUF2_DMA_CONTIG
- select REGMAP_MMIO
- help
- Support for the encoder video IP core by Allegro DVT. This core is
- found for example on the Xilinx ZynqMP SoC in the EV family and is
- called VCU in the reference manual.
-
- To compile this driver as a module, choose M here: the module
- will be called allegro.
+source "drivers/media/platform/allegro-dvt/zynqmp/Kconfig"
diff --git a/drivers/media/platform/allegro-dvt/Makefile b/drivers/media/platform/allegro-dvt/Makefile
index 66108a3037747020d549bc0a427881e0667a3f0a..d2aa6875edcf7760901996aac4d5ac98282cce20 100644
--- a/drivers/media/platform/allegro-dvt/Makefile
+++ b/drivers/media/platform/allegro-dvt/Makefile
@@ -1,6 +1,3 @@
# SPDX-License-Identifier: GPL-2.0
-allegro-objs := allegro-core.o allegro-mail.o
-allegro-objs += nal-rbsp.o nal-h264.o nal-hevc.o
-
-obj-$(CONFIG_VIDEO_ALLEGRO_DVT) += allegro.o
+obj-y += zynqmp/
diff --git a/drivers/media/platform/allegro-dvt/zynqmp/Kconfig b/drivers/media/platform/allegro-dvt/zynqmp/Kconfig
new file mode 100644
index 0000000000000000000000000000000000000000..0a0a697c420da47f87f05153a2dbfbe5d3ccf988
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/zynqmp/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config VIDEO_ALLEGRO_DVT
+ tristate "Allegro DVT Video IP Core for ZynqMP"
+ depends on V4L_MEM2MEM_DRIVERS
+ depends on VIDEO_DEV
+ depends on ARCH_ZYNQMP || COMPILE_TEST
+ select V4L2_MEM2MEM_DEV
+ select VIDEOBUF2_DMA_CONTIG
+ select REGMAP_MMIO
+ help
+ Support for the encoder video IP core by Allegro DVT. This core is
+ found for example on the Xilinx ZynqMP SoC in the EV family and is
+ called VCU in the reference manual.
+
+ To compile this driver as a module, choose M here: the module
+ will be called allegro.
diff --git a/drivers/media/platform/allegro-dvt/zynqmp/Makefile b/drivers/media/platform/allegro-dvt/zynqmp/Makefile
new file mode 100644
index 0000000000000000000000000000000000000000..66108a3037747020d549bc0a427881e0667a3f0a
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/zynqmp/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+allegro-objs := allegro-core.o allegro-mail.o
+allegro-objs += nal-rbsp.o nal-h264.o nal-hevc.o
+
+obj-$(CONFIG_VIDEO_ALLEGRO_DVT) += allegro.o
diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/zynqmp/allegro-core.c
similarity index 100%
rename from drivers/media/platform/allegro-dvt/allegro-core.c
rename to drivers/media/platform/allegro-dvt/zynqmp/allegro-core.c
diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.c b/drivers/media/platform/allegro-dvt/zynqmp/allegro-mail.c
similarity index 100%
rename from drivers/media/platform/allegro-dvt/allegro-mail.c
rename to drivers/media/platform/allegro-dvt/zynqmp/allegro-mail.c
diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.h b/drivers/media/platform/allegro-dvt/zynqmp/allegro-mail.h
similarity index 100%
rename from drivers/media/platform/allegro-dvt/allegro-mail.h
rename to drivers/media/platform/allegro-dvt/zynqmp/allegro-mail.h
diff --git a/drivers/media/platform/allegro-dvt/nal-h264.c b/drivers/media/platform/allegro-dvt/zynqmp/nal-h264.c
similarity index 100%
rename from drivers/media/platform/allegro-dvt/nal-h264.c
rename to drivers/media/platform/allegro-dvt/zynqmp/nal-h264.c
diff --git a/drivers/media/platform/allegro-dvt/nal-h264.h b/drivers/media/platform/allegro-dvt/zynqmp/nal-h264.h
similarity index 100%
rename from drivers/media/platform/allegro-dvt/nal-h264.h
rename to drivers/media/platform/allegro-dvt/zynqmp/nal-h264.h
diff --git a/drivers/media/platform/allegro-dvt/nal-hevc.c b/drivers/media/platform/allegro-dvt/zynqmp/nal-hevc.c
similarity index 100%
rename from drivers/media/platform/allegro-dvt/nal-hevc.c
rename to drivers/media/platform/allegro-dvt/zynqmp/nal-hevc.c
diff --git a/drivers/media/platform/allegro-dvt/nal-hevc.h b/drivers/media/platform/allegro-dvt/zynqmp/nal-hevc.h
similarity index 100%
rename from drivers/media/platform/allegro-dvt/nal-hevc.h
rename to drivers/media/platform/allegro-dvt/zynqmp/nal-hevc.h
diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.c b/drivers/media/platform/allegro-dvt/zynqmp/nal-rbsp.c
similarity index 100%
rename from drivers/media/platform/allegro-dvt/nal-rbsp.c
rename to drivers/media/platform/allegro-dvt/zynqmp/nal-rbsp.c
diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.h b/drivers/media/platform/allegro-dvt/zynqmp/nal-rbsp.h
similarity index 100%
rename from drivers/media/platform/allegro-dvt/nal-rbsp.h
rename to drivers/media/platform/allegro-dvt/zynqmp/nal-rbsp.h
--
2.30.2
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP
2025-06-05 12:26 [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 1/4] media: allegro-dvt: Move the current driver to a subdirectory Yassine Ouaissa via B4 Relay
@ 2025-06-05 12:26 ` Yassine Ouaissa via B4 Relay
2025-06-05 13:01 ` Krzysztof Kozlowski
2025-06-12 12:42 ` Michael Tretter
2025-06-05 12:26 ` [PATCH v2 3/4] dt-bindings: vendor-prefixes: Update the description of allegro prefix Yassine Ouaissa via B4 Relay
` (2 subsequent siblings)
4 siblings, 2 replies; 13+ messages in thread
From: Yassine Ouaissa via B4 Relay @ 2025-06-05 12:26 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Michael Tretter, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Yassine OUAISSA, Nicolas Dufresne
Cc: linux-kernel, linux-media, linux-arm-kernel, devicetree
From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
Add compatible for video decoder on allegrodvt Gen 3 IP.
v2:
- Change the YAML file name, use the existing vendor-prefix.
- Improuve the dt-bindings description.
- Change the device compatible identifier, from "allegrodvt, al300-vdec",
to "allegro, al300-vdec"
- Simplify the register property specification,
by using the simple min/max items constraint (Krzysztof Kozlowski)
- Remove the clock-names property. And remove it from the required
properties list (Krzysztof Kozlowski) (Conor Dooley)
- Use the simple maxItems constraint for the memory-region property.
Also for the firmware-name (Krzysztof Kozlowski)
- Example changes:
- Use header provides definitions for the interrupts (Conor Dooley)
- Improuve Interrupt specification using GIC constants (Conor Dooley)
- Use generic node name "video-decoder" (Krzysztof Kozlowski) (Conor Dooley)
- Remove unused label (Krzysztof Kozlowski)
- Change clock reference from <&mcu_clock_dec> to <&mcu_core_clk>
- Use hex format for reg property (Krzysztof Kozlowski) (Conor Dooley)
- Reduce memory region size (Krzysztof Kozlowski) (Conor Dooley)
- Link v1: https://patchwork.linuxtv.org/project/linux-media/patch/20250511144752.504162-4-yassine.ouaissa@allegrodvt.com/
Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
---
.../bindings/media/allegro,al300-vdec.yaml | 75 ++++++++++++++++++++++
MAINTAINERS | 2 +
2 files changed, 77 insertions(+)
diff --git a/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..26f9ac39682431b1d4828aed5d1ed43ef099e204
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
@@ -0,0 +1,75 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/allegro,al300-vdec.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Allegro DVT Video IP Decoder Gen 3
+
+maintainers:
+ - Yassine OUAISSA <yassine.ouaissa@allegrodvt.com>
+
+description: The al300-vdec represents the gen 3 of Allegro DVT IP video
+ decoding technology, offering significant advancements over its
+ predecessors. This new decoder features enhanced processing capabilities
+ with improved throughput and reduced latency.
+
+ Communication between the host driver software and the MCU is implemented
+ through a specialized mailbox interface mechanism. This mailbox system
+ provides a structured channel for exchanging commands, parameters, and
+ status information between the host CPU and the MCU controlling the codec
+ engines.
+
+properties:
+ compatible:
+ const: allegro,al300-vdec
+
+ reg:
+ maxItems: 2
+ minItems: 2
+
+ reg-names:
+ items:
+ - const: regs
+ - const: apb
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ items:
+ - description: MCU core clock
+
+ memory-region:
+ maxItems: 1
+
+ firmware-name:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - reg-names
+ - interrupts
+ - clocks
+
+additionalProperties: False
+
+examples:
+ - |
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ axi {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ video-decoder@a0120000 {
+ compatible = "allegro,al300-vdec";
+ reg = <0x00 0xa0120000 0x00 0x10000>,
+ <0x01 0x80000000 0x00 0x8000>;
+ reg-names = "regs", "apb";
+ interrupts = <GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&mcu_core_clk>;
+ firmware-name = "al300_vdec.fw";
+ };
+ };
diff --git a/MAINTAINERS b/MAINTAINERS
index abc6ba61048771303bc219102f2db602266b7c30..1ff78b9a76cb8cdf32263fcd9b4579b4a2bb6b2a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -816,7 +816,9 @@ M: Michael Tretter <m.tretter@pengutronix.de>
R: Pengutronix Kernel Team <kernel@pengutronix.de>
L: linux-media@vger.kernel.org
S: Maintained
+F: Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
F: Documentation/devicetree/bindings/media/allegro,al5e.yaml
+F: drivers/media/platform/allegro-dvt/al300
F: drivers/media/platform/allegro-dvt/zynqmp
ALLIED VISION ALVIUM CAMERA DRIVER
--
2.30.2
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v2 3/4] dt-bindings: vendor-prefixes: Update the description of allegro prefix
2025-06-05 12:26 [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 1/4] media: allegro-dvt: Move the current driver to a subdirectory Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP Yassine Ouaissa via B4 Relay
@ 2025-06-05 12:26 ` Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
2025-06-05 12:57 ` [PATCH v2 0/4] media: " Krzysztof Kozlowski
4 siblings, 0 replies; 13+ messages in thread
From: Yassine Ouaissa via B4 Relay @ 2025-06-05 12:26 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Michael Tretter, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Yassine OUAISSA, Nicolas Dufresne
Cc: linux-kernel, linux-media, linux-arm-kernel, devicetree
From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
Add SAS (Société par actions simplifiée) to the allegro of vendor
prefixe description to include French simplified joint-stock company
legal structure.
Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
---
Documentation/devicetree/bindings/vendor-prefixes.yaml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/devicetree/bindings/vendor-prefixes.yaml b/Documentation/devicetree/bindings/vendor-prefixes.yaml
index c01adbaacbbbcafa3e174153cdc5d2ca28cf703e..02e70a31477d742c606d63228b97037375442e49 100644
--- a/Documentation/devicetree/bindings/vendor-prefixes.yaml
+++ b/Documentation/devicetree/bindings/vendor-prefixes.yaml
@@ -82,7 +82,7 @@ patternProperties:
"^alfa-network,.*":
description: ALFA Network Inc.
"^allegro,.*":
- description: Allegro DVT
+ description: Allegro DVT, SAS.
"^allegromicro,.*":
description: Allegro MicroSystems, Inc.
"^alliedvision,.*":
--
2.30.2
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver
2025-06-05 12:26 [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
` (2 preceding siblings ...)
2025-06-05 12:26 ` [PATCH v2 3/4] dt-bindings: vendor-prefixes: Update the description of allegro prefix Yassine Ouaissa via B4 Relay
@ 2025-06-05 12:26 ` Yassine Ouaissa via B4 Relay
2025-06-12 13:30 ` Michael Tretter
2025-06-05 12:57 ` [PATCH v2 0/4] media: " Krzysztof Kozlowski
4 siblings, 1 reply; 13+ messages in thread
From: Yassine Ouaissa via B4 Relay @ 2025-06-05 12:26 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Michael Tretter, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Yassine OUAISSA, Nicolas Dufresne
Cc: linux-kernel, linux-media, linux-arm-kernel, devicetree
From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
This commit introduces a new allegro-dvt V4L2 stateful decoder driverfor
the Gen 3 IP with support for:
- AVC (H.264), HEVC (H.265), and JPEG decoding
- Output formats: NV12, NV16, I420, and P010 for capture
v2:
- Replace the mutex_(lock/unlock) with the guard(mutex), that manage
mutexes more efficiently.
- Set DMA_BIT_MASK to 39, and drop the paddr check when allocating
dma_memory.
- Use dma_coerce_mask_and_coherent to set the DMA_MASK.
- Use static initializer for some structs.
- use #ifdef instead of #if defined
- Optimize some function.
- Use the declaration in the loop.
- Use codec for al_codec_dev instead of dev, to not get confused with
the device struct.
- Remove the codec member of the al_codec_dev, use the fmt->pixelformat
when request creating decoder instance.
Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
---
drivers/media/platform/allegro-dvt/Kconfig | 1 +
drivers/media/platform/allegro-dvt/Makefile | 1 +
drivers/media/platform/allegro-dvt/al300/Kconfig | 23 +
drivers/media/platform/allegro-dvt/al300/Makefile | 6 +
.../platform/allegro-dvt/al300/al_codec_common.c | 733 ++++++++++
.../platform/allegro-dvt/al300/al_codec_common.h | 248 ++++
.../platform/allegro-dvt/al300/al_codec_util.c | 174 +++
.../platform/allegro-dvt/al300/al_codec_util.h | 186 +++
.../media/platform/allegro-dvt/al300/al_vdec_drv.c | 1518 ++++++++++++++++++++
.../media/platform/allegro-dvt/al300/al_vdec_drv.h | 93 ++
10 files changed, 2983 insertions(+)
diff --git a/drivers/media/platform/allegro-dvt/Kconfig b/drivers/media/platform/allegro-dvt/Kconfig
index e9008614c27b9490d1cd29fab887977a1918ede4..0d01ed0ad08ab3bf63fc6bc60ac6c8ad9b31c9ab 100644
--- a/drivers/media/platform/allegro-dvt/Kconfig
+++ b/drivers/media/platform/allegro-dvt/Kconfig
@@ -2,4 +2,5 @@
comment "Allegro DVT media platform drivers"
+source "drivers/media/platform/allegro-dvt/al300/Kconfig"
source "drivers/media/platform/allegro-dvt/zynqmp/Kconfig"
diff --git a/drivers/media/platform/allegro-dvt/Makefile b/drivers/media/platform/allegro-dvt/Makefile
index d2aa6875edcf7760901996aac4d5ac98282cce20..c70ca19a47fb7a50a568b37ce519bbedbefe670d 100644
--- a/drivers/media/platform/allegro-dvt/Makefile
+++ b/drivers/media/platform/allegro-dvt/Makefile
@@ -1,3 +1,4 @@
# SPDX-License-Identifier: GPL-2.0
+obj-y += al300/
obj-y += zynqmp/
diff --git a/drivers/media/platform/allegro-dvt/al300/Kconfig b/drivers/media/platform/allegro-dvt/al300/Kconfig
new file mode 100644
index 0000000000000000000000000000000000000000..0bc3d7a79f14038a4f497f736b14a7fc7cca0aeb
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/Kconfig
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config VIDEO_ALLEGRO_DVT_D300
+ tristate "Allegro DVT Video IP Decode Gen 3"
+ depends on V4L_MEM2MEM_DRIVERS
+ depends on VIDEO_DEV && OF && HAS_DMA
+ select V4L2_MEM2MEM_DEV
+ select VIDEOBUF2_DMA_CONTIG
+ help
+ This is a video4linux2 driver for the Allegro DVT IP Decode Gen 3,
+ that support codecs : AVC (H.264), HEVC (H.265), and JPEG.
+
+ The driver provides hardware acceleration for video decoding operations,
+ enabling efficient processing of compressed video streams on platforms
+ featuring this IP block. It handles memory management, buffer allocation,
+ and decoder command sequencing to deliver optimized performance.
+
+ The driver integrates with the V4L2 framework and videobuf2 subsystem
+ to provide a standard interface for applications requiring video
+ decoding capabilities.
+
+ To compile this driver as a module, choose M here. The module
+ will be called al300-vdec.
diff --git a/drivers/media/platform/allegro-dvt/al300/Makefile b/drivers/media/platform/allegro-dvt/al300/Makefile
new file mode 100644
index 0000000000000000000000000000000000000000..3c50caccb731e98163bf6b693dcb6f98e7d20dfa
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/Makefile
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+
+al300-vdec-objs := al_codec_common.o al_codec_util.o
+al300-vdec-objs += al_vdec_drv.o
+
+obj-$(CONFIG_VIDEO_ALLEGRO_DVT_D300) += al300-vdec.o
diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_common.c b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
new file mode 100644
index 0000000000000000000000000000000000000000..716d0004482702537ea89ec4abecd6af26654b32
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
@@ -0,0 +1,733 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Core MCU functionality including firmware loading,
+ * memory allocation, and general MCU interaction interfaces
+ *
+ * Copyright (c) 2025 Allegro DVT.
+ * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
+ */
+
+#include <linux/cleanup.h>
+#include <linux/clk.h>
+#include <linux/dma-mapping.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include "al_codec_common.h"
+
+#define AL_CODEC_UID 0x0000
+#define AL_CODEC_RESET 0x0010
+#define AL_CODEC_IRQ_MASK 0x0014
+#define AL_CODEC_IRQ_STATUS_CLEAR 0x0018
+#define AL_CODEC_MCU_CLK 0x0400
+#define AL_CODEC_MCU_RST 0x0404
+#define AL_CODEC_MCU_IRQ 0x040C
+#define AL_CODEC_MCU_BOOT_ADDR_HI 0x0410
+#define AL_CODEC_MCU_BOOT_ADDR_LO 0x0414
+#define AL_CODEC_MCU_IP_START_ADDR_HI 0x0418
+#define AL_CODEC_MCU_IP_START_ADDR_LO 0x041C
+#define AL_CODEC_MCU_IP_END_ADDR_HI 0x0420
+#define AL_CODEC_MCU_IP_END_ADDR_LO 0x0424
+#define AL_CODEC_MCU_PERIPH_ADDR_HI 0x0428
+#define AL_CODEC_MCU_PERIPH_ADDR_LO 0x042C
+#define AL_CODEC_MCU_IRQ_MASK 0x0440
+#define AL_CODEC_INST_OFFSET_HI 0x0450
+#define AL_CODEC_INST_OFFSET_LO 0x0454
+#define AL_CODEC_DATA_OFFSET_HI 0x0458
+#define AL_CODEC_DATA_OFFSET_LO 0x045C
+
+#define AL_CODEC_UID_ID 0x30AB6E51
+#define AL_CODEC_IRQ_MCU_2_CPU BIT(30)
+#define AL_CODEC_IP_OFFSET GENMASK(26, 25)
+#define AL_CODEC_APB_MASK GENMASK(26, 0)
+
+#define AL_CODEC_MCU_BOOT_RESET_WAIT 2 /* in ms */
+#define AL_CODEC_REG_ENABLE BIT(0)
+#define AL_CODEC_REG_DISABLE 0
+
+/*
+ * struct codec_dma_buf - Allocated dma buffer
+ *
+ * @list: list head for buffer queue
+ * @paddr: physical address of the allcated DMA buffer
+ * @vaddr: virtual address of the allocated DMA buffer
+ * @size: Size of allocated dma memory
+ */
+struct codec_dma_buf {
+ void *vaddr;
+ dma_addr_t paddr;
+ u32 size;
+ struct list_head list;
+};
+
+struct mb_header {
+ u64 start;
+ u64 end;
+} __packed;
+
+struct boot_header {
+ u32 bh_version;
+ u32 fw_version;
+ char model[16];
+ u64 vaddr_start;
+ u64 vaddr_end;
+ u64 vaddr_boot;
+ struct mb_header h2m;
+ struct mb_header m2h;
+ u64 machine_id;
+ /* fill by driver before fw boot */
+ u64 ip_start;
+ u64 ip_end;
+ u64 mcu_clk_rate;
+} __packed;
+
+static u32 al_common_read(struct al_codec_dev *codec, u32 offset)
+{
+ return readl(codec->regs + offset);
+}
+
+static void al_common_write(struct al_codec_dev *codec, u32 offset, u32 val)
+{
+ writel(val, codec->regs + offset);
+}
+
+static void al_common_trigger_mcu_irq(void *arg)
+{
+ struct al_codec_dev *codec = arg;
+
+ al_common_write(codec, AL_CODEC_MCU_IRQ, BIT(0));
+}
+
+static inline void al_common_reset(struct al_codec_dev *codec)
+{
+ /* reset ip */
+ al_common_write(codec, AL_CODEC_RESET, AL_CODEC_REG_ENABLE);
+
+ /* reset and stop mcu */
+ al_common_write(codec, AL_CODEC_MCU_CLK, AL_CODEC_REG_ENABLE);
+ al_common_write(codec, AL_CODEC_MCU_RST, AL_CODEC_REG_ENABLE);
+ /* time to reset the mct */
+ mdelay(AL_CODEC_MCU_BOOT_RESET_WAIT);
+ al_common_write(codec, AL_CODEC_MCU_CLK, AL_CODEC_REG_DISABLE);
+
+ al_common_write(codec, AL_CODEC_MCU_IRQ, AL_CODEC_REG_DISABLE);
+ al_common_write(codec, AL_CODEC_MCU_IRQ_MASK, AL_CODEC_REG_DISABLE);
+
+ mdelay(AL_CODEC_MCU_BOOT_RESET_WAIT * 5);
+ al_common_write(codec, AL_CODEC_MCU_RST, AL_CODEC_REG_DISABLE);
+}
+
+static int al_common_setup_hw_regs(struct al_codec_dev *codec)
+{
+ u64 reg_start, reg_end;
+ dma_addr_t boot_addr;
+ unsigned int id;
+
+ id = al_common_read(codec, AL_CODEC_UID);
+
+ if (id != AL_CODEC_UID_ID) {
+ al_codec_err(codec,
+ "bad device id, expected 0x%08x, got 0x%08x",
+ AL_CODEC_UID_ID, id);
+ return -ENODEV;
+ }
+
+ boot_addr = codec->firmware.phys + codec->firmware.bin_data.offset;
+
+ /* Reset MCU step */
+ al_common_reset(codec);
+
+ /* Configure the MCU*/
+ al_common_write(codec, AL_CODEC_IRQ_MASK, AL_CODEC_IRQ_MCU_2_CPU);
+ /* Set Instruction and data offset */
+ al_common_write(codec, AL_CODEC_INST_OFFSET_HI, 0);
+ al_common_write(codec, AL_CODEC_INST_OFFSET_LO, 0);
+ al_common_write(codec, AL_CODEC_DATA_OFFSET_HI, 0);
+ al_common_write(codec, AL_CODEC_DATA_OFFSET_LO, 0);
+
+ reg_start = codec->regs_info->start;
+ reg_end = reg_start + resource_size(codec->regs_info);
+ al_common_write(codec, AL_CODEC_MCU_IP_START_ADDR_HI,
+ upper_32_bits(reg_start));
+ al_common_write(codec, AL_CODEC_MCU_IP_START_ADDR_LO,
+ lower_32_bits(reg_start));
+ al_common_write(codec, AL_CODEC_MCU_IP_END_ADDR_HI,
+ upper_32_bits(reg_end));
+ al_common_write(codec, AL_CODEC_MCU_IP_END_ADDR_HI,
+ lower_32_bits(reg_end));
+
+ al_common_write(codec, AL_CODEC_MCU_PERIPH_ADDR_HI,
+ upper_32_bits(codec->apb));
+ al_common_write(codec, AL_CODEC_MCU_PERIPH_ADDR_LO,
+ lower_32_bits(codec->apb));
+
+ al_common_write(codec, AL_CODEC_MCU_BOOT_ADDR_HI,
+ upper_32_bits(boot_addr));
+ al_common_write(codec, AL_CODEC_MCU_BOOT_ADDR_LO,
+ lower_32_bits(boot_addr));
+
+ return 0;
+}
+
+static void al_common_dma_buf_insert(struct al_codec_dev *codec,
+ struct codec_dma_buf *buf)
+{
+ guard(mutex)(&codec->buf_lock);
+ list_add(&buf->list, &codec->alloc_buffers);
+}
+
+static void al_common_dma_buf_remove(struct al_codec_dev *codec,
+ struct codec_dma_buf *buf)
+{
+ struct device *dev = &codec->pdev->dev;
+
+ guard(mutex)(&codec->buf_lock);
+ dma_free_coherent(dev, buf->size, buf->vaddr, buf->paddr);
+ list_del(&buf->list);
+ kfree(buf);
+}
+
+static struct codec_dma_buf *
+al_common_dma_buf_lookup(struct al_codec_dev *codec, dma_addr_t buf_paddr)
+{
+ struct codec_dma_buf *buf = NULL;
+
+ guard(mutex)(&codec->buf_lock);
+ list_for_each_entry(buf, &codec->alloc_buffers, list)
+ if (likely(buf->paddr == buf_paddr))
+ break;
+
+ return list_entry_is_head(buf, &codec->alloc_buffers, list) ? NULL :
+ buf;
+}
+
+static void al_common_dma_buf_cleanup(struct al_codec_dev *codec)
+{
+ struct device *dev = &codec->pdev->dev;
+ struct codec_dma_buf *buf, *tmp;
+
+ guard(mutex)(&codec->buf_lock);
+ list_for_each_entry_safe(buf, tmp, &codec->alloc_buffers, list) {
+ dma_free_coherent(dev, buf->size, buf->vaddr, buf->paddr);
+ list_del(&buf->list);
+ kfree(buf);
+ }
+}
+
+static int al_common_setup_dma(struct al_codec_dev *codec)
+{
+ struct device *dev = &codec->pdev->dev;
+ int ret;
+
+ /* setup dma memory mask */
+ if (!dev->coherent_dma_mask) {
+ ret = dma_coerce_mask_and_coherent(dev, DMA_BIT_MASK(39));
+ if (ret) {
+ al_codec_err(codec, "Failed to set dma mask %d\n", ret);
+ return ret;
+ }
+ }
+
+ /* Try to use reserved memory if we got one */
+ ret = of_reserved_mem_device_init(dev);
+ if (ret && ret != ENODEV)
+ dev_warn(dev, "No reserved memory, use cma instead\n");
+
+ return 0;
+}
+
+void al_common_remove(struct al_codec_dev *codec)
+{
+ /* Cleanup allocated internal buffers used by the mcu*/
+ al_common_dma_buf_cleanup(codec);
+
+ /* reset codecice */
+ al_common_reset(codec);
+ clk_disable_unprepare(codec->clk);
+ /* Free the allocated memory for the firmware */
+ dma_free_coherent(&codec->pdev->dev, codec->firmware.size,
+ codec->firmware.virt, codec->firmware.phys);
+
+ if (codec->firmware.firmware)
+ release_firmware(codec->firmware.firmware);
+}
+
+static void handle_alloc_memory_req(struct al_codec_dev *codec,
+ struct msg_itf_header *hdr)
+{
+ struct device *dev = &codec->pdev->dev;
+ struct msg_itf_alloc_mem_req req;
+ struct msg_itf_alloc_mem_reply_full reply = {
+ .reply.phyAddr = 0,
+ .hdr.type = MSG_ITF_TYPE_ALLOC_MEM_REPLY,
+ .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
+ .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
+ .hdr.payload_len = sizeof(struct msg_itf_alloc_mem_reply),
+ };
+ struct codec_dma_buf *buf;
+ int ret;
+
+ ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
+ if (ret) {
+ al_codec_err(codec, "Unable to get cma req %d", ret);
+ return;
+ }
+
+ buf = kmalloc(sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ goto send_reply;
+
+ buf->size = req.uSize;
+ buf->vaddr =
+ dma_alloc_coherent(dev, buf->size, &buf->paddr, GFP_KERNEL);
+ if (!buf->vaddr) {
+ dev_warn(dev, "Failed to allocate DMA buffer\n");
+ goto send_reply;
+ }
+
+ reply.reply.phyAddr = (u64)buf->paddr;
+ al_common_dma_buf_insert(codec, buf);
+
+send_reply:
+ ret = al_common_send(codec, &reply.hdr);
+ if (ret) {
+ al_codec_err(codec, "Unable to reply to cma alloc");
+ al_common_dma_buf_remove(codec, buf);
+ }
+}
+
+static void handle_free_memory_req(struct al_codec_dev *codec,
+ struct msg_itf_header *hdr)
+{
+ struct msg_itf_free_mem_req req;
+ struct msg_itf_free_mem_reply_full reply = {
+ .hdr.type = MSG_ITF_TYPE_FREE_MEM_REPLY,
+ .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
+ .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
+ .hdr.payload_len = sizeof(struct msg_itf_free_mem_reply),
+ .reply.ret = -1,
+ };
+ struct codec_dma_buf *buf;
+ int ret;
+
+ ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
+ if (ret) {
+ al_codec_err(codec, "Unable to put cma req");
+ return;
+ }
+
+ buf = al_common_dma_buf_lookup(codec, req.phyAddr);
+ if (!buf) {
+ al_codec_err(codec, "Unable to get dma handle for %p",
+ (void *)(long)req.phyAddr);
+ reply.reply.ret = -EINVAL;
+ goto send_reply;
+ }
+
+ al_codec_dbg(codec, "Free memory %p, size %d",
+ (void *)(long)req.phyAddr, buf->size);
+
+ al_common_dma_buf_remove(codec, buf);
+ reply.reply.ret = 0;
+
+send_reply:
+ ret = al_common_send(codec, &reply.hdr);
+ if (ret)
+ al_codec_err(codec, "Unable to reply to cma free");
+}
+
+static void handle_mcu_console_print(struct al_codec_dev *codec,
+ struct msg_itf_header *hdr)
+{
+#ifdef DEBUG
+ struct msg_itf_write_req *req;
+ int ret;
+
+ /* one more byte to be sure to have a zero terminated string */
+ req = kzalloc(hdr->payload_len + 1, GFP_KERNEL);
+ if (!req) {
+ al_common_skip_data(codec, hdr->payload_len);
+ al_codec_err(codec, "Unable to alloc memory");
+ return;
+ }
+
+ ret = al_codec_msg_get_data(&codec->mb_m2h, (char *)req,
+ hdr->payload_len);
+ if (ret) {
+ al_codec_err(codec, "Unable to get request");
+ kfree(req);
+ return;
+ }
+
+ /* Print the mcu logs */
+ dev_dbg(&codec->pdev->dev, "[ALG_MCU] %s(),%d: %s\n", __func__,
+ __LINE__, (char *)(req + 1));
+ kfree(req);
+#else
+ al_common_skip_data(codec, hdr->payload_len);
+#endif
+}
+
+static void process_one_message(struct al_codec_dev *codec,
+ struct msg_itf_header *hdr)
+{
+ switch (hdr->type) {
+ case MSG_ITF_TYPE_ALLOC_MEM_REQ:
+ handle_alloc_memory_req(codec, hdr);
+ break;
+ case MSG_ITF_TYPE_FREE_MEM_REQ:
+ handle_free_memory_req(codec, hdr);
+ break;
+ case MSG_ITF_TYPE_WRITE_REQ:
+ handle_mcu_console_print(codec, hdr);
+ break;
+ case MSG_ITF_TYPE_MCU_ALIVE:
+ complete(&codec->completion);
+ break;
+ default:
+ codec->process_msg_cb(codec->cb_arg, hdr);
+ break;
+ }
+}
+
+static irqreturn_t al_common_irq_handler(int irq, void *data)
+{
+ struct al_codec_dev *codec = data;
+ struct msg_itf_header hdr;
+
+ /* poll all messages */
+ while (al_codec_msg_get_header(&codec->mb_m2h, &hdr) == 0)
+ process_one_message(codec, &hdr);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t al_common_hardirq_handler(int irq, void *data)
+{
+ struct al_codec_dev *codec = data;
+
+ if (al_common_read(codec, AL_CODEC_IRQ_STATUS_CLEAR) == 0)
+ return IRQ_NONE;
+
+ al_common_write(codec, AL_CODEC_IRQ_STATUS_CLEAR,
+ AL_CODEC_IRQ_MCU_2_CPU);
+
+ return IRQ_WAKE_THREAD;
+}
+
+static int al_common_start_fw(struct al_codec_dev *codec)
+{
+ /* Enable the MCU clock */
+ al_common_write(codec, AL_CODEC_MCU_CLK, AL_CODEC_REG_ENABLE);
+
+ return !wait_for_completion_timeout(&codec->completion, 2 * HZ);
+}
+
+static void al_common_copy_firmware_image(struct al_codec_dev *codec)
+{
+ const struct firmware *firmware = codec->firmware.firmware;
+ u32 *virt = codec->firmware.virt;
+
+ /* copy the whole thing taking into account endianness */
+ for (size_t i = 0; i < firmware->size / sizeof(u32); i++)
+ virt[i] = le32_to_cpu(((__le32 *)firmware->data)[i]);
+}
+
+static int al_common_read_firmware(struct al_codec_dev *codec, const char *name)
+{
+ struct device *dev = &codec->pdev->dev;
+ const struct boot_header *bh;
+ int err;
+
+ /* request_firmware prints error if it fails */
+ err = request_firmware(&codec->firmware.firmware, name, dev);
+ if (err < 0)
+ return err;
+
+ bh = (struct boot_header *)codec->firmware.firmware->data;
+ codec->firmware.size = bh->vaddr_end - bh->vaddr_start;
+
+ return 0;
+}
+
+static int al_common_parse_firmware_image(struct al_codec_dev *codec)
+{
+ struct boot_header *bh = (void *)codec->firmware.virt;
+
+ if (bh->bh_version < AL_BOOT_VERSION(2, 0, 0) ||
+ bh->bh_version >= AL_BOOT_VERSION(3, 0, 0)) {
+ al_codec_err(codec, "Unsupported firmware version");
+ return -EINVAL;
+ }
+
+ codec->firmware.bin_data.offset = bh->vaddr_boot - bh->vaddr_start;
+ codec->firmware.bin_data.size = bh->vaddr_end - bh->vaddr_start;
+
+ codec->firmware.mb_h2m.offset = bh->h2m.start - bh->vaddr_start;
+ codec->firmware.mb_h2m.size = bh->h2m.end - bh->h2m.start;
+ codec->firmware.mb_m2h.offset = bh->m2h.start - bh->vaddr_start;
+ codec->firmware.mb_m2h.size = bh->m2h.end - bh->m2h.start;
+
+ /* Override some data */
+ bh->ip_start = codec->apb + AL_CODEC_IP_OFFSET;
+ bh->ip_end = bh->ip_start + resource_size(codec->regs_info);
+ bh->mcu_clk_rate = clk_get_rate(codec->clk);
+
+ al_codec_dbg(codec, "bh version = 0x%08x", bh->bh_version);
+ al_codec_dbg(codec, "fw version = 0x%08x", bh->fw_version);
+ al_codec_dbg(codec, "fw model = %s", bh->model);
+ al_codec_dbg(codec, "vaddress start = 0x%016llx", bh->vaddr_start);
+ al_codec_dbg(codec, "vaddress end = 0x%016llx", bh->vaddr_end);
+ al_codec_dbg(codec, "boot address = 0x%016llx", bh->vaddr_boot);
+ al_codec_dbg(codec, "machineid = %lld", bh->machine_id);
+ al_codec_dbg(codec, "periph address = 0x%016llx", codec->apb);
+ al_codec_dbg(codec, "ip start = 0x%016llx", bh->ip_start);
+ al_codec_dbg(codec, "ip end = 0x%016llx", bh->ip_end);
+ al_codec_dbg(codec, "mcu clk = %llu", bh->mcu_clk_rate);
+
+ return 0;
+}
+
+static int al_common_load_firmware_start(struct al_codec_dev *codec,
+ const char *name)
+{
+ struct device *dev = &codec->pdev->dev;
+ dma_addr_t phys;
+ size_t size;
+ void *virt;
+ int err;
+
+ if (codec->firmware.virt)
+ return 0;
+
+ err = al_common_read_firmware(codec, name);
+ if (err)
+ return err;
+
+ size = codec->firmware.size;
+
+ virt = dma_alloc_coherent(dev, size, &phys, GFP_KERNEL);
+ err = dma_mapping_error(dev, phys);
+ if (err < 0)
+ return err;
+
+ codec->firmware.virt = virt;
+ codec->firmware.phys = phys;
+
+ al_common_copy_firmware_image(codec);
+ err = al_common_parse_firmware_image(codec);
+ if (err) {
+ al_codec_err(codec, "failed to parse firmware image");
+ goto cleanup;
+ }
+
+ err = al_common_setup_hw_regs(codec);
+ if (err) {
+ al_codec_err(codec, "Unable to setup hw registers");
+ goto cleanup;
+ }
+
+ al_codec_mb_init(&codec->mb_h2m, virt + codec->firmware.mb_h2m.offset,
+ codec->firmware.mb_h2m.size, MB_IFT_MAGIC_H2M);
+
+ al_codec_mb_init(&codec->mb_m2h, virt + codec->firmware.mb_m2h.offset,
+ codec->firmware.mb_m2h.size, MB_IFT_MAGIC_M2H);
+
+ err = al_common_start_fw(codec);
+ if (err) {
+ al_codec_err(codec, "fw start has failed");
+ goto cleanup;
+ }
+
+ al_codec_dbg(codec, "mcu has boot successfully !");
+ codec->fw_ready_cb(codec->cb_arg);
+
+ release_firmware(codec->firmware.firmware);
+ codec->firmware.firmware = NULL;
+
+ return 0;
+
+cleanup:
+ dma_free_coherent(dev, size, virt, phys);
+
+ return err;
+}
+
+static u64 al_common_get_periph_addr(struct al_codec_dev *codec)
+{
+ struct resource *res;
+
+ res = platform_get_resource_byname(codec->pdev, IORESOURCE_MEM, "apb");
+ if (!res) {
+ al_codec_err(codec, "Unable to find APB start address");
+ return 0;
+ }
+
+ if (res->start & AL_CODEC_APB_MASK) {
+ al_codec_err(codec, "APB start address is invalid");
+ return 0;
+ }
+
+ return res->start;
+}
+
+int al_common_probe(struct al_codec_dev *codec, const char *name)
+{
+ struct platform_device *pdev = codec->pdev;
+ int irq;
+ int ret;
+
+ mutex_init(&codec->buf_lock);
+ INIT_LIST_HEAD(&codec->alloc_buffers);
+ init_completion(&codec->completion);
+
+ /* setup dma memory */
+ ret = al_common_setup_dma(codec);
+ if (ret)
+ return ret;
+
+ /* Hw registers */
+ codec->regs_info =
+ platform_get_resource_byname(pdev, IORESOURCE_MEM, "regs");
+ if (!codec->regs_info) {
+ al_codec_err(codec, "regs resource missing from device tree");
+ return -EINVAL;
+ }
+
+ codec->regs = devm_ioremap_resource(&pdev->dev, codec->regs_info);
+ if (!codec->regs) {
+ al_codec_err(codec, "failed to map registers");
+ return -ENOMEM;
+ }
+
+ codec->apb = al_common_get_periph_addr(codec);
+ if (!codec->apb)
+ return -EINVAL;
+
+ /* The MCU has already default clock value */
+ codec->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(codec->clk)) {
+ al_codec_err(codec, "failed to get MCU core clock");
+ return PTR_ERR(codec->clk);
+ }
+
+ ret = clk_prepare_enable(codec->clk);
+ if (ret) {
+ al_codec_err(codec, "Cannot enable MCU clock: %d\n", ret);
+ return ret;
+ }
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0) {
+ al_codec_err(codec, "Failed to get IRQ");
+ ret = -EINVAL;
+ goto disable_clk;
+ }
+
+ ret = devm_request_threaded_irq(&pdev->dev, irq,
+ al_common_hardirq_handler,
+ al_common_irq_handler, IRQF_SHARED,
+ dev_name(&pdev->dev), codec);
+ if (ret) {
+ al_codec_err(codec, "Unable to register irq handler");
+ goto disable_clk;
+ }
+
+ /* ok so request the fw */
+ ret = al_common_load_firmware_start(codec, name);
+ if (ret) {
+ al_codec_err(codec, "failed to load firmware : %s", name);
+ goto disable_clk;
+ }
+
+ return 0;
+
+disable_clk:
+ clk_disable_unprepare(codec->clk);
+
+ return ret;
+}
+
+int al_common_send(struct al_codec_dev *codec, struct msg_itf_header *hdr)
+{
+ return al_codec_msg_send(&codec->mb_h2m, hdr, al_common_trigger_mcu_irq,
+ codec);
+}
+
+int al_common_send_req_reply(struct al_codec_dev *codec,
+ struct list_head *cmd_list,
+ struct msg_itf_header *hdr,
+ struct al_common_mcu_req *req)
+{
+ struct al_codec_cmd *cmd = NULL;
+ int ret;
+
+ hdr->drv_cmd_hdl = 0;
+
+ if (req->reply_size && req->reply) {
+ cmd = al_codec_cmd_create(req->reply_size);
+ if (!cmd)
+ return -ENOMEM;
+
+ hdr->drv_cmd_hdl = al_virt_to_phys(cmd);
+ }
+
+ hdr->drv_ctx_hdl = req->pCtx;
+ hdr->type = req->req_type;
+ hdr->payload_len = req->req_size;
+
+ /* Add the list to the cmd list */
+ if (cmd)
+ list_add(&cmd->list, cmd_list);
+
+ ret = al_common_send(codec, hdr);
+ if (ret)
+ goto remove_cmd;
+
+ al_codec_dbg(codec, "Send req to mcu %d : %ld ", req->req_type,
+ req->req_size);
+
+ if (!cmd)
+ return 0;
+
+ ret = wait_for_completion_timeout(&cmd->done, 5 * HZ);
+ if (ret <= 0) {
+ al_codec_err(codec, "cmd %p has %d (%s)", cmd, ret,
+ (ret == 0) ? "failed" : "timedout");
+ ret = -ETIMEDOUT;
+ goto remove_cmd;
+ }
+
+ ret = 0;
+ memcpy(req->reply, cmd->reply, req->reply_size);
+
+remove_cmd:
+
+ if (cmd) {
+ list_del(&cmd->list);
+ al_codec_cmd_put(cmd);
+ }
+ return ret;
+}
+
+bool al_common_mcu_is_alive(struct al_codec_dev *codec)
+{
+ static const struct msg_itf_header hdr = {
+ .type = MSG_ITF_TYPE_MCU_ALIVE,
+ .payload_len = 0,
+ };
+ int ret;
+
+ ret = al_common_send(codec, (struct msg_itf_header *)&hdr);
+ if (ret)
+ return false;
+
+ ret = wait_for_completion_timeout(&codec->completion, 5 * HZ);
+ if (ret <= 0)
+ return false;
+
+ return true;
+}
diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_common.h b/drivers/media/platform/allegro-dvt/al300/al_codec_common.h
new file mode 100644
index 0000000000000000000000000000000000000000..b739c29e9a544671adb1961f291024aa2bd008c0
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/al_codec_common.h
@@ -0,0 +1,248 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2025 Allegro DVT.
+ * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
+ */
+
+#ifndef __AL_CODEC_COMMON__
+#define __AL_CODEC_COMMON__
+
+#include <linux/cleanup.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <media/v4l2-device.h>
+
+#include "al_codec_util.h"
+
+#define fh_to_ctx(ptr, type) container_of(ptr, type, fh)
+
+enum {
+ MSG_ITF_TYPE_CREATE_INST_REQ = MSG_ITF_TYPE_NEXT_REQ,
+ MSG_ITF_TYPE_DESTROY_INST_REQ,
+ MSG_ITF_TYPE_PUSH_BITSTREAM_BUFFER_REQ,
+ MSG_ITF_TYPE_PUT_DISPLAY_PICTURE_REQ,
+ MSG_ITF_TYPE_FLUSH_REQ,
+ MSG_ITF_TYPE_INFO_REQ,
+ MSG_ITF_TYPE_CREATE_INST_REPLY = MSG_ITF_TYPE_NEXT_REPLY,
+ MSG_ITF_TYPE_DESTROY_INST_REPLY,
+ MSG_ITF_TYPE_PUSH_BITSTREAM_BUFFER_REPLY,
+ MSG_ITF_TYPE_PUT_DISPLAY_PICTURE_REPLY,
+ MSG_ITF_TYPE_FLUSH_REPLY,
+ MSG_ITF_TYPE_INFO_REPLY,
+ MSG_ITF_TYPE_EVT_ERROR = MSG_ITF_TYPE_NEXT_EVT,
+};
+
+struct msg_itf_write_req {
+ u32 fd;
+ u32 len;
+ /* payload follow */
+} __packed;
+DECLARE_FULL_REQ(msg_itf_write_req);
+
+struct msg_itf_free_mem_req {
+ phys_addr_t phyAddr;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_free_mem_req);
+
+struct msg_itf_alloc_mem_req {
+ u64 uSize;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_alloc_mem_req);
+
+struct msg_itf_alloc_mem_reply {
+ phys_addr_t phyAddr;
+} __packed;
+DECLARE_FULL_REPLY(msg_itf_alloc_mem_reply);
+
+struct msg_itf_free_mem_reply {
+ s64 ret;
+};
+DECLARE_FULL_REPLY(msg_itf_free_mem_reply);
+
+struct msg_itf_create_codec_reply {
+ phys_addr_t hCodec;
+ s32 ret;
+} __packed;
+DECLARE_FULL_REPLY(msg_itf_create_codec_reply);
+
+struct msg_itf_destroy_codec_req {
+ phys_addr_t hCodec;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_destroy_codec_req);
+
+/*
+ * Note : no need to know the status of this request
+ * The codec should be destroyed, in case of the mcu
+ * hasn't received any request with the codec handler
+ */
+struct msg_itf_destroy_codec_reply {
+ u32 unused;
+} __packed;
+DECLARE_FULL_REPLY(msg_itf_destroy_codec_reply);
+
+struct al_buffer_meta {
+ u64 timestamp;
+ struct v4l2_timecode timecode;
+ bool last;
+};
+
+struct msg_itf_push_src_buf_req {
+ phys_addr_t hCodec;
+ phys_addr_t bufferHandle;
+ phys_addr_t phyAddr;
+ u64 size;
+ struct al_buffer_meta meta;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_push_src_buf_req);
+
+struct msg_itf_push_dst_buf_req {
+ phys_addr_t hCodec;
+ phys_addr_t bufferHandle;
+ phys_addr_t phyAddr;
+ u64 size;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_push_dst_buf_req);
+
+struct msg_itf_push_buffer_req {
+ phys_addr_t hCodec;
+ phys_addr_t bufferHandle;
+ phys_addr_t phyAddr;
+ u64 size;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_push_buffer_req);
+
+struct msg_itf_push_buffer_reply {
+ s32 res;
+} __packed;
+DECLARE_FULL_REPLY(msg_itf_push_buffer_reply);
+
+struct msg_itf_info_req {
+ u64 unused;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_info_req);
+
+struct msg_itf_flush_req {
+ phys_addr_t hCodec;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_flush_req);
+
+struct msg_itf_flush_reply {
+ int32_t unused;
+} __packed;
+DECLARE_FULL_REPLY(msg_itf_flush_reply);
+
+struct msg_itf_evt_error {
+ uint32_t errno;
+} __packed;
+DECLARE_FULL_EVENT(msg_itf_evt_error);
+
+struct al_match_data {
+ const char *fw_name;
+};
+
+struct al_common_mcu_req {
+ phys_addr_t pCtx;
+ int req_type;
+ size_t req_size;
+ size_t reply_size;
+ void *reply;
+} __packed;
+
+struct al_firmware_section {
+ u64 offset;
+ size_t size;
+};
+
+struct al_firmware {
+ /* Firmware after it is read but not loaded */
+ const struct firmware *firmware;
+
+ /* Raw firmware data */
+ dma_addr_t phys;
+ void *virt;
+ size_t size;
+
+ /* Parsed firmware information */
+ struct al_firmware_section bin_data;
+ struct al_firmware_section mb_m2h;
+ struct al_firmware_section mb_h2m;
+};
+
+struct al_codec_dev {
+ struct platform_device *pdev;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_m2m_dev *m2m_dev;
+ struct video_device video_dev;
+
+ /* Firmware */
+ struct al_firmware firmware;
+ dma_addr_t apb;
+
+ struct clk *clk;
+ void __iomem *regs;
+ struct resource *regs_info;
+
+ /* Mailbox structs */
+ struct al_codec_mb mb_h2m;
+ struct al_codec_mb mb_m2h;
+
+ /* list of buffers used by the MCU */
+ struct list_head alloc_buffers;
+ struct mutex buf_lock;
+
+ /* mutex protecting vb2_queue structure */
+ struct mutex lock;
+
+ /* list of ctx (aka decoder) */
+ struct mutex ctx_mlock;
+ struct list_head ctx_q_list;
+ int is_video_init_done;
+
+ /* list of cap/out supported formats */
+ struct list_head codec_q_list;
+ struct al_codec_cmd *codec_info_cmd;
+
+ /* Command completion */
+ struct completion completion;
+ /* Resolution found completion */
+ struct completion res_done;
+
+ /* callbacks set by client before common_probe */
+ void *cb_arg;
+ void (*process_msg_cb)(void *cb_arg, struct msg_itf_header *hdr);
+ void (*fw_ready_cb)(void *cb_arg);
+};
+
+static inline int al_common_get_header(struct al_codec_dev *codec,
+ struct msg_itf_header *hdr)
+{
+ return al_codec_msg_get_header(&codec->mb_m2h, hdr);
+}
+
+static inline int al_common_get_data(struct al_codec_dev *codec, char *data,
+ int len)
+{
+ return al_codec_msg_get_data(&codec->mb_m2h, data, len);
+}
+
+static inline int al_common_skip_data(struct al_codec_dev *codec, int len)
+{
+ return al_common_get_data(codec, NULL, len);
+}
+
+int al_common_send(struct al_codec_dev *codec, struct msg_itf_header *hdr);
+int al_common_send_req_reply(struct al_codec_dev *codec,
+ struct list_head *cmd_list,
+ struct msg_itf_header *hdr,
+ struct al_common_mcu_req *req);
+bool al_common_mcu_is_alive(struct al_codec_dev *codec);
+
+int al_common_probe(struct al_codec_dev *codec, const char *name);
+void al_common_remove(struct al_codec_dev *codec);
+
+#endif /*__AL_CODEC_COMMON__*/
diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_util.c b/drivers/media/platform/allegro-dvt/al300/al_codec_util.c
new file mode 100644
index 0000000000000000000000000000000000000000..99483b040623d44be9b710a6c60beed8a3a97c3c
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/al_codec_util.c
@@ -0,0 +1,174 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Mailbox communication utilities for command creation
+ * and message exchange with the MCU
+ *
+ * Copyright (c) 2025 Allegro DVT.
+ * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
+ */
+
+#include <asm-generic/errno.h>
+#include <linux/errno.h>
+#include <linux/jiffies.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+
+#include "al_codec_util.h"
+
+static int al_get_used_space(struct al_codec_mb *mb)
+{
+ u32 head = mb->hdr->head;
+ u32 tail = mb->hdr->tail;
+
+ return head >= tail ? head - tail : mb->size - (tail - head);
+}
+
+static int al_get_free_space(struct al_codec_mb *mb)
+{
+ return mb->size - al_get_used_space(mb) - 1;
+}
+
+static int al_has_enough_space(struct al_codec_mb *mb, int len)
+{
+ return al_get_free_space(mb) >= len;
+}
+
+static inline void al_copy_to_mb(struct al_codec_mb *mb, char *data, int len)
+{
+ u32 head = mb->hdr->head;
+ int copy_len = min(mb->size - head, (unsigned int)len);
+ int copied_len = len;
+
+ memcpy(&mb->data[head], data, copy_len);
+ len -= copy_len;
+ if (len)
+ memcpy(&mb->data[0], &data[copy_len], len);
+
+ /* Make sure that all messages are written before updating the head */
+ dma_wmb();
+ mb->hdr->head = (head + copied_len) % mb->size;
+ /* Make sure that the head is updated in DDR instead of cache */
+ dma_wmb();
+}
+
+static inline void al_copy_from_mb(struct al_codec_mb *mb, char *data, int len)
+{
+ u32 tail = mb->hdr->tail;
+ int copy_len = min(mb->size - tail, (unsigned int)len);
+ int copied_len = len;
+
+ if (!data)
+ goto update_tail;
+
+ memcpy(data, &mb->data[tail], copy_len);
+ len -= copy_len;
+ if (len)
+ memcpy(&data[copy_len], &mb->data[0], len);
+
+update_tail:
+ mb->hdr->tail = (tail + copied_len) % mb->size;
+ /* Make sure that the head is updated in DDR instead of cache */
+ dma_wmb();
+}
+
+static int al_codec_mb_send(struct al_codec_mb *mb, char *data, int len)
+{
+ if (!al_has_enough_space(mb, len))
+ return -ENOMEM;
+
+ al_copy_to_mb(mb, data, len);
+
+ return 0;
+}
+
+static int al_codec_mb_receive(struct al_codec_mb *mb, char *data, int len)
+{
+ if (al_get_used_space(mb) < len)
+ return -ENOMEM;
+
+ al_copy_from_mb(mb, data, len);
+
+ return 0;
+}
+
+void al_codec_mb_init(struct al_codec_mb *mb, char *addr, int size, u32 magic)
+{
+ mb->hdr = (struct al_mb_itf *)addr;
+ mb->hdr->magic = magic;
+ mb->hdr->version = MB_IFT_VERSION;
+ mb->hdr->head = 0;
+ mb->hdr->tail = 0;
+ mb->data = addr + sizeof(struct al_mb_itf);
+ mb->size = size - sizeof(struct al_mb_itf);
+ mutex_init(&mb->lock);
+}
+
+int al_codec_msg_get_header(struct al_codec_mb *mb, struct msg_itf_header *hdr)
+{
+ return al_codec_mb_receive(mb, (char *)hdr, sizeof(*hdr));
+}
+
+int al_codec_msg_get_data(struct al_codec_mb *mb, char *data, int len)
+{
+ return al_codec_mb_receive(mb, data, len);
+}
+
+int al_codec_msg_send(struct al_codec_mb *mb, struct msg_itf_header *hdr,
+ void (*trigger)(void *), void *trigger_arg)
+{
+ const unsigned long timeout = jiffies + HZ;
+ int ret;
+
+ guard(mutex)(&mb->lock);
+ do {
+ if (time_after(jiffies, timeout))
+ return -ETIMEDOUT;
+
+ ret = al_codec_mb_send(mb, (char *)hdr,
+ hdr->payload_len +
+ sizeof(struct msg_itf_header));
+
+ } while (ret);
+
+ trigger(trigger_arg);
+
+ return 0;
+}
+
+static void al_codec_cmd_cleanup(struct kref *ref)
+{
+ struct al_codec_cmd *cmd = container_of(ref, typeof(*cmd), refcount);
+
+ kfree(cmd->reply);
+ kfree(cmd);
+}
+
+void al_codec_cmd_put(struct al_codec_cmd *cmd)
+{
+ if (WARN_ON(!cmd))
+ return;
+
+ kref_put(&cmd->refcount, al_codec_cmd_cleanup);
+}
+
+struct al_codec_cmd *al_codec_cmd_create(int reply_size)
+{
+ struct al_codec_cmd *cmd;
+
+ cmd = kmalloc(sizeof(*cmd), GFP_KERNEL);
+ if (!cmd)
+ return NULL;
+
+ cmd->reply = kmalloc(reply_size, GFP_KERNEL);
+ if (!cmd->reply) {
+ kfree(cmd);
+ return NULL;
+ }
+
+ kref_init(&cmd->refcount);
+ cmd->reply_size = reply_size;
+ init_completion(&cmd->done);
+
+ return cmd;
+}
diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_util.h b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
new file mode 100644
index 0000000000000000000000000000000000000000..5f893db4a1a3f2b9e6e9109b81a956bcaa71851c
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
@@ -0,0 +1,186 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2025 Allegro DVT.
+ * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
+ */
+
+#ifndef __AL_CODEC_UTIL__
+#define __AL_CODEC_UTIL__
+
+#include <linux/mutex.h>
+#include <linux/types.h>
+#include <linux/v4l2-common.h>
+
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-v4l2.h>
+
+#define MB_IFT_MAGIC_H2M 0xabcd1230
+#define MB_IFT_MAGIC_M2H 0xabcd1231
+#define MB_IFT_VERSION 0x00010000
+
+#define MAJOR_SHIFT 20
+#define MAJOR_MASK 0xfff
+#define MINOR_SHIFT 8
+#define MINOR_MASK 0xfff
+#define PATCH_SHIFT 0
+#define PATCH_MASK 0xff
+
+/*
+ * AL_BOOT_VERSION() - Version format 32-bit, 12 bits for the major,
+ * the same for minor, 8bits for the patch
+ */
+#define AL_BOOT_VERSION(major, minor, patch) \
+ ((((major) & MAJOR_MASK) << MAJOR_SHIFT) | \
+ (((minor) & MINOR_MASK) << MINOR_SHIFT) | \
+ (((patch) & PATCH_MASK) << PATCH_SHIFT))
+
+#define al_phys_to_virt(x) ((void *)(uintptr_t)x)
+#define al_virt_to_phys(x) ((phys_addr_t)(uintptr_t)x)
+
+#define DECLARE_FULL_REQ(s) \
+ struct s##_full { \
+ struct msg_itf_header hdr; \
+ struct s req; \
+ } __packed
+
+#define DECLARE_FULL_REPLY(s) \
+ struct s##_full { \
+ struct msg_itf_header hdr; \
+ struct s reply; \
+ } __packed
+
+#define DECLARE_FULL_EVENT(s) \
+ struct s##_full { \
+ struct msg_itf_header hdr; \
+ struct s event; \
+ } __packed
+
+struct al_mb_itf {
+ u32 magic;
+ u32 version;
+ u32 head;
+ u32 tail;
+} __packed;
+
+struct al_codec_mb {
+ struct al_mb_itf *hdr;
+ struct mutex lock;
+ char *data;
+ int size;
+};
+
+struct al_codec_cmd {
+ struct kref refcount;
+ struct list_head list;
+ struct completion done;
+ int reply_size;
+ void *reply;
+};
+
+#define al_codec_err(codec, fmt, args...) \
+ dev_err(&(codec)->pdev->dev, "[ALG_CODEC][ERROR] %s():%d: " fmt "\n", \
+ __func__, __LINE__, ##args)
+
+#define al_v4l2_err(codec, fmt, args...) \
+ dev_err(&(codec)->pdev->dev, "[ALG_V4L2][ERROR] %s():%d: " fmt "\n", \
+ __func__, __LINE__, ##args)
+
+#if defined(DEBUG)
+
+extern int debug;
+
+/* V4L2 logs */
+#define al_v4l2_dbg(codec, level, fmt, args...) \
+ do { \
+ if (debug >= level) \
+ dev_dbg(&(codec)->pdev->dev, \
+ "[ALG_V4L2] level=%d %s(),%d: " fmt "\n", \
+ level, __func__, __LINE__, ##args); \
+ } while (0)
+
+/* Codec logs */
+#define al_codec_dbg(codec, fmt, args...) \
+ do { \
+ if (debug) \
+ dev_dbg(&(codec)->pdev->dev, \
+ "[ALG_CODEC] %s(),%d: " fmt "\n", __func__, \
+ __LINE__, ##args); \
+ } while (0)
+
+#else
+
+#define al_v4l2_dbg(codec, level, fmt, args...) \
+ do { \
+ (void)level; \
+ dev_dbg(&(codec)->pdev->dev, "[ALG_V4L2]: " fmt "\n", \
+ ##args); \
+ } while (0)
+
+#define al_codec_dbg(codec, fmt, args...) \
+ dev_dbg(&(codec)->pdev->dev, "[ALG_CODEC]: " fmt "\n", ##args)
+
+#endif
+
+#define MSG_ITF_TYPE_LIMIT BIT(10)
+
+/* Message types host <-> mcu */
+enum {
+ MSG_ITF_TYPE_MCU_ALIVE = 0,
+ MSG_ITF_TYPE_WRITE_REQ = 2,
+ MSG_ITF_TYPE_FIRST_REQ = 1024,
+ MSG_ITF_TYPE_NEXT_REQ,
+ MSG_ITF_TYPE_FIRST_REPLY = 2048,
+ MSG_ITF_TYPE_NEXT_REPLY,
+ MSG_ITF_TYPE_ALLOC_MEM_REQ = 3072,
+ MSG_ITF_TYPE_FREE_MEM_REQ,
+ MSG_ITF_TYPE_ALLOC_MEM_REPLY = 4096,
+ MSG_ITF_TYPE_FREE_MEM_REPLY,
+ MSG_ITF_TYPE_FIRST_EVT = 5120,
+ MSG_ITF_TYPE_NEXT_EVT = MSG_ITF_TYPE_FIRST_EVT
+};
+
+struct msg_itf_header {
+ u64 drv_ctx_hdl;
+ u64 drv_cmd_hdl;
+ u16 type;
+ u16 payload_len;
+ u16 padding[2];
+} __packed;
+
+void al_codec_mb_init(struct al_codec_mb *mb, char *addr, int size, u32 magic);
+int al_codec_msg_get_header(struct al_codec_mb *mb, struct msg_itf_header *hdr);
+int al_codec_msg_get_data(struct al_codec_mb *mb, char *data, int len);
+int al_codec_msg_send(struct al_codec_mb *mb, struct msg_itf_header *hdr,
+ void (*trigger)(void *), void *trigger_arg);
+
+static inline bool is_type_reply(uint16_t type)
+{
+ return type >= MSG_ITF_TYPE_FIRST_REPLY &&
+ type < MSG_ITF_TYPE_FIRST_REPLY + MSG_ITF_TYPE_LIMIT;
+}
+
+static inline bool is_type_event(uint16_t type)
+{
+ return type >= MSG_ITF_TYPE_FIRST_EVT &&
+ type < MSG_ITF_TYPE_FIRST_EVT + MSG_ITF_TYPE_LIMIT;
+}
+
+void al_codec_cmd_put(struct al_codec_cmd *cmd);
+
+struct al_codec_cmd *al_codec_cmd_create(int reply_size);
+
+static inline struct al_codec_cmd *al_codec_cmd_get(struct list_head *cmd_list,
+ uint64_t hdl)
+{
+ struct al_codec_cmd *cmd = NULL;
+
+ list_for_each_entry(cmd, cmd_list, list) {
+ if (likely(cmd == al_phys_to_virt(hdl))) {
+ kref_get(&cmd->refcount);
+ break;
+ }
+ }
+ return list_entry_is_head(cmd, cmd_list, list) ? NULL : cmd;
+}
+
+#endif /* __AL_CODEC_UTIL__ */
diff --git a/drivers/media/platform/allegro-dvt/al300/al_vdec_drv.c b/drivers/media/platform/allegro-dvt/al300/al_vdec_drv.c
new file mode 100644
index 0000000000000000000000000000000000000000..bb8f768328b449575b877508e86ffcca92008085
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/al_vdec_drv.c
@@ -0,0 +1,1518 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (c) 2025 Allegro DVT.
+ * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
+ *
+ * Allegro DVT stateful video decoder driver for the IP Gen 3
+ */
+
+#include <asm-generic/errno-base.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/string.h>
+#include <linux/v4l2-controls.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "al_codec_common.h"
+#include "al_vdec_drv.h"
+
+#if defined(DEBUG)
+/* Log level */
+int debug;
+module_param(debug, int, 0644);
+MODULE_PARM_DESC(debug, "Debug level (0-3)");
+#endif
+
+/* default decoder params */
+#define DECODER_WIDTH_DEFAULT 640
+#define DECODER_HEIGHT_DEFAULT 480
+#define DECODER_WIDTH_MAX 3840
+#define DECODER_HEIGHT_MAX 2160
+#define DECODER_WIDTH_MIN 16
+#define DECODER_HEIGHT_MIN 16
+#define DEC_REQ_TIMEOUT msecs_to_jiffies(1000)
+#define DEC_RES_EVT_TIMEOUT DEC_REQ_TIMEOUT
+
+/* Supported formats */
+static const struct al_fmt al_src_formats[] = {
+ {
+ .pixelformat = V4L2_PIX_FMT_H264,
+ .bpp = 20,
+ },
+ {
+ .pixelformat = V4L2_PIX_FMT_HEVC,
+ .bpp = 20,
+ },
+ {
+ .pixelformat = V4L2_PIX_FMT_JPEG,
+ .bpp = 8,
+ }
+};
+
+static const struct al_fmt al_dst_formats[] = {
+ {
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .bpp = 12,
+ },
+ {
+ .pixelformat = V4L2_PIX_FMT_P010,
+ .bpp = 12,
+ },
+ {
+ .pixelformat = V4L2_PIX_FMT_NV16,
+ .bpp = 16,
+ },
+ {
+ .pixelformat = V4L2_PIX_FMT_YUV420, /* YUV 4:2:0 */
+ .bpp = 12,
+ },
+ {
+ .pixelformat = V4L2_PIX_FMT_YVU420, /* YVU 4:2:0 */
+ .bpp = 12,
+ },
+};
+
+/* Default format */
+static const struct al_frame al_default_fmt = {
+
+ .width = DECODER_WIDTH_DEFAULT,
+ .height = DECODER_HEIGHT_DEFAULT,
+ .bytesperline = DECODER_WIDTH_MAX * 4,
+ .sizeimage = DECODER_WIDTH_DEFAULT * DECODER_HEIGHT_DEFAULT * 4,
+ .nbuffers = 1,
+ .fmt = &al_dst_formats[0],
+ .field = V4L2_FIELD_NONE,
+ .colorspace = V4L2_COLORSPACE_REC709,
+ .ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT,
+ .quantization = V4L2_QUANTIZATION_DEFAULT,
+ .xfer_func = V4L2_XFER_FUNC_DEFAULT
+};
+
+static struct al_frame *al_get_frame(struct al_dec_ctx *ctx,
+ enum v4l2_buf_type type)
+{
+ if (WARN_ON(!ctx))
+ return ERR_PTR(-EINVAL);
+
+ if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ return &ctx->src;
+ else if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return &ctx->dst;
+
+ al_v4l2_err(ctx->codec, "Unsupported type (%d)", type);
+
+ return ERR_PTR(-EINVAL);
+}
+
+static const struct al_fmt *al_find_fmt(u32 pixelformat)
+{
+ const struct al_fmt *fmt;
+ unsigned int i;
+
+ /* check if the pixelformat exist in the src formats list */
+ for (i = 0; i < ARRAY_SIZE(al_src_formats); i++) {
+ fmt = &al_src_formats[i];
+ if (fmt->pixelformat == pixelformat)
+ return fmt;
+ }
+
+ /* check if the pixelformat exist in the dst formats list */
+ for (i = 0; i < ARRAY_SIZE(al_dst_formats); i++) {
+ fmt = &al_dst_formats[i];
+ if (fmt->pixelformat == pixelformat)
+ return fmt;
+ }
+
+ return NULL;
+}
+
+static int dec_fw_create_decoder(struct al_dec_ctx *ctx)
+{
+ struct msg_itf_create_decoder_req_full req;
+ struct msg_itf_create_codec_reply reply;
+ const struct al_frame *frame = &ctx->src;
+ struct al_common_mcu_req mreq = {
+ .pCtx = al_virt_to_phys(ctx),
+ .req_type = MSG_ITF_TYPE_CREATE_INST_REQ,
+ .req_size = sizeof(req.req),
+ .reply_size = sizeof(reply),
+ .reply = &reply,
+ };
+ int ret;
+
+ if (ctx->hDec) {
+ al_v4l2_dbg(ctx->codec, 3, "fw decoder already exist\n");
+ return 0;
+ }
+
+ req.req.codec = frame->fmt->pixelformat;
+
+ ret = al_common_send_req_reply(ctx->codec, &ctx->cmd_q_list, &req.hdr,
+ &mreq);
+
+ if (!ret && !reply.ret)
+ ctx->hDec = reply.hCodec;
+ else if (reply.ret)
+ ret = -ENODEV;
+
+ return ret;
+}
+
+static void dec_fw_destroy_decoder(struct al_dec_ctx *ctx)
+{
+ struct msg_itf_destroy_codec_req_full req;
+ struct msg_itf_destroy_codec_reply reply;
+ struct al_common_mcu_req mreq = {
+ .req_type = MSG_ITF_TYPE_DESTROY_INST_REQ,
+ .pCtx = al_virt_to_phys(ctx),
+ .req_size = sizeof(req.req),
+ .reply_size = sizeof(reply),
+ .reply = &reply,
+ };
+ int ret;
+
+ if (WARN(!ctx->hDec, "NULL Decoder to destroy !"))
+ return;
+
+ al_v4l2_dbg(ctx->codec, 3, "Destroy decoder %lld ", ctx->hDec);
+
+ req.req.hCodec = ctx->hDec;
+
+ ret = al_common_send_req_reply(ctx->codec, &ctx->cmd_q_list, &req.hdr,
+ &mreq);
+
+ if (!ret)
+ ctx->hDec = 0;
+}
+
+static int al_dec_fw_push_frame_buf(struct al_dec_ctx *ctx,
+ struct vb2_v4l2_buffer *vbuf)
+{
+ struct msg_itf_push_dst_buf_req_full req;
+ struct v4l2_m2m_buffer *m2m_buf;
+ struct al_common_mcu_req mreq = {
+ .req_type = MSG_ITF_TYPE_PUT_DISPLAY_PICTURE_REQ,
+ .pCtx = al_virt_to_phys(ctx),
+ .req_size = sizeof(req.req),
+ };
+ int ret;
+
+ if (WARN(!vbuf, "NULL frame Buffer to push!!"))
+ return -EINVAL;
+
+ m2m_buf = container_of(vbuf, struct v4l2_m2m_buffer, vb);
+ req.req.hCodec = ctx->hDec;
+ req.req.bufferHandle = al_virt_to_phys(m2m_buf);
+ req.req.phyAddr = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
+ req.req.size = vb2_plane_size(&vbuf->vb2_buf, 0);
+
+ ret = al_common_send_req_reply(ctx->codec, &ctx->cmd_q_list, &req.hdr,
+ &mreq);
+ if (ret)
+ al_v4l2_err(ctx->codec, "Failed to push frame buffer %p %d",
+ m2m_buf, ret);
+
+ return ret;
+}
+
+static int al_dec_fw_push_bitstream_buf(struct al_dec_ctx *ctx,
+ struct vb2_v4l2_buffer *vbuf)
+{
+ struct msg_itf_push_src_buf_req_full req;
+ struct v4l2_m2m_buffer *m2m_buf;
+ struct al_common_mcu_req mreq = {
+ .req_type = MSG_ITF_TYPE_PUSH_BITSTREAM_BUFFER_REQ,
+ .pCtx = al_virt_to_phys(ctx),
+ .req_size = sizeof(req.req),
+ };
+ int ret;
+
+ if (WARN(!vbuf, "NULL Buffer to push!!"))
+ return -EINVAL;
+
+ m2m_buf = container_of(vbuf, struct v4l2_m2m_buffer, vb);
+ req.req.hCodec = ctx->hDec;
+ req.req.bufferHandle = al_virt_to_phys(m2m_buf);
+ req.req.phyAddr = vb2_dma_contig_plane_dma_addr(&vbuf->vb2_buf, 0);
+ req.req.size = vb2_plane_size(&vbuf->vb2_buf, 0);
+
+ /* Fill the v4l2 metadata*/
+ req.req.meta.timestamp = vbuf->vb2_buf.timestamp;
+ req.req.meta.timecode = vbuf->timecode;
+ req.req.meta.last = vbuf->flags & V4L2_BUF_FLAG_LAST;
+
+ ret = al_common_send_req_reply(ctx->codec, &ctx->cmd_q_list, &req.hdr,
+ &mreq);
+ if (ret)
+ al_v4l2_err(ctx->codec, "Failed to push bitstream buffer %p %d",
+ m2m_buf, ret);
+
+ return ret;
+}
+
+static int dec_fw_flush_req(struct al_dec_ctx *ctx)
+{
+ struct msg_itf_flush_req_full req;
+ struct msg_itf_flush_reply reply;
+ struct al_common_mcu_req mreq = {
+ .req_type = MSG_ITF_TYPE_FLUSH_REQ,
+ .pCtx = al_virt_to_phys(ctx),
+ .req_size = sizeof(req.req),
+ .reply_size = sizeof(reply),
+ .reply = &reply,
+ };
+ int ret;
+
+ req.req.hCodec = ctx->hDec;
+
+ ret = al_common_send_req_reply(ctx->codec, &ctx->cmd_q_list, &req.hdr,
+ &mreq);
+
+ if (ret)
+ al_v4l2_err(ctx->codec, "Failed to flush the decoder %d", ret);
+
+ return ret;
+}
+
+static struct vb2_v4l2_buffer *al_dec_dequeue_buf(struct al_dec_ctx *ctx,
+ uint64_t hdl,
+ struct list_head *buffer_list)
+{
+ struct v4l2_m2m_buffer *buf, *tmp;
+ struct vb2_v4l2_buffer *ret = NULL;
+
+ guard(mutex)(&ctx->buf_q_mlock);
+ list_for_each_entry_safe(buf, tmp, buffer_list, list) {
+ if (buf == al_phys_to_virt(hdl)) {
+ list_del(&buf->list);
+ ret = &buf->vb;
+ break;
+ }
+ }
+
+ return ret;
+}
+
+static struct vb2_v4l2_buffer *al_dec_dequeue_src_buf(struct al_dec_ctx *ctx,
+ uint64_t hdl)
+{
+ return al_dec_dequeue_buf(ctx, hdl, &ctx->stream_q_list);
+}
+
+static struct vb2_v4l2_buffer *al_dec_dequeue_dst_buf(struct al_dec_ctx *ctx,
+ uint64_t hdl)
+{
+ return al_dec_dequeue_buf(ctx, hdl, &ctx->frame_q_list);
+}
+
+static void al_ctx_cleanup(struct kref *ref)
+{
+ struct al_dec_ctx *ctx = container_of(ref, struct al_dec_ctx, refcount);
+
+ kfree(ctx);
+}
+
+static struct al_dec_ctx *al_ctx_get(struct al_codec_dev *codec, uint64_t hdl)
+{
+ struct al_dec_ctx *ctx;
+ struct al_dec_ctx *ret = NULL;
+
+ guard(mutex)(&codec->ctx_mlock);
+ list_for_each_entry(ctx, &codec->ctx_q_list, list) {
+ if (ctx == al_phys_to_virt(hdl)) {
+ kref_get(&ctx->refcount);
+ ret = ctx;
+ break;
+ }
+ }
+
+ return ret;
+}
+
+static void al_ctx_put(struct al_dec_ctx *ctx)
+{
+ kref_put(&ctx->refcount, al_ctx_cleanup);
+}
+
+static int al_dec_start_streaming(struct vb2_queue *q, unsigned int count)
+{
+ struct al_dec_ctx *ctx = vb2_get_drv_priv(q);
+ struct al_codec_dev *codec = ctx->codec;
+
+ v4l2_m2m_update_start_streaming_state(ctx->fh.m2m_ctx, q);
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type)) {
+ struct v4l2_m2m_buffer *buf;
+ int ret;
+
+ if (list_empty(&ctx->stream_q_list)) {
+ al_v4l2_err(codec, "Empty stream list.");
+ return -EINVAL;
+ }
+
+ if (!al_common_mcu_is_alive(codec)) {
+ al_v4l2_err(codec, "Unable to ping the mcu");
+ return -ENODEV;
+ }
+
+ ret = dec_fw_create_decoder(ctx);
+ if (ret) {
+ al_v4l2_err(codec, "Unable to create the fw decoder %d",
+ ret);
+ return ret;
+ }
+
+ /* Get the first vid-out queued buffer */
+ buf = list_first_entry(&ctx->stream_q_list,
+ struct v4l2_m2m_buffer, list);
+
+ if (al_dec_fw_push_bitstream_buf(ctx, &buf->vb)) {
+ al_v4l2_err(codec,
+ "Unable to push the bitstream buffer");
+ return -EINVAL;
+ }
+
+ /* Wait until the mcu detect the resolution of the stream */
+ ret = wait_for_completion_timeout(&ctx->res_done,
+ DEC_RES_EVT_TIMEOUT);
+ if (!ret) {
+ al_v4l2_err(codec, "unsupported stream");
+ ctx->aborting = true;
+ }
+ }
+
+ return 0;
+}
+
+static void al_dec_stop_streaming_cap(struct al_dec_ctx *ctx)
+{
+ struct vb2_v4l2_buffer *vbuf;
+ struct v4l2_m2m_buffer *entry, *tmp;
+
+ mutex_lock(&ctx->buf_q_mlock);
+ if (!list_empty(&ctx->frame_q_list))
+ list_for_each_entry_safe(entry, tmp, &ctx->frame_q_list, list) {
+ list_del(&entry->list);
+ vbuf = &entry->vb;
+ vb2_set_plane_payload(&vbuf->vb2_buf, 0, 0);
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+ }
+ mutex_unlock(&ctx->buf_q_mlock);
+
+ while (v4l2_m2m_num_dst_bufs_ready(ctx->fh.m2m_ctx)) {
+ vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+ if (vbuf) {
+ vb2_set_plane_payload(&vbuf->vb2_buf, 0, 0);
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+ }
+ }
+
+ v4l2_m2m_mark_stopped(ctx->fh.m2m_ctx);
+}
+
+static void al_dec_stop_streaming_out(struct al_dec_ctx *ctx)
+{
+ struct vb2_v4l2_buffer *vbuf;
+ struct v4l2_m2m_buffer *entry, *tmp;
+
+ mutex_lock(&ctx->buf_q_mlock);
+ if (!list_empty(&ctx->stream_q_list))
+ list_for_each_entry_safe(entry, tmp, &ctx->stream_q_list,
+ list) {
+ list_del(&entry->list);
+ v4l2_m2m_buf_done(&entry->vb, VB2_BUF_STATE_ERROR);
+ }
+ mutex_unlock(&ctx->buf_q_mlock);
+
+ if (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx)) {
+ while ((vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx)))
+ if (vbuf->vb2_buf.state == VB2_BUF_STATE_ACTIVE)
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+ }
+
+ dec_fw_destroy_decoder(ctx);
+}
+
+static void al_dec_stop_streaming(struct vb2_queue *q)
+{
+ struct al_dec_ctx *ctx = vb2_get_drv_priv(q);
+
+ v4l2_m2m_update_stop_streaming_state(ctx->fh.m2m_ctx, q);
+
+ /* Releasing the dst and src buffers */
+ ctx->stopped = true;
+
+ if (V4L2_TYPE_IS_OUTPUT(q->type))
+ al_dec_stop_streaming_out(ctx);
+ else
+ al_dec_stop_streaming_cap(ctx);
+}
+
+static int al_dec_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers,
+ unsigned int *nplanes, unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct al_dec_ctx *ctx = vb2_get_drv_priv(vq);
+ struct al_frame *format = al_get_frame(ctx, vq->type);
+
+ if (IS_ERR(format)) {
+ al_v4l2_err(ctx->codec, "Invalid format %p", format);
+ return PTR_ERR(format);
+ }
+
+ if (*nplanes)
+ return ((sizes[0] < format->sizeimage) ? -EINVAL : 0);
+
+ /* update queue num buffers */
+ format->nbuffers = max(*nbuffers, format->nbuffers);
+
+ *nplanes = 1;
+ sizes[0] = format->sizeimage;
+ *nbuffers = format->nbuffers;
+
+ al_v4l2_dbg(ctx->codec, 2, "%s: Get %d buffers of size %d each ",
+ (vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) ? "OUT" : "CAP",
+ *nbuffers, sizes[0]);
+
+ return 0;
+}
+
+static int al_dec_buf_prepare(struct vb2_buffer *vb)
+{
+ struct al_dec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+ if (ctx->aborting)
+ return -EINVAL;
+
+ if (V4L2_TYPE_IS_CAPTURE(vb->type)) {
+ if (vbuf->field == V4L2_FIELD_ANY)
+ vbuf->field = V4L2_FIELD_NONE;
+ if (vbuf->field != V4L2_FIELD_NONE)
+ return -EINVAL;
+ }
+
+ al_v4l2_dbg(ctx->codec, 3, "%s : Buffer (%p) prepared ",
+ (V4L2_TYPE_IS_OUTPUT(vb->type) ? "OUT" : "CAP"), vbuf);
+
+ return 0;
+}
+
+static inline void al_dec_fill_bitstream(struct al_dec_ctx *ctx)
+{
+ struct vb2_v4l2_buffer *src_buf;
+ struct v4l2_m2m_buffer *m2m_buf;
+ struct vb2_queue *src_vq;
+
+ lockdep_assert_held(&ctx->buf_q_mlock);
+
+ if (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) > 0) {
+ src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+ if (!src_buf)
+ return;
+
+ /* Dump empty buffers */
+ if (!vb2_get_plane_payload(&src_buf->vb2_buf, 0)) {
+ src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+ return;
+ }
+
+ src_vq = v4l2_m2m_get_src_vq(ctx->fh.m2m_ctx);
+ src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+
+ if (src_buf) {
+ src_buf->sequence = ctx->osequence++;
+
+ if (vb2_is_streaming(src_vq) &&
+ al_dec_fw_push_bitstream_buf(ctx, src_buf)) {
+ v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+ return;
+ }
+
+ m2m_buf = container_of(src_buf, struct v4l2_m2m_buffer,
+ vb);
+ list_add_tail(&m2m_buf->list, &ctx->stream_q_list);
+ }
+ }
+}
+
+static void al_dec_buf_queue(struct vb2_buffer *vb)
+{
+ struct al_dec_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+ v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->type)) {
+ mutex_lock(&ctx->buf_q_mlock);
+ al_dec_fill_bitstream(ctx);
+ mutex_unlock(&ctx->buf_q_mlock);
+ }
+
+ al_v4l2_dbg(ctx->codec, 3, "%s queued (%p) - (%d)",
+ V4L2_TYPE_IS_OUTPUT(vb->type) ? "OUT" : "CAP", vbuf,
+ vb->num_planes);
+}
+
+static const struct vb2_ops dec_queue_ops = {
+ .queue_setup = al_dec_queue_setup,
+ .buf_prepare = al_dec_buf_prepare,
+ .buf_queue = al_dec_buf_queue,
+ .start_streaming = al_dec_start_streaming,
+ .stop_streaming = al_dec_stop_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+};
+
+static int al_dec_queue_init(void *priv, struct vb2_queue *src_vq,
+ struct vb2_queue *dst_vq)
+{
+ struct al_dec_ctx *ctx = priv;
+ struct al_codec_dev *codec = ctx->codec;
+ int ret;
+
+ src_vq->dev = &codec->pdev->dev;
+ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
+ src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ src_vq->non_coherent_mem = false;
+ src_vq->dma_attrs |= DMA_ATTR_FORCE_CONTIGUOUS;
+ src_vq->mem_ops = &vb2_dma_contig_memops;
+ src_vq->drv_priv = ctx;
+ src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ src_vq->ops = &dec_queue_ops;
+ src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ src_vq->lock = &ctx->codec->lock;
+ ret = vb2_queue_init(src_vq);
+ if (ret)
+ return ret;
+
+ dst_vq->dev = &codec->pdev->dev;
+ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+ dst_vq->non_coherent_mem = false;
+ dst_vq->dma_attrs |= DMA_ATTR_FORCE_CONTIGUOUS;
+ dst_vq->mem_ops = &vb2_dma_contig_memops;
+ dst_vq->drv_priv = ctx;
+ dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ dst_vq->ops = &dec_queue_ops;
+ dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+ dst_vq->lock = &ctx->codec->lock;
+ ret = vb2_queue_init(dst_vq);
+ if (ret) {
+ vb2_queue_release(src_vq);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int al_dec_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ struct al_codec_dev *codec = video_drvdata(file);
+
+ strscpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
+ strscpy(cap->card, "Allegro DVT Video Decoder", sizeof(cap->card));
+ snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+ dev_name(&codec->pdev->dev));
+
+ return 0;
+}
+
+static int al_dec_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f)
+{
+ const struct al_fmt *fmt;
+
+ if (f->type != V4L2_BUF_TYPE_VIDEO_OUTPUT &&
+ f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return -EINVAL;
+
+ if (V4L2_TYPE_IS_OUTPUT(f->type)) {
+ if (f->index >= ARRAY_SIZE(al_src_formats))
+ return -EINVAL;
+
+ fmt = &al_src_formats[f->index];
+ } else {
+ if (f->index >= ARRAY_SIZE(al_dst_formats))
+ return -EINVAL;
+
+ fmt = &al_dst_formats[f->index];
+ }
+
+ f->pixelformat = fmt->pixelformat;
+ return 0;
+}
+
+static int al_dec_try_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct al_dec_ctx *ctx = fh_to_ctx(fh, struct al_dec_ctx);
+ struct v4l2_pix_format *pix = &f->fmt.pix;
+ struct al_frame *pix_fmt;
+
+ pix_fmt = al_get_frame(ctx, f->type);
+ if (IS_ERR(pix_fmt)) {
+ al_v4l2_err(ctx->codec, "Invalid frame (%p)", pix_fmt);
+ return PTR_ERR(pix_fmt);
+ }
+
+ pix_fmt->fmt = al_find_fmt(pix->pixelformat);
+ if (!pix_fmt->fmt) {
+ al_v4l2_err(ctx->codec, "Unknown format 0x%x",
+ pix->pixelformat);
+ return -EINVAL;
+ }
+ pix->field = V4L2_FIELD_NONE;
+ pix->width = clamp_t(__u32, pix->width, DECODER_WIDTH_MIN,
+ DECODER_WIDTH_MAX);
+ pix->height = clamp_t(__u32, pix->height, DECODER_HEIGHT_MIN,
+ DECODER_HEIGHT_MAX);
+
+ pix->bytesperline = pix->width;
+ pix->sizeimage = (pix->width * pix->height * pix_fmt->fmt->bpp) / 8;
+
+ if (V4L2_TYPE_IS_CAPTURE(f->type))
+ if (pix->sizeimage < pix_fmt->sizeimage)
+ pix->sizeimage = pix_fmt->sizeimage;
+
+ al_v4l2_dbg(
+ ctx->codec, 3,
+ "%s : width (%d) , height (%d), bytesperline (%d), sizeimage (%d) ",
+ (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) ? "CAP" : "OUT",
+ pix->width, pix->height, pix->bytesperline, pix->sizeimage);
+
+ return 0;
+}
+
+static int al_dec_g_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct al_dec_ctx *ctx = fh_to_ctx(fh, struct al_dec_ctx);
+ struct al_frame *pix_fmt = al_get_frame(ctx, f->type);
+ struct v4l2_pix_format *pix;
+
+ if (IS_ERR(pix_fmt)) {
+ al_v4l2_err(ctx->codec, "Invalid pixel format %p", pix_fmt);
+ return PTR_ERR(pix_fmt);
+ }
+
+ if (!pix_fmt->fmt) {
+ al_v4l2_err(ctx->codec, "Unknown format for %d", f->type);
+ return -EINVAL;
+ }
+
+ pix = &f->fmt.pix;
+ pix->width = pix_fmt->width;
+ pix->height = pix_fmt->height;
+ pix->bytesperline = pix_fmt->bytesperline;
+ pix->sizeimage = pix_fmt->sizeimage;
+ pix->pixelformat = pix_fmt->fmt->pixelformat;
+ pix->field = V4L2_FIELD_NONE;
+
+ if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ pix->bytesperline = 0;
+
+ pix->ycbcr_enc = pix_fmt->ycbcr_enc;
+ pix->quantization = pix_fmt->quantization;
+ pix->xfer_func = pix_fmt->xfer_func;
+ pix->colorspace = pix_fmt->colorspace;
+
+ al_v4l2_dbg(
+ ctx->codec, 3,
+ "%s : width (%d) , height (%d), bytesperline (%d) , sizeimage (%d)",
+ (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) ? "CAP" : "OUT",
+ pix->width, pix->height, pix->bytesperline, pix->sizeimage);
+
+ return 0;
+}
+
+static int al_dec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
+{
+ struct al_dec_ctx *ctx = fh_to_ctx(fh, struct al_dec_ctx);
+ struct v4l2_pix_format *pix;
+ struct al_frame *frame;
+ struct vb2_queue *vq;
+ int ret;
+
+ ret = al_dec_try_fmt(file, fh, f);
+ if (ret) {
+ al_v4l2_err(ctx->codec, "Cannot set format (%d)", f->type);
+ return ret;
+ }
+
+ frame = (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) ? &ctx->src : &ctx->dst;
+
+ pix = &f->fmt.pix;
+ frame->fmt = al_find_fmt(pix->pixelformat);
+ if (!frame->fmt) {
+ al_v4l2_err(ctx->codec, "Unknown format for %d",
+ pix->pixelformat);
+ return -EINVAL;
+ }
+
+ vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
+ if (vb2_is_streaming(vq)) {
+ al_v4l2_err(ctx->codec, "queue %d busy", f->type);
+ return -EBUSY;
+ }
+
+ frame->width = pix->width;
+ frame->height = pix->height;
+ frame->bytesperline = pix->bytesperline;
+ frame->sizeimage = pix->sizeimage;
+ frame->field = pix->field;
+
+ frame->ycbcr_enc = pix->ycbcr_enc;
+ frame->quantization = pix->quantization;
+ frame->xfer_func = pix->xfer_func;
+ frame->colorspace = pix->colorspace;
+
+ al_v4l2_dbg(
+ ctx->codec, 3,
+ " %s : width (%d) , height (%d), bytesperline (%d), sizeimage (%d)",
+ (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) ? "CAP" : "OUT",
+ pix->width, pix->height, pix->bytesperline, pix->sizeimage);
+
+ return 0;
+}
+
+static void al_queue_eos_event(struct al_dec_ctx *ctx)
+{
+ const struct v4l2_event eos_event = {
+ .id = 0,
+ .type = V4L2_EVENT_EOS,
+ };
+
+ v4l2_event_queue_fh(&ctx->fh, &eos_event);
+}
+
+static void al_queue_res_chg_event(struct al_dec_ctx *ctx)
+{
+ static const struct v4l2_event ev_src_ch = {
+ .id = 0,
+ .type = V4L2_EVENT_SOURCE_CHANGE,
+ .u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
+ };
+
+ v4l2_event_queue_fh(&ctx->fh, &ev_src_ch);
+}
+
+static int al_dec_decoder_cmd(struct file *file, void *fh,
+ struct v4l2_decoder_cmd *dcmd)
+{
+ struct al_dec_ctx *ctx = fh_to_ctx(fh, struct al_dec_ctx);
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ struct vb2_v4l2_buffer *vbuf;
+ struct vb2_queue *dst_vq;
+ int ret;
+
+ ret = v4l2_m2m_ioctl_try_decoder_cmd(file, fh, dcmd);
+ if (ret)
+ return ret;
+
+ /* Get the vb2 queue for the Capture */
+ dst_vq = v4l2_m2m_get_dst_vq(m2m_ctx);
+
+ switch (dcmd->cmd) {
+ case V4L2_DEC_CMD_START:
+ vb2_clear_last_buffer_dequeued(dst_vq);
+ break;
+ case V4L2_DEC_CMD_STOP:
+ vbuf = v4l2_m2m_last_src_buf(m2m_ctx);
+ if (vbuf) {
+ al_v4l2_dbg(ctx->codec, 1,
+ "marking last pending buffer");
+
+ vbuf->flags |= V4L2_BUF_FLAG_LAST;
+ if (v4l2_m2m_num_src_bufs_ready(m2m_ctx) == 0) {
+ al_v4l2_dbg(ctx->codec, 1,
+ "all remaining buffers queued");
+ v4l2_m2m_try_schedule(m2m_ctx);
+ }
+ }
+ dec_fw_flush_req(ctx);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int al_dec_enum_framesizes(struct file *file, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ if (!al_find_fmt(fsize->pixel_format))
+ return -EINVAL;
+
+ /* FIXME : check step size */
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise.min_width = DECODER_WIDTH_MIN;
+ fsize->stepwise.max_width = DECODER_WIDTH_MAX;
+ fsize->stepwise.step_width = 8;
+ fsize->stepwise.min_height = DECODER_HEIGHT_MIN;
+ fsize->stepwise.max_height = DECODER_HEIGHT_MAX;
+ fsize->stepwise.step_height = 8;
+
+ return 0;
+}
+
+static int al_dec_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_EOS:
+ return v4l2_event_subscribe(fh, sub, 0, NULL);
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return v4l2_src_change_event_subscribe(fh, sub);
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int al_dec_log_status(struct file *file, void *fh)
+{
+ struct al_codec_dev *codec = video_drvdata(file);
+
+ v4l2_device_call_all(&codec->v4l2_dev, 0, core, log_status);
+ return 0;
+}
+
+static const struct v4l2_ioctl_ops al_dec_ioctl_ops = {
+ .vidioc_querycap = al_dec_querycap,
+ .vidioc_enum_fmt_vid_cap = al_dec_enum_fmt,
+ .vidioc_enum_fmt_vid_out = al_dec_enum_fmt,
+ .vidioc_g_fmt_vid_cap = al_dec_g_fmt,
+ .vidioc_g_fmt_vid_out = al_dec_g_fmt,
+ .vidioc_try_fmt_vid_cap = al_dec_try_fmt,
+ .vidioc_try_fmt_vid_out = al_dec_try_fmt,
+ .vidioc_s_fmt_vid_cap = al_dec_s_fmt,
+ .vidioc_s_fmt_vid_out = al_dec_s_fmt,
+
+ .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs,
+ .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs,
+
+ .vidioc_expbuf = v4l2_m2m_ioctl_expbuf,
+ .vidioc_querybuf = v4l2_m2m_ioctl_querybuf,
+ .vidioc_qbuf = v4l2_m2m_ioctl_qbuf,
+ .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf,
+ .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf,
+
+ .vidioc_streamon = v4l2_m2m_ioctl_streamon,
+ .vidioc_streamoff = v4l2_m2m_ioctl_streamoff,
+ .vidioc_log_status = al_dec_log_status,
+
+ .vidioc_try_decoder_cmd = v4l2_m2m_ioctl_try_decoder_cmd,
+ .vidioc_decoder_cmd = al_dec_decoder_cmd,
+ .vidioc_enum_framesizes = al_dec_enum_framesizes,
+
+ .vidioc_subscribe_event = al_dec_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static void al_device_run(void *priv)
+{
+ struct al_dec_ctx *ctx = priv;
+ struct vb2_v4l2_buffer *dst_buf;
+ struct v4l2_m2m_buffer *m2m_buf;
+
+ if (unlikely(!ctx))
+ return;
+
+ if (ctx->aborting) {
+ vb2_queue_error(v4l2_m2m_get_src_vq(ctx->fh.m2m_ctx));
+ vb2_queue_error(v4l2_m2m_get_dst_vq(ctx->fh.m2m_ctx));
+ return;
+ }
+
+ if (!v4l2_m2m_num_dst_bufs_ready(ctx->fh.m2m_ctx))
+ goto job_finish;
+
+ dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+ if (!dst_buf)
+ goto job_finish;
+
+ if (!al_common_mcu_is_alive(ctx->codec) ||
+ al_dec_fw_push_frame_buf(ctx, dst_buf)) {
+ vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
+ v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+ goto job_finish;
+ }
+
+ mutex_lock(&ctx->buf_q_mlock);
+ m2m_buf = container_of(dst_buf, struct v4l2_m2m_buffer, vb);
+ list_add_tail(&m2m_buf->list, &ctx->frame_q_list);
+ mutex_unlock(&ctx->buf_q_mlock);
+
+job_finish:
+ v4l2_m2m_job_finish(ctx->codec->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+static const struct v4l2_m2m_ops al_dec_m2m_ops = {
+ .device_run = al_device_run,
+};
+
+static int al_dec_open(struct file *file)
+{
+ struct video_device *vdev = video_devdata(file);
+ struct al_codec_dev *codec = video_get_drvdata(vdev);
+ struct al_dec_ctx *ctx = NULL;
+ int ret;
+
+ if (mutex_lock_interruptible(&codec->ctx_mlock))
+ return -ERESTARTSYS;
+
+ /* Aloocate memory for the dec ctx */
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx) {
+ ret = -ENOMEM;
+ goto unlock;
+ }
+
+ ctx->codec = codec;
+ /* Init ctx mutex */
+ mutex_init(&ctx->buf_q_mlock);
+ /* Init ctx LISTHEADs*/
+ INIT_LIST_HEAD(&ctx->cmd_q_list);
+ INIT_LIST_HEAD(&ctx->frame_q_list);
+ INIT_LIST_HEAD(&ctx->stream_q_list);
+
+ /* Init the irq queue */
+ init_completion(&ctx->res_done);
+
+ v4l2_fh_init(&ctx->fh, vdev);
+
+ v4l2_ctrl_handler_init(&ctx->ctrl_handler, 0);
+ if (ctx->ctrl_handler.error) {
+ ret = ctx->ctrl_handler.error;
+ al_v4l2_err(codec, "Failed to create control %d", ret);
+ goto handler_error;
+ }
+
+ ctx->fh.ctrl_handler = &ctx->ctrl_handler;
+ v4l2_ctrl_handler_setup(&ctx->ctrl_handler);
+
+ file->private_data = &ctx->fh;
+ v4l2_fh_add(&ctx->fh);
+
+ /* Set default formats */
+ ctx->src = ctx->dst = al_default_fmt;
+ ctx->csequence = ctx->osequence = 0;
+
+ ctx->stopped = false;
+ ctx->aborting = false;
+
+ /* Setup the ctx for m2m mode */
+ ctx->fh.m2m_ctx =
+ v4l2_m2m_ctx_init(codec->m2m_dev, ctx, al_dec_queue_init);
+ if (IS_ERR(ctx->fh.m2m_ctx)) {
+ ret = PTR_ERR(ctx->fh.m2m_ctx);
+ al_v4l2_err(codec, "Failed to initialize m2m mode %d", ret);
+ goto error_ctrls;
+ }
+
+ v4l2_m2m_set_src_buffered(ctx->fh.m2m_ctx, true);
+
+ /* Add ctx to the LIST */
+ kref_init(&ctx->refcount);
+ list_add(&ctx->list, &codec->ctx_q_list);
+
+ mutex_unlock(&codec->ctx_mlock);
+
+ return 0;
+
+error_ctrls:
+ v4l2_fh_del(&ctx->fh);
+handler_error:
+ v4l2_ctrl_handler_free(&ctx->ctrl_handler);
+ v4l2_fh_exit(&ctx->fh);
+ kfree(ctx);
+
+unlock:
+ mutex_unlock(&codec->ctx_mlock);
+ return ret;
+}
+
+static int al_dec_release(struct file *file)
+{
+ struct al_dec_ctx *ctx =
+ fh_to_ctx(file->private_data, struct al_dec_ctx);
+ struct al_codec_dev *codec = ctx->codec;
+
+ guard(mutex)(&codec->ctx_mlock);
+
+ /* It is important to do this before removing ctx from dev list.
+ * Those commands will trigger some traffic towards fw and so we
+ * need completion to avoid deadlock if cmds can't find ctx.
+ */
+ v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+ v4l2_ctrl_handler_free(&ctx->ctrl_handler);
+ v4l2_fh_del(&ctx->fh);
+ v4l2_fh_exit(&ctx->fh);
+
+ list_del(&ctx->list);
+ al_ctx_put(ctx);
+
+ return 0;
+}
+
+static inline bool al_mark_last_dst_buf(struct al_dec_ctx *ctx)
+{
+ struct vb2_v4l2_buffer *buf;
+ struct vb2_buffer *dst_vb;
+ struct vb2_queue *dst_vq;
+ unsigned long flags;
+
+ al_v4l2_dbg(ctx->codec, 1, "marking last capture buffer");
+
+ dst_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+ spin_lock_irqsave(&dst_vq->done_lock, flags);
+ if (list_empty(&dst_vq->done_list)) {
+ spin_unlock_irqrestore(&dst_vq->done_lock, flags);
+ return false;
+ }
+
+ dst_vb = list_last_entry(&dst_vq->done_list, struct vb2_buffer,
+ done_entry);
+ buf = to_vb2_v4l2_buffer(dst_vb);
+ buf->flags |= V4L2_BUF_FLAG_LAST;
+
+ spin_unlock_irqrestore(&dst_vq->done_lock, flags);
+ return true;
+}
+
+static const struct v4l2_file_operations al_dec_file_ops = {
+ .owner = THIS_MODULE,
+ .open = al_dec_open,
+ .release = al_dec_release,
+ .poll = v4l2_m2m_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = v4l2_m2m_fop_mmap,
+};
+
+static void handle_error_evt(struct al_dec_ctx *ctx, struct msg_itf_header *hdr)
+{
+ struct al_codec_dev *codec = ctx->codec;
+ struct msg_itf_evt_error evt;
+ struct v4l2_m2m_buffer *vbuf;
+
+ if (al_common_get_data(codec, (char *)&evt, hdr->payload_len)) {
+ al_v4l2_err(codec, "Unable to get resolution found event");
+ return;
+ }
+
+ al_v4l2_err(codec, "Decoding error %d", evt.errno);
+
+ guard(mutex)(&ctx->buf_q_mlock);
+ if (!list_empty(&ctx->stream_q_list)) {
+ vbuf = list_last_entry(&ctx->frame_q_list,
+ struct v4l2_m2m_buffer, list);
+ list_del(&vbuf->list);
+ v4l2_m2m_buf_done(&vbuf->vb, VB2_BUF_STATE_ERROR);
+ }
+}
+
+static void handle_resolution_found_evt(struct al_dec_ctx *ctx,
+ struct msg_itf_header *hdr)
+{
+ struct msg_itf_evt_resolution_found evt;
+ struct al_codec_dev *codec = ctx->codec;
+ struct al_frame *frame;
+ struct vb2_queue *dst_vq;
+
+ if (al_common_get_data(codec, (char *)&evt, hdr->payload_len)) {
+ al_v4l2_err(codec, "Unable to get resolution found event");
+ return;
+ }
+
+ frame = &ctx->dst;
+
+ if (frame->width != evt.width || frame->height != evt.height ||
+ frame->nbuffers < evt.buffer_nb) {
+ /* Update frame properties */
+ frame->width = evt.width;
+ frame->height = evt.height;
+ frame->bytesperline = evt.bytesperline;
+ frame->sizeimage = evt.sizeimage;
+ frame->nbuffers = evt.buffer_nb;
+ frame->fmt = al_find_fmt(evt.pixelformat);
+
+ /* This has to be changed */
+ if (!frame->fmt)
+ return;
+
+ al_queue_res_chg_event(ctx);
+ }
+
+ dst_vq = v4l2_m2m_get_dst_vq(ctx->fh.m2m_ctx);
+ if (!vb2_is_streaming(dst_vq))
+ complete(&ctx->res_done);
+
+ al_v4l2_dbg(
+ codec, 3,
+ "width(%d) , height(%d), bytesperline(%d), sizeimage(%d), n_bufs(%d)",
+ frame->width, frame->height, frame->bytesperline,
+ frame->sizeimage, frame->nbuffers);
+}
+
+static void handle_bitstream_buffer_release_evt(struct al_dec_ctx *ctx,
+ struct msg_itf_header *hdr)
+{
+ struct msg_itf_evt_bitstream_buffer_release evt;
+ struct al_codec_dev *codec = ctx->codec;
+ struct vb2_v4l2_buffer *vbuf;
+
+ if (al_common_get_data(codec, (char *)&evt, hdr->payload_len)) {
+ al_v4l2_err(codec, "Unable to get buffer release event");
+ return;
+ }
+
+ if (ctx->stopped)
+ return;
+
+ vbuf = al_dec_dequeue_src_buf(ctx, evt.bufferHandle);
+ if (!vbuf) {
+ al_v4l2_err(codec, "Unable to find bitsream buffer 0x%llx",
+ evt.bufferHandle);
+ return;
+ }
+
+ al_v4l2_dbg(codec, 3, "Release bitstream buffer %p", vbuf);
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
+}
+
+static void handle_eos_evt(struct al_dec_ctx *ctx, struct msg_itf_header *hdr)
+{
+ struct msg_itf_evt_frame_buffer_decode evt;
+ struct al_codec_dev *codec = ctx->codec;
+
+ if (al_common_get_data(codec, (char *)&evt, hdr->payload_len)) {
+ al_v4l2_err(codec, "Unable to get frame buffer event");
+ return;
+ }
+
+ /* set LAST_FLAG to the last done CAPTURE buffer*/
+ al_mark_last_dst_buf(ctx);
+ /* Set eos event */
+ al_queue_eos_event(ctx);
+}
+
+static void handle_frame_buffer_decode_evt(struct al_dec_ctx *ctx,
+ struct msg_itf_header *hdr)
+{
+ struct msg_itf_evt_frame_buffer_decode evt;
+ struct al_codec_dev *codec = ctx->codec;
+ struct vb2_v4l2_buffer *vbuf;
+ struct al_buffer_meta *meta;
+
+ if (al_common_get_data(codec, (char *)&evt, hdr->payload_len)) {
+ al_v4l2_err(codec, "Unable to get frame buffer event");
+ return;
+ }
+
+ vbuf = al_dec_dequeue_dst_buf(ctx, evt.bufferHandle);
+ if (!vbuf) {
+ al_v4l2_err(codec, "Unable to find frame buffer 0x%llx",
+ evt.bufferHandle);
+ return;
+ }
+
+ meta = &evt.meta;
+ al_v4l2_dbg(codec, 3, "Decoded frame done for buffer %p (%d) (%lld)",
+ vbuf, meta->last, evt.size);
+
+ vb2_set_plane_payload(&vbuf->vb2_buf, 0, evt.size);
+ vbuf->field = V4L2_FIELD_NONE;
+ vbuf->sequence = ctx->csequence++;
+ vbuf->timecode = meta->timecode;
+ vbuf->vb2_buf.timestamp = meta->timestamp;
+
+ if (meta->last || (vbuf->flags & V4L2_BUF_FLAG_LAST)) {
+ vbuf->flags |= V4L2_BUF_FLAG_LAST;
+ v4l2_m2m_mark_stopped(ctx->fh.m2m_ctx);
+ }
+
+ v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
+}
+
+static int al_handle_cmd_reply(struct al_codec_dev *codec,
+ struct msg_itf_header *hdr)
+{
+ struct al_dec_ctx *ctx;
+ struct al_codec_cmd *cmd = NULL;
+ int ret = 0;
+
+ ctx = al_ctx_get(codec, hdr->drv_ctx_hdl);
+ if (IS_ERR_OR_NULL(ctx)) {
+ al_v4l2_err(codec, "Unable to find ctx %p for reply %d",
+ al_phys_to_virt(hdr->drv_ctx_hdl), hdr->type);
+ return -EINVAL;
+ }
+
+ cmd = al_codec_cmd_get(&ctx->cmd_q_list, hdr->drv_cmd_hdl);
+ if (!cmd) {
+ al_v4l2_err(codec, "Unable to find command %p for reply %d",
+ al_phys_to_virt(hdr->drv_cmd_hdl), hdr->type);
+ ret = -EINVAL;
+ goto ctx_put;
+ }
+
+ if (cmd->reply_size != hdr->payload_len) {
+ al_v4l2_err(codec, "mismatch size %d %d", cmd->reply_size,
+ hdr->payload_len);
+ ret = -EINVAL;
+ goto cmd_put;
+ }
+
+ ret = al_common_get_data(codec, cmd->reply, hdr->payload_len);
+ if (ret)
+ al_v4l2_err(codec, "Unable to copy reply");
+
+ complete(&cmd->done);
+ ret = 0;
+
+cmd_put:
+ al_codec_cmd_put(cmd);
+ctx_put:
+ al_ctx_put(ctx);
+
+ return ret;
+}
+
+static int al_handle_cmd_evt(struct al_codec_dev *dev,
+ struct msg_itf_header *hdr, int type)
+{
+ static u32 evt_sizes[] = {
+ sizeof(struct msg_itf_evt_error),
+ sizeof(struct msg_itf_evt_resolution_found),
+ sizeof(struct msg_itf_evt_bitstream_buffer_release),
+ sizeof(struct msg_itf_evt_frame_buffer_decode),
+ sizeof(struct msg_itf_evt_eos),
+ };
+
+ u32 evt_size;
+ struct al_dec_ctx *ctx = NULL;
+ int ret = 0;
+
+ if (type < MSG_ITF_TYPE_NEXT_EVT || type > MSG_ITF_TYPE_END_EVT) {
+ al_v4l2_err(dev, "Unsupporting event type %d", type);
+ return -EINVAL;
+ }
+
+ ctx = al_ctx_get(dev, hdr->drv_ctx_hdl);
+ if (!ctx) {
+ al_v4l2_err(dev, "Unable to find ctx %p for evt %d",
+ al_phys_to_virt(hdr->drv_ctx_hdl), type);
+ return -EINVAL;
+ }
+
+ // Check the received event size and the expected one
+ evt_size = evt_sizes[type - MSG_ITF_TYPE_NEXT_EVT];
+ if (hdr->payload_len != evt_size) {
+ al_v4l2_err(
+ dev,
+ "Invalid event size for client (%p) for evt (%d) : Got (%d), expected (%d)",
+ al_phys_to_virt(hdr->drv_ctx_hdl), type,
+ hdr->payload_len, evt_size);
+ ret = -EINVAL;
+ goto clean_ctx;
+ }
+
+ al_v4l2_dbg(dev, 3, "Event received from MCU (%d)", type);
+
+ switch (type) {
+ case MSG_ITF_TYPE_EVT_ERROR:
+ handle_error_evt(ctx, hdr);
+ break;
+ case MSG_ITF_TYPE_EVT_RESOLUTION_FOUND:
+ handle_resolution_found_evt(ctx, hdr);
+ break;
+ case MSG_ITF_TYPE_EVT_BITSTREAM_BUFFER_RELEASE:
+ handle_bitstream_buffer_release_evt(ctx, hdr);
+ break;
+ case MSG_ITF_TYPE_EVT_FRAME_BUFFER_DECODE:
+ handle_frame_buffer_decode_evt(ctx, hdr);
+ break;
+ case MSG_ITF_TYPE_EVT_EOS:
+ handle_eos_evt(ctx, hdr);
+ break;
+ default:
+ break;
+ }
+
+clean_ctx:
+ al_ctx_put(ctx);
+ return ret;
+}
+
+static void al_dec_process_msg(void *cb_arg, struct msg_itf_header *hdr)
+{
+ struct al_codec_dev *dev = cb_arg;
+ int ret;
+
+ if (is_type_reply(hdr->type))
+ ret = al_handle_cmd_reply(dev, hdr);
+ else if (is_type_event(hdr->type))
+ ret = al_handle_cmd_evt(dev, hdr, hdr->type);
+ else {
+ al_v4l2_err(dev, "Unsupported message type %d", hdr->type);
+ ret = -EINVAL;
+ }
+
+ if (ret) {
+ al_v4l2_err(dev, "Skip received data");
+ al_common_skip_data(dev, hdr->payload_len);
+ }
+}
+
+static const struct video_device al_videodev = {
+ .name = "allegro-decoder",
+ .fops = &al_dec_file_ops,
+ .ioctl_ops = &al_dec_ioctl_ops,
+ .minor = -1,
+ .release = video_device_release_empty,
+ .vfl_dir = VFL_DIR_M2M,
+ .device_caps = V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING,
+};
+
+static void al_dec_register_v4l2(void *cb_arg)
+{
+ struct al_codec_dev *dev = cb_arg;
+ struct video_device *video_dev = NULL;
+ int ret;
+
+ ret = v4l2_device_register(&dev->pdev->dev, &dev->v4l2_dev);
+ if (ret) {
+ al_v4l2_err(dev, "Unable to register v4l2 device %d", ret);
+ return;
+ }
+
+ dev->m2m_dev = v4l2_m2m_init(&al_dec_m2m_ops);
+ if (IS_ERR(dev->m2m_dev)) {
+ ret = PTR_ERR(dev->m2m_dev);
+ al_v4l2_err(dev, "failed to init mem2mem device %d", ret);
+ goto v4l2_m2m_init_error;
+ }
+
+ video_dev = &dev->video_dev;
+ *video_dev = al_videodev;
+
+ video_dev->lock = &dev->lock;
+ video_dev->v4l2_dev = &dev->v4l2_dev;
+
+ video_set_drvdata(video_dev, dev);
+ ret = video_register_device(video_dev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ al_v4l2_err(dev, "failed to register video device %d", ret);
+ goto video_register_device_error;
+ }
+
+ v4l2_info(&dev->v4l2_dev, "registered as /dev/video%d\n",
+ dev->video_dev.num);
+
+ dev->is_video_init_done = 1;
+
+ return;
+
+video_register_device_error:
+ v4l2_m2m_release(dev->m2m_dev);
+v4l2_m2m_init_error:
+ v4l2_device_unregister(&dev->v4l2_dev);
+}
+
+static int al_dec_probe(struct platform_device *pdev)
+{
+ struct al_codec_dev *al_dev;
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
+ const struct al_match_data *match_data;
+ const char *firmware;
+ int ret;
+
+ dev_info(dev, "Probing ...\n");
+
+ match_data = device_get_match_data(dev);
+ if (!match_data) {
+ dev_err(dev, "Missing device match data\n");
+ return -EINVAL;
+ }
+
+ al_dev = devm_kzalloc(dev, sizeof(*al_dev), GFP_KERNEL);
+ if (!al_dev)
+ return -ENOMEM;
+
+ al_dev->pdev = pdev;
+ al_dev->is_video_init_done = 0;
+ mutex_init(&al_dev->lock);
+ mutex_init(&al_dev->ctx_mlock);
+ INIT_LIST_HEAD(&al_dev->ctx_q_list);
+
+ al_dev->cb_arg = al_dev;
+ al_dev->process_msg_cb = al_dec_process_msg;
+ al_dev->fw_ready_cb = al_dec_register_v4l2;
+
+ /* firmware-name is optional in DT */
+ of_property_read_string(np, "firmware-name", &firmware);
+ if (!firmware)
+ firmware = match_data->fw_name;
+
+ ret = al_common_probe(al_dev, firmware);
+ if (ret)
+ return ret;
+
+ platform_set_drvdata(pdev, al_dev);
+ dev_info(dev, "Probing done successfully %p\n", al_dev);
+
+ return 0;
+}
+
+static void al_dec_remove(struct platform_device *pdev)
+{
+ struct al_codec_dev *dev = platform_get_drvdata(pdev);
+
+ dev_info(&pdev->dev, "remove %p\n", dev);
+
+ if (dev->is_video_init_done) {
+ video_unregister_device(&dev->video_dev);
+ if (dev->m2m_dev)
+ v4l2_m2m_release(dev->m2m_dev);
+ v4l2_device_unregister(&dev->v4l2_dev);
+ }
+
+ al_common_remove(dev);
+}
+
+static const struct al_match_data ald300_data = {
+ .fw_name = "al300-vdec.fw",
+};
+
+static const struct of_device_id v4l2_al_dec_dt_match[] = {
+ { .compatible = "allegro,al300-vdec", .data = &ald300_data },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, v4l2_al_dec_dt_match);
+
+static struct platform_driver al300_vdec_drv = {
+ .probe = al_dec_probe,
+ .remove = al_dec_remove,
+ .driver = {
+ .name = "al300_vdec",
+ .of_match_table = of_match_ptr(v4l2_al_dec_dt_match),
+ },
+};
+
+module_platform_driver(al300_vdec_drv);
+
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:al300-vdec");
+MODULE_AUTHOR("Yassine OUAISSA <yassine.ouaissa@allegrodvt.com>");
+MODULE_DESCRIPTION("Allegro DVT V4l2 decoder driver gen 3");
diff --git a/drivers/media/platform/allegro-dvt/al300/al_vdec_drv.h b/drivers/media/platform/allegro-dvt/al300/al_vdec_drv.h
new file mode 100644
index 0000000000000000000000000000000000000000..c0893f134b345c693d72649c43f23d808b16f4c3
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/al300/al_vdec_drv.h
@@ -0,0 +1,93 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (c) 2025 Allegro DVT.
+ * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
+ */
+
+#ifndef __AL_VDEC_DRV__
+#define __AL_VDEC_DRV__
+
+#include "al_codec_util.h"
+
+enum {
+ MSG_ITF_TYPE_EVT_RESOLUTION_FOUND = MSG_ITF_TYPE_NEXT_EVT + 1,
+ MSG_ITF_TYPE_EVT_BITSTREAM_BUFFER_RELEASE,
+ MSG_ITF_TYPE_EVT_FRAME_BUFFER_DECODE,
+ MSG_ITF_TYPE_EVT_EOS,
+ /* Mark the end of the events list.*/
+ MSG_ITF_TYPE_END_EVT,
+};
+
+struct msg_itf_create_decoder_req {
+ unsigned int codec;
+} __packed;
+DECLARE_FULL_REQ(msg_itf_create_decoder_req);
+
+struct msg_itf_evt_resolution_found {
+ u16 buffer_nb;
+ u16 width;
+ u16 height;
+ u32 pixelformat;
+ u32 sizeimage;
+ u32 bytesperline;
+} __packed;
+DECLARE_FULL_EVENT(msg_itf_evt_resolution_found);
+
+struct msg_itf_evt_bitstream_buffer_release {
+ u64 bufferHandle;
+} __packed;
+DECLARE_FULL_EVENT(msg_itf_evt_bitstream_buffer_release);
+
+struct msg_itf_evt_frame_buffer_decode {
+ u64 bufferHandle;
+ u64 size;
+ struct al_buffer_meta meta;
+} __packed;
+DECLARE_FULL_EVENT(msg_itf_evt_frame_buffer_decode);
+
+struct msg_itf_evt_eos {
+ u32 unused;
+} __packed;
+DECLARE_FULL_EVENT(msg_itf_evt_eos);
+
+struct al_fmt {
+ u32 pixelformat;
+ u8 bpp;
+};
+
+struct al_frame {
+ u32 width;
+ u32 height;
+ u32 bytesperline;
+ u32 sizeimage;
+ u32 nbuffers;
+ const struct al_fmt *fmt;
+ enum v4l2_field field;
+ enum v4l2_colorspace colorspace;
+ enum v4l2_ycbcr_encoding ycbcr_enc;
+ enum v4l2_quantization quantization;
+ enum v4l2_xfer_func xfer_func;
+};
+
+struct al_dec_ctx {
+ struct al_codec_dev *codec;
+ struct v4l2_fh fh;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct kref refcount;
+ struct list_head list;
+ /* CAP and OUT frames */
+ struct al_frame src;
+ struct al_frame dst;
+ struct completion res_done; /* Resolution found event */
+ struct list_head cmd_q_list; /* Store active commands */
+ struct mutex buf_q_mlock;
+ struct list_head frame_q_list;
+ struct list_head stream_q_list;
+ u64 hDec;
+ u32 csequence;
+ u32 osequence;
+ bool stopped;
+ bool aborting;
+};
+
+#endif /*__AL_VDEC_DRV__*/
--
2.30.2
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver
2025-06-05 12:26 [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
` (3 preceding siblings ...)
2025-06-05 12:26 ` [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
@ 2025-06-05 12:57 ` Krzysztof Kozlowski
4 siblings, 0 replies; 13+ messages in thread
From: Krzysztof Kozlowski @ 2025-06-05 12:57 UTC (permalink / raw)
To: yassine.ouaissa, Mauro Carvalho Chehab, Michael Tretter,
Pengutronix Kernel Team, Michal Simek, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Nicolas Dufresne
Cc: linux-kernel, linux-media, linux-arm-kernel, devicetree
On 05/06/2025 14:26, Yassine Ouaissa via B4 Relay wrote:
> This patch series introduces a new stateful decoder driver for the
> allegrodvt GEN 3 IPs.
>
> Changes in v2:
> - patch 2: Fix dt-bindings reviews (Krzysztof Kozlowski) (Conor Dooley)
That's just way too vague. What change specifically?
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP
2025-06-05 12:26 ` [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP Yassine Ouaissa via B4 Relay
@ 2025-06-05 13:01 ` Krzysztof Kozlowski
2025-06-05 13:29 ` Yassine OUAISSA
2025-06-12 12:42 ` Michael Tretter
1 sibling, 1 reply; 13+ messages in thread
From: Krzysztof Kozlowski @ 2025-06-05 13:01 UTC (permalink / raw)
To: yassine.ouaissa, Mauro Carvalho Chehab, Michael Tretter,
Pengutronix Kernel Team, Michal Simek, Rob Herring,
Krzysztof Kozlowski, Conor Dooley, Nicolas Dufresne
Cc: linux-kernel, linux-media, linux-arm-kernel, devicetree
On 05/06/2025 14:26, Yassine Ouaissa via B4 Relay wrote:
> From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>
> Add compatible for video decoder on allegrodvt Gen 3 IP.
A nit, subject: drop second/last, redundant "dt-bindings". The
"dt-bindings" prefix is already stating that these are bindings.
See also:
https://elixir.bootlin.com/linux/v6.7-rc8/source/Documentation/devicetree/bindings/submitting-patches.rst#L18
Subject prefix(es): still wrong. You can get them for example with `git
log --oneline -- DIRECTORY_OR_FILE` on the directory your patch is
touching. For bindings, the preferred subjects are explained here:
https://www.kernel.org/doc/html/latest/devicetree/bindings/submitting-patches.html#i-for-patch-submitters
>
> v2:
> - Change the YAML file name, use the existing vendor-prefix.
> - Improuve the dt-bindings description.
> - Change the device compatible identifier, from "allegrodvt, al300-vdec",
> to "allegro, al300-vdec"
> - Simplify the register property specification,
> by using the simple min/max items constraint (Krzysztof Kozlowski)
> - Remove the clock-names property. And remove it from the required
> properties list (Krzysztof Kozlowski) (Conor Dooley)
> - Use the simple maxItems constraint for the memory-region property.
> Also for the firmware-name (Krzysztof Kozlowski)
> - Example changes:
> - Use header provides definitions for the interrupts (Conor Dooley)
> - Improuve Interrupt specification using GIC constants (Conor Dooley)
> - Use generic node name "video-decoder" (Krzysztof Kozlowski) (Conor Dooley)
> - Remove unused label (Krzysztof Kozlowski)
> - Change clock reference from <&mcu_clock_dec> to <&mcu_core_clk>
> - Use hex format for reg property (Krzysztof Kozlowski) (Conor Dooley)
> - Reduce memory region size (Krzysztof Kozlowski) (Conor Dooley)
All this goes to changelog
>
> - Link v1: https://patchwork.linuxtv.org/project/linux-media/patch/20250511144752.504162-4-yassine.ouaissa@allegrodvt.com/
Drop
>
> Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
> ---
> .../bindings/media/allegro,al300-vdec.yaml | 75 ++++++++++++++++++++++
> MAINTAINERS | 2 +
> 2 files changed, 77 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
> new file mode 100644
> index 0000000000000000000000000000000000000000..26f9ac39682431b1d4828aed5d1ed43ef099e204
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
> @@ -0,0 +1,75 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/allegro,al300-vdec.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Allegro DVT Video IP Decoder Gen 3
> +
> +maintainers:
> + - Yassine OUAISSA <yassine.ouaissa@allegrodvt.com>
> +
> +description: The al300-vdec represents the gen 3 of Allegro DVT IP video
Blank line after description:
> + decoding technology, offering significant advancements over its
> + predecessors. This new decoder features enhanced processing capabilities
> + with improved throughput and reduced latency.
> +
> + Communication between the host driver software and the MCU is implemented
> + through a specialized mailbox interface mechanism. This mailbox system
> + provides a structured channel for exchanging commands, parameters, and
> + status information between the host CPU and the MCU controlling the codec
> + engines.
> +
> +properties:
> + compatible:
> + const: allegro,al300-vdec
> +
> + reg:
> + maxItems: 2
> + minItems: 2
Drop
> +
> + reg-names:
> + items:
> + - const: regs
base? apb is also "regs", because this is "reg" property, so "regs"
feels redundant.
Unless this is something entirely else (quite different address in
example), so maybe this should not be reg at all.
Also, make the example complete - missing memory region.
> + - const: apb
> +
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP
2025-06-05 13:01 ` Krzysztof Kozlowski
@ 2025-06-05 13:29 ` Yassine OUAISSA
0 siblings, 0 replies; 13+ messages in thread
From: Yassine OUAISSA @ 2025-06-05 13:29 UTC (permalink / raw)
To: Krzysztof Kozlowski
Cc: Mauro Carvalho Chehab, Michael Tretter, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Nicolas Dufresne, linux-kernel, linux-media, linux-arm-kernel,
devicetree
Hi Krzysztof,
On 05.06.2025 15:01, Krzysztof Kozlowski wrote:
>On 05/06/2025 14:26, Yassine Ouaissa via B4 Relay wrote:
>> From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>>
>> Add compatible for video decoder on allegrodvt Gen 3 IP.
>
>A nit, subject: drop second/last, redundant "dt-bindings". The
>"dt-bindings" prefix is already stating that these are bindings.
>See also:
>https://elixir.bootlin.com/linux/v6.7-rc8/source/Documentation/devicetree/bindings/submitting-patches.rst#L18
>
>
>Subject prefix(es): still wrong. You can get them for example with `git
>log --oneline -- DIRECTORY_OR_FILE` on the directory your patch is
>touching. For bindings, the preferred subjects are explained here:
>https://www.kernel.org/doc/html/latest/devicetree/bindings/submitting-patches.html#i-for-patch-submitters
>
Got it, thanks.
>>
>> v2:
>> - Change the YAML file name, use the existing vendor-prefix.
>> - Improuve the dt-bindings description.
>> - Change the device compatible identifier, from "allegrodvt, al300-vdec",
>> to "allegro, al300-vdec"
>> - Simplify the register property specification,
>> by using the simple min/max items constraint (Krzysztof Kozlowski)
>> - Remove the clock-names property. And remove it from the required
>> properties list (Krzysztof Kozlowski) (Conor Dooley)
>> - Use the simple maxItems constraint for the memory-region property.
>> Also for the firmware-name (Krzysztof Kozlowski)
>> - Example changes:
>> - Use header provides definitions for the interrupts (Conor Dooley)
>> - Improuve Interrupt specification using GIC constants (Conor Dooley)
>> - Use generic node name "video-decoder" (Krzysztof Kozlowski) (Conor Dooley)
>> - Remove unused label (Krzysztof Kozlowski)
>> - Change clock reference from <&mcu_clock_dec> to <&mcu_core_clk>
>> - Use hex format for reg property (Krzysztof Kozlowski) (Conor Dooley)
>> - Reduce memory region size (Krzysztof Kozlowski) (Conor Dooley)
>
>All this goes to changelog
>
I'll move it to the changelog.
>>
>> - Link v1: https://patchwork.linuxtv.org/project/linux-media/patch/20250511144752.504162-4-yassine.ouaissa@allegrodvt.com/
>
>Drop
>
>>
>> Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>> ---
>> .../bindings/media/allegro,al300-vdec.yaml | 75 ++++++++++++++++++++++
>> MAINTAINERS | 2 +
>> 2 files changed, 77 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..26f9ac39682431b1d4828aed5d1ed43ef099e204
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
>> @@ -0,0 +1,75 @@
>> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/media/allegro,al300-vdec.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Allegro DVT Video IP Decoder Gen 3
>> +
>> +maintainers:
>> + - Yassine OUAISSA <yassine.ouaissa@allegrodvt.com>
>> +
>> +description: The al300-vdec represents the gen 3 of Allegro DVT IP video
>
>Blank line after description:
I'll change it in the next version.
>
>> + decoding technology, offering significant advancements over its
>> + predecessors. This new decoder features enhanced processing capabilities
>> + with improved throughput and reduced latency.
>> +
>> + Communication between the host driver software and the MCU is implemented
>> + through a specialized mailbox interface mechanism. This mailbox system
>> + provides a structured channel for exchanging commands, parameters, and
>> + status information between the host CPU and the MCU controlling the codec
>> + engines.
>> +
>> +properties:
>> + compatible:
>> + const: allegro,al300-vdec
>> +
>> + reg:
>> + maxItems: 2
>> + minItems: 2
>
>Drop
I'll change it in the next version.
>
>> +
>> + reg-names:
>> + items:
>> + - const: regs
>
>base? apb is also "regs", because this is "reg" property, so "regs"
>feels redundant.
>
the regs, is where the device is mapped, but the apb is used by the MCU.
the reg 'apb' is used to map the MCU peripherals.
>Unless this is something entirely else (quite different address in
>example), so maybe this should not be reg at all.
>
>Also, make the example complete - missing memory region.
>
I'll change it in the next version.
>> + - const: apb
>> +
>Best regards,
>Krzysztof
Best regards,
Yassine OUAISSA
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP
2025-06-05 12:26 ` [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP Yassine Ouaissa via B4 Relay
2025-06-05 13:01 ` Krzysztof Kozlowski
@ 2025-06-12 12:42 ` Michael Tretter
2025-06-17 6:33 ` Yassine OUAISSA
1 sibling, 1 reply; 13+ messages in thread
From: Michael Tretter @ 2025-06-12 12:42 UTC (permalink / raw)
To: yassine.ouaissa
Cc: Mauro Carvalho Chehab, Pengutronix Kernel Team, Michal Simek,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Nicolas Dufresne,
linux-kernel, linux-media, linux-arm-kernel, devicetree
On Thu, 05 Jun 2025 12:26:57 +0000, Yassine Ouaissa via B4 Relay wrote:
> From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>
> Add compatible for video decoder on allegrodvt Gen 3 IP.
>
> v2:
> - Change the YAML file name, use the existing vendor-prefix.
> - Improuve the dt-bindings description.
> - Change the device compatible identifier, from "allegrodvt, al300-vdec",
> to "allegro, al300-vdec"
> - Simplify the register property specification,
> by using the simple min/max items constraint (Krzysztof Kozlowski)
> - Remove the clock-names property. And remove it from the required
> properties list (Krzysztof Kozlowski) (Conor Dooley)
> - Use the simple maxItems constraint for the memory-region property.
> Also for the firmware-name (Krzysztof Kozlowski)
> - Example changes:
> - Use header provides definitions for the interrupts (Conor Dooley)
> - Improuve Interrupt specification using GIC constants (Conor Dooley)
> - Use generic node name "video-decoder" (Krzysztof Kozlowski) (Conor Dooley)
> - Remove unused label (Krzysztof Kozlowski)
> - Change clock reference from <&mcu_clock_dec> to <&mcu_core_clk>
> - Use hex format for reg property (Krzysztof Kozlowski) (Conor Dooley)
> - Reduce memory region size (Krzysztof Kozlowski) (Conor Dooley)
>
> - Link v1: https://patchwork.linuxtv.org/project/linux-media/patch/20250511144752.504162-4-yassine.ouaissa@allegrodvt.com/
>
> Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
> ---
> .../bindings/media/allegro,al300-vdec.yaml | 75 ++++++++++++++++++++++
> MAINTAINERS | 2 +
> 2 files changed, 77 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
> new file mode 100644
> index 0000000000000000000000000000000000000000..26f9ac39682431b1d4828aed5d1ed43ef099e204
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
> @@ -0,0 +1,75 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/allegro,al300-vdec.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Allegro DVT Video IP Decoder Gen 3
> +
> +maintainers:
> + - Yassine OUAISSA <yassine.ouaissa@allegrodvt.com>
> +
> +description: The al300-vdec represents the gen 3 of Allegro DVT IP video
> + decoding technology, offering significant advancements over its
> + predecessors. This new decoder features enhanced processing capabilities
> + with improved throughput and reduced latency.
> +
> + Communication between the host driver software and the MCU is implemented
> + through a specialized mailbox interface mechanism. This mailbox system
> + provides a structured channel for exchanging commands, parameters, and
> + status information between the host CPU and the MCU controlling the codec
> + engines.
> +
> +properties:
> + compatible:
> + const: allegro,al300-vdec
> +
> + reg:
> + maxItems: 2
> + minItems: 2
> +
> + reg-names:
> + items:
> + - const: regs
> + - const: apb
If I understand correctly, "regs" are the registers to control the MCU
and "apb" are the registers of the actual codec engines, which is
controlled by the MCU. The driver never accesses the apb registers, but
uses the apb address only to configure the firmware and tell it, where
the registers of the codec engines are found.
Maybe a separate node for the actual codec that is referred via a
phandle could be a better description of the hardware?
> +
> + interrupts:
> + maxItems: 1
> +
> + clocks:
> + items:
> + - description: MCU core clock
> +
> + memory-region:
> + maxItems: 1
> +
> + firmware-name:
> + maxItems: 1
> +
> +required:
> + - compatible
> + - reg
> + - reg-names
> + - interrupts
> + - clocks
> +
> +additionalProperties: False
> +
> +examples:
> + - |
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> +
> + axi {
> + #address-cells = <2>;
> + #size-cells = <2>;
> +
> + video-decoder@a0120000 {
> + compatible = "allegro,al300-vdec";
> + reg = <0x00 0xa0120000 0x00 0x10000>,
> + <0x01 0x80000000 0x00 0x8000>;
> + reg-names = "regs", "apb";
> + interrupts = <GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH>;
> + clocks = <&mcu_core_clk>;
> + firmware-name = "al300_vdec.fw";
> + };
> + };
> diff --git a/MAINTAINERS b/MAINTAINERS
> index abc6ba61048771303bc219102f2db602266b7c30..1ff78b9a76cb8cdf32263fcd9b4579b4a2bb6b2a 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -816,7 +816,9 @@ M: Michael Tretter <m.tretter@pengutronix.de>
> R: Pengutronix Kernel Team <kernel@pengutronix.de>
> L: linux-media@vger.kernel.org
> S: Maintained
> +F: Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
> F: Documentation/devicetree/bindings/media/allegro,al5e.yaml
> +F: drivers/media/platform/allegro-dvt/al300
> F: drivers/media/platform/allegro-dvt/zynqmp
>
> ALLIED VISION ALVIUM CAMERA DRIVER
>
> --
> 2.30.2
>
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver
2025-06-05 12:26 ` [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
@ 2025-06-12 13:30 ` Michael Tretter
2025-06-16 8:46 ` Yassine OUAISSA
2025-06-16 12:56 ` Yassine OUAISSA
0 siblings, 2 replies; 13+ messages in thread
From: Michael Tretter @ 2025-06-12 13:30 UTC (permalink / raw)
To: yassine.ouaissa
Cc: Mauro Carvalho Chehab, Pengutronix Kernel Team, Michal Simek,
Rob Herring, Krzysztof Kozlowski, Conor Dooley, Nicolas Dufresne,
linux-kernel, linux-media, linux-arm-kernel, devicetree
Hi Yassine,
Thanks for the patch.
The overall architecture looks a lot like the video encoder for the
ZynqMP, but with some significant differences in the details. I didn't
manage to look more closely at the driver, yet, but I have a few high
level questions.
On Thu, 05 Jun 2025 12:26:59 +0000, Yassine Ouaissa via B4 Relay wrote:
> From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>
> This commit introduces a new allegro-dvt V4L2 stateful decoder driverfor
> the Gen 3 IP with support for:
> - AVC (H.264), HEVC (H.265), and JPEG decoding
> - Output formats: NV12, NV16, I420, and P010 for capture
>
> v2:
> - Replace the mutex_(lock/unlock) with the guard(mutex), that manage
> mutexes more efficiently.
> - Set DMA_BIT_MASK to 39, and drop the paddr check when allocating
> dma_memory.
> - Use dma_coerce_mask_and_coherent to set the DMA_MASK.
> - Use static initializer for some structs.
> - use #ifdef instead of #if defined
> - Optimize some function.
> - Use the declaration in the loop.
> - Use codec for al_codec_dev instead of dev, to not get confused with
> the device struct.
> - Remove the codec member of the al_codec_dev, use the fmt->pixelformat
> when request creating decoder instance.
>
> Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
> ---
> drivers/media/platform/allegro-dvt/Kconfig | 1 +
> drivers/media/platform/allegro-dvt/Makefile | 1 +
> drivers/media/platform/allegro-dvt/al300/Kconfig | 23 +
> drivers/media/platform/allegro-dvt/al300/Makefile | 6 +
> .../platform/allegro-dvt/al300/al_codec_common.c | 733 ++++++++++
> .../platform/allegro-dvt/al300/al_codec_common.h | 248 ++++
> .../platform/allegro-dvt/al300/al_codec_util.c | 174 +++
> .../platform/allegro-dvt/al300/al_codec_util.h | 186 +++
> .../media/platform/allegro-dvt/al300/al_vdec_drv.c | 1518 ++++++++++++++++++++
> .../media/platform/allegro-dvt/al300/al_vdec_drv.h | 93 ++
> 10 files changed, 2983 insertions(+)
>
[...]
> diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_common.c b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
> new file mode 100644
> index 0000000000000000000000000000000000000000..716d0004482702537ea89ec4abecd6af26654b32
> --- /dev/null
> +++ b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
[...]
> +static void handle_alloc_memory_req(struct al_codec_dev *codec,
> + struct msg_itf_header *hdr)
> +{
> + struct device *dev = &codec->pdev->dev;
> + struct msg_itf_alloc_mem_req req;
> + struct msg_itf_alloc_mem_reply_full reply = {
> + .reply.phyAddr = 0,
> + .hdr.type = MSG_ITF_TYPE_ALLOC_MEM_REPLY,
> + .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
> + .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
> + .hdr.payload_len = sizeof(struct msg_itf_alloc_mem_reply),
> + };
> + struct codec_dma_buf *buf;
> + int ret;
> +
> + ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
> + if (ret) {
> + al_codec_err(codec, "Unable to get cma req %d", ret);
> + return;
> + }
> +
> + buf = kmalloc(sizeof(*buf), GFP_KERNEL);
> + if (!buf)
> + goto send_reply;
> +
> + buf->size = req.uSize;
> + buf->vaddr =
> + dma_alloc_coherent(dev, buf->size, &buf->paddr, GFP_KERNEL);
> + if (!buf->vaddr) {
> + dev_warn(dev, "Failed to allocate DMA buffer\n");
> + goto send_reply;
> + }
> +
> + reply.reply.phyAddr = (u64)buf->paddr;
> + al_common_dma_buf_insert(codec, buf);
The buffers allocated by the firmware are tracked by device. Thus, there
is only one list for all buffers used by the firmware.
I guess that the buffers are be allocated per context. If that's the
case, maybe tracking them per context in the driver would be a better
option.
> +
> +send_reply:
> + ret = al_common_send(codec, &reply.hdr);
> + if (ret) {
> + al_codec_err(codec, "Unable to reply to cma alloc");
> + al_common_dma_buf_remove(codec, buf);
> + }
> +}
> +
> +static void handle_free_memory_req(struct al_codec_dev *codec,
> + struct msg_itf_header *hdr)
> +{
> + struct msg_itf_free_mem_req req;
> + struct msg_itf_free_mem_reply_full reply = {
> + .hdr.type = MSG_ITF_TYPE_FREE_MEM_REPLY,
> + .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
> + .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
> + .hdr.payload_len = sizeof(struct msg_itf_free_mem_reply),
> + .reply.ret = -1,
> + };
> + struct codec_dma_buf *buf;
> + int ret;
> +
> + ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
> + if (ret) {
> + al_codec_err(codec, "Unable to put cma req");
> + return;
> + }
> +
> + buf = al_common_dma_buf_lookup(codec, req.phyAddr);
> + if (!buf) {
> + al_codec_err(codec, "Unable to get dma handle for %p",
> + (void *)(long)req.phyAddr);
> + reply.reply.ret = -EINVAL;
> + goto send_reply;
> + }
> +
> + al_codec_dbg(codec, "Free memory %p, size %d",
> + (void *)(long)req.phyAddr, buf->size);
> +
> + al_common_dma_buf_remove(codec, buf);
> + reply.reply.ret = 0;
> +
> +send_reply:
> + ret = al_common_send(codec, &reply.hdr);
> + if (ret)
> + al_codec_err(codec, "Unable to reply to cma free");
> +}
> +
> +static void handle_mcu_console_print(struct al_codec_dev *codec,
> + struct msg_itf_header *hdr)
> +{
> +#ifdef DEBUG
What's the reason for making this a compile time option? Maybe a
module parameter to enable MCU logging at runtime would be more
friendly for debugging.
> + struct msg_itf_write_req *req;
> + int ret;
> +
> + /* one more byte to be sure to have a zero terminated string */
> + req = kzalloc(hdr->payload_len + 1, GFP_KERNEL);
> + if (!req) {
> + al_common_skip_data(codec, hdr->payload_len);
> + al_codec_err(codec, "Unable to alloc memory");
> + return;
> + }
> +
> + ret = al_codec_msg_get_data(&codec->mb_m2h, (char *)req,
> + hdr->payload_len);
> + if (ret) {
> + al_codec_err(codec, "Unable to get request");
> + kfree(req);
> + return;
> + }
> +
> + /* Print the mcu logs */
> + dev_dbg(&codec->pdev->dev, "[ALG_MCU] %s(),%d: %s\n", __func__,
> + __LINE__, (char *)(req + 1));
> + kfree(req);
> +#else
> + al_common_skip_data(codec, hdr->payload_len);
> +#endif
> +}
> +
[...]
> +static int al_common_load_firmware_start(struct al_codec_dev *codec,
> + const char *name)
> +{
> + struct device *dev = &codec->pdev->dev;
> + dma_addr_t phys;
> + size_t size;
> + void *virt;
> + int err;
> +
> + if (codec->firmware.virt)
> + return 0;
> +
> + err = al_common_read_firmware(codec, name);
> + if (err)
> + return err;
> +
> + size = codec->firmware.size;
> +
> + virt = dma_alloc_coherent(dev, size, &phys, GFP_KERNEL);
> + err = dma_mapping_error(dev, phys);
> + if (err < 0)
> + return err;
> +
> + codec->firmware.virt = virt;
> + codec->firmware.phys = phys;
> +
> + al_common_copy_firmware_image(codec);
> + err = al_common_parse_firmware_image(codec);
> + if (err) {
> + al_codec_err(codec, "failed to parse firmware image");
> + goto cleanup;
> + }
> +
> + err = al_common_setup_hw_regs(codec);
> + if (err) {
> + al_codec_err(codec, "Unable to setup hw registers");
> + goto cleanup;
> + }
> +
> + al_codec_mb_init(&codec->mb_h2m, virt + codec->firmware.mb_h2m.offset,
> + codec->firmware.mb_h2m.size, MB_IFT_MAGIC_H2M);
> +
> + al_codec_mb_init(&codec->mb_m2h, virt + codec->firmware.mb_m2h.offset,
> + codec->firmware.mb_m2h.size, MB_IFT_MAGIC_M2H);
> +
> + err = al_common_start_fw(codec);
> + if (err) {
> + al_codec_err(codec, "fw start has failed");
> + goto cleanup;
> + }
If I understand correctly, the difference to the ZynqMP firmware is that
the firmware can be configured for different addresses. Thus, the
firmware and mailbox addresses on ZynqMP are determined by the bitstream
synthesis, which this driver is free to allocate memory for the firmware
and mailboxes wherever it wants. Correct?
> +
> + al_codec_dbg(codec, "mcu has boot successfully !");
> + codec->fw_ready_cb(codec->cb_arg);
> +
> + release_firmware(codec->firmware.firmware);
> + codec->firmware.firmware = NULL;
> +
> + return 0;
> +
> +cleanup:
> + dma_free_coherent(dev, size, virt, phys);
> +
> + return err;
> +}
> +
> +static u64 al_common_get_periph_addr(struct al_codec_dev *codec)
> +{
> + struct resource *res;
> +
> + res = platform_get_resource_byname(codec->pdev, IORESOURCE_MEM, "apb");
> + if (!res) {
> + al_codec_err(codec, "Unable to find APB start address");
> + return 0;
> + }
> +
> + if (res->start & AL_CODEC_APB_MASK) {
> + al_codec_err(codec, "APB start address is invalid");
> + return 0;
> + }
> +
> + return res->start;
> +}
> +
> +int al_common_probe(struct al_codec_dev *codec, const char *name)
> +{
> + struct platform_device *pdev = codec->pdev;
> + int irq;
> + int ret;
> +
> + mutex_init(&codec->buf_lock);
> + INIT_LIST_HEAD(&codec->alloc_buffers);
> + init_completion(&codec->completion);
> +
> + /* setup dma memory */
> + ret = al_common_setup_dma(codec);
> + if (ret)
> + return ret;
> +
> + /* Hw registers */
> + codec->regs_info =
> + platform_get_resource_byname(pdev, IORESOURCE_MEM, "regs");
> + if (!codec->regs_info) {
> + al_codec_err(codec, "regs resource missing from device tree");
> + return -EINVAL;
> + }
> +
> + codec->regs = devm_ioremap_resource(&pdev->dev, codec->regs_info);
> + if (!codec->regs) {
> + al_codec_err(codec, "failed to map registers");
> + return -ENOMEM;
> + }
> +
> + codec->apb = al_common_get_periph_addr(codec);
> + if (!codec->apb)
> + return -EINVAL;
> +
> + /* The MCU has already default clock value */
> + codec->clk = devm_clk_get(&pdev->dev, NULL);
> + if (IS_ERR(codec->clk)) {
> + al_codec_err(codec, "failed to get MCU core clock");
> + return PTR_ERR(codec->clk);
> + }
> +
> + ret = clk_prepare_enable(codec->clk);
> + if (ret) {
> + al_codec_err(codec, "Cannot enable MCU clock: %d\n", ret);
> + return ret;
> + }
> +
> + irq = platform_get_irq(pdev, 0);
> + if (irq < 0) {
> + al_codec_err(codec, "Failed to get IRQ");
> + ret = -EINVAL;
> + goto disable_clk;
> + }
> +
> + ret = devm_request_threaded_irq(&pdev->dev, irq,
> + al_common_hardirq_handler,
> + al_common_irq_handler, IRQF_SHARED,
> + dev_name(&pdev->dev), codec);
> + if (ret) {
> + al_codec_err(codec, "Unable to register irq handler");
> + goto disable_clk;
> + }
> +
> + /* ok so request the fw */
> + ret = al_common_load_firmware_start(codec, name);
> + if (ret) {
> + al_codec_err(codec, "failed to load firmware : %s", name);
> + goto disable_clk;
> + }
> +
> + return 0;
> +
> +disable_clk:
> + clk_disable_unprepare(codec->clk);
> +
> + return ret;
> +}
> +
[...]
> diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_util.h b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
> new file mode 100644
> index 0000000000000000000000000000000000000000..5f893db4a1a3f2b9e6e9109b81a956bcaa71851c
> --- /dev/null
> +++ b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
> @@ -0,0 +1,186 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * Copyright (c) 2025 Allegro DVT.
> + * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
> + */
> +
> +#ifndef __AL_CODEC_UTIL__
> +#define __AL_CODEC_UTIL__
> +
> +#include <linux/mutex.h>
> +#include <linux/types.h>
> +#include <linux/v4l2-common.h>
> +
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-v4l2.h>
> +
> +#define MB_IFT_MAGIC_H2M 0xabcd1230
> +#define MB_IFT_MAGIC_M2H 0xabcd1231
> +#define MB_IFT_VERSION 0x00010000
> +
> +#define MAJOR_SHIFT 20
> +#define MAJOR_MASK 0xfff
> +#define MINOR_SHIFT 8
> +#define MINOR_MASK 0xfff
> +#define PATCH_SHIFT 0
> +#define PATCH_MASK 0xff
> +
> +/*
> + * AL_BOOT_VERSION() - Version format 32-bit, 12 bits for the major,
> + * the same for minor, 8bits for the patch
> + */
> +#define AL_BOOT_VERSION(major, minor, patch) \
> + ((((major) & MAJOR_MASK) << MAJOR_SHIFT) | \
> + (((minor) & MINOR_MASK) << MINOR_SHIFT) | \
> + (((patch) & PATCH_MASK) << PATCH_SHIFT))
> +
> +#define al_phys_to_virt(x) ((void *)(uintptr_t)x)
> +#define al_virt_to_phys(x) ((phys_addr_t)(uintptr_t)x)
> +
> +#define DECLARE_FULL_REQ(s) \
> + struct s##_full { \
> + struct msg_itf_header hdr; \
> + struct s req; \
> + } __packed
> +
> +#define DECLARE_FULL_REPLY(s) \
> + struct s##_full { \
> + struct msg_itf_header hdr; \
> + struct s reply; \
> + } __packed
> +
> +#define DECLARE_FULL_EVENT(s) \
> + struct s##_full { \
> + struct msg_itf_header hdr; \
> + struct s event; \
> + } __packed
> +
> +struct al_mb_itf {
> + u32 magic;
> + u32 version;
> + u32 head;
> + u32 tail;
> +} __packed;
> +
> +struct al_codec_mb {
> + struct al_mb_itf *hdr;
> + struct mutex lock;
> + char *data;
> + int size;
> +};
On a first glance, this looks a lot like the allegro_mbox in the ZynqMP
encoder driver. Even though the message format is different, would it be
possible to generalize the general mailbox handling and use it for both
drivers?
Michael
> +
> +struct al_codec_cmd {
> + struct kref refcount;
> + struct list_head list;
> + struct completion done;
> + int reply_size;
> + void *reply;
> +};
> +
> +#define al_codec_err(codec, fmt, args...) \
> + dev_err(&(codec)->pdev->dev, "[ALG_CODEC][ERROR] %s():%d: " fmt "\n", \
> + __func__, __LINE__, ##args)
> +
> +#define al_v4l2_err(codec, fmt, args...) \
> + dev_err(&(codec)->pdev->dev, "[ALG_V4L2][ERROR] %s():%d: " fmt "\n", \
> + __func__, __LINE__, ##args)
> +
> +#if defined(DEBUG)
> +
> +extern int debug;
> +
> +/* V4L2 logs */
> +#define al_v4l2_dbg(codec, level, fmt, args...) \
> + do { \
> + if (debug >= level) \
> + dev_dbg(&(codec)->pdev->dev, \
> + "[ALG_V4L2] level=%d %s(),%d: " fmt "\n", \
> + level, __func__, __LINE__, ##args); \
> + } while (0)
> +
> +/* Codec logs */
> +#define al_codec_dbg(codec, fmt, args...) \
> + do { \
> + if (debug) \
> + dev_dbg(&(codec)->pdev->dev, \
> + "[ALG_CODEC] %s(),%d: " fmt "\n", __func__, \
> + __LINE__, ##args); \
> + } while (0)
> +
> +#else
> +
> +#define al_v4l2_dbg(codec, level, fmt, args...) \
> + do { \
> + (void)level; \
> + dev_dbg(&(codec)->pdev->dev, "[ALG_V4L2]: " fmt "\n", \
> + ##args); \
> + } while (0)
> +
> +#define al_codec_dbg(codec, fmt, args...) \
> + dev_dbg(&(codec)->pdev->dev, "[ALG_CODEC]: " fmt "\n", ##args)
> +
> +#endif
> +
> +#define MSG_ITF_TYPE_LIMIT BIT(10)
> +
> +/* Message types host <-> mcu */
> +enum {
> + MSG_ITF_TYPE_MCU_ALIVE = 0,
> + MSG_ITF_TYPE_WRITE_REQ = 2,
> + MSG_ITF_TYPE_FIRST_REQ = 1024,
> + MSG_ITF_TYPE_NEXT_REQ,
> + MSG_ITF_TYPE_FIRST_REPLY = 2048,
> + MSG_ITF_TYPE_NEXT_REPLY,
> + MSG_ITF_TYPE_ALLOC_MEM_REQ = 3072,
> + MSG_ITF_TYPE_FREE_MEM_REQ,
> + MSG_ITF_TYPE_ALLOC_MEM_REPLY = 4096,
> + MSG_ITF_TYPE_FREE_MEM_REPLY,
> + MSG_ITF_TYPE_FIRST_EVT = 5120,
> + MSG_ITF_TYPE_NEXT_EVT = MSG_ITF_TYPE_FIRST_EVT
> +};
> +
> +struct msg_itf_header {
> + u64 drv_ctx_hdl;
> + u64 drv_cmd_hdl;
> + u16 type;
> + u16 payload_len;
> + u16 padding[2];
> +} __packed;
> +
> +void al_codec_mb_init(struct al_codec_mb *mb, char *addr, int size, u32 magic);
> +int al_codec_msg_get_header(struct al_codec_mb *mb, struct msg_itf_header *hdr);
> +int al_codec_msg_get_data(struct al_codec_mb *mb, char *data, int len);
> +int al_codec_msg_send(struct al_codec_mb *mb, struct msg_itf_header *hdr,
> + void (*trigger)(void *), void *trigger_arg);
> +
> +static inline bool is_type_reply(uint16_t type)
> +{
> + return type >= MSG_ITF_TYPE_FIRST_REPLY &&
> + type < MSG_ITF_TYPE_FIRST_REPLY + MSG_ITF_TYPE_LIMIT;
> +}
> +
> +static inline bool is_type_event(uint16_t type)
> +{
> + return type >= MSG_ITF_TYPE_FIRST_EVT &&
> + type < MSG_ITF_TYPE_FIRST_EVT + MSG_ITF_TYPE_LIMIT;
> +}
> +
> +void al_codec_cmd_put(struct al_codec_cmd *cmd);
> +
> +struct al_codec_cmd *al_codec_cmd_create(int reply_size);
> +
> +static inline struct al_codec_cmd *al_codec_cmd_get(struct list_head *cmd_list,
> + uint64_t hdl)
> +{
> + struct al_codec_cmd *cmd = NULL;
> +
> + list_for_each_entry(cmd, cmd_list, list) {
> + if (likely(cmd == al_phys_to_virt(hdl))) {
> + kref_get(&cmd->refcount);
> + break;
> + }
> + }
> + return list_entry_is_head(cmd, cmd_list, list) ? NULL : cmd;
> +}
> +
> +#endif /* __AL_CODEC_UTIL__ */
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver
2025-06-12 13:30 ` Michael Tretter
@ 2025-06-16 8:46 ` Yassine OUAISSA
2025-06-16 12:56 ` Yassine OUAISSA
1 sibling, 0 replies; 13+ messages in thread
From: Yassine OUAISSA @ 2025-06-16 8:46 UTC (permalink / raw)
To: Michael Tretter, Mauro Carvalho Chehab, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Nicolas Dufresne, linux-kernel, linux-media, linux-arm-kernel,
devicetree
On 12.06.2025 15:30, Michael Tretter wrote:
Hi Michael,
Thank you for your review.
>Hi Yassine,
>
>Thanks for the patch.
>
>The overall architecture looks a lot like the video encoder for the
>ZynqMP, but with some significant differences in the details. I didn't
>manage to look more closely at the driver, yet, but I have a few high
>level questions.
>
There are a many difference between the ZynqMP, and the al300 drivers,
espicially on the mcu control. ( the al300 mcu is a 64bit ).
>On Thu, 05 Jun 2025 12:26:59 +0000, Yassine Ouaissa via B4 Relay wrote:
>> From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>>
>> This commit introduces a new allegro-dvt V4L2 stateful decoder driverfor
>> the Gen 3 IP with support for:
>> - AVC (H.264), HEVC (H.265), and JPEG decoding
>> - Output formats: NV12, NV16, I420, and P010 for capture
>>
>> v2:
>> - Replace the mutex_(lock/unlock) with the guard(mutex), that manage
>> mutexes more efficiently.
>> - Set DMA_BIT_MASK to 39, and drop the paddr check when allocating
>> dma_memory.
>> - Use dma_coerce_mask_and_coherent to set the DMA_MASK.
>> - Use static initializer for some structs.
>> - use #ifdef instead of #if defined
>> - Optimize some function.
>> - Use the declaration in the loop.
>> - Use codec for al_codec_dev instead of dev, to not get confused with
>> the device struct.
>> - Remove the codec member of the al_codec_dev, use the fmt->pixelformat
>> when request creating decoder instance.
>>
>> Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>> ---
>> drivers/media/platform/allegro-dvt/Kconfig | 1 +
>> drivers/media/platform/allegro-dvt/Makefile | 1 +
>> drivers/media/platform/allegro-dvt/al300/Kconfig | 23 +
>> drivers/media/platform/allegro-dvt/al300/Makefile | 6 +
>> .../platform/allegro-dvt/al300/al_codec_common.c | 733 ++++++++++
>> .../platform/allegro-dvt/al300/al_codec_common.h | 248 ++++
>> .../platform/allegro-dvt/al300/al_codec_util.c | 174 +++
>> .../platform/allegro-dvt/al300/al_codec_util.h | 186 +++
>> .../media/platform/allegro-dvt/al300/al_vdec_drv.c | 1518 ++++++++++++++++++++
>> .../media/platform/allegro-dvt/al300/al_vdec_drv.h | 93 ++
>> 10 files changed, 2983 insertions(+)
>>
>[...]
>> diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_common.c b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..716d0004482702537ea89ec4abecd6af26654b32
>> --- /dev/null
>> +++ b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
>[...]
>> +static void handle_alloc_memory_req(struct al_codec_dev *codec,
>> + struct msg_itf_header *hdr)
>> +{
>> + struct device *dev = &codec->pdev->dev;
>> + struct msg_itf_alloc_mem_req req;
>> + struct msg_itf_alloc_mem_reply_full reply = {
>> + .reply.phyAddr = 0,
>> + .hdr.type = MSG_ITF_TYPE_ALLOC_MEM_REPLY,
>> + .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
>> + .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
>> + .hdr.payload_len = sizeof(struct msg_itf_alloc_mem_reply),
>> + };
>> + struct codec_dma_buf *buf;
>> + int ret;
>> +
>> + ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to get cma req %d", ret);
>> + return;
>> + }
>> +
>> + buf = kmalloc(sizeof(*buf), GFP_KERNEL);
>> + if (!buf)
>> + goto send_reply;
>> +
>> + buf->size = req.uSize;
>> + buf->vaddr =
>> + dma_alloc_coherent(dev, buf->size, &buf->paddr, GFP_KERNEL);
>> + if (!buf->vaddr) {
>> + dev_warn(dev, "Failed to allocate DMA buffer\n");
>> + goto send_reply;
>> + }
>> +
>> + reply.reply.phyAddr = (u64)buf->paddr;
>> + al_common_dma_buf_insert(codec, buf);
>
>The buffers allocated by the firmware are tracked by device. Thus, there
>is only one list for all buffers used by the firmware.
>
>I guess that the buffers are be allocated per context. If that's the
>case, maybe tracking them per context in the driver would be a better
>option.
>
The mcu can ask for buffers at the startup, and also it can reuse some
buffers, with different contexts.
>> +
>> +send_reply:
>> + ret = al_common_send(codec, &reply.hdr);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to reply to cma alloc");
>> + al_common_dma_buf_remove(codec, buf);
>> + }
>> +}
>> +
>> +static void handle_free_memory_req(struct al_codec_dev *codec,
>> + struct msg_itf_header *hdr)
>> +{
>> + struct msg_itf_free_mem_req req;
>> + struct msg_itf_free_mem_reply_full reply = {
>> + .hdr.type = MSG_ITF_TYPE_FREE_MEM_REPLY,
>> + .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
>> + .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
>> + .hdr.payload_len = sizeof(struct msg_itf_free_mem_reply),
>> + .reply.ret = -1,
>> + };
>> + struct codec_dma_buf *buf;
>> + int ret;
>> +
>> + ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to put cma req");
>> + return;
>> + }
>> +
>> + buf = al_common_dma_buf_lookup(codec, req.phyAddr);
>> + if (!buf) {
>> + al_codec_err(codec, "Unable to get dma handle for %p",
>> + (void *)(long)req.phyAddr);
>> + reply.reply.ret = -EINVAL;
>> + goto send_reply;
>> + }
>> +
>> + al_codec_dbg(codec, "Free memory %p, size %d",
>> + (void *)(long)req.phyAddr, buf->size);
>> +
>> + al_common_dma_buf_remove(codec, buf);
>> + reply.reply.ret = 0;
>> +
>> +send_reply:
>> + ret = al_common_send(codec, &reply.hdr);
>> + if (ret)
>> + al_codec_err(codec, "Unable to reply to cma free");
>> +}
>> +
>> +static void handle_mcu_console_print(struct al_codec_dev *codec,
>> + struct msg_itf_header *hdr)
>> +{
>> +#ifdef DEBUG
>
>What's the reason for making this a compile time option? Maybe a
>module parameter to enable MCU logging at runtime would be more
>friendly for debugging.
>
This will change to add debugfs. thanks
>> + struct msg_itf_write_req *req;
>> + int ret;
>> +
>> + /* one more byte to be sure to have a zero terminated string */
>> + req = kzalloc(hdr->payload_len + 1, GFP_KERNEL);
>> + if (!req) {
>> + al_common_skip_data(codec, hdr->payload_len);
>> + al_codec_err(codec, "Unable to alloc memory");
>> + return;
>> + }
>> +
>> + ret = al_codec_msg_get_data(&codec->mb_m2h, (char *)req,
>> + hdr->payload_len);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to get request");
>> + kfree(req);
>> + return;
>> + }
>> +
>> + /* Print the mcu logs */
>> + dev_dbg(&codec->pdev->dev, "[ALG_MCU] %s(),%d: %s\n", __func__,
>> + __LINE__, (char *)(req + 1));
>> + kfree(req);
>> +#else
>> + al_common_skip_data(codec, hdr->payload_len);
>> +#endif
>> +}
>> +
>[...]
>> +static int al_common_load_firmware_start(struct al_codec_dev *codec,
>> + const char *name)
>> +{
>> + struct device *dev = &codec->pdev->dev;
>> + dma_addr_t phys;
>> + size_t size;
>> + void *virt;
>> + int err;
>> +
>> + if (codec->firmware.virt)
>> + return 0;
>> +
>> + err = al_common_read_firmware(codec, name);
>> + if (err)
>> + return err;
>> +
>> + size = codec->firmware.size;
>> +
>> + virt = dma_alloc_coherent(dev, size, &phys, GFP_KERNEL);
>> + err = dma_mapping_error(dev, phys);
>> + if (err < 0)
>> + return err;
>> +
>> + codec->firmware.virt = virt;
>> + codec->firmware.phys = phys;
>> +
>> + al_common_copy_firmware_image(codec);
>> + err = al_common_parse_firmware_image(codec);
>> + if (err) {
>> + al_codec_err(codec, "failed to parse firmware image");
>> + goto cleanup;
>> + }
>> +
>> + err = al_common_setup_hw_regs(codec);
>> + if (err) {
>> + al_codec_err(codec, "Unable to setup hw registers");
>> + goto cleanup;
>> + }
>> +
>> + al_codec_mb_init(&codec->mb_h2m, virt + codec->firmware.mb_h2m.offset,
>> + codec->firmware.mb_h2m.size, MB_IFT_MAGIC_H2M);
>> +
>> + al_codec_mb_init(&codec->mb_m2h, virt + codec->firmware.mb_m2h.offset,
>> + codec->firmware.mb_m2h.size, MB_IFT_MAGIC_M2H);
>> +
>> + err = al_common_start_fw(codec);
>> + if (err) {
>> + al_codec_err(codec, "fw start has failed");
>> + goto cleanup;
>> + }
>
>If I understand correctly, the difference to the ZynqMP firmware is that
>the firmware can be configured for different addresses. Thus, the
>firmware and mailbox addresses on ZynqMP are determined by the bitstream
>synthesis, which this driver is free to allocate memory for the firmware
>and mailboxes wherever it wants. Correct?
>
There are some similarities between mailbox of the mcus ( zynqMP and
al300 ), but i'm actually working on the remoteproc and rpmsg for the mcu
control and IPC and messaging between host and the mcu.
the feature should be enabled with the encoder patches.
this should make the v4l2 easy with less files.
>> +
>> + al_codec_dbg(codec, "mcu has boot successfully !");
>> + codec->fw_ready_cb(codec->cb_arg);
>> +
>> + release_firmware(codec->firmware.firmware);
>> + codec->firmware.firmware = NULL;
>> +
>> + return 0;
>> +
>> +cleanup:
>> + dma_free_coherent(dev, size, virt, phys);
>> +
>> + return err;
>> +}
>> +
>> +static u64 al_common_get_periph_addr(struct al_codec_dev *codec)
>> +{
>> + struct resource *res;
>> +
>> + res = platform_get_resource_byname(codec->pdev, IORESOURCE_MEM, "apb");
>> + if (!res) {
>> + al_codec_err(codec, "Unable to find APB start address");
>> + return 0;
>> + }
>> +
>> + if (res->start & AL_CODEC_APB_MASK) {
>> + al_codec_err(codec, "APB start address is invalid");
>> + return 0;
>> + }
>> +
>> + return res->start;
>> +}
>> +
>> +int al_common_probe(struct al_codec_dev *codec, const char *name)
>> +{
>> + struct platform_device *pdev = codec->pdev;
>> + int irq;
>> + int ret;
>> +
>> + mutex_init(&codec->buf_lock);
>> + INIT_LIST_HEAD(&codec->alloc_buffers);
>> + init_completion(&codec->completion);
>> +
>> + /* setup dma memory */
>> + ret = al_common_setup_dma(codec);
>> + if (ret)
>> + return ret;
>> +
>> + /* Hw registers */
>> + codec->regs_info =
>> + platform_get_resource_byname(pdev, IORESOURCE_MEM, "regs");
>> + if (!codec->regs_info) {
>> + al_codec_err(codec, "regs resource missing from device tree");
>> + return -EINVAL;
>> + }
>> +
>> + codec->regs = devm_ioremap_resource(&pdev->dev, codec->regs_info);
>> + if (!codec->regs) {
>> + al_codec_err(codec, "failed to map registers");
>> + return -ENOMEM;
>> + }
>> +
>> + codec->apb = al_common_get_periph_addr(codec);
>> + if (!codec->apb)
>> + return -EINVAL;
>> +
>> + /* The MCU has already default clock value */
>> + codec->clk = devm_clk_get(&pdev->dev, NULL);
>> + if (IS_ERR(codec->clk)) {
>> + al_codec_err(codec, "failed to get MCU core clock");
>> + return PTR_ERR(codec->clk);
>> + }
>> +
>> + ret = clk_prepare_enable(codec->clk);
>> + if (ret) {
>> + al_codec_err(codec, "Cannot enable MCU clock: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + irq = platform_get_irq(pdev, 0);
>> + if (irq < 0) {
>> + al_codec_err(codec, "Failed to get IRQ");
>> + ret = -EINVAL;
>> + goto disable_clk;
>> + }
>> +
>> + ret = devm_request_threaded_irq(&pdev->dev, irq,
>> + al_common_hardirq_handler,
>> + al_common_irq_handler, IRQF_SHARED,
>> + dev_name(&pdev->dev), codec);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to register irq handler");
>> + goto disable_clk;
>> + }
>> +
>> + /* ok so request the fw */
>> + ret = al_common_load_firmware_start(codec, name);
>> + if (ret) {
>> + al_codec_err(codec, "failed to load firmware : %s", name);
>> + goto disable_clk;
>> + }
>> +
>> + return 0;
>> +
>> +disable_clk:
>> + clk_disable_unprepare(codec->clk);
>> +
>> + return ret;
>> +}
>> +
>[...]
>> diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_util.h b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..5f893db4a1a3f2b9e6e9109b81a956bcaa71851c
>> --- /dev/null
>> +++ b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
>> @@ -0,0 +1,186 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +/*
>> + * Copyright (c) 2025 Allegro DVT.
>> + * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
>> + */
>> +
>> +#ifndef __AL_CODEC_UTIL__
>> +#define __AL_CODEC_UTIL__
>> +
>> +#include <linux/mutex.h>
>> +#include <linux/types.h>
>> +#include <linux/v4l2-common.h>
>> +
>> +#include <media/v4l2-mem2mem.h>
>> +#include <media/videobuf2-v4l2.h>
>> +
>> +#define MB_IFT_MAGIC_H2M 0xabcd1230
>> +#define MB_IFT_MAGIC_M2H 0xabcd1231
>> +#define MB_IFT_VERSION 0x00010000
>> +
>> +#define MAJOR_SHIFT 20
>> +#define MAJOR_MASK 0xfff
>> +#define MINOR_SHIFT 8
>> +#define MINOR_MASK 0xfff
>> +#define PATCH_SHIFT 0
>> +#define PATCH_MASK 0xff
>> +
>> +/*
>> + * AL_BOOT_VERSION() - Version format 32-bit, 12 bits for the major,
>> + * the same for minor, 8bits for the patch
>> + */
>> +#define AL_BOOT_VERSION(major, minor, patch) \
>> + ((((major) & MAJOR_MASK) << MAJOR_SHIFT) | \
>> + (((minor) & MINOR_MASK) << MINOR_SHIFT) | \
>> + (((patch) & PATCH_MASK) << PATCH_SHIFT))
>> +
>> +#define al_phys_to_virt(x) ((void *)(uintptr_t)x)
>> +#define al_virt_to_phys(x) ((phys_addr_t)(uintptr_t)x)
>> +
>> +#define DECLARE_FULL_REQ(s) \
>> + struct s##_full { \
>> + struct msg_itf_header hdr; \
>> + struct s req; \
>> + } __packed
>> +
>> +#define DECLARE_FULL_REPLY(s) \
>> + struct s##_full { \
>> + struct msg_itf_header hdr; \
>> + struct s reply; \
>> + } __packed
>> +
>> +#define DECLARE_FULL_EVENT(s) \
>> + struct s##_full { \
>> + struct msg_itf_header hdr; \
>> + struct s event; \
>> + } __packed
>> +
>> +struct al_mb_itf {
>> + u32 magic;
>> + u32 version;
>> + u32 head;
>> + u32 tail;
>> +} __packed;
>> +
>> +struct al_codec_mb {
>> + struct al_mb_itf *hdr;
>> + struct mutex lock;
>> + char *data;
>> + int size;
>> +};
>
>On a first glance, this looks a lot like the allegro_mbox in the ZynqMP
>encoder driver. Even though the message format is different, would it be
>possible to generalize the general mailbox handling and use it for both
>drivers?
>
>Michael
>
>> +
>> +struct al_codec_cmd {
>> + struct kref refcount;
>> + struct list_head list;
>> + struct completion done;
>> + int reply_size;
>> + void *reply;
>> +};
>> +
>> +#define al_codec_err(codec, fmt, args...) \
>> + dev_err(&(codec)->pdev->dev, "[ALG_CODEC][ERROR] %s():%d: " fmt "\n", \
>> + __func__, __LINE__, ##args)
>> +
>> +#define al_v4l2_err(codec, fmt, args...) \
>> + dev_err(&(codec)->pdev->dev, "[ALG_V4L2][ERROR] %s():%d: " fmt "\n", \
>> + __func__, __LINE__, ##args)
>> +
>> +#if defined(DEBUG)
>> +
>> +extern int debug;
>> +
>> +/* V4L2 logs */
>> +#define al_v4l2_dbg(codec, level, fmt, args...) \
>> + do { \
>> + if (debug >= level) \
>> + dev_dbg(&(codec)->pdev->dev, \
>> + "[ALG_V4L2] level=%d %s(),%d: " fmt "\n", \
>> + level, __func__, __LINE__, ##args); \
>> + } while (0)
>> +
>> +/* Codec logs */
>> +#define al_codec_dbg(codec, fmt, args...) \
>> + do { \
>> + if (debug) \
>> + dev_dbg(&(codec)->pdev->dev, \
>> + "[ALG_CODEC] %s(),%d: " fmt "\n", __func__, \
>> + __LINE__, ##args); \
>> + } while (0)
>> +
>> +#else
>> +
>> +#define al_v4l2_dbg(codec, level, fmt, args...) \
>> + do { \
>> + (void)level; \
>> + dev_dbg(&(codec)->pdev->dev, "[ALG_V4L2]: " fmt "\n", \
>> + ##args); \
>> + } while (0)
>> +
>> +#define al_codec_dbg(codec, fmt, args...) \
>> + dev_dbg(&(codec)->pdev->dev, "[ALG_CODEC]: " fmt "\n", ##args)
>> +
>> +#endif
>> +
>> +#define MSG_ITF_TYPE_LIMIT BIT(10)
>> +
>> +/* Message types host <-> mcu */
>> +enum {
>> + MSG_ITF_TYPE_MCU_ALIVE = 0,
>> + MSG_ITF_TYPE_WRITE_REQ = 2,
>> + MSG_ITF_TYPE_FIRST_REQ = 1024,
>> + MSG_ITF_TYPE_NEXT_REQ,
>> + MSG_ITF_TYPE_FIRST_REPLY = 2048,
>> + MSG_ITF_TYPE_NEXT_REPLY,
>> + MSG_ITF_TYPE_ALLOC_MEM_REQ = 3072,
>> + MSG_ITF_TYPE_FREE_MEM_REQ,
>> + MSG_ITF_TYPE_ALLOC_MEM_REPLY = 4096,
>> + MSG_ITF_TYPE_FREE_MEM_REPLY,
>> + MSG_ITF_TYPE_FIRST_EVT = 5120,
>> + MSG_ITF_TYPE_NEXT_EVT = MSG_ITF_TYPE_FIRST_EVT
>> +};
>> +
>> +struct msg_itf_header {
>> + u64 drv_ctx_hdl;
>> + u64 drv_cmd_hdl;
>> + u16 type;
>> + u16 payload_len;
>> + u16 padding[2];
>> +} __packed;
>> +
>> +void al_codec_mb_init(struct al_codec_mb *mb, char *addr, int size, u32 magic);
>> +int al_codec_msg_get_header(struct al_codec_mb *mb, struct msg_itf_header *hdr);
>> +int al_codec_msg_get_data(struct al_codec_mb *mb, char *data, int len);
>> +int al_codec_msg_send(struct al_codec_mb *mb, struct msg_itf_header *hdr,
>> + void (*trigger)(void *), void *trigger_arg);
>> +
>> +static inline bool is_type_reply(uint16_t type)
>> +{
>> + return type >= MSG_ITF_TYPE_FIRST_REPLY &&
>> + type < MSG_ITF_TYPE_FIRST_REPLY + MSG_ITF_TYPE_LIMIT;
>> +}
>> +
>> +static inline bool is_type_event(uint16_t type)
>> +{
>> + return type >= MSG_ITF_TYPE_FIRST_EVT &&
>> + type < MSG_ITF_TYPE_FIRST_EVT + MSG_ITF_TYPE_LIMIT;
>> +}
>> +
>> +void al_codec_cmd_put(struct al_codec_cmd *cmd);
>> +
>> +struct al_codec_cmd *al_codec_cmd_create(int reply_size);
>> +
>> +static inline struct al_codec_cmd *al_codec_cmd_get(struct list_head *cmd_list,
>> + uint64_t hdl)
>> +{
>> + struct al_codec_cmd *cmd = NULL;
>> +
>> + list_for_each_entry(cmd, cmd_list, list) {
>> + if (likely(cmd == al_phys_to_virt(hdl))) {
>> + kref_get(&cmd->refcount);
>> + break;
>> + }
>> + }
>> + return list_entry_is_head(cmd, cmd_list, list) ? NULL : cmd;
>> +}
>> +
>> +#endif /* __AL_CODEC_UTIL__ */
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver
2025-06-12 13:30 ` Michael Tretter
2025-06-16 8:46 ` Yassine OUAISSA
@ 2025-06-16 12:56 ` Yassine OUAISSA
1 sibling, 0 replies; 13+ messages in thread
From: Yassine OUAISSA @ 2025-06-16 12:56 UTC (permalink / raw)
To: Michael Tretter, Mauro Carvalho Chehab, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Nicolas Dufresne, linux-kernel, linux-media, linux-arm-kernel,
devicetree
On 12.06.2025 15:30, Michael Tretter wrote:
Hi Michael,
Thank you for your review.
>Hi Yassine,
>
>Thanks for the patch.
>
>The overall architecture looks a lot like the video encoder for the
>ZynqMP, but with some significant differences in the details. I didn't
>manage to look more closely at the driver, yet, but I have a few high
>level questions.
>
There are a many difference between the ZynqMP, and the al300 drivers,
espicially on the mcu control. ( the al300 mcu is a 64bit ).
>On Thu, 05 Jun 2025 12:26:59 +0000, Yassine Ouaissa via B4 Relay wrote:
>> From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>>
>> This commit introduces a new allegro-dvt V4L2 stateful decoder driverfor
>> the Gen 3 IP with support for:
>> - AVC (H.264), HEVC (H.265), and JPEG decoding
>> - Output formats: NV12, NV16, I420, and P010 for capture
>>
>> v2:
>> - Replace the mutex_(lock/unlock) with the guard(mutex), that manage
>> mutexes more efficiently.
>> - Set DMA_BIT_MASK to 39, and drop the paddr check when allocating
>> dma_memory.
>> - Use dma_coerce_mask_and_coherent to set the DMA_MASK.
>> - Use static initializer for some structs.
>> - use #ifdef instead of #if defined
>> - Optimize some function.
>> - Use the declaration in the loop.
>> - Use codec for al_codec_dev instead of dev, to not get confused with
>> the device struct.
>> - Remove the codec member of the al_codec_dev, use the fmt->pixelformat
>> when request creating decoder instance.
>>
>> Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>> ---
>> drivers/media/platform/allegro-dvt/Kconfig | 1 +
>> drivers/media/platform/allegro-dvt/Makefile | 1 +
>> drivers/media/platform/allegro-dvt/al300/Kconfig | 23 +
>> drivers/media/platform/allegro-dvt/al300/Makefile | 6 +
>> .../platform/allegro-dvt/al300/al_codec_common.c | 733 ++++++++++
>> .../platform/allegro-dvt/al300/al_codec_common.h | 248 ++++
>> .../platform/allegro-dvt/al300/al_codec_util.c | 174 +++
>> .../platform/allegro-dvt/al300/al_codec_util.h | 186 +++
>> .../media/platform/allegro-dvt/al300/al_vdec_drv.c | 1518 ++++++++++++++++++++
>> .../media/platform/allegro-dvt/al300/al_vdec_drv.h | 93 ++
>> 10 files changed, 2983 insertions(+)
>>
>[...]
>> diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_common.c b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..716d0004482702537ea89ec4abecd6af26654b32
>> --- /dev/null
>> +++ b/drivers/media/platform/allegro-dvt/al300/al_codec_common.c
>[...]
>> +static void handle_alloc_memory_req(struct al_codec_dev *codec,
>> + struct msg_itf_header *hdr)
>> +{
>> + struct device *dev = &codec->pdev->dev;
>> + struct msg_itf_alloc_mem_req req;
>> + struct msg_itf_alloc_mem_reply_full reply = {
>> + .reply.phyAddr = 0,
>> + .hdr.type = MSG_ITF_TYPE_ALLOC_MEM_REPLY,
>> + .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
>> + .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
>> + .hdr.payload_len = sizeof(struct msg_itf_alloc_mem_reply),
>> + };
>> + struct codec_dma_buf *buf;
>> + int ret;
>> +
>> + ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to get cma req %d", ret);
>> + return;
>> + }
>> +
>> + buf = kmalloc(sizeof(*buf), GFP_KERNEL);
>> + if (!buf)
>> + goto send_reply;
>> +
>> + buf->size = req.uSize;
>> + buf->vaddr =
>> + dma_alloc_coherent(dev, buf->size, &buf->paddr, GFP_KERNEL);
>> + if (!buf->vaddr) {
>> + dev_warn(dev, "Failed to allocate DMA buffer\n");
>> + goto send_reply;
>> + }
>> +
>> + reply.reply.phyAddr = (u64)buf->paddr;
>> + al_common_dma_buf_insert(codec, buf);
>
>The buffers allocated by the firmware are tracked by device. Thus, there
>is only one list for all buffers used by the firmware.
>
>I guess that the buffers are be allocated per context. If that's the
>case, maybe tracking them per context in the driver would be a better
>option.
>
The mcu can ask for buffers at the startup, and also it can reuse some
buffers, with different contexts.
>> +
>> +send_reply:
>> + ret = al_common_send(codec, &reply.hdr);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to reply to cma alloc");
>> + al_common_dma_buf_remove(codec, buf);
>> + }
>> +}
>> +
>> +static void handle_free_memory_req(struct al_codec_dev *codec,
>> + struct msg_itf_header *hdr)
>> +{
>> + struct msg_itf_free_mem_req req;
>> + struct msg_itf_free_mem_reply_full reply = {
>> + .hdr.type = MSG_ITF_TYPE_FREE_MEM_REPLY,
>> + .hdr.drv_ctx_hdl = hdr->drv_ctx_hdl,
>> + .hdr.drv_cmd_hdl = hdr->drv_cmd_hdl,
>> + .hdr.payload_len = sizeof(struct msg_itf_free_mem_reply),
>> + .reply.ret = -1,
>> + };
>> + struct codec_dma_buf *buf;
>> + int ret;
>> +
>> + ret = al_common_get_data(codec, (char *)&req, hdr->payload_len);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to put cma req");
>> + return;
>> + }
>> +
>> + buf = al_common_dma_buf_lookup(codec, req.phyAddr);
>> + if (!buf) {
>> + al_codec_err(codec, "Unable to get dma handle for %p",
>> + (void *)(long)req.phyAddr);
>> + reply.reply.ret = -EINVAL;
>> + goto send_reply;
>> + }
>> +
>> + al_codec_dbg(codec, "Free memory %p, size %d",
>> + (void *)(long)req.phyAddr, buf->size);
>> +
>> + al_common_dma_buf_remove(codec, buf);
>> + reply.reply.ret = 0;
>> +
>> +send_reply:
>> + ret = al_common_send(codec, &reply.hdr);
>> + if (ret)
>> + al_codec_err(codec, "Unable to reply to cma free");
>> +}
>> +
>> +static void handle_mcu_console_print(struct al_codec_dev *codec,
>> + struct msg_itf_header *hdr)
>> +{
>> +#ifdef DEBUG
>
>What's the reason for making this a compile time option? Maybe a
>module parameter to enable MCU logging at runtime would be more
>friendly for debugging.
>
This will change to add debugfs. thanks
>> + struct msg_itf_write_req *req;
>> + int ret;
>> +
>> + /* one more byte to be sure to have a zero terminated string */
>> + req = kzalloc(hdr->payload_len + 1, GFP_KERNEL);
>> + if (!req) {
>> + al_common_skip_data(codec, hdr->payload_len);
>> + al_codec_err(codec, "Unable to alloc memory");
>> + return;
>> + }
>> +
>> + ret = al_codec_msg_get_data(&codec->mb_m2h, (char *)req,
>> + hdr->payload_len);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to get request");
>> + kfree(req);
>> + return;
>> + }
>> +
>> + /* Print the mcu logs */
>> + dev_dbg(&codec->pdev->dev, "[ALG_MCU] %s(),%d: %s\n", __func__,
>> + __LINE__, (char *)(req + 1));
>> + kfree(req);
>> +#else
>> + al_common_skip_data(codec, hdr->payload_len);
>> +#endif
>> +}
>> +
>[...]
>> +static int al_common_load_firmware_start(struct al_codec_dev *codec,
>> + const char *name)
>> +{
>> + struct device *dev = &codec->pdev->dev;
>> + dma_addr_t phys;
>> + size_t size;
>> + void *virt;
>> + int err;
>> +
>> + if (codec->firmware.virt)
>> + return 0;
>> +
>> + err = al_common_read_firmware(codec, name);
>> + if (err)
>> + return err;
>> +
>> + size = codec->firmware.size;
>> +
>> + virt = dma_alloc_coherent(dev, size, &phys, GFP_KERNEL);
>> + err = dma_mapping_error(dev, phys);
>> + if (err < 0)
>> + return err;
>> +
>> + codec->firmware.virt = virt;
>> + codec->firmware.phys = phys;
>> +
>> + al_common_copy_firmware_image(codec);
>> + err = al_common_parse_firmware_image(codec);
>> + if (err) {
>> + al_codec_err(codec, "failed to parse firmware image");
>> + goto cleanup;
>> + }
>> +
>> + err = al_common_setup_hw_regs(codec);
>> + if (err) {
>> + al_codec_err(codec, "Unable to setup hw registers");
>> + goto cleanup;
>> + }
>> +
>> + al_codec_mb_init(&codec->mb_h2m, virt + codec->firmware.mb_h2m.offset,
>> + codec->firmware.mb_h2m.size, MB_IFT_MAGIC_H2M);
>> +
>> + al_codec_mb_init(&codec->mb_m2h, virt + codec->firmware.mb_m2h.offset,
>> + codec->firmware.mb_m2h.size, MB_IFT_MAGIC_M2H);
>> +
>> + err = al_common_start_fw(codec);
>> + if (err) {
>> + al_codec_err(codec, "fw start has failed");
>> + goto cleanup;
>> + }
>
>If I understand correctly, the difference to the ZynqMP firmware is that
>the firmware can be configured for different addresses. Thus, the
>firmware and mailbox addresses on ZynqMP are determined by the bitstream
>synthesis, which this driver is free to allocate memory for the firmware
>and mailboxes wherever it wants. Correct?
>
There are some similarities between mailbox of the mcus ( zynqMP and
al300 ), but i'm actually working on the remoteproc and rpmsg for the mcu
control and IPC and messaging between host and the mcu.
the feature should be enabled with the encoder patches.
this should make the v4l2 easy with less files.
>> +
>> + al_codec_dbg(codec, "mcu has boot successfully !");
>> + codec->fw_ready_cb(codec->cb_arg);
>> +
>> + release_firmware(codec->firmware.firmware);
>> + codec->firmware.firmware = NULL;
>> +
>> + return 0;
>> +
>> +cleanup:
>> + dma_free_coherent(dev, size, virt, phys);
>> +
>> + return err;
>> +}
>> +
>> +static u64 al_common_get_periph_addr(struct al_codec_dev *codec)
>> +{
>> + struct resource *res;
>> +
>> + res = platform_get_resource_byname(codec->pdev, IORESOURCE_MEM, "apb");
>> + if (!res) {
>> + al_codec_err(codec, "Unable to find APB start address");
>> + return 0;
>> + }
>> +
>> + if (res->start & AL_CODEC_APB_MASK) {
>> + al_codec_err(codec, "APB start address is invalid");
>> + return 0;
>> + }
>> +
>> + return res->start;
>> +}
>> +
>> +int al_common_probe(struct al_codec_dev *codec, const char *name)
>> +{
>> + struct platform_device *pdev = codec->pdev;
>> + int irq;
>> + int ret;
>> +
>> + mutex_init(&codec->buf_lock);
>> + INIT_LIST_HEAD(&codec->alloc_buffers);
>> + init_completion(&codec->completion);
>> +
>> + /* setup dma memory */
>> + ret = al_common_setup_dma(codec);
>> + if (ret)
>> + return ret;
>> +
>> + /* Hw registers */
>> + codec->regs_info =
>> + platform_get_resource_byname(pdev, IORESOURCE_MEM, "regs");
>> + if (!codec->regs_info) {
>> + al_codec_err(codec, "regs resource missing from device tree");
>> + return -EINVAL;
>> + }
>> +
>> + codec->regs = devm_ioremap_resource(&pdev->dev, codec->regs_info);
>> + if (!codec->regs) {
>> + al_codec_err(codec, "failed to map registers");
>> + return -ENOMEM;
>> + }
>> +
>> + codec->apb = al_common_get_periph_addr(codec);
>> + if (!codec->apb)
>> + return -EINVAL;
>> +
>> + /* The MCU has already default clock value */
>> + codec->clk = devm_clk_get(&pdev->dev, NULL);
>> + if (IS_ERR(codec->clk)) {
>> + al_codec_err(codec, "failed to get MCU core clock");
>> + return PTR_ERR(codec->clk);
>> + }
>> +
>> + ret = clk_prepare_enable(codec->clk);
>> + if (ret) {
>> + al_codec_err(codec, "Cannot enable MCU clock: %d\n", ret);
>> + return ret;
>> + }
>> +
>> + irq = platform_get_irq(pdev, 0);
>> + if (irq < 0) {
>> + al_codec_err(codec, "Failed to get IRQ");
>> + ret = -EINVAL;
>> + goto disable_clk;
>> + }
>> +
>> + ret = devm_request_threaded_irq(&pdev->dev, irq,
>> + al_common_hardirq_handler,
>> + al_common_irq_handler, IRQF_SHARED,
>> + dev_name(&pdev->dev), codec);
>> + if (ret) {
>> + al_codec_err(codec, "Unable to register irq handler");
>> + goto disable_clk;
>> + }
>> +
>> + /* ok so request the fw */
>> + ret = al_common_load_firmware_start(codec, name);
>> + if (ret) {
>> + al_codec_err(codec, "failed to load firmware : %s", name);
>> + goto disable_clk;
>> + }
>> +
>> + return 0;
>> +
>> +disable_clk:
>> + clk_disable_unprepare(codec->clk);
>> +
>> + return ret;
>> +}
>> +
>[...]
>> diff --git a/drivers/media/platform/allegro-dvt/al300/al_codec_util.h b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..5f893db4a1a3f2b9e6e9109b81a956bcaa71851c
>> --- /dev/null
>> +++ b/drivers/media/platform/allegro-dvt/al300/al_codec_util.h
>> @@ -0,0 +1,186 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +/*
>> + * Copyright (c) 2025 Allegro DVT.
>> + * Author: Yassine OUAISSA <yassine.ouaissa@allegrodvt.fr>
>> + */
>> +
>> +#ifndef __AL_CODEC_UTIL__
>> +#define __AL_CODEC_UTIL__
>> +
>> +#include <linux/mutex.h>
>> +#include <linux/types.h>
>> +#include <linux/v4l2-common.h>
>> +
>> +#include <media/v4l2-mem2mem.h>
>> +#include <media/videobuf2-v4l2.h>
>> +
>> +#define MB_IFT_MAGIC_H2M 0xabcd1230
>> +#define MB_IFT_MAGIC_M2H 0xabcd1231
>> +#define MB_IFT_VERSION 0x00010000
>> +
>> +#define MAJOR_SHIFT 20
>> +#define MAJOR_MASK 0xfff
>> +#define MINOR_SHIFT 8
>> +#define MINOR_MASK 0xfff
>> +#define PATCH_SHIFT 0
>> +#define PATCH_MASK 0xff
>> +
>> +/*
>> + * AL_BOOT_VERSION() - Version format 32-bit, 12 bits for the major,
>> + * the same for minor, 8bits for the patch
>> + */
>> +#define AL_BOOT_VERSION(major, minor, patch) \
>> + ((((major) & MAJOR_MASK) << MAJOR_SHIFT) | \
>> + (((minor) & MINOR_MASK) << MINOR_SHIFT) | \
>> + (((patch) & PATCH_MASK) << PATCH_SHIFT))
>> +
>> +#define al_phys_to_virt(x) ((void *)(uintptr_t)x)
>> +#define al_virt_to_phys(x) ((phys_addr_t)(uintptr_t)x)
>> +
>> +#define DECLARE_FULL_REQ(s) \
>> + struct s##_full { \
>> + struct msg_itf_header hdr; \
>> + struct s req; \
>> + } __packed
>> +
>> +#define DECLARE_FULL_REPLY(s) \
>> + struct s##_full { \
>> + struct msg_itf_header hdr; \
>> + struct s reply; \
>> + } __packed
>> +
>> +#define DECLARE_FULL_EVENT(s) \
>> + struct s##_full { \
>> + struct msg_itf_header hdr; \
>> + struct s event; \
>> + } __packed
>> +
>> +struct al_mb_itf {
>> + u32 magic;
>> + u32 version;
>> + u32 head;
>> + u32 tail;
>> +} __packed;
>> +
>> +struct al_codec_mb {
>> + struct al_mb_itf *hdr;
>> + struct mutex lock;
>> + char *data;
>> + int size;
>> +};
>
>On a first glance, this looks a lot like the allegro_mbox in the ZynqMP
>encoder driver. Even though the message format is different, would it be
>possible to generalize the general mailbox handling and use it for both
>drivers?
>
>Michael
>
>> +
>> +struct al_codec_cmd {
>> + struct kref refcount;
>> + struct list_head list;
>> + struct completion done;
>> + int reply_size;
>> + void *reply;
>> +};
>> +
>> +#define al_codec_err(codec, fmt, args...) \
>> + dev_err(&(codec)->pdev->dev, "[ALG_CODEC][ERROR] %s():%d: " fmt "\n", \
>> + __func__, __LINE__, ##args)
>> +
>> +#define al_v4l2_err(codec, fmt, args...) \
>> + dev_err(&(codec)->pdev->dev, "[ALG_V4L2][ERROR] %s():%d: " fmt "\n", \
>> + __func__, __LINE__, ##args)
>> +
>> +#if defined(DEBUG)
>> +
>> +extern int debug;
>> +
>> +/* V4L2 logs */
>> +#define al_v4l2_dbg(codec, level, fmt, args...) \
>> + do { \
>> + if (debug >= level) \
>> + dev_dbg(&(codec)->pdev->dev, \
>> + "[ALG_V4L2] level=%d %s(),%d: " fmt "\n", \
>> + level, __func__, __LINE__, ##args); \
>> + } while (0)
>> +
>> +/* Codec logs */
>> +#define al_codec_dbg(codec, fmt, args...) \
>> + do { \
>> + if (debug) \
>> + dev_dbg(&(codec)->pdev->dev, \
>> + "[ALG_CODEC] %s(),%d: " fmt "\n", __func__, \
>> + __LINE__, ##args); \
>> + } while (0)
>> +
>> +#else
>> +
>> +#define al_v4l2_dbg(codec, level, fmt, args...) \
>> + do { \
>> + (void)level; \
>> + dev_dbg(&(codec)->pdev->dev, "[ALG_V4L2]: " fmt "\n", \
>> + ##args); \
>> + } while (0)
>> +
>> +#define al_codec_dbg(codec, fmt, args...) \
>> + dev_dbg(&(codec)->pdev->dev, "[ALG_CODEC]: " fmt "\n", ##args)
>> +
>> +#endif
>> +
>> +#define MSG_ITF_TYPE_LIMIT BIT(10)
>> +
>> +/* Message types host <-> mcu */
>> +enum {
>> + MSG_ITF_TYPE_MCU_ALIVE = 0,
>> + MSG_ITF_TYPE_WRITE_REQ = 2,
>> + MSG_ITF_TYPE_FIRST_REQ = 1024,
>> + MSG_ITF_TYPE_NEXT_REQ,
>> + MSG_ITF_TYPE_FIRST_REPLY = 2048,
>> + MSG_ITF_TYPE_NEXT_REPLY,
>> + MSG_ITF_TYPE_ALLOC_MEM_REQ = 3072,
>> + MSG_ITF_TYPE_FREE_MEM_REQ,
>> + MSG_ITF_TYPE_ALLOC_MEM_REPLY = 4096,
>> + MSG_ITF_TYPE_FREE_MEM_REPLY,
>> + MSG_ITF_TYPE_FIRST_EVT = 5120,
>> + MSG_ITF_TYPE_NEXT_EVT = MSG_ITF_TYPE_FIRST_EVT
>> +};
>> +
>> +struct msg_itf_header {
>> + u64 drv_ctx_hdl;
>> + u64 drv_cmd_hdl;
>> + u16 type;
>> + u16 payload_len;
>> + u16 padding[2];
>> +} __packed;
>> +
>> +void al_codec_mb_init(struct al_codec_mb *mb, char *addr, int size, u32 magic);
>> +int al_codec_msg_get_header(struct al_codec_mb *mb, struct msg_itf_header *hdr);
>> +int al_codec_msg_get_data(struct al_codec_mb *mb, char *data, int len);
>> +int al_codec_msg_send(struct al_codec_mb *mb, struct msg_itf_header *hdr,
>> + void (*trigger)(void *), void *trigger_arg);
>> +
>> +static inline bool is_type_reply(uint16_t type)
>> +{
>> + return type >= MSG_ITF_TYPE_FIRST_REPLY &&
>> + type < MSG_ITF_TYPE_FIRST_REPLY + MSG_ITF_TYPE_LIMIT;
>> +}
>> +
>> +static inline bool is_type_event(uint16_t type)
>> +{
>> + return type >= MSG_ITF_TYPE_FIRST_EVT &&
>> + type < MSG_ITF_TYPE_FIRST_EVT + MSG_ITF_TYPE_LIMIT;
>> +}
>> +
>> +void al_codec_cmd_put(struct al_codec_cmd *cmd);
>> +
>> +struct al_codec_cmd *al_codec_cmd_create(int reply_size);
>> +
>> +static inline struct al_codec_cmd *al_codec_cmd_get(struct list_head *cmd_list,
>> + uint64_t hdl)
>> +{
>> + struct al_codec_cmd *cmd = NULL;
>> +
>> + list_for_each_entry(cmd, cmd_list, list) {
>> + if (likely(cmd == al_phys_to_virt(hdl))) {
>> + kref_get(&cmd->refcount);
>> + break;
>> + }
>> + }
>> + return list_entry_is_head(cmd, cmd_list, list) ? NULL : cmd;
>> +}
>> +
>> +#endif /* __AL_CODEC_UTIL__ */
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP
2025-06-12 12:42 ` Michael Tretter
@ 2025-06-17 6:33 ` Yassine OUAISSA
0 siblings, 0 replies; 13+ messages in thread
From: Yassine OUAISSA @ 2025-06-17 6:33 UTC (permalink / raw)
To: Michael Tretter, Mauro Carvalho Chehab, Pengutronix Kernel Team,
Michal Simek, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Nicolas Dufresne, linux-kernel, linux-media, linux-arm-kernel,
devicetree
On 12.06.2025 14:42, Michael Tretter wrote:
>On Thu, 05 Jun 2025 12:26:57 +0000, Yassine Ouaissa via B4 Relay wrote:
>> From: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>>
>> Add compatible for video decoder on allegrodvt Gen 3 IP.
>>
>> v2:
>> - Change the YAML file name, use the existing vendor-prefix.
>> - Improuve the dt-bindings description.
>> - Change the device compatible identifier, from "allegrodvt, al300-vdec",
>> to "allegro, al300-vdec"
>> - Simplify the register property specification,
>> by using the simple min/max items constraint (Krzysztof Kozlowski)
>> - Remove the clock-names property. And remove it from the required
>> properties list (Krzysztof Kozlowski) (Conor Dooley)
>> - Use the simple maxItems constraint for the memory-region property.
>> Also for the firmware-name (Krzysztof Kozlowski)
>> - Example changes:
>> - Use header provides definitions for the interrupts (Conor Dooley)
>> - Improuve Interrupt specification using GIC constants (Conor Dooley)
>> - Use generic node name "video-decoder" (Krzysztof Kozlowski) (Conor Dooley)
>> - Remove unused label (Krzysztof Kozlowski)
>> - Change clock reference from <&mcu_clock_dec> to <&mcu_core_clk>
>> - Use hex format for reg property (Krzysztof Kozlowski) (Conor Dooley)
>> - Reduce memory region size (Krzysztof Kozlowski) (Conor Dooley)
>>
>> - Link v1: https://patchwork.linuxtv.org/project/linux-media/patch/20250511144752.504162-4-yassine.ouaissa@allegrodvt.com/
>>
>> Signed-off-by: Yassine Ouaissa <yassine.ouaissa@allegrodvt.com>
>> ---
>> .../bindings/media/allegro,al300-vdec.yaml | 75 ++++++++++++++++++++++
>> MAINTAINERS | 2 +
>> 2 files changed, 77 insertions(+)
>>
>> diff --git a/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..26f9ac39682431b1d4828aed5d1ed43ef099e204
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
>> @@ -0,0 +1,75 @@
>> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/media/allegro,al300-vdec.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Allegro DVT Video IP Decoder Gen 3
>> +
>> +maintainers:
>> + - Yassine OUAISSA <yassine.ouaissa@allegrodvt.com>
>> +
>> +description: The al300-vdec represents the gen 3 of Allegro DVT IP video
>> + decoding technology, offering significant advancements over its
>> + predecessors. This new decoder features enhanced processing capabilities
>> + with improved throughput and reduced latency.
>> +
>> + Communication between the host driver software and the MCU is implemented
>> + through a specialized mailbox interface mechanism. This mailbox system
>> + provides a structured channel for exchanging commands, parameters, and
>> + status information between the host CPU and the MCU controlling the codec
>> + engines.
>> +
>> +properties:
>> + compatible:
>> + const: allegro,al300-vdec
>> +
>> + reg:
>> + maxItems: 2
>> + minItems: 2
>> +
>> + reg-names:
>> + items:
>> + - const: regs
>> + - const: apb
>
>If I understand correctly, "regs" are the registers to control the MCU
>and "apb" are the registers of the actual codec engines, which is
>controlled by the MCU. The driver never accesses the apb registers, but
>uses the apb address only to configure the firmware and tell it, where
>the registers of the codec engines are found.
>
>Maybe a separate node for the actual codec that is referred via a
>phandle could be a better description of the hardware?
>
the regs is actually is used to configure the decoder and also the mcu.
the APB address is used to map the mcu peripheral, which includes the
codec engine.
I think it's sampler to let it as reg.
the naming has changed in the v3.
>> +
>> + interrupts:
>> + maxItems: 1
>> +
>> + clocks:
>> + items:
>> + - description: MCU core clock
>> +
>> + memory-region:
>> + maxItems: 1
>> +
>> + firmware-name:
>> + maxItems: 1
>> +
>> +required:
>> + - compatible
>> + - reg
>> + - reg-names
>> + - interrupts
>> + - clocks
>> +
>> +additionalProperties: False
>> +
>> +examples:
>> + - |
>> + #include <dt-bindings/interrupt-controller/arm-gic.h>
>> +
>> + axi {
>> + #address-cells = <2>;
>> + #size-cells = <2>;
>> +
>> + video-decoder@a0120000 {
>> + compatible = "allegro,al300-vdec";
>> + reg = <0x00 0xa0120000 0x00 0x10000>,
>> + <0x01 0x80000000 0x00 0x8000>;
>> + reg-names = "regs", "apb";
>> + interrupts = <GIC_SPI 96 IRQ_TYPE_LEVEL_HIGH>;
>> + clocks = <&mcu_core_clk>;
>> + firmware-name = "al300_vdec.fw";
>> + };
>> + };
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index abc6ba61048771303bc219102f2db602266b7c30..1ff78b9a76cb8cdf32263fcd9b4579b4a2bb6b2a 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -816,7 +816,9 @@ M: Michael Tretter <m.tretter@pengutronix.de>
>> R: Pengutronix Kernel Team <kernel@pengutronix.de>
>> L: linux-media@vger.kernel.org
>> S: Maintained
>> +F: Documentation/devicetree/bindings/media/allegro,al300-vdec.yaml
>> F: Documentation/devicetree/bindings/media/allegro,al5e.yaml
>> +F: drivers/media/platform/allegro-dvt/al300
>> F: drivers/media/platform/allegro-dvt/zynqmp
>>
>> ALLIED VISION ALVIUM CAMERA DRIVER
>>
>> --
>> 2.30.2
>>
>>
>>
Best regards,
Yassine OUAISSA
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2025-06-17 6:34 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-05 12:26 [PATCH v2 0/4] media: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 1/4] media: allegro-dvt: Move the current driver to a subdirectory Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 2/4] dt-bindings: media: allegro-dvt: add decoder dt-bindings for Gen3 IP Yassine Ouaissa via B4 Relay
2025-06-05 13:01 ` Krzysztof Kozlowski
2025-06-05 13:29 ` Yassine OUAISSA
2025-06-12 12:42 ` Michael Tretter
2025-06-17 6:33 ` Yassine OUAISSA
2025-06-05 12:26 ` [PATCH v2 3/4] dt-bindings: vendor-prefixes: Update the description of allegro prefix Yassine Ouaissa via B4 Relay
2025-06-05 12:26 ` [PATCH v2 4/4] media: allegro-dvt: Add Gen 3 IP stateful decoder driver Yassine Ouaissa via B4 Relay
2025-06-12 13:30 ` Michael Tretter
2025-06-16 8:46 ` Yassine OUAISSA
2025-06-16 12:56 ` Yassine OUAISSA
2025-06-05 12:57 ` [PATCH v2 0/4] media: " Krzysztof Kozlowski
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).