* [PATCH 0/4] media: cedrus: Add a JPEG decoder
@ 2024-07-31 16:44 Emmanuel Gil Peyrot
2024-07-31 16:44 ` [PATCH 1/4] media: cedrus: Setup secondary output formats Emmanuel Gil Peyrot
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Emmanuel Gil Peyrot @ 2024-07-31 16:44 UTC (permalink / raw)
To: linux-sunxi
Cc: Emmanuel Gil Peyrot, Maxime Ripard, Paul Kocialkowski,
Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai,
Jernej Skrabec, Samuel Holland, linux-kernel, linux-media,
linux-staging, linux-arm-kernel
The Video Engine present on all (?) AllWinner SoCs includes a JPEG
decoder, which shares the same engine as the MPEG2 decoder. I mostly
used the linux-sunxi resources on the engine[0], as well as some
reverse-engineered userland using the legacy /dev/cedar_dev API[1].
I wrote the Onix library[2] to exercise this driver. With it, I have
tested this series on the A10, A20 and A64, as found in Olimex Lime,
Lime 2 and PinePhone respectively, so further testing on other SoCs
would be very welcome.
On the A10 and A20, since only the tiled ST12 format is supported, the
file dump binary is a bit unusable, but I have tested displaying to DRM
and to Wayland just fine on these SoCs.
They also only support downscaling from 4:4:4 to 4:2:2, unlike newer
SoCs which support downscaling from either 4:4:4 or 4:2:2 to 4:2:2 or
4:2:0. This prompts the issue of which formats to advertise: currently
I chose to only expose NV12, but ideally NV16 would be exposed when the
user tries to decode a 4:2:2 or 4:4:4 image. By the time the user has
called ENUM_FMTS on the capture queue, the driver doesn’t know the
subsampling in the JPEG file, so it can’t expose only the supported
formats. A possible solution could be to add a new v4l2_ext_ctrl
describing the JPEG, but this is a new uAPI which should get discussed
outside of this driver.
[0] https://linux-sunxi.org/VE_Register_guide
[1] https://github.com/jemk/cedrus
[2] https://linkmauve.fr/dev/onix/
Emmanuel Gil Peyrot (1):
media: cedrus: Add JPEG decoder
Jernej Skrabec (3):
media: cedrus: Setup secondary output formats
media: cedrus: Enable JPEG decoding in various variants
media: cedrus: Don't require requests for all codecs
drivers/staging/media/sunxi/cedrus/Kconfig | 1 +
drivers/staging/media/sunxi/cedrus/Makefile | 2 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 10 +
drivers/staging/media/sunxi/cedrus/cedrus.h | 5 +
.../staging/media/sunxi/cedrus/cedrus_hw.c | 14 +-
.../staging/media/sunxi/cedrus/cedrus_jpeg.c | 352 ++++++++++++++++++
.../staging/media/sunxi/cedrus/cedrus_regs.h | 25 +-
.../staging/media/sunxi/cedrus/cedrus_video.c | 11 +-
8 files changed, 413 insertions(+), 7 deletions(-)
create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c
--
2.45.2
^ permalink raw reply [flat|nested] 9+ messages in thread* [PATCH 1/4] media: cedrus: Setup secondary output formats 2024-07-31 16:44 [PATCH 0/4] media: cedrus: Add a JPEG decoder Emmanuel Gil Peyrot @ 2024-07-31 16:44 ` Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 2/4] media: cedrus: Add JPEG decoder Emmanuel Gil Peyrot ` (2 subsequent siblings) 3 siblings, 0 replies; 9+ messages in thread From: Emmanuel Gil Peyrot @ 2024-07-31 16:44 UTC (permalink / raw) To: linux-sunxi Cc: Jernej Skrabec, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel From: Jernej Skrabec <jernej.skrabec@gmail.com> It's needed for JPEG codec, since engine outputs to it for some reason. Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> --- drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 12 +++++++++--- drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 9 +++++++-- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c index 32af0e96e762..582b6cb796ea 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c @@ -88,15 +88,21 @@ void cedrus_dst_format_set(struct cedrus_dev *dev, case V4L2_PIX_FMT_NV12: chroma_size = ALIGN(width, 16) * ALIGN(height, 16) / 2; - reg = VE_PRIMARY_OUT_FMT_NV12; + reg = VE_PRIMARY_OUT_FMT_NV12 | VE_SECONDARY_OUT_FMT_EXT_NV12; cedrus_write(dev, VE_PRIMARY_OUT_FMT, reg); reg = chroma_size / 2; cedrus_write(dev, VE_PRIMARY_CHROMA_BUF_LEN, reg); - reg = VE_PRIMARY_FB_LINE_STRIDE_LUMA(ALIGN(width, 16)) | - VE_PRIMARY_FB_LINE_STRIDE_CHROMA(ALIGN(width, 16) / 2); + reg = VE_CHROMA_BUF_LEN_SDRT(chroma_size / 2) | + VE_CHROMA_WIDTH_ALIGN_8 | + VE_SECONDARY_OUT_FMT_EXT; + cedrus_write(dev, VE_CHROMA_BUF_LEN, reg); + + reg = VE_FB_LINE_STRIDE_LUMA(ALIGN(width, 16)) | + VE_FB_LINE_STRIDE_CHROMA(ALIGN(width, 16) / 2); cedrus_write(dev, VE_PRIMARY_FB_LINE_STRIDE, reg); + cedrus_write(dev, VE_SECONDARY_FB_LINE_STRIDE, reg); break; case V4L2_PIX_FMT_NV12_32L32: diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h index 05e6cbc548ab..3acc05e0fb54 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h @@ -59,9 +59,10 @@ #define VE_INTRAPRED_DRAM_BUF_ADDR 0x58 #define VE_PRIMARY_CHROMA_BUF_LEN 0xc4 #define VE_PRIMARY_FB_LINE_STRIDE 0xc8 +#define VE_SECONDARY_FB_LINE_STRIDE 0xcc -#define VE_PRIMARY_FB_LINE_STRIDE_CHROMA(s) SHIFT_AND_MASK_BITS(s, 31, 16) -#define VE_PRIMARY_FB_LINE_STRIDE_LUMA(s) SHIFT_AND_MASK_BITS(s, 15, 0) +#define VE_FB_LINE_STRIDE_CHROMA(s) SHIFT_AND_MASK_BITS(s, 31, 16) +#define VE_FB_LINE_STRIDE_LUMA(s) SHIFT_AND_MASK_BITS(s, 15, 0) #define VE_CHROMA_BUF_LEN 0xe8 @@ -69,6 +70,10 @@ #define VE_SECONDARY_OUT_FMT_EXT (0x01 << 30) #define VE_SECONDARY_OUT_FMT_YU12 (0x02 << 30) #define VE_SECONDARY_OUT_FMT_YV12 (0x03 << 30) +#define VE_CHROMA_WIDTH_ALIGN_16 (0x00 << 29) +#define VE_CHROMA_WIDTH_ALIGN_8 (0x01 << 29) +#define VE_LUMA_WIDTH_ALIGN_16 (0x00 << 28) +#define VE_LUMA_WIDTH_ALIGN_32 (0x01 << 28) #define VE_CHROMA_BUF_LEN_SDRT(l) SHIFT_AND_MASK_BITS(l, 27, 0) #define VE_PRIMARY_OUT_FMT 0xec -- 2.45.2 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/4] media: cedrus: Add JPEG decoder 2024-07-31 16:44 [PATCH 0/4] media: cedrus: Add a JPEG decoder Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 1/4] media: cedrus: Setup secondary output formats Emmanuel Gil Peyrot @ 2024-07-31 16:44 ` Emmanuel Gil Peyrot 2025-02-13 17:12 ` Sebastian Fricke 2024-07-31 16:44 ` [PATCH 3/4] media: cedrus: Enable JPEG decoding in various variants Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 4/4] media: cedrus: Don't require requests for all codecs Emmanuel Gil Peyrot 3 siblings, 1 reply; 9+ messages in thread From: Emmanuel Gil Peyrot @ 2024-07-31 16:44 UTC (permalink / raw) To: linux-sunxi Cc: Emmanuel Gil Peyrot, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Jernej Skrabec, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel Basically all Cedrus variants support JPEG decoding. Add code for it. Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> --- drivers/staging/media/sunxi/cedrus/Kconfig | 1 + drivers/staging/media/sunxi/cedrus/Makefile | 2 +- drivers/staging/media/sunxi/cedrus/cedrus.h | 5 + .../staging/media/sunxi/cedrus/cedrus_hw.c | 2 + .../staging/media/sunxi/cedrus/cedrus_jpeg.c | 352 ++++++++++++++++++ .../staging/media/sunxi/cedrus/cedrus_regs.h | 16 + .../staging/media/sunxi/cedrus/cedrus_video.c | 9 + 7 files changed, 386 insertions(+), 1 deletion(-) create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c diff --git a/drivers/staging/media/sunxi/cedrus/Kconfig b/drivers/staging/media/sunxi/cedrus/Kconfig index cb07a343c9c2..5683519aead0 100644 --- a/drivers/staging/media/sunxi/cedrus/Kconfig +++ b/drivers/staging/media/sunxi/cedrus/Kconfig @@ -9,6 +9,7 @@ config VIDEO_SUNXI_CEDRUS select SUNXI_SRAM select VIDEOBUF2_DMA_CONTIG select V4L2_MEM2MEM_DEV + select V4L2_JPEG_HELPER help Support for the VPU found in Allwinner SoCs, also known as the Cedar video engine. diff --git a/drivers/staging/media/sunxi/cedrus/Makefile b/drivers/staging/media/sunxi/cedrus/Makefile index a647b3690bf8..fa3e949e0788 100644 --- a/drivers/staging/media/sunxi/cedrus/Makefile +++ b/drivers/staging/media/sunxi/cedrus/Makefile @@ -3,4 +3,4 @@ obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o \ cedrus_mpeg2.o cedrus_h264.o cedrus_h265.o \ - cedrus_vp8.o + cedrus_vp8.o cedrus_jpeg.o diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h index 522c184e2afc..555f8d124d47 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h @@ -34,6 +34,7 @@ #define CEDRUS_CAPABILITY_MPEG2_DEC BIT(3) #define CEDRUS_CAPABILITY_VP8_DEC BIT(4) #define CEDRUS_CAPABILITY_H265_10_DEC BIT(5) +#define CEDRUS_CAPABILITY_JPEG_DEC BIT(6) enum cedrus_irq_status { CEDRUS_IRQ_NONE, @@ -152,6 +153,9 @@ struct cedrus_ctx { u8 *entropy_probs_buf; dma_addr_t entropy_probs_buf_dma; } vp8; + struct { + unsigned int subsampling; + } jpeg; } codec; }; @@ -201,6 +205,7 @@ extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2; extern struct cedrus_dec_ops cedrus_dec_ops_h264; extern struct cedrus_dec_ops cedrus_dec_ops_h265; extern struct cedrus_dec_ops cedrus_dec_ops_vp8; +extern struct cedrus_dec_ops cedrus_dec_ops_jpeg; static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val) { diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c index 582b6cb796ea..e237f7d66f7e 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c @@ -43,7 +43,9 @@ int cedrus_engine_enable(struct cedrus_ctx *ctx) reg |= VE_MODE_DDR_MODE_BW_128; switch (ctx->src_fmt.pixelformat) { + /* MPEG2 and JPEG both use the same decoding mode bit. */ case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_JPEG: reg |= VE_MODE_DEC_MPEG; break; diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c b/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c new file mode 100644 index 000000000000..1e8978ebf9dd --- /dev/null +++ b/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c @@ -0,0 +1,352 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Cedrus VPU driver + * + * Copyright (C) 2022 Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> + */ + +#include <media/videobuf2-dma-contig.h> +#include <media/v4l2-jpeg.h> + +#include "cedrus.h" +#include "cedrus_hw.h" +#include "cedrus_regs.h" + +static enum cedrus_irq_status cedrus_jpeg_irq_status(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + u32 reg; + + reg = cedrus_read(dev, VE_DEC_MPEG_STATUS); + reg &= VE_DEC_MPEG_STATUS_CHECK_MASK; + + if (!reg) + return CEDRUS_IRQ_NONE; + + if (reg & VE_DEC_MPEG_STATUS_CHECK_ERROR) + return CEDRUS_IRQ_ERROR; + + return CEDRUS_IRQ_OK; +} + +static void cedrus_jpeg_irq_clear(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + + cedrus_write(dev, VE_DEC_MPEG_STATUS, VE_DEC_MPEG_STATUS_CHECK_MASK); +} + +static void cedrus_jpeg_irq_disable(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + u32 reg = cedrus_read(dev, VE_DEC_MPEG_CTRL); + + reg &= ~VE_DEC_MPEG_CTRL_IRQ_MASK; + + cedrus_write(dev, VE_DEC_MPEG_CTRL, reg); +} + +static int cedrus_write_table_header(struct cedrus_dev *dev, + struct v4l2_jpeg_reference *table) +{ + u16 start_codes[16], code; + u8 offsets[16], *ptr; + unsigned int count; + u32 *ptr32; + int i; + + ptr = table->start; + if (!ptr) + return -EINVAL; + + count = 0; + code = 0; + for (i = 0; i < 16; i++) { + offsets[i] = count; + start_codes[i] = code; + count += ptr[i]; + code += ptr[i]; + code *= 2; + } + + for (i = 15; i >= 0 && !ptr[i]; i--) + start_codes[i] = 0xffff; + + ptr32 = (u32*)start_codes; + for (i = 0; i < 8; i++) + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr32[i]); + + ptr32 = (u32*)offsets; + for (i = 0; i < 4; i++) + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr32[i]); + + for (i = 0; i < 4; i++) + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); + + return 0; +} + +static int cedrus_jpeg_write_dh_tables(struct cedrus_dev *dev, + struct v4l2_jpeg_header *hdr) +{ + struct v4l2_jpeg_reference *tables[4], *table; + struct v4l2_jpeg_scan_component_spec *comp; + unsigned int i, j, ret; + size_t length; + u32 *ptr, val; + + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_OFFSET, 0); + + j = 0; + for (i = 0; i < 2; i++) { + comp = &hdr->scan->component[i]; + + tables[j++] = &hdr->huffman_tables[comp->dc_entropy_coding_table_selector]; + tables[j++] = &hdr->huffman_tables[comp->ac_entropy_coding_table_selector + 2]; + } + + for (i = 0; i < 4; i++) { + ret = cedrus_write_table_header(dev, tables[i]); + if (ret) + return ret; + } + + for (i = 0; i < 192; i++) + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); + + for (i = 0; i < 4; i++) { + table = tables[i]; + ptr = (u32*)&table->start[16]; + length = table->length - 16; + + for (j = 0; j < length / 4; j++) + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr[j]); + + if (length & 3) { + val = 0; + for (j = 0; j < (length & 3); j++) + val = (val << 8) | table->start[15 + length - j]; + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, val); + } + + for (j = 0; j < 64 - DIV_ROUND_UP(length, 4); j++) + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); + } + + return 0; +} + +static int cedrus_write_quantization_matrix(struct cedrus_dev *dev, u32 flags, + struct v4l2_jpeg_reference *table) +{ + const u8 *matrix; + u32 reg, val; + int i; + + matrix = table->start; + if (!matrix) + return -EINVAL; + + for (i = 0; i < 64; i++) { + /* determine if values are 8 or 16 bits */ + val = *matrix++; + if (table->length > 64) + val = (val << 8) | *matrix++; + + reg = VE_DEC_MPEG_IQMINPUT_WEIGHT(i, val); + reg |= flags; + + cedrus_write(dev, VE_DEC_MPEG_IQMINPUT, reg); + } + + return 0; +} + +static int cedrus_jpeg_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) +{ + struct cedrus_dev *dev = ctx->dev; + dma_addr_t src_buf_addr, dst_luma_addr, dst_chroma_addr; + struct v4l2_jpeg_scan_header scan_header; + struct v4l2_jpeg_reference quantization_tables[4] = { }; + struct v4l2_jpeg_reference huffman_tables[4] = { }; + struct v4l2_jpeg_header header = { + .scan = &scan_header, + .quantization_tables = quantization_tables, + .huffman_tables = huffman_tables, + }; + struct vb2_buffer *src_buf = &run->src->vb2_buf; + struct v4l2_jpeg_frame_component_spec *components; + u32 reg, subsampling; + unsigned long size; + int ret, index; + u8 hmax, vmax; + u16 width, height; + + size = vb2_get_plane_payload(src_buf, 0); + ret = v4l2_jpeg_parse_header(vb2_plane_vaddr(src_buf, 0), size, &header); + if (ret < 0) { + v4l2_err(&dev->v4l2_dev, + "failed to parse JPEG header: %d\n", ret); + return -EINVAL; + } + + width = header.frame.width; + height = header.frame.height; + if (width > 2048 || height > 2048) { + v4l2_err(&dev->v4l2_dev, + "unsupported JPEG of resolution %ux%u (max 2048x2048)\n", + width, height); + return -EINVAL; + } + + if (header.frame.precision != 8) { + v4l2_err(&dev->v4l2_dev, + "unsupported JPEG with %u bits of precision (8 required)\n", + header.frame.precision); + return -EINVAL; + } + + if (header.frame.num_components != 3) { + v4l2_err(&dev->v4l2_dev, + "unsupported JPEG with %u components (3 required)\n", + header.frame.num_components); + return -EINVAL; + } + + components = header.frame.component; + index = components[0].horizontal_sampling_factor << 20 | + components[0].vertical_sampling_factor << 16 | + components[1].horizontal_sampling_factor << 12 | + components[1].vertical_sampling_factor << 8 | + components[2].horizontal_sampling_factor << 4 | + components[2].vertical_sampling_factor; + + switch (index) { + case 0x221111: + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_420; + break; + case 0x211111: + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422; + break; + case 0x111111: + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444; + break; + case 0x121111: + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T; + break; + default: + v4l2_err(&dev->v4l2_dev, "unsupported subsampling %d\n", index); + return -EINVAL; + } + + ctx->codec.jpeg.subsampling = subsampling; + + /* Activate MPEG engine and select JPEG subengine. */ + cedrus_engine_enable(ctx); + + reg = VE_DEC_MPEG_TRIGGER_JPEG | subsampling; + cedrus_write(dev, VE_DEC_MPEG_TRIGGER, reg); + + /* Set restart interval. */ + cedrus_write(dev, VE_DEC_MPEG_JPEG_RES_INT, header.restart_interval); + + /* Set resolution in blocks. */ + hmax = components[0].horizontal_sampling_factor; + vmax = components[0].vertical_sampling_factor; + for (index = 1; index < 3; index++) { + if (hmax < components[index].horizontal_sampling_factor) + hmax = components[index].horizontal_sampling_factor; + if (vmax < components[index].vertical_sampling_factor) + vmax = components[index].vertical_sampling_factor; + } + + reg = VE_DEC_MPEG_JPEG_SIZE_WIDTH(DIV_ROUND_UP(width, 8 * hmax)); + reg |= VE_DEC_MPEG_JPEG_SIZE_HEIGHT(DIV_ROUND_UP(height, 8 * vmax)); + cedrus_write(dev, VE_DEC_MPEG_JPEG_SIZE, reg); + + /* Set intra quantisation matrix. */ + index = components[0].quantization_table_selector; + ret = cedrus_write_quantization_matrix(dev, + VE_DEC_MPEG_IQMINPUT_FLAG_INTRA, + &quantization_tables[index]); + if (ret) + return ret; + + /* Set non-intra quantisation matrix. */ + index = components[1].quantization_table_selector; + ret = cedrus_write_quantization_matrix(dev, + VE_DEC_MPEG_IQMINPUT_FLAG_NON_INTRA, + &quantization_tables[index]); + if (ret) + return ret; + + /* Set Diffie-Huffman tables. */ + ret = cedrus_jpeg_write_dh_tables(dev, &header); + if (ret) + return ret; + + /* Destination luma and chroma buffers. */ + + dst_luma_addr = cedrus_dst_buf_addr(ctx, &run->dst->vb2_buf, 0); + dst_chroma_addr = cedrus_dst_buf_addr(ctx, &run->dst->vb2_buf, 1); + + /* JPEG outputs to rotation/scale down output buffers */ + cedrus_write(dev, VE_DEC_MPEG_ROT_LUMA, dst_luma_addr); + cedrus_write(dev, VE_DEC_MPEG_ROT_CHROMA, dst_chroma_addr); + + /* Disable rotation and scaling. */ + cedrus_write(dev, VE_DEC_MPEG_SD_ROT_DBLK_CTL, 0); + + /* Source offset and length in bits. */ + + cedrus_write(dev, VE_DEC_MPEG_VLD_OFFSET, 8 * header.ecs_offset); + + reg = size * 8; + cedrus_write(dev, VE_DEC_MPEG_VLD_LEN, reg); + + /* Source beginning and end addresses. */ + + src_buf_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0); + + reg = VE_DEC_MPEG_VLD_ADDR_BASE(src_buf_addr); + reg |= VE_DEC_MPEG_VLD_ADDR_VALID_PIC_DATA; + reg |= VE_DEC_MPEG_VLD_ADDR_LAST_PIC_DATA; + reg |= VE_DEC_MPEG_VLD_ADDR_FIRST_PIC_DATA; + + cedrus_write(dev, VE_DEC_MPEG_VLD_ADDR, reg); + + reg = src_buf_addr + size; + cedrus_write(dev, VE_DEC_MPEG_VLD_END_ADDR, reg); + + /* Enable appropriate interrupts and components. */ + + reg = VE_DEC_MPEG_CTRL_IRQ_MASK; + if (subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422 || + subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T || + subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444) + reg |= VE_DEC_MPEG_CTRL_JPEG_FORCE_420; + + cedrus_write(dev, VE_DEC_MPEG_CTRL, reg); + + return 0; +} + +static void cedrus_jpeg_trigger(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + u32 reg; + + /* Trigger JPEG engine. */ + reg = VE_DEC_MPEG_TRIGGER_HW_JPEG_VLD | VE_DEC_MPEG_TRIGGER_JPEG; + reg |= ctx->codec.jpeg.subsampling; + + cedrus_write(dev, VE_DEC_MPEG_TRIGGER, reg); +} + +struct cedrus_dec_ops cedrus_dec_ops_jpeg = { + .irq_clear = cedrus_jpeg_irq_clear, + .irq_disable = cedrus_jpeg_irq_disable, + .irq_status = cedrus_jpeg_irq_status, + .setup = cedrus_jpeg_setup, + .trigger = cedrus_jpeg_trigger, +}; diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h index 3acc05e0fb54..cb8b4bb4f44e 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h @@ -144,6 +144,7 @@ #define VE_DEC_MPEG_CTRL_MC_CACHE_EN BIT(31) #define VE_DEC_MPEG_CTRL_SW_VLD BIT(27) +#define VE_DEC_MPEG_CTRL_JPEG_FORCE_420 BIT(18) #define VE_DEC_MPEG_CTRL_SW_IQ_IS BIT(17) #define VE_DEC_MPEG_CTRL_QP_AC_DC_OUT_EN BIT(14) #define VE_DEC_MPEG_CTRL_ROTATE_SCALE_OUT_EN BIT(8) @@ -165,6 +166,7 @@ #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422 (0x02 << 27) #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444 (0x03 << 27) #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T (0x04 << 27) +#define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_400 (0x05 << 27) #define VE_DEC_MPEG_TRIGGER_MPEG1 (0x01 << 24) #define VE_DEC_MPEG_TRIGGER_MPEG2 (0x02 << 24) @@ -255,10 +257,24 @@ #define VE_DEC_MPEG_IQMINPUT_WEIGHT(i, v) \ (SHIFT_AND_MASK_BITS(i, 13, 8) | SHIFT_AND_MASK_BITS(v, 7, 0)) +#define VE_DEC_MPEG_JPEG_SIZE (VE_ENGINE_DEC_MPEG + 0xb8) + +#define VE_DEC_MPEG_JPEG_SIZE_WIDTH(w) \ + SHIFT_AND_MASK_BITS((w) - 1, 10, 0) +#define VE_DEC_MPEG_JPEG_SIZE_HEIGHT(h) \ + SHIFT_AND_MASK_BITS((h) - 1, 26, 16) + +#define VE_DEC_MPEG_JPEG_MCU (VE_ENGINE_DEC_MPEG + 0xbc) +#define VE_DEC_MPEG_JPEG_RES_INT (VE_ENGINE_DEC_MPEG + 0xc0) #define VE_DEC_MPEG_ERROR (VE_ENGINE_DEC_MPEG + 0xc4) #define VE_DEC_MPEG_CRTMBADDR (VE_ENGINE_DEC_MPEG + 0xc8) #define VE_DEC_MPEG_ROT_LUMA (VE_ENGINE_DEC_MPEG + 0xcc) #define VE_DEC_MPEG_ROT_CHROMA (VE_ENGINE_DEC_MPEG + 0xd0) +#define VE_DEC_MPEG_SD_ROT_DBLK_CTL (VE_ENGINE_DEC_MPEG + 0xd4) +#define VE_DEC_MPEG_JPEG_MCU_START (VE_ENGINE_DEC_MPEG + 0xd8) +#define VE_DEC_MPEG_JPEG_MCU_END (VE_ENGINE_DEC_MPEG + 0xdc) +#define VE_DEC_MPEG_SRAM_RW_OFFSET (VE_ENGINE_DEC_MPEG + 0xe0) +#define VE_DEC_MPEG_SRAM_RW_DATA (VE_ENGINE_DEC_MPEG + 0xe4) #define VE_DEC_H265_DEC_NAL_HDR (VE_ENGINE_DEC_H265 + 0x00) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c index b00feaf4072c..7205c2315bc5 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c @@ -55,6 +55,11 @@ static struct cedrus_format cedrus_formats[] = { .directions = CEDRUS_DECODE_SRC, .capabilities = CEDRUS_CAPABILITY_VP8_DEC, }, + { + .pixelformat = V4L2_PIX_FMT_JPEG, + .directions = CEDRUS_DECODE_SRC, + .capabilities = CEDRUS_CAPABILITY_JPEG_DEC, + }, { .pixelformat = V4L2_PIX_FMT_NV12, .directions = CEDRUS_DECODE_DST, @@ -118,6 +123,7 @@ void cedrus_prepare_format(struct v4l2_pix_format *pix_fmt) case V4L2_PIX_FMT_H264_SLICE: case V4L2_PIX_FMT_HEVC_SLICE: case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_JPEG: /* Zero bytes per line for encoded source. */ bytesperline = 0; /* Choose some minimum size since this can't be 0 */ @@ -350,6 +356,9 @@ static int cedrus_s_fmt_vid_out_p(struct cedrus_ctx *ctx, case V4L2_PIX_FMT_VP8_FRAME: ctx->current_codec = &cedrus_dec_ops_vp8; break; + case V4L2_PIX_FMT_JPEG: + ctx->current_codec = &cedrus_dec_ops_jpeg; + break; } /* Propagate format information to capture. */ -- 2.45.2 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 2/4] media: cedrus: Add JPEG decoder 2024-07-31 16:44 ` [PATCH 2/4] media: cedrus: Add JPEG decoder Emmanuel Gil Peyrot @ 2025-02-13 17:12 ` Sebastian Fricke 2025-02-20 17:32 ` Jernej Škrabec 0 siblings, 1 reply; 9+ messages in thread From: Sebastian Fricke @ 2025-02-13 17:12 UTC (permalink / raw) To: Emmanuel Gil Peyrot Cc: linux-sunxi, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Jernej Skrabec, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel Hey Emmanuel, first of all sorry for taking that long to reply to your patches ... I notice that there has been no movement on this so I wonder how we should proceed with this patch, our CI shows at least that you have some style issues on the patches (https://gitlab.freedesktop.org/linux-media/users/sebastianfricke/-/jobs/71044070) and I would like to have a reviewer first before I can think about merging this. For a start can you rebase this patch series and apply the required fixes to make our CI happy and then we have to look for a reviewer as this is a new driver and I don't want to solely rely on my opinion as I am not well versed in the Cedrus driver. Regards, Sebastian ---- On Wed, 31 Jul 2024 18:44:12 +0200 Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> wrote --- > Basically all Cedrus variants support JPEG decoding. Add code for it. > > Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> > Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> > --- > drivers/staging/media/sunxi/cedrus/Kconfig | 1 + > drivers/staging/media/sunxi/cedrus/Makefile | 2 +- > drivers/staging/media/sunxi/cedrus/cedrus.h | 5 + > .../staging/media/sunxi/cedrus/cedrus_hw.c | 2 + > .../staging/media/sunxi/cedrus/cedrus_jpeg.c | 352 ++++++++++++++++++ > .../staging/media/sunxi/cedrus/cedrus_regs.h | 16 + > .../staging/media/sunxi/cedrus/cedrus_video.c | 9 + > 7 files changed, 386 insertions(+), 1 deletion(-) > create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c > > diff --git a/drivers/staging/media/sunxi/cedrus/Kconfig b/drivers/staging/media/sunxi/cedrus/Kconfig > index cb07a343c9c2..5683519aead0 100644 > --- a/drivers/staging/media/sunxi/cedrus/Kconfig > +++ b/drivers/staging/media/sunxi/cedrus/Kconfig > @@ -9,6 +9,7 @@ config VIDEO_SUNXI_CEDRUS > select SUNXI_SRAM > select VIDEOBUF2_DMA_CONTIG > select V4L2_MEM2MEM_DEV > + select V4L2_JPEG_HELPER > help > Support for the VPU found in Allwinner SoCs, also known as the Cedar > video engine. > diff --git a/drivers/staging/media/sunxi/cedrus/Makefile b/drivers/staging/media/sunxi/cedrus/Makefile > index a647b3690bf8..fa3e949e0788 100644 > --- a/drivers/staging/media/sunxi/cedrus/Makefile > +++ b/drivers/staging/media/sunxi/cedrus/Makefile > @@ -3,4 +3,4 @@ obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o > > sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o \ > cedrus_mpeg2.o cedrus_h264.o cedrus_h265.o \ > - cedrus_vp8.o > + cedrus_vp8.o cedrus_jpeg.o > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h > index 522c184e2afc..555f8d124d47 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus.h > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h > @@ -34,6 +34,7 @@ > #define CEDRUS_CAPABILITY_MPEG2_DEC BIT(3) > #define CEDRUS_CAPABILITY_VP8_DEC BIT(4) > #define CEDRUS_CAPABILITY_H265_10_DEC BIT(5) > +#define CEDRUS_CAPABILITY_JPEG_DEC BIT(6) > > enum cedrus_irq_status { > CEDRUS_IRQ_NONE, > @@ -152,6 +153,9 @@ struct cedrus_ctx { > u8 *entropy_probs_buf; > dma_addr_t entropy_probs_buf_dma; > } vp8; > + struct { > + unsigned int subsampling; > + } jpeg; > } codec; > }; > > @@ -201,6 +205,7 @@ extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2; > extern struct cedrus_dec_ops cedrus_dec_ops_h264; > extern struct cedrus_dec_ops cedrus_dec_ops_h265; > extern struct cedrus_dec_ops cedrus_dec_ops_vp8; > +extern struct cedrus_dec_ops cedrus_dec_ops_jpeg; > > static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val) > { > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > index 582b6cb796ea..e237f7d66f7e 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > @@ -43,7 +43,9 @@ int cedrus_engine_enable(struct cedrus_ctx *ctx) > reg |= VE_MODE_DDR_MODE_BW_128; > > switch (ctx->src_fmt.pixelformat) { > + /* MPEG2 and JPEG both use the same decoding mode bit. */ > case V4L2_PIX_FMT_MPEG2_SLICE: > + case V4L2_PIX_FMT_JPEG: > reg |= VE_MODE_DEC_MPEG; > break; > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c b/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c > new file mode 100644 > index 000000000000..1e8978ebf9dd > --- /dev/null > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c > @@ -0,0 +1,352 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* > + * Cedrus VPU driver > + * > + * Copyright (C) 2022 Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> > + */ > + > +#include <media/videobuf2-dma-contig.h> > +#include <media/v4l2-jpeg.h> > + > +#include "cedrus.h" > +#include "cedrus_hw.h" > +#include "cedrus_regs.h" > + > +static enum cedrus_irq_status cedrus_jpeg_irq_status(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + u32 reg; > + > + reg = cedrus_read(dev, VE_DEC_MPEG_STATUS); > + reg &= VE_DEC_MPEG_STATUS_CHECK_MASK; > + > + if (!reg) > + return CEDRUS_IRQ_NONE; > + > + if (reg & VE_DEC_MPEG_STATUS_CHECK_ERROR) > + return CEDRUS_IRQ_ERROR; > + > + return CEDRUS_IRQ_OK; > +} > + > +static void cedrus_jpeg_irq_clear(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_write(dev, VE_DEC_MPEG_STATUS, VE_DEC_MPEG_STATUS_CHECK_MASK); > +} > + > +static void cedrus_jpeg_irq_disable(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + u32 reg = cedrus_read(dev, VE_DEC_MPEG_CTRL); > + > + reg &= ~VE_DEC_MPEG_CTRL_IRQ_MASK; > + > + cedrus_write(dev, VE_DEC_MPEG_CTRL, reg); > +} > + > +static int cedrus_write_table_header(struct cedrus_dev *dev, > + struct v4l2_jpeg_reference *table) > +{ > + u16 start_codes[16], code; > + u8 offsets[16], *ptr; > + unsigned int count; > + u32 *ptr32; > + int i; > + > + ptr = table->start; > + if (!ptr) > + return -EINVAL; > + > + count = 0; > + code = 0; > + for (i = 0; i < 16; i++) { > + offsets[i] = count; > + start_codes[i] = code; > + count += ptr[i]; > + code += ptr[i]; > + code *= 2; > + } > + > + for (i = 15; i >= 0 && !ptr[i]; i--) > + start_codes[i] = 0xffff; > + > + ptr32 = (u32*)start_codes; > + for (i = 0; i < 8; i++) > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr32[i]); > + > + ptr32 = (u32*)offsets; > + for (i = 0; i < 4; i++) > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr32[i]); > + > + for (i = 0; i < 4; i++) > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); > + > + return 0; > +} > + > +static int cedrus_jpeg_write_dh_tables(struct cedrus_dev *dev, > + struct v4l2_jpeg_header *hdr) > +{ > + struct v4l2_jpeg_reference *tables[4], *table; > + struct v4l2_jpeg_scan_component_spec *comp; > + unsigned int i, j, ret; > + size_t length; > + u32 *ptr, val; > + > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_OFFSET, 0); > + > + j = 0; > + for (i = 0; i < 2; i++) { > + comp = &hdr->scan->component[i]; > + > + tables[j++] = &hdr->huffman_tables[comp->dc_entropy_coding_table_selector]; > + tables[j++] = &hdr->huffman_tables[comp->ac_entropy_coding_table_selector + 2]; > + } > + > + for (i = 0; i < 4; i++) { > + ret = cedrus_write_table_header(dev, tables[i]); > + if (ret) > + return ret; > + } > + > + for (i = 0; i < 192; i++) > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); > + > + for (i = 0; i < 4; i++) { > + table = tables[i]; > + ptr = (u32*)&table->start[16]; > + length = table->length - 16; > + > + for (j = 0; j < length / 4; j++) > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr[j]); > + > + if (length & 3) { > + val = 0; > + for (j = 0; j < (length & 3); j++) > + val = (val << 8) | table->start[15 + length - j]; > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, val); > + } > + > + for (j = 0; j < 64 - DIV_ROUND_UP(length, 4); j++) > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); > + } > + > + return 0; > +} > + > +static int cedrus_write_quantization_matrix(struct cedrus_dev *dev, u32 flags, > + struct v4l2_jpeg_reference *table) > +{ > + const u8 *matrix; > + u32 reg, val; > + int i; > + > + matrix = table->start; > + if (!matrix) > + return -EINVAL; > + > + for (i = 0; i < 64; i++) { > + /* determine if values are 8 or 16 bits */ > + val = *matrix++; > + if (table->length > 64) > + val = (val << 8) | *matrix++; > + > + reg = VE_DEC_MPEG_IQMINPUT_WEIGHT(i, val); > + reg |= flags; > + > + cedrus_write(dev, VE_DEC_MPEG_IQMINPUT, reg); > + } > + > + return 0; > +} > + > +static int cedrus_jpeg_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) > +{ > + struct cedrus_dev *dev = ctx->dev; > + dma_addr_t src_buf_addr, dst_luma_addr, dst_chroma_addr; > + struct v4l2_jpeg_scan_header scan_header; > + struct v4l2_jpeg_reference quantization_tables[4] = { }; > + struct v4l2_jpeg_reference huffman_tables[4] = { }; > + struct v4l2_jpeg_header header = { > + .scan = &scan_header, > + .quantization_tables = quantization_tables, > + .huffman_tables = huffman_tables, > + }; > + struct vb2_buffer *src_buf = &run->src->vb2_buf; > + struct v4l2_jpeg_frame_component_spec *components; > + u32 reg, subsampling; > + unsigned long size; > + int ret, index; > + u8 hmax, vmax; > + u16 width, height; > + > + size = vb2_get_plane_payload(src_buf, 0); > + ret = v4l2_jpeg_parse_header(vb2_plane_vaddr(src_buf, 0), size, &header); > + if (ret < 0) { > + v4l2_err(&dev->v4l2_dev, > + "failed to parse JPEG header: %d\n", ret); > + return -EINVAL; > + } > + > + width = header.frame.width; > + height = header.frame.height; > + if (width > 2048 || height > 2048) { > + v4l2_err(&dev->v4l2_dev, > + "unsupported JPEG of resolution %ux%u (max 2048x2048)\n", > + width, height); > + return -EINVAL; > + } > + > + if (header.frame.precision != 8) { > + v4l2_err(&dev->v4l2_dev, > + "unsupported JPEG with %u bits of precision (8 required)\n", > + header.frame.precision); > + return -EINVAL; > + } > + > + if (header.frame.num_components != 3) { > + v4l2_err(&dev->v4l2_dev, > + "unsupported JPEG with %u components (3 required)\n", > + header.frame.num_components); > + return -EINVAL; > + } > + > + components = header.frame.component; > + index = components[0].horizontal_sampling_factor << 20 | > + components[0].vertical_sampling_factor << 16 | > + components[1].horizontal_sampling_factor << 12 | > + components[1].vertical_sampling_factor << 8 | > + components[2].horizontal_sampling_factor << 4 | > + components[2].vertical_sampling_factor; > + > + switch (index) { > + case 0x221111: > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_420; > + break; > + case 0x211111: > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422; > + break; > + case 0x111111: > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444; > + break; > + case 0x121111: > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T; > + break; > + default: > + v4l2_err(&dev->v4l2_dev, "unsupported subsampling %d\n", index); > + return -EINVAL; > + } > + > + ctx->codec.jpeg.subsampling = subsampling; > + > + /* Activate MPEG engine and select JPEG subengine. */ > + cedrus_engine_enable(ctx); > + > + reg = VE_DEC_MPEG_TRIGGER_JPEG | subsampling; > + cedrus_write(dev, VE_DEC_MPEG_TRIGGER, reg); > + > + /* Set restart interval. */ > + cedrus_write(dev, VE_DEC_MPEG_JPEG_RES_INT, header.restart_interval); > + > + /* Set resolution in blocks. */ > + hmax = components[0].horizontal_sampling_factor; > + vmax = components[0].vertical_sampling_factor; > + for (index = 1; index < 3; index++) { > + if (hmax < components[index].horizontal_sampling_factor) > + hmax = components[index].horizontal_sampling_factor; > + if (vmax < components[index].vertical_sampling_factor) > + vmax = components[index].vertical_sampling_factor; > + } > + > + reg = VE_DEC_MPEG_JPEG_SIZE_WIDTH(DIV_ROUND_UP(width, 8 * hmax)); > + reg |= VE_DEC_MPEG_JPEG_SIZE_HEIGHT(DIV_ROUND_UP(height, 8 * vmax)); > + cedrus_write(dev, VE_DEC_MPEG_JPEG_SIZE, reg); > + > + /* Set intra quantisation matrix. */ > + index = components[0].quantization_table_selector; > + ret = cedrus_write_quantization_matrix(dev, > + VE_DEC_MPEG_IQMINPUT_FLAG_INTRA, > + &quantization_tables[index]); > + if (ret) > + return ret; > + > + /* Set non-intra quantisation matrix. */ > + index = components[1].quantization_table_selector; > + ret = cedrus_write_quantization_matrix(dev, > + VE_DEC_MPEG_IQMINPUT_FLAG_NON_INTRA, > + &quantization_tables[index]); > + if (ret) > + return ret; > + > + /* Set Diffie-Huffman tables. */ > + ret = cedrus_jpeg_write_dh_tables(dev, &header); > + if (ret) > + return ret; > + > + /* Destination luma and chroma buffers. */ > + > + dst_luma_addr = cedrus_dst_buf_addr(ctx, &run->dst->vb2_buf, 0); > + dst_chroma_addr = cedrus_dst_buf_addr(ctx, &run->dst->vb2_buf, 1); > + > + /* JPEG outputs to rotation/scale down output buffers */ > + cedrus_write(dev, VE_DEC_MPEG_ROT_LUMA, dst_luma_addr); > + cedrus_write(dev, VE_DEC_MPEG_ROT_CHROMA, dst_chroma_addr); > + > + /* Disable rotation and scaling. */ > + cedrus_write(dev, VE_DEC_MPEG_SD_ROT_DBLK_CTL, 0); > + > + /* Source offset and length in bits. */ > + > + cedrus_write(dev, VE_DEC_MPEG_VLD_OFFSET, 8 * header.ecs_offset); > + > + reg = size * 8; > + cedrus_write(dev, VE_DEC_MPEG_VLD_LEN, reg); > + > + /* Source beginning and end addresses. */ > + > + src_buf_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0); > + > + reg = VE_DEC_MPEG_VLD_ADDR_BASE(src_buf_addr); > + reg |= VE_DEC_MPEG_VLD_ADDR_VALID_PIC_DATA; > + reg |= VE_DEC_MPEG_VLD_ADDR_LAST_PIC_DATA; > + reg |= VE_DEC_MPEG_VLD_ADDR_FIRST_PIC_DATA; > + > + cedrus_write(dev, VE_DEC_MPEG_VLD_ADDR, reg); > + > + reg = src_buf_addr + size; > + cedrus_write(dev, VE_DEC_MPEG_VLD_END_ADDR, reg); > + > + /* Enable appropriate interrupts and components. */ > + > + reg = VE_DEC_MPEG_CTRL_IRQ_MASK; > + if (subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422 || > + subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T || > + subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444) > + reg |= VE_DEC_MPEG_CTRL_JPEG_FORCE_420; > + > + cedrus_write(dev, VE_DEC_MPEG_CTRL, reg); > + > + return 0; > +} > + > +static void cedrus_jpeg_trigger(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + u32 reg; > + > + /* Trigger JPEG engine. */ > + reg = VE_DEC_MPEG_TRIGGER_HW_JPEG_VLD | VE_DEC_MPEG_TRIGGER_JPEG; > + reg |= ctx->codec.jpeg.subsampling; > + > + cedrus_write(dev, VE_DEC_MPEG_TRIGGER, reg); > +} > + > +struct cedrus_dec_ops cedrus_dec_ops_jpeg = { > + .irq_clear = cedrus_jpeg_irq_clear, > + .irq_disable = cedrus_jpeg_irq_disable, > + .irq_status = cedrus_jpeg_irq_status, > + .setup = cedrus_jpeg_setup, > + .trigger = cedrus_jpeg_trigger, > +}; > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > index 3acc05e0fb54..cb8b4bb4f44e 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > @@ -144,6 +144,7 @@ > > #define VE_DEC_MPEG_CTRL_MC_CACHE_EN BIT(31) > #define VE_DEC_MPEG_CTRL_SW_VLD BIT(27) > +#define VE_DEC_MPEG_CTRL_JPEG_FORCE_420 BIT(18) > #define VE_DEC_MPEG_CTRL_SW_IQ_IS BIT(17) > #define VE_DEC_MPEG_CTRL_QP_AC_DC_OUT_EN BIT(14) > #define VE_DEC_MPEG_CTRL_ROTATE_SCALE_OUT_EN BIT(8) > @@ -165,6 +166,7 @@ > #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422 (0x02 << 27) > #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444 (0x03 << 27) > #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T (0x04 << 27) > +#define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_400 (0x05 << 27) > > #define VE_DEC_MPEG_TRIGGER_MPEG1 (0x01 << 24) > #define VE_DEC_MPEG_TRIGGER_MPEG2 (0x02 << 24) > @@ -255,10 +257,24 @@ > #define VE_DEC_MPEG_IQMINPUT_WEIGHT(i, v) \ > (SHIFT_AND_MASK_BITS(i, 13, 8) | SHIFT_AND_MASK_BITS(v, 7, 0)) > > +#define VE_DEC_MPEG_JPEG_SIZE (VE_ENGINE_DEC_MPEG + 0xb8) > + > +#define VE_DEC_MPEG_JPEG_SIZE_WIDTH(w) \ > + SHIFT_AND_MASK_BITS((w) - 1, 10, 0) > +#define VE_DEC_MPEG_JPEG_SIZE_HEIGHT(h) \ > + SHIFT_AND_MASK_BITS((h) - 1, 26, 16) > + > +#define VE_DEC_MPEG_JPEG_MCU (VE_ENGINE_DEC_MPEG + 0xbc) > +#define VE_DEC_MPEG_JPEG_RES_INT (VE_ENGINE_DEC_MPEG + 0xc0) > #define VE_DEC_MPEG_ERROR (VE_ENGINE_DEC_MPEG + 0xc4) > #define VE_DEC_MPEG_CRTMBADDR (VE_ENGINE_DEC_MPEG + 0xc8) > #define VE_DEC_MPEG_ROT_LUMA (VE_ENGINE_DEC_MPEG + 0xcc) > #define VE_DEC_MPEG_ROT_CHROMA (VE_ENGINE_DEC_MPEG + 0xd0) > +#define VE_DEC_MPEG_SD_ROT_DBLK_CTL (VE_ENGINE_DEC_MPEG + 0xd4) > +#define VE_DEC_MPEG_JPEG_MCU_START (VE_ENGINE_DEC_MPEG + 0xd8) > +#define VE_DEC_MPEG_JPEG_MCU_END (VE_ENGINE_DEC_MPEG + 0xdc) > +#define VE_DEC_MPEG_SRAM_RW_OFFSET (VE_ENGINE_DEC_MPEG + 0xe0) > +#define VE_DEC_MPEG_SRAM_RW_DATA (VE_ENGINE_DEC_MPEG + 0xe4) > > #define VE_DEC_H265_DEC_NAL_HDR (VE_ENGINE_DEC_H265 + 0x00) > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c > index b00feaf4072c..7205c2315bc5 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c > @@ -55,6 +55,11 @@ static struct cedrus_format cedrus_formats[] = { > .directions = CEDRUS_DECODE_SRC, > .capabilities = CEDRUS_CAPABILITY_VP8_DEC, > }, > + { > + .pixelformat = V4L2_PIX_FMT_JPEG, > + .directions = CEDRUS_DECODE_SRC, > + .capabilities = CEDRUS_CAPABILITY_JPEG_DEC, > + }, > { > .pixelformat = V4L2_PIX_FMT_NV12, > .directions = CEDRUS_DECODE_DST, > @@ -118,6 +123,7 @@ void cedrus_prepare_format(struct v4l2_pix_format *pix_fmt) > case V4L2_PIX_FMT_H264_SLICE: > case V4L2_PIX_FMT_HEVC_SLICE: > case V4L2_PIX_FMT_VP8_FRAME: > + case V4L2_PIX_FMT_JPEG: > /* Zero bytes per line for encoded source. */ > bytesperline = 0; > /* Choose some minimum size since this can't be 0 */ > @@ -350,6 +356,9 @@ static int cedrus_s_fmt_vid_out_p(struct cedrus_ctx *ctx, > case V4L2_PIX_FMT_VP8_FRAME: > ctx->current_codec = &cedrus_dec_ops_vp8; > break; > + case V4L2_PIX_FMT_JPEG: > + ctx->current_codec = &cedrus_dec_ops_jpeg; > + break; > } > > /* Propagate format information to capture. */ > -- > 2.45.2 > > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 2/4] media: cedrus: Add JPEG decoder 2025-02-13 17:12 ` Sebastian Fricke @ 2025-02-20 17:32 ` Jernej Škrabec 0 siblings, 0 replies; 9+ messages in thread From: Jernej Škrabec @ 2025-02-20 17:32 UTC (permalink / raw) To: Emmanuel Gil Peyrot, Sebastian Fricke Cc: linux-sunxi, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel Hi Sebastian, Dne četrtek, 13. februar 2025 ob 18:12:56 Srednjeevropski standardni čas je Sebastian Fricke napisal(a): > Hey Emmanuel, > > first of all sorry for taking that long to reply to your patches ... > > I notice that there has been no movement on this so I wonder how we should proceed with this patch, our CI shows at least that you have some style issues on the patches (https://gitlab.freedesktop.org/linux-media/users/sebastianfricke/-/jobs/71044070) and I would like to have a reviewer first before I can think about merging this. > > For a start can you rebase this patch series and apply the required fixes to make our CI happy and then we have to look for a reviewer as this is a new driver and I don't want to solely rely on my opinion as I am not well versed in the Cedrus driver. I would usually review such contribution, but since I'm co-author, there is a conflict of interests. You can ask Paul Kocialkowski. Best regards, Jernej > > Regards, > Sebastian > > ---- On Wed, 31 Jul 2024 18:44:12 +0200 Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> wrote --- > > Basically all Cedrus variants support JPEG decoding. Add code for it. > > > > Signed-off-by: Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> > > Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> > > --- > > drivers/staging/media/sunxi/cedrus/Kconfig | 1 + > > drivers/staging/media/sunxi/cedrus/Makefile | 2 +- > > drivers/staging/media/sunxi/cedrus/cedrus.h | 5 + > > .../staging/media/sunxi/cedrus/cedrus_hw.c | 2 + > > .../staging/media/sunxi/cedrus/cedrus_jpeg.c | 352 ++++++++++++++++++ > > .../staging/media/sunxi/cedrus/cedrus_regs.h | 16 + > > .../staging/media/sunxi/cedrus/cedrus_video.c | 9 + > > 7 files changed, 386 insertions(+), 1 deletion(-) > > create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c > > > > diff --git a/drivers/staging/media/sunxi/cedrus/Kconfig b/drivers/staging/media/sunxi/cedrus/Kconfig > > index cb07a343c9c2..5683519aead0 100644 > > --- a/drivers/staging/media/sunxi/cedrus/Kconfig > > +++ b/drivers/staging/media/sunxi/cedrus/Kconfig > > @@ -9,6 +9,7 @@ config VIDEO_SUNXI_CEDRUS > > select SUNXI_SRAM > > select VIDEOBUF2_DMA_CONTIG > > select V4L2_MEM2MEM_DEV > > + select V4L2_JPEG_HELPER > > help > > Support for the VPU found in Allwinner SoCs, also known as the Cedar > > video engine. > > diff --git a/drivers/staging/media/sunxi/cedrus/Makefile b/drivers/staging/media/sunxi/cedrus/Makefile > > index a647b3690bf8..fa3e949e0788 100644 > > --- a/drivers/staging/media/sunxi/cedrus/Makefile > > +++ b/drivers/staging/media/sunxi/cedrus/Makefile > > @@ -3,4 +3,4 @@ obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o > > > > sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o \ > > cedrus_mpeg2.o cedrus_h264.o cedrus_h265.o \ > > - cedrus_vp8.o > > + cedrus_vp8.o cedrus_jpeg.o > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h > > index 522c184e2afc..555f8d124d47 100644 > > --- a/drivers/staging/media/sunxi/cedrus/cedrus.h > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h > > @@ -34,6 +34,7 @@ > > #define CEDRUS_CAPABILITY_MPEG2_DEC BIT(3) > > #define CEDRUS_CAPABILITY_VP8_DEC BIT(4) > > #define CEDRUS_CAPABILITY_H265_10_DEC BIT(5) > > +#define CEDRUS_CAPABILITY_JPEG_DEC BIT(6) > > > > enum cedrus_irq_status { > > CEDRUS_IRQ_NONE, > > @@ -152,6 +153,9 @@ struct cedrus_ctx { > > u8 *entropy_probs_buf; > > dma_addr_t entropy_probs_buf_dma; > > } vp8; > > + struct { > > + unsigned int subsampling; > > + } jpeg; > > } codec; > > }; > > > > @@ -201,6 +205,7 @@ extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2; > > extern struct cedrus_dec_ops cedrus_dec_ops_h264; > > extern struct cedrus_dec_ops cedrus_dec_ops_h265; > > extern struct cedrus_dec_ops cedrus_dec_ops_vp8; > > +extern struct cedrus_dec_ops cedrus_dec_ops_jpeg; > > > > static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val) > > { > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > > index 582b6cb796ea..e237f7d66f7e 100644 > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > > @@ -43,7 +43,9 @@ int cedrus_engine_enable(struct cedrus_ctx *ctx) > > reg |= VE_MODE_DDR_MODE_BW_128; > > > > switch (ctx->src_fmt.pixelformat) { > > + /* MPEG2 and JPEG both use the same decoding mode bit. */ > > case V4L2_PIX_FMT_MPEG2_SLICE: > > + case V4L2_PIX_FMT_JPEG: > > reg |= VE_MODE_DEC_MPEG; > > break; > > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c b/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c > > new file mode 100644 > > index 000000000000..1e8978ebf9dd > > --- /dev/null > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_jpeg.c > > @@ -0,0 +1,352 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > +/* > > + * Cedrus VPU driver > > + * > > + * Copyright (C) 2022 Emmanuel Gil Peyrot <linkmauve@linkmauve.fr> > > + */ > > + > > +#include <media/videobuf2-dma-contig.h> > > +#include <media/v4l2-jpeg.h> > > + > > +#include "cedrus.h" > > +#include "cedrus_hw.h" > > +#include "cedrus_regs.h" > > + > > +static enum cedrus_irq_status cedrus_jpeg_irq_status(struct cedrus_ctx *ctx) > > +{ > > + struct cedrus_dev *dev = ctx->dev; > > + u32 reg; > > + > > + reg = cedrus_read(dev, VE_DEC_MPEG_STATUS); > > + reg &= VE_DEC_MPEG_STATUS_CHECK_MASK; > > + > > + if (!reg) > > + return CEDRUS_IRQ_NONE; > > + > > + if (reg & VE_DEC_MPEG_STATUS_CHECK_ERROR) > > + return CEDRUS_IRQ_ERROR; > > + > > + return CEDRUS_IRQ_OK; > > +} > > + > > +static void cedrus_jpeg_irq_clear(struct cedrus_ctx *ctx) > > +{ > > + struct cedrus_dev *dev = ctx->dev; > > + > > + cedrus_write(dev, VE_DEC_MPEG_STATUS, VE_DEC_MPEG_STATUS_CHECK_MASK); > > +} > > + > > +static void cedrus_jpeg_irq_disable(struct cedrus_ctx *ctx) > > +{ > > + struct cedrus_dev *dev = ctx->dev; > > + u32 reg = cedrus_read(dev, VE_DEC_MPEG_CTRL); > > + > > + reg &= ~VE_DEC_MPEG_CTRL_IRQ_MASK; > > + > > + cedrus_write(dev, VE_DEC_MPEG_CTRL, reg); > > +} > > + > > +static int cedrus_write_table_header(struct cedrus_dev *dev, > > + struct v4l2_jpeg_reference *table) > > +{ > > + u16 start_codes[16], code; > > + u8 offsets[16], *ptr; > > + unsigned int count; > > + u32 *ptr32; > > + int i; > > + > > + ptr = table->start; > > + if (!ptr) > > + return -EINVAL; > > + > > + count = 0; > > + code = 0; > > + for (i = 0; i < 16; i++) { > > + offsets[i] = count; > > + start_codes[i] = code; > > + count += ptr[i]; > > + code += ptr[i]; > > + code *= 2; > > + } > > + > > + for (i = 15; i >= 0 && !ptr[i]; i--) > > + start_codes[i] = 0xffff; > > + > > + ptr32 = (u32*)start_codes; > > + for (i = 0; i < 8; i++) > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr32[i]); > > + > > + ptr32 = (u32*)offsets; > > + for (i = 0; i < 4; i++) > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr32[i]); > > + > > + for (i = 0; i < 4; i++) > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); > > + > > + return 0; > > +} > > + > > +static int cedrus_jpeg_write_dh_tables(struct cedrus_dev *dev, > > + struct v4l2_jpeg_header *hdr) > > +{ > > + struct v4l2_jpeg_reference *tables[4], *table; > > + struct v4l2_jpeg_scan_component_spec *comp; > > + unsigned int i, j, ret; > > + size_t length; > > + u32 *ptr, val; > > + > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_OFFSET, 0); > > + > > + j = 0; > > + for (i = 0; i < 2; i++) { > > + comp = &hdr->scan->component[i]; > > + > > + tables[j++] = &hdr->huffman_tables[comp->dc_entropy_coding_table_selector]; > > + tables[j++] = &hdr->huffman_tables[comp->ac_entropy_coding_table_selector + 2]; > > + } > > + > > + for (i = 0; i < 4; i++) { > > + ret = cedrus_write_table_header(dev, tables[i]); > > + if (ret) > > + return ret; > > + } > > + > > + for (i = 0; i < 192; i++) > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); > > + > > + for (i = 0; i < 4; i++) { > > + table = tables[i]; > > + ptr = (u32*)&table->start[16]; > > + length = table->length - 16; > > + > > + for (j = 0; j < length / 4; j++) > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, ptr[j]); > > + > > + if (length & 3) { > > + val = 0; > > + for (j = 0; j < (length & 3); j++) > > + val = (val << 8) | table->start[15 + length - j]; > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, val); > > + } > > + > > + for (j = 0; j < 64 - DIV_ROUND_UP(length, 4); j++) > > + cedrus_write(dev, VE_DEC_MPEG_SRAM_RW_DATA, 0); > > + } > > + > > + return 0; > > +} > > + > > +static int cedrus_write_quantization_matrix(struct cedrus_dev *dev, u32 flags, > > + struct v4l2_jpeg_reference *table) > > +{ > > + const u8 *matrix; > > + u32 reg, val; > > + int i; > > + > > + matrix = table->start; > > + if (!matrix) > > + return -EINVAL; > > + > > + for (i = 0; i < 64; i++) { > > + /* determine if values are 8 or 16 bits */ > > + val = *matrix++; > > + if (table->length > 64) > > + val = (val << 8) | *matrix++; > > + > > + reg = VE_DEC_MPEG_IQMINPUT_WEIGHT(i, val); > > + reg |= flags; > > + > > + cedrus_write(dev, VE_DEC_MPEG_IQMINPUT, reg); > > + } > > + > > + return 0; > > +} > > + > > +static int cedrus_jpeg_setup(struct cedrus_ctx *ctx, struct cedrus_run *run) > > +{ > > + struct cedrus_dev *dev = ctx->dev; > > + dma_addr_t src_buf_addr, dst_luma_addr, dst_chroma_addr; > > + struct v4l2_jpeg_scan_header scan_header; > > + struct v4l2_jpeg_reference quantization_tables[4] = { }; > > + struct v4l2_jpeg_reference huffman_tables[4] = { }; > > + struct v4l2_jpeg_header header = { > > + .scan = &scan_header, > > + .quantization_tables = quantization_tables, > > + .huffman_tables = huffman_tables, > > + }; > > + struct vb2_buffer *src_buf = &run->src->vb2_buf; > > + struct v4l2_jpeg_frame_component_spec *components; > > + u32 reg, subsampling; > > + unsigned long size; > > + int ret, index; > > + u8 hmax, vmax; > > + u16 width, height; > > + > > + size = vb2_get_plane_payload(src_buf, 0); > > + ret = v4l2_jpeg_parse_header(vb2_plane_vaddr(src_buf, 0), size, &header); > > + if (ret < 0) { > > + v4l2_err(&dev->v4l2_dev, > > + "failed to parse JPEG header: %d\n", ret); > > + return -EINVAL; > > + } > > + > > + width = header.frame.width; > > + height = header.frame.height; > > + if (width > 2048 || height > 2048) { > > + v4l2_err(&dev->v4l2_dev, > > + "unsupported JPEG of resolution %ux%u (max 2048x2048)\n", > > + width, height); > > + return -EINVAL; > > + } > > + > > + if (header.frame.precision != 8) { > > + v4l2_err(&dev->v4l2_dev, > > + "unsupported JPEG with %u bits of precision (8 required)\n", > > + header.frame.precision); > > + return -EINVAL; > > + } > > + > > + if (header.frame.num_components != 3) { > > + v4l2_err(&dev->v4l2_dev, > > + "unsupported JPEG with %u components (3 required)\n", > > + header.frame.num_components); > > + return -EINVAL; > > + } > > + > > + components = header.frame.component; > > + index = components[0].horizontal_sampling_factor << 20 | > > + components[0].vertical_sampling_factor << 16 | > > + components[1].horizontal_sampling_factor << 12 | > > + components[1].vertical_sampling_factor << 8 | > > + components[2].horizontal_sampling_factor << 4 | > > + components[2].vertical_sampling_factor; > > + > > + switch (index) { > > + case 0x221111: > > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_420; > > + break; > > + case 0x211111: > > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422; > > + break; > > + case 0x111111: > > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444; > > + break; > > + case 0x121111: > > + subsampling = VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T; > > + break; > > + default: > > + v4l2_err(&dev->v4l2_dev, "unsupported subsampling %d\n", index); > > + return -EINVAL; > > + } > > + > > + ctx->codec.jpeg.subsampling = subsampling; > > + > > + /* Activate MPEG engine and select JPEG subengine. */ > > + cedrus_engine_enable(ctx); > > + > > + reg = VE_DEC_MPEG_TRIGGER_JPEG | subsampling; > > + cedrus_write(dev, VE_DEC_MPEG_TRIGGER, reg); > > + > > + /* Set restart interval. */ > > + cedrus_write(dev, VE_DEC_MPEG_JPEG_RES_INT, header.restart_interval); > > + > > + /* Set resolution in blocks. */ > > + hmax = components[0].horizontal_sampling_factor; > > + vmax = components[0].vertical_sampling_factor; > > + for (index = 1; index < 3; index++) { > > + if (hmax < components[index].horizontal_sampling_factor) > > + hmax = components[index].horizontal_sampling_factor; > > + if (vmax < components[index].vertical_sampling_factor) > > + vmax = components[index].vertical_sampling_factor; > > + } > > + > > + reg = VE_DEC_MPEG_JPEG_SIZE_WIDTH(DIV_ROUND_UP(width, 8 * hmax)); > > + reg |= VE_DEC_MPEG_JPEG_SIZE_HEIGHT(DIV_ROUND_UP(height, 8 * vmax)); > > + cedrus_write(dev, VE_DEC_MPEG_JPEG_SIZE, reg); > > + > > + /* Set intra quantisation matrix. */ > > + index = components[0].quantization_table_selector; > > + ret = cedrus_write_quantization_matrix(dev, > > + VE_DEC_MPEG_IQMINPUT_FLAG_INTRA, > > + &quantization_tables[index]); > > + if (ret) > > + return ret; > > + > > + /* Set non-intra quantisation matrix. */ > > + index = components[1].quantization_table_selector; > > + ret = cedrus_write_quantization_matrix(dev, > > + VE_DEC_MPEG_IQMINPUT_FLAG_NON_INTRA, > > + &quantization_tables[index]); > > + if (ret) > > + return ret; > > + > > + /* Set Diffie-Huffman tables. */ > > + ret = cedrus_jpeg_write_dh_tables(dev, &header); > > + if (ret) > > + return ret; > > + > > + /* Destination luma and chroma buffers. */ > > + > > + dst_luma_addr = cedrus_dst_buf_addr(ctx, &run->dst->vb2_buf, 0); > > + dst_chroma_addr = cedrus_dst_buf_addr(ctx, &run->dst->vb2_buf, 1); > > + > > + /* JPEG outputs to rotation/scale down output buffers */ > > + cedrus_write(dev, VE_DEC_MPEG_ROT_LUMA, dst_luma_addr); > > + cedrus_write(dev, VE_DEC_MPEG_ROT_CHROMA, dst_chroma_addr); > > + > > + /* Disable rotation and scaling. */ > > + cedrus_write(dev, VE_DEC_MPEG_SD_ROT_DBLK_CTL, 0); > > + > > + /* Source offset and length in bits. */ > > + > > + cedrus_write(dev, VE_DEC_MPEG_VLD_OFFSET, 8 * header.ecs_offset); > > + > > + reg = size * 8; > > + cedrus_write(dev, VE_DEC_MPEG_VLD_LEN, reg); > > + > > + /* Source beginning and end addresses. */ > > + > > + src_buf_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0); > > + > > + reg = VE_DEC_MPEG_VLD_ADDR_BASE(src_buf_addr); > > + reg |= VE_DEC_MPEG_VLD_ADDR_VALID_PIC_DATA; > > + reg |= VE_DEC_MPEG_VLD_ADDR_LAST_PIC_DATA; > > + reg |= VE_DEC_MPEG_VLD_ADDR_FIRST_PIC_DATA; > > + > > + cedrus_write(dev, VE_DEC_MPEG_VLD_ADDR, reg); > > + > > + reg = src_buf_addr + size; > > + cedrus_write(dev, VE_DEC_MPEG_VLD_END_ADDR, reg); > > + > > + /* Enable appropriate interrupts and components. */ > > + > > + reg = VE_DEC_MPEG_CTRL_IRQ_MASK; > > + if (subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422 || > > + subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T || > > + subsampling == VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444) > > + reg |= VE_DEC_MPEG_CTRL_JPEG_FORCE_420; > > + > > + cedrus_write(dev, VE_DEC_MPEG_CTRL, reg); > > + > > + return 0; > > +} > > + > > +static void cedrus_jpeg_trigger(struct cedrus_ctx *ctx) > > +{ > > + struct cedrus_dev *dev = ctx->dev; > > + u32 reg; > > + > > + /* Trigger JPEG engine. */ > > + reg = VE_DEC_MPEG_TRIGGER_HW_JPEG_VLD | VE_DEC_MPEG_TRIGGER_JPEG; > > + reg |= ctx->codec.jpeg.subsampling; > > + > > + cedrus_write(dev, VE_DEC_MPEG_TRIGGER, reg); > > +} > > + > > +struct cedrus_dec_ops cedrus_dec_ops_jpeg = { > > + .irq_clear = cedrus_jpeg_irq_clear, > > + .irq_disable = cedrus_jpeg_irq_disable, > > + .irq_status = cedrus_jpeg_irq_status, > > + .setup = cedrus_jpeg_setup, > > + .trigger = cedrus_jpeg_trigger, > > +}; > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > > index 3acc05e0fb54..cb8b4bb4f44e 100644 > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > > @@ -144,6 +144,7 @@ > > > > #define VE_DEC_MPEG_CTRL_MC_CACHE_EN BIT(31) > > #define VE_DEC_MPEG_CTRL_SW_VLD BIT(27) > > +#define VE_DEC_MPEG_CTRL_JPEG_FORCE_420 BIT(18) > > #define VE_DEC_MPEG_CTRL_SW_IQ_IS BIT(17) > > #define VE_DEC_MPEG_CTRL_QP_AC_DC_OUT_EN BIT(14) > > #define VE_DEC_MPEG_CTRL_ROTATE_SCALE_OUT_EN BIT(8) > > @@ -165,6 +166,7 @@ > > #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422 (0x02 << 27) > > #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_444 (0x03 << 27) > > #define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_422T (0x04 << 27) > > +#define VE_DEC_MPEG_TRIGGER_CHROMA_FMT_400 (0x05 << 27) > > > > #define VE_DEC_MPEG_TRIGGER_MPEG1 (0x01 << 24) > > #define VE_DEC_MPEG_TRIGGER_MPEG2 (0x02 << 24) > > @@ -255,10 +257,24 @@ > > #define VE_DEC_MPEG_IQMINPUT_WEIGHT(i, v) \ > > (SHIFT_AND_MASK_BITS(i, 13, 8) | SHIFT_AND_MASK_BITS(v, 7, 0)) > > > > +#define VE_DEC_MPEG_JPEG_SIZE (VE_ENGINE_DEC_MPEG + 0xb8) > > + > > +#define VE_DEC_MPEG_JPEG_SIZE_WIDTH(w) \ > > + SHIFT_AND_MASK_BITS((w) - 1, 10, 0) > > +#define VE_DEC_MPEG_JPEG_SIZE_HEIGHT(h) \ > > + SHIFT_AND_MASK_BITS((h) - 1, 26, 16) > > + > > +#define VE_DEC_MPEG_JPEG_MCU (VE_ENGINE_DEC_MPEG + 0xbc) > > +#define VE_DEC_MPEG_JPEG_RES_INT (VE_ENGINE_DEC_MPEG + 0xc0) > > #define VE_DEC_MPEG_ERROR (VE_ENGINE_DEC_MPEG + 0xc4) > > #define VE_DEC_MPEG_CRTMBADDR (VE_ENGINE_DEC_MPEG + 0xc8) > > #define VE_DEC_MPEG_ROT_LUMA (VE_ENGINE_DEC_MPEG + 0xcc) > > #define VE_DEC_MPEG_ROT_CHROMA (VE_ENGINE_DEC_MPEG + 0xd0) > > +#define VE_DEC_MPEG_SD_ROT_DBLK_CTL (VE_ENGINE_DEC_MPEG + 0xd4) > > +#define VE_DEC_MPEG_JPEG_MCU_START (VE_ENGINE_DEC_MPEG + 0xd8) > > +#define VE_DEC_MPEG_JPEG_MCU_END (VE_ENGINE_DEC_MPEG + 0xdc) > > +#define VE_DEC_MPEG_SRAM_RW_OFFSET (VE_ENGINE_DEC_MPEG + 0xe0) > > +#define VE_DEC_MPEG_SRAM_RW_DATA (VE_ENGINE_DEC_MPEG + 0xe4) > > > > #define VE_DEC_H265_DEC_NAL_HDR (VE_ENGINE_DEC_H265 + 0x00) > > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c > > index b00feaf4072c..7205c2315bc5 100644 > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c > > @@ -55,6 +55,11 @@ static struct cedrus_format cedrus_formats[] = { > > .directions = CEDRUS_DECODE_SRC, > > .capabilities = CEDRUS_CAPABILITY_VP8_DEC, > > }, > > + { > > + .pixelformat = V4L2_PIX_FMT_JPEG, > > + .directions = CEDRUS_DECODE_SRC, > > + .capabilities = CEDRUS_CAPABILITY_JPEG_DEC, > > + }, > > { > > .pixelformat = V4L2_PIX_FMT_NV12, > > .directions = CEDRUS_DECODE_DST, > > @@ -118,6 +123,7 @@ void cedrus_prepare_format(struct v4l2_pix_format *pix_fmt) > > case V4L2_PIX_FMT_H264_SLICE: > > case V4L2_PIX_FMT_HEVC_SLICE: > > case V4L2_PIX_FMT_VP8_FRAME: > > + case V4L2_PIX_FMT_JPEG: > > /* Zero bytes per line for encoded source. */ > > bytesperline = 0; > > /* Choose some minimum size since this can't be 0 */ > > @@ -350,6 +356,9 @@ static int cedrus_s_fmt_vid_out_p(struct cedrus_ctx *ctx, > > case V4L2_PIX_FMT_VP8_FRAME: > > ctx->current_codec = &cedrus_dec_ops_vp8; > > break; > > + case V4L2_PIX_FMT_JPEG: > > + ctx->current_codec = &cedrus_dec_ops_jpeg; > > + break; > > } > > > > /* Propagate format information to capture. */ > > ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 3/4] media: cedrus: Enable JPEG decoding in various variants 2024-07-31 16:44 [PATCH 0/4] media: cedrus: Add a JPEG decoder Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 1/4] media: cedrus: Setup secondary output formats Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 2/4] media: cedrus: Add JPEG decoder Emmanuel Gil Peyrot @ 2024-07-31 16:44 ` Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 4/4] media: cedrus: Don't require requests for all codecs Emmanuel Gil Peyrot 3 siblings, 0 replies; 9+ messages in thread From: Emmanuel Gil Peyrot @ 2024-07-31 16:44 UTC (permalink / raw) To: linux-sunxi Cc: Jernej Skrabec, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel From: Jernej Skrabec <jernej.skrabec@gmail.com> As far as manuals go, only A10 and A13 don't have JPEG support mentioned, all others do. To be safe, enable it on all variants except A13, since it was tested to be functional on A10. Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> --- drivers/staging/media/sunxi/cedrus/cedrus.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index f52df6836045..f586bd95c163 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -563,6 +563,7 @@ static void cedrus_remove(struct platform_device *pdev) static const struct cedrus_variant sun4i_a10_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_VP8_DEC, .mod_rate = 320000000, @@ -577,6 +578,7 @@ static const struct cedrus_variant sun5i_a13_cedrus_variant = { static const struct cedrus_variant sun7i_a20_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_VP8_DEC, .mod_rate = 320000000, @@ -585,6 +587,7 @@ static const struct cedrus_variant sun7i_a20_cedrus_variant = { static const struct cedrus_variant sun8i_a33_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_VP8_DEC, .mod_rate = 320000000, @@ -593,6 +596,7 @@ static const struct cedrus_variant sun8i_a33_cedrus_variant = { static const struct cedrus_variant sun8i_h3_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_H265_DEC | CEDRUS_CAPABILITY_VP8_DEC, @@ -601,6 +605,7 @@ static const struct cedrus_variant sun8i_h3_cedrus_variant = { static const struct cedrus_variant sun8i_v3s_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC, .mod_rate = 297000000, }; @@ -608,6 +613,7 @@ static const struct cedrus_variant sun8i_v3s_cedrus_variant = { static const struct cedrus_variant sun8i_r40_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_VP8_DEC, .mod_rate = 297000000, @@ -616,6 +622,7 @@ static const struct cedrus_variant sun8i_r40_cedrus_variant = { static const struct cedrus_variant sun20i_d1_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_H265_DEC, .mod_rate = 432000000, @@ -624,6 +631,7 @@ static const struct cedrus_variant sun20i_d1_cedrus_variant = { static const struct cedrus_variant sun50i_a64_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_H265_DEC | CEDRUS_CAPABILITY_VP8_DEC, @@ -633,6 +641,7 @@ static const struct cedrus_variant sun50i_a64_cedrus_variant = { static const struct cedrus_variant sun50i_h5_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_H265_DEC | CEDRUS_CAPABILITY_VP8_DEC, @@ -642,6 +651,7 @@ static const struct cedrus_variant sun50i_h5_cedrus_variant = { static const struct cedrus_variant sun50i_h6_cedrus_variant = { .capabilities = CEDRUS_CAPABILITY_UNTILED | CEDRUS_CAPABILITY_MPEG2_DEC | + CEDRUS_CAPABILITY_JPEG_DEC | CEDRUS_CAPABILITY_H264_DEC | CEDRUS_CAPABILITY_H265_DEC | CEDRUS_CAPABILITY_H265_10_DEC | -- 2.45.2 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 4/4] media: cedrus: Don't require requests for all codecs 2024-07-31 16:44 [PATCH 0/4] media: cedrus: Add a JPEG decoder Emmanuel Gil Peyrot ` (2 preceding siblings ...) 2024-07-31 16:44 ` [PATCH 3/4] media: cedrus: Enable JPEG decoding in various variants Emmanuel Gil Peyrot @ 2024-07-31 16:44 ` Emmanuel Gil Peyrot 2024-07-31 16:56 ` Dan Carpenter 3 siblings, 1 reply; 9+ messages in thread From: Emmanuel Gil Peyrot @ 2024-07-31 16:44 UTC (permalink / raw) To: linux-sunxi Cc: Jernej Skrabec, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel From: Jernej Skrabec <jernej.skrabec@gmail.com> JPEG decoding doesn’t need it currently. Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> --- drivers/staging/media/sunxi/cedrus/cedrus_video.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c index 7205c2315bc5..30821d81d0db 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c @@ -599,7 +599,7 @@ int cedrus_queue_init(void *priv, struct vb2_queue *src_vq, src_vq->lock = &ctx->dev->dev_mutex; src_vq->dev = ctx->dev->dev; src_vq->supports_requests = true; - src_vq->requires_requests = true; + src_vq->requires_requests = false; ret = vb2_queue_init(src_vq); if (ret) -- 2.45.2 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 4/4] media: cedrus: Don't require requests for all codecs 2024-07-31 16:44 ` [PATCH 4/4] media: cedrus: Don't require requests for all codecs Emmanuel Gil Peyrot @ 2024-07-31 16:56 ` Dan Carpenter 2025-02-20 17:34 ` Jernej Škrabec 0 siblings, 1 reply; 9+ messages in thread From: Dan Carpenter @ 2024-07-31 16:56 UTC (permalink / raw) To: Emmanuel Gil Peyrot Cc: linux-sunxi, Jernej Skrabec, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel On Wed, Jul 31, 2024 at 06:44:14PM +0200, Emmanuel Gil Peyrot wrote: > From: Jernej Skrabec <jernej.skrabec@gmail.com> > > JPEG decoding doesn’t need it currently. > > Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> > --- Does this change have an effect on runtime? regards, dan carpenter ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 4/4] media: cedrus: Don't require requests for all codecs 2024-07-31 16:56 ` Dan Carpenter @ 2025-02-20 17:34 ` Jernej Škrabec 0 siblings, 0 replies; 9+ messages in thread From: Jernej Škrabec @ 2025-02-20 17:34 UTC (permalink / raw) To: Emmanuel Gil Peyrot, Dan Carpenter Cc: linux-sunxi, Maxime Ripard, Paul Kocialkowski, Mauro Carvalho Chehab, Greg Kroah-Hartman, Chen-Yu Tsai, Samuel Holland, linux-kernel, linux-media, linux-staging, linux-arm-kernel Dne sreda, 31. julij 2024 ob 18:56:15 Srednjeevropski standardni čas je Dan Carpenter napisal(a): > On Wed, Jul 31, 2024 at 06:44:14PM +0200, Emmanuel Gil Peyrot wrote: > > From: Jernej Skrabec <jernej.skrabec@gmail.com> > > > > JPEG decoding doesn’t need it currently. > > > > Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com> > > --- > > Does this change have an effect on runtime? Ideally, driver should allow only JPEG format to be used without request api. Other codecs still require it (decoding doesn't work properly without it). Best regards, Jernej > > regards, > dan carpenter > > > ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2025-02-20 17:40 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-07-31 16:44 [PATCH 0/4] media: cedrus: Add a JPEG decoder Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 1/4] media: cedrus: Setup secondary output formats Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 2/4] media: cedrus: Add JPEG decoder Emmanuel Gil Peyrot 2025-02-13 17:12 ` Sebastian Fricke 2025-02-20 17:32 ` Jernej Škrabec 2024-07-31 16:44 ` [PATCH 3/4] media: cedrus: Enable JPEG decoding in various variants Emmanuel Gil Peyrot 2024-07-31 16:44 ` [PATCH 4/4] media: cedrus: Don't require requests for all codecs Emmanuel Gil Peyrot 2024-07-31 16:56 ` Dan Carpenter 2025-02-20 17:34 ` Jernej Škrabec
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox