* [PATCH 0/3] media: imx-jpeg: Fix some motion-jpeg decoding issues
@ 2025-03-27 2:37 ming.qian
2025-03-27 2:37 ` [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation ming.qian
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: ming.qian @ 2025-03-27 2:37 UTC (permalink / raw)
To: mchehab, hverkuil-cisco, mirela.rabulea
Cc: shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou,
linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel
From: Ming Qian <ming.qian@oss.nxp.com>
To support decoding motion-jpeg without DHT, driver will try to decode a
pattern jpeg before actual jpeg frame by use of linked descriptors
(This is called "repeat mode"), then the DHT in the pattern jpeg can be
used for decoding the motion-jpeg.
But there is some hardware limitation in the repeat mode, that may cause
corruption or decoding timeout.
Try to make workaround for these limitation in this patchset.
Ming Qian (3):
media: imx-jpeg: Enhance error handling in buffer allocation
media: imx-jpeg: Change the pattern size to 128x64
media: imx-jpeg: Check decoding is ongoing for motion-jpeg
.../media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h | 1 +
.../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 109 +++++++++++++-----
.../media/platform/nxp/imx-jpeg/mxc-jpeg.h | 5 +
3 files changed, 86 insertions(+), 29 deletions(-)
--
2.43.0-rc1
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation 2025-03-27 2:37 [PATCH 0/3] media: imx-jpeg: Fix some motion-jpeg decoding issues ming.qian @ 2025-03-27 2:37 ` ming.qian 2025-03-27 14:35 ` Frank Li 2025-03-27 2:37 ` [PATCH 2/3] media: imx-jpeg: Change the pattern size to 128x64 ming.qian 2025-03-27 2:37 ` [PATCH 3/3] media: imx-jpeg: Check decoding is ongoing for motion-jpeg ming.qian 2 siblings, 1 reply; 8+ messages in thread From: ming.qian @ 2025-03-27 2:37 UTC (permalink / raw) To: mchehab, hverkuil-cisco, mirela.rabulea Cc: shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou, linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel From: Ming Qian <ming.qian@oss.nxp.com> In function mxc_jpeg_alloc_slot_data, driver will allocate some dma buffer, but only return error if certain allocation failed. Without cleanup the allocation failure, the next time it will return success directly, but let some buffer be uninitialized. It may result in accessing a null pointer. Clean up if error occurs in the allocation. Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> --- .../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 47 +++++++++++-------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c index 0e6ee997284b..12661c177f5a 100644 --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c @@ -752,6 +752,32 @@ static int mxc_get_free_slot(struct mxc_jpeg_slot_data *slot_data) return -1; } +static void mxc_jpeg_free_slot_data(struct mxc_jpeg_dev *jpeg) +{ + /* free descriptor for decoding/encoding phase */ + dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), + jpeg->slot_data.desc, + jpeg->slot_data.desc_handle); + jpeg->slot_data.desc = NULL; + jpeg->slot_data.desc_handle = 0; + + /* free descriptor for encoder configuration phase / decoder DHT */ + dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), + jpeg->slot_data.cfg_desc, + jpeg->slot_data.cfg_desc_handle); + jpeg->slot_data.cfg_desc_handle = 0; + jpeg->slot_data.cfg_desc = NULL; + + /* free configuration stream */ + dma_free_coherent(jpeg->dev, MXC_JPEG_MAX_CFG_STREAM, + jpeg->slot_data.cfg_stream_vaddr, + jpeg->slot_data.cfg_stream_handle); + jpeg->slot_data.cfg_stream_vaddr = NULL; + jpeg->slot_data.cfg_stream_handle = 0; + + jpeg->slot_data.used = false; +} + static bool mxc_jpeg_alloc_slot_data(struct mxc_jpeg_dev *jpeg) { struct mxc_jpeg_desc *desc; @@ -794,30 +820,11 @@ static bool mxc_jpeg_alloc_slot_data(struct mxc_jpeg_dev *jpeg) return true; err: dev_err(jpeg->dev, "Could not allocate descriptors for slot %d", jpeg->slot_data.slot); + mxc_jpeg_free_slot_data(jpeg); return false; } -static void mxc_jpeg_free_slot_data(struct mxc_jpeg_dev *jpeg) -{ - /* free descriptor for decoding/encoding phase */ - dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), - jpeg->slot_data.desc, - jpeg->slot_data.desc_handle); - - /* free descriptor for encoder configuration phase / decoder DHT */ - dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), - jpeg->slot_data.cfg_desc, - jpeg->slot_data.cfg_desc_handle); - - /* free configuration stream */ - dma_free_coherent(jpeg->dev, MXC_JPEG_MAX_CFG_STREAM, - jpeg->slot_data.cfg_stream_vaddr, - jpeg->slot_data.cfg_stream_handle); - - jpeg->slot_data.used = false; -} - static void mxc_jpeg_check_and_set_last_buffer(struct mxc_jpeg_ctx *ctx, struct vb2_v4l2_buffer *src_buf, struct vb2_v4l2_buffer *dst_buf) -- 2.43.0-rc1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation 2025-03-27 2:37 ` [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation ming.qian @ 2025-03-27 14:35 ` Frank Li 2025-03-28 1:22 ` Ming Qian(OSS) 0 siblings, 1 reply; 8+ messages in thread From: Frank Li @ 2025-03-27 14:35 UTC (permalink / raw) To: ming.qian Cc: mchehab, hverkuil-cisco, mirela.rabulea, shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou, linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel On Thu, Mar 27, 2025 at 10:37:05AM +0800, ming.qian@oss.nxp.com wrote: > From: Ming Qian <ming.qian@oss.nxp.com> > > In function mxc_jpeg_alloc_slot_data, driver will allocate some dma > buffer, but only return error if certain allocation failed. > > Without cleanup the allocation failure, the next time it will return > success directly, but let some buffer be uninitialized. > It may result in accessing a null pointer. > > Clean up if error occurs in the allocation. > > Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> Look like it is bug fix, add fixes tags Frank > --- > .../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 47 +++++++++++-------- > 1 file changed, 27 insertions(+), 20 deletions(-) > > diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c > index 0e6ee997284b..12661c177f5a 100644 > --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c > +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c > @@ -752,6 +752,32 @@ static int mxc_get_free_slot(struct mxc_jpeg_slot_data *slot_data) > return -1; > } > > +static void mxc_jpeg_free_slot_data(struct mxc_jpeg_dev *jpeg) > +{ > + /* free descriptor for decoding/encoding phase */ > + dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), > + jpeg->slot_data.desc, > + jpeg->slot_data.desc_handle); > + jpeg->slot_data.desc = NULL; > + jpeg->slot_data.desc_handle = 0; > + > + /* free descriptor for encoder configuration phase / decoder DHT */ > + dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), > + jpeg->slot_data.cfg_desc, > + jpeg->slot_data.cfg_desc_handle); > + jpeg->slot_data.cfg_desc_handle = 0; > + jpeg->slot_data.cfg_desc = NULL; > + > + /* free configuration stream */ > + dma_free_coherent(jpeg->dev, MXC_JPEG_MAX_CFG_STREAM, > + jpeg->slot_data.cfg_stream_vaddr, > + jpeg->slot_data.cfg_stream_handle); > + jpeg->slot_data.cfg_stream_vaddr = NULL; > + jpeg->slot_data.cfg_stream_handle = 0; > + > + jpeg->slot_data.used = false; > +} > + > static bool mxc_jpeg_alloc_slot_data(struct mxc_jpeg_dev *jpeg) > { > struct mxc_jpeg_desc *desc; > @@ -794,30 +820,11 @@ static bool mxc_jpeg_alloc_slot_data(struct mxc_jpeg_dev *jpeg) > return true; > err: > dev_err(jpeg->dev, "Could not allocate descriptors for slot %d", jpeg->slot_data.slot); > + mxc_jpeg_free_slot_data(jpeg); > > return false; > } > > -static void mxc_jpeg_free_slot_data(struct mxc_jpeg_dev *jpeg) > -{ > - /* free descriptor for decoding/encoding phase */ > - dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), > - jpeg->slot_data.desc, > - jpeg->slot_data.desc_handle); > - > - /* free descriptor for encoder configuration phase / decoder DHT */ > - dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), > - jpeg->slot_data.cfg_desc, > - jpeg->slot_data.cfg_desc_handle); > - > - /* free configuration stream */ > - dma_free_coherent(jpeg->dev, MXC_JPEG_MAX_CFG_STREAM, > - jpeg->slot_data.cfg_stream_vaddr, > - jpeg->slot_data.cfg_stream_handle); > - > - jpeg->slot_data.used = false; > -} > - > static void mxc_jpeg_check_and_set_last_buffer(struct mxc_jpeg_ctx *ctx, > struct vb2_v4l2_buffer *src_buf, > struct vb2_v4l2_buffer *dst_buf) > -- > 2.43.0-rc1 > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation 2025-03-27 14:35 ` Frank Li @ 2025-03-28 1:22 ` Ming Qian(OSS) 0 siblings, 0 replies; 8+ messages in thread From: Ming Qian(OSS) @ 2025-03-28 1:22 UTC (permalink / raw) To: Frank Li Cc: mchehab, hverkuil-cisco, mirela.rabulea, shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou, linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel Hi Frank, On 2025/3/27 22:35, Frank Li wrote: > On Thu, Mar 27, 2025 at 10:37:05AM +0800, ming.qian@oss.nxp.com wrote: >> From: Ming Qian <ming.qian@oss.nxp.com> >> >> In function mxc_jpeg_alloc_slot_data, driver will allocate some dma >> buffer, but only return error if certain allocation failed. >> >> Without cleanup the allocation failure, the next time it will return >> success directly, but let some buffer be uninitialized. >> It may result in accessing a null pointer. >> >> Clean up if error occurs in the allocation. >> >> Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> > > Look like it is bug fix, add fixes tags Yes, I missed it, I will add it in v2 Thanks, Ming > > Frank >> --- >> .../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 47 +++++++++++-------- >> 1 file changed, 27 insertions(+), 20 deletions(-) >> >> diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c >> index 0e6ee997284b..12661c177f5a 100644 >> --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c >> +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c >> @@ -752,6 +752,32 @@ static int mxc_get_free_slot(struct mxc_jpeg_slot_data *slot_data) >> return -1; >> } >> >> +static void mxc_jpeg_free_slot_data(struct mxc_jpeg_dev *jpeg) >> +{ >> + /* free descriptor for decoding/encoding phase */ >> + dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), >> + jpeg->slot_data.desc, >> + jpeg->slot_data.desc_handle); >> + jpeg->slot_data.desc = NULL; >> + jpeg->slot_data.desc_handle = 0; >> + >> + /* free descriptor for encoder configuration phase / decoder DHT */ >> + dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), >> + jpeg->slot_data.cfg_desc, >> + jpeg->slot_data.cfg_desc_handle); >> + jpeg->slot_data.cfg_desc_handle = 0; >> + jpeg->slot_data.cfg_desc = NULL; >> + >> + /* free configuration stream */ >> + dma_free_coherent(jpeg->dev, MXC_JPEG_MAX_CFG_STREAM, >> + jpeg->slot_data.cfg_stream_vaddr, >> + jpeg->slot_data.cfg_stream_handle); >> + jpeg->slot_data.cfg_stream_vaddr = NULL; >> + jpeg->slot_data.cfg_stream_handle = 0; >> + >> + jpeg->slot_data.used = false; >> +} >> + >> static bool mxc_jpeg_alloc_slot_data(struct mxc_jpeg_dev *jpeg) >> { >> struct mxc_jpeg_desc *desc; >> @@ -794,30 +820,11 @@ static bool mxc_jpeg_alloc_slot_data(struct mxc_jpeg_dev *jpeg) >> return true; >> err: >> dev_err(jpeg->dev, "Could not allocate descriptors for slot %d", jpeg->slot_data.slot); >> + mxc_jpeg_free_slot_data(jpeg); >> >> return false; >> } >> >> -static void mxc_jpeg_free_slot_data(struct mxc_jpeg_dev *jpeg) >> -{ >> - /* free descriptor for decoding/encoding phase */ >> - dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), >> - jpeg->slot_data.desc, >> - jpeg->slot_data.desc_handle); >> - >> - /* free descriptor for encoder configuration phase / decoder DHT */ >> - dma_free_coherent(jpeg->dev, sizeof(struct mxc_jpeg_desc), >> - jpeg->slot_data.cfg_desc, >> - jpeg->slot_data.cfg_desc_handle); >> - >> - /* free configuration stream */ >> - dma_free_coherent(jpeg->dev, MXC_JPEG_MAX_CFG_STREAM, >> - jpeg->slot_data.cfg_stream_vaddr, >> - jpeg->slot_data.cfg_stream_handle); >> - >> - jpeg->slot_data.used = false; >> -} >> - >> static void mxc_jpeg_check_and_set_last_buffer(struct mxc_jpeg_ctx *ctx, >> struct vb2_v4l2_buffer *src_buf, >> struct vb2_v4l2_buffer *dst_buf) >> -- >> 2.43.0-rc1 >> ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 2/3] media: imx-jpeg: Change the pattern size to 128x64 2025-03-27 2:37 [PATCH 0/3] media: imx-jpeg: Fix some motion-jpeg decoding issues ming.qian 2025-03-27 2:37 ` [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation ming.qian @ 2025-03-27 2:37 ` ming.qian 2025-03-27 2:37 ` [PATCH 3/3] media: imx-jpeg: Check decoding is ongoing for motion-jpeg ming.qian 2 siblings, 0 replies; 8+ messages in thread From: ming.qian @ 2025-03-27 2:37 UTC (permalink / raw) To: mchehab, hverkuil-cisco, mirela.rabulea Cc: shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou, linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel From: Ming Qian <ming.qian@oss.nxp.com> To support decoding motion-jpeg without DHT, driver will try to decode a pattern jpeg before actual jpeg frame by use of linked descriptors (This is called "repeat mode"), then the DHT in the pattern jpeg can be used for decoding the motion-jpeg. To avoid performance loss, use the smallest supported resolution 64x64 as the pattern jpeg size. But there is a hardware issue: when the JPEG decoded frame with a resolution that is no larger than 64x64 and it is followed by a next decoded frame with a larger resolution but not 64 aligned, then this next decoded frame may be corrupted. To avoid corruption of the decoded image, we change the pattern jpeg size to 128x64, as we confirmed with the hardware designer that this is a safe size. Besides, we also need to allocate a dma buffer to store the decoded picture for the pattern image. Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> --- .../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 42 +++++++++++++++---- .../media/platform/nxp/imx-jpeg/mxc-jpeg.h | 5 +++ 2 files changed, 39 insertions(+), 8 deletions(-) diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c index 12661c177f5a..45705c606769 100644 --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c @@ -535,7 +535,18 @@ static const unsigned char jpeg_sos_maximal[] = { }; static const unsigned char jpeg_image_red[] = { - 0xFC, 0x5F, 0xA2, 0xBF, 0xCA, 0x73, 0xFE, 0xFE, + 0xF9, 0xFE, 0x8A, 0xFC, 0x34, 0xFD, 0xC4, 0x28, + 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, + 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, + 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, + 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, + 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, + 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, + 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, + 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, + 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, + 0x8A, 0x00, 0x28, 0xA0, 0x0F, 0xFF, 0xD0, 0xF9, + 0xFE, 0x8A, 0xFC, 0x34, 0xFD, 0xC4, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, @@ -545,7 +556,7 @@ static const unsigned char jpeg_image_red[] = { 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00, 0x28, 0xA0, 0x02, 0x8A, - 0x00, 0x28, 0xA0, 0x02, 0x8A, 0x00 + 0x00, 0x28, 0xA0, 0x0F }; static const unsigned char jpeg_eoi[] = { @@ -775,6 +786,13 @@ static void mxc_jpeg_free_slot_data(struct mxc_jpeg_dev *jpeg) jpeg->slot_data.cfg_stream_vaddr = NULL; jpeg->slot_data.cfg_stream_handle = 0; + dma_free_coherent(jpeg->dev, jpeg->slot_data.cfg_dec_size, + jpeg->slot_data.cfg_dec_vaddr, + jpeg->slot_data.cfg_dec_daddr); + jpeg->slot_data.cfg_dec_size = 0; + jpeg->slot_data.cfg_dec_vaddr = NULL; + jpeg->slot_data.cfg_dec_daddr = 0; + jpeg->slot_data.used = false; } @@ -814,6 +832,14 @@ static bool mxc_jpeg_alloc_slot_data(struct mxc_jpeg_dev *jpeg) goto err; jpeg->slot_data.cfg_stream_vaddr = cfg_stm; + jpeg->slot_data.cfg_dec_size = MXC_JPEG_PATTERN_WIDTH * MXC_JPEG_PATTERN_HEIGHT * 2; + jpeg->slot_data.cfg_dec_vaddr = dma_alloc_coherent(jpeg->dev, + jpeg->slot_data.cfg_dec_size, + &jpeg->slot_data.cfg_dec_daddr, + GFP_ATOMIC); + if (!jpeg->slot_data.cfg_dec_vaddr) + goto err; + skip_alloc: jpeg->slot_data.used = true; @@ -1216,14 +1242,14 @@ static void mxc_jpeg_config_dec_desc(struct vb2_buffer *out_buf, */ *cfg_size = mxc_jpeg_setup_cfg_stream(cfg_stream_vaddr, V4L2_PIX_FMT_YUYV, - MXC_JPEG_MIN_WIDTH, - MXC_JPEG_MIN_HEIGHT); + MXC_JPEG_PATTERN_WIDTH, + MXC_JPEG_PATTERN_HEIGHT); cfg_desc->next_descpt_ptr = desc_handle | MXC_NXT_DESCPT_EN; - cfg_desc->buf_base0 = vb2_dma_contig_plane_dma_addr(dst_buf, 0); + cfg_desc->buf_base0 = jpeg->slot_data.cfg_dec_daddr; cfg_desc->buf_base1 = 0; - cfg_desc->imgsize = MXC_JPEG_MIN_WIDTH << 16; - cfg_desc->imgsize |= MXC_JPEG_MIN_HEIGHT; - cfg_desc->line_pitch = MXC_JPEG_MIN_WIDTH * 2; + cfg_desc->imgsize = MXC_JPEG_PATTERN_WIDTH << 16; + cfg_desc->imgsize |= MXC_JPEG_PATTERN_HEIGHT; + cfg_desc->line_pitch = MXC_JPEG_PATTERN_WIDTH * 2; cfg_desc->stm_ctrl = STM_CTRL_IMAGE_FORMAT(MXC_JPEG_YUV422); cfg_desc->stm_ctrl |= STM_CTRL_BITBUF_PTR_CLR(1); cfg_desc->stm_bufbase = cfg_stream_handle; diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.h b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.h index 86e324b21aed..fdde45f7e163 100644 --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.h +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.h @@ -28,6 +28,8 @@ #define MXC_JPEG_W_ALIGN 3 #define MXC_JPEG_MAX_SIZEIMAGE 0xFFFFFC00 #define MXC_JPEG_MAX_PLANES 2 +#define MXC_JPEG_PATTERN_WIDTH 128 +#define MXC_JPEG_PATTERN_HEIGHT 64 enum mxc_jpeg_enc_state { MXC_JPEG_ENCODING = 0, /* jpeg encode phase */ @@ -117,6 +119,9 @@ struct mxc_jpeg_slot_data { dma_addr_t desc_handle; dma_addr_t cfg_desc_handle; // configuration descriptor dma address dma_addr_t cfg_stream_handle; // configuration bitstream dma address + dma_addr_t cfg_dec_size; + void *cfg_dec_vaddr; + dma_addr_t cfg_dec_daddr; }; struct mxc_jpeg_dev { -- 2.43.0-rc1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/3] media: imx-jpeg: Check decoding is ongoing for motion-jpeg 2025-03-27 2:37 [PATCH 0/3] media: imx-jpeg: Fix some motion-jpeg decoding issues ming.qian 2025-03-27 2:37 ` [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation ming.qian 2025-03-27 2:37 ` [PATCH 2/3] media: imx-jpeg: Change the pattern size to 128x64 ming.qian @ 2025-03-27 2:37 ` ming.qian 2025-03-27 14:41 ` Frank Li 2 siblings, 1 reply; 8+ messages in thread From: ming.qian @ 2025-03-27 2:37 UTC (permalink / raw) To: mchehab, hverkuil-cisco, mirela.rabulea Cc: shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou, linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel From: Ming Qian <ming.qian@oss.nxp.com> To support decoding motion-jpeg without DHT, driver will try to decode a pattern jpeg before actual jpeg frame by use of linked descriptors (This is called "repeat mode"), then the DHT in the pattern jpeg can be used for decoding the motion-jpeg. In other words, 2 frame done interrupts will be triggered, driver will ignore the first interrupt, and wait for the second interrupt. If the resolution is small, and the 2 interrupts may be too close, when driver is handling the first interrupt, two frames are done, then driver will fail to wait for the second interrupt. In such case, driver can check whether the decoding is still ongoing, if not, just done the current decoding. Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> --- .../media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h | 1 + .../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 20 ++++++++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h index d579c804b047..adb93e977be9 100644 --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h @@ -89,6 +89,7 @@ /* SLOT_STATUS fields for slots 0..3 */ #define SLOT_STATUS_FRMDONE (0x1 << 3) #define SLOT_STATUS_ENC_CONFIG_ERR (0x1 << 8) +#define SLOT_STATUS_ONGOING (0x1 << 31) /* SLOT_IRQ_EN fields TBD */ diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c index 45705c606769..e6bb45633a19 100644 --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c @@ -910,6 +910,23 @@ static u32 mxc_jpeg_get_plane_size(struct mxc_jpeg_q_data *q_data, u32 plane_no) return size; } +static bool mxc_dec_is_ongoing(struct mxc_jpeg_ctx *ctx) +{ + struct mxc_jpeg_dev *jpeg = ctx->mxc_jpeg; + u32 curr_desc; + u32 slot_status; + + slot_status = readl(jpeg->base_reg + MXC_SLOT_OFFSET(ctx->slot, SLOT_STATUS)); + curr_desc = readl(jpeg->base_reg + MXC_SLOT_OFFSET(ctx->slot, SLOT_CUR_DESCPT_PTR)); + + if (curr_desc == jpeg->slot_data.cfg_desc_handle) + return true; + if (slot_status & SLOT_STATUS_ONGOING) + return true; + + return false; +} + static irqreturn_t mxc_jpeg_dec_irq(int irq, void *priv) { struct mxc_jpeg_dev *jpeg = priv; @@ -979,7 +996,8 @@ static irqreturn_t mxc_jpeg_dec_irq(int irq, void *priv) mxc_jpeg_enc_mode_go(dev, reg, mxc_jpeg_is_extended_sequential(q_data->fmt)); goto job_unlock; } - if (jpeg->mode == MXC_JPEG_DECODE && jpeg_src_buf->dht_needed) { + if (jpeg->mode == MXC_JPEG_DECODE && jpeg_src_buf->dht_needed && + mxc_dec_is_ongoing(ctx)) { jpeg_src_buf->dht_needed = false; dev_dbg(dev, "Decoder DHT cfg finished. Start decoding...\n"); goto job_unlock; -- 2.43.0-rc1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] media: imx-jpeg: Check decoding is ongoing for motion-jpeg 2025-03-27 2:37 ` [PATCH 3/3] media: imx-jpeg: Check decoding is ongoing for motion-jpeg ming.qian @ 2025-03-27 14:41 ` Frank Li 2025-03-28 1:30 ` Ming Qian(OSS) 0 siblings, 1 reply; 8+ messages in thread From: Frank Li @ 2025-03-27 14:41 UTC (permalink / raw) To: ming.qian Cc: mchehab, hverkuil-cisco, mirela.rabulea, shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou, linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel On Thu, Mar 27, 2025 at 10:37:07AM +0800, ming.qian@oss.nxp.com wrote: > From: Ming Qian <ming.qian@oss.nxp.com> > > To support decoding motion-jpeg without DHT, driver will try to decode a > pattern jpeg before actual jpeg frame by use of linked descriptors > (This is called "repeat mode"), then the DHT in the pattern jpeg can be > used for decoding the motion-jpeg. > > In other words, 2 frame done interrupts will be triggered, driver will > ignore the first interrupt, and wait for the second interrupt. > If the resolution is small, and the 2 interrupts may be too close, > when driver is handling the first interrupt, two frames are done, then > driver will fail to wait for the second interrupt. > > In such case, driver can check whether the decoding is still ongoing, > if not, just done the current decoding. > > Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> > --- > .../media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h | 1 + > .../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 20 ++++++++++++++++++- > 2 files changed, 20 insertions(+), 1 deletion(-) > > diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h > index d579c804b047..adb93e977be9 100644 > --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h > +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h > @@ -89,6 +89,7 @@ > /* SLOT_STATUS fields for slots 0..3 */ > #define SLOT_STATUS_FRMDONE (0x1 << 3) > #define SLOT_STATUS_ENC_CONFIG_ERR (0x1 << 8) > +#define SLOT_STATUS_ONGOING (0x1 << 31) > > /* SLOT_IRQ_EN fields TBD */ > > diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c > index 45705c606769..e6bb45633a19 100644 > --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c > +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c > @@ -910,6 +910,23 @@ static u32 mxc_jpeg_get_plane_size(struct mxc_jpeg_q_data *q_data, u32 plane_no) > return size; > } > > +static bool mxc_dec_is_ongoing(struct mxc_jpeg_ctx *ctx) > +{ > + struct mxc_jpeg_dev *jpeg = ctx->mxc_jpeg; > + u32 curr_desc; > + u32 slot_status; > + > + slot_status = readl(jpeg->base_reg + MXC_SLOT_OFFSET(ctx->slot, SLOT_STATUS)); > + curr_desc = readl(jpeg->base_reg + MXC_SLOT_OFFSET(ctx->slot, SLOT_CUR_DESCPT_PTR)); > + > + if (curr_desc == jpeg->slot_data.cfg_desc_handle) > + return true; > + if (slot_status & SLOT_STATUS_ONGOING) > + return true; > + > + return false; > +} > + > static irqreturn_t mxc_jpeg_dec_irq(int irq, void *priv) > { > struct mxc_jpeg_dev *jpeg = priv; > @@ -979,7 +996,8 @@ static irqreturn_t mxc_jpeg_dec_irq(int irq, void *priv) > mxc_jpeg_enc_mode_go(dev, reg, mxc_jpeg_is_extended_sequential(q_data->fmt)); > goto job_unlock; > } > - if (jpeg->mode == MXC_JPEG_DECODE && jpeg_src_buf->dht_needed) { > + if (jpeg->mode == MXC_JPEG_DECODE && jpeg_src_buf->dht_needed && > + mxc_dec_is_ongoing(ctx)) { what happen if hardware completed just after you call mxc_dec_is_ongoing(), any thing will be missed? Frank > jpeg_src_buf->dht_needed = false; > dev_dbg(dev, "Decoder DHT cfg finished. Start decoding...\n"); > goto job_unlock; > -- > 2.43.0-rc1 > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/3] media: imx-jpeg: Check decoding is ongoing for motion-jpeg 2025-03-27 14:41 ` Frank Li @ 2025-03-28 1:30 ` Ming Qian(OSS) 0 siblings, 0 replies; 8+ messages in thread From: Ming Qian(OSS) @ 2025-03-28 1:30 UTC (permalink / raw) To: Frank Li Cc: mchehab, hverkuil-cisco, mirela.rabulea, shawnguo, s.hauer, kernel, festevam, xiahong.bao, eagle.zhou, linux-imx, imx, linux-media, linux-kernel, linux-arm-kernel Hi Frank, On 2025/3/27 22:41, Frank Li wrote: > On Thu, Mar 27, 2025 at 10:37:07AM +0800, ming.qian@oss.nxp.com wrote: >> From: Ming Qian <ming.qian@oss.nxp.com> >> >> To support decoding motion-jpeg without DHT, driver will try to decode a >> pattern jpeg before actual jpeg frame by use of linked descriptors >> (This is called "repeat mode"), then the DHT in the pattern jpeg can be >> used for decoding the motion-jpeg. >> >> In other words, 2 frame done interrupts will be triggered, driver will >> ignore the first interrupt, and wait for the second interrupt. >> If the resolution is small, and the 2 interrupts may be too close, >> when driver is handling the first interrupt, two frames are done, then >> driver will fail to wait for the second interrupt. >> >> In such case, driver can check whether the decoding is still ongoing, >> if not, just done the current decoding. >> >> Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> >> --- >> .../media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h | 1 + >> .../media/platform/nxp/imx-jpeg/mxc-jpeg.c | 20 ++++++++++++++++++- >> 2 files changed, 20 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h >> index d579c804b047..adb93e977be9 100644 >> --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h >> +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg-hw.h >> @@ -89,6 +89,7 @@ >> /* SLOT_STATUS fields for slots 0..3 */ >> #define SLOT_STATUS_FRMDONE (0x1 << 3) >> #define SLOT_STATUS_ENC_CONFIG_ERR (0x1 << 8) >> +#define SLOT_STATUS_ONGOING (0x1 << 31) >> >> /* SLOT_IRQ_EN fields TBD */ >> >> diff --git a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c >> index 45705c606769..e6bb45633a19 100644 >> --- a/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c >> +++ b/drivers/media/platform/nxp/imx-jpeg/mxc-jpeg.c >> @@ -910,6 +910,23 @@ static u32 mxc_jpeg_get_plane_size(struct mxc_jpeg_q_data *q_data, u32 plane_no) >> return size; >> } >> >> +static bool mxc_dec_is_ongoing(struct mxc_jpeg_ctx *ctx) >> +{ >> + struct mxc_jpeg_dev *jpeg = ctx->mxc_jpeg; >> + u32 curr_desc; >> + u32 slot_status; >> + >> + slot_status = readl(jpeg->base_reg + MXC_SLOT_OFFSET(ctx->slot, SLOT_STATUS)); >> + curr_desc = readl(jpeg->base_reg + MXC_SLOT_OFFSET(ctx->slot, SLOT_CUR_DESCPT_PTR)); >> + >> + if (curr_desc == jpeg->slot_data.cfg_desc_handle) >> + return true; >> + if (slot_status & SLOT_STATUS_ONGOING) >> + return true; >> + >> + return false; >> +} >> + >> static irqreturn_t mxc_jpeg_dec_irq(int irq, void *priv) >> { >> struct mxc_jpeg_dev *jpeg = priv; >> @@ -979,7 +996,8 @@ static irqreturn_t mxc_jpeg_dec_irq(int irq, void *priv) >> mxc_jpeg_enc_mode_go(dev, reg, mxc_jpeg_is_extended_sequential(q_data->fmt)); >> goto job_unlock; >> } >> - if (jpeg->mode == MXC_JPEG_DECODE && jpeg_src_buf->dht_needed) { >> + if (jpeg->mode == MXC_JPEG_DECODE && jpeg_src_buf->dht_needed && >> + mxc_dec_is_ongoing(ctx)) { > > what happen if hardware completed just after you call mxc_dec_is_ongoing(), > any thing will be missed? The interrupt status is cleared before the ongoing check, so if hardware complete after mxc_dec_is_ongoing(), a new interrupt is triggered, driver will finish the decoding in the next irq().It's the same normal process as before. Thanks, Ming > > Frank > >> jpeg_src_buf->dht_needed = false; >> dev_dbg(dev, "Decoder DHT cfg finished. Start decoding...\n"); >> goto job_unlock; >> -- >> 2.43.0-rc1 >> ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-03-28 1:33 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-03-27 2:37 [PATCH 0/3] media: imx-jpeg: Fix some motion-jpeg decoding issues ming.qian 2025-03-27 2:37 ` [PATCH 1/3] media: imx-jpeg: Enhance error handling in buffer allocation ming.qian 2025-03-27 14:35 ` Frank Li 2025-03-28 1:22 ` Ming Qian(OSS) 2025-03-27 2:37 ` [PATCH 2/3] media: imx-jpeg: Change the pattern size to 128x64 ming.qian 2025-03-27 2:37 ` [PATCH 3/3] media: imx-jpeg: Check decoding is ongoing for motion-jpeg ming.qian 2025-03-27 14:41 ` Frank Li 2025-03-28 1:30 ` Ming Qian(OSS)
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox