* [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec
@ 2026-05-30 9:42 Anand Moon
2026-05-30 9:42 ` [PATCH v6 1/8] media: meson: vdec: Fix memory leaks and lifetime of m2m device Anand Moon
` (7 more replies)
0 siblings, 8 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Hans Verkuil, Maxime Jourdan,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
v6: Changes
The previous approach had some technical issues, so this new version
takes a slightly different approach, I have fixed the DMA warnings
found during basic testing.
I have donse basic testing on the Odroid N2+ and found that
the clocks are not enabling for decoder.
It also seems some Mali GPU configurations are still missing.
You can reproduce the test case using:
mpv --hwdec=v4l2m2m Big_Buck_Bunny_1080_10s_30MB.mp4
Please let me know your feedback so we can discuss and address these
points!
Thanks
-Anand
V5: Changes
[v5] https://lore.kernel.org/all/20260525095216.12078-2-linux.amoon@gmail.com/
Following chamges try to fix the memory leak reported by Sashiko
New issues:
- [High] The newly added error path in `vdec_start_streaming()` leaks
`sess->priv` when `kthread_run()` fails.
Pre-existing issues:
- [Critical] Race condition between hardware power-on and `core->cur_sess`
initialization leads to a NULL pointer dereference in the IRQ handler.
- [High] Returning buffers for both source and destination queues upon
single-queue failure orphans active queue buffers.
- [High] Concurrent sessions can bypass the hardware exclusivity check,
leading to simultaneous hardware programming.
--
V4: Changes:
v4: https://lore.kernel.org/all/20260521073449.10057-2-linux.amoon@gmail.com/
Following chamges try to fix the memory leak reported by Sashiko
Pre-existing issues:
- [Critical] The `sess->esparser_queue_work` work item is not canceled
before freeing the session context, leading to a potential Use-After-Free
vulnerability.
- [High] The patch attempts to fix a memory leak reported by kmemleak,
but misdiagnoses the root cause and leaves the primary memory leak
(the V4L2 control handler) unresolved.
- [High] The driver does not verify if `kthread_run()` returns an `ERR_PTR`,
leading to a kernel panic when `kthread_stop()` is called.
Thanks
-Anand
Anand Moon (8):
media: meson: vdec: Fix memory leaks and lifetime of m2m device
media: meson: vdec: Fix concurrent STREAMON / STREAMOFF race
conditions
media: meson: vdec: Handle kthread failure and free codec state
media: meson: vdec: Condition buffer flushing on queue type in
start_streaming
media: meson: vdec: Cancel esparser work during teardown
media: meson: vdec: Configure DMA mask and segment size in probe
media: meson: vdec: Fix NULL pointer dereference in ISR handlers
gpu: drm: meson: Fix DMA max segment size for DMABUF imports
drivers/gpu/drm/meson/meson_drv.c | 2 +
drivers/staging/media/meson/vdec/vdec.c | 179 +++++++++++++++++-------
drivers/staging/media/meson/vdec/vdec.h | 4 +-
3 files changed, 136 insertions(+), 49 deletions(-)
base-commit: f5e5d3509bffb95c6648eb9795f7f236852ae62d
--
2.50.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v6 1/8] media: meson: vdec: Fix memory leaks and lifetime of m2m device
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
2026-05-30 9:42 ` [PATCH v6 2/8] media: meson: vdec: Fix concurrent STREAMON / STREAMOFF race conditions Anand Moon
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Maxime Jourdan, Hans Verkuil,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Sashiko, Nicolas Dufresne
The driver was initializing the v4l2 m2m device instance per-session
within vdec_open() and releasing it inside vdec_close(). This approach
is faulty because the m2m device represents the hardware context and
should persist across multiple open sessions.
Fix this design flaw by shifting v4l2_m2m_init() to vdec_probe() and
v4l2_m2m_release() to vdec_remove(). Correspondingly, move the m2m_dev
pointer from struct amvdec_session to struct amvdec_core.
Additionally, this patch addresses two critical resource leaks:
1. Adds a missing v4l2_ctrl_handler_free() in vdec_close() to clean up
allocated control handlers upon session closure.
2. Introduces proper unwinding logic via a new 'err_fh_del' label in
vdec_open() to ensure that file handles (v4l2_fh) are fully deregistered
if subsequent session resource allocations fail.
This was identified via kmemleak:
unreferenced object 0xffff0000205d6878 (size 8):
comm "v4l_id", pid 5289, jiffies 4294938580
hex dump (first 8 bytes):
40 d2 49 18 00 00 ff ff @.I.....
backtrace (crc d3204599):
kmemleak_alloc+0xc8/0xf0
__kvmalloc_node_noprof+0x60c/0x850
v4l2_ctrl_handler_init_class+0x1b4/0x2e8 [videodev]
vdec_open+0x1f4/0x788 [meson_vdec]
v4l2_open+0x144/0x460 [videodev]
chrdev_open+0x1ac/0x500
do_dentry_open+0x3f0/0xfe8
vfs_open+0x68/0x320
do_open+0x2d8/0x9a8
path_openat+0x1d0/0x4f0
do_filp_open+0x190/0x380
do_sys_openat2+0xf8/0x1b0
__arm64_sys_openat+0x13c/0x1e8
invoke_syscall+0xdc/0x268
el0_svc_common.constprop.0+0x178/0x258
do_el0_svc+0x4c/0x70
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260520045905.6ACBA1F000E9@smtp.kernel.org/#t
Fixes: 3e7f51bd9607 ("media: meson: add v4l2 m2m video decoder driver")
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/staging/media/meson/vdec/vdec.c | 33 ++++++++++++++-----------
drivers/staging/media/meson/vdec/vdec.h | 4 +--
2 files changed, 20 insertions(+), 17 deletions(-)
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 4b77ec1af5a7..4ffebba2341d 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -153,7 +153,7 @@ static void vdec_m2m_job_abort(void *priv)
{
struct amvdec_session *sess = priv;
- v4l2_m2m_job_finish(sess->m2m_dev, sess->m2m_ctx);
+ v4l2_m2m_job_finish(sess->core->m2m_dev, sess->m2m_ctx);
}
static const struct v4l2_m2m_ops vdec_m2m_ops = {
@@ -873,23 +873,16 @@ static int vdec_open(struct file *file)
sess->core = core;
- sess->m2m_dev = v4l2_m2m_init(&vdec_m2m_ops);
- if (IS_ERR(sess->m2m_dev)) {
- dev_err(dev, "Fail to v4l2_m2m_init\n");
- ret = PTR_ERR(sess->m2m_dev);
- goto err_free_sess;
- }
-
- sess->m2m_ctx = v4l2_m2m_ctx_init(sess->m2m_dev, sess, m2m_queue_init);
+ sess->m2m_ctx = v4l2_m2m_ctx_init(core->m2m_dev, sess, m2m_queue_init);
if (IS_ERR(sess->m2m_ctx)) {
dev_err(dev, "Fail to v4l2_m2m_ctx_init\n");
ret = PTR_ERR(sess->m2m_ctx);
- goto err_m2m_release;
+ goto err_fh_del;
}
ret = vdec_init_ctrls(sess);
if (ret)
- goto err_m2m_release;
+ goto err_free_sess;
sess->pixfmt_cap = formats[0].pixfmts_cap[0];
sess->fmt_out = &formats[0];
@@ -913,8 +906,8 @@ static int vdec_open(struct file *file)
return 0;
-err_m2m_release:
- v4l2_m2m_release(sess->m2m_dev);
+err_fh_del:
+ v4l2_fh_exit(&sess->fh);
err_free_sess:
kfree(sess);
return ret;
@@ -925,9 +918,9 @@ static int vdec_close(struct file *file)
struct amvdec_session *sess = file_to_amvdec_session(file);
v4l2_m2m_ctx_release(sess->m2m_ctx);
- v4l2_m2m_release(sess->m2m_dev);
v4l2_fh_del(&sess->fh, file);
v4l2_fh_exit(&sess->fh);
+ v4l2_ctrl_handler_free(&sess->ctrl_handler);
mutex_destroy(&sess->lock);
mutex_destroy(&sess->bufs_recycle_lock);
@@ -1057,10 +1050,17 @@ static int vdec_probe(struct platform_device *pdev)
if (ret)
return ret;
+ core->m2m_dev = v4l2_m2m_init(&vdec_m2m_ops);
+ if (IS_ERR(core->m2m_dev)) {
+ dev_err(dev, "Failed to initialize v4l2 m2m device\n");
+ return PTR_ERR(core->m2m_dev);
+ }
+
ret = v4l2_device_register(dev, &core->v4l2_dev);
if (ret) {
dev_err(dev, "Couldn't register v4l2 device\n");
- return -ENOMEM;
+ ret = -ENOMEM;
+ goto err_m2m_release;
}
vdev = video_device_alloc();
@@ -1095,6 +1095,8 @@ static int vdec_probe(struct platform_device *pdev)
err_vdev_release:
video_device_release(vdev);
v4l2_device_unregister(&core->v4l2_dev);
+err_m2m_release:
+ v4l2_m2m_release(core->m2m_dev);
return ret;
}
@@ -1104,6 +1106,7 @@ static void vdec_remove(struct platform_device *pdev)
video_unregister_device(core->vdev_dec);
v4l2_device_unregister(&core->v4l2_dev);
+ v4l2_m2m_release(core->m2m_dev);
}
static struct platform_driver meson_vdec_driver = {
diff --git a/drivers/staging/media/meson/vdec/vdec.h b/drivers/staging/media/meson/vdec/vdec.h
index 7a5d8e871d70..cc0cfafb8a95 100644
--- a/drivers/staging/media/meson/vdec/vdec.h
+++ b/drivers/staging/media/meson/vdec/vdec.h
@@ -63,6 +63,7 @@ struct amvdec_session;
* @vdec_hevcf_clk: VDEC_HEVCF clock
* @esparser_reset: RESET for the PARSER
* @vdev_dec: video device for the decoder
+ * @m2m_dev: v4l2 m2m device
* @v4l2_dev: v4l2 device
* @cur_sess: current decoding session
* @lock: video device lock
@@ -87,6 +88,7 @@ struct amvdec_core {
struct reset_control *esparser_reset;
struct video_device *vdev_dec;
+ struct v4l2_m2m_dev *m2m_dev;
struct v4l2_device v4l2_dev;
struct amvdec_session *cur_sess;
@@ -183,7 +185,6 @@ enum amvdec_status {
*
* @core: reference to the vdec core struct
* @fh: v4l2 file handle
- * @m2m_dev: v4l2 m2m device
* @m2m_ctx: v4l2 m2m context
* @ctrl_handler: V4L2 control handler
* @ctrl_min_buf_capture: V4L2 control V4L2_CID_MIN_BUFFERS_FOR_CAPTURE
@@ -230,7 +231,6 @@ struct amvdec_session {
struct amvdec_core *core;
struct v4l2_fh fh;
- struct v4l2_m2m_dev *m2m_dev;
struct v4l2_m2m_ctx *m2m_ctx;
struct v4l2_ctrl_handler ctrl_handler;
struct v4l2_ctrl *ctrl_min_buf_capture;
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v6 2/8] media: meson: vdec: Fix concurrent STREAMON / STREAMOFF race conditions
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
2026-05-30 9:42 ` [PATCH v6 1/8] media: meson: vdec: Fix memory leaks and lifetime of m2m device Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
2026-05-30 9:42 ` [PATCH v6 3/8] media: meson: vdec: Handle kthread failure and free codec state Anand Moon
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Hans Verkuil, Maxime Jourdan,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Sashiko, Nicolas Dufresne
The Meson VDEC driver’s start/stop streaming paths previously
updated core->cur_sess and sess->status without synchronization,
leaving a race window between concurrent STREAMON/STREAMOFF calls.
Following change introduces proper locking discipline:
- Hold core->lock when checking or updating core->cur_sess and
sess->status in vdec_start_streaming().
- Snapshot sess->status under the lock in vdec_stop_streaming()
to safely evaluate hardware state after releasing the mutex.
- Ensure error unwind paths clear core->cur_sess and reset
sess->status inside the lock.
This prevents TOCTOU races, avoids data corruption when multiple
sessions contend for the hardware, and ensures consistent session
lifecycle management.
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260525104345.C8D501F00A3C@smtp.kernel.org/
Fixes: 3e7f51bd9607 ("media: meson: add v4l2 m2m video decoder driver")
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/staging/media/meson/vdec/vdec.c | 62 ++++++++++++++++++-------
1 file changed, 46 insertions(+), 16 deletions(-)
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 4ffebba2341d..7233000e2232 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -286,11 +286,6 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
struct vb2_v4l2_buffer *buf;
int ret;
- if (core->cur_sess && core->cur_sess != sess) {
- ret = -EBUSY;
- goto bufs_done;
- }
-
if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
sess->streamon_out = 1;
else
@@ -308,9 +303,29 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
}
if (sess->status == STATUS_RUNNING ||
- sess->status == STATUS_NEEDS_RESUME ||
- sess->status == STATUS_INIT)
+ sess->status == STATUS_NEEDS_RESUME)
+ return 0;
+
+ /*
+ * Secure the core hardware lock before checking availability
+ * and updating session states to prevent STREAMON race conditions.
+ */
+ mutex_lock(&core->lock);
+ if (core->cur_sess && core->cur_sess != sess) {
+ mutex_unlock(&core->lock);
+ ret = -EBUSY;
+ goto bufs_done;
+ }
+
+ /* If already half-initialized, do not re-initialize */
+ if (sess->status == STATUS_INIT) {
+ mutex_unlock(&core->lock);
return 0;
+ }
+
+ sess->status = STATUS_INIT;
+ core->cur_sess = sess;
+ mutex_unlock(&core->lock);
sess->vififo_size = SIZE_VIFIFO;
sess->vififo_vaddr =
@@ -341,8 +356,6 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
sess->recycle_thread = kthread_run(vdec_recycle_thread, sess,
"vdec_recycle");
- sess->status = STATUS_INIT;
- core->cur_sess = sess;
schedule_work(&sess->esparser_queue_work);
return 0;
@@ -350,6 +363,12 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
dma_free_coherent(sess->core->dev, sess->vififo_size,
sess->vififo_vaddr, sess->vififo_paddr);
bufs_done:
+ mutex_lock(&core->lock);
+ if (core->cur_sess == sess)
+ core->cur_sess = NULL;
+ sess->status = STATUS_STOPPED;
+ mutex_unlock(&core->lock);
+
while ((buf = v4l2_m2m_src_buf_remove(sess->m2m_ctx)))
v4l2_m2m_buf_done(buf, VB2_BUF_STATE_QUEUED);
while ((buf = v4l2_m2m_dst_buf_remove(sess->m2m_ctx)))
@@ -399,10 +418,23 @@ static void vdec_stop_streaming(struct vb2_queue *q)
struct amvdec_codec_ops *codec_ops = sess->fmt_out->codec_ops;
struct amvdec_core *core = sess->core;
struct vb2_v4l2_buffer *buf;
+ enum amvdec_status old_status;
- if (sess->status == STATUS_RUNNING ||
- sess->status == STATUS_INIT ||
- (sess->status == STATUS_NEEDS_RESUME &&
+ /*
+ * Safely snapshot the status and clear the hardware owner inside
+ * the mutex to prevent data races with concurrent STREAMON requests.
+ */
+ mutex_lock(&core->lock);
+ old_status = sess->status;
+ if (core->cur_sess == sess)
+ core->cur_sess = NULL;
+ sess->status = STATUS_STOPPED;
+ mutex_unlock(&core->lock);
+
+ /* Evaluate the hardware state using our snapshot */
+ if (old_status == STATUS_RUNNING ||
+ old_status == STATUS_INIT ||
+ (old_status == STATUS_NEEDS_RESUME &&
(!sess->streamon_out || !sess->streamon_cap))) {
if (vdec_codec_needs_recycle(sess))
kthread_stop(sess->recycle_thread);
@@ -415,8 +447,6 @@ static void vdec_stop_streaming(struct vb2_queue *q)
vdec_reset_bufs_recycle(sess);
kfree(sess->priv);
sess->priv = NULL;
- core->cur_sess = NULL;
- sess->status = STATUS_STOPPED;
}
if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
@@ -425,8 +455,8 @@ static void vdec_stop_streaming(struct vb2_queue *q)
sess->streamon_out = 0;
} else {
- /* Drain remaining refs if was still running */
- if (sess->status >= STATUS_RUNNING && codec_ops->drain)
+ /* Drain remaining refs if was still running using the snapshot */
+ if (old_status >= STATUS_RUNNING && codec_ops->drain)
codec_ops->drain(sess);
while ((buf = v4l2_m2m_dst_buf_remove(sess->m2m_ctx)))
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v6 3/8] media: meson: vdec: Handle kthread failure and free codec state
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
2026-05-30 9:42 ` [PATCH v6 1/8] media: meson: vdec: Fix memory leaks and lifetime of m2m device Anand Moon
2026-05-30 9:42 ` [PATCH v6 2/8] media: meson: vdec: Fix concurrent STREAMON / STREAMOFF race conditions Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
2026-05-30 9:42 ` [PATCH v6 4/8] media: meson: vdec: Condition buffer flushing on queue type in start_streaming Anand Moon
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Maxime Jourdan, Hans Verkuil,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Sashiko, Nicolas Dufresne
vdec_start_streaming() launches a recycle thread when required by the
codec. If kthread_run() fails, the previous error path only powered off
the hardware, leaving sess->priv and codec state allocated. This caused
a permanent leak of the codec context and associated DMA buffers.
Fix this by introducing a dedicated err_cleanup path:
- Call codec_ops->stop() to release the codec context and clear sess->priv.
- Power off the hardware before freeing buffers to avoid DMA faults.
- Free canvas IDs explicitly and set sess->vififo_vaddr = NULL after
dma_free_coherent() to guard against double‑free in fallback stop paths.
- Reset core->cur_sess and sess->status to prevent stale references.
Following change closes the memory leak on kthread_run() failure and ensures
robust cleanup of codec resources in both error and stop paths.
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260521090944.F35401F00A3D@smtp.kernel.org/
Fixes: 3e7f51bd9607 ("media: meson: add v4l2 m2m video decoder driver")
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/staging/media/meson/vdec/vdec.c | 26 ++++++++++++++++++++++---
1 file changed, 23 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 7233000e2232..8a5bf1a96830 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -32,6 +32,8 @@ struct dummy_buf {
/* 16 MiB for parsed bitstream swap exchange */
#define SIZE_VIFIFO SZ_16M
+static void vdec_free_canvas(struct amvdec_session *sess);
+
static u32 get_output_size(u32 width, u32 height)
{
return ALIGN(width * height, SZ_64K);
@@ -352,16 +354,31 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
sess->sequence_cap = 0;
sess->sequence_out = 0;
- if (vdec_codec_needs_recycle(sess))
+ if (vdec_codec_needs_recycle(sess)) {
sess->recycle_thread = kthread_run(vdec_recycle_thread, sess,
"vdec_recycle");
+ if (IS_ERR(sess->recycle_thread)) {
+ ret = PTR_ERR(sess->recycle_thread);
+ sess->recycle_thread = NULL;
+ goto err_cleanup;
+ }
+ }
schedule_work(&sess->esparser_queue_work);
return 0;
+err_cleanup:
+ vdec_free_canvas(sess);
+ vdec_poweroff(sess);
+ if (codec_ops && codec_ops->stop && sess->priv) {
+ codec_ops->stop(sess);
+ kfree(sess->priv);
+ sess->priv = NULL;
+ }
vififo_free:
dma_free_coherent(sess->core->dev, sess->vififo_size,
sess->vififo_vaddr, sess->vififo_paddr);
+ sess->vififo_vaddr = NULL;
bufs_done:
mutex_lock(&core->lock);
if (core->cur_sess == sess)
@@ -441,8 +458,11 @@ static void vdec_stop_streaming(struct vb2_queue *q)
vdec_poweroff(sess);
vdec_free_canvas(sess);
- dma_free_coherent(sess->core->dev, sess->vififo_size,
- sess->vififo_vaddr, sess->vififo_paddr);
+ if (sess->vififo_vaddr) {
+ dma_free_coherent(sess->core->dev, sess->vififo_size,
+ sess->vififo_vaddr,
+ sess->vififo_paddr);
+ }
vdec_reset_timestamps(sess);
vdec_reset_bufs_recycle(sess);
kfree(sess->priv);
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v6 4/8] media: meson: vdec: Condition buffer flushing on queue type in start_streaming
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
` (2 preceding siblings ...)
2026-05-30 9:42 ` [PATCH v6 3/8] media: meson: vdec: Handle kthread failure and free codec state Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
2026-05-30 9:42 ` [PATCH v6 5/8] media: meson: vdec: Cancel esparser work during teardown Anand Moon
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Maxime Jourdan, Hans Verkuil,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Sashiko, Nicolas Dufresne
When vdec_start_streaming() fails, the error path clears buffers from both
the source and destination queues unconditionally. If one queue was already
streaming successfully from a prior invocation, flushing its buffers behind
its back leaves videobuf2 deadlocked waiting for completions.
Fix this by only sweeping buffers from the specific queue type container
that failed to initialize.
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260521090944.F35401F00A3D@smtp.kernel.org/
Fixes: 3e7f51bd9607 ("media: meson: add v4l2 m2m video decoder driver")
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/staging/media/meson/vdec/vdec.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 8a5bf1a96830..698a95566ad2 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -386,15 +386,15 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
sess->status = STATUS_STOPPED;
mutex_unlock(&core->lock);
- while ((buf = v4l2_m2m_src_buf_remove(sess->m2m_ctx)))
- v4l2_m2m_buf_done(buf, VB2_BUF_STATE_QUEUED);
- while ((buf = v4l2_m2m_dst_buf_remove(sess->m2m_ctx)))
- v4l2_m2m_buf_done(buf, VB2_BUF_STATE_QUEUED);
-
- if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
sess->streamon_out = 0;
- else
+ while ((buf = v4l2_m2m_src_buf_remove(sess->m2m_ctx)))
+ v4l2_m2m_buf_done(buf, VB2_BUF_STATE_QUEUED);
+ } else {
sess->streamon_cap = 0;
+ while ((buf = v4l2_m2m_dst_buf_remove(sess->m2m_ctx)))
+ v4l2_m2m_buf_done(buf, VB2_BUF_STATE_QUEUED);
+ }
return ret;
}
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v6 5/8] media: meson: vdec: Cancel esparser work during teardown
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
` (3 preceding siblings ...)
2026-05-30 9:42 ` [PATCH v6 4/8] media: meson: vdec: Condition buffer flushing on queue type in start_streaming Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
2026-05-30 9:42 ` [PATCH v6 6/8] media: meson: vdec: Configure DMA mask and segment size in probe Anand Moon
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Hans Verkuil, Maxime Jourdan,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Sashiko, Nicolas Dufresne
The esparser workqueue could remain active during error unwind,
streaming stop, or device close, leading to use‑after‑free when
work items accessed freed session memory.
Fix this by explicitly cancelling the work in all teardown paths:
- Call cancel_work_sync(&sess->esparser_queue_work) in
vdec_start_streaming() error unwind, vdec_stop_streaming(),
and vdec_close().
- Ensure the workqueue is drained before releasing session
state and buffers.
- Move codec_ops->drain() evaluation earlier in stop_streaming()
using the status snapshot, so draining occurs before buffer
cleanup.
Following change prevents dangling work execution, eliminates
use‑after‑free hazards, and ensures orderly teardown of decoder
resources.
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260521090944.F35401F00A3D@smtp.kernel.org/
Fixes: 3e7f51bd9607 ("media: meson: add v4l2 m2m video decoder driver")
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/staging/media/meson/vdec/vdec.c | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 698a95566ad2..4884ee04b352 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -380,6 +380,8 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
sess->vififo_vaddr, sess->vififo_paddr);
sess->vififo_vaddr = NULL;
bufs_done:
+ cancel_work_sync(&sess->esparser_queue_work);
+
mutex_lock(&core->lock);
if (core->cur_sess == sess)
core->cur_sess = NULL;
@@ -437,6 +439,8 @@ static void vdec_stop_streaming(struct vb2_queue *q)
struct vb2_v4l2_buffer *buf;
enum amvdec_status old_status;
+ cancel_work_sync(&sess->esparser_queue_work);
+
/*
* Safely snapshot the status and clear the hardware owner inside
* the mutex to prevent data races with concurrent STREAMON requests.
@@ -448,7 +452,11 @@ static void vdec_stop_streaming(struct vb2_queue *q)
sess->status = STATUS_STOPPED;
mutex_unlock(&core->lock);
- /* Evaluate the hardware state using our snapshot */
+ if (q->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+ if (old_status >= STATUS_RUNNING && codec_ops->drain)
+ codec_ops->drain(sess);
+ }
+
if (old_status == STATUS_RUNNING ||
old_status == STATUS_INIT ||
(old_status == STATUS_NEEDS_RESUME &&
@@ -472,16 +480,10 @@ static void vdec_stop_streaming(struct vb2_queue *q)
if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
while ((buf = v4l2_m2m_src_buf_remove(sess->m2m_ctx)))
v4l2_m2m_buf_done(buf, VB2_BUF_STATE_ERROR);
-
sess->streamon_out = 0;
} else {
- /* Drain remaining refs if was still running using the snapshot */
- if (old_status >= STATUS_RUNNING && codec_ops->drain)
- codec_ops->drain(sess);
-
while ((buf = v4l2_m2m_dst_buf_remove(sess->m2m_ctx)))
v4l2_m2m_buf_done(buf, VB2_BUF_STATE_ERROR);
-
sess->streamon_cap = 0;
}
}
@@ -967,6 +969,8 @@ static int vdec_close(struct file *file)
{
struct amvdec_session *sess = file_to_amvdec_session(file);
+ cancel_work_sync(&sess->esparser_queue_work);
+
v4l2_m2m_ctx_release(sess->m2m_ctx);
v4l2_fh_del(&sess->fh, file);
v4l2_fh_exit(&sess->fh);
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v6 6/8] media: meson: vdec: Configure DMA mask and segment size in probe
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
` (4 preceding siblings ...)
2026-05-30 9:42 ` [PATCH v6 5/8] media: meson: vdec: Cancel esparser work during teardown Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
2026-05-30 9:42 ` [PATCH v6 7/8] media: meson: vdec: Fix NULL pointer dereference in ISR handlers Anand Moon
2026-05-30 9:42 ` [PATCH v6 8/8] gpu: drm: meson: Fix DMA max segment size for DMABUF imports Anand Moon
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Hans Verkuil, Maxime Jourdan,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Sashiko, Nicolas Dufresne
The vdec probe routine did not set explicit DMA constraints,
leaving the driver dependent on platform defaults. This could
cause allocation failures or fragmented buffer handling on
systems with stricter DMA limits.
Fix this by:
- Setting a 64 bit coherent DMA mask with
dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)).
- Configuring the maximum contiguous segment size to UINT_MAX
via vb2_dma_contig_set_max_seg_size().
This aligns the driver with common DMA setup practices and guarantees
large buffer allocations work reliably across platforms.
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260521090944.F35401F00A3D@smtp.kernel.org/
Fixes: 3e7f51bd9607 ("media: meson: add v4l2 m2m video decoder driver")
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/staging/media/meson/vdec/vdec.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 4884ee04b352..f99335effe17 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -1064,6 +1064,15 @@ static int vdec_probe(struct platform_device *pdev)
if (IS_ERR(core->canvas))
return PTR_ERR(core->canvas);
+ ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
+ if (ret)
+ return dev_err_probe(dev, ret, "Failed to set DMA mask\n");
+
+ ret = vb2_dma_contig_set_max_seg_size(dev, UINT_MAX);
+ if (ret)
+ return dev_err_probe(dev, ret,
+ "Failed to set DMA max segment size\n");
+
of_id = of_match_node(vdec_dt_match, dev->of_node);
core->platform = of_id->data;
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v6 7/8] media: meson: vdec: Fix NULL pointer dereference in ISR handlers
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
` (5 preceding siblings ...)
2026-05-30 9:42 ` [PATCH v6 6/8] media: meson: vdec: Configure DMA mask and segment size in probe Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
2026-05-30 9:42 ` [PATCH v6 8/8] gpu: drm: meson: Fix DMA max segment size for DMABUF imports Anand Moon
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Maxime Jourdan, Hans Verkuil,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Sashiko, Nicolas Dufresne
The hard interrupt handler (vdec_isr) and the threaded interrupt handler
(vdec_threaded_isr) directly read core->cur_sess without synchronization
or validation. If a streaming teardown concurrently clears core->cur_sess
to NULL while an interrupt is being processed, a NULL pointer dereference
occurs when accessing the session fields or codec operations.
Fix this race condition by using READ_ONCE() to obtain a stable, atomic
snapshot of core->cur_sess. Check if the returned session pointer is NULL,
and return IRQ_NONE immediately if the session has already been torn down.
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260521090944.F35401F00A3D@smtp.kernel.org/
Fixes: 3e7f51bd9607 ("media: meson: add v4l2 m2m video decoder driver")
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/staging/media/meson/vdec/vdec.c | 25 ++++++++++++++++++++++---
1 file changed, 22 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index f99335effe17..3897c75b19c8 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -996,17 +996,36 @@ static const struct v4l2_file_operations vdec_fops = {
static irqreturn_t vdec_isr(int irq, void *data)
{
struct amvdec_core *core = data;
- struct amvdec_session *sess = core->cur_sess;
+ struct amvdec_session *sess;
+ irqreturn_t ret = IRQ_HANDLED;
+
+ /*
+ * Use READ_ONCE to secure an atomic snapshot of the pointer,
+ * protecting against concurrent clearing during streaming
+ * teardowns.
+ */
+ sess = READ_ONCE(core->cur_sess);
+ if (!sess)
+ return IRQ_NONE;
sess->last_irq_jiffies = get_jiffies_64();
+ ret = sess->fmt_out->codec_ops->isr(sess);
- return sess->fmt_out->codec_ops->isr(sess);
+ return ret;
}
static irqreturn_t vdec_threaded_isr(int irq, void *data)
{
struct amvdec_core *core = data;
- struct amvdec_session *sess = core->cur_sess;
+ struct amvdec_session *sess;
+
+ /*
+ * Prevent late-stage threaded interrupts from dereferencing a NULL
+ * session.
+ */
+ sess = READ_ONCE(core->cur_sess);
+ if (!sess)
+ return IRQ_NONE;
return sess->fmt_out->codec_ops->threaded_isr(sess);
}
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v6 8/8] gpu: drm: meson: Fix DMA max segment size for DMABUF imports
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
` (6 preceding siblings ...)
2026-05-30 9:42 ` [PATCH v6 7/8] media: meson: vdec: Fix NULL pointer dereference in ISR handlers Anand Moon
@ 2026-05-30 9:42 ` Anand Moon
7 siblings, 0 replies; 9+ messages in thread
From: Anand Moon @ 2026-05-30 9:42 UTC (permalink / raw)
To: Neil Armstrong, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter, Kevin Hilman,
Jerome Brunet, Martin Blumenstingl, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Hans Verkuil, Maxime Jourdan,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
open list:DRM DRIVERS FOR AMLOGIC SOCS,
moderated list:ARM/Amlogic Meson SoC support, open list,
open list:MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS,
open list:STAGING SUBSYSTEM
Cc: Nicolas Dufresne
When using hardware-accelerated video decoding via v4l2m2m-copy with mpv or
similar players, the video decoder driver (`meson_vdec`) exports contiguous
memory allocations as DMABUFs. When these buffers are subsequently imported
by the display controller driver (`meson-drm`) for rendering via the GPU or
compositor, the DMA API throws constraint validation warnings.
- Call dma_set_max_seg_size(dev, UINT_MAX) to allow large
scatter‑gather segments.
- Ensures the DRM core and canvas allocations can handle
full sized buffers without hitting DMA‑API warnings.
This aligns the driver with common DMA setup practices and
avoids failures on platforms with strict segment limits.
Cc: Nicolas Dufresne <nicolas@ndufresne.ca>
Signed-off-by: Anand Moon <linux.amoon@gmail.com>
---
drivers/gpu/drm/meson/meson_drv.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/meson/meson_drv.c b/drivers/gpu/drm/meson/meson_drv.c
index 49ff9f1f16d3..8570add8b831 100644
--- a/drivers/gpu/drm/meson/meson_drv.c
+++ b/drivers/gpu/drm/meson/meson_drv.c
@@ -247,6 +247,8 @@ static int meson_drv_bind_master(struct device *dev, bool has_components)
goto free_drm;
}
+ dma_set_max_seg_size(dev, UINT_MAX);
+
ret = meson_canvas_alloc(priv->canvas, &priv->canvas_id_osd1);
if (ret)
goto free_drm;
--
2.50.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-05-30 9:45 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-05-30 9:42 [PATCH v6 0/8] media: meson: Fix memory leak in error path in vdec Anand Moon
2026-05-30 9:42 ` [PATCH v6 1/8] media: meson: vdec: Fix memory leaks and lifetime of m2m device Anand Moon
2026-05-30 9:42 ` [PATCH v6 2/8] media: meson: vdec: Fix concurrent STREAMON / STREAMOFF race conditions Anand Moon
2026-05-30 9:42 ` [PATCH v6 3/8] media: meson: vdec: Handle kthread failure and free codec state Anand Moon
2026-05-30 9:42 ` [PATCH v6 4/8] media: meson: vdec: Condition buffer flushing on queue type in start_streaming Anand Moon
2026-05-30 9:42 ` [PATCH v6 5/8] media: meson: vdec: Cancel esparser work during teardown Anand Moon
2026-05-30 9:42 ` [PATCH v6 6/8] media: meson: vdec: Configure DMA mask and segment size in probe Anand Moon
2026-05-30 9:42 ` [PATCH v6 7/8] media: meson: vdec: Fix NULL pointer dereference in ISR handlers Anand Moon
2026-05-30 9:42 ` [PATCH v6 8/8] gpu: drm: meson: Fix DMA max segment size for DMABUF imports Anand Moon
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox