public inbox for linux-media@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH] omap_vout: Added check in reqbuf & mmap for buf_size allocation
@ 2011-06-16 20:14 hvaibhav
  2011-06-17  8:45 ` Archit Taneja
  0 siblings, 1 reply; 6+ messages in thread
From: hvaibhav @ 2011-06-16 20:14 UTC (permalink / raw)
  To: linux-media; +Cc: mchehab, hverkuil, Vaibhav Hiremath

From: Vaibhav Hiremath <hvaibhav@ti.com>

The usecase where, user allocates small size of buffer
through bootargs (video1_bufsize/video2_bufsize) and later from application
tries to set the format which requires larger buffer size, driver doesn't
check for insufficient buffer size and allows application to map extra buffer.
This leads to kernel crash, when user application tries to access memory
beyond the allocation size.

Added check in both mmap and reqbuf call back function,
and return error if the size of the buffer allocated by user through
bootargs is less than the S_FMT size.

Signed-off-by: Vaibhav Hiremath <hvaibhav@ti.com>
---
 drivers/media/video/omap/omap_vout.c |   16 ++++++++++++++++
 1 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/drivers/media/video/omap/omap_vout.c b/drivers/media/video/omap/omap_vout.c
index 3bc909a..343b50c 100644
--- a/drivers/media/video/omap/omap_vout.c
+++ b/drivers/media/video/omap/omap_vout.c
@@ -678,6 +678,14 @@ static int omap_vout_buffer_setup(struct videobuf_queue *q, unsigned int *count,
 	startindex = (vout->vid == OMAP_VIDEO1) ?
 		video1_numbuffers : video2_numbuffers;

+	/* Check the size of the buffer */
+	if (*size > vout->buffer_size) {
+		v4l2_err(&vout->vid_dev->v4l2_dev,
+				"buffer allocation mismatch [%u] [%u]\n",
+				*size, vout->buffer_size);
+		return -ENOMEM;
+	}
+
 	for (i = startindex; i < *count; i++) {
 		vout->buffer_size = *size;

@@ -856,6 +864,14 @@ static int omap_vout_mmap(struct file *file, struct vm_area_struct *vma)
 				(vma->vm_pgoff << PAGE_SHIFT));
 		return -EINVAL;
 	}
+	/* Check the size of the buffer */
+	if (size > vout->buffer_size) {
+		v4l2_err(&vout->vid_dev->v4l2_dev,
+				"insufficient memory [%lu] [%u]\n",
+				size, vout->buffer_size);
+		return -ENOMEM;
+	}
+
 	q->bufs[i]->baddr = vma->vm_start;

 	vma->vm_flags |= VM_RESERVED;
--
1.6.2.4


^ permalink raw reply related	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2011-06-17 10:42 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-16 20:14 [PATCH] omap_vout: Added check in reqbuf & mmap for buf_size allocation hvaibhav
2011-06-17  8:45 ` Archit Taneja
2011-06-17 10:03   ` Hiremath, Vaibhav
2011-06-17 10:27     ` Archit Taneja
2011-06-17 10:23       ` Hiremath, Vaibhav
2011-06-17 10:49         ` Archit Taneja

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox