public inbox for linux-media@vger.kernel.org
 help / color / mirror / Atom feed
* Dead code in v4l2-mem2mem.c?
@ 2017-02-17  3:47 Shaobo
  0 siblings, 0 replies; 6+ messages in thread
From: Shaobo @ 2017-02-17  3:47 UTC (permalink / raw)
  To: linux-media

Hi there,

My name is Shaobo He and I am a graduate student at University of Utah. 
I am applying a static analysis tool to the Linux device drivers, 
looking for NULL pointer dereference and accidentally found a plausible 
dead code location in v4l2-mem2mem.c due to undefined behavior.

The following is the problematic code segment 
(drivers/media/v4l2-core/v4l2-mem2mem.c),

> 70 static struct v4l2_m2m_queue_ctx *get_queue_ctx(struct v4l2_m2m_ctx 
> *m2m_ctx,
> 71                                                 enum v4l2_buf_type 
> type)
> 72 {
> 73         if (V4L2_TYPE_IS_OUTPUT(type))
> 74                 return &m2m_ctx->out_q_ctx;
> 75         else
> 76                 return &m2m_ctx->cap_q_ctx;
> 77 }
> 78
> 79 struct vb2_queue *v4l2_m2m_get_vq(struct v4l2_m2m_ctx *m2m_ctx,
> 80                                        enum v4l2_buf_type type)
> 81 {
> 82         struct v4l2_m2m_queue_ctx *q_ctx;
> 83
> 84         q_ctx = get_queue_ctx(m2m_ctx, type);
> 85         if (!q_ctx)
> 86                 return NULL;
> 87
> 88         return &q_ctx->q;
> 89 }

`get_queue_ctx` returns a pointer value that is an addition of the base 
pointer address (`m2m_ctx`) to a non-zero offset. The following is the 
definition of struct v4l2_m2m_ctx (include/media/v4l2-mem2mem.h),

> 94 struct v4l2_m2m_ctx {
> 95         /* optional cap/out vb2 queues lock */
> 96         struct mutex                    *q_lock;
> 97
> 98         /* internal use only */
> 99         struct v4l2_m2m_dev             *m2m_dev;
> 100
> 101         struct v4l2_m2m_queue_ctx       cap_q_ctx;
> 102
> 103         struct v4l2_m2m_queue_ctx       out_q_ctx;
> 104
> 105         /* For device job queue */
> 106         struct list_head                queue;
> 107         unsigned long                   job_flags;
> 108         wait_queue_head_t               finished;
> 109
> 110         void                            *priv;
> 111 };

There is a NULL test in a caller of `get_queue_ctx` (line 85), which 
appears problematic to me. I’m not sure if it is defined or feasible 
under the context of Linux kernel. This blog 
(https://wdtz.org/undefined-behavior-in-binutils-causes-segfault.html) 
suggests that the NULL check can be optimized away because the only case 
that the return value can be NULL triggers pointer overflow, which is 
undefined.

Please let me know if it makes sense or not. Thanks for your time and I 
am looking forward to your reply.

Best,
Shaobo

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2017-02-22 19:54 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <002201d288a9$93dd7360$bb985a20$@cs.utah.edu>
2017-02-17 10:26 ` Dead code in v4l2-mem2mem.c? Laurent Pinchart
2017-02-17 18:42   ` Shaobo
2017-02-18 10:53     ` Laurent Pinchart
2017-02-20 19:49       ` Shaobo
2017-02-22 19:54         ` Laurent Pinchart
2017-02-17  3:47 Shaobo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox