Re: [PATCH] Check vmalloc return value in vpe_open

[Ralf Baechle <ralf@linux-mips.org>]

On Sat, Oct 30, 2010 at 06:37:16PM +0200, Jesper Juhl wrote:

> I noticed that the return value of the 
> vmalloc() call in arch/mips/kernel/vpe.c::vpe_open() is not checked, so we 
> potentially store a null pointer in v->pbuffer. As far as I can tell this 
> will be a problem. However, I don't know the mips code at all, so there 
> may be something, somewhere where I did not look, that handles this in a 
> safe manner but I couldn't find it.
> 
> To me it looks like we should do what the patch below implements and check 
> for a null return and then return -ENOMEM in that case. Comments?

All users check if the buffer was successfully allocated so the code is
safe wrt. to that.

Doesn't mean that it's not a pukeogenic piece of code.  Look at the use of
v->pbuffer in vpe_release for example.  First use it the vmalloc'ed memory
then carefully check the pointer for being non-NULL before calling vfree.
If the pointer could actually be non-NULL that's too late and vfree does
that check itself anyway.  And more such gems, general uglyness and
freedom of concept.  It used to be even worse.

  Ralf