Re: [PATCH v2] fs: use kmalloc() to allocate fdmem if possible

[Andrew Morton <akpm@linux-foundation.org>]

On Mon,  3 May 2010 03:02:00 +0800
Changli Gao <xiaosuo@gmail.com> wrote:

> use kmalloc() to allocate fdmem if possible.
> 
> vmalloc() is used as a fallback solution for fdmem allocation. A new helper
> function __free_fdtable() is introduced to reduce the lines of code.
> 
> A potential bug, vfree() a memory allocated by kmalloc(), is fixed.
> 

Seems a reasonable thing to do.  It might also be reasonable to make
vmalloc() try kmalloc() first, but that's a separate exercise.

> ----
>  fs/file.c |   55 ++++++++++++++++++++++---------------------------------
>  1 file changed, 22 insertions(+), 33 deletions(-)
> diff --git a/fs/file.c b/fs/file.c
> index 34bb7f7..9af31ae 100644
> --- a/fs/file.c
> +++ b/fs/file.c
> @@ -39,28 +39,27 @@ int sysctl_nr_open_max = 1024 * 1024; /* raised later */
>   */
>  static DEFINE_PER_CPU(struct fdtable_defer, fdtable_defer_list);
>  
> -static inline void * alloc_fdmem(unsigned int size)
> +static inline void *alloc_fdmem(unsigned int size)
>  {
> -	if (size <= PAGE_SIZE)
> -		return kmalloc(size, GFP_KERNEL);
> -	else
> -		return vmalloc(size);
> +	void *data;
> +
> +	data = kmalloc(size, GFP_KERNEL);

This most definitely should have __GFP_NOWARN.

> +	if (data != NULL)
> +		return data;
> +
> +	return vmalloc(size);
>  }
>  
> -static inline void free_fdarr(struct fdtable *fdt)
> +static inline void free_fdmem(void *ptr)
>  {
> -	if (fdt->max_fds <= (PAGE_SIZE / sizeof(struct file *)))
> -		kfree(fdt->fd);
> -	else
> -		vfree(fdt->fd);
> +	is_vmalloc_addr(ptr) ? vfree(ptr) : kfree(ptr);
>  }
>  
> -static inline void free_fdset(struct fdtable *fdt)
> +static inline void __free_fdtable(struct fdtable *fdt)
>  {
> -	if (fdt->max_fds <= (PAGE_SIZE * BITS_PER_BYTE / 2))
> -		kfree(fdt->open_fds);
> -	else
> -		vfree(fdt->open_fds);
> +	free_fdmem(fdt->fd);
> +	free_fdmem(fdt->open_fds);
> +	kfree(fdt);
>  }

And these should be uninlined - they're too large to be inlined.

My version of gcc seems to just uninline them anyway, but forcing them
to be inlined with __always_inline indeed causes 70-80 bytes more text,
and we figure that larger text generally causes a slower kernel due to
cache eviction effects.