Re: Fio benchmark output "error=Too many open files

[Jens Axboe <axboe@kernel.dk>]

On 2011-09-20 12:06, yan li wrote:
> Hi fio users and developers,
>     When I run fio benchmark, there are some error in the screen.
>     Does someone meet the same circumstances ?
>     Thanks, ^_^
> 
> #fio fio.cfg
> ......
> fio: pid=29128, err=24/file:filesetup.c:529,
> func=open(/mnt/rw1/rw1.168.2), error=Too many open files
> fio: pid=29039, err=24/file:filesetup.c:529, func=open(/mnt/rw.79.2),
> error=Too many open files
> fio: pid=29040, err=24/file:filesetup.c:529, func=open(/mnt/rw.80.2),
> error=Too many open files
> fio: pid=29074, err=24/file:filesetup.c:529, func=open(/mnt/rw.114.2),
> error=Too many open files
> fio: pid=29026, err=24/file:filesetup.c:529, func=open(/mnt/rw.66.2),
> error=Too many open files
> ......

Fio keeps all of its data files open, and from the jobs below, you have

3*8*128 = 3072

in data files alone, add some house keeping, and perhaps you hit the
limit? Seems a bit odd, given that the typical max is somewhat larger,
and your selinux settings allow higher as well. What do

/proc/sys/fs/file-{nr,max}

show when this happens? You could also freeze fio at this point and ls
-l /proc/pid-of-fio/fd/ to see if something strange is being held open.

-- 
Jens Axboe