Re: [PATCH] generic/558: avoid forkbombs on filesystems with many free inodes

["Darrick J. Wong" <djwong@kernel.org>]

On Fri, Jul 14, 2023 at 07:59:00AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Mikulas reported that this test became a forkbomb on his system when he
> tested it with bcachefs.  Unlike XFS and ext4, which have large inodes
> consuming hundreds of bytes, bcachefs has very tiny ones.  Therefore, it
> reports a large number of free inodes on a freshly mounted 1GB fs (~15
> million), which causes this test to try to create 15000 processes.
> 
> There's really no reason to do that -- all this test wanted to do was to
> exhaust the number of inodes as quickly as possible using all available
> CPUs, and then it ran xfs_repair to try to reproduce a bug.  Set the
> number of subshells to 4x the CPU count and spread the work among them
> instead of forking thousands of processes.
> 
> Reported-by: Mikulas Patocka <mpatocka@redhat.com>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> Tested-by: Mikulas Patocka <mpatocka@redhat.com>
> ---
>  tests/generic/558 |   18 ++++++++++++------
>  1 file changed, 12 insertions(+), 6 deletions(-)
> 
> diff --git a/tests/generic/558 b/tests/generic/558
> index 4e22ce656b..de5c28d00d 100755
> --- a/tests/generic/558
> +++ b/tests/generic/558
> @@ -39,15 +39,21 @@ _scratch_mkfs_sized $((1024 * 1024 * 1024)) >>$seqres.full 2>&1
>  _scratch_mount
>  
>  i=0
> -free_inode=`_get_free_inode $SCRATCH_MNT`
> -file_per_dir=1000
> -loop=$((free_inode / file_per_dir + 1))

NAK.  Here, the old code effectively does:

loop = howmany(free_inode, 1000);
for i in 0...loop:
	create_file ... 1000files... &

IOWs, it rounds the number of files to create up to the nearest 1000,
which I overlooked because I was overloaded and words are easier than
resurrecting mathematical concepts from raw formulae.

If, say, the 1G fs claims to have 524,288 free inodes, the test will
start *525* create_file subshells to create 1000 files each, or 525,000
files.

The /new/ code does this instead:

nr_cpus=(cpu count * 4)
files_per_dir = howmany(free_inodes, nr_cpus)
for i in 0..nr_cpus:
	create_file ... files_per_dir... &

If nr_cpu is a factor of free_inodes, we don't do /any/ roundup at all.
524,288 free inodes with 4 CPUs gets you 16 threads and 32768 files per
thread.

Apparently this is significant somehow, because on a lark I decided to
revert the referenced commit and the new code doesn't reliably
reproduce the failure when parent pointers are enabled.

Reintroducing the "rounding free_inodes up to the nearest 1000" does
make it trip, though.

Sooooo... I'll have a new version out tomorrow after some testing.
Please do not apply this patch until then, unless you are testing
bcachefs.

--D

> +free_inodes=$(_get_free_inode $SCRATCH_MNT)
> +nr_cpus=$(( $($here/src/feature -o) * 4 * LOAD_FACTOR ))
> +echo "free inodes: $free_inodes nr_cpus: $nr_cpus" >> $seqres.full
> +
> +if ((free_inodes <= nr_cpus)); then
> +	nr_cpus=1
> +	files_per_dir=$free_inodes
> +else
> +	files_per_dir=$(( (free_inodes + nr_cpus - 1) / nr_cpus ))
> +fi
>  mkdir -p $SCRATCH_MNT/testdir
>  
>  echo "Create $((loop * file_per_dir)) files in $SCRATCH_MNT/testdir" >>$seqres.full
> -while [ $i -lt $loop ]; do
> -	create_file $SCRATCH_MNT/testdir $file_per_dir $i >>$seqres.full 2>&1 &
> -	let i=$i+1
> +for ((i = 0; i < nr_cpus; i++)); do
> +	create_file $SCRATCH_MNT/testdir $files_per_dir $i >>$seqres.full 2>&1 &
>  done
>  wait
>