soft lockup when leaf and node bigger than 4KB

soft lockup when leaf and node bigger than 4KB

[Zhang Jingwang]

I make a btrfs use 'mkfs.btrfs -m single -l 16384 -n 16384 /dev/xxx'.
After mount it, I run a test script to create lots of files, then soft
lockup occurs.

After digging into the source code, I think there is a problem with
bio->bi_end_io.

when a bio is done, end_io function is called, it does following things:
1: put the bio into async thread waiting for checksum. (end_workqueue_bio)
2: check whether the bio can be checksum-ed. If not, put it back to
wait queue.(end_workqueue_fn)
3: If it can be checksum-ed, call end_bio_extent_readpage().
4: checksum the extent, set_extent_uptodate() and set uptodate flag of
the pages belong to the bio.(end_bio_extent_readpage)

But when checking whether the bio can be checksum-ed in step 2, it
examine the extent_range_uptodate() and uptodate flag of these pages
which is set in step 4. So I think there will be a endless loop here.

When we say a page is uptodate, we means its checksum is correct, but
we must ensure that all of the pages belongs to a btree node is
uptodate in order to calculate its checksum. So it's a logical
paradox.

I am new to btrfs and there may be some mistake about this problem.
Any comment is welcome and thanks for your time!

-- 
Zhang Jingwang
National Research Centre for High Performance Computers
Institute of Computing Technology, Chinese Academy of Sciences
No. 6, South Kexueyuan Road, Haidian District
Beijing, China

Re: soft lockup when leaf and node bigger than 4KB

[Chris Mason]

On Mon, Oct 12, 2009 at 05:55:47PM +0800, Zhang Jingwang wrote:
> I make a btrfs use 'mkfs.btrfs -m single -l 16384 -n 16384 /dev/xxx'.
> After mount it, I run a test script to create lots of files, then soft
> lockup occurs.
> 
> After digging into the source code, I think there is a problem with
> bio->bi_end_io.

Unfortunately, the larger sizes are not supported right now at all, as
there are a number of problems to handle.

In my queue of btrfs-progs updates, I have it limited to the page size
for all of these options.

-chris