Re: Bug#605009: serious performance regression with ext4

[Jonathan Nieder <jrnieder@gmail.com>]

Hi,

Ted Ts'o wrote:

> I did some experimenting, and I figured out what was going on.  You're
> right, (c) doesn't quite work, because delayed allocation meant that
> the writeout didn't take place until the fsync() for each file
> happened.  I didn't see this at first; my apologies.

Thanks for a clear analysis[1].  I am still confused about something,
though.  If the answer is "stop wasting my time, just read the source", I
can accept that.

>                        sync_file_range() is a Linux specific system
> call that has been around for a while.  It allows program to control
> when writeback happens in a very low-level fashion.  The first set of
> sync_file_range() system calls causes the system to start writing back
> each file once it has finished being extracted.  It doesn't actually
> wait for the write to finish; it just starts the writeback.

True, using sync_file_range(..., SYNC_FILE_RANGE_WRITE) for each file
makes later fsync() much faster.  But why?  Is this a matter of allowing
writeback to overlap with write() or is something else going on?

I'm thinking it has to be something else, since sync() is fast without the
sync_file_range().

> I've attached the program I used to test and prove this mechanism, as
> well as the kernel tracepoint script I used to debug why (c) wasn't
> working, which might be of interest to folks on debian-kernel.
> Basically it's a demonstration of how cool ftrace is.  :-)

Perhaps the answer can be phrased in terms of the output of this script.

> #!/bin/sh
> cd /sys/kernel/debug/tracing
> echo blk > current_tracer
> echo 1 > /sys/block/dm-5/trace/enable
> echo 1 > events/ext4/ext4_sync_file/enable
> echo 1 > events/ext4/ext4_da_writepages/enable
> echo 1 > events/ext4/ext4_mark_inode_dirty/enable
> echo 1 > events/jbd2/jbd2_run_stats/enable
> echo 1 > events/jbd2/jbd2_start_commit/enable
> echo 1 > events/jbd2/jbd2_end_commit/enable
> (cd /kbuild; /home/tytso/src/mass-sync-tester -n)
> cat trace > /tmp/trace
> echo 0 > events/jbd2/jbd2_start_commit/enable
> echo 0 > events/jbd2/jbd2_end_commit/enable
> echo 0 > events/jbd2/jbd2_run_stats/enable
> echo 0 > events/ext4/ext4_sync_file/enable
> echo 0 > events/ext4/ext4_da_writepages/enable
> echo 0 > events/ext4/ext4_mark_inode_dirty/enable
> echo 0 > /sys/block/dm-5/trace/enable
> echo nop > current_tracer

Jonathan

[1] http://lists.debian.org/debian-devel/2010/11/msg00577.html