Re: Problem with direct IO

[Zhengyuan Liu <liuzhengyuang521@gmail.com>]

On Mon, Oct 25, 2021 at 11:57 PM Jan Kara <jack@suse.cz> wrote:
>
> On Sat 23-10-21 10:06:24, Zhengyuan Liu wrote:
> > On Fri, Oct 22, 2021 at 5:31 PM Jan Kara <jack@suse.cz> wrote:
> > > > > > > Can you post output of "dumpe2fs -h <device>" for the filesystem where the
> > > > > > > problem happens? Thanks!
> > > > > >
> > > > > > Sure, the output is:
> > > > > >
> > > > > > # dumpe2fs -h /dev/sda3
> > > > > > dumpe2fs 1.45.3 (14-Jul-2019)
> > > > > > Filesystem volume name:   <none>
> > > > > > Last mounted on:          /data
> > > > > > Filesystem UUID:          09a51146-b325-48bb-be63-c9df539a90a1
> > > > > > Filesystem magic number:  0xEF53
> > > > > > Filesystem revision #:    1 (dynamic)
> > > > > > Filesystem features:      has_journal ext_attr resize_inode dir_index
> > > > > > filetype needs_recovery sparse_super large_file
> > > > >
> > > > > Thanks for the data. OK, a filesystem without extents. Does your test by
> > > > > any chance try to do direct IO to a hole in a file? Because that is not
> > > > > (and never was) supported without extents. Also the fact that you don't see
> > > > > the problem with ext4 (which means extents support) would be pointing in
> > > > > that direction.
> > > >
> > > > I am not sure if it trys to do direct IO to a hole or not, is there any
> > > > way to check?  If you have a simple test to reproduce please let me know,
> > > > we are glad to try.
> > >
> > > Can you enable following tracing?
> >
> > Sure, but let's confirm before doing that, it seems Ext4 doesn't
> > support iomap in
> > V4.19 which could also reproduce the problem, so if it is necessary to
> > do the following
> > tracing? or should we modify the tracing if under V4.19?
>
> Well, iomap is just a different generic framework for doing direct IO. The
> fact that you can observe the problem both with iomap and the old direct IO
> framework is one of the reasons why I think the problem is actually that
> the file has holes (unallocated space in the middle).
>
> > > echo 1 >/sys/kernel/debug/tracing/events/ext4/ext4_ind_map_blocks_exit/enable
> > > echo iomap_dio_rw >/sys/kernel/debug/tracing/set_ftrace_filter
> > > echo "function_graph" >/sys/kernel/debug/tracing/current_tracer
>
> If you want to trace a kernel were ext4 direct IO path is not yet
> converted to iomap framework you need to replace tracing of iomap_dio_rw
> with:
>
> echo __blockdev_direct_IO >/sys/kernel/debug/tracing/set_ftrace_filter

Sorry for the late reply, because it is hard to reproduce when enabling ftrace.
I'll post here right after getting useful data.

> > > And then gather output from /sys/kernel/debug/tracing/trace_pipe. Once the
> > > problem reproduces, you can gather the problematic file name from dmesg, find
> > > inode number from "stat <filename>" and provide that all to me? Thanks!
>