Re: Btrfs autodefrag wrote 5TB in one day to a 0.5TB SSD without a measurable benefit

[Zygo Blaxell <ce3g8jdj@umail.furryterror.org>]

On Tue, Mar 08, 2022 at 10:57:51PM +0100, Jan Ziak wrote:
> On Mon, Mar 7, 2022 at 3:39 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> > On 2022/3/7 10:23, Jan Ziak wrote:
> > > BTW: "compsize file-with-million-extents" finishes in 0.2 seconds
> > > (uses BTRFS_IOC_TREE_SEARCH_V2 ioctl), but "filefrag
> > > file-with-million-extents" doesn't finish even after several minutes
> > > of time (uses FS_IOC_FIEMAP ioctl - manages to perform only about 5
> > > ioctl syscalls per second - and appears to be slowing down as the
> > > value of the "fm_start" ioctl argument grows; e2fsprogs version
> > > 1.46.5). It would be nice if filefrag was faster than just a few
> > > ioctls per second.
> >
> > This is mostly a race with autodefrag.
> >
> > Both are using file extent map, thus if autodefrag is still trying to
> > redirty the file again and again, it would definitely cause problems for
> > anything also using file extent map.
> 
> It isn't caused by a race with autodefrag, but by something else.
> Autodefrag was turned off when I was running "filefrag
> file-with-million-extents".
> 
> $ /usr/bin/time filefrag file-with-million-extents.sqlite
> Ctrl+C Command terminated by signal 2
> 0.000000 user, 4.327145 system, 0:04.331167 elapsed, 99% CPU

FIEMAP will try to populate the SHARED bit for each extent, which requires
checking every extent that overlaps a block range to see if the block is
present.  It can be very expensive for large, random-written files.

No way to fix that without disabling the SHARED bit in FIEMAP.

> Sincerely
> Jan