Re: Still seeing high autodefrag IO with kernel 5.16.5

[Benjamin Xiao <ben.r.xiao@gmail.com>]

Okay, I just compiled a custom Arch kernel with your patches applied. 
Will test soon. Besides enabling autodefrag and redownloading a game 
from Steam, what other sorts of tests should I do?

Ben

On Fri, Feb 4 2022 at 09:54:19 AM +0800, Qu Wenruo 
<quwenruo.btrfs@gmx.com> wrote:
> 
> 
> On 2022/2/4 09:17, Qu Wenruo wrote:
>> 
>> 
>> On 2022/2/4 04:05, Benjamin Xiao wrote:
>>> Hello all,
>>> 
>>> Even after the defrag patches that landed in 5.16.5, I am still 
>>> seeing
>>> lots of cpu usage and disk writes to my SSD when autodefrag is 
>>> enabled.
>>> I kinda expected slightly more IO during writes compared to 5.15, 
>>> but
>>> what I am actually seeing is massive amounts of btrfs-cleaner i/o 
>>> even
>>> when no programs are actively writing to the disk.
>>> 
>>> I can reproduce it quite reliably on my 2TB Btrfs Steam library
>>> partition. In my case, I was downloading Strange Brigade, which is a
>>> roughly 25GB download and 33.65GB on disk. Somewhere during the
>>> download, iostat will start reporting disk writes around 300 MB/s, 
>>> even
>>> though Steam itself reports disk usage of 40-45MB/s. After the 
>>> download
>>> finishes and nothing else is being written to disk, I still see 
>>> around
>>> 90-150MB/s worth of disk writes. Checking in iotop, I can see btrfs
>>> cleaner and other btrfs processes writing a lot of data.
>>> 
>>> I left it running for a while to see if it was just some maintenance
>>> tasks that Btrfs needed to do, but it just kept going. I tried to
>>> reboot, but it actually prevented me from properly rebooting. After
>>> systemd timed out, my system finally shutdown.
>>> 
>>> Here are my mount options:
>>> rw,relatime,compress-force=zstd:2,ssd,autodefrag,space_cache=v2,subvolid=5,subvol=/
>>> 
>> 
>> Compression, I guess that's the reason.
>> 
>>  From the very beginning, btrfs defrag doesn't handle compressed 
>> extent
>> well.
>> 
>> Even if a compressed extent is already at its maximum capacity, btrfs
>> will still try to defrag it.
>> 
>> I believe the behavior is masked by other problems in older kernels 
>> thus
>> not that obvious.
>> 
>> But after rework of defrag in v5.16, this behavior is more exposed.
> 
> And if possible, please try this diff on v5.15.x, and see if v5.15 is
> really doing less IO than v5.16.x.
> 
> The diff will solve a problem in the old code, where autodefrag is
> almost not working.
> 
> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> index cc61813213d8..f6f2468d4883 100644
> --- a/fs/btrfs/ioctl.c
> +++ b/fs/btrfs/ioctl.c
> @@ -1524,13 +1524,8 @@ int btrfs_defrag_file(struct inode *inode, 
> struct
> file *file,
>                         continue;
>                 }
> 
> -               if (!newer_than) {
> -                       cluster = (PAGE_ALIGN(defrag_end) >>
> -                                  PAGE_SHIFT) - i;
> -                       cluster = min(cluster, max_cluster);
> -               } else {
> -                       cluster = max_cluster;
> -               }
> +               cluster = (PAGE_ALIGN(defrag_end) >> PAGE_SHIFT) - i;
> +               cluster = min(cluster, max_cluster);
> 
>                 if (i + cluster > ra_index) {
>                         ra_index = max(i, ra_index);
> 
>> 
>> There are patches to address the compression related problem, but not
>> yet merged:
>> 
>> https://patchwork.kernel.org/project/linux-btrfs/list/?series=609387
>> 
>> Mind to test them to see if that's the case?
>> 
>> Thanks,
>> Qu
>>> 
>>> 
>>> I've disabled autodefrag again for now to save my SSD, but just 
>>> wanted
>>> to say that there is still an issue. Have the defrag issues been 
>>> fully
>>> fixed or are there more patches incoming despite what Reddit and
>>> Phoronix say? XD
>>> 
>>> Thanks!
>>> Ben
>>> 
>>>