Re: fstrim is takes a long time on Btrfs and NVMe

Nikolay Borisov
2019-12-22T10:40:51Z

On 21.12.19 г. 8:24 ч., Chris Murphy wrote:
> Hi,
>
> Recent kernels, I think since 5.1 or 5.2, but tested today on 5.3.18,
> 5.4.5, 5.5.0rc2, takes quite a long time for `fstrim /` to complete,
> just over 1 minute.
>
> Filesystem Size Used Avail Use% Mounted on
> /dev/nvme0n1p7 178G 16G 161G 9% /
>
> fstrim stops on this for pretty much the entire time:
> ioctl(3, FITRIM, {start=0, len=0xffffffffffffffff, minlen=0}) = 0
>
> top shows the fstrim process itself isn't consuming much CPU, about
> 2-3%. Top five items in per top, not much more revealing.
>
> Samples: 220K of event 'cycles', 4000 Hz, Event count (approx.):
> 3463316966 lost: 0/0 drop: 0/0
> Overhead Shared Object Symbol
> 1.62% [kernel] [k] find_next_zero_bit
> 1.59% perf [.] 0x00000000002ae063
> 1.52% [kernel] [k] psi_task_change
> 1.41% [kernel] [k] update_blocked_averages
> 1.33% [unknown] [.] 0000000000000000
>
> On a different system, with older Samsung 840 SATA SSD, and a fresh
> Btrfs, I can't reproduce. It takes less than 1s. Not sure how to get
> more information.
>
>

Ok, indeed your device seems to be taking a long time to do discards,
perhahps it's not using NCQ. OTOH btrfs currently issues synchronous
discards since it's using: blkdev_issue_discard. So naturally every
discard request is blocking.

The solution would be to rework how discard in btrfs works by utilising
the asynchronous discard interface __blkdev_issue_discard et al.
However, that would need a more careful analysis since freespace which
could be waiting to be discarded might be allocated. So for correctness
reasons some sort of synchronization would need to be devised.