Re: fstrim is takes a long time on Btrfs and NVMe

[Nikolay Borisov <nborisov@suse.com>]

On 21.12.19 г. 8:24 ч., Chris Murphy wrote:
> Hi,
> 
> Recent kernels, I think since 5.1 or 5.2, but tested today on 5.3.18,
> 5.4.5, 5.5.0rc2, takes quite a long time for `fstrim /` to complete,
> just over 1 minute.
> 
> Filesystem      Size  Used Avail Use% Mounted on
> /dev/nvme0n1p7  178G   16G  161G   9% /
> 
> fstrim stops on this for pretty much the entire time:
> ioctl(3, FITRIM, {start=0, len=0xffffffffffffffff, minlen=0}) = 0
> 
> top shows the fstrim process itself isn't consuming much CPU, about
> 2-3%. Top five items in per top, not much more revealing.
> 
> Samples: 220K of event 'cycles', 4000 Hz, Event count (approx.):
> 3463316966 lost: 0/0 drop: 0/0
> Overhead  Shared Object                    Symbol
>    1.62%  [kernel]                         [k] find_next_zero_bit
>    1.59%  perf                             [.] 0x00000000002ae063
>    1.52%  [kernel]                         [k] psi_task_change
>    1.41%  [kernel]                         [k] update_blocked_averages
>    1.33%  [unknown]                        [.] 0000000000000000
> 
> On a different system, with older Samsung 840 SATA SSD, and a fresh
> Btrfs, I can't reproduce. It takes less than 1s. Not sure how to get
> more information.
> 
> 


Ok, indeed your device seems to be taking a long time to do discards,
perhahps it's not using NCQ. OTOH btrfs currently issues synchronous
discards since it's using: blkdev_issue_discard. So naturally every
discard request is blocking.

The solution would be to rework how discard in btrfs works by utilising
the asynchronous discard interface __blkdev_issue_discard et al.
However, that would need a more careful analysis since freespace which
could be waiting to be discarded might be allocated. So for correctness
reasons some sort of synchronization would need to be devised.