From: Mike Snitzer <snitzer@kernel.org>
To: Kirill Kirilenko <kirill@ultracoder.org>
Cc: linux-raid@vger.kernel.org, heinzm@redhat.com,
dm-devel@redhat.com, Alasdair Kergon <agk@redhat.com>
Subject: Re: [dm-devel] fstrim on raid1 LV with writemostly PV leads to system freeze
Date: Thu, 21 Sep 2023 17:45:24 -0400 [thread overview]
Message-ID: <ZQy5dClooWaZoS/N@redhat.com> (raw)
In-Reply-To: <0e15b760-2d5f-f639-0fc7-eed67f8c385c@ultracoder.org>
[cc'ing Heinz and the linux-raid mailing list]
On Thu, Sep 21 2023 at 4:34P -0400,
Kirill Kirilenko <kirill@ultracoder.org> wrote:
> Hello.
>
> I created two LVM physical volumes: one on NVMe device and one on SATA SSD.
> I added them to a volume group and created a logical RAID1 volume in it.
> Then I enabled 'writemostly' flag on the second (slowest) PV.
> And my system started to freeze at random times with no messages in syslog.
> I was able to determine that the freezing was happening during execution of
> 'fstrim' (via systemd timer). I checked this by running 'fstrim' manually.
> If I disable the 'writemostly' flag, I experience no freezes. I can
> reproduce this behavior on vanilla 6.5.0 kernel.
>
> My LV is 150 GB ext4 volume, and it has lots of files in it, so running
> 'fstrim' takes around a minute. This may be important.
>
> Additional information:
> OS: Linux Mint 21.2
> CPU: AMD Ryzen 7 5800X
> NVMe: Samsung SSD 980 500GB
> SATA SSD: Samsung SSD 850 EVO M.2 250GB
>
> Best regards,
> Kirill Kirilenko.
>
I just verified that 6.5.0 does have this DM core fix (needed to
prevent excessive splitting of discard IO.. which could cause fstrim
to take longer for a DM device), but again 6.5.0 has this fix so it
isn't relevant:
be04c14a1bd2 dm: use op specific max_sectors when splitting abnormal io
Given your use of 'writemostly' I'm inferring you're using lvm2's
raid1 that uses MD raid1 code in terms of the dm-raid target.
Discards (more generic term for fstrim) are considered writes, so
writemostly really shouldn't matter... but I know that there have been
issues with MD's writemostly code (identified by others relatively
recently).
All said: hopefully someone more MD oriented can review your report
and help you further.
Mike
--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel
WARNING: multiple messages have this Message-ID (diff)
From: Mike Snitzer <snitzer@kernel.org>
To: Kirill Kirilenko <kirill@ultracoder.org>
Cc: Alasdair Kergon <agk@redhat.com>,
dm-devel@redhat.com, heinzm@redhat.com,
linux-raid@vger.kernel.org
Subject: Re: fstrim on raid1 LV with writemostly PV leads to system freeze
Date: Thu, 21 Sep 2023 17:45:24 -0400 [thread overview]
Message-ID: <ZQy5dClooWaZoS/N@redhat.com> (raw)
In-Reply-To: <0e15b760-2d5f-f639-0fc7-eed67f8c385c@ultracoder.org>
[cc'ing Heinz and the linux-raid mailing list]
On Thu, Sep 21 2023 at 4:34P -0400,
Kirill Kirilenko <kirill@ultracoder.org> wrote:
> Hello.
>
> I created two LVM physical volumes: one on NVMe device and one on SATA SSD.
> I added them to a volume group and created a logical RAID1 volume in it.
> Then I enabled 'writemostly' flag on the second (slowest) PV.
> And my system started to freeze at random times with no messages in syslog.
> I was able to determine that the freezing was happening during execution of
> 'fstrim' (via systemd timer). I checked this by running 'fstrim' manually.
> If I disable the 'writemostly' flag, I experience no freezes. I can
> reproduce this behavior on vanilla 6.5.0 kernel.
>
> My LV is 150 GB ext4 volume, and it has lots of files in it, so running
> 'fstrim' takes around a minute. This may be important.
>
> Additional information:
> OS: Linux Mint 21.2
> CPU: AMD Ryzen 7 5800X
> NVMe: Samsung SSD 980 500GB
> SATA SSD: Samsung SSD 850 EVO M.2 250GB
>
> Best regards,
> Kirill Kirilenko.
>
I just verified that 6.5.0 does have this DM core fix (needed to
prevent excessive splitting of discard IO.. which could cause fstrim
to take longer for a DM device), but again 6.5.0 has this fix so it
isn't relevant:
be04c14a1bd2 dm: use op specific max_sectors when splitting abnormal io
Given your use of 'writemostly' I'm inferring you're using lvm2's
raid1 that uses MD raid1 code in terms of the dm-raid target.
Discards (more generic term for fstrim) are considered writes, so
writemostly really shouldn't matter... but I know that there have been
issues with MD's writemostly code (identified by others relatively
recently).
All said: hopefully someone more MD oriented can review your report
and help you further.
Mike
next prev parent reply other threads:[~2023-09-21 21:49 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-21 20:34 [dm-devel] fstrim on raid1 LV with writemostly PV leads to system freeze Kirill Kirilenko
2023-09-21 21:45 ` Mike Snitzer [this message]
2023-09-21 21:45 ` Mike Snitzer
2023-09-21 22:03 ` [dm-devel] " Roman Mamedov
2023-09-21 22:03 ` Roman Mamedov
2023-09-22 16:16 ` [dm-devel] " Kirill Kirilenko
2023-09-22 16:16 ` Kirill Kirilenko
2023-09-22 23:08 ` [dm-devel] " Song Liu
2023-09-22 23:08 ` Song Liu
2023-09-25 2:58 ` [dm-devel] " Yu Kuai
2023-09-25 2:58 ` Yu Kuai
2023-09-25 23:59 ` [dm-devel] " Kirill Kirilenko
2023-09-25 23:59 ` Kirill Kirilenko
2023-09-26 3:28 ` [dm-devel] " Yu Kuai
2023-09-26 3:28 ` Yu Kuai
[not found] ` <a4d3f9b0-15d5-4a90-f2c1-cad633badbbf@ultracoder.org>
2023-09-26 13:21 ` [dm-devel] " Yu Kuai
2023-09-26 13:21 ` Yu Kuai
2023-09-26 20:27 ` [dm-devel] " Kirill Kirilenko
2023-09-26 20:27 ` Kirill Kirilenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZQy5dClooWaZoS/N@redhat.com \
--to=snitzer@kernel.org \
--cc=agk@redhat.com \
--cc=dm-devel@redhat.com \
--cc=heinzm@redhat.com \
--cc=kirill@ultracoder.org \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.