From: Dave Chinner <david@fromorbit.com>
To: Brian Foster <bfoster@redhat.com>
Cc: linux-xfs@vger.kernel.org, Dave Chinner <dchinner@redhat.com>
Subject: Re: [BUG] XFS (delalloc) writeback livelock writing to -ENOSPC on dm-thin
Date: Sat, 22 Apr 2023 08:27:25 +1000 [thread overview]
Message-ID: <20230421222725.GG3223426@dread.disaster.area> (raw)
In-Reply-To: <ZEJtT7vJ9RA4pno4@bfoster>
On Fri, Apr 21, 2023 at 07:02:39AM -0400, Brian Foster wrote:
> Hi all,
>
> The test case is a simple sequential write to XFS backed by a thin
> volume. The test vm is running latest 6.3.0-rc7, has 8xcpu and 8GB RAM,
> and the thin volume is backed by sufficient space in the thin pool.
> I.e.:
>
> lvcreate --type thin-pool -n tpool -L30G test
> lvcreate -V 20G -n tvol test/tpool
> mkfs.xfs /dev/test/tvol
> mount /dev/test/tvol /mnt
> dd if=/dev/zero of=/mnt/file bs=1M
>
> The dd command writes until ~1GB or so free space is left in the fs and
> then seems to hit a livelock. From a quick look at tracepoints, XFS
> seems to be spinning in the xfs_convert_blocks() writeback path. df
> shows space consumption no longer changing, the flush worker is spinning
> at 100% and dd is blocked in balance_dirty_pages(). If I kill dd, the
> writeback worker continues spinning and an fsync of the file blocks
> indefinitely.
>
> If I reset the vm, remount and run the following:
>
> dd if=/dev/zero of=/mnt/file bs=1M conv=notrunc oflag=append
>
> ... it then runs to -ENOSPC, as expected.
>
> I haven't seen this occur when running on a non-thin lvm volume, not
> sure why.
thinp volumes trigger stripe alignment in mkfs, which then use
different allocation strategies at writeback time.
Patch to fix it already sent to the list.
-Dave.
--
Dave Chinner
david@fromorbit.com
prev parent reply other threads:[~2023-04-21 22:27 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-21 11:02 [BUG] XFS (delalloc) writeback livelock writing to -ENOSPC on dm-thin Brian Foster
2023-04-21 22:27 ` Dave Chinner [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230421222725.GG3223426@dread.disaster.area \
--to=david@fromorbit.com \
--cc=bfoster@redhat.com \
--cc=dchinner@redhat.com \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox