public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Dave Chinner <david@fromorbit.com>
To: Brian Foster <bfoster@redhat.com>
Cc: linux-xfs@vger.kernel.org, Dave Chinner <dchinner@redhat.com>
Subject: Re: [BUG] XFS (delalloc) writeback livelock writing to -ENOSPC on dm-thin
Date: Sat, 22 Apr 2023 08:27:25 +1000	[thread overview]
Message-ID: <20230421222725.GG3223426@dread.disaster.area> (raw)
In-Reply-To: <ZEJtT7vJ9RA4pno4@bfoster>

On Fri, Apr 21, 2023 at 07:02:39AM -0400, Brian Foster wrote:
> Hi all,
> 
> The test case is a simple sequential write to XFS backed by a thin
> volume. The test vm is running latest 6.3.0-rc7, has 8xcpu and 8GB RAM,
> and the thin volume is backed by sufficient space in the thin pool.
> I.e.:
> 
> lvcreate --type thin-pool -n tpool -L30G test
> lvcreate -V 20G -n tvol test/tpool
> mkfs.xfs /dev/test/tvol
> mount /dev/test/tvol /mnt
> dd if=/dev/zero of=/mnt/file bs=1M
> 
> The dd command writes until ~1GB or so free space is left in the fs and
> then seems to hit a livelock. From a quick look at tracepoints, XFS
> seems to be spinning in the xfs_convert_blocks() writeback path. df
> shows space consumption no longer changing, the flush worker is spinning
> at 100% and dd is blocked in balance_dirty_pages(). If I kill dd, the
> writeback worker continues spinning and an fsync of the file blocks
> indefinitely.
> 
> If I reset the vm, remount and run the following:
> 
> dd if=/dev/zero of=/mnt/file bs=1M conv=notrunc oflag=append
> 
> ... it then runs to -ENOSPC, as expected.
> 
> I haven't seen this occur when running on a non-thin lvm volume, not
> sure why.

thinp volumes trigger stripe alignment in mkfs, which then use
different allocation strategies at writeback time.

Patch to fix it already sent to the list.

-Dave.
-- 
Dave Chinner
david@fromorbit.com

      reply	other threads:[~2023-04-21 22:27 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-21 11:02 [BUG] XFS (delalloc) writeback livelock writing to -ENOSPC on dm-thin Brian Foster
2023-04-21 22:27 ` Dave Chinner [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230421222725.GG3223426@dread.disaster.area \
    --to=david@fromorbit.com \
    --cc=bfoster@redhat.com \
    --cc=dchinner@redhat.com \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox