All of lore.kernel.org
 help / color / mirror / Atom feed
From: Josef Bacik <josef@toxicpanda.com>
To: linux-btrfs@vger.kernel.org, kernel-team@fb.com
Subject: [PATCH 0/4] btrfs: shrink delalloc fixes
Date: Fri, 11 Jun 2021 09:53:56 -0400	[thread overview]
Message-ID: <cover.1623419155.git.josef@toxicpanda.com> (raw)

Hello,

I backported the patch to switch us to using sync_inode() to our kernel inside
Facebook to fix a deadlock when using the async delalloc shrinker threads.  This
uncovered a bunch of problems with how we shrink delalloc, as we use -o
compress-force, and thus everything goes through the async compression threads.

I ripped out the async pages stuff because originally I had switched us to just
writing the whole inode.  This caused a performance regression, and so I
switched us to calling sync_inode() twice to handle the async extent case.  The
problem is that sync_inode() can skip writing inodes sometimes, and thus we
weren't properly waiting on the async helpers.  We really do need to wait for
the async delalloc pages to go down before continuing to shrink delalloc.  There
was also a race in how we woke up the async delalloc pages waiter which could be
problematic.

And then finally there is our use of sync_inode().  It tries to be too clever
for us, when in reality we want to make sure all pages are under writeback
before we come back to the shrinking loop.  I've added a small helper to give us
this flexibilty and have switched us to that helper.

With these patches, and others that will be sent separately, the early ENOSPC
problems we were experiencing have been eliminated.  Thanks,

Josef Bacik (4):
  btrfs: wait on async extents when flushing delalloc
  btrfs: wake up async_delalloc_pages waiters after submit
  fs: add a filemap_fdatawrite_wbc helper
  btrfs: use the filemap_fdatawrite_wbc helper for delalloc shrinking

 fs/btrfs/inode.c      | 16 ++++++----------
 fs/btrfs/space-info.c | 33 +++++++++++++++++++++++++++++++++
 include/linux/fs.h    |  2 ++
 mm/filemap.c          | 29 ++++++++++++++++++++++++-----
 4 files changed, 65 insertions(+), 15 deletions(-)

-- 
2.26.3


             reply	other threads:[~2021-06-11 13:54 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-11 13:53 Josef Bacik [this message]
2021-06-11 13:53 ` [PATCH 1/4] btrfs: wait on async extents when flushing delalloc Josef Bacik
2021-06-14 10:05   ` Nikolay Borisov
2021-06-14 18:30     ` Josef Bacik
2021-06-11 13:53 ` [PATCH 2/4] btrfs: wake up async_delalloc_pages waiters after submit Josef Bacik
2021-06-14 10:20   ` Nikolay Borisov
2021-06-11 13:53 ` [PATCH 3/4] fs: add a filemap_fdatawrite_wbc helper Josef Bacik
2021-06-14 10:16   ` Nikolay Borisov
2021-06-15 13:37     ` Josef Bacik
2021-06-16  4:41       ` Christoph Hellwig
2021-06-11 13:54 ` [PATCH 4/4] btrfs: use the filemap_fdatawrite_wbc helper for delalloc shrinking Josef Bacik
2021-06-14 10:57   ` Nikolay Borisov
2021-06-14 11:13 ` [PATCH 0/4] btrfs: shrink delalloc fixes Johannes Thumshirn
2021-06-14 18:38   ` Josef Bacik

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1623419155.git.josef@toxicpanda.com \
    --to=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.