public inbox for linux-btrfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Qu Wenruo <wqu@suse.com>
To: linux-btrfs@vger.kernel.org
Subject: [PATCH 0/3] btrfs: reduce the memory usage for replace in btrfs_io_context.
Date: Sat, 28 Jan 2023 16:23:13 +0800	[thread overview]
Message-ID: <cover.1674893735.git.wqu@suse.com> (raw)

In btrfs_io_context, we have two members dedicated for dev-replace:

- num_tgtdevs
  This is straight-forward, just the number of extra stripes for replace
  usage.

- tgtdev_map[]
  This is a little complex, it represents the mapping between the
  original stripes and dev-replace stripes.

  This is mostly for RAID56, as only in RAID56 the stripes contain
  different contents, thus it's important to know the mapping.

  It goes like this:

    num_stripes = 4 (3 + 1 for replace)
    stripes[0]:		dev = devid 1, physical = X
    stripes[1]:		dev = devid 2, physical = Y
    stripes[2]:		dev = devid 3, physical = Z
    stripes[3]:		dev = devid 0, physical = Y

    num_tgtdevs = 1
    tgtdev_map[0] = 0	<- Means stripes[0] is not involved in replace.
    tgtdev_map[1] = 3	<- Means stripes[1] is involved in replace,
			   and it's duplicated to stripes[3].
    tgtdev_map[2] = 0	<- Means stripes[2] is not involved in replace.

  Thus most space is wasted, and the more devices in the array, the more
  space wasted.


For the current tgtdev_map[] design, it's wasting quite some space.
E.g. in the above case, we only need on slot to record the source stripe
number, and the other two slots are just a waste of space.

The existing tgtdev_map[] will make more sense if we support multiple
running dev-replaces, but that's not the case.

So this patch would mostly change it to a new, and more space efficient
way, by going something like this for the same example:

  replace_nr_stripes = 1
  tgtdev_map[0] = 1	<- Means stripes[1] is involved in replace.
  tgtdev_map[1] = -1	<- Means the second slot is not used.
		 	   (Only DUP can use this slot, but they
			    don't really care)

Furthermore we reduce the width of nr_stripes related member to u16, the
same as on-disk format width.

This not only saved some space for btrfs_io_context structure, but also
allows the following cleanups:

- Streamline handle_ops_on_dev_replace()
  We go a common path for both WRITE and GET_READ_MIRRORS, and only
  for DUP and GET_READ_MIRRORS, we shrink the bioc to keep the same
  old behavior.

- Remove some unnecessary variables

Although the series still increases the number of lines, the net
increase mostly comes from comments, in fact around 70 lines of comments
are added around the replace related members.


Qu Wenruo (3):
  btrfs: simplify the @bioc argument for handle_ops_on_dev_replace()
  btrfs: small improvement for btrfs_io_context structure
  btrfs: use a more space efficient way to represent the source of
    duplicated stripes

 fs/btrfs/raid56.c  |  44 +++++++++--
 fs/btrfs/scrub.c   |   4 +-
 fs/btrfs/volumes.c | 187 +++++++++++++++++++++------------------------
 fs/btrfs/volumes.h |  52 +++++++++++--
 4 files changed, 174 insertions(+), 113 deletions(-)

-- 
2.39.1


             reply	other threads:[~2023-01-28  8:23 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-28  8:23 Qu Wenruo [this message]
2023-01-28  8:23 ` [PATCH 1/3] btrfs: simplify the @bioc argument for handle_ops_on_dev_replace() Qu Wenruo
2023-01-30  7:17   ` Johannes Thumshirn
2023-01-31 11:54   ` Anand Jain
2023-01-28  8:23 ` [PATCH 2/3] btrfs: small improvement for btrfs_io_context structure Qu Wenruo
2023-01-30  7:27   ` Johannes Thumshirn
2023-01-30  7:31     ` Qu Wenruo
2023-02-02  6:28   ` Anand Jain
2023-02-02  6:47     ` Qu Wenruo
2023-02-02  9:12       ` Anand Jain
2023-02-07  2:19         ` Qu Wenruo
2023-02-03  6:19       ` Christoph Hellwig
2023-02-03  6:44         ` Qu Wenruo
2023-01-28  8:23 ` [PATCH 3/3] btrfs: use a more space efficient way to represent the source of duplicated stripes Qu Wenruo
2023-01-30  7:40   ` Johannes Thumshirn

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1674893735.git.wqu@suse.com \
    --to=wqu@suse.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox