Re: [PATCH 1/1] btrfs: Allow replacing device with a smaller one if possible

[David Sterba <dsterba@suse.cz>]

On Mon, Dec 09, 2019 at 10:26:52AM +0000, Filipe Manana wrote:
> 2) A simple solution, but often less efficient: before starting the
> actual replace operation, shrink the source device to the size of the
> target device - just use the existing btrfs_shrink_device(), which
> will relocate chunks beyond the new size, and if there's not enough
> space it just returns -ENOSPC.  This means no changes to the actual
> way replace copies data - it does extra IO, due to the relocation but
> keeps things simple, and it should still be significantly more
> efficient then doing a device remove + device add operation, maybe
> except if all or most of the allocated chunks (in the device to be
> replaced) cross or start beyond an offset matching the new device's
> size.
> 
>    Also, since the shrink can take some time due to relocation of
>    chunks, we would need to teach btrfs_shrink_device() to check for
>    device replace cancel requests as well.  And such request is
>    detected, restore the device's size to the original value.

The shrinking can be done completely in userspace, calling one more
ioctl before device replace. Handling the error cases will be simplified
(and not necessarily done in kernel at all).

So something like that:

  $ btrfs device replace 2 /dev/sdx /mnt
  (fail because the device is too small, print a message that the target
  device needs to be shrunk manually or there's an option eg.
  --shrink-target that will do that in one go)

  $ btrfs device replace --shrink-target 2 /dev/sdx /mnt
  Shrink device 2 from 12345678 to 123456 (you can cancel that by 'btrfs resize --cancel)
  Done
  Starting devicr replace

> I think option 2 may actually be acceptable for an initial version. Option 1 is
> complex and increases the risk for data loss. Also, for option 2, there's the
> possible downside of requiring writes to the source device - one might
> be replacing
> it because the device is not healthy, writes into some regions are
> failing, which
> can prevent the shrink/relocation process from suceeding, in that case only
> a device remove followed by a device add operation would work.

That's a good point and giving user more options how to replace the
device sounds a like a better option than implementing all of that in
kernel.