Hi,

I have a failing device that's part of my my root filesystem. I'm not
using a data-redundant RAID configuration. I'd like to know how to
remove this device without corrupting the data I care about (more on
this momentarily).

To provide some background, the affected device is /dev/sdb:

$ sudo smartctl -a /dev/sdb
[...]
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed: read failure       90%     11729         3678402408
# 2  Short offline       Completed: read failure       90%     11729         3678402408
[...]

The only partition on this device is mapped to /dev/mapper/cryptsdd1,
which is part of my root filesystem:

$ sudo btrfs fi show /
Label: 'btrfs'  uuid: d31878d6-3a77-4f0f-9fdd-bb9a2c4e578b
        Total devices 4 FS bytes used 2.76TiB
        devid    1 size 931.01GiB used 928.99GiB path /dev/mapper/cryptnvme
        devid    2 size 931.50GiB used 931.50GiB path /dev/mapper/cryptnvme1
        devid    3 size 1.82TiB used 1.55TiB path /dev/mapper/cryptsdd1
        devid    4 size 15.88TiB used 0.00B path /dev/mapper/cryptsdc2

I *think* all data I care about is safe: I snapshot my home directory
every hour and backup these snapshots to a remote backup service
periodically. I haven't run into any issues doing that. I believe the
lost data is from an old snapshot, which I'm fine with (assuming that's
true).

I'd like to remove this failing device without corrupting any data I do
care about. How can I accomplish this?

I attempted scrubbing the partition, but that seems to have no
effect. Running

$ sudo btrfs scrub start /dev/mapper/cryptsdd1

doesn't produce any output and doesn't return. Opening another terminal
and using status doesn't return either. I'm also unable to terminate
either of these processes. I was hoping to use this to confirm that the
affected files are confined to old snapshots.

I tried removing the device, but that fails:

$ sudo btrfs device remove /dev/mapper/cryptsdd1 /
ERROR: error removing device '/dev/mapper/cryptsdd1': Input/output error

What can I do?

Thanks!
Matt

FYI:

$ uname -a
Linux ryzen3950 5.12.4 #1-NixOS SMP Fri May 14 08:53:07 UTC 2021 x86_64 GNU/Linux

$ btrfs --version
btrfs-progs v5.11.1

$ sudo btrfs fi show
Label: 'btrfs'  uuid: d31878d6-3a77-4f0f-9fdd-bb9a2c4e578b
        Total devices 4 FS bytes used 2.76TiB
        devid    1 size 931.01GiB used 928.99GiB path /dev/mapper/cryptnvme
        devid    2 size 931.50GiB used 931.50GiB path /dev/mapper/cryptnvme1
        devid    3 size 1.82TiB used 1.55TiB path /dev/mapper/cryptsdd1
        devid    4 size 15.88TiB used 0.00B path /dev/mapper/cryptsdc2

Label: 'backup'  uuid: 0bd10808-0330-4736-9425-059d4a0a300e
        Total devices 2 FS bytes used 1.08TiB
        devid    1 size 1.82TiB used 593.00GiB path /dev/mapper/cryptsdc1
        devid    2 size 1.82TiB used 588.00GiB path /dev/mapper/cryptsdb1

(the above is printed, but the command doesn't return...)

$ sudo btrfs fi df /
Data, single: total=3.15TiB, used=2.74TiB
System, RAID1: total=32.00MiB, used=432.00KiB
Metadata, RAID1: total=110.00GiB, used=18.25GiB
GlobalReserve, single: total=512.00MiB, used=0.00B