Re: [PATCH v2] btrfs: validate device maj:min during open

[Boris Burkov <boris@bur.io>]

On Fri, Mar 08, 2024 at 09:23:52AM -0800, Christoph Hellwig wrote:
> On Fri, Mar 08, 2024 at 09:53:07PM +0530, Anand Jain wrote:
> > It's a bit complex, as Boris discovered and has provided a testcase
> > for here:
> > 
> > https://lore.kernel.org/fstests/f40e347d5a4b4b28201b1a088d38a3c75dd10ebd.1709251328.git.boris@bur.io/
> > 
> > In essence:
> > 
> >  - Create two devices, d1 and d2.
> >  - Both devices will be scanned into the kernel by Mfks.
> >  - Use an external method to alter the devt of the d2 device.
> >  - Mount using d1.
> >  - You end up with a 2 devices Btrfs with an incorrect device->devt.
> >  - Delete d1.
> >  - Now you have a single-device Btrfs on d2 with a stale device->devt.
> 
> But how do you get mismatching devices in this exact place?
> 
>   - bdev->bd_dev is immutable and never updated
>   - device->devt can be changed by device_list_add, but if that happens
>     underneath us here between btrfs_get_bdev_and_sb and the code a few
>     lines below the call to it in btrfs_open_one_device there is a huge
>     synchronization problem
> 

You remove/add the device in a way that results in a new bd_dev while
the filesystem is unmounted but btrfs is still caching the struct
btrfs_device. When we unmount a multi-device fs, we don't clear the
device cache, since we need it to remount with just one device name
later.

The mechanism I used for getting a different bd_dev was partitioning two
different devices in two different orders.

I sent the repro script as an fstest last week:
https://lore.kernel.org/linux-btrfs/f40e347d5a4b4b28201b1a088d38a3c75dd10ebd.1709251328.git.boris@bur.io/T/#u

Boris