Re: Seed device is broken, again.

[Anand Jain <anand.jain@oracle.com>]

On 28/02/2022 10:35, Qu Wenruo wrote:
> 
> 
> On 2022/2/28 10:01, Anand Jain wrote:
>> On 28/02/2022 07:56, Qu Wenruo wrote:
>>>
>>>
>>> On 2022/2/26 03:18, Omar Sandoval wrote:
>>>> On Fri, Feb 25, 2022 at 09:36:32PM +0800, Qu Wenruo wrote:
>>>>>
>>>>>
>>>>> On 2022/2/25 19:47, David Sterba wrote:
>>>>>> On Fri, Feb 25, 2022 at 06:08:20PM +0800, Qu Wenruo wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> The very basic seed device usage is broken again:
>>>>>>>
>>>>>>>     mkfs.btrfs -f $dev1 > /dev/null
>>>>>>>     btrfstune -S 1 $dev1
>>>>>>>     mount $dev1 $mnt
>>>>>>>     btrfs dev add $dev2 $mnt
>>>>>>>     umount $mnt
>>>>>>>
>>>>>>>
>>>>>>> I'm not sure how many guys are really using seed device.
>>>>>>>
>>>>>>> But I see a lot of weird operations, like calling a definite write
>>>>>>> operation (device add) on a RO mounted fs.
>>>>>>
>>>>>> That's how the seeding device works, in what way would you want to do
>>>>>> the ro->rw change?
>>>>>
>>>>> In progs-only, so that in kernel we can make dev add ioctl as a 
>>>>> real RW
>>>>> operation, not adding an exception for dev add.
>>>>>
>>>>>>
>>>>>>> Can we make at least the seed sprouting part into btrfs-progs
>>>>>>> instead?
>>>>>>
>>>>>> How? What do you mean? This is an in kernel operation that is done
>>>>>> on a
>>>>>> mounted filesystem, progs can't do that.
>>>>>
>>>>> Why not?
>>>>>
>>>>> Progs has the same ability to read-write the fs, I see no reason 
>>>>> why we
>>>>> shouldn't do it in user space.
>>>>>
>>>>> We're just adding a device, update related trees (which will only be
>>>>> written to the new device). I see no special reason why it can't be
>>>>> done
>>>>> in user space.
>>>>>
>>>>> Furthermore, the ability to add device in user space can be a good
>>>>> safenet for some ENOSPC space.
>>>>>
>>>>>>
>>>>>>> And can seed device even support the upcoming extent-tree-v2?
>>>>>>
>>>>>> I should, it operates on the device level.
>>>>>>
>>>>>>> Personally speaking I prefer to mark seed device deprecated
>>>>>>> completely.
>>>>>>
>>>>>> If we did that with every feature some developer does not like we'd
>>>>>> have
>>>>>> nothing left you know. Seeding is a documented usecase, as long as it
>>>>>> works it's fine to have it available.
>>>>>
>>>>> A documented usecase doesn't mean it can't be deprecated.
>>>>>
>>>>> Furthermore, a documented use case doesn't validate the use case
>>>>> itself.
>>>>>
>>>>> So, please tell me when did you use seed device last time?
>>>>> Or anyone in the mail list, please tell me some real world usecase for
>>>>> seed devices.
>>>>>
>>>>> I did remember some planned use case like a way to use seed device 
>>>>> as a
>>>>> VM/container template, but no, it doesn't get much attention.
>>>>>
>>>>>
>>>>> I'm not asking for deprecate the feature just because I don't like it.
>>>>>
>>>>> This feature is asking for too many exceptions, from the extra
>>>>> fs_devices hack (we have in fact two fs_devices, one for rw devices,
>>>>> one
>>>>> for seed only), to the dev add ioctl.
>>>>>
>>>>> But the little benefit is not really worthy for the cost.
>>>>
>>>> We use seed devices in production. The use case is for servers where we
>>>> don't want to persist any sensitive data. The seed device contains a
>>>> basic boot environment, then we sprout it with a dm-crypt device and
>>>> throw away the encryption key. This ensures that nothing sensitive can
>>>> ever be recovered. We previously did this with overlayfs, but seed
>>>> devices ended up working better for reasons I don't remember.
>>>>
>>>> Davide Cavalca also told me that "Fedora is also interested in
>>>> leveraging seed devices down the road. e.g. doing seed/sprout for cloud
>>>> provisioning, and using seed devices for OEM factory recovery on
>>>> laptops."
>>>>
>>>> There are definitely hacks we need to fix for seed devices, but there
>>>> are valid use cases and we can't just deprecate it.
>>>
>>> OK, then it looks we need to keep the feature.
>>>
>>> Then would you mind to share your preference between things like:
>>>
>>> a) The current way
>>>     mkfs + btrfstune + mount + dev add
>>
>>    And further, dev-remove seed if needed.
>>
>>> b) All user space way
>>>     mkfs /dev/seed
>>>     btrfstune -S 1 /dev/seed
>>>     mkfs -S /dev/seed /dev/new
>>>     (using seed device as seed, and sprout to the new device)
>>
>>   Does the -S option copy all blocks here?
> 
> Nope, just the same as dev add.
> 
>>   How does it work if /dev/new needed to be an independent filesystem
>>   only at some later point? Add another btrfstune option?
> 
> The same dev remove.
> 
>>
>>> With method b, at least we can make dev add ioctl to be completely RW in
>>> the long run.
>>
>>   Could you please add more clarity here? Very confusing.
> 
> Isn't it an awful thing that device add ioctl can even be executed on a
> RO mounted fs?
> 

Ah. That's fine, IMO. It is a matter of understanding the nature of the
seed device. No?
The RO mount used to turn into an RW mount after the device-add a long
ago. It got changed without a trace.

Thanks, Anand


> Thanks,
> Qu
> 
>>
>> Thanks, Anand
>>
>>> Thanks,
>>> Qu
>>