Re: [PATCH 0/5] Mkfs: Rework --rootdir to a more generic behavior

["Austin S. Hemmelgarn" <ahferroin7@gmail.com>]

On 2017-09-06 12:43, Goffredo Baroncelli wrote:
> On 09/06/2017 01:31 PM, Austin S. Hemmelgarn wrote:
>> On 2017-09-05 15:05, Goffredo Baroncelli wrote:
>>> On 09/05/2017 10:19 AM, Qu Wenruo wrote:
>>>>
>>>>
>>>> On 2017年09月05日 02:08, David Sterba wrote:
>>>>> On Mon, Sep 04, 2017 at 03:41:05PM +0900, Qu Wenruo wrote:
>>>>>> mkfs.btrfs --rootdir provides user a method to generate btrfs with
>>>>>> pre-written content while without the need of root privilege.
>>>>>>
>>>>>> However the code is quite old and doesn't get much review or test.
>>>>>> This makes some strange behavior, from customized chunk allocation
>>>>>> (which uses the reserved 0~1M device space) to lack of special file
>>>>>> handler (Fixed in previous 2 patches).
>>>>>
>>>>> The cleanup in this area is most welcome. The patches look good after a
>>>>> quick look, I'll do another review round.
>>>>
>>>> To save you some time, I found that my rework can't create new image which old --rootdir can do. So it's still not completely the same behavior.
>>>> I can fix it by creating a large sparse file first and then truncate it using current method easily.
>>>>
>>>> But this really concerns me, do we need to shrink the fs?
>>>
>>> I still fatigue to understand in what "mkfs.btrfs --rootdir" would be better than a "simple tar....";
>>>
>>> in the first case I have to do
>>> a1) mkfs.btrfs --root-dir....  (create the archive)
>>> a2) dd  (copy and truncate the image and store it in the archive)
>>> a3) dd  (take the archived image, and restore it)
>>> a4) btrfs fi resize (expand the image)
>> The primary use case for this is to generate installation images.  Using this method removes the need for tar in the installation environment, and if you defer step a4 until the first boot of the system, it also removes the need to have btrfs-progs in the installation environment. Taken together, that's a pretty significant space savings.
> 
> Sorry but I don't understand. If you reach the step a3; you have:
> - the final disk, and an environment fully working. So I am still inclined to think that using "mkfs.btrfs --root-dir" is more complicated in *this case*.
With the current released code (without these patches), `-r` can be used 
to generate a filesystem image that has zero free space.  In that case, 
step a3 does not give you a fully working environment, it gives you a 
basic environment that can probably be booted to a minimal degree and 
run just enough to be able to resize the filesystem to take up the whole 
disk.

IOW, the use case I'm referring to here is more akin to that of 
genext2fs -d than mkfs.ext4 -d, although mkfs.btrfs -r generates a 
minimally sized FS right now, while neither of the other two do so by 
default (although I could have sworn that genext2fs could).
>>
>> It's also somewhat useful for creating minimalistic seed device images, which have a couple of interesting uses, namely:
>>
>> * Base system images for 'factory reset'.  The general principal is simple, your base system is a seed device, plus a storage device associated with it.  When you want to do a factory reset, you wipe the storage partition, and recreate an empty one associated with the seed image.  This usage pretty much requires a minimally sized filesystem, as anything more wastes space that would be otherwise usable by the end user.
>>
>> * Seed-device based install images.  The general concept for this has been tossed around a couple of times before.  You start with a minimal seed device, boot to a live system using that and a temporary in-memory device for root, set up the persistent storage, re-balance everything to persistent storage, then remove the seed device and in-memory device so the user can keep using the system without needing to reboot.  This also needs a minimalistic image, for the same reason any install disc needs to have a minimal base image.
> 
> For the above cases I agree that this could be useful to have "--rootdir"
>>
>> Note that without resize being able to shrink chunks (and ideally completely shrink them so there is no slack space in the FS), you have to use a hex editor to get a truly minimalistic filesystem image.
>>>
>>> in the second case I have to
>>> b1) tar cf ... (create the image an store it in the archive, this is a1+a2)
>>> b2) mkfs,btrfs (create the filesystem with the final size)
>>> b3) tar xf ... (take the archived image and restore it)
>>>
>>>
>>> However the code is already written (and it seems simple enough), so a possible compromise could be to have the "shrinking" only if another option is passed; eg.
>>>
>>> mkfs.btrfs --root ...                --> populate the filesystem
>>> mkfs.btrfs --shrink --root           --> populate and shrink the filesystem
>>>
>>> however I find this useful only if it is possible to creating the filesystem in a file; ie.
>>>
>>> mkfs.btrfs --shrink --root <path-to-source-fs> <file-to-be-created>
>>>
>>> where <file-to-be-created> doesn't have to exists before mkfs.btrfs, and after
>>> a) <file-to-be-created> contains the image
>>> b) <file-to-be-created> is the smallest possible size.
>>>
>>> Definitely I don't like the truncate done by the operator by hand after the mkfs.btrfs (current behavior).
>> FWIW, the current release behavior doesn't require the truncate, and properly generates the file for the filesystem.
> If you don't do truncate, you have the fully partition... Or there is something that I miss ?
The current release, without these patches, run using a non-existent 
file and the `-r` option, will produce a filesystem image of the exact 
size needed to hold everything in the directory passed to `-r`.  It 
doesn't require truncation unless used on a file that already exists.