Re: Why special linux code in grub2/util/grub-setup.c

["Vladimir 'φ-coder/phcoder' Serbinenko" <phcoder@gmail.com>]

On 25.07.2013 21:40, Dr. Tilmann Bubeck wrote:
>
>
> Am 25.07.2013 17:29, schrieb Vladimir 'φ-coder/phcoder' Serbinenko:
>> On 12.07.2013 14:43, Dr. Tilmann Bubeck wrote:
>>> Hi Vladimir,
>>>
>>> Am 12.07.2013 09:05, schrieb Vladimir 'φ-coder/phcoder' Serbinenko:
>>>> On 12.07.2013 08:52, Dr. Tilmann Bubeck wrote:
>>>>> Hi Vladimir,
>>>>>
>>>>> I am curently preparing an improvement for GRUB2 to let it embed into
>>>>> ext4
>>>>> based upon the new ioctl EXT4_IOC_SWAP_BOOT which I wrote for linux
>>>>> 3.10.
>>>>> This would allow to install GRUB SAFELY into a file system and omit
>>>>> the
>>>>> warning about block list being unreliable (for ext4).
>>>>>
>>>> What does this ioctl do?
>>>
>>> It was discussed with Ted T'so and modelled after his suggestion and
>>> included in 3.10. It takes the file associated with the file descriptor
>>> passed to the ioctl and stores the data blocks of that file in a special
>>> (reserved) inode called "BOOT_IMAGE_INO". This inode is reserved
>>> specifically for storing boot loaders inside the filesystem. Technically
>>> it copies the inodes data block pointers and some more data like
>>> timestamps from the given file's inode to the BOOT_IMAGE_INO.
>>>
>>> This means, that we have a reserved area for storing the boot file and
>>> that area is never touched again and therefore "sealed". No user tool is
>>> able to get access to that data (except debug2fs and similar). It is not
>>> visible in any directory and not destroyed or reported by e2fsck. It is
>>> similar to the reserved area for booting in btrfs.
>>>
>>> It is called "SWAP", because the previous content of the BOOT_IMAGE_INO
>>> is linked back to the given fd. Therefore one is able to get the
>>> previous boot code in exchange. GRUB would delete that right away.
>>>
>>> Using this ioctl we could make ext4 able to embed reliably which would
>>> allow installing GRUB into a partition without requiring "--force".
>>>
>>>>> To do so I need code to get the blocks of a file. I found that this is
>>>>> already present in grub2/util/grub-setup.c. I have a few questons
>>>>> based
>>>>> upon your changes in revision 4033.
>>>>>
>>>>> 1. Why did you add additional (linux only) methods to get block list
>>>>> based
>>>>> upon FS_IOC_FIEMAP and FIBMAP instead of sticking with the existing
>>>>> (generic) code based upon  file->read_hook = save_blocklists?
>>>>>
>>>> Because Linux buffering and caching is piece of shit and even issuing
>>>> flush, sync and flushing ioctl doesn't make metadata go to disk.
>>>  >
>>>>> 2. I would like to refactor the above code, so that there is a new
>>>>> method
>>>>> called "grub_getblocklist(dir, core_file, &blocklist);". This method
>>>>> move
>>>>> contain your code together with the generic code to find the
>>>>> blocklist.
>>>>> This method would then be called from the existing place (and my new
>>>>> place
>>>>> in fs/ext2.c if GRUB_UTIL).
>>>>>
>>>>> What do you think?
>>>>>
>>>> Relying on blocklist retrieving is likely wrong  and doesn't solve any
>>>> of blocklist problems. But I lack details to know for sure and right
>>>> now
>>>> I'm under a lot of stress with my thesis on neural learning.
>>>
>>> I would use the blocklist of the above BOOT_IMAGE_INO which will never
>>> change and therefore will never break. However, I need a way to know
>>> where the blocks are stored, even if they are "save".
>>>
>>> Would you accept a patch from me, using the above IOCTL to make
>>> embedding to ext4 work reliable? Sure, we have to discuss the details
>>> but before I start I would like to get a feeling, if this would have a
>>> chance for inclusion.
>>>
>
>> What's with the allocation issues? The blocks have to be as near to the
>> beginning as possible. Is it possible to modify the embedding zone
>> without doing the "swapping"? There is a parallel issues in LVM which
>> would prompt for the ability of overwrite only part of embedded zone.
>
> The embedding zone is not accessible and could not be changed. This
> ensure, that block lists keep stable.
>
> If you want to change the content, then you have to install a new file
> using IOCTL (and you will get back the previous embedded file in exchange).
>
That's a problem since the old zone will stay here and it's one of big 
vectors of problems with blocklists. This approach actually *ensures* 
that blocklist will *change* every time so it makes some simultaneous 
installations impossible and will create an illusion that any lingering 
bootsector is still active. Better way would be:
1) have a way to reserve some space, near the beginning. E.g. IOCTL 
CREATE_EMBEDDING_ZONE with argument size=5M (number coming from my draft 
on LVM zones). This has to have synchronous semantics. Blocklist is 
fixed after this operation.
2) A way to retrieve its blocklist
3) A way to overwrite parts of its contents in a way that ensures 
bypassing journal, COW, compression, encryption, hyperspace or any other 
advanced features.