From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1V2SuO-0002GG-2W for mharc-grub-devel@gnu.org; Thu, 25 Jul 2013 17:17:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40318) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2SuL-0002CL-9j for grub-devel@gnu.org; Thu, 25 Jul 2013 17:17:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V2SuJ-0005Tr-JA for grub-devel@gnu.org; Thu, 25 Jul 2013 17:17:09 -0400 Received: from mail-we0-x232.google.com ([2a00:1450:400c:c03::232]:63241) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V2SuJ-0005S5-9x for grub-devel@gnu.org; Thu, 25 Jul 2013 17:17:07 -0400 Received: by mail-we0-f178.google.com with SMTP id u57so1174789wes.9 for ; Thu, 25 Jul 2013 14:17:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=hoRXO/tQqVVWElt/u6aEMiwbVwWcQguzLk4odPvTbG8=; b=r2Ir5QToqwIghMLztVyDDFk7ed05IIUTTW3iomHlSLRgcfTJ1boJ6shxxc12P8t+oC SWWAo/M7MIRNRTBH0JBKSSz/An/hczgOD83blNFrJo7HbHVgOaoF8WwCDnUHqg+oi/jx r97bKpMavyGaC2pgKf2ZGDwFQGfxOw5ioSPDa+ziFqPmY9kelx9/tDDg8QrK7tv1E17W 1fKAPrhb75aDzLwV0iCknTmPgl+yXjHQXB2nFdpMO2ZKFloJfTP/LPQmR2Oc4VbBXDkV znukbk1G3cwZU9xBFHCAGYBwtG7hdk7LV2F0CTJyqXdEUSuYSKd+3V2yhkHK5Mu02B7T TA5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=hoRXO/tQqVVWElt/u6aEMiwbVwWcQguzLk4odPvTbG8=; b=cIh9FBw+rUnElL6akqd6KK/WEsaw81yalykflcXNLFSv0Dct+Y71fBtdQkWA7paOkS w/IdY+RBJKHR8r/hSmDM6QksDn9LmlGsnFzeSYYaWqLw2l+Dva3HC0+Z7LtvPrsck9rS HuvY4FXxLFw5AmkL6DRO8weOWJcVxDY8N7PGgRzEUsCMz75m9KDnPJbhytSFqjHcrWYA 21ThzwXRsoLhSLKZkZan8j+03gMjKUIVD5t2lTRTRNtQ8zizMPLK4tN4mxMKnsK2Vg8y iqyVj0zGEw3PUE223oKr9xMTdqBMTg6F0Hg8oE9mV5S6UdEs+ZlW/A5h5zM7qUp8dW1s /k6g== X-Received: by 10.180.9.212 with SMTP id c20mr3396576wib.65.1374787026162; Thu, 25 Jul 2013 14:17:06 -0700 (PDT) Received: from [192.168.1.113] (31-249.1-85.cust.bluewin.ch. [85.1.249.31]) by mx.google.com with ESMTPSA id u7sm554042wiw.9.2013.07.25.14.17.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 25 Jul 2013 14:17:05 -0700 (PDT) Message-ID: <51F195D2.1070007@gmail.com> Date: Thu, 25 Jul 2013 23:17:06 +0200 From: =?UTF-8?B?VmxhZGltaXIgJ8+GLWNvZGVyL3BoY29kZXInIFNlcmJpbmVua28=?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130630 Icedove/17.0.7 MIME-Version: 1.0 To: The development of GNU GRUB Subject: Re: Why special linux code in grub2/util/grub-setup.c References: <8513f87fd2e877871174acaf00583071.squirrel@bubecks.de> <51DFAAB0.2040908@gmail.com> <51DFF9EC.7010801@bubecks.de> <51F14448.2050103@gmail.com> <51F17F1E.4050400@bubecks.de> In-Reply-To: <51F17F1E.4050400@bubecks.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:400c:c03::232 X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: The development of GNU GRUB List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jul 2013 21:17:11 -0000 On 25.07.2013 21:40, Dr. Tilmann Bubeck wrote: > > > Am 25.07.2013 17:29, schrieb Vladimir 'φ-coder/phcoder' Serbinenko: >> On 12.07.2013 14:43, Dr. Tilmann Bubeck wrote: >>> Hi Vladimir, >>> >>> Am 12.07.2013 09:05, schrieb Vladimir 'φ-coder/phcoder' Serbinenko: >>>> On 12.07.2013 08:52, Dr. Tilmann Bubeck wrote: >>>>> Hi Vladimir, >>>>> >>>>> I am curently preparing an improvement for GRUB2 to let it embed into >>>>> ext4 >>>>> based upon the new ioctl EXT4_IOC_SWAP_BOOT which I wrote for linux >>>>> 3.10. >>>>> This would allow to install GRUB SAFELY into a file system and omit >>>>> the >>>>> warning about block list being unreliable (for ext4). >>>>> >>>> What does this ioctl do? >>> >>> It was discussed with Ted T'so and modelled after his suggestion and >>> included in 3.10. It takes the file associated with the file descriptor >>> passed to the ioctl and stores the data blocks of that file in a special >>> (reserved) inode called "BOOT_IMAGE_INO". This inode is reserved >>> specifically for storing boot loaders inside the filesystem. Technically >>> it copies the inodes data block pointers and some more data like >>> timestamps from the given file's inode to the BOOT_IMAGE_INO. >>> >>> This means, that we have a reserved area for storing the boot file and >>> that area is never touched again and therefore "sealed". No user tool is >>> able to get access to that data (except debug2fs and similar). It is not >>> visible in any directory and not destroyed or reported by e2fsck. It is >>> similar to the reserved area for booting in btrfs. >>> >>> It is called "SWAP", because the previous content of the BOOT_IMAGE_INO >>> is linked back to the given fd. Therefore one is able to get the >>> previous boot code in exchange. GRUB would delete that right away. >>> >>> Using this ioctl we could make ext4 able to embed reliably which would >>> allow installing GRUB into a partition without requiring "--force". >>> >>>>> To do so I need code to get the blocks of a file. I found that this is >>>>> already present in grub2/util/grub-setup.c. I have a few questons >>>>> based >>>>> upon your changes in revision 4033. >>>>> >>>>> 1. Why did you add additional (linux only) methods to get block list >>>>> based >>>>> upon FS_IOC_FIEMAP and FIBMAP instead of sticking with the existing >>>>> (generic) code based upon file->read_hook = save_blocklists? >>>>> >>>> Because Linux buffering and caching is piece of shit and even issuing >>>> flush, sync and flushing ioctl doesn't make metadata go to disk. >>> > >>>>> 2. I would like to refactor the above code, so that there is a new >>>>> method >>>>> called "grub_getblocklist(dir, core_file, &blocklist);". This method >>>>> move >>>>> contain your code together with the generic code to find the >>>>> blocklist. >>>>> This method would then be called from the existing place (and my new >>>>> place >>>>> in fs/ext2.c if GRUB_UTIL). >>>>> >>>>> What do you think? >>>>> >>>> Relying on blocklist retrieving is likely wrong and doesn't solve any >>>> of blocklist problems. But I lack details to know for sure and right >>>> now >>>> I'm under a lot of stress with my thesis on neural learning. >>> >>> I would use the blocklist of the above BOOT_IMAGE_INO which will never >>> change and therefore will never break. However, I need a way to know >>> where the blocks are stored, even if they are "save". >>> >>> Would you accept a patch from me, using the above IOCTL to make >>> embedding to ext4 work reliable? Sure, we have to discuss the details >>> but before I start I would like to get a feeling, if this would have a >>> chance for inclusion. >>> > >> What's with the allocation issues? The blocks have to be as near to the >> beginning as possible. Is it possible to modify the embedding zone >> without doing the "swapping"? There is a parallel issues in LVM which >> would prompt for the ability of overwrite only part of embedded zone. > > The embedding zone is not accessible and could not be changed. This > ensure, that block lists keep stable. > > If you want to change the content, then you have to install a new file > using IOCTL (and you will get back the previous embedded file in exchange). > That's a problem since the old zone will stay here and it's one of big vectors of problems with blocklists. This approach actually *ensures* that blocklist will *change* every time so it makes some simultaneous installations impossible and will create an illusion that any lingering bootsector is still active. Better way would be: 1) have a way to reserve some space, near the beginning. E.g. IOCTL CREATE_EMBEDDING_ZONE with argument size=5M (number coming from my draft on LVM zones). This has to have synchronous semantics. Blocklist is fixed after this operation. 2) A way to retrieve its blocklist 3) A way to overwrite parts of its contents in a way that ensures bypassing journal, COW, compression, encryption, hyperspace or any other advanced features.