Everything and the kitchen sink

Everything and the kitchen sink

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I recently had an incident that reminded me that while the grub rescue
shell is better than nothing when there's a problem, it often isn't
enough to troubleshoot and work around the error.  These days most
disks start the first partition at 1 MiB, so there is no need to keep
the core.img slim and trim, so it would be really nice to throw all of
the modules in there, so you can have a fully functional grub to help
troubleshoot when things go wrong.

Now I realize that I can manually build a ramdisk image to contain all
modules, and pass that to grub-mkimage, but then as soon as grub is
updated, and grub-install is re-run automatically, my changes would be
wiped out.

So I think what needs to happen is for grub-install to grow an
- --everything option where it would build the ramdisk and attach it
automatically.  Additionally there needs to be a conf file or
something where this could be enabled so that it will be applied when
grub is updated.  It may also be nice to have this enabled by default
when the embed area is detected to be large enough.

Thoughts?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJRHP1UAAoJEJrBOlT6nu751SMH/3i+PnYOXL5DAtA2CuP+iWcL
AP/HN3OPHRoGo/WrV+UZoF7gKnZBYOCV+57zU/6FKEOENjs7eVBas3hTlYBLl3vy
6fjZasC5gg3Fu6MGfZjK4y17Dj87G9osPYr51OQ1T2dYD907yiZkcJwFvKFKtWt4
qhxAfUtk3vtxZb9tLzAz6hzgHCUkdPD35iaH7p8/2L/sAK3m1I6Qj+CiemaAsycl
UTXoh65GOX8o4DuuOqQcO/Y+AuvYOq86rkHoFRsEMgL1TnZKHuplwqGgDmgV+7S8
S94xKbVziXtQ8RA1CiXXotEMTWyqLiMszYZ6eKos3RBGr2XN2BKGkr7C0UK12ic=
=MLu2
-----END PGP SIGNATURE-----

Re: Everything and the kitchen sink

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1252 bytes --]

On 14.02.2013 16:05, Phillip Susi wrote:

> I recently had an incident that reminded me that while the grub rescue
> shell is better than nothing when there's a problem, it often isn't
> enough to troubleshoot and work around the error.  These days most
> disks start the first partition at 1 MiB, so there is no need to keep
> the core.img slim and trim, so it would be really nice to throw all of
> the modules in there, so you can have a fully functional grub to help
> troubleshoot when things go wrong.
> 
> Now I realize that I can manually build a ramdisk image to contain all
> modules, and pass that to grub-mkimage, but then as soon as grub is
> updated, and grub-install is re-run automatically, my changes would be
> wiped out.
> 
> So I think what needs to happen is for grub-install to grow an
> --everything option where it would build the ramdisk and attach it
> automatically.  Additionally there needs to be a conf file or
> something where this could be enabled so that it will be applied when
> grub is updated.  It may also be nice to have this enabled by default
> when the embed area is detected to be large enough.
> 
> Thoughts?

GRUB diskboot.S runs in real mode. No way it can load more than ~512K


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Everything and the kitchen sink

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2/14/2013 10:48 AM, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> GRUB diskboot.S runs in real mode. No way it can load more than
> ~512K

Yes, I ran into both the 512k limit that grub-setup enforces, and even
when under that, but still close, my test system throws a read error
from diskboot.S trying to load the whole image.  Is the decompression
done in real mode too, so even uncompressed, it has to fit in 512k?

Maybe diskboot.S could load only the kernel+builtin modules part of
the image, make sure biosdisk is built in, and have memdisk load the
memdisk part of the image once it initializes and is in protected mode?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJRHTxLAAoJEJrBOlT6nu75rJgIANA23r5beH5ML7LJelw+5oxk
kVkcr+gqITGLZG66wTG9UGx10j3AZ/OGOdzEzCMMjIzzH69u8bPRg2YjKFSHkPWi
9IGK8vtqBxQxxgJWOahD/mhzYlwjyIvnCMKTk9jG/e1fAY8FKs/cRtk844RvfEX9
fL1pDPS9PTqYD8nc79voH3vOLkrd7aUjf6JECCnMQPh6CTQN6ieTm1+1n3BoUK3W
EPKxrzMxPTNt6VsGX/VnhWT6luqXo+r7mzt9kYogzUw31JjuyJwj6/6tz9fQNjqp
aR29GoZxdcvD4Zvuw0WuyssDA/9gReUJ6oUVsUzhHQUggnXUMakVxW0SNqAOn9A=
=lfBh
-----END PGP SIGNATURE-----

Re: O_SYNC

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

While playing around with fully featured images, I noticed that
grub-setup takes ages to write the image to disk.  It seems that this
is because the disk is opened with O_SYNC, and sectors are written one
at a time.  This is terribly silly, so I was going to submit a patch
removing the flag, but I noticed that the flush code that should fsync
after writing the whole image is not actually being called.  I traced
it down to here:

grub_err_t
grub_util_biosdisk_flush (struct grub_disk *disk)
{
  struct grub_util_biosdisk_data *data = disk->data;

  if (disk->dev->id != GRUB_DISK_DEVICE_BIOSDISK_ID)
=>  return GRUB_ERR_NONE;

It seems the id is GRUB_DISK_DEVICE_HOSTDISK_ID.  What is the
intention of this check and why is it skipping the fsync?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJRHUyuAAoJEJrBOlT6nu754CEH/19lSCLHRqQlGpqGDdjWvlRS
R2uGCLdHEdfn/8+Upm4PrkWn5LH0TYTDLHDtPEhnMxbe0TfpV17KovqATwLrjVJi
k13J4JY8lhmou0JKzFx3/lP9nWcM/4GGB1nowf3VxMnfFkJqYSBiKNbL9WuJCnaV
HQgLXwRqbi2EeBUe99HM+0TRbiOoZF0umtjAVX+Wa9OikFofoTpVf+bu0eA6ysZ3
jSgjXTVmdVT/FQzUtRA0vIieqabRe7PJZv1ilNxLCUoYcK6LfMpw72jbZ3lQlMRe
kku4hnx4ZjZFv38qNdLMrr5B3D48P7+PQe+FFV04sIocsp0FidrT3mUOb/RO1Gw=
=XiGl
-----END PGP SIGNATURE-----

Do not use O_SYNC

[Phillip Susi]

revno: 164
committer: Phillip Susi <psusi@ubuntu.com>
branch nick: raring
timestamp: Wed 2013-03-20 10:00:03 -0400
message:
  Do not use O_SYNC

  The disk was being opened with O_SYNC, and written to one sector
  at a time.  This has a huge time penalty, causing at least one
  revolution of the disk per sector written.
diff:
=== modified file 'grub-core/kern/emu/hostdisk.c'
--- grub-core/kern/emu/hostdisk.c       2012-09-27 16:16:51 +0000
+++ grub-core/kern/emu/hostdisk.c       2013-03-20 14:00:03 +0000
@@ -848,9 +848,6 @@
 #ifdef O_LARGEFILE
   flags |= O_LARGEFILE;
 #endif
-#ifdef O_SYNC
-  flags |= O_SYNC;
-#endif
 #ifdef O_FSYNC
   flags |= O_FSYNC;
 #endif

Re: Do not use O_SYNC

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Little old, but seems this was lost.  The relevant code seems to have
been moved to osdep/hostfile_unix.h, but it is still using O_SYNC and
I really don't see any reason why it should.  Can we please get rid of
this?

On 3/20/2013 10:15 AM, Phillip Susi wrote:
> revno: 164 committer: Phillip Susi <psusi@ubuntu.com> branch nick:
> raring timestamp: Wed 2013-03-20 10:00:03 -0400 message: Do not use
> O_SYNC
> 
> The disk was being opened with O_SYNC, and written to one sector at
> a time.  This has a huge time penalty, causing at least one 
> revolution of the disk per sector written. diff: === modified file
> 'grub-core/kern/emu/hostdisk.c' --- grub-core/kern/emu/hostdisk.c
> 2012-09-27 16:16:51 +0000 +++ grub-core/kern/emu/hostdisk.c
> 2013-03-20 14:00:03 +0000 @@ -848,9 +848,6 @@ #ifdef O_LARGEFILE 
> flags |= O_LARGEFILE; #endif -#ifdef O_SYNC -  flags |= O_SYNC; 
> -#endif #ifdef O_FSYNC flags |= O_FSYNC; #endif

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)

iQEcBAEBAgAGBQJUquEDAAoJENRVrw2cjl5RDW4H/2U8ICjqHovw8C/s+CFg7smW
q4/Wqpj8bEXHThL3yO+88cTB3q3I2/TGsunq/g4IsFqda3buXrRunHOg7IcNQQhy
JPSBWX7ucsFvi97bAwyh/iqns3MOlJlmnAGpM6yfwYPpyZOaCkU7ulJP0Hopc3LT
+JAq5V3V64GcknjbYhlvWB8K+eXaZ2s0NoIXAFA/ewhi7CQVE22PeQEhxH9rRbKQ
Or4FbGsoFwi+SL4PNuPShp+4H/sg8ewDA/gIVSjsThp9c3whq+nnGhVt28GfLZBc
1KtyeJCJ5HeKOgqzSh7BleUdxtApxFsUU2Fz5M/zyV6f5MidKfHlm8B65sUc+xM=
=HJul
-----END PGP SIGNATURE-----