A thread on grub-bug could need attention

A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

i am unhappy about the state of the thread

  http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00010.html
  "HP ProBook x360 11 G1 EE incompatible with grub"

which is about an EFI implementation which SYSLINUX for EFI can operate
from USB stick whereas GRUB from USB stick ends up with black screen.

An allergy of the firmware towards ISO 9660 hybrid images on USB stick
can surely be ruled out by
  http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00022.html
  (The only known production ISO with SYSLINUX for EFI does work)
  http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00024.html
  (A normal distro installation works after replacing GRUB by SYSLINUX)

The bug reporter, Michel Bouissou, seems well apt enough to do experiments.
But i am not apt enough to propose any beyond ISO 9660.
Could people from this list please give Michel some hints about how to
retrieve more information about the problem between GRUB and this
peculiar EFI ?


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Daniel Kiper]

On Wed, Dec 20, 2017 at 12:35:20PM +0100, Thomas Schmitt wrote:
> Hi,
>
> i am unhappy about the state of the thread
>
>   http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00010.html
>   "HP ProBook x360 11 G1 EE incompatible with grub"
>
> which is about an EFI implementation which SYSLINUX for EFI can operate
> from USB stick whereas GRUB from USB stick ends up with black screen.
>
> An allergy of the firmware towards ISO 9660 hybrid images on USB stick
> can surely be ruled out by
>   http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00022.html
>   (The only known production ISO with SYSLINUX for EFI does work)
>   http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00024.html
>   (A normal distro installation works after replacing GRUB by SYSLINUX)
>
> The bug reporter, Michel Bouissou, seems well apt enough to do experiments.
> But i am not apt enough to propose any beyond ISO 9660.
> Could people from this list please give Michel some hints about how to
> retrieve more information about the problem between GRUB and this
> peculiar EFI ?

Thank you for a notice. Sorry for late reply. I am just back after a bit
longer vacation. Now I am recovering and trying to get at speed. I will
take closer look at the thread next week.

Thanks,

Daniel

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hello,

Le 08/01/2018 à 13:48, Daniel Kiper a écrit :
> On Wed, Dec 20, 2017 at 12:35:20PM +0100, Thomas Schmitt wrote:
>> Hi,
>>
>> i am unhappy about the state of the thread
>>
>>   http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00010.html
>>   "HP ProBook x360 11 G1 EE incompatible with grub"
>>
>> which is about an EFI implementation which SYSLINUX for EFI can operate
>> from USB stick whereas GRUB from USB stick ends up with black screen.
>>
>> An allergy of the firmware towards ISO 9660 hybrid images on USB stick
>> can surely be ruled out by
And I believe it has been. I have made a complete Linux install on an
USB stick of the latest Manjaro, so the stick is « setup like a hard
disk ». It will boot on other UEFI machines but will not boot on the
ProBook x360 11 G1 EE.

>> The bug reporter, Michel Bouissou, seems well apt enough to do experiments.
>> But i am not apt enough to propose any beyond ISO 9660.
>> Could people from this list please give Michel some hints about how to
>> retrieve more information about the problem between GRUB and this
>> peculiar EFI ?
As a side note, this is my son's student machine, he needs it daily in
college and also often needs it at home, so I don't have it available
all day and I cannot do experiments that would break the installed
Windows (alas).

I have also been on travel between XMas and january 1st, my son at
another place with his machine, and I will be in travel again all february.

So I volunteer to perform all necessary tests to understand this issue
better, however for the reasons above it may take some time before I can
both have the machine at hand and time to work on the issue. But I will
try my best to perform all proposed tests - and already did quite a
number of them...

So far my understanding of the issue is : as soon as the computer loads
grub, it freezes, independantly of the distro, used, whether be it from
an ISO image or a "normally" installed distro, and whether grub was
directly loaded from UEFI, or chained from rEFINd.

rEFInd works, Syslinux loads and boots ; grub hangs.

Best regards.

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Daniel Kiper]

On Mon, Jan 08, 2018 at 01:48:42PM +0100, Daniel Kiper wrote:
> On Wed, Dec 20, 2017 at 12:35:20PM +0100, Thomas Schmitt wrote:
> > Hi,
> >
> > i am unhappy about the state of the thread
> >
> >   http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00010.html
> >   "HP ProBook x360 11 G1 EE incompatible with grub"
> >
> > which is about an EFI implementation which SYSLINUX for EFI can operate
> > from USB stick whereas GRUB from USB stick ends up with black screen.
> >
> > An allergy of the firmware towards ISO 9660 hybrid images on USB stick
> > can surely be ruled out by
> >   http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00022.html
> >   (The only known production ISO with SYSLINUX for EFI does work)
> >   http://lists.gnu.org/archive/html/bug-grub/2017-12/msg00024.html
> >   (A normal distro installation works after replacing GRUB by SYSLINUX)
> >
> > The bug reporter, Michel Bouissou, seems well apt enough to do experiments.
> > But i am not apt enough to propose any beyond ISO 9660.
> > Could people from this list please give Michel some hints about how to
> > retrieve more information about the problem between GRUB and this
> > peculiar EFI ?
>
> Thank you for a notice. Sorry for late reply. I am just back after a bit
> longer vacation. Now I am recovering and trying to get at speed. I will
> take closer look at the thread next week.

OK, I have looked through the email exchange. I have not seen any info
about the UEFI/firmware update. Have you done that? If you have not
please do. If it does not help please create bootable USB stick in
following way. I assume that USB stick is /dev/sdb device. If not please
change device path below accordingly. Additionally, if you use Debian,
IIRC you do, please install at least following extra packages: dosfstools,
gdisk, grub-common and grub-efi-amd64-bin. I assume that other distros
has similar ones.

1) Zap GPT and MBR - DANGEROUS - it removes all GPT and MBR partitions
   # sgdisk -Z /dev/sdb

2) Create EFI System Partition (code EF00) using gdisk. 256.0 MiB will suffice.

3) Format EFI System Partition
   # mkdosfs /dev/sdb1

4) Create GRUB2 EFI bootable image
   # mount /dev/sdb1 /mnt
   # mkdir /mnt/efi/boot
   # grub-mkstandalone -O x86_64-efi -o /mnt/efi/boot/bootx64.efi
   # umount /dev/sdb1

Reboot HP machine from this USB stick. You should see normal GRUB2 prompt.
If it does not happen then I will think about next steps.

If you have any questions please drop me a line.

Additionally, I will be at FOSDEM this weekend. So, if you wish and you
will be there then you can bring the machine and I can take a look.

Daniel

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hello,

Le 29/01/2018 à 13:49, Daniel Kiper a écrit :
> OK, I have looked through the email exchange. I have not seen any info
> about the UEFI/firmware update. Have you done that? If you have not
> please do.
I have installed the latest HP UEFI/firmware from HP on this machine the
day I got it (last december)

Version is 01.11 (Machine came with 01.09 AFAIR)

>  If it does not help please create bootable USB stick in
> following way.

I created the USB stick per your instructions. I will try to give it a
shot tonight on my son's machine.

For your reference, the machine from which I created the stick is a
Manjaro and grub package version is 2.02.0-4.

After the grub-mkstandalone phase, the /mnt/efi/boot contains :
-rwxr-xr-x 1 root root 12068352 29 janv. 16:19 bootx64.efi

Thanks for your help.

Kind regards.

-- 

ॐ

Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi again,

Le 29/01/2018 à 16:29, Michel Bouissou a écrit :
> I created the USB stick per your instructions. I will try to give it a
> shot tonight on my son's machine.

I tested the USB stick on my son's machine, and I do get a grub prompt.

From there, if I type "help", I get a list of grub shell commands.

And if I type "reboot", the machine reboots.

So tha'ts a progress.

Thanks for the suggestion.

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

Daniel Kiper wrote:
> > 2) Create EFI System Partition (code EF00) using gdisk. 256.0 MiB will
> > suffice.
> > 3) Format EFI System Partition
> >   # mkdosfs /dev/sdb1
> >
> >4) Create GRUB2 EFI bootable image
> >   # mount /dev/sdb1 /mnt
> >   # mkdir /mnt/efi/boot
> >   # grub-mkstandalone -O x86_64-efi -o /mnt/efi/boot/bootx64.efi

Michel Bouissou wrote:
> I tested the USB stick on my son's machine, and I do get a grub prompt.

Does this mean that the decisive trick is to use GPT with EFI partition
instead of a MBR partition of type 0xEF ?

Or is there something missing in the FAT filesystem images of e.g.
Debian installation ISOs like
  https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.3.0-amd64-netinst.iso 
?

Michel: Does a simple ISO "output.iso" work, when made by

  mkdir ./minimal
  echo dummy >./minimal/dummy
  grub-mkrescue -o output.iso ./minimal

and then copied onto the plain USB stick device (e.g. /dev/sdc) ?
grub-mkrescue produces valid GPT in ISO 9660.


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi there,

Le 29/01/2018 à 19:53, Thomas Schmitt a écrit :
>
> Does this mean that the decisive trick is to use GPT with EFI partition
> instead of a MBR partition of type 0xEF ?
>
> Or is there something missing in the FAT filesystem images of e.g.
> Debian installation ISOs like
>   https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.3.0-amd64-netinst.iso 
> ?
It's probably not related to any specific images, as I could boot no
standard Linux USB install stick made from an ISO image on this machine,
besides Knoppix or Tails.

I have installed a Manjaro with grub onto a blank USB stick, in EFI
mode, and it would boot perfectly on other machines, but not on affected
machine.

But AFAIR the USB partition table was DOS, not GPT.

So might this machine boot only from GPT disks ? Well, then it would'tn
boot a Knoppix ISO image stick. But it does...
> Michel: Does a simple ISO "output.iso" work, when made by
>
>   mkdir ./minimal
>   echo dummy >./minimal/dummy
>   grub-mkrescue -o output.iso ./minimal
>
> and then copied onto the plain USB stick device (e.g. /dev/sdc) ?
> grub-mkrescue produces valid GPT in ISO 9660.
>
I've produced an USB stick per your instructions above and tested it OK
on another machine. I will try to test it tonight on the affected
machine and let you know.

Thanks for your help.

Kind regards.

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

I wrote:
> > Does this mean that the decisive trick is to use GPT with EFI partition
> > instead of a MBR partition of type 0xEF ?

Michel Bouissou wrote:
> So might this machine boot only from GPT disks ? Well, then it would'tn
> boot a Knoppix ISO image stick. But it does...

Yes, i proposed to Klaus Knopper to deviate from Matthew J. Garrett's
layout which is used by Fedora, Debian, and others.
So Knoppix 8.1 has the EFI partition appended outside the ISO 9660
filesystem range, with a neat MBR partition table without nested
partitions. No GPT gets written, because it would be invalid.

Nevertheless, Knoppix uses SYSLINUX which could be less dependent on
a neat GPT than GRUB2 could be ... theoretically guessing ...


> I have installed a Manjaro with grub onto a blank USB stick, in EFI
> mode, and it would boot perfectly on other machines, but not on affected
> machine.

It would be interesting to compare the particular actions of the
installation with the procedure proposed by Daniel Kiper which works.


> I could boot no
> standard Linux USB install stick made from an ISO image on this machine,
> besides Knoppix or Tails.

Oh. I forgot your success with Tails. My mailbox says i inspected 3.3
and that it was BIOS-only with ISOLINUX as bootloader.
Just to be sure:
  http://dl.amnesia.boum.org/tails/stable/tails-amd64-3.5/tails-amd64-3.5.iso
No EFI partition at all. And no GRUB.
This corresponds to the "mac" ISOs of Debian, which are BIOS-only because
some MacIntoshs are said to take offense on the EFI equipment.
E.g. this 293 MB image:

  https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-mac-9.3.0-amd64-netinst.iso


> >   grub-mkrescue -o output.iso ./minimal

> I will try to test it tonight on the affected machine and let you know.

I am curious what will happen.


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Le 30/01/2018 à 17:33, Thomas Schmitt a écrit :
>> I will try to test it tonight on the affected machine and let you know.
> I am curious what will happen.

It boots into grub properly.

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Daniel Kiper]

On Mon, Jan 29, 2018 at 07:20:25PM +0100, Michel Bouissou wrote:
> Hi again,
>
> Le 29/01/2018 ?? 16:29, Michel Bouissou a ??crit??:
> > I created the USB stick per your instructions. I will try to give it a
> > shot tonight on my son's machine.
>
> I tested the USB stick on my son's machine, and I do get a grub prompt.
>
> >From there, if I type "help", I get a list of grub shell commands.
>
> And if I type "reboot", the machine reboots.
>
> So tha'ts a progress.

Great!

> Thanks for the suggestion.

You are welcome!

IIRC you see blank screen after installing some distros. Right? Could you send
me grub.cfg/grub2.cfg generated by the installer? It should be somewhere in
/boot/grub or /boot/grub2 or even on EFI System Partition.

Daniel

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

Daniel Kiper wrote:
> IIRC you see blank screen after installing some distros. Right?

It also does not work when booting the ISO images which shall install
the systems.
E.g. debian-9.3.0-amd64-netinst.iso :

  mount debian-9.3.0-amd64-netinst.iso /mnt/iso
  mount /mnt/iso/boot/grub/efi.img /mnt/fat

At the end of the binary /mnt/fat/efi/boot/bootx64.efi, "strings" sees:

  search --file --set=root /.disk/info
  set prefix=($root)/boot/grub
  source $prefix/x86_64-efi/grub.cfg
  (memdisk)/boot/grub

The search leads to the ISO 9660 filesystem.
/mnt/iso/boot/grub/x86_64-efi/grub.cfg has

  insmod part_acorn
  insmod part_amiga
  insmod part_apple
  insmod part_bsd
  insmod part_dfly
  insmod part_dvh
  insmod part_gpt
  insmod part_msdos
  insmod part_plan
  insmod part_sun
  insmod part_sunpc
  source /boot/grub/grub.cfg

/mnt/iso/boot/grub/grub.cfg has

  if loadfont $prefix/font.pf2 ; then
    set gfxmode=800x600
    insmod efi_gop
    insmod efi_uga
    insmod video_bochs
    insmod video_cirrus
    insmod gfxterm
    insmod png
    terminal_output gfxterm
  fi
  
  if background_image /isolinux/splash.png; then
    set color_normal=light-gray/black
    set color_highlight=white/black
  else
    set menu_color_normal=cyan/blue
    set menu_color_highlight=white/blue
  fi
  
  insmod play
  play 960 440 1 0 4 440 1
  set theme=/boot/grub/theme/1
  menuentry --hotkey=g 'Graphical install' {
      set background_color=black
      linux    /install.amd/vmlinuz vga=788 --- quiet 
      initrd   /install.amd/gtk/initrd.gz
  }
  menuentry --hotkey=i 'Install' {
      set background_color=black
      linux    /install.amd/vmlinuz vga=788 --- quiet 
      initrd   /install.amd/initrd.gz
  }
  submenu --hotkey=a 'Advanced options ...' {
      set menu_color_normal=cyan/blue
      set menu_color_highlight=white/blue
      set theme=/boot/grub/theme/1-1
      menuentry '... Graphical expert install' {
          set background_color=black
          linux    /install.amd/vmlinuz priority=low vga=788 --- 
          initrd   /install.amd/gtk/initrd.gz
      }
      menuentry '... Graphical rescue mode' {
          set background_color=black
          linux    /install.amd/vmlinuz vga=788 rescue/enable=true --- quiet  
          initrd   /install.amd/gtk/initrd.gz
      }
      menuentry '... Graphical automated install' {
          set background_color=black
          linux    /install.amd/vmlinuz auto=true priority=critical vga=788 --- quiet 
          initrd   /install.amd/gtk/initrd.gz
      }
      menuentry --hotkey=x '... Expert install' {
          set background_color=black
          linux    /install.amd/vmlinuz priority=low vga=788 --- 
          initrd   /install.amd/initrd.gz
      }
      menuentry --hotkey=r '... Rescue mode' {
          set background_color=black
          linux    /install.amd/vmlinuz vga=788 rescue/enable=true --- quiet 
          initrd   /install.amd/initrd.gz
      }
      menuentry --hotkey=a '... Automated install' {
          set background_color=black
          linux    /install.amd/vmlinuz auto=true priority=critical vga=788 --- quiet 
          initrd   /install.amd/initrd.gz
      }
      submenu --hotkey=s '... Speech-enabled advanced options ...' {
          set menu_color_normal=cyan/blue
          set menu_color_highlight=white/blue
          set theme=/boot/grub/theme/1-1-1
          menuentry --hotkey=x '... Expert speech install' {
              set background_color=black
              linux    /install.amd/vmlinuz priority=low vga=788 speakup.synth=soft --- 
              initrd   /install.amd/gtk/initrd.gz
          }
          menuentry --hotkey=r '... Rescue speech mode' {
              set background_color=black
              linux    /install.amd/vmlinuz vga=788 rescue/enable=true speakup.synth=soft --- quiet  
              initrd   /install.amd/gtk/initrd.gz
          }
          menuentry --hotkey=a '... Automated speech install' {
              set background_color=black
              linux    /install.amd/vmlinuz auto=true priority=critical vga=788 speakup.synth=soft --- quiet 
              initrd   /install.amd/gtk/initrd.gz
          }
      }
  }
  menuentry --hotkey=s 'Install with speech synthesis' {
      set background_color=black
      linux    /install.amd/vmlinuz vga=788 speakup.synth=soft --- quiet 
      initrd   /install.amd/gtk/initrd.gz
  }

--------------------------------------------------------------------

Well, my MBR versus GPT theory fades away by Daniel Kiper's above question.

The next step would have been to build a grub-mkrescue image with
a partition layout like Knoppix 8.1. You'd take script
  xorriso-1.4.8/frontend/grub-mkrescue-sed.sh
from a GNU xorriso source installation or from
  https://dev.lovelyhq.com/libburnia/libisoburn/raw/master/frontend/grub-mkrescue-sed.sh
and use it as "xorriso" for the grub-mkrescue run.

It re-arranges the options for the xorriso run to achieve other partition
layouts and/or positions of the EFI System Partition.
Default is mode "mbr_only", which puts the EFI partition image after
the ISO 9660 partition and marks both partitions by MBR rather than GPT.

Best with xorrisofs option "-partition_offset 16", which lets the ISO
partition start at byte 32768 with an own superblock and directory tree,
while the overall ISO filesystem starts at byte 0 and claims the whole
image size including the EFI partition as its filesystem size.

  grub-mkrescue -o output.iso minimal \
                --xorriso=...path.../grub-mkrescue-sed.sh \
                -partition_offset 16  

If xorriso-1.4.8 is installed, you should afterwards get told by

  xorriso -indev output.iso -report_system_area plain

something like:

  System area summary: MBR grub2-mbr cyl-align-off
  ISO image size/512 : 33864
  Partition offset   : 16
  MBR heads per cyl  : 64
  MBR secs per head  : 32
  MBR partition table:   N Status  Type        Start       Blocks
  MBR partition      :   1   0x80  0x00           64        28040
  MBR partition      :   2   0x00  0xef        28104         5760

after this harmless warning:

  libisofs: WARNING : Found hidden El-Torito image. Its size could not be figured out, so image modify or boot image patching may lead to bad results.

(I will have to improve the multi-session capabilities with boot images
 which are not reachable as files of the ISO 9660 filesystem.)


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hello,

Le 30/01/2018 à 20:15, Thomas Schmitt a écrit :
> It also does not work when booting the ISO images which shall install
> the systems.
> E.g. debian-9.3.0-amd64-netinst.iso :
Yes. AFAIR I had tested all the latest (in december) standard
installation media for at least :
- Manjaro Cinnamon x64 (created either with dd or unetbootin)
- Mint Cinnamon x64 (created either with dd or unetbootin)
- Ubuntu 17.10
- Debian live
- Parted Magic (created with dd)
(and maybe a few more, see my first mails...)

- Manjaro Cinnamon after it being "normally" installed on a USB key
instead of an HD on another machine - and tested working.

None of them booted, all showed the same behaviour :
- The machine's UEFI BIOS displays the USB Key as « Windows bootloader »
(and the true on-HD Windows bootloader bears the same label)
- Selecting is causes the screen to go black with a cursor on the top
left and nothing else ever happens.

Now what works :
- rEFInd USB key (made with dd)
- Knoppix USB key (made with dd from DVD image)
- Tails USB key (both made with dd or from the Tails installer)
- Manjaro install key created with unetbootin THEN manually doctored to
remove grub and install syslinux instead.

- Plus the grub test keys you asked me to create and test in your
previous mails.

Interestingly, for "working" keys, the machines UEFI  displays the USB
key brand / model instead of « Windows bootloader »

Even more interesting, keys that do NOT work do not work either when
chained to from rEFInd (inserting both keys, booting on rEFInd then
chaining to the other key), with the same symptoms - where keys that
work, also work when performing the same test.

As I side note I'm currently packing for a trip to India and will be
unable to perform any further tests from next week to the end of
february (and will have very little time left for doing so before leaving)

Have a nice day :)

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

i wrote:
> > It also does not work when booting the ISO images which shall install
> > the systems.

Michel Bouissou wrote:
> Yes. AFAIR I had tested all the latest (in december) standard
> installation media for at least :
> [...]
> - Debian live

debian-live-9.3.0-amd64-xfce.iso looks much like the Debian netinst ISO.

Strings from end of /mnt/fat/efi/boot/bootx64.efi :
-------------------------------------------------------------------------
search --file --set=root /.disk/info
set prefix=($root)/boot/grub
source $prefix/x86_64-efi/grub.cfg
(memdisk)/boot/grub
-------------------------------------------------------------------------

/mnt/iso/boot/grub/x86_64-efi/grub.cfg :
-------------------------------------------------------------------------
insmod part_acorn
insmod part_amiga
insmod part_apple
insmod part_bsd
insmod part_dfly
insmod part_dvh
insmod part_gpt
insmod part_msdos
insmod part_plan
insmod part_sun
insmod part_sunpc
source /boot/grub/grub.cfg
-------------------------------------------------------------------------

/mnt/iso/boot/grub/grub.cfg :
-------------------------------------------------------------------------
if loadfont $prefix/font.pf2 ; then
  set gfxmode=800x600
  insmod efi_gop
  insmod efi_uga
  insmod video_bochs
  insmod video_cirrus
  insmod gfxterm
  insmod png
  terminal_output gfxterm
fi

if background_image /isolinux/splash.png; then
  set color_normal=light-gray/black
  set color_highlight=white/black
else
  set menu_color_normal=cyan/blue
  set menu_color_highlight=white/blue
fi

insmod play
play 960 440 1 0 4 440 1
if [ ${iso_path} ] ; then
set loopback="findiso=${iso_path}"
fi

menuentry "Debian GNU/Linux Live (kernel 4.9.0-4-amd64)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
submenu "Debian Live with Localisation Support" {
menuentry "Albanian (sq)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=sq_AL.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Amharic (am)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=am_ET "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Arabic (ar)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ar_EG.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Asturian (ast)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ast_ES.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Basque (eu)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=eu_ES.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Belarusian (be)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=be_BY.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Bangla (bn)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bn_BD "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Bosnian (bs)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bs_BA.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Bulgarian (bg)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bg_BG.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Tibetan (bo)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=bo_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "C (C)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=C "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Catalan (ca)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ca_ES.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Chinese (Simplified) (zh_CN)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=zh_CN.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Chinese (Traditional) (zh_TW)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=zh_TW.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Croatian (hr)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=hr_HR.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Czech (cs)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=cs_CZ.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Danish (da)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=da_DK.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Dutch (nl)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=nl_NL.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Dzongkha (dz)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=dz_BT "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "English (en)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=en_US.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Esperanto (eo)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=eo.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Estonian (et)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=et_EE.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Finnish (fi)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=fi_FI.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "French (fr)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=fr_FR.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Galician (gl)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=gl_ES.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Georgian (ka)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ka_GE.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "German (de)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=de_DE.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Greek (el)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=el_GR.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Gujarati (gu)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=gu_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Hebrew (he)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=he_IL.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Hindi (hi)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=hi_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Hungarian (hu)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=hu_HU.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Icelandic (is)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=is_IS.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Indonesian (id)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=id_ID.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Irish (ga)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ga_IE.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Italian (it)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=it_IT.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Japanese (ja)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ja_JP.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Kazakh (kk)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=kk_KZ.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Khmer (km)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=km_KH "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Kannada (kn)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=kn_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Korean (ko)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ko_KR.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Kurdish (ku)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ku_TR.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Lao (lo)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=lo_LA "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Latvian (lv)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=lv_LV.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Lithuanian (lt)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=lt_LT.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Malayalam (ml)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ml_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Marathi (mr)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=mr_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Macedonian (mk)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=mk_MK.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Burmese (my)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=my_MM "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Nepali (ne)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ne_NP "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Northern Sami (se_NO)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=se_NO "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Norwegian Bokmaal (nb_NO)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=nb_NO.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Norwegian Nynorsk (nn_NO)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=nn_NO.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Persian (fa)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=fa_IR "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Polish (pl)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=pl_PL.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Portuguese (pt)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=pt_PT.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Portuguese (Brazil) (pt_BR)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=pt_BR.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Punjabi (Gurmukhi) (pa)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=pa_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Romanian (ro)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ro_RO.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Russian (ru)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ru_RU.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Sinhala (si)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=si_LK "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Serbian (Cyrillic) (sr)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=sr_RS "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Slovak (sk)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=sk_SK.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Slovenian (sl)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=sl_SI.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Spanish (es)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=es_ES.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Swedish (sv)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=sv_SE.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Tagalog (tl)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=tl_PH.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Tamil (ta)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ta_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Telugu (te)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=te_IN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Tajik (tg)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=tg_TJ.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Thai (th)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=th_TH.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Turkish (tr)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=tr_TR.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Uyghur (ug)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=ug_CN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Ukrainian (uk)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=uk_UA.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Vietnamese (vi)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=vi_VN "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
menuentry "Welsh (cy)" {
  linux  /live/vmlinuz-4.9.0-4-amd64 boot=live components locales=cy_GB.UTF-8 "${loopback}"
  initrd /live/initrd.img-4.9.0-4-amd64
}
}
menuentry "Graphical Debian Installer" {
  linux  /d-i/gtk/vmlinuz append video=vesa:ywrap,mtrr vga=788 "${loopback}"
  initrd /d-i/gtk/initrd.gz
}
menuentry "Debian Installer" {
  linux  /d-i/vmlinuz  "${loopback}"
  initrd /d-i/initrd.gz
}
menuentry "Debian Installer with Speech Synthesis" {
  linux  /d-i/gtk/vmlinuz speakup.synth=soft "${loopback}"
  initrd /d-i/gtk/initrd.gz
}
-------------------------------------------------------------------------

> Now what works :
> [...]
> - Plus the grub test keys you asked me to create and test in your
> previous mails.

This run too ?

  grub-mkrescue -o output.iso minimal \
                --xorriso=...path.../grub-mkrescue-sed.sh \
                -partition_offset 16  

Its success would remove the last possibility that the partition table
type is involved.


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi,

Le 31/01/2018 à 13:20, Thomas Schmitt a écrit :
> This run too ?
>
>   grub-mkrescue -o output.iso minimal \
>                 --xorriso=...path.../grub-mkrescue-sed.sh \
>                 -partition_offset 16  
>
I would assume that I need to create a "minimal" directory with a
"dummy" file inside like previously ?

However, I don't have any "grub-mkrescue-sed.sh" file on the system on
which I would try to build the USB key...

[root@tiru ~]# pacman -Ql grub | grep rescue
grub /usr/bin/grub-mkrescue
grub /usr/share/man/man1/grub-mkrescue.1.gz

Kind regards.

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

Michel Bouissou wrote:
> I would assume that I need to create a "minimal" directory with a
> "dummy" file inside like previously ?

Yes.
The experiment can well wait until you are back from travel.


> However, I don't have any "grub-mkrescue-sed.sh" file 

Get it from 
  https://dev.lovelyhq.com/libburnia/libisoburn/raw/master/frontend/grub-mkrescue-sed.sh

It belongs to the upstream sources of xorriso, where i am developer.
The GNU xorriso tarball has it as demo in its ./frontend directory.
You can trust its good intentions as much as the ones of xorriso.
File operations only affect the grub-mkrescue temporary directory and
the /tmp directory. Files get moved, not deleted.
(The practical consequences of running it are subject to the usual
 risk of bugs, i fear.)

It works with xorriso-1.4.4 or newer. Best with current release 1.4.8.

If your installed xorriso tells an older revision when run without any
arguments, then consider to get xorriso and grub-mkrescue-sed.sh by the
tarball at
  https://www.gnu.org/software/xorriso/#download
This xorriso needs no installation but only a place where to unpack
and compile
  tar xzf xorriso-1.4.8.tar.gz
  cd xorriso-1.4.8
  ./configure && make
If you run its fellow ./frontend/grub-mkrescue-sed.sh then it will make
use of the compiled binary ./xorriso/xorriso.
Existing installed xorriso and its libraries will not be affected.

----------------------------------------------------------------------------
Lengthy motivation:

The purpose of grub-mkrescue-sed.sh is to modify the xorriso run underneath
grub-mkrescue to get other partition layouts.

The native layout as defined by Vladimir Serbinenko and implemented by me
obviously works fine. Its main disadvantage is that it shows at least
two not mountable GPT partitions (e.g. /dev/sdc1 and /dev/sdc4).
The ISO 9660 filesystem can only be mounted by the base device /dev/sdc.

GPT has the further disadvantage that it prescribes to have a backup
partition table at the end of the storage device. But when bootable ISOs
are made, it is not yet known how large the device will be. And even if so,
one would have to pad up the image to write that backup GPT.
So any ISO copied onto USB stick has its backup GPT at the wrong place
and any remnant backup GPT at the correct position would be in conflict
with the main GPT at image start.

The MBR partition table doesn't refer to the device end. UEFI specifies
that a MBR partition of type 0xEF shall be regarded as EFI System
Partition.

By some extra measures, the default mode of grub-mkrescue-sed.sh achieves
that all partitions are mountable and that no nested partitions emerge.
This partition table is fully acceptable to MBR partition editors like
fdisk which can then be used to add partitions and so give the rest of
the USB stick a life.
Because of the 64 block gap before partition 1, it is even possible to
install a new boot loader and use the ISO as read-only data partition
rather than as home of the operating system.

The same measures can help to make GPT with mountable ISO partition
(see description of "gpt_appended" in the script). But the problem of
misplaced backup GPT cannot be solved at the time when the ISO gets
produced.

Most modes avoid HFS+ partition and Apple Partition Map. Partition editors
and operating systems do not expect APM to be combined with MBR or GPT
by some hot x86 machine code stunt and an unusual APM block size. 


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

it comes to me that possibly Michel's grub-mkrescue run was BIOS-only.
So Michel, take much care to come back in good shape. We have experiments
to do. :))

-----------------------------------------------------------------------

At least on Debian the boot equipment prepared by grub-mkrescue depends
on which grub-* Debian packages are installed.

"grub-pc" enables BIOS equipment. In the case of an USB stick, it's the
MBR x86 boot code which comes into effect. It knows the block address of
the El Torito boot image. So i assume it loads and executes that binary.

"grub-efi-amd64" causes grub-mkrescue to prepare an EFI System Partition
with binary /efi/boot/bootx64.efi . 

"grub-efi-ia32" causes an EFI System Partition with binary
/efi/boot/bootia32.efi .

All three can be combined.

So after a vanilla run with grub-mkrescue, it is essential to inspect
the partition tables and El Torito boot equipment by

  xorriso -indev output.iso \
          -report_el_torito plain -report_system_area plain

If no lines like
  GPT disk GUID      :      04a5adf35d1adb4382bf8300bebe08a1
  ...
  GPT start and size :   4  34132  600
are reported, then the ISO has no EFI System Partition.

In this case a run with grub-mkrescue-sed.sh will not make much sense.
First one will have to configure GRUB to enable at least one of the EFI
variants. I dimly remember that the machine was 64 bit, i.e. should run
bootx64.efi.


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi,

Le 31/01/2018 à 22:37, Thomas Schmitt a écrit :
> 
> In this case a run with grub-mkrescue-sed.sh will not make much sense.
> First one will have to configure GRUB to enable at least one of the EFI
> variants. I dimly remember that the machine was 64 bit, i.e. should run
> bootx64.efi.

I'm happy to report that the USB stick I made today with
grub-mkrescue-sed.sh does boot alright on tested machines.

As a side note, the machine on wich I made it is a Manjaro that boots in
UEFI 64 mode, so for sure it's grub is UEFI-capable.

The HP tested machine has, AFAIK, no CSM "legacy" boot mode at all and I
believe it doesn't boot anything but UEFI.

That said, I don't have the machine on which I made the stick at hand
and I made the ISO image in a tmpfs, so...

But directly analyzing the USB stick I made, on another (Arch) machine
now gives :


root@zafu:/# xorriso -indev stdio:/dev/sdb -report_el_torito plain
-report_system_area plain
xorriso 1.4.8 : RockRidge filesystem manipulator, libburnia project.

xorriso : NOTE : Loading ISO image tree from LBA 0
xorriso : UPDATE : 632 nodes read in 1 seconds
libisofs: WARNING : Found hidden El-Torito image. Its size could not be
figured out, so image modify or boot image patching may lead to bad results.
xorriso : NOTE : Detected El-Torito boot information which currently is
set to be discarded
Drive current: -indev 'stdio:/dev/sdb'
Media current: stdio file, overwriteable
Media status : is written , is appendable
Boot record  : El Torito , MBR grub2-mbr cyl-align-off
Media summary: 1 session, 8594 data blocks, 16.8m data, 14.8g free
Volume id    : 'ISOIMAGE'
El Torito catalog  : 141  1
El Torito cat path : /boot.catalog
El Torito images   :   N  Pltf  B   Emul  Ld_seg  Hdpt  Ldsiz         LBA
El Torito boot img :   1  BIOS  y   none  0x0000  0x00      4        1487
El Torito boot img :   2  UEFI  y   none  0x0000  0x00   5760        7154
El Torito img path :   1  /boot/grub/i386-pc/eltorito.img
El Torito img opts :   1  boot-info-table grub2-boot-info
El Torito img blks :   2  1440
System area options: 0x00004a00
System area summary: MBR grub2-mbr cyl-align-off
ISO image size/512 : 34376
Partition offset   : 16
MBR heads per cyl  : 64
MBR secs per head  : 32
MBR partition table:   N Status  Type        Start       Blocks
MBR partition      :   1   0x80  0x00           64        28552
MBR partition      :   2   0x00  0xef        28616         5760


Hope this helps (although I don't get much out of this myself ;-)

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

Michel Bouissou wrote:
> I'm happy to report that the USB stick I made today with
> grub-mkrescue-sed.sh does boot alright on tested machines.

So GRUB itself is ok with the machine. GPT and MBR partitions alike.


> The HP tested machine has, AFAIK, no CSM "legacy" boot mode at all and I
> believe it doesn't boot anything but UEFI.

It booted the tails ISO which has no EFI boot equipment but only
an ISOLINUX El Torito boot image for BIOS and an ISOLINUX isohybrid MBR
with machine code which BIOS will execute when presented on USB stick.

The machine's firmware probably falls back to legacy mode when detecting
BIOS stuff but no EFI partition.

-------------------------------------------------------------------------

What only do our users at the distro ISO projects miss which grub-mkrescue
does right ?

Is it simply the invalid GPT which Matthew J. Garrett introduced
to get some of his test machines booting ? (Or did he introduce the MBR
partition without noticing that this invalidates GPT ?)
See
  "Anatomy of a Fedora 17 ISO image"
  https://mjg59.dreamwidth.org/11285.html
preceeded by
  "Further adventures in EFI booting"
  https://mjg59.dreamwidth.org/4957.html
The EFI images, which he and nearly all others use, are made with GRUB2.

grub-mkrescue does a similar stunt about x86 machine code without
side effects and the Block0 of an Apple Partition Map. But it does not
combine a non-protective MBR with GPT.

Michel - on his way to India - could zeroize 512-byte block 1 of the
Debian Live ISO and then try to boot it from USB stick

  dd if=/dev/zero conv=notrunc bs=512 seek=1 count=1 of=...image.or.stick...

where "...image.or.stick..." mabe either something like debian-live-*.iso
or /dev/sdc.
This would further deface the GPT and possibly de-confuse the firmware.

Michel, i will try to remember to remind you, when you are back. :))

-------------------------------------------------------------------------
The partition tables of modified and vanilla grub-mkrescue ISOs:

This is the modified one from grub-mkrescue-sed.sh :

> root@zafu:/# xorriso -indev stdio:/dev/sdb -report_el_torito plain -report_system_area plain
> ...
> El Torito images   :   N  Pltf  B   Emul  Ld_seg  Hdpt  Ldsiz         LBA
> El Torito boot img :   1  BIOS  y   none  0x0000  0x00      4        1487
> El Torito boot img :   2  UEFI  y   none  0x0000  0x00   5760        7154
> El Torito img path :   1  /boot/grub/i386-pc/eltorito.img
> El Torito img opts :   1  boot-info-table grub2-boot-info

We see that it's with EFI equipment.
(LBA counts blocks of 2048 bytes.  Ldsiz counts blocks of 512 bytes.
 Blame IBM and Phoenix, not me.)


> MBR partition table:   N Status  Type        Start       Blocks
> MBR partition      :   1   0x80  0x00           64        28552
> MBR partition      :   2   0x00  0xef        28616         5760

(Here all block numbers are counted with 512 bytes per block.)

By the options manipulation of grub-mkrescue-sed.sh the EFI partition
image was appended after the mountable ISO 9660 partition. Other than
with vanilla grub-mkrescue both partitions are marked in the MBR partition
table.
Partition 1 has the Boot/Active flag set, because some firmwares ignore
media which do not have it on any partition.
Partition 2 is the EFI partition. The same bytes as the El Torito UEFI
boot image. (7154 * 2048 / 512 = 28616)

Vanilla grub-mkrescue would rather look like:

  El Torito images   :   N  Pltf  B   Emul  Ld_seg  Hdpt  Ldsiz         LBA
  El Torito boot img :   1  BIOS  y   none  0x0000  0x00      4        4047
  El Torito boot img :   2  UEFI  y   none  0x0000  0x00   5760          85
  El Torito img path :   1  /boot/grub/i386-pc/eltorito.img
  El Torito img opts :   1  boot-info-table grub2-boot-info
  El Torito img path :   2  /efi.img
  ...
  MBR partition table:   N Status  Type        Start       Blocks
  MBR partition      :   1   0x00  0xee            1        34779
  GPT                :   N  Info
  GPT disk GUID      :      04a5adf35d1adb4382bf8300bebe08a1
  GPT entry array    :      20  176  separated
  GPT lba range      :      64  34734  34779
  GPT partition name :   1  4700610070003000
  GPT partname local :   1  Gap0
  ...
  GPT partition name :   2  450046004900200062006f006f007400200070006100720074006900740069006f006e00
  GPT partname local :   2  EFI boot partition
  GPT partition GUID :   2  04a5adf35d1adb4382bd8300bebe08a1
  GPT type GUID      :   2  28732ac11ff8d211ba4b00a0c93ec93b
  GPT partition flags:   2  0x1000000000000001
  GPT start and size :   2  340  5760
  GPT partition path :   2  /efi.img
  ...
  GPT start and size :   4  34132  600

plus maybe

  APM                :   N  Info
  APM block size     :      2048
  ...

Other than with the mbr_only version, the EFI partition is a data file
inside the ISO 9660 filesystem: /efi.img

247 lines of explanation of the report format are available with
  xorriso -report_system_area help -report_el_torito help | less


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi,

Le 01/02/2018 à 01:02, Thomas Schmitt a écrit :
> Michel - on his way to India - could zeroize 512-byte block 1 of the
> Debian Live ISO and then try to boot it from USB stick
> 
>   dd if=/dev/zero conv=notrunc bs=512 seek=1 count=1 of=...image.or.stick...
> 
> where "...image.or.stick..." mabe either something like debian-live-*.iso
> or /dev/sdc.
> This would further deface the GPT and possibly de-confuse the firmware.

I made it (with latest debian-gnome 9.3 live), and tested it OK on
*another* machine.

But on the machine where the problem is, it does *NOT* boot.

It does display the key brand/model (instead of "Windows boot manager")
but when selecting it, the punishment is as usual : black screen with
cursor, game over.

No grub message or output of any kind.

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

i wrote:
> > zeroize 512-byte block 1 of the
> > Debian Live ISO and then try to boot it from USB stick

Michel Bouissou wrote:
> I made it (with latest debian-gnome 9.3 live),i

Now we have no surplus GPT any more. The EFI system partition is only
marked by the MBR partition 2 (and by the El Torito catalog which should
be ignored when found on USB stick).


> and tested it OK on *another* machine.
> But on the machine where the problem is, it does *NOT* boot.

Normally i would now again exclude the theory that the partitions inside
the ISO are to blame. (We still have to explain why the GRUB installations
on hard-disk-like partitioned USB sticks failed too.)


> It does display the key brand/model (instead of "Windows boot manager")

But this slight change in firmware behavior could be significant.

Under the theory that it's the partition layout and not the EFI partition
image content or its way to hop on the ISO, there remains the nested
partition situation as main suspect:

  $ /sbin/fdisk -l debian-live-9.3.0-amd64-xfce.iso
  ...
  Device                            Boot Start     End Sectors  Size Id Type
  debian-live-9.3.0-amd64-xfce.iso1 *        0 3803903 3803904  1.8G  0 Empty
  debian-live-9.3.0-amd64-xfce.iso2       1548    2379     832  416K ef EFI

The block range of partition 2 is inside partition 1.
Normally the type 0 ("Empty") of partition 1 is enough to let firmwares
tolerate this.

Partition 1 is not needed for the boot process. It just protects the ISO
from partition editors and provides a mountable partition to operating
systems.

So the next experiment would be to remove it, i.e. zeroize its slot in
the MBR.

One can do it in the ISO file
  img=debian-live-9.3.0-amd64-xfce.iso
or directly on the USB stick where it was copied
  img=/dev/sdc

Either by:
  /sbin/fdisk "$img" <<+
  d
  1
  w
  +

Or by:
  dd if=/dev/zero bs=1 seek=446 count=16 conv=notrunc of="$img"


The result should then be
  $ /sbin/fdisk -l "$img"
  ...
  Device                            Boot Start     End Sectors  Size Id Type
  debian-live-9.3.0-amd64-xfce.iso2       1548    2379     832  416K ef EFI 


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Michel Bouissou]

Hi Thomas,


Le 02/02/2018 à 09:15, Thomas Schmitt a écrit :
> So the next experiment would be to remove it, i.e. zeroize its slot in
> the MBR.
Well, my son left this morning for the week-end with the machine, and I
will fly to India myself before I get a chance to do any further testing.

So I'm afraid we'll have to wait for season 2 by early march to see the
continuation of the story ;-)

Kind regards and thanks again for all your help and suggestions.

ॐ

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xEB04D09C

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

Michel Bouissou wrote:
> So I'm afraid we'll have to wait for season 2 by early march to see the
> continuation of the story ;-)

A classical cliff hanger. :))


> Kind regards and thanks again for all your help and suggestions.

Thanks to you for being a very endurant bug reporter.
And thanks to your son for letting his dad play with the computer.
Mine would have met more resistence.

-----------------------------------------------------------------------

(I only got one copy of this mail. Probably the Cc: for me.
 The list seems not to work for me as receiver or for the archive
   http://lists.gnu.org/archive/html/grub-devel/2018-02/threads.html
  Michel obviously got my mail from 09:15:42 +0100.
)


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Thomas Schmitt]

Hi,

about two hours ago i sent an answer to
  Date: Thu, 1 Feb 2018 22:56:26 +0100
  From: Michel Bouissou <michel@bouissou.net>
  Subject: Re: A thread on grub-bug could need attention
with own
  Date: Fri, 02 Feb 2018 09:15:42 +0100
  Message-Id: <28716785462546254838@scdbackup.webframe.org>

It did not show up in the mailing list yet. Neither in my mailbox nor at
  http://lists.gnu.org/archive/html/grub-devel/2018-02/threads.html

This is an attempt to resend it. I again Cc: Michel Bouissou because i
don't know whether the problem is with my mail provider.

-----------------------------------------------------------------------------

Hi,

i wrote:
> > zeroize 512-byte block 1 of the
> > Debian Live ISO and then try to boot it from USB stick

Michel Bouissou wrote:
> I made it (with latest debian-gnome 9.3 live),i

Now we have no surplus GPT any more. The EFI system partition is only
marked by the MBR partition 2 (and by the El Torito catalog which should
be ignored when found on USB stick).


> and tested it OK on *another* machine.
> But on the machine where the problem is, it does *NOT* boot.

Normally i would now again exclude the theory that the partitions inside
the ISO are to blame. (We still have to explain why the GRUB installations
on hard-disk-like partitioned USB sticks failed too.)


> It does display the key brand/model (instead of "Windows boot manager")

But this slight change in firmware behavior could be significant.

Under the theory that it's the partition layout and not the EFI partition
image content or its way to hop on the ISO, there remains the nested
partition situation as main suspect:

  $ /sbin/fdisk -l debian-live-9.3.0-amd64-xfce.iso
  ...
  Device                            Boot Start     End Sectors  Size Id Type
  debian-live-9.3.0-amd64-xfce.iso1 *        0 3803903 3803904  1.8G  0 Empty
  debian-live-9.3.0-amd64-xfce.iso2       1548    2379     832  416K ef EFI

The block range of partition 2 is inside partition 1.
Normally the type 0 ("Empty") of partition 1 is enough to let firmwares
tolerate this.

Partition 1 is not needed for the boot process. It just protects the ISO
from partition editors and provides a mountable partition to operating
systems.

So the next experiment would be to remove it, i.e. zeroize its slot in
the MBR.

One can do it in the ISO file
  img=debian-live-9.3.0-amd64-xfce.iso
or directly on the USB stick where it was copied
  img=/dev/sdc

Either by:
  /sbin/fdisk "$img" <<+
  d
  1
  w
  +

Or by:
  dd if=/dev/zero bs=1 seek=446 count=16 conv=notrunc of="$img"


The result should then be
  $ /sbin/fdisk -l "$img"
  ...
  Device                            Boot Start     End Sectors  Size Id Type
  debian-live-9.3.0-amd64-xfce.iso2       1548    2379     832  416K ef EFI 


Have a nice day :)

Thomas

Re: A thread on grub-bug could need attention

[Daniel Kiper]

On Wed, Jan 31, 2018 at 10:38:03AM +0100, Michel Bouissou wrote:
> Hello,
>
> Le 30/01/2018 ?? 20:15, Thomas Schmitt a ??crit??:
> > It also does not work when booting the ISO images which shall install
> > the systems.
> > E.g. debian-9.3.0-amd64-netinst.iso :
> Yes. AFAIR I had tested all the latest (in december) standard
> installation media for at least :
> - Manjaro Cinnamon x64 (created either with dd or unetbootin)
> - Mint Cinnamon x64 (created either with dd or unetbootin)
> - Ubuntu 17.10
> - Debian live
> - Parted Magic (created with dd)
> (and maybe a few more, see my first mails...)
>
> - Manjaro Cinnamon after it being "normally" installed on a USB key
> instead of an HD on another machine - and tested working.
>
> None of them booted, all showed the same behaviour :
> - The machine's UEFI BIOS displays the USB Key as ?? Windows bootloader ??
> (and the true on-HD Windows bootloader bears the same label)
> - Selecting is causes the screen to go black with a cursor on the top
> left and nothing else ever happens.
>
> Now what works :
> - rEFInd USB key (made with dd)
> - Knoppix USB key (made with dd from DVD image)
> - Tails USB key (both made with dd or from the Tails installer)
> - Manjaro install key created with unetbootin THEN manually doctored to
> remove grub and install syslinux instead.
>
> - Plus the grub test keys you asked me to create and test in your
> previous mails.
>
> Interestingly, for "working" keys, the machines UEFI?? displays the USB
> key brand / model instead of ?? Windows bootloader ??

Could you check partitioning type used on each of above mentioned devices?
Is it MBR or GPT? Please remember that GPT have to be always protected by
something called "protective MBR". So, you will see MBR everywhere but in
case of GPT it should contain one entry with 0xee (GPT) type.

> Even more interesting, keys that do NOT work do not work either when
> chained to from rEFInd (inserting both keys, booting on rEFInd then
> chaining to the other key), with the same symptoms - where keys that
> work, also work when performing the same test.
>
> As I side note I'm currently packing for a trip to India and will be
> unable to perform any further tests from next week to the end of
> february (and will have very little time left for doing so before leaving)
>
> Have a nice day :)

Have a nice vacation! Drop us a line when you are back.

Daniel