"error: invalid magic number" when booting from ISO loopback

"error: invalid magic number" when booting from ISO loopback

[Ron]

Hi,

I was playing around with Grub2 (latest svn version) and its loopback
feature and had some problems to boot an ISO. The ISO file is the 32Bit
version of Ubuntu 9.10 which is located on a 8GB USB flash drive with
one FAT32 partition. I've tried to boot it with the following menu
entry:

menuentry "Ubuntu" {
    set isofile="/boot/isos/ubuntu-9.10-desktop-i386.iso"

    loopback loop $isofile 
    linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile
quiet splash noprompt --
    initrd (loop)/casper/initrd.lz
}

This entry fails silently and when the commands were executed one by one
the linux command shows the error message "error: invalid magic number".

A short google search gave me the following similar bug report but no
solution:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=543924

I believe that Grub2 has some problems when the file is located nearly
at the end of the filesystem because I've put the file on the drive when
it was nearly full. Also I've checked the md5 value of the file under
Linux and it was correct but under Grub2 a crc value check shows a
mismatch. But the most interesting fact is that after I've deleted some
other files to make some space I made a copy from the ISO file on the
USB flash drive which booted without any problems.

Re: "error: invalid magic number" when booting from ISO loopback

[Robert Millan]

On Sat, Oct 31, 2009 at 06:01:56PM +0100, Ron wrote:
> Hi,
> 
> I was playing around with Grub2 (latest svn version) and its loopback
> feature and had some problems to boot an ISO. The ISO file is the 32Bit
> version of Ubuntu 9.10 which is located on a 8GB USB flash drive with
> one FAT32 partition. I've tried to boot it with the following menu
> entry:
> 
> menuentry "Ubuntu" {
>     set isofile="/boot/isos/ubuntu-9.10-desktop-i386.iso"
> 
>     loopback loop $isofile 
>     linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile
> quiet splash noprompt --
>     initrd (loop)/casper/initrd.lz
> }
> 
> This entry fails silently and when the commands were executed one by one
> the linux command shows the error message "error: invalid magic number".

This error means vmlinuz wasn't read correctly.

This could be either a problem in loopback/iso9660 or a problem in the
partition that contains ubuntu-9.10-desktop-i386.iso.

Can you figure out which applies?  For example, if you put a physical CD
with ubuntu-9.10-desktop-i386.iso in the drive and access it from
GRUB (via ata.mod), are you able to load Linux this way?

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Slow grub boot when /boot/grub is not on first partition

[Simon Wagner]

Hello dear GRUB2 developers,

I am a user of Ubuntu 9.10, which uses GRUB2 1.97. Unfortunately GRUB 
needs a rather long time loading the modules. For 2 minutes or so it 
just displays "GRUB loading..." until the boot menu is shown and I can 
start Ubuntu.

The bug has already been reported at Launchpad: 
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/420933

I tried to find out what causes the problem. I have the following disk 
layout:

A "Windows HD" with the following partitions
/dev/sda1
/dev/sda2
/dev/sda3
/dev/sda4
/dev/sda5

A "Ubuntu HD":
/dev/sdb2 (Extended Partition)
/dev/sdb5 (swap)
/dev/sdb6 (small partition, holds some misc data)
/dev/sdb7 (Ubuntu 9.10)
/dev/sdb8 (Backup)

So when I add
    set debug="disk"
at grub.cfg I can see grub accessing the disk. It cycles through all the 
partitions, then reads data from sdb7, cycles again through the 
partitions, reads again from sdb7, and so on...

I can't really tell whats causing this, I used the Ubuntu 9.04 package 
of grub2 before and that had not this problem. The package dates back to 
to the grub2 version from the 24th July of 2008.

Maybe it is possible to do some caching in GRUB2? So for example, if
    search -u someuuid
is done, the result is saved, and if we do that again, we can lookup in 
the cache which drive has that uuid? The same for the FS type. If we 
once did detect ext2 on hd1,7 we should cache that, so we don't need to 
detect that again.

I will try to add more grub_dprintf calls, so that I can better see what 
is going on.

Sincerly
Simon W.

Re: Slow grub boot when /boot/grub is not on first partition

[Vladimir 'phcoder' Serbinenko]

Simon Wagner wrote:
> Hello dear GRUB2 developers,
>
> I am a user of Ubuntu 9.10, which uses GRUB2 1.97. Unfortunately GRUB
> needs a rather long time loading the modules. For 2 minutes or so it
> just displays "GRUB loading..." until the boot menu is shown and I can
> start Ubuntu.
>
> The bug has already been reported at Launchpad:
> https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/420933
>
> I tried to find out what causes the problem. I have the following disk
> layout:
>
> A "Windows HD" with the following partitions
> /dev/sda1
> /dev/sda2
> /dev/sda3
> /dev/sda4
> /dev/sda5
>
> A "Ubuntu HD":
> /dev/sdb2 (Extended Partition)
> /dev/sdb5 (swap)
> /dev/sdb6 (small partition, holds some misc data)
> /dev/sdb7 (Ubuntu 9.10)
> /dev/sdb8 (Backup)
>
> So when I add
>    set debug="disk"
> at grub.cfg I can see grub accessing the disk. It cycles through all
> the partitions, then reads data from sdb7, cycles again through the
> partitions, reads again from sdb7, and so on...
>
> I can't really tell whats causing this, I used the Ubuntu 9.04 package
> of grub2 before and that had not this problem. The package dates back
> to to the grub2 version from the 24th July of 2008.
>
> Maybe it is possible to do some caching in GRUB2? So for example, if
>    search -u someuuid
> is done, the result is saved, and if we do that again, we can lookup
> in the cache which drive has that uuid? The same for the FS type. If
> we once did detect ext2 on hd1,7 we should cache that, so we don't
> need to detect that again.
>
> I will try to add more grub_dprintf calls, so that I can better see
> what is going on.
>
> Sincerly
> Simon W.
>
We're aware of this problem and have patch which I'll merge as soon as
I'm back home
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>


-- 
Regards
Vladimir 'phcoder' Serbinenko
Personal git repository: http://repo.or.cz/w/grub2/phcoder.git 

Re: Slow grub boot when /boot/grub is not on first partition

[Felix Zielcke]

Am Samstag, den 31.10.2009, 20:03 +0100 schrieb Vladimir 'phcoder'
Serbinenko:
> Simon Wagner wrote:
> > Hello dear GRUB2 developers,
> >
> > I am a user of Ubuntu 9.10, which uses GRUB2 1.97. Unfortunately
> GRUB
> > needs a rather long time loading the modules. For 2 minutes or so it
> > just displays "GRUB loading..." until the boot menu is shown and I
> can
> > start Ubuntu.
> >
> > The bug has already been reported at Launchpad:
> > https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/420933
> >
> > I tried to find out what causes the problem. I have the following
> disk
> > layout:
> >
> > A "Windows HD" with the following partitions
> > /dev/sda1
> > /dev/sda2
> > /dev/sda3
> > /dev/sda4
> > /dev/sda5
> >
> > A "Ubuntu HD":
> > /dev/sdb2 (Extended Partition)
> > /dev/sdb5 (swap)
> > /dev/sdb6 (small partition, holds some misc data)
> > /dev/sdb7 (Ubuntu 9.10)
> > /dev/sdb8 (Backup)
> >
> > So when I add
> >    set debug="disk"
> > at grub.cfg I can see grub accessing the disk. It cycles through all
> > the partitions, then reads data from sdb7, cycles again through the
> > partitions, reads again from sdb7, and so on...
> >
> > I can't really tell whats causing this, I used the Ubuntu 9.04
> package
> > of grub2 before and that had not this problem. The package dates
> back
> > to to the grub2 version from the 24th July of 2008.
> >
> > Maybe it is possible to do some caching in GRUB2? So for example, if
> >    search -u someuuid
> > is done, the result is saved, and if we do that again, we can lookup
> > in the cache which drive has that uuid? The same for the FS type. If
> > we once did detect ext2 on hd1,7 we should cache that, so we don't
> > need to detect that again.
> >
> > I will try to add more grub_dprintf calls, so that I can better see
> > what is going on.
> >
> > Sincerly
> > Simon W.
> >
> We're aware of this problem and have patch which I'll merge as soon as
> I'm back home

Vladimir any news?
The people on the LP bug now seem to more revert to Legacy instead of
following the advice to boot from the disk where their /boot/grub is on.

-- 
Felix Zielcke
Proud Debian Maintainer and GNU GRUB developer

Re: "error: invalid magic number" when booting from ISO loopback

[Ron]

On Sat, 2009-10-31 at 18:30 +0100, Robert Millan wrote:
> On Sat, Oct 31, 2009 at 06:01:56PM +0100, Ron wrote:
> > Hi,
> > 
> > I was playing around with Grub2 (latest svn version) and its loopback
> > feature and had some problems to boot an ISO. The ISO file is the 32Bit
> > version of Ubuntu 9.10 which is located on a 8GB USB flash drive with
> > one FAT32 partition. I've tried to boot it with the following menu
> > entry:
> > 
> > menuentry "Ubuntu" {
> >     set isofile="/boot/isos/ubuntu-9.10-desktop-i386.iso"
> > 
> >     loopback loop $isofile 
> >     linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile
> > quiet splash noprompt --
> >     initrd (loop)/casper/initrd.lz
> > }
> > 
> > This entry fails silently and when the commands were executed one by one
> > the linux command shows the error message "error: invalid magic number".
> 
> This error means vmlinuz wasn't read correctly.
> 
> This could be either a problem in loopback/iso9660 or a problem in the
> partition that contains ubuntu-9.10-desktop-i386.iso.
> 
> Can you figure out which applies?  For example, if you put a physical CD
> with ubuntu-9.10-desktop-i386.iso in the drive and access it from
> GRUB (via ata.mod), are you able to load Linux this way?
> 

I'm sorry but right now I can't burn a CD. But I made some more test
with the ISO file that doesn't work and the copy from it that does work.
As I've mentioned both files shows the same correct MD5 sum under Linux.
Also GRUB shows with the ls command the same file size of 723488768
Bytes (690 MiB) for both of them. But the output of the blocklist
command is looking wrong.

blocklist output for the ISO that doesn't work:
11833144+161304, 13766112+8, 13853680+218800, 14072488+118288

blocklist output for the ISO that does work:
15128592+328728, 11652008+16, 11654088+1792, 11655888+8,11655912+440,
12089440+16, 12404120+126080, 12530256+99464, 12629728+761984, 13413432
+94536

So if you sum up the sectors from the ISO that doesn't work and multiply
it with 512 you get only 255180800 Bytes (243 MiB). It looks like the
blocklist command encounters an error and exits before it is done, but
no error message is visible.

Some more info that may help.

blocklist output from the vmlinuz file from the mounted ISO file (both
ISO files show the same output):
1384372+7598, 1391970[0-224]

fdisk -lu output from the USB flash drive:
Disk /dev/sdb: 8086 MB, 8086618112 bytes
255 heads, 63 sectors/track, 983 cylinders, total 15794176 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x2c6b7369

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *          63    15791894     7895916    b  W95 FAT32

Re: "error: invalid magic number" when booting from ISO loopback

[Vladimir 'phcoder' Serbinenko]

Ron wrote:
> On Sat, 2009-10-31 at 18:30 +0100, Robert Millan wrote:
>   
>> On Sat, Oct 31, 2009 at 06:01:56PM +0100, Ron wrote:
>>     
>>> Hi,
>>>
>>> I was playing around with Grub2 (latest svn version) and its loopback
>>> feature and had some problems to boot an ISO. The ISO file is the 32Bit
>>> version of Ubuntu 9.10 which is located on a 8GB USB flash drive with
>>> one FAT32 partition. I've tried to boot it with the following menu
>>> entry:
>>>
>>> menuentry "Ubuntu" {
>>>     set isofile="/boot/isos/ubuntu-9.10-desktop-i386.iso"
>>>
>>>     loopback loop $isofile 
>>>     linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile
>>> quiet splash noprompt --
>>>     initrd (loop)/casper/initrd.lz
>>> }
>>>
>>> This entry fails silently and when the commands were executed one by one
>>> the linux command shows the error message "error: invalid magic number".
>>>       
>> This error means vmlinuz wasn't read correctly.
>>
>> This could be either a problem in loopback/iso9660 or a problem in the
>> partition that contains ubuntu-9.10-desktop-i386.iso.
>>
>> Can you figure out which applies?  For example, if you put a physical CD
>> with ubuntu-9.10-desktop-i386.iso in the drive and access it from
>> GRUB (via ata.mod), are you able to load Linux this way?
>>
>>     
>
> I'm sorry but right now I can't burn a CD. But I made some more test
> with the ISO file that doesn't work and the copy from it that does work.
> As I've mentioned both files shows the same correct MD5 sum under Linux.
> Also GRUB shows with the ls command the same file size of 723488768
> Bytes (690 MiB) for both of them. But the output of the blocklist
> command is looking wrong.
>
> blocklist output for the ISO that doesn't work:
> 11833144+161304, 13766112+8, 13853680+218800, 14072488+118288
>
> blocklist output for the ISO that does work:
> 15128592+328728, 11652008+16, 11654088+1792, 11655888+8,11655912+440,
> 12089440+16, 12404120+126080, 12530256+99464, 12629728+761984, 13413432
> +94536
>
> So if you sum up the sectors from the ISO that doesn't work and multiply
> it with 512 you get only 255180800 Bytes (243 MiB). It looks like the
> blocklist command encounters an error and exits before it is done, but
> no error message is visible.
>
>   
Can you disable offset checking in kern/disk.c and retry?
> Some more info that may help.
>
> blocklist output from the vmlinuz file from the mounted ISO file (both
> ISO files show the same output):
> 1384372+7598, 1391970[0-224]
>
> fdisk -lu output from the USB flash drive:
> Disk /dev/sdb: 8086 MB, 8086618112 bytes
> 255 heads, 63 sectors/track, 983 cylinders, total 15794176 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Disk identifier: 0x2c6b7369
>
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1   *          63    15791894     7895916    b  W95 FAT32
>
>   
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>
>   


-- 
Regards
Vladimir 'phcoder' Serbinenko
Personal git repository: http://repo.or.cz/w/grub2/phcoder.git 

Re: "error: invalid magic number" when booting from ISO loopback

[Ron]

On Sun, 2009-11-01 at 08:53 +0100, Vladimir 'phcoder' Serbinenko wrote:
> Ron wrote:
> > On Sat, 2009-10-31 at 18:30 +0100, Robert Millan wrote:
> >   
> >> On Sat, Oct 31, 2009 at 06:01:56PM +0100, Ron wrote:
> >>     
> >>> Hi,
> >>>
> >>> I was playing around with Grub2 (latest svn version) and its loopback
> >>> feature and had some problems to boot an ISO. The ISO file is the 32Bit
> >>> version of Ubuntu 9.10 which is located on a 8GB USB flash drive with
> >>> one FAT32 partition. I've tried to boot it with the following menu
> >>> entry:
> >>>
> >>> menuentry "Ubuntu" {
> >>>     set isofile="/boot/isos/ubuntu-9.10-desktop-i386.iso"
> >>>
> >>>     loopback loop $isofile 
> >>>     linux (loop)/casper/vmlinuz boot=casper iso-scan/filename=$isofile
> >>> quiet splash noprompt --
> >>>     initrd (loop)/casper/initrd.lz
> >>> }
> >>>
> >>> This entry fails silently and when the commands were executed one by one
> >>> the linux command shows the error message "error: invalid magic number".
> >>>       
> >> This error means vmlinuz wasn't read correctly.
> >>
> >> This could be either a problem in loopback/iso9660 or a problem in the
> >> partition that contains ubuntu-9.10-desktop-i386.iso.
> >>
> >> Can you figure out which applies?  For example, if you put a physical CD
> >> with ubuntu-9.10-desktop-i386.iso in the drive and access it from
> >> GRUB (via ata.mod), are you able to load Linux this way?
> >>
> >>     
> >
> > I'm sorry but right now I can't burn a CD. But I made some more test
> > with the ISO file that doesn't work and the copy from it that does work.
> > As I've mentioned both files shows the same correct MD5 sum under Linux.
> > Also GRUB shows with the ls command the same file size of 723488768
> > Bytes (690 MiB) for both of them. But the output of the blocklist
> > command is looking wrong.
> >
> > blocklist output for the ISO that doesn't work:
> > 11833144+161304, 13766112+8, 13853680+218800, 14072488+118288
> >
> > blocklist output for the ISO that does work:
> > 15128592+328728, 11652008+16, 11654088+1792, 11655888+8,11655912+440,
> > 12089440+16, 12404120+126080, 12530256+99464, 12629728+761984, 13413432
> > +94536
> >
> > So if you sum up the sectors from the ISO that doesn't work and multiply
> > it with 512 you get only 255180800 Bytes (243 MiB). It looks like the
> > blocklist command encounters an error and exits before it is done, but
> > no error message is visible.
> >
> >   
> Can you disable offset checking in kern/disk.c and retry?

I've looked at the source code but I'm unsure where this offset checking
is placed. Can you give me more infos?

Re: "error: invalid magic number" when booting from ISO loopback

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 729 bytes --]

On 10/31/2009 06:01 PM, Ron wrote:
> This entry fails silently and when the commands were executed one by one
> the linux command shows the error message "error: invalid magic number".
>
>   
This can be caused by BIOS cutting last sectors of the disk to make it
appear to have C/H/S geometry of some kind. Can you compare "ls -l" from
GRUB shell and
grub-fstest /dev/sdX ls -- -l
especially compare the total sector count in both cases.
> A short google search gave me the following similar bug report but no
> solution:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=543924
>
>   
This was on ext2 and the cause were triple indirect blocks



-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]