[Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Anatol Pomozov]

It is possible to create a 64 bit elf image that has valid multiboot header.
qemu should be able to boot such images.

Tested with homemade 64bit OS - now it boots fine with 'qemu -kernel'
and as a grub image.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
---
 hw/i386/multiboot.c | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/hw/i386/multiboot.c b/hw/i386/multiboot.c
index 663f35a658..cf1b4f5fb3 100644
--- a/hw/i386/multiboot.c
+++ b/hw/i386/multiboot.c
@@ -192,11 +192,6 @@ int load_multiboot(FWCfgState *fw_cfg,
         int kernel_size;
         fclose(f);
 
-        if (((struct elf64_hdr*)header)->e_machine == EM_X86_64) {
-            fprintf(stderr, "Cannot load x86-64 image, give a 32bit one.\n");
-            exit(1);
-        }
-
         kernel_size = load_elf(kernel_filename, NULL, NULL, &elf_entry,
                                &elf_low, &elf_high, 0, I386_ELF_MACHINE,
                                0, 0);
-- 
2.13.1

Re: [Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Anatol Pomozov]

+ more folks who made changes to hw/i386/multiboot.c

On Tue, Jun 6, 2017 at 9:41 PM, Anatol Pomozov <anatol.pomozov@gmail.com> wrote:
> It is possible to create a 64 bit elf image that has valid multiboot header.
> qemu should be able to boot such images.
>
> Tested with homemade 64bit OS - now it boots fine with 'qemu -kernel'
> and as a grub image.
>
> Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
> ---
>  hw/i386/multiboot.c | 5 -----
>  1 file changed, 5 deletions(-)
>
> diff --git a/hw/i386/multiboot.c b/hw/i386/multiboot.c
> index 663f35a658..cf1b4f5fb3 100644
> --- a/hw/i386/multiboot.c
> +++ b/hw/i386/multiboot.c
> @@ -192,11 +192,6 @@ int load_multiboot(FWCfgState *fw_cfg,
>          int kernel_size;
>          fclose(f);
>
> -        if (((struct elf64_hdr*)header)->e_machine == EM_X86_64) {
> -            fprintf(stderr, "Cannot load x86-64 image, give a 32bit one.\n");
> -            exit(1);
> -        }
> -
>          kernel_size = load_elf(kernel_filename, NULL, NULL, &elf_entry,
>                                 &elf_low, &elf_high, 0, I386_ELF_MACHINE,
>                                 0, 0);
> --
> 2.13.1
>

Re: [Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Adam Lackorzynski]

On Tue Jun 06, 2017 at 21:41:48 -0700, Anatol Pomozov wrote:
> It is possible to create a 64 bit elf image that has valid multiboot header.
> qemu should be able to boot such images.

But this 64bit image actually starts with 32bit code, right?
So it's a 32bit program and the check verifies that this is the case.



Adam

Re: [Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Anatol Pomozov]

+reply-all

On Thu, Jun 8, 2017 at 1:41 PM, Adam Lackorzynski
<adam@os.inf.tu-dresden.de> wrote:
>
> On Tue Jun 06, 2017 at 21:41:48 -0700, Anatol Pomozov wrote:
>> It is possible to create a 64 bit elf image that has valid multiboot header.
>> qemu should be able to boot such images.
>
> But this 64bit image actually starts with 32bit code, right?

Correct. The very first part of the startup code has to be 32bit.
After it sets "long mode" it can use 64bit instructions. To make sure
that the preamble has only 32bit instructions one have to use asm
directive such as ".code32".

Here is an example from LitleKernel sturtup code:

https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L50

.code32 tells assembler to treat following text as 32 bit code. And
later when it jumps into "long mode"

https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L214
one can use 64bit code.

> So it's a 32bit program and the check verifies that this is the case.

While preamble have to contain 32 only instructions the rest of the
image can perfectly contain 64bit code. Right now 64bit binary cannot
be run with "qemu-system-x86_64 -kernel". But the same binary runs
fine if packed with GRUB as iso.

I tried to hack around this restriction by adding
"OUTPUT_FORMAT(elf32-i386)" to the linker file and compiling project
with 64bit support. But GNU ld program crashed at Ubuntu 14.04. It
means not that many people use this code path. GNU ld compiled from
HEAD does not have this problem but now GDB is confused by the fact
that ELF contains 64bit code while header reports i386.

Practically there is no reason for this check as it prevents running
64bit binaries with "qemu-system-x86_64 -kernel".

Re: [Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Anatol Pomozov]

Hi Adam

Do these arguments sound reasonable to apply the patch?

On Thu, Jun 8, 2017 at 2:07 PM, Anatol Pomozov <anatol.pomozov@gmail.com> wrote:
> +reply-all
>
> On Thu, Jun 8, 2017 at 1:41 PM, Adam Lackorzynski
> <adam@os.inf.tu-dresden.de> wrote:
>>
>> On Tue Jun 06, 2017 at 21:41:48 -0700, Anatol Pomozov wrote:
>>> It is possible to create a 64 bit elf image that has valid multiboot header.
>>> qemu should be able to boot such images.
>>
>> But this 64bit image actually starts with 32bit code, right?
>
> Correct. The very first part of the startup code has to be 32bit.
> After it sets "long mode" it can use 64bit instructions. To make sure
> that the preamble has only 32bit instructions one have to use asm
> directive such as ".code32".
>
> Here is an example from LitleKernel sturtup code:
>
> https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L50
>
> .code32 tells assembler to treat following text as 32 bit code. And
> later when it jumps into "long mode"
>
> https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L214
> one can use 64bit code.
>
>> So it's a 32bit program and the check verifies that this is the case.
>
> While preamble have to contain 32 only instructions the rest of the
> image can perfectly contain 64bit code. Right now 64bit binary cannot
> be run with "qemu-system-x86_64 -kernel". But the same binary runs
> fine if packed with GRUB as iso.
>
> I tried to hack around this restriction by adding
> "OUTPUT_FORMAT(elf32-i386)" to the linker file and compiling project
> with 64bit support. But GNU ld program crashed at Ubuntu 14.04. It
> means not that many people use this code path. GNU ld compiled from
> HEAD does not have this problem but now GDB is confused by the fact
> that ELF contains 64bit code while header reports i386.
>
> Practically there is no reason for this check as it prevents running
> 64bit binaries with "qemu-system-x86_64 -kernel".

Re: [Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Adam Lackorzynski]

Hi,

On Tue Jun 13, 2017 at 17:05:41 -0700, Anatol Pomozov wrote:
> Do these arguments sound reasonable to apply the patch?

I'm not really convinced.

> On Thu, Jun 8, 2017 at 2:07 PM, Anatol Pomozov <anatol.pomozov@gmail.com> wrote:
> > +reply-all
> >
> > On Thu, Jun 8, 2017 at 1:41 PM, Adam Lackorzynski
> > <adam@os.inf.tu-dresden.de> wrote:
> >>
> >> On Tue Jun 06, 2017 at 21:41:48 -0700, Anatol Pomozov wrote:
> >>> It is possible to create a 64 bit elf image that has valid multiboot header.
> >>> qemu should be able to boot such images.
> >>
> >> But this 64bit image actually starts with 32bit code, right?
> >
> > Correct. The very first part of the startup code has to be 32bit.
> > After it sets "long mode" it can use 64bit instructions. To make sure
> > that the preamble has only 32bit instructions one have to use asm
> > directive such as ".code32".
> >
> > Here is an example from LitleKernel sturtup code:
> >
> > https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L50
> >
> > .code32 tells assembler to treat following text as 32 bit code. And
> > later when it jumps into "long mode"
> >
> > https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L214
> > one can use 64bit code.
> >
> >> So it's a 32bit program and the check verifies that this is the case.
> >
> > While preamble have to contain 32 only instructions the rest of the
> > image can perfectly contain 64bit code. Right now 64bit binary cannot
> > be run with "qemu-system-x86_64 -kernel". But the same binary runs
> > fine if packed with GRUB as iso.
> >
> > I tried to hack around this restriction by adding
> > "OUTPUT_FORMAT(elf32-i386)" to the linker file and compiling project
> > with 64bit support. But GNU ld program crashed at Ubuntu 14.04. It
> > means not that many people use this code path. GNU ld compiled from
> > HEAD does not have this problem but now GDB is confused by the fact
> > that ELF contains 64bit code while header reports i386.

That's unfortunate.

> > Practically there is no reason for this check as it prevents running
> > 64bit binaries with "qemu-system-x86_64 -kernel".

One reason for the check is that it prevents that one loads a 64bit ELF
binary that then fails strangely because it does not have the magic
32bit code to set up things. At least there needs to be an override
(could also be in the ELF info).

Doing a proper 32bit wrapper is also possible, although that would need
a little ELF loader, or a custom loader, probably. We've done it this
way but that also requires some more lines.

If allowing 64bit binaries it should also be checked that all relevant
values fit into 32bit.



Adam

Re: [Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[Anatol Pomozov]

Hi Adam, thank you for your reply.

On Mon, Jun 19, 2017 at 2:08 PM, Adam Lackorzynski
<adam@os.inf.tu-dresden.de> wrote:
> Hi,
>
> On Tue Jun 13, 2017 at 17:05:41 -0700, Anatol Pomozov wrote:
>> Do these arguments sound reasonable to apply the patch?
>
> I'm not really convinced.
>
>> On Thu, Jun 8, 2017 at 2:07 PM, Anatol Pomozov <anatol.pomozov@gmail.com> wrote:
>> > +reply-all
>> >
>> > On Thu, Jun 8, 2017 at 1:41 PM, Adam Lackorzynski
>> > <adam@os.inf.tu-dresden.de> wrote:
>> >>
>> >> On Tue Jun 06, 2017 at 21:41:48 -0700, Anatol Pomozov wrote:
>> >>> It is possible to create a 64 bit elf image that has valid multiboot header.
>> >>> qemu should be able to boot such images.
>> >>
>> >> But this 64bit image actually starts with 32bit code, right?
>> >
>> > Correct. The very first part of the startup code has to be 32bit.
>> > After it sets "long mode" it can use 64bit instructions. To make sure
>> > that the preamble has only 32bit instructions one have to use asm
>> > directive such as ".code32".
>> >
>> > Here is an example from LitleKernel sturtup code:
>> >
>> > https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L50
>> >
>> > .code32 tells assembler to treat following text as 32 bit code. And
>> > later when it jumps into "long mode"
>> >
>> > https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L214
>> > one can use 64bit code.
>> >
>> >> So it's a 32bit program and the check verifies that this is the case.
>> >
>> > While preamble have to contain 32 only instructions the rest of the
>> > image can perfectly contain 64bit code. Right now 64bit binary cannot
>> > be run with "qemu-system-x86_64 -kernel". But the same binary runs
>> > fine if packed with GRUB as iso.
>> >
>> > I tried to hack around this restriction by adding
>> > "OUTPUT_FORMAT(elf32-i386)" to the linker file and compiling project
>> > with 64bit support. But GNU ld program crashed at Ubuntu 14.04. It
>> > means not that many people use this code path. GNU ld compiled from
>> > HEAD does not have this problem but now GDB is confused by the fact
>> > that ELF contains 64bit code while header reports i386.
>
> That's unfortunate.
>
>> > Practically there is no reason for this check as it prevents running
>> > 64bit binaries with "qemu-system-x86_64 -kernel".
>
> One reason for the check is that it prevents that one loads a 64bit ELF
> binary that then fails strangely because it does not have the magic
> 32bit code to set up things.

I would not call the 32bit preamble a magic - it is fairly well
documented; it is just a result of backward compatibility that
AMD/Intel carry for a long time. Every single 64-bit operation system
(Linux, Windows, FreeBSD, ...) has such 32bit init sequence.

And while I understand your concerns about bugs in 64-bit mode
initialization I do not think that preventing elf64 images altogether
is a right fix for it. Loading 64bit OS with multiboot is a totally
valid use-case, you can find plenty of examples/OS that show how to do
it. Adding the restriction is just another hoop that users need to
jump over.

It is a good idea to check how other multiboot-compliant bootloaders
deal with such situation. GRUB has the best support for multiboot and
it works great both with elf64 bit images. You can see its code here
http://git.savannah.gnu.org/cgit/grub.git/tree/grub-core/loader/multiboot.c#n208

/* Load ELF32 or ELF64.  */
grub_err_t
grub_multiboot_load_elf (mbi_load_data_t *mld)
{
  if (grub_multiboot_is_elf32 (mld->buffer))
    return grub_multiboot_load_elf32 (mld);
  else if (grub_multiboot_is_elf64 (mld->buffer))
    return grub_multiboot_load_elf64 (mld);

  return grub_error (GRUB_ERR_UNKNOWN_OS, N_("invalid arch-dependent
ELF magic"));
}


I tried my elf64 OS in following configuration:
  Qemu + GRUB = works
  VMWare + GRUB = works
  GRUB at bare hardware = works

Qemu with -kernel loading elf64 directly - works flawlessly if the
patch above applied.

So currently QEMU breaks compatibility with GRUB. elf64 images boot
fine with GRUB while fail to boot with QEMU.

> At least there needs to be an override
> (could also be in the ELF info).
>
> Doing a proper 32bit wrapper is also possible, although that would need
> a little ELF loader, or a custom loader, probably. We've done it this
> way but that also requires some more lines.

No need for a wrapper. Current QEMU code works fine with elf64. I did
not spot any problems so far.


BTW I saw an argument in qemu maillist saying elf64 is not possible to
use in multiboot as there is no way to link 32bit code with 64bit
code. And while linkers do not allow it explicitly there are some
tricks that allow to merge two different architectures into one
binary. The easiest way is to use ".code32" directive in asm and
compile it with x86_64 architecture. "objcopy" is also possible to
use. Linux has its own simple linker to handle this situation
https://github.com/torvalds/linux/blob/master/arch/x86/boot/tools/build.c

> If allowing 64bit binaries it should also be checked that all relevant
> values fit into 32bit.

Agree that additional checks are useful. One thing that comes to my
mind is that elf entry addresses should be below 4GiB to fit the
multiboot field.

Re: [Qemu-devel] [PATCH] Remove restriction that prevents bootimg elf64 images

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 3261 bytes --]

On Wed, 2017-06-21 at 13:45 -0700, Anatol Pomozov wrote:
> Hi Adam, thank you for your reply.
> 
> On Mon, Jun 19, 2017 at 2:08 PM, Adam Lackorzynski
> <adam@os.inf.tu-dresden.de> wrote:
> > Hi,
> > 
> > On Tue Jun 13, 2017 at 17:05:41 -0700, Anatol Pomozov wrote:
> > > Do these arguments sound reasonable to apply the patch?
> > 
> > I'm not really convinced.
> > 
> > > On Thu, Jun 8, 2017 at 2:07 PM, Anatol Pomozov <anatol.pomozov@gmail.com> wrote:
> > > > +reply-all
> > > > 
> > > > On Thu, Jun 8, 2017 at 1:41 PM, Adam Lackorzynski
> > > > <adam@os.inf.tu-dresden.de> wrote:
> > > > > 
> > > > > On Tue Jun 06, 2017 at 21:41:48 -0700, Anatol Pomozov wrote:
> > > > > > It is possible to create a 64 bit elf image that has valid multiboot header.
> > > > > > qemu should be able to boot such images.
> > > > > 
> > > > > But this 64bit image actually starts with 32bit code, right?
> > > > 
> > > > Correct. The very first part of the startup code has to be 32bit.
> > > > After it sets "long mode" it can use 64bit instructions. To make sure
> > > > that the preamble has only 32bit instructions one have to use asm
> > > > directive such as ".code32".
> > > > 
> > > > Here is an example from LitleKernel sturtup code:
> > > > 
> > > > https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L50
> > > > 
> > > > .code32 tells assembler to treat following text as 32 bit code. And
> > > > later when it jumps into "long mode"
> > > > 
> > > > https://github.com/littlekernel/lk/blob/master/arch/x86/64/start.S#L214
> > > > one can use 64bit code.
> > > > 
> > > > > So it's a 32bit program and the check verifies that this is the case.
> > > > 
> > > > While preamble have to contain 32 only instructions the rest of the
> > > > image can perfectly contain 64bit code. Right now 64bit binary cannot
> > > > be run with "qemu-system-x86_64 -kernel". But the same binary runs
> > > > fine if packed with GRUB as iso.
> > > > 
> > > > I tried to hack around this restriction by adding
> > > > "OUTPUT_FORMAT(elf32-i386)" to the linker file and compiling project
> > > > with 64bit support. But GNU ld program crashed at Ubuntu 14.04. It
> > > > means not that many people use this code path. GNU ld compiled from
> > > > HEAD does not have this problem but now GDB is confused by the fact
> > > > that ELF contains 64bit code while header reports i386.
> > 
> > That's unfortunate.
> > 
> > > > Practically there is no reason for this check as it prevents running
> > > > 64bit binaries with "qemu-system-x86_64 -kernel".
> > 
> > One reason for the check is that it prevents that one loads a 64bit ELF
> > binary that then fails strangely because it does not have the magic
> > 32bit code to set up things.
> 
> I would not call the 32bit preamble a magic - it is fairly well
> documented; it is just a result of backward compatibility that
> AMD/Intel carry for a long time. Every single 64-bit operation system
> (Linux, Windows, FreeBSD, ...) has such 32bit init sequence.

FWIW, Anatol's patch also makes it possible for qemu to load the 64-bit
Xen XTF tests.

It would be a useful to at least have a way to override the check, even
if it's still present by default.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]