[Qemu-devel] Allow ARMv7M to be started without a kernel

[Qemu-devel] Allow ARMv7M to be started without a kernel

[Ben Leslie]

Hi all,

For some current software development I'm doing I've found it most easy
to use Qemu in the following manner

qemu-system-arm -M lm3s811evb  -s -S & arm-eabi-gdb

>From GDB I then load any code I want to debug and test and run it.

For this to work however, I needed to make a small change to armv7m.c
to enable starting the simulator without specifying a "-kernel" option.

Is there any reason why such a change is a bad idea? (If not, I'll
submit an appropriate patch).

FWIW, the reason why I'm not using -kernel is that the current
way the armv7m code works, it expects the provided kernel to
be a full flash image including appropriate vector table, whereas
right now I just want to debug some stand-alone code, not the full
system, which the above gdb approach works perfectly for.

Cheers,

Benno

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Peter Maydell]

On 5 May 2011 09:23, Ben Leslie <benno@benno.id.au> wrote:
> FWIW, the reason why I'm not using -kernel is that the current
> way the armv7m code works, it expects the provided kernel to
> be a full flash image including appropriate vector table, whereas
> right now I just want to debug some stand-alone code, not the full
> system, which the above gdb approach works perfectly for.

It would probably be better for the -kernel option to honour the
entry point in the ELF file rather than insisting on full reset
(and to try to load the reset SP from the vector table but not
insist on that working). That is, we should support both "load
this ELF image which is a full system image with a vector table"
and "load this ELF image which is just a bare-metal (possibly
semihosting) application".

The combination of v7M reset with image loading and the possibility
of a debugger altering the pc/sp while the core is in reset is a bit
complicated, though :-)

As an aside: I think QEMU should have an option which is "just
load a plain ELF or raw binary, with no funny Linux-kernel-specific
behaviour" rather than overloading -kernel to mean "if it's a
raw image it's Linux and if it's an ELF file it's not".

-- PMM

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Ben Leslie]

On Thu, May 5, 2011 at 19:56, Peter Maydell <peter.maydell@linaro.org> wrote:
> On 5 May 2011 09:23, Ben Leslie <benno@benno.id.au> wrote:
>> FWIW, the reason why I'm not using -kernel is that the current
>> way the armv7m code works, it expects the provided kernel to
>> be a full flash image including appropriate vector table, whereas
>> right now I just want to debug some stand-alone code, not the full
>> system, which the above gdb approach works perfectly for.
>
> It would probably be better for the -kernel option to honour the
> entry point in the ELF file rather than insisting on full reset
> (and to try to load the reset SP from the vector table but not
> insist on that working). That is, we should support both "load
> this ELF image which is a full system image with a vector table"
> and "load this ELF image which is just a bare-metal (possibly
> semihosting) application".

Ideally that should work too, well in fact it would be more important
to get that working, and I guess the utility of my suggestion is
significantly lowered if that worked correctly.

I still think it is somewhat nice that the simulator target can work
just like a blank board though, and then connect GDB to it either
directly for the sim or via JTAG for a real board. Then it is the
same work flow for simulated or real hardware. (And the code change
is just one if statement). I guess the 'against' case is that if you
start up like that, and aren't running GDB it may be slightly confusing
for the user. However you would just get the same behaviour as you get
now when you pass in an image without a vector table.

(As an aside it would probably be good to implement the lockup mechanism
of ARMv7M so that with an invalid reset vector it goes into lockup and
something useful could be  displayed on stderr.)

> The combination of v7M reset with image loading and the possibility
> of a debugger altering the pc/sp while the core is in reset is a bit
> complicated, though :-)

I'm not sure if it is that complicated, I would assume that in armv7_reset
after calling cpu_reset the PC is set to the entry point in a similar way
to to how it is done in arm_boot.c

I don't think GDB gets a chance to touch the registers until after CPU reset
so I'm not sure if that will cause an issue.

(Another aside: ARMv7M is meant to cause a Hard Fault if something tries to
set the 'thumb' bit to zero, which would also help avoid some of the
silly problems
I had when getting my code to run.)

> As an aside: I think QEMU should have an option which is "just
> load a plain ELF or raw binary, with no funny Linux-kernel-specific
> behaviour" rather than overloading -kernel to mean "if it's a
> raw image it's Linux and if it's an ELF file it's not".

I agree, as far as I can tell right now (at least on arm), kernel can mean:

 1: A Linux kernel in linux kernel format.
 2: An ELF file (which may or may not have entry honoured)
 3: A raw binary.

OK, I think I may have forked this conversation with too may asides. Unless
there is a better suggestion I'll put together two patches, one that allows
-kernel to go unspecified, and a second that honours the ELF entry point
after reset. (Unless someone else wants to do it first!)

Cheers,

Ben

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Peter Maydell]

On 5 May 2011 13:03, Ben Leslie <benno@benno.id.au> wrote:
> I still think it is somewhat nice that the simulator target can work
> just like a blank board though, and then connect GDB to it either
> directly for the sim or via JTAG for a real board. Then it is the
> same work flow for simulated or real hardware. (And the code change
> is just one if statement).

I would personally be happy with that (or with a "no really I don't
want an image" option or something), but I would like consistency
across targets rather than an armv7m specific change.

> (As an aside it would probably be good to implement the lockup mechanism
> of ARMv7M so that with an invalid reset vector it goes into lockup and
> something useful could be  displayed on stderr.)

There are a few known bugs in QEMU's v7m exception model, and yes,
not implementing lockup is one of them. Incidentally, technically
it's possible to write an architecturally compliant program that puts
the core into lockup and then rescues itself via an NMI handler, but
"gone into lockup" would be a useful thing to trace if we had a consistent
mechanism for enabling "emit trace for events indicating likely OS bugs".
I don't have any time personally to look at v7M issues (Linaro's focus
is A profile cores) but I'll review patches if anybody submits them.

[The best way to think of Lockup is as a continual attempt to execute
an instruction until either it works or you get a reset or suitably high
priority exception; so you assert the lockup signal for the things the
spec says cause lockup, and you deassert lockup if you find you have
managed to successfully execute an instruction. Implementing this in
QEMU is left as an exercise for the reader :-)]

>> The combination of v7M reset with image loading and the possibility
>> of a debugger altering the pc/sp while the core is in reset is a bit
>> complicated, though :-)
>
> I'm not sure if it is that complicated, I would assume that in armv7_reset
> after calling cpu_reset the PC is set to the entry point in a similar way
> to to how it is done in arm_boot.c

> I don't think GDB gets a chance to touch the registers until after CPU reset
> so I'm not sure if that will cause an issue.

The trouble is that unlike A&R profiles (where the reset PC is a constant
value), the reset PC/SP are loaded from memory. So PC/SP aren't actually
set when the core goes into reset, but as the first thing that happens
when we come out of reset and start doing work. So if QEMU does the "load
initial PC/SP" in reset then (a) this isn't what the hardware does and
(b) trying to load an image with a vector table via the debugger won't
work (because we read PC from RAM before the debugger wrote to it). If
you do the "load PC/SP" after reset, then any change the user made to
PC/SP in gdb gets overridden (unless you take special measures to avoid that).
(See also the comment in target-arm/helper.c:cpu_reset() about another
case we're not getting right.)

I think you need to work out a consistent way everything should behave
first, rather than trying to generate patches to make point fixes to
cases that are causing problems.

> (Another aside: ARMv7M is meant to cause a Hard Fault if something tries to
> set the 'thumb' bit to zero, which would also help avoid some of the
> silly problems I had when getting my code to run.)

Strictly, it should generate a UsageFault (with UFSR.INVSTATE set) when
the core next tries to execute an instruction with the T bit clear. (If
this happens immediately after reset then the UsageFault will be escalated
to HardFault; if your HardFault handler also tries to execute in ARM
mode then we go into Lockup.) QEMU might not get the right fault status
bits, but it should certainly generate some kind of fault because
disas_arm_insn() causes an exception to be generated if it's invoked for
an M profile core.

-- PMM

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Alexander Graf]

On 05.05.2011, at 11:56, Peter Maydell wrote:

> On 5 May 2011 09:23, Ben Leslie <benno@benno.id.au> wrote:
>> FWIW, the reason why I'm not using -kernel is that the current
>> way the armv7m code works, it expects the provided kernel to
>> be a full flash image including appropriate vector table, whereas
>> right now I just want to debug some stand-alone code, not the full
>> system, which the above gdb approach works perfectly for.
> 
> It would probably be better for the -kernel option to honour the
> entry point in the ELF file rather than insisting on full reset
> (and to try to load the reset SP from the vector table but not
> insist on that working). That is, we should support both "load
> this ELF image which is a full system image with a vector table"
> and "load this ELF image which is just a bare-metal (possibly
> semihosting) application".
> 
> The combination of v7M reset with image loading and the possibility
> of a debugger altering the pc/sp while the core is in reset is a bit
> complicated, though :-)
> 
> As an aside: I think QEMU should have an option which is "just
> load a plain ELF or raw binary, with no funny Linux-kernel-specific
> behaviour" rather than overloading -kernel to mean "if it's a
> raw image it's Linux and if it's an ELF file it's not".

Traditionally, -bios has been that one. -kernel is more of a real bootloader replacement, including all the weirdness a bootloader does :).


Alex

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Rob Landley]

On 05/05/2011 06:26 PM, Alexander Graf wrote:
>> As an aside: I think QEMU should have an option which is "just load
>> a plain ELF or raw binary, with no funny Linux-kernel-specific 
>> behaviour" rather than overloading -kernel to mean "if it's a raw
>> image it's Linux and if it's an ELF file it's not".
> 
> Traditionally, -bios has been that one. -kernel is more of a real
> bootloader replacement, including all the weirdness a bootloader does
> :).

Except that neither "qemu-system-x86_64 -bios vmlinux" nor
qemu-system-x86_64 -kernel vmlinux" will load an ELF kernel on x86-64.

The code to do this _exists_ within qemu, it's just not hooked up
consistently on all targets.  We have a universal cross-platform image
format, and we have support in qemu for loading that format, and for
some reason it's only enabled on certain targets.  I've never understood
why...

Rob

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Alexander Graf]

On 06.05.2011, at 01:50, Rob Landley wrote:

> On 05/05/2011 06:26 PM, Alexander Graf wrote:
>>> As an aside: I think QEMU should have an option which is "just load
>>> a plain ELF or raw binary, with no funny Linux-kernel-specific 
>>> behaviour" rather than overloading -kernel to mean "if it's a raw
>>> image it's Linux and if it's an ELF file it's not".
>> 
>> Traditionally, -bios has been that one. -kernel is more of a real
>> bootloader replacement, including all the weirdness a bootloader does
>> :).
> 
> Except that neither "qemu-system-x86_64 -bios vmlinux" nor
> qemu-system-x86_64 -kernel vmlinux" will load an ELF kernel on x86-64.
> 
> The code to do this _exists_ within qemu, it's just not hooked up
> consistently on all targets.  We have a universal cross-platform image
> format, and we have support in qemu for loading that format, and for
> some reason it's only enabled on certain targets.  I've never understood
> why...

Unfortunately, booting isn't that easy :). The kernel needs some more data than just itself and an entry point to properly load. On some architectures, it requires a device tree (and pointer to it in a register), it might require page tables / TLB entries to be set up, if you want -initrd and -append to work you also need to tell the kernel where to find those.

So while ELF is a nice container for binaries, there are still some pieces missing to actually make it a kernel loader. But that's what multiboot was invented for. I haven't seen too much effort going on around multiboot on non-x86, but it basically fills exactly the gap you're wondering about. It uses ELF and adds a simple boot protocol for all the other fancy stuff a kernel needs.


Alex

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Andreas Färber]

Am 06.05.2011 um 14:48 schrieb Alexander Graf:

> On 06.05.2011, at 01:50, Rob Landley wrote:
>
>> On 05/05/2011 06:26 PM, Alexander Graf wrote:
>>>> As an aside: I think QEMU should have an option which is "just load
>>>> a plain ELF or raw binary, with no funny Linux-kernel-specific
>>>> behaviour" rather than overloading -kernel to mean "if it's a raw
>>>> image it's Linux and if it's an ELF file it's not".
>>>
>>> Traditionally, -bios has been that one. -kernel is more of a real
>>> bootloader replacement, including all the weirdness a bootloader  
>>> does
>>> :).
>>
>> Except that neither "qemu-system-x86_64 -bios vmlinux" nor
>> qemu-system-x86_64 -kernel vmlinux" will load an ELF kernel on  
>> x86-64.
>>
>> The code to do this _exists_ within qemu, it's just not hooked up
>> consistently on all targets.  We have a universal cross-platform  
>> image
>> format, and we have support in qemu for loading that format, and for
>> some reason it's only enabled on certain targets.  I've never  
>> understood
>> why...
>
> Unfortunately, booting isn't that easy :). The kernel needs some  
> more data than just itself and an entry point to properly load. On  
> some architectures, it requires a device tree (and pointer to it in  
> a register), it might require page tables / TLB entries to be set  
> up, if you want -initrd and -append to work you also need to tell  
> the kernel where to find those.
>
> So while ELF is a nice container for binaries, there are still some  
> pieces missing to actually make it a kernel loader. But that's what  
> multiboot was invented for. I haven't seen too much effort going on  
> around multiboot on non-x86, but it basically fills exactly the gap  
> you're wondering about. It uses ELF and adds a simple boot protocol  
> for all the other fancy stuff a kernel needs.

I guess Rob meant something different: -bios does not load ELF files  
for all machines (raw neither). For example, for raw-only PReP I had  
posted a patch that enables loading ELF firmware like OpenBIOS that  
you turned down.

Point being, the handling of -bios is board-specific (not even  
architecture-specific) and thus inconsistent. Your points above - page  
tables, TLB entries etc. - only apply to -kernel but not to -bios,  
that should give us a bare-metal machine.
Unfortunately I didn't see a way to make this handling more generic  
either. Some magic defines get passed into the ELF load function, so  
the call can't be moved to generic sources, and validation and  
placement of raw firmware is architecture- and board-specific from my  
view...

Andreas

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Rob Landley]

On 05/08/2011 09:10 AM, Andreas Färber wrote:
> Am 06.05.2011 um 14:48 schrieb Alexander Graf:
> 
>> On 06.05.2011, at 01:50, Rob Landley wrote:
>>
>>> On 05/05/2011 06:26 PM, Alexander Graf wrote:
>>>>> As an aside: I think QEMU should have an option which is "just load
>>>>> a plain ELF or raw binary, with no funny Linux-kernel-specific
>>>>> behaviour" rather than overloading -kernel to mean "if it's a raw
>>>>> image it's Linux and if it's an ELF file it's not".
>>>>
>>>> Traditionally, -bios has been that one. -kernel is more of a real
>>>> bootloader replacement, including all the weirdness a bootloader does
>>>> :).
>>>
>>> Except that neither "qemu-system-x86_64 -bios vmlinux" nor
>>> qemu-system-x86_64 -kernel vmlinux" will load an ELF kernel on x86-64.
>>>
>>> The code to do this _exists_ within qemu, it's just not hooked up
>>> consistently on all targets.  We have a universal cross-platform image
>>> format, and we have support in qemu for loading that format, and for
>>> some reason it's only enabled on certain targets.  I've never understood
>>> why...
>>
>> Unfortunately, booting isn't that easy :). The kernel needs some more
>> data than just itself and an entry point to properly load. On some
>> architectures, it requires a device tree (and pointer to it in a
>> register), it might require page tables / TLB entries to be set up, if
>> you want -initrd and -append to work you also need to tell the kernel
>> where to find those.
>>
>> So while ELF is a nice container for binaries, there are still some
>> pieces missing to actually make it a kernel loader. But that's what
>> multiboot was invented for. I haven't seen too much effort going on
>> around multiboot on non-x86, but it basically fills exactly the gap
>> you're wondering about. It uses ELF and adds a simple boot protocol
>> for all the other fancy stuff a kernel needs.
> 
> I guess Rob meant something different: -bios does not load ELF files for
> all machines (raw neither). For example, for raw-only PReP I had posted
> a patch that enables loading ELF firmware like OpenBIOS that you turned
> down.

I actually wrote a long reply and then save-as-draft rather than sending
it because it boils down to "I need to do the work myself and send you a
patch".

What I meant was that the build for every Linux architecture produces a
vmlinux file, which it often then packs into another format but on mips
and powerpc the file I feed to -kernel is the elf vmlinux file.
(Doesn't even have to be stripped.)

Which is nice, because when I debug stuff with the -s option I need to
feed the vmlinux file to gdb, so just having to keep track of one of the
suckers is incredibly convenient.  Being able to build 5 architectures
without having to record an arbitrary path to whatever the output file
is called this time is also nice.

Yes, -kernel is linux-specific: so is vmlinux.  Yes -kernel does various
other stuff to boot a linux kernel, but at least half of it is obsolete
things for 2.2 and earlier kernels, as described in the rdev man page.
(And if you want to use such ancient kernels, you can always make a boot
floppy image and use rdev, just just make a disk image with a bootloader
in it from one of the install cds on
http://archive.download.redhat.com/pub/redhat/linux or something.)

Pretty much everything the kernel needs is on the command line these
days, including vga settings and the physical starting address of the
initial ram disk ("ramdisk_start=", see init/do_mounts_rd.c in the
kernel).  On most platforms the one thing it needs is a device tree
pointer (in a register) and then it finds the command line data blob
through that.

I have no interest in GNU/multiboot, and neither does most of the rest
of the Linux community from what I've seen.  The GNU/hurd is welcome to
it, the FSF has nothing to do with Linux.  The people porting device
trees to x86 soo the mechanism will be properly universal (that went
into the 2.6.39 development series, by the way) are not referencing
GNU/multiboot.  I want to teach -kernel to accept a vmlinux file, not
introduce crazy FSF code dependencies, which don't seem to be supported
on any non-x86 platform yet despite being designed with Solaris in mind,
into my build.

What I want to do is actually somewhat straightforward, load_elf() is
already used directly on several platforms like mips_r4k.c, the hard
parts of making it work on x86 are A) properly editing the kernel
command line instead of trying to recreate rdev, B) figuring out where
to splice the kernel command line start address into a vmlinux instead
of a bzImage, and C) starting in the right mode.

C) requires more research, because I have to make sure the entry point
is either doing the 16->32 (or 64) bit startup dance or that it's being
launched in the right mode (which the bios isn't doing), but vmlinux
doesn't need to be decompressed and copied so it's just making sure
we're expecting the right layer of stuff.  Which means reading through
arch/x86/boot to see what the vmlinux->bzImage packaging is adding, I
suppose.

But "-kernel vmlinux" is not currently at the top of my todo list, lemme
finish studying the v9fs code first...

> Point being, the handling of -bios is board-specific (not even
> architecture-specific) and thus inconsistent.

So, currently, is -kernel.  Having it accept vmlinux on more platforms
would make it more consistent.  (Having -bios accept that too sounds
great, but is orthogonal to what I'm trying to do.)

> Your points above - page
> tables, TLB entries etc. - only apply to -kernel but not to -bios, that
> should give us a bare-metal machine.

In 16 bit mode.

Rob

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Alexander Graf]

On 05/08/2011 08:25 PM, Rob Landley wrote:
> On 05/08/2011 09:10 AM, Andreas Färber wrote:
>> Am 06.05.2011 um 14:48 schrieb Alexander Graf:
>>
>>> On 06.05.2011, at 01:50, Rob Landley wrote:
>>>
>>>> On 05/05/2011 06:26 PM, Alexander Graf wrote:
>>>>>> As an aside: I think QEMU should have an option which is "just load
>>>>>> a plain ELF or raw binary, with no funny Linux-kernel-specific
>>>>>> behaviour" rather than overloading -kernel to mean "if it's a raw
>>>>>> image it's Linux and if it's an ELF file it's not".
>>>>> Traditionally, -bios has been that one. -kernel is more of a real
>>>>> bootloader replacement, including all the weirdness a bootloader does
>>>>> :).
>>>> Except that neither "qemu-system-x86_64 -bios vmlinux" nor
>>>> qemu-system-x86_64 -kernel vmlinux" will load an ELF kernel on x86-64.
>>>>
>>>> The code to do this _exists_ within qemu, it's just not hooked up
>>>> consistently on all targets.  We have a universal cross-platform image
>>>> format, and we have support in qemu for loading that format, and for
>>>> some reason it's only enabled on certain targets.  I've never understood
>>>> why...
>>> Unfortunately, booting isn't that easy :). The kernel needs some more
>>> data than just itself and an entry point to properly load. On some
>>> architectures, it requires a device tree (and pointer to it in a
>>> register), it might require page tables / TLB entries to be set up, if
>>> you want -initrd and -append to work you also need to tell the kernel
>>> where to find those.
>>>
>>> So while ELF is a nice container for binaries, there are still some
>>> pieces missing to actually make it a kernel loader. But that's what
>>> multiboot was invented for. I haven't seen too much effort going on
>>> around multiboot on non-x86, but it basically fills exactly the gap
>>> you're wondering about. It uses ELF and adds a simple boot protocol
>>> for all the other fancy stuff a kernel needs.
>> I guess Rob meant something different: -bios does not load ELF files for
>> all machines (raw neither). For example, for raw-only PReP I had posted
>> a patch that enables loading ELF firmware like OpenBIOS that you turned
>> down.
> I actually wrote a long reply and then save-as-draft rather than sending
> it because it boils down to "I need to do the work myself and send you a
> patch".
>
> What I meant was that the build for every Linux architecture produces a
> vmlinux file, which it often then packs into another format but on mips
> and powerpc the file I feed to -kernel is the elf vmlinux file.
> (Doesn't even have to be stripped.)
>
> Which is nice, because when I debug stuff with the -s option I need to
> feed the vmlinux file to gdb, so just having to keep track of one of the
> suckers is incredibly convenient.  Being able to build 5 architectures
> without having to record an arbitrary path to whatever the output file
> is called this time is also nice.
>
> Yes, -kernel is linux-specific: so is vmlinux.  Yes -kernel does various
> other stuff to boot a linux kernel, but at least half of it is obsolete
> things for 2.2 and earlier kernels, as described in the rdev man page.
> (And if you want to use such ancient kernels, you can always make a boot
> floppy image and use rdev, just just make a disk image with a bootloader
> in it from one of the install cds on
> http://archive.download.redhat.com/pub/redhat/linux or something.)
>
> Pretty much everything the kernel needs is on the command line these
> days, including vga settings and the physical starting address of the
> initial ram disk ("ramdisk_start=", see init/do_mounts_rd.c in the
> kernel).  On most platforms the one thing it needs is a device tree
> pointer (in a register) and then it finds the command line data blob
> through that.
>
> I have no interest in GNU/multiboot, and neither does most of the rest
> of the Linux community from what I've seen.  The GNU/hurd is welcome to
> it, the FSF has nothing to do with Linux.  The people porting device
> trees to x86 soo the mechanism will be properly universal (that went
> into the 2.6.39 development series, by the way) are not referencing
> GNU/multiboot.  I want to teach -kernel to accept a vmlinux file, not
> introduce crazy FSF code dependencies, which don't seem to be supported
> on any non-x86 platform yet despite being designed with Solaris in mind,
> into my build.
>
> What I want to do is actually somewhat straightforward, load_elf() is
> already used directly on several platforms like mips_r4k.c, the hard
> parts of making it work on x86 are A) properly editing the kernel
> command line instead of trying to recreate rdev, B) figuring out where
> to splice the kernel command line start address into a vmlinux instead
> of a bzImage, and C) starting in the right mode.
>
> C) requires more research, because I have to make sure the entry point
> is either doing the 16->32 (or 64) bit startup dance or that it's being
> launched in the right mode (which the bios isn't doing), but vmlinux
> doesn't need to be decompressed and copied so it's just making sure
> we're expecting the right layer of stuff.  Which means reading through
> arch/x86/boot to see what the vmlinux->bzImage packaging is adding, I
> suppose.

The issue is that this is not how it works on real hardware. Grub won't 
just load a vmlinux file and boot it. I'm not even sure how much exactly 
the early entry code handles in Linux before it jumps to the ELF entry 
point.

Either way, if you get something rolling that also ensures that it fails 
when it's an ELF file that's not Linux, I'd be very open to it :).

> But "-kernel vmlinux" is not currently at the top of my todo list, lemme
> finish studying the v9fs code first...
>
>> Point being, the handling of -bios is board-specific (not even
>> architecture-specific) and thus inconsistent.
> So, currently, is -kernel.  Having it accept vmlinux on more platforms
> would make it more consistent.  (Having -bios accept that too sounds
> great, but is orthogonal to what I'm trying to do.)
>
>> Your points above - page
>> tables, TLB entries etc. - only apply to -kernel but not to -bios, that
>> should give us a bare-metal machine.
> In 16 bit mode.

Depends on the machine. On PPC e500 for example, even with -bios we have 
to set up TLB entries, but those are at least defined by the spec :).


Alex

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Peter Maydell]

On 9 May 2011 16:11, Alexander Graf <agraf@suse.de> wrote:
[about -kernel, unless I've got confused]
> The issue is that this is not how it works on real hardware. Grub won't just
> load a vmlinux file and boot it. I'm not even sure how much exactly the
> early entry code handles in Linux before it jumps to the ELF entry point.
>
> Either way, if you get something rolling that also ensures that it fails
> when it's an ELF file that's not Linux, I'd be very open to it :).

If we do that we need to document what the new way of doing "just load
and jump to the entry point of my not-a-linux-kernel ELF image" is; at
the moment for ARM that use case is supported by -kernel (the code
specifically handles ELF images as not-kernels), so changing that would
be a back-compatibility break...

(maybe this should be '-bios' but that seems a bit of an obscure name.)

-- PMM

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Rob Landley]

On 05/09/2011 09:11 AM, Alexander Graf wrote:
>> C) requires more research, because I have to make sure the entry point
>> is either doing the 16->32 (or 64) bit startup dance or that it's being
>> launched in the right mode (which the bios isn't doing), but vmlinux
>> doesn't need to be decompressed and copied so it's just making sure
>> we're expecting the right layer of stuff.  Which means reading through
>> arch/x86/boot to see what the vmlinux->bzImage packaging is adding, I
>> suppose.
> 
> The issue is that this is not how it works on real hardware.

Real hardware doesn't have the -kernel option.  On x86 that option is
currently patching the rdev area after loading the blob into DRAM.

> Grub won't just load a vmlinux file and boot it.

The -kernel option doesn't load grub, it loads a _kernel_.  -kernel is a
linux-specific bootloader.  It doesn't chainload other bootloaders.

> I'm not even sure how much exactly
> the early entry code handles in Linux before it jumps to the ELF entry
> point.
> 
> Either way, if you get something rolling that also ensures that it fails
> when it's an ELF file that's not Linux, I'd be very open to it :).

If it's an ELF file that's not Linux, I can ensure the result won't boot
Linux.  It's guaranteed to fail without me doing anything.

I don't understand your objection.  (The option is called -kernel.  It
does what it says on the tin.  If you feed it something that isn't a
kernel, which you can do now, this is pilot error.)

By the way, I just did "qemu -kernel qemu-img" an the emulated vga
screen was very colorful.  Feeding it qemu-io gave a different pattern
of vga text mode garbage.  It's happily loading an elf file _now_, and
spinning its little CPU on the resulting garbage...

>> But "-kernel vmlinux" is not currently at the top of my todo list, lemme
>> finish studying the v9fs code first...
>>
>>> Point being, the handling of -bios is board-specific (not even
>>> architecture-specific) and thus inconsistent.
>> So, currently, is -kernel.  Having it accept vmlinux on more platforms
>> would make it more consistent.  (Having -bios accept that too sounds
>> great, but is orthogonal to what I'm trying to do.)
>>
>>> Your points above - page
>>> tables, TLB entries etc. - only apply to -kernel but not to -bios, that
>>> should give us a bare-metal machine.
>> In 16 bit mode.
> 
> Depends on the machine. On PPC e500 for example, even with -bios we have
> to set up TLB entries, but those are at least defined by the spec :).

Yes, -kernel has to do setup work before starting the kernel.  This is
true now, it remains true if the kernel is in a different file format.
-kernel is a linux bootloader, which is not the same thing as the -bios
option.

Rob

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Alexander Graf]

On 10.05.2011, at 06:58, Rob Landley wrote:

> On 05/09/2011 09:11 AM, Alexander Graf wrote:
>>> C) requires more research, because I have to make sure the entry point
>>> is either doing the 16->32 (or 64) bit startup dance or that it's being
>>> launched in the right mode (which the bios isn't doing), but vmlinux
>>> doesn't need to be decompressed and copied so it's just making sure
>>> we're expecting the right layer of stuff.  Which means reading through
>>> arch/x86/boot to see what the vmlinux->bzImage packaging is adding, I
>>> suppose.
>> 
>> The issue is that this is not how it works on real hardware.
> 
> Real hardware doesn't have the -kernel option.  On x86 that option is
> currently patching the rdev area after loading the blob into DRAM.
> 
>> Grub won't just load a vmlinux file and boot it.
> 
> The -kernel option doesn't load grub, it loads a _kernel_.  -kernel is a
> linux-specific bootloader.  It doesn't chainload other bootloaders.

My point is that -kernel is on the same level as grub. It's a bootloader - just (mostly) implemented in host mode, not guest mode. I'm not objecting to your idea. Just saying that what you're doing isn't streamlined in Linux atm, so you might hit barriers that people didn't expect :).


> 
>> I'm not even sure how much exactly
>> the early entry code handles in Linux before it jumps to the ELF entry
>> point.
>> 
>> Either way, if you get something rolling that also ensures that it fails
>> when it's an ELF file that's not Linux, I'd be very open to it :).
> 
> If it's an ELF file that's not Linux, I can ensure the result won't boot
> Linux.  It's guaranteed to fail without me doing anything.
> 
> I don't understand your objection.  (The option is called -kernel.  It
> does what it says on the tin.  If you feed it something that isn't a
> kernel, which you can do now, this is pilot error.)
> 
> By the way, I just did "qemu -kernel qemu-img" an the emulated vga
> screen was very colorful.  Feeding it qemu-io gave a different pattern
> of vga text mode garbage.  It's happily loading an elf file _now_, and
> spinning its little CPU on the resulting garbage...

IIRC -kernel loads ELF images as raw blobs atm. Just try to implement it and you'll see what I mean, seriously :).

Basically, -kernel with an OpenBSD ELF kernel would be different. You somehow need to be able to find out the boot protocol. Either it's defined by architecture or by random OS conventions. If you can somehow make sure that a kernel really is Linux and nothing else, speaking some random Linux boot protocol with it is just fine!

> 
>>> But "-kernel vmlinux" is not currently at the top of my todo list, lemme
>>> finish studying the v9fs code first...
>>> 
>>>> Point being, the handling of -bios is board-specific (not even
>>>> architecture-specific) and thus inconsistent.
>>> So, currently, is -kernel.  Having it accept vmlinux on more platforms
>>> would make it more consistent.  (Having -bios accept that too sounds
>>> great, but is orthogonal to what I'm trying to do.)
>>> 
>>>> Your points above - page
>>>> tables, TLB entries etc. - only apply to -kernel but not to -bios, that
>>>> should give us a bare-metal machine.
>>> In 16 bit mode.
>> 
>> Depends on the machine. On PPC e500 for example, even with -bios we have
>> to set up TLB entries, but those are at least defined by the spec :).
> 
> Yes, -kernel has to do setup work before starting the kernel.  This is
> true now, it remains true if the kernel is in a different file format.
> -kernel is a linux bootloader, which is not the same thing as the -bios
> option.

I disagree. -kernel is a _kernel_ bootloader. There's more out there than just Linux :).


Alex




> 
> Rob

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Rob Landley]

On 05/09/2011 10:50 AM, Peter Maydell wrote:
> On 9 May 2011 16:11, Alexander Graf <agraf@suse.de> wrote:
> [about -kernel, unless I've got confused]
>> The issue is that this is not how it works on real hardware. Grub won't just
>> load a vmlinux file and boot it. I'm not even sure how much exactly the
>> early entry code handles in Linux before it jumps to the ELF entry point.
>>
>> Either way, if you get something rolling that also ensures that it fails
>> when it's an ELF file that's not Linux, I'd be very open to it :).
> 
> If we do that we need to document what the new way of doing "just load
> and jump to the entry point of my not-a-linux-kernel ELF image" is; at
> the moment for ARM that use case is supported by -kernel (the code
> specifically handles ELF images as not-kernels), so changing that would
> be a back-compatibility break...

Arm doesn't need nearly as much setup as x86, some boards just map flash
at the physical start address with a jump straight to the kernel entry
point.

On arm there's no legacy start mode, meaning no 16->32 bit transfer
requiring mmu initialization as part of the setup.  (I believe arm
starts with a 1-1 virtual/physical mapping in the absence of initialized
page tables, or something like that.)  The big black magic thing
coreboot/bios and uboot do is DRAM refresh, which QEMU simply doesn't
care about: we map dram from the host and it gets refreshed at that level.

So this behavior may actually be correct for Linux in the absence of a
device tree (which just means initializinng a register to point to it on
ppc, dunno what ARM does).

And if you feed in an "-append" option to set a kernel command line,
then you know it's a linux kernel.  If we explicitly ask for more setup,
then you know to do more setup.  (I think that bit still needs to be
written, I have to go read the code...)

Rob

Re: [Qemu-devel] Allow ARMv7M to be started without a kernel

[Rob Landley]

On 05/10/2011 12:13 AM, Alexander Graf wrote:
> 
> On 10.05.2011, at 06:58, Rob Landley wrote:
> 
>> On 05/09/2011 09:11 AM, Alexander Graf wrote:
>>>> C) requires more research, because I have to make sure the
>>>> entry point is either doing the 16->32 (or 64) bit startup
>>>> dance or that it's being launched in the right mode (which the
>>>> bios isn't doing), but vmlinux doesn't need to be decompressed
>>>> and copied so it's just making sure we're expecting the right
>>>> layer of stuff.  Which means reading through arch/x86/boot to
>>>> see what the vmlinux->bzImage packaging is adding, I suppose.
>>> 
>>> The issue is that this is not how it works on real hardware.
>> 
>> Real hardware doesn't have the -kernel option.  On x86 that option
>> is currently patching the rdev area after loading the blob into
>> DRAM.
>> 
>>> Grub won't just load a vmlinux file and boot it.
>> 
>> The -kernel option doesn't load grub, it loads a _kernel_.  -kernel
>> is a linux-specific bootloader.  It doesn't chainload other
>> bootloaders.
> 
> My point is that -kernel is on the same level as grub. It's a
> bootloader - just (mostly) implemented in host mode, not guest mode.
> I'm not objecting to your idea. Just saying that what you're doing
> isn't streamlined in Linux atm, so you might hit barriers that people
> didn't expect :).

Oh sure.  That's why I haven't posted the patch yet.

I read through enough to assemble a very rough todo list (call
load_elf(), figure out if the kernel can run from the read-only ROM blob
or if I need to copy it to normal DRAM like some targets do, figure out
where the structure rdev used to patch lives in the blob the ELF loaded
so we can set the right field in it to specify where the command line
lives, figure out what we need to add to said command line...)

But it's not in the top 5 of my todo list today...

>> By the way, I just did "qemu -kernel qemu-img" an the emulated vga 
>> screen was very colorful.  Feeding it qemu-io gave a different
>> pattern of vga text mode garbage.  It's happily loading an elf file
>> _now_, and spinning its little CPU on the resulting garbage...
> 
> IIRC -kernel loads ELF images as raw blobs atm. Just try to implement
> it and you'll see what I mean, seriously :).

Which is why -kernel needs to fix up the result (if nothing else, to
feed it a command line).

> Basically, -kernel with an OpenBSD ELF kernel would be different.

My lack of caring about OpenBSD cannot be expressed in words.  Does the
current -kernel option support OpenBSD kernels?  If so, I wouldn't be
introducing any regressions.

> You somehow need to be able to find out the boot protocol.

No, I don't.  I need to ignore OpenBSD with the force of a thousand DMV
agents on break.

"Your new thing would continue not to do something the current -kernel
option doesn't support".  Yes.  Yes it would.  I don't see how this is
considered a valid objection.

> Either it's
> defined by architecture or by random OS conventions. If you can
> somehow make sure that a kernel really is Linux and nothing else,
> speaking some random Linux boot protocol with it is just fine!

The current -kernel option will happily load a TEXT FILE.  The result
does not boot, but it makes pretty pictures in the VGA screen.

Feeding -kernel an OpenBSD kernel is only slightly less useful than
feeding it a text file, and if you want multiboot you know where to find
it.  (There's code for that in qemu now, it's just not at all
interesting to me.)

>> Yes, -kernel has to do setup work before starting the kernel.  This
>> is true now, it remains true if the kernel is in a different file
>> format. -kernel is a linux bootloader, which is not the same thing
>> as the -bios option.
> 
> I disagree. -kernel is a _kernel_ bootloader.

A linux kernel bootloader.  It's 100% linux-specific.  Always has been.

Feel free to provide a counter-example.  Show me how to boot an OpenBSD
kernel with "qemu-system-x86_64 -kernel".

> There's more out there than just Linux :).

The current code disagrees with you.  This project's been around for a
decade and nobody's ever bothered to add BSD support to -kernel, and I
won't either.

If you'd be more comfortable if the command line option was renamed
"-linux", feel free to submit a patch, but you're currently arguing
against the existing code, not against the patch I'm writing.

Rob