Re: [Qemu-devel] I got a kernel booted under qemu-system-ppc !

[Milton Miller <miltonm@bga.com>]

On Oct 18, 2007, at 6:46 PM, J. Mayer wrote:
> On Thu, 2007-10-18 at 19:12 -0500, Rob Landley wrote:
>> The easy way to reproduce this is go to  
>> "http://landley.net/hg/firmware",
>> download tip, and "./build.sh powerpc".  When it finishes building
>> everything, cd build and "./run-powerpc.sh".
>>
> [...]
>
>> The downside is that the result boots fine under qemu-0.9.0, but is  
>> broken
>> with current cvs.  I tracked it down to the specific patch with "git  
>> bisect",
>> and it's this one:
>>
>> http://git.kernel.dk/?p=qemu.git;a=commit; 
>> h=36f447f730f61ac413c5b1c4a512781f5dea0c94
>>
>> author  j_mayer <j_mayer>
>> 	Mon, 9 Apr 2007 22:45:36 +0000 (22:45 +0000)
>> committer  j_mayer <j_mayer>
>> 	Mon, 9 Apr 2007 22:45:36 +0000 (22:45 +0000)
>
> What is strange is that it was reported as unable to boot long before
> this patch. It was broken when the Qemu PCI architecture has been
> redesigned and have never been reported booting since then (2 years  
> ago,
> if I remember well).

My job includes getting the linux kernel to run on various PowerPC  
hardware.  While searching for a small file system, I came across Rob's  
work and it was close to what I needed so I started working with it.   
Rob was having trouble getting the kernel to build and run on qemu with  
Open Hackware.

I took the approach of treating qemu as another embedded board and  
describing the hardware to the kernel.  I quickly got ide and serial  
working, and later got pci config working.  I haven't gotten pci memory  
working yet; I don't know if that is the emulation or the description.   
Since hackware can initialize the video, its probably the later.   
Here's some history.

In the linux kernel, the ppc arch is being replaced by the powerpc  
arch, which has already replaced the ppc64 arch and supports all  
powerpc processors.  The ppc arch is slated to be removed by June 2008,  
having been in process over two years.  Part of the merge process,  
however, is to standardize the entry requirements of the kernel.  The  
ppc port had been accepting data from random firmwares in random  
structures, and it promoted drivers that only worked on one platform.   
The powerpc kernel requires that this information be translated to a  
device tree, similar to that provided by Open Firmware.  By forcing the  
device tree, the drivers are loaded based on the platform supplied data  
instead of knowing the platform and searching for data.  Instead of  
requiring a client interface however, a powerpc kenrel will accept the  
information as a data structure known as the flat device tree.  In  
fact, when booted with a client interface, after interacting with  
firmware (including storing some properties in the tree ond opening  
video cards to initialize them), the kernel builds this data structure  
and restarts itself with it.  The kernel tree also provides a library  
of boot code, called the wrapper or zImage, for translating random  
firmware structures into a flattened device tree.  It also provides  
support for loading compressed kernels and combining an initrd with the  
kernel into a single image.

Rob was complaining he needed arch ppc to boot prep qemu but  
kernel_headers only worked on powerpc.  Rob choose prep because the  
header on the prep kernel was the only one that satisfied Open Hackware  
when supplied to -kernel.  About this time Dave Gibson started code to  
run prep hardware under powerpc.  It had several assumptions on what  
the machine looked like.  As I remember it, I tried Dave's port on the  
qemu 0.6.2 found on the Knoppix 4.0.2 dvd and caused qemu to segv.  I  
was not prepared to debug qemu; I assumed the assumptions were not met.

Taking a look at Open Hackware 0.4, it appears that it treats the  
memory given to -kernel as a block device.  That makes no sense to me.   
I would expect it to treat the memory as the file loaded from the block  
device.  Seeing that no updates to Open Hackware had been made in  
years, and that it only pretended to have a open firmware client  
interface and contained a mostly useless rtas (eg no pci config  
methods), I decided to try bypassing it and just give the kernel what  
it was looking for: a description of the hardware.

I choose to continue with prep because I knew the expected hardware  
much better than pmac (the heathrow emulation).  I started by mining  
Open Hackware 0.4 for information describing the hardware and its  
startup.  I found the entry point was 0xfffffffc and after a bit of  
experimentation I found qemu required the rom to be 512k and was read  
only.  I found that since there were no caches I could do IO to serial  
in real mode, which meant I didn't have to decide what to map via bats.  
  Since the zImage code runs where it starts execution, I wrote some  
code to copy the rom to ram and start it there.  With only 512k the  
zImage couldn't contain the kernel, even compressed, to support  
something like kboot or petiboot (where the first kenrel is used with  
kexec to become a bootloader), but I already had patches to use the  
kernel loaded elsewhere.  I found the code in hackware to read nvram to  
find the memory size, loaded kernel, initrd, and command line and  
taught zImage to fetch the info from nvram and modify a device tree.  I  
used David's device tree and port as a baseline to modify.  When I  
didn't find code to describe an openpic or mpic, I decided to try using  
the 8259 as the interrupt controller.  After cutting down Dave's prep  
kernel code and changing its expectations to match the device tree, I  
quickly had interrupts, serial, and ide working.

http://patchwork.ozlabs.org/linuxppc/patch?person=79&id=13140

After a peek at qemu I got the right address for pci config and had  
that working. Rob complained my instructions were too hard to follow  
and had too much unmerged code, so I split and rebased them.  The  
kernel code patched cleanly against 2.6.23 but but the zImage code  
required patches queued for 2.6.24 and about 4 more not yet accepted.   
After looking at what the wrapper was doing, I decided it was was  
easier to just write the assembly code to copy the device tree to ram,  
copy the nvram to ram, and patch the 5 or so items based on the nvram.   
That resulted in the code for the rom that Rob posted.

> In march 2007, I did test and reported that PreP
> and heathrow target were unable to boot, not receiving any IRQ from PCI
> (in fact, adding traces proves there are even no IRQ generated in the
> PCI code). Someone reported the faulty patch during this summer (but
> I've not been unable to find the mail in the mailing list archive
> tonight).

As I stated, I haven't gotten pci memory working yet; my attempt at  
using -usb on under Q 0.9.0 (the os X compiled qemu) had the linux code  
failing to reset the chip.  However, since Open Hackware can initialize  
the graphics card, its probably an error in my description of the  
memory map in the device tree.  The linux driver for the video card  
also dies, I don't know if its because of no bios, the memory map, or  
other issues.

> I have to admit I never put the focus on trying to solve this issue,  
> has
> I usually use Mac99 or PowerPC 405 targets for tests and that PreP
> machines are long obsolete and the heathrow target does not reflect any
> real machine. But this is to be solved, for sure.

 From the patch description you inserted a openpic between the 8259 and  
the cpu.  If this is true, at a minimum it will require this to be  
described to be in the device tree for linux (or in the prep residual  
data in general for hackware).  My qemu linux code wasn't expecting an  
openpic so it would need to adjust; or perhaps David's kernel code  
would now work.

 From a linux on powerpc standpoint, I don't care if we found the  
hardware with the same platform code as a real machine or if a custom  
platform and boot code to create the device tree was used.  If the  
device tree matches the standard firmware, I can use the standard linux  
platform code.  If not, I can still use all the drivers.  If qemu  
misses a piece of hardware in the platform, I can choose to modify an  
existing one or create a new one.  However, I see being able to test  
the zImage with -kernel as a requirement (be it the standard platform  
zImage or a custom one customized to qemu), and if hackware is getting  
in the way it needs to be fixed or replaced.

I can help you debug from the linux side, but I have no interest in  
debugging qemu.  To complete the port to the 0.9 prep, I need to learn  
a bit more about the actual emulated machine and how its pci is hooked  
up.

The rom I provided was aimed at running linux on the hardware I knew,  
bypassing broken firmware.  I took advantage of the hardware being  
setup (eg ram working) and do the minimum to start the loaded linux.   
This means I support the image being an elf with a single load segment  
and the entry point can run where the file is loaded.  The code could  
be extended to check the platform and jump to another copy of itself  
when it doesn't match, meaning that through chaining it would search  
until it found the device tree for the platform being emulated and then  
start the kernel with that tree.  This would allow a rom to be built  
supporting several platforms, but it requires the kernel to be loaded  
via -kernel (as I don't have any floppy or hard drive code in it).   
This approach doesn't support booting from file systems, but an  
approach using kexec with kernel in rom could.

milton