[Qemu-devel] problems with freeBSD

[Qemu-devel] problems with freeBSD

[Dietmar Maurer]

[-- Attachment #1: Type: text/plain, Size: 422 bytes --]

Using qemu 1.4.0:

# qemu -hda test.raw -m 512 -cdrom pfSense-LiveCD-2.0.2-RELEASE-amd64-20121207-2239.iso

Results in:

trap 12: page fault while in kernel mode
...
stopped at x86bios_emu_rdw+0x2f: movzwl (%rbx),%eax

Any ideas? Can somebody reproduce that?

To get the FreeBSD VM boot use the console, enter the boot loader, then:
# set hint.atkbd.0.disabled="1"
# boot

But that disables the keyboard.

[-- Attachment #2: Type: text/html, Size: 3175 bytes --]

Re: [Qemu-devel] problems with freeBSD

[Alexandre DERUMIER]

I can also reproduce it. (Host cpu intel or amd, Guest cpu qemu64/kvm64/host).

They are also a bugreport on the freebsd mailing here:

http://lists.freebsd.org/pipermail/freebsd-amd64/2013-January/015092.html



----- Mail original -----

De: "Dietmar Maurer" <dietmar@proxmox.com>
À: qemu-devel@nongnu.org
Envoyé: Mercredi 6 Mars 2013 09:21:11
Objet: [Qemu-devel] problems with freeBSD



Using qemu 1.4.0:

# qemu -hda test.raw -m 512 -cdrom pfSense-LiveCD-2.0.2-RELEASE-amd64-20121207-2239.iso

Results in:

trap 12: page fault while in kernel mode
…
stopped at x86bios_emu_rdw+0x2f: movzwl (%rbx),%eax

Any ideas? Can somebody reproduce that?

To get the FreeBSD VM boot use the console, enter the boot loader, then:
# set hint.atkbd.0.disabled="1"
# boot

But that disables the keyboard.

Re: [Qemu-devel] problems with freeBSD

[Dietmar Maurer]

I already sent a mail to the qemu-devel list - no answer so far.

> -----Original Message-----
> From: Alexandre DERUMIER [mailto:aderumier@odiso.com]
> Sent: Mittwoch, 06. März 2013 13:42
> To: Dietmar Maurer
> Cc: qemu-devel@nongnu.org
> Subject: Re: [Qemu-devel] problems with freeBSD
> 
> I can also reproduce it. (Host cpu intel or amd, Guest cpu qemu64/kvm64/host).
> 
> They are also a bugreport on the freebsd mailing here:
> 
> http://lists.freebsd.org/pipermail/freebsd-amd64/2013-January/015092.html

Re: [Qemu-devel] problems with freeBSD

[Laszlo Ersek]

On 03/06/13 09:21, Dietmar Maurer wrote:
> Using qemu 1.4.0:
> 
>  
> 
> # qemu -hda test.raw -m 512 -cdrom
> pfSense-LiveCD-2.0.2-RELEASE-amd64-20121207-2239.iso
> 
>  
> 
> Results in:
> 
>  
> 
> trap 12: page fault while in kernel mode
> 
> …
> 
> stopped at x86bios_emu_rdw+0x2f: movzwl (%rbx),%eax
> 
>  
> 
> Any ideas? Can somebody reproduce that?

I found this on the web:

351	static uint16_t
352	x86bios_emu_rdw(struct x86emu *emu, uint32_t addr)
353	{
354		uint16_t *va;
355	
356		va = x86bios_get_pages(addr, sizeof(*va));
357		if (va == NULL)
358			x86bios_set_fault(emu, addr);
359	
360	#ifndef __NO_STRICT_ALIGNMENT
361		if ((addr & 1) != 0)
362			return (le16dec(va));
363		else
364	#endif
365		return (le16toh(*va));
366	}

"movzwl (%rbx),%eax" reads the 16-bit word at %rbx, copies it into the
low 16 bits of %eax, and clears the rest of the bits in %eax.

This probably corresponds to the *va dereference at the end of the
function. (le16toh() is compiled out if the platform is little endian.)
Either x86bios_get_pages() returns garbage, or "addr" is garbage on input.

You'll have to trace back the callpath from x86bios_emu_rdw(), possibly
instrumenting it with printk()s (or whatever it's called in FreeBSD).

Laszlo

Re: [Qemu-devel] problems with freeBSD

[Aurelien Jarno]

On Wed, Mar 06, 2013 at 08:21:11AM +0000, Dietmar Maurer wrote:
> Using qemu 1.4.0:
> 
> # qemu -hda test.raw -m 512 -cdrom pfSense-LiveCD-2.0.2-RELEASE-amd64-20121207-2239.iso
> 
> Results in:
> 
> trap 12: page fault while in kernel mode
> ...
> stopped at x86bios_emu_rdw+0x2f: movzwl (%rbx),%eax
> 
> Any ideas? Can somebody reproduce that?
> 
> To get the FreeBSD VM boot use the console, enter the boot loader, then:
> # set hint.atkbd.0.disabled="1"
> # boot
> 
> But that disables the keyboard.

I was actually digging about that problem. It is indeed present in
version 1.4.0, but is fixed in the current git master. The problem is
actually not directly in QEMU but in seabios, the update to version
1.7.2.1 commit 5c75fb10) fixes the issue. Maybe it is worth 
cherry-picking it into stable-1.4 (hence the Cc:). In the meantime
using bios.bin from master with QEMU version 1.4.0 should also fix the
issue.

What is strange is the seabios commit fixing the issue:

    commit 4219149ad2b783abfa61e80e9e9f6910db0c76c9
    Author: Kevin O'Connor <kevin@koconnor.net>
    Date:   Sun Feb 17 10:56:10 2013 -0500
    
        build: Don't require $(OUT) to be a sub-directory of the main directory.
        
        Remove references to "../" and "out/" from the build so that "make
        OUT=/a/b/c/" will work.
        
        Signed-off-by: Kevin O'Connor <kevin@koconnor.net>
   
Maybe Kevin has an explanation?

-- 
Aurelien Jarno                          GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] problems with freeBSD

[Kevin O'Connor]

On Thu, Mar 07, 2013 at 12:12:08AM +0100, Aurelien Jarno wrote:
> On Wed, Mar 06, 2013 at 08:21:11AM +0000, Dietmar Maurer wrote:
> > Using qemu 1.4.0:
> > 
> > # qemu -hda test.raw -m 512 -cdrom pfSense-LiveCD-2.0.2-RELEASE-amd64-20121207-2239.iso
> > 
> > Results in:
> > 
> > trap 12: page fault while in kernel mode
> > ...
> > stopped at x86bios_emu_rdw+0x2f: movzwl (%rbx),%eax
> > 
> > Any ideas? Can somebody reproduce that?
> > 
> > To get the FreeBSD VM boot use the console, enter the boot loader, then:
> > # set hint.atkbd.0.disabled="1"
> > # boot
> > 
> > But that disables the keyboard.
> 
> I was actually digging about that problem. It is indeed present in
> version 1.4.0, but is fixed in the current git master. The problem is
> actually not directly in QEMU but in seabios, the update to version
> 1.7.2.1 commit 5c75fb10) fixes the issue. Maybe it is worth 
> cherry-picking it into stable-1.4 (hence the Cc:). In the meantime
> using bios.bin from master with QEMU version 1.4.0 should also fix the
> issue.
> 
> What is strange is the seabios commit fixing the issue:
> 
>     commit 4219149ad2b783abfa61e80e9e9f6910db0c76c9
>     Author: Kevin O'Connor <kevin@koconnor.net>
>     Date:   Sun Feb 17 10:56:10 2013 -0500
>     
>         build: Don't require $(OUT) to be a sub-directory of the main directory.

That change is definitely just build related - I don't see how it
could impact the final SeaBIOS binary.  How did you conclude that this
commit is what fixes the issue?

-Kevin

Re: [Qemu-devel] problems with freeBSD

[Laszlo Ersek]

On 03/07/13 01:53, Kevin O'Connor wrote:
> On Thu, Mar 07, 2013 at 12:12:08AM +0100, Aurelien Jarno wrote:
>> On Wed, Mar 06, 2013 at 08:21:11AM +0000, Dietmar Maurer wrote:
>>> Using qemu 1.4.0:
>>>
>>> # qemu -hda test.raw -m 512 -cdrom pfSense-LiveCD-2.0.2-RELEASE-amd64-20121207-2239.iso
>>>
>>> Results in:
>>>
>>> trap 12: page fault while in kernel mode
>>> ...
>>> stopped at x86bios_emu_rdw+0x2f: movzwl (%rbx),%eax
>>>
>>> Any ideas? Can somebody reproduce that?
>>>
>>> To get the FreeBSD VM boot use the console, enter the boot loader, then:
>>> # set hint.atkbd.0.disabled="1"
>>> # boot
>>>
>>> But that disables the keyboard.

Apparently the call may come from get_typematic()
[sys/dev/atkbdc/atkbd.c]; it wants to retrieve the typematic rate of the
keyboard using the BIOS.

>>
>> I was actually digging about that problem. It is indeed present in
>> version 1.4.0, but is fixed in the current git master. The problem is
>> actually not directly in QEMU but in seabios, the update to version
>> 1.7.2.1 commit 5c75fb10) fixes the issue. Maybe it is worth
>> cherry-picking it into stable-1.4 (hence the Cc:). In the meantime
>> using bios.bin from master with QEMU version 1.4.0 should also fix the
>> issue.
>>
>> What is strange is the seabios commit fixing the issue:
>>
>>     commit 4219149ad2b783abfa61e80e9e9f6910db0c76c9
>>     Author: Kevin O'Connor <kevin@koconnor.net>
>>     Date:   Sun Feb 17 10:56:10 2013 -0500
>>
>>         build: Don't require $(OUT) to be a sub-directory of the main directory.
>
> That change is definitely just build related - I don't see how it
> could impact the final SeaBIOS binary.  How did you conclude that this
> commit is what fixes the issue?

Going out on a limb, I suspect qemu commit 5f876756 instead.

(It's a bit risky for me to say that, as Aurelien may have taken
qemu-1.4.0 as fixed point and bisected seabios rel-1.7.2..rel-1.7.2.1
against it:

$ git log --oneline --reverse rel-1.7.2..rel-1.7.2.1
f396871 Update tools/acpi_extract.py to handle iasl 20130117 release.
12e8199 USB-EHCI: Fix null pointer assignment
d75c22f Fix Makefile - don't reference "out/" directly, instead use "$(OUT)".
4219149 build: Don't require $(OUT) to be a sub-directory of the main directory.
e5fe4f9 Verify CC is valid during build tests.
2b57726 seabios q35: Enable all PIRQn IRQs at startup
985a9d3 seabios q35: Add new PCI slot to irq routing function
88cb66e seabios: Add a dummy PCI slot to irq mapping function
)

I'm suspecting said qemu commit because:
- it's the final commit in 1.4 for file "pc-bios/bios.bin",
- somewhat out of the ordinary, apparently, it was Anthony to rebuild
  the bios, and he used gcc-4.7.2 on Fedora 18,
- while normally Gerd does the updates (see both before and after
  5f876756), and I know for a fact Gerd uses RHEL-6.

I think the gcc version Anthony was using miscompiled SeaBIOS (in the
sense that FreeBSD chokes on it), and the 1.7.2.1 binary from Gerd
restores peace *only* because Gerd relied on RHEL-6 gcc, and not because
of the SeaBIOS changes from 1.7.2 to 1.7.2.1.

$ git log --reverse -- pc-bios/bios.bin

Probably works, but never appeared in a separate release:

    commit 3588185b8396eb97fd9efd41c2b97775465f67c4
    Author: Gerd Hoffmann <kraxel@redhat.com>
    Date:   Mon Jan 21 09:17:16 2013 +0100

        seabios: update to 1.7.2 release

        Not that many changes as we have a pretty recent git snapshot in
        master already:

        Hannes Reinecke (1):
              megasas: Invert PCI device selection

        Kevin O'Connor (2):
              Minor: Separate UUID display from F12 boot prompt.
              boot: Support "halt" in the boot order to prevent default
                boot attempts.

        Laszlo Ersek (1):
              display_uuid(): fix incomplete check after the loop

        Paolo Bonzini (1):
              vgabios: implement AX=1120H..1124H functions

Exposes problem (released in qemu-1.4.0):

    commit 5f876756c57c15f5e14d4136fc432b74f05f082b
    Author: Anthony Liguori <aliguori@us.ibm.com>
    Date:   Wed Feb 6 05:12:06 2013 -0600

        bios: recompile BIOS

        SeaBIOS is really close to spilling over to 256k.  Until we can
        better handle migration across RAM block size changes, recompile
        SeaBIOS with a compiler that causes the binary to still fit in
        128k.

        This was built with:

        gcc version 4.7.2 20121109 (Red Hat 4.7.2-8) (GCC)

        On 64-bit Fedora 18.

        Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

Works again (unreleased), according to Aurelien's testing:

    commit 5c75fb10029c5fd1e705a6ef5d698fbea06c7a33
    Author: Gerd Hoffmann <kraxel@redhat.com>
    Date:   Thu Feb 28 09:18:56 2013 +0100

        update seabios to 1.7.2.1

        Alex Williamson (3):
              seabios q35: Enable all PIRQn IRQs at startup
              seabios q35: Add new PCI slot to irq routing function
              seabios: Add a dummy PCI slot to irq mapping function

        Avik Sil (1):
              USB-EHCI: Fix null pointer assignment

        Kevin O'Connor (4):
              Update tools/acpi_extract.py to handle iasl 20130117
                release.
              Fix Makefile - don't reference "out/" directly, instead
                use "$(OUT)".
              build: Don't require $(OUT) to be a sub-directory of the
                main directory.
              Verify CC is valid during build tests.

        Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

(I re-wrapped the commit messages for legibility.)

Laszlo

Re: [Qemu-devel] [SeaBIOS] problems with freeBSD

[Peter Stuge]

Laszlo Ersek wrote:
> Going out on a limb, I suspect qemu commit 5f876756 instead.
..
> I think the gcc version Anthony was using miscompiled SeaBIOS

Yeah, it is very common. Lots of distributions patch their
toolchains such that they don't compile firmware code correctly.

Firmware isn't a common target, so I understand that it happens.

For coreboot we recommend to build a vanilla toolchain using known
good versions. We also have a script to do that. (The script is at
util/crossgcc/buildgcc in the coreboot repo.)


//Peter

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Doug Goldstein]

On Wed, Mar 6, 2013 at 7:58 PM, Peter Stuge <peter@stuge.se> wrote:
> Laszlo Ersek wrote:
>> Going out on a limb, I suspect qemu commit 5f876756 instead.
> ..
>> I think the gcc version Anthony was using miscompiled SeaBIOS
>
> Yeah, it is very common. Lots of distributions patch their
> toolchains such that they don't compile firmware code correctly.
>
> Firmware isn't a common target, so I understand that it happens.
>
> For coreboot we recommend to build a vanilla toolchain using known
> good versions. We also have a script to do that. (The script is at
> util/crossgcc/buildgcc in the coreboot repo.)
>
>
> //Peter
>

Peter,

Not sure if you or Kevin builds the coreboot images at
http://code.coreboot.org/p/seabios/downloads/, but would you guys
consider building point release as well (e.g. 1.7.2.1), if yes then my
next question...

The rest of the ML,

Would qemu consider using those blobs rather than different developers
using their distro toolchain to build up a random commit ID. I say
random only because often qemu releases ship with a non-release
seabios.

In this past there's been other issues related to seabios + qemu, that
I've solved in Gentoo by simply using the coreboot seabios images
after some troubleshooting help from Peter. Similarly our Ubuntu
OpenStack machines at work had quirks resolved by dropping the
coreboot seabios images on them.

-- 
Doug Goldstein

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Peter Maydell]

On 7 March 2013 14:12, Doug Goldstein <cardoe@gentoo.org> wrote:
> On Wed, Mar 6, 2013 at 7:58 PM, Peter Stuge <peter@stuge.se> wrote:
>> For coreboot we recommend to build a vanilla toolchain using known
>> good versions. We also have a script to do that. (The script is at
>> util/crossgcc/buildgcc in the coreboot repo.)

> Would qemu consider using those blobs rather than different developers
> using their distro toolchain to build up a random commit ID. I say
> random only because often qemu releases ship with a non-release
> seabios.

The right way to fix this is for QEMU to stop shipping binary BIOS
blobs in its source tarballs. Instead the developers of the BIOS
should release them separately with an appropriate testing
and release process which ensures that they're compiled with
whichever toolchain is required.

(just my opinion of course, which I state mostly to give people
something to disagree with...)

-- PMM

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Michael Tokarev]

07.03.2013 10:12, Doug Goldstein wrote:
> On Wed, Mar 6, 2013 at 7:58 PM, Peter Stuge <peter@stuge.se> wrote:

>> Yeah, it is very common. Lots of distributions patch their
>> toolchains such that they don't compile firmware code correctly.
>>
>> Firmware isn't a common target, so I understand that it happens.
>>
>> For coreboot we recommend to build a vanilla toolchain using known
>> good versions. We also have a script to do that. (The script is at
>> util/crossgcc/buildgcc in the coreboot repo.)
[]
> Not sure if you or Kevin builds the coreboot images at
> http://code.coreboot.org/p/seabios/downloads/, but would you guys
> consider building point release as well (e.g. 1.7.2.1), if yes then my
> next question...
> 
> The rest of the ML,
> 
> Would qemu consider using those blobs rather than different developers
> using their distro toolchain to build up a random commit ID. I say
> random only because often qemu releases ship with a non-release
> seabios.
> 
> In this past there's been other issues related to seabios + qemu, that
> I've solved in Gentoo by simply using the coreboot seabios images
> after some troubleshooting help from Peter. Similarly our Ubuntu
> OpenStack machines at work had quirks resolved by dropping the
> coreboot seabios images on them.

Just a note, or an "alternative opinion", so to say.  In Debian, we have
a social contract which, among other things, ensures that the binaries
you, as a user, get, comes with source which you can modify on the
system you installed.  This requires, for example, that all binaries
shipped are actually built from the corresponding source, and that
no blobs from whatever other sources are used, ever.

We don't even ship any upstream blobs in the debian qemu _source_
package: we repack upstream qemu.tar.gz by removing these blobs.

So from this PoV, it'd be better if upstream didn't ship any blobs
at all, -- we wont need to repack the upstream .tar.gz ;)

FWIW, prpbably we were lucky so far: we had almost no issues with
(mis)compiling of various ROM codes (bochsbios, seabios, vgabios,
pxe roms, various sparc/ppc firmware etc) using toolchains provided
by Debian.

Thanks,

/mjt

Re: [Qemu-devel] problems with freeBSD

[Aurelien Jarno]

On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
> On Thu, Mar 07, 2013 at 12:12:08AM +0100, Aurelien Jarno wrote:
> > On Wed, Mar 06, 2013 at 08:21:11AM +0000, Dietmar Maurer wrote:
> > > Using qemu 1.4.0:
> > > 
> > > # qemu -hda test.raw -m 512 -cdrom pfSense-LiveCD-2.0.2-RELEASE-amd64-20121207-2239.iso
> > > 
> > > Results in:
> > > 
> > > trap 12: page fault while in kernel mode
> > > ...
> > > stopped at x86bios_emu_rdw+0x2f: movzwl (%rbx),%eax
> > > 
> > > Any ideas? Can somebody reproduce that?
> > > 
> > > To get the FreeBSD VM boot use the console, enter the boot loader, then:
> > > # set hint.atkbd.0.disabled="1"
> > > # boot
> > > 
> > > But that disables the keyboard.
> > 
> > I was actually digging about that problem. It is indeed present in
> > version 1.4.0, but is fixed in the current git master. The problem is
> > actually not directly in QEMU but in seabios, the update to version
> > 1.7.2.1 commit 5c75fb10) fixes the issue. Maybe it is worth 
> > cherry-picking it into stable-1.4 (hence the Cc:). In the meantime
> > using bios.bin from master with QEMU version 1.4.0 should also fix the
> > issue.
> > 
> > What is strange is the seabios commit fixing the issue:
> > 
> >     commit 4219149ad2b783abfa61e80e9e9f6910db0c76c9
> >     Author: Kevin O'Connor <kevin@koconnor.net>
> >     Date:   Sun Feb 17 10:56:10 2013 -0500
> >     
> >         build: Don't require $(OUT) to be a sub-directory of the main directory.
> 
> That change is definitely just build related - I don't see how it
> could impact the final SeaBIOS binary.  How did you conclude that this
> commit is what fixes the issue?
> 

I did a git bisect to find the commit fixing the issue. Then, as I was
not believing the result, I tried the following sequence a dozen of
times (for some unknown reasons the FreeBSD install CD doesn't exhibit
the issue, so I used the Debian GNU/kFreeBSD installer):

| mkdir qemu-freebsd-bug
| cd qemu-freebsd-bug
|
| wget http://ftp.debian.org/debian/dists/squeeze/main/installer-kfreebsd-amd64/current/images/netboot/mini.iso 
|
| git clone git://git.qemu.org/qemu.git
| cd qemu
| git checkout -b stable-1.4 v1.4.0
| ./configure --target-list=x86_64-softmmu
| make
| cd ..
|
| git clone git://git.seabios.org/seabios.git
| cd seabios
| git checkout -b 1.7.2-stable origin/1.7.2-stable
| git reset --hard 4219149ad2b783abfa61e80e9e9f6910db0c76c9
| make
| cp out/bios.bin ../qemu/pc-bios
| cd..
|
| # debian-installer boots correctly 
| ./qemu/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cdrom mini.iso
|
| cd seabios
| git reset --hard d75c22fcb6521dad11428b65789d92f89675c600 
| git clean -fdx
| make
| cp out/bios.bin ../qemu/pc-bios
| cd ..
|
| # debian-installer fails to boot
| ./qemu/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cdrom mini.iso 


Maybe I am doing something wrong or there is a bug in my toolchain
(Debian Sid). It would be nice if someone could try to reproduce that on
another distro/system.

-- 
Aurelien Jarno                          GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] problems with freeBSD

[Gerd Hoffmann]

  Hi,

> Probably works, but never appeared in a separate release:
> 
>     commit 3588185b8396eb97fd9efd41c2b97775465f67c4
>     Author: Gerd Hoffmann <kraxel@redhat.com>
>     Date:   Mon Jan 21 09:17:16 2013 +0100
> 
>         seabios: update to 1.7.2 release

Built with "gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-3)" (rhel-6).

>     commit 5f876756c57c15f5e14d4136fc432b74f05f082b
>     Author: Anthony Liguori <aliguori@us.ibm.com>
>     Date:   Wed Feb 6 05:12:06 2013 -0600
> 
>         bios: recompile BIOS

>         gcc version 4.7.2 20121109 (Red Hat 4.7.2-8) (GCC)

>     commit 5c75fb10029c5fd1e705a6ef5d698fbea06c7a33
>     Author: Gerd Hoffmann <kraxel@redhat.com>
>     Date:   Thu Feb 28 09:18:56 2013 +0100
> 
>         update seabios to 1.7.2.1

Built with "gcc (GCC) 4.7.2 20121015 (Red Hat 4.7.2-5)"
(rhel-6 devtoolkit-1.1) as gcc 4.4 builds are too big and bumb seabios
size from 128k to 256k.  Which was the reason for anthonys rebuild.

cheers,
  Gerd

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Gerd Hoffmann]

  Hi,

>> Would qemu consider using those blobs rather than different developers
>> using their distro toolchain to build up a random commit ID. I say
>> random only because often qemu releases ship with a non-release
>> seabios.

It happened (well, the snapshot thing).  But that isn't our preferred
model, it was just bad release cycle coordination and we'll try to avoid
doing it again.  There is reason why we have a 1.7.2 stable branch now.

The binary blobs are just a convenience thing because rebuilding the
firmware might not be that easy.  Not so much for virtualization where
host + firmware are the same arch.  But for emulation, where you need
the -- say -- ppc slof on a x86 machine, and the binaries shipped allow
people to get started without having to install (or compile) a cross
compiler first.

It is certainly not required to use blobs shipped.  In fact the release
tarballs include all the firmware submodules, so you can build the
firmware yourself if you want.  I've created roms/Makefile so you can
easily rebuild the (x86) firmware.  seabios is there, vgabios is there,
patches for ipxe are in flight.

> Just a note, or an "alternative opinion", so to say.  In Debian, we have
> a social contract which, among other things, ensures that the binaries
> you, as a user, get, comes with source which you can modify on the
> system you installed.  This requires, for example, that all binaries
> shipped are actually built from the corresponding source, and that
> no blobs from whatever other sources are used, ever.

That is perfectly fine.  How to you handle ppc firmware for x86 hosts
btw?  Build on ppc buildhost and ship as noarch package?  Or do you do
cross compiler builds, so that users can patch+rebuild it on their x86
host too?

> We don't even ship any upstream blobs in the debian qemu _source_
> package: we repack upstream qemu.tar.gz by removing these blobs.

That's a bit over the top for my taste as the release tarballs include
both source and blobs.  Although it might be the debian release is just
a bit too old for that, not fully sure with which release anthony
started to include the firmware submodules, might be it was after 1.1

cheers,
  Gerd

Re: [Qemu-devel] problems with freeBSD

[Laszlo Ersek]

On 03/07/13 09:43, Aurelien Jarno wrote:

> I did a git bisect to find the commit fixing the issue. Then, as I was
> not believing the result, I tried the following sequence a dozen of
> times (for some unknown reasons the FreeBSD install CD doesn't exhibit
> the issue, so I used the Debian GNU/kFreeBSD installer):
> 
> | mkdir qemu-freebsd-bug
> | cd qemu-freebsd-bug
> |
> | wget http://ftp.debian.org/debian/dists/squeeze/main/installer-kfreebsd-amd64/current/images/netboot/mini.iso 
> |
> | git clone git://git.qemu.org/qemu.git
> | cd qemu
> | git checkout -b stable-1.4 v1.4.0
> | ./configure --target-list=x86_64-softmmu
> | make
> | cd ..
> |
> | git clone git://git.seabios.org/seabios.git
> | cd seabios
> | git checkout -b 1.7.2-stable origin/1.7.2-stable
> | git reset --hard 4219149ad2b783abfa61e80e9e9f6910db0c76c9
> | make
> | cp out/bios.bin ../qemu/pc-bios
> | cd..
> |
> | # debian-installer boots correctly 
> | ./qemu/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cdrom mini.iso
> |
> | cd seabios
> | git reset --hard d75c22fcb6521dad11428b65789d92f89675c600 
> | git clean -fdx
> | make
> | cp out/bios.bin ../qemu/pc-bios
> | cd ..
> |
> | # debian-installer fails to boot
> | ./qemu/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cdrom mini.iso 
> 
> 
> Maybe I am doing something wrong or there is a bug in my toolchain
> (Debian Sid). It would be nice if someone could try to reproduce that on
> another distro/system.
> 

Can you save the out/ directory from both builds and "diff -ur" them
(maybe just the *.lds files)?

I'm noticing that pathnames are embedded in some ELF section names (I
hope this sentence makes sense), and when you build at d75c22fc, those
pathnames contain dot-dot (".."), ie. two section name separators next
to each other. Maybe that's not good; no idea.

Laszlo

Re: [Qemu-devel] problems with freeBSD

[Don Slutz]

[-- Attachment #1: Type: text/plain, Size: 640 bytes --]

On 03/07/13 03:43, Aurelien Jarno wrote:
> On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
>> >On Thu, Mar 07, 2013 at 12:12:08AM +0100, Aurelien Jarno wrote:
>>> > >On Wed, Mar 06, 2013 at 08:21:11AM +0000, Dietmar Maurer wrote:
[snip]
>>> Maybe I am doing something wrong or there is a bug in my toolchain 
>>> (Debian Sid). It would be nice if someone could try to reproduce 
>>> that on another distro/system. 
> -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurelien@aurel32.net 
> http://www.aurel32.net

I reproduced this on "Fedora release 17 (Beefy Miracle)" 
3.7.3-101.fc17.x86_64 doing the steps provided.
    -Don Slutz

[-- Attachment #2: Type: text/html, Size: 2258 bytes --]

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Michael Tokarev]

07.03.2013 15:56, Gerd Hoffmann wrote:

>> Just a note, or an "alternative opinion", so to say.  In Debian, we have
>> a social contract which, among other things, ensures that the binaries
>> you, as a user, get, comes with source which you can modify on the
>> system you installed.  This requires, for example, that all binaries
>> shipped are actually built from the corresponding source, and that
>> no blobs from whatever other sources are used, ever.
> 
> That is perfectly fine.  How to you handle ppc firmware for x86 hosts
> btw?  Build on ppc buildhost and ship as noarch package?  Or do you do
> cross compiler builds, so that users can patch+rebuild it on their x86
> host too?

"Foreign" firmware handling is an unsolved issue.  Yes, basically,
it is built on the corresponding build machine (eg, openbios-ppc is
built on a ppc machine) and shipped as "noarch" package.  The same
is for X86 seabios/vgabios/etc stuff actually.

So technically this is a violation in some way.  However, that same
ppc firmware actually can be rebuilt in a (qemu) virtual machine with
the same debian release installed.

That's why I say it is basically unsolved issue -- the "solution"
isn't actuall a solution but a workaround instead.

For added "fun", currently we ship optionroms from qemu in seabios
source package, exactly because of this reason - seabios is what
gets built on x86 only and is distributed on as "noarch" binary,
while qemu is built on all architectures.

And for another fun, we've several broken qemu architectures in
Debian at the moment -- s390 and ppc64 are missing firmwares
exactly because of the difficulties building them, despite the
fact they're shipping in upstream qemu tarball.

>> We don't even ship any upstream blobs in the debian qemu _source_
>> package: we repack upstream qemu.tar.gz by removing these blobs.
> 
> That's a bit over the top for my taste as the release tarballs include
> both source and blobs.

Yes this is a bit extremistic, so to say.  But this is the same
base principle of Debian: we should ensure and demonstrate it all
is buildable.  Just mere presence of a blob in a tarball is already
suspicious :)

>   Although it might be the debian release is just
> a bit too old for that, not fully sure with which release anthony
> started to include the firmware submodules, might be it was after 1.1

Nope, qemu started including sources before that.  But it doesn't
matter really, it is just the way how debian works, nothing to
do with qemu really.  Especially since most of these submodules
are already packaged in Debian separately (be it because the
qemu needs or due to other means).

Thanks,

/mjt

Re: [Qemu-devel] problems with freeBSD

[Aurelien Jarno]

[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

On Thu, Mar 07, 2013 at 01:16:54PM +0100, Laszlo Ersek wrote:
> On 03/07/13 09:43, Aurelien Jarno wrote:
> 
> > I did a git bisect to find the commit fixing the issue. Then, as I was
> > not believing the result, I tried the following sequence a dozen of
> > times (for some unknown reasons the FreeBSD install CD doesn't exhibit
> > the issue, so I used the Debian GNU/kFreeBSD installer):
> > 
> > | mkdir qemu-freebsd-bug
> > | cd qemu-freebsd-bug
> > |
> > | wget http://ftp.debian.org/debian/dists/squeeze/main/installer-kfreebsd-amd64/current/images/netboot/mini.iso 
> > |
> > | git clone git://git.qemu.org/qemu.git
> > | cd qemu
> > | git checkout -b stable-1.4 v1.4.0
> > | ./configure --target-list=x86_64-softmmu
> > | make
> > | cd ..
> > |
> > | git clone git://git.seabios.org/seabios.git
> > | cd seabios
> > | git checkout -b 1.7.2-stable origin/1.7.2-stable
> > | git reset --hard 4219149ad2b783abfa61e80e9e9f6910db0c76c9
> > | make
> > | cp out/bios.bin ../qemu/pc-bios
> > | cd..
> > |
> > | # debian-installer boots correctly 
> > | ./qemu/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cdrom mini.iso
> > |
> > | cd seabios
> > | git reset --hard d75c22fcb6521dad11428b65789d92f89675c600 
> > | git clean -fdx
> > | make
> > | cp out/bios.bin ../qemu/pc-bios
> > | cd ..
> > |
> > | # debian-installer fails to boot
> > | ./qemu/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -cdrom mini.iso 
> > 
> > 
> > Maybe I am doing something wrong or there is a bug in my toolchain
> > (Debian Sid). It would be nice if someone could try to reproduce that on
> > another distro/system.
> > 
> 
> Can you save the out/ directory from both builds and "diff -ur" them
> (maybe just the *.lds files)?

Please find it attached.

> I'm noticing that pathnames are embedded in some ELF section names (I
> hope this sentence makes sense), and when you build at d75c22fc, those
> pathnames contain dot-dot (".."), ie. two section name separators next
> to each other. Maybe that's not good; no idea.
> 

It's clearly the case, but I also don't know if it is a good solution or
not. What is sure is that it also changes the address of the text
section in bios.bin.elf.

-- 
Aurelien Jarno	                        GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

[-- Attachment #2: seabios.qemu-freebsd-bug.diff.xz --]
[-- Type: application/octet-stream, Size: 37720 bytes --]

Re: [Qemu-devel] problems with freeBSD

[Kevin O'Connor]

On Thu, Mar 07, 2013 at 09:43:04AM +0100, Aurelien Jarno wrote:
> On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
> > That change is definitely just build related - I don't see how it
> > could impact the final SeaBIOS binary.  How did you conclude that this
> > commit is what fixes the issue?
> > 
> 
> I did a git bisect to find the commit fixing the issue. Then, as I was
> not believing the result, I tried the following sequence a dozen of
> times (for some unknown reasons the FreeBSD install CD doesn't exhibit
> the issue, so I used the Debian GNU/kFreeBSD installer):

Thanks.  I'll take a look at this tonight.  If you get a chance first,
could you try adding "make distclean; make clean" into the seabios
steps?

-Kevin

Re: [Qemu-devel] problems with freeBSD

[Don Slutz]

[-- Attachment #1: Type: text/plain, Size: 4421 bytes --]

On 03/07/13 08:02, Don Slutz wrote:
> On 03/07/13 03:43, Aurelien Jarno wrote:
>> On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
>>> >On Thu, Mar 07, 2013 at 12:12:08AM +0100, Aurelien Jarno wrote:
>>>> > >On Wed, Mar 06, 2013 at 08:21:11AM +0000, Dietmar Maurer wrote:
> [snip]
>>>> Maybe I am doing something wrong or there is a bug in my toolchain 
>>>> (Debian Sid). It would be nice if someone could try to reproduce 
>>>> that on another distro/system. 
>> -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurelien@aurel32.net 
>> http://www.aurel32.net
>
> I reproduced this on "Fedora release 17 (Beefy Miracle)" 
> 3.7.3-101.fc17.x86_64 doing the steps provided.
>    -Don Slutz

Turns out that this is not the normal kind of issue.  Newer seabios 
works, older does not:

good   * 88cb66e (HEAD, tag: rel-1.7.2.1, origin/1.7.2-stable, 
1.7.2-stable) seabios: Add a dummy PCI slot to irq mapping funct
good   * 985a9d3 seabios q35: Add new PCI slot to irq routing function
good   * 2b57726 seabios q35: Enable all PIRQn IRQs at startup
good   * e5fe4f9 Verify CC is valid during build tests.
good   * 4219149 build: Don't require $(OUT) to be a sub-directory of 
the main directory.
bad    * d75c22f Fix Makefile - don't reference "out/" directly, instead 
use "$(OUT)".
bad    * 12e8199 USB-EHCI: Fix null pointer assignment
bad    * f396871 Update tools/acpi_extract.py to handle iasl 20130117 
release.
bad    * 4bd8aeb (tag: rel-1.7.2) vgabios: implement AX=1120H..1124H 
functions


good:
Version: rel-1.7.2-4-g4219149-20130307_085117-don-lt.don.CloudSwitch.com
Fixed space: 0xe05b-0x10000  total: 8101  slack: 5  Percent slack: 0.1%
16bit size:           39856
32bit segmented size: 1430
32bit flat size:      18778
32bit flat init size: 62400
Lowmem size:          2176

bad:
Version: rel-1.7.2-3-gd75c22f-20130307_085345-don-lt.don.CloudSwitch.com
Fixed space: 0xe05b-0x10000  total: 8101  slack: 5  Percent slack: 0.1%
16bit size:           39808
32bit segmented size: 1430
32bit flat size:      18778
32bit flat init size: 62400
Lowmem size:          2176


The 16bit size change is the only output change.

The changed that fixed it:

commit 4219149ad2b783abfa61e80e9e9f6910db0c76c9
Author: Kevin O'Connor <kevin@koconnor.net>
Date:   Sun Feb 17 10:56:10 2013 -0500

     build: Don't require $(OUT) to be a sub-directory of the main 
directory.

     Remove references to "../" and "out/" from the build so that "make
     OUT=/a/b/c/" will work.

     Signed-off-by: Kevin O'Connor <kevin@koconnor.net>

diff --git a/Makefile b/Makefile
index a482c94..20da6d0 100644
--- a/Makefile
+++ b/Makefile
@@ -84,7 +84,7 @@ vpath %.S src vgasrc
  ################ Common build rules

  # Verify the build environment works.
-TESTGCC:=$(shell CC="$(CC)" LD="$(LD)" IASL="$(IASL)" tools/test-build.sh)
+TESTGCC:=$(shell OUT="$(OUT)" CC="$(CC)" LD="$(LD)" IASL="$(IASL)" 
tools/test-build.sh)
  ifeq "$(TESTGCC)" "-1"
  $(error "Please upgrade the build environment")
  endif
@@ -97,7 +97,7 @@ endif
  # Do a whole file compile by textually including all C code.
  define whole-compile
  @echo "  Compiling whole program $3"
-$(Q)printf '$(foreach i,$2,#include "../$i"\n)' > $3.tmp.c
+$(Q)printf '$(foreach i,$2,#include "$(CURDIR)/$i"\n)' > $3.tmp.c
  $(Q)$(CC) $1 $(CFLAGSWHOLE) -c $3.tmp.c -o $3
  endef

diff --git a/src/Kconfig b/src/Kconfig
index 0b112ed..2c9100d 100644
--- a/src/Kconfig
+++ b/src/Kconfig
@@ -367,7 +367,7 @@ menu "BIOS Tables"
              Support generation of ACPI tables.
  endmenu

-source ../vgasrc/Kconfig
+source vgasrc/Kconfig

  menu "Debugging"
      config DEBUG_LEVEL
diff --git a/tools/test-build.sh b/tools/test-build.sh
index 7bd6d1f..ce0aca9 100755
--- a/tools/test-build.sh
+++ b/tools/test-build.sh
@@ -14,13 +14,13 @@ if [ $? -ne 0 ]; then
      exit 0
  fi

-mkdir -p out
-TMPFILE1=out/tmp_testcompile1.c
-TMPFILE1o=out/tmp_testcompile1.o
-TMPFILE1_ld=out/tmp_testcompile1.lds
-TMPFILE2=out/tmp_testcompile2.c
-TMPFILE2o=out/tmp_testcompile2.o
-TMPFILE3o=out/tmp_testcompile3.o
+mkdir -p ${OUT}
+TMPFILE1=${OUT}/tmp_testcompile1.c
+TMPFILE1o=${OUT}/tmp_testcompile1.o
+TMPFILE1_ld=${OUT}/tmp_testcompile1.lds
+TMPFILE2=${OUT}/tmp_testcompile2.c
+TMPFILE2o=${OUT}/tmp_testcompile2.o
+TMPFILE3o=${OUT}/tmp_testcompile3.o

  # Test if ld's alignment handling is correct.  This is a known problem
  # with the linker that ships with Ubuntu 11.04.

   -Don Slutz

[-- Attachment #2: Type: text/html, Size: 7373 bytes --]

Re: [Qemu-devel] [SeaBIOS] problems with freeBSD

[Don Slutz]

On 03/07/13 08:57, Kevin O'Connor wrote:
> On Thu, Mar 07, 2013 at 09:43:04AM +0100, Aurelien Jarno wrote:
>> On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
>>> That change is definitely just build related - I don't see how it
>>> could impact the final SeaBIOS binary.  How did you conclude that this
>>> commit is what fixes the issue?
>>>
>> I did a git bisect to find the commit fixing the issue. Then, as I was
>> not believing the result, I tried the following sequence a dozen of
>> times (for some unknown reasons the FreeBSD install CD doesn't exhibit
>> the issue, so I used the Debian GNU/kFreeBSD installer):
> Thanks.  I'll take a look at this tonight.  If you get a chance first,
> could you try adding "make distclean; make clean" into the seabios
> steps?
>
> -Kevin
>
> _______________________________________________
> SeaBIOS mailing list
> SeaBIOS@seabios.org
> http://www.seabios.org/mailman/listinfo/seabios
    make distclean;make clean

Do not change what happens.

    git clean -fdx

looks to be good enough.
     -Don

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Peter Maydell]

On 7 March 2013 19:56, Gerd Hoffmann <kraxel@redhat.com> wrote:

>> We don't even ship any upstream blobs in the debian qemu _source_
>> package: we repack upstream qemu.tar.gz by removing these blobs.
>
> That's a bit over the top for my taste as the release tarballs include
> both source and blobs.

qemu-linaro's source tarballs (and indeed git tree) drops the
blobs as well. I didn't want to have to deal with the issues
of making sure the sources were present and matched the blobs.

In general having blobs in our allegedly source tarball is pretty
ugly. Either it's a source release or it isn't. We can do a
release-of-blobs tarball too if we want, but it doesn't need
to be in the source tarball and it doesn't need to be in git
either IMHO.

-- PMM

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Gerd Hoffmann]

  Hi,

> In general having blobs in our allegedly source tarball is pretty
> ugly. Either it's a source release or it isn't. We can do a
> release-of-blobs tarball too if we want, but it doesn't need
> to be in the source tarball

This is easily doable, just an update to scripts/make-release to spew
two tarballs (one source, one blobs).

We probably want update the build process to build the blobs by default
(if the compilers needed are installed).  Blueswirl started that already
for sparc firmware I think.

> and it doesn't need to be in git
> either IMHO.

That one is a bit more tricky.  The big advantage git has here is that
the update of a blob is not different from other updates.  It is just a
pull request.  Keeping source+binaries in sync is easy too: we can
update submodule hash + blobs with a single git commit.

The process needs to stay simple, otherwise we'll risk that the blobs we
provide will bitrot ...

cheers,
  Gerd

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Peter Maydell]

On 7 March 2013 23:56, Gerd Hoffmann <kraxel@redhat.com> wrote:
> Peter Maydell wrote:
>> In general having blobs in our allegedly source tarball is pretty
>> ugly. Either it's a source release or it isn't. We can do a
>> release-of-blobs tarball too if we want, but it doesn't need
>> to be in the source tarball
>
> This is easily doable, just an update to scripts/make-release to spew
> two tarballs (one source, one blobs).
>
> We probably want update the build process to build the blobs by default
> (if the compilers needed are installed).  Blueswirl started that already
> for sparc firmware I think.

Earlier in this thread it's been stated that this often produces
subtly broken blobs...

>> and it doesn't need to be in git
>> either IMHO.
>
> That one is a bit more tricky.  The big advantage git has here is that
> the update of a blob is not different from other updates.  It is just a
> pull request.  Keeping source+binaries in sync is easy too: we can
> update submodule hash + blobs with a single git commit.

You could put the sources for the binary blobs in their own git
repo, and then have the blobs in that git repo rather than qemu
mainline.

-- PMM

Re: [Qemu-devel] problems with freeBSD

[Paolo Bonzini]

Il 07/03/2013 15:00, Don Slutz ha scritto:
> Turns out that this is not the normal kind of issue.  Newer seabios
> works, older does not:
> 
> good   * 88cb66e (HEAD, tag: rel-1.7.2.1, origin/1.7.2-stable,
> 1.7.2-stable) seabios: Add a dummy PCI slot to irq mapping funct
> good   * 985a9d3 seabios q35: Add new PCI slot to irq routing function
> good   * 2b57726 seabios q35: Enable all PIRQn IRQs at startup
> good   * e5fe4f9 Verify CC is valid during build tests.
> good   * 4219149 build: Don't require $(OUT) to be a sub-directory of
> the main directory.
> bad    * d75c22f Fix Makefile - don't reference "out/" directly, instead
> use "$(OUT)".
> bad    * 12e8199 USB-EHCI: Fix null pointer assignment
> bad    * f396871 Update tools/acpi_extract.py to handle iasl 20130117
> release.
> bad    * 4bd8aeb (tag: rel-1.7.2) vgabios: implement AX=1120H..1124H
> functions

If so, you need to invert "bad" and "good" when bisecting.  Did you?

Paolo

Re: [Qemu-devel] problems with freeBSD

[Aurelien Jarno]

On Thu, Mar 07, 2013 at 08:57:06AM -0500, Kevin O'Connor wrote:
> On Thu, Mar 07, 2013 at 09:43:04AM +0100, Aurelien Jarno wrote:
> > On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
> > > That change is definitely just build related - I don't see how it
> > > could impact the final SeaBIOS binary.  How did you conclude that this
> > > commit is what fixes the issue?
> > > 
> > 
> > I did a git bisect to find the commit fixing the issue. Then, as I was
> > not believing the result, I tried the following sequence a dozen of
> > times (for some unknown reasons the FreeBSD install CD doesn't exhibit
> > the issue, so I used the Debian GNU/kFreeBSD installer):
> 
> Thanks.  I'll take a look at this tonight.  If you get a chance first,
> could you try adding "make distclean; make clean" into the seabios
> steps?
> 

It doesn't change anything (which is more or less expected due to the
git clean -fdx). I have tried to also copy acpi-dsdt.aml and it doesn't
help (the file is the same in qemu v1.4.0, and in the "bad" and "good"
version of seabios).

-- 
Aurelien Jarno                          GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Aurelien Jarno]

On Fri, Mar 08, 2013 at 12:03:15AM +0800, Peter Maydell wrote:
> On 7 March 2013 23:56, Gerd Hoffmann <kraxel@redhat.com> wrote:
> > Peter Maydell wrote:
> >> In general having blobs in our allegedly source tarball is pretty
> >> ugly. Either it's a source release or it isn't. We can do a
> >> release-of-blobs tarball too if we want, but it doesn't need
> >> to be in the source tarball
> >
> > This is easily doable, just an update to scripts/make-release to spew
> > two tarballs (one source, one blobs).
> >
> > We probably want update the build process to build the blobs by default
> > (if the compilers needed are installed).  Blueswirl started that already
> > for sparc firmware I think.
> 
> Earlier in this thread it's been stated that this often produces
> subtly broken blobs...
> 

Would it be possible to have a testsuite to validate such blobs. For
example the breakage which happens recently is due to the size of the
bios reaching 128kB. This is something easy to check.

On the QEMU side, we should definitely have an autotest system trying to
boot a few guests (and if possible with some !linux) to detect such
issues. That's not something easy to do, maybe that could be a good
GSoC project?

-- 
Aurelien Jarno                          GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Dietmar Maurer]

> In general having blobs in our allegedly source tarball is pretty ugly. Either it's a
> source release or it isn't. We can do a release-of-blobs tarball too if we want,
> but it doesn't need to be in the source tarball and it doesn't need to be in git
> either IMHO.

I think it is very convenient to have that in git, because everyone can test with the
same binary - to detect error like this one.

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[Peter Stuge]

Aurelien Jarno wrote:
> > > We probably want update the build process to build the blobs by default
> > 
> > Earlier in this thread it's been stated that this often produces
> > subtly broken blobs...
> 
> Would it be possible to have a testsuite to validate such blobs.

The coreboot project has a test system which although not used much
does allow distributed testing. Maybe it could be replicated for
SeaBIOS.


//Peter

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

On Thu, 2013-03-07 at 16:56 +0100, Gerd Hoffmann wrote:
> > and it doesn't need to be in git either IMHO.
> 
> That one is a bit more tricky.  The big advantage git has here is that
> the update of a blob is not different from other updates.  It is just
> a pull request.  Keeping source+binaries in sync is easy too: we can
> update submodule hash + blobs with a single git commit.

FWIW I find the binaries in git to be a PITA. I submitted a patch in
January to fix the VGA BIOS but it's still broken in git. The one
installed in /usr/share/qemu is fine, but whenever I build qemu from git
I have to remember to *remove* the one in the git tree or it uses that
instead...

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]

Re: [Qemu-devel] problems with freeBSD

[Kevin O'Connor]

On Thu, Mar 07, 2013 at 09:43:04AM +0100, Aurelien Jarno wrote:
> On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
> > That change is definitely just build related - I don't see how it
> > could impact the final SeaBIOS binary.  How did you conclude that this
> > commit is what fixes the issue?
> > 
> 
> I did a git bisect to find the commit fixing the issue. Then, as I was
> not believing the result, I tried the following sequence a dozen of
> times (for some unknown reasons the FreeBSD install CD doesn't exhibit
> the issue, so I used the Debian GNU/kFreeBSD installer):
[...]

Thanks for the detailed bug report.  Here's what I see going on:

- the SeaBIOS 4219149a commit does change the resulting binary ever so
  slightly - the src/virtio_ring.c code has a reference to __FILE__
  (the only code in SeaBIOS that does that), and due to slightly
  different build rules in this commit it evaluates to a slightly
  different string.

- the freebsd crash has nothing to do with 4219149a or
  src/virtio_ring.c - instead, random changes in the seabios binary
  layout can cause (or avoid) the crash.  You can see this in action
  by modifying seabios to have higher debug levels, commenting out
  code, adding dprintf statements, etc.

- the crash happens when freebsd attempts to emulate the bios code (!)
  in order to determine the keyboard typematic rate (!).  (See
  sys/dev/atkbdc/atkbd.c.) Since SeaBIOS doesn't support the typematic
  callback rate (int 0x16 ax=0x0306) this doesn't actually achieve
  anything in practice were the call to not crash.  However, a crash
  does (sometimes) result.

- the freebsd x86bios_get_pages() code is buggy (See
  sys/compat/x86bios/x86bios.c).  It attempts to check that its x86
  emulater (!) doesn't access a page it hasn't mapped.  However, it
  does not check for the case where a two byte access spans two pages.
  If the first page is mapped, but the second is not - splat.  The
  crash I've seen in QEMU had a two byte access to 0xffffff8000015fff
  with the fault at 0xffffff8000016000.

- I have not been able to determine why an attempt was made to access
  a non-mapped page.  My best guess is that the x86emu code (!) goes
  off the deep-end in all cases - just some cases lead it to the bug
  above and other cases lead it to a more friendly termination.
  (Recall that SeaBIOS doesn't support the typematic call anyway.)  It
  should be possible to track this down by adding debug statements to
  the freebsd code if anyone is familiar with the freebsd kernel
  compile-deploy-run cycle.

-Kevin

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[Paolo Bonzini]

Il 07/03/2013 21:24, David Woodhouse ha scritto:
> On Thu, 2013-03-07 at 16:56 +0100, Gerd Hoffmann wrote:
>>> and it doesn't need to be in git either IMHO.
>>
>> That one is a bit more tricky.  The big advantage git has here is that
>> the update of a blob is not different from other updates.  It is just
>> a pull request.  Keeping source+binaries in sync is easy too: we can
>> update submodule hash + blobs with a single git commit.
> 
> FWIW I find the binaries in git to be a PITA. I submitted a patch in
> January to fix the VGA BIOS but it's still broken in git. The one
> installed in /usr/share/qemu is fine, but whenever I build qemu from git
> I have to remember to *remove* the one in the git tree or it uses that
> instead...

Wasn't that a fix for the SeaBIOS VGA BIOS?  The one in qemu.git is
still built from the Bochs VGA BIOS.

Last year, malc blocked the switch on the lack of protected mode
interface in SeaVGABIOS.

Paolo

Re: [Qemu-devel] [Qemu-stable] problems with freeBSD

[Paolo Bonzini]

Il 07/03/2013 17:21, Aurelien Jarno ha scritto:
> 
> On the QEMU side, we should definitely have an autotest system trying to
> boot a few guests (and if possible with some !linux) to detect such
> issues. That's not something easy to do, maybe that could be a good
> GSoC project?

There is virt-test (formerly known as KVM Autotest).  I did some of the
lifting required to have !x86 guests, but haven't yet actually created
the images.

We (Red Hat) use that to run sanity tests on qemu.git.  These tests last
~2.5 hours and include guest installations.

Alternatively, you can use a pre-built image (the download is ~150MB
compressed) and run tests on it.  See the patch at
http://permalink.gmane.org/gmane.comp.emulators.qemu/197856 that adds
"make check-autotest" to QEMU.  The tests it runs last a little less
than 1 hour.

Paolo

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 286 bytes --]

On Fri, 2013-03-08 at 08:47 +0100, Paolo Bonzini wrote:
> Wasn't that a fix for the SeaBIOS VGA BIOS?  The one in qemu.git is
> still built from the Bochs VGA BIOS.

No, it was for Bochs VGA BIOS.

http://lists.gnu.org/archive/html/qemu-devel/2013-01/msg03650.html

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 6171 bytes --]

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[Gerd Hoffmann]

On 03/08/13 08:47, Paolo Bonzini wrote:
>> FWIW I find the binaries in git to be a PITA. I submitted a patch in
>> January to fix the VGA BIOS but it's still broken in git. The one
>> installed in /usr/share/qemu is fine, but whenever I build qemu from git
>> I have to remember to *remove* the one in the git tree or it uses that
>> instead...
> 
> Wasn't that a fix for the SeaBIOS VGA BIOS?  The one in qemu.git is
> still built from the Bochs VGA BIOS.
> 
> Last year, malc blocked the switch on the lack of protected mode
> interface in SeaVGABIOS.

Fedora switched though, so the distro binaries in /usr/share/qemu are
the seavgabios ones.

"make -C roms seavgabios" in the qemu source tree will upgrade the blobs
in pc-bios/.

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[Paolo Bonzini]

Il 08/03/2013 09:20, Gerd Hoffmann ha scritto:
> On 03/08/13 08:47, Paolo Bonzini wrote:
>>> FWIW I find the binaries in git to be a PITA. I submitted a patch in
>>> January to fix the VGA BIOS but it's still broken in git. The one
>>> installed in /usr/share/qemu is fine, but whenever I build qemu from git
>>> I have to remember to *remove* the one in the git tree or it uses that
>>> instead...
>>
>> Wasn't that a fix for the SeaBIOS VGA BIOS?  The one in qemu.git is
>> still built from the Bochs VGA BIOS.
>>
>> Last year, malc blocked the switch on the lack of protected mode
>> interface in SeaVGABIOS.
> 
> Fedora switched though, so the distro binaries in /usr/share/qemu are
> the seavgabios ones.

Did it?  This is F18:

$ rpm -q --requires qemu-system-x86|grep bios
seabios-bin >= 0.6.0-2
sgabios-bin
vgabios >= 0.6c-2

$ rpm -q vgabios
vgabios-0.6c-5.fc18.noarch

$ rpm -ql seabios-bin
/usr/share/seabios
/usr/share/seabios/bios.bin

$ rpm -qf /usr/share/vgabios/
vgabios-0.6c-5.fc18.noarch

Paolo

> "make -C roms seavgabios" in the qemu source tree will upgrade the blobs
> in pc-bios/.
> 
> cheers,
>   Gerd
> 
> 

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[Gerd Hoffmann]

> Did it?  This is F18:
> 
> $ rpm -q --requires qemu-system-x86|grep bios
> seabios-bin >= 0.6.0-2
> sgabios-bin
> vgabios >= 0.6c-2

Oops.

There is a seavgabios-bin package too, and I through cole switched qemu
over, but apparently this didn't happen (or maybe in rawhide only) ...

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] problems with freeBSD

[Laszlo Ersek]

On 03/08/13 04:35, Kevin O'Connor wrote:
> On Thu, Mar 07, 2013 at 09:43:04AM +0100, Aurelien Jarno wrote:
>> On Wed, Mar 06, 2013 at 07:53:51PM -0500, Kevin O'Connor wrote:
>>> That change is definitely just build related - I don't see how it
>>> could impact the final SeaBIOS binary.  How did you conclude that this
>>> commit is what fixes the issue?
>>>
>>
>> I did a git bisect to find the commit fixing the issue. Then, as I was
>> not believing the result, I tried the following sequence a dozen of
>> times (for some unknown reasons the FreeBSD install CD doesn't exhibit
>> the issue, so I used the Debian GNU/kFreeBSD installer):
> [...]
> 
> Thanks for the detailed bug report.  Here's what I see going on:
> 
> - the SeaBIOS 4219149a commit does change the resulting binary ever so
>   slightly - the src/virtio_ring.c code has a reference to __FILE__
>   (the only code in SeaBIOS that does that), and due to slightly
>   different build rules in this commit it evaluates to a slightly
>   different string.
> 
> - the freebsd crash has nothing to do with 4219149a or
>   src/virtio_ring.c - instead, random changes in the seabios binary
>   layout can cause (or avoid) the crash.  You can see this in action
>   by modifying seabios to have higher debug levels, commenting out
>   code, adding dprintf statements, etc.
> 
> - the crash happens when freebsd attempts to emulate the bios code (!)
>   in order to determine the keyboard typematic rate (!).  (See
>   sys/dev/atkbdc/atkbd.c.) Since SeaBIOS doesn't support the typematic
>   callback rate (int 0x16 ax=0x0306) this doesn't actually achieve
>   anything in practice were the call to not crash.  However, a crash
>   does (sometimes) result.
> 
> - the freebsd x86bios_get_pages() code is buggy (See
>   sys/compat/x86bios/x86bios.c).  It attempts to check that its x86
>   emulater (!) doesn't access a page it hasn't mapped.  However, it
>   does not check for the case where a two byte access spans two pages.
>   If the first page is mapped, but the second is not - splat.  The
>   crash I've seen in QEMU had a two byte access to 0xffffff8000015fff
>   with the fault at 0xffffff8000016000.
> 
> - I have not been able to determine why an attempt was made to access
>   a non-mapped page.  My best guess is that the x86emu code (!) goes
>   off the deep-end in all cases - just some cases lead it to the bug
>   above and other cases lead it to a more friendly termination.
>   (Recall that SeaBIOS doesn't support the typematic call anyway.)  It
>   should be possible to track this down by adding debug statements to
>   the freebsd code if anyone is familiar with the freebsd kernel
>   compile-deploy-run cycle.

Great analysis!

Laszlo
(sorry for the noise)

Re: [Qemu-devel] [Qemu-stable] [SeaBIOS] problems with freeBSD

[Kevin O'Connor]

On Fri, Mar 08, 2013 at 12:03:15AM +0800, Peter Maydell wrote:
> Earlier in this thread it's been stated that this often produces
> subtly broken blobs...

I think there have been some far reaching conclusions in this thread
based on incorrect premises.

SeaBIOS has had problems with mis-compilations in the past.  To handle
this, the SeaBIOS build runs test cases on every compilation to verify
the tool chain (see tools/test-build.sh).  I have not seen a report of
a guest visible SeaBIOS mis-compilation in many years.

There have been a few cases of guest bugs that get tickled when
changes to SeaBIOS cause slightly different memory layouts.  The last
case of this that comes to mind is the JunOS bug that was tracked down
a year and a half ago.  The root cause of the FreeBSD issue raised in
this thread is very similar - the full details aren't known yet, but
we do know that a slightly different memory layout is ticking a bug in
a FreeBSD page presence check.

Since changes are inevitable anyway (the only alternative would be to
never enhance SeaBIOS), I don't believe that dictating one "true
compiler" or one "true blob" is necessary or desirable.

-Kevin

Re: [Qemu-devel] [SeaBIOS] [Qemu-stable] problems with freeBSD

[Peter Stuge]

Kevin O'Connor wrote:
> I don't believe that dictating one "true compiler" or one "true
> blob" is necessary or desirable.

Sure. The point isn't to have only one correct way, but to have
reliable results in the common case.

A reference toolchain and a reference blob both help accomplish
that, but they're by no means the only, or even best, methods.


//Peter