From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: Kevin O'Connor <kevin@koconnor.net>
Cc: Andrey Korolyov <andrey@xdel.ru>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
Bandan Das <bsd@redhat.com>,
kraxel@redhat.com, Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [Qemu-devel] E5-2620v2 - emulation stop error
Date: Wed, 11 Mar 2015 16:52:03 +0000 [thread overview]
Message-ID: <20150311165203.GL2334@work-vm> (raw)
In-Reply-To: <20150311163739.GA29522@morn.localdomain>
* Kevin O'Connor (kevin@koconnor.net) wrote:
> On Wed, Mar 11, 2015 at 03:53:07PM +0000, Dr. David Alan Gilbert wrote:
> > * Kevin O'Connor (kevin@koconnor.net) wrote:
> > > On Wed, Mar 11, 2015 at 01:45:57PM +0000, Dr. David Alan Gilbert wrote:
> > > > * Bandan Das (bsd@redhat.com) wrote:
> > > > > "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> > > > > > while true; do (sleep 5; echo -e '\001cq\n')|/opt/qemu-try-world3/bin/qemu-system-x86_64 -machine pc-i440fx-2.0,accel=kvm -m 1024 -smp 128 -nographic -device sga 2>&1 | tee /tmp/qemu.op; grep "internal error" /tmp/qemu.op -q && break; done
>
> That is a truly impressive command line, BTW.
Thanks :-)
> > > > > > [root@virtlab413 qemu-world3]# git bisect bad
> > > > > > 21f5826a04d38e19488f917e1eef22751490c769 is the first bad commit
> > > > >
> > > > > I can reproduce this on E5-2620 v2 with David's "while true" test.
> > > > > (The emulation failure I mean, not the suberror 2 that Andrey is seeing)
> > > > > The commit that seems to have introduced this is -
> > > > >
> > > > > commit 0673b7870063a3affbad9046fb6d385a4e734c19
> > > > > Author: Kevin O'Connor <kevin@koconnor.net>
> > > > > Date: Sat May 24 10:49:50 2014 -0400
> > > > >
> > > > > smp: Replace QEMU SMP init assembler code with C; run only in 32bit mode.
> > > [...]
> > > > Turning on debug logging
> > > > ( -chardev file,id=log,path=/tmp/debugcon.$$ -device isa-debugcon,chardev=log,iobase=0x402 )
> > > >
> > > > SeaBIOS (version rel-1.8.0-0-g4c59f5d-20150219_092859-nilsson.home.kraxel.org)
> > > [...]
> > > > Found 1 cpu(s) max supported 128 cpu(s)
> > >
> > > Something is very odd here. When I run the above command (on an older
> > > AMD machine) I get:
> > >
> > > Found 128 cpu(s) max supported 128 cpu(s)
> > >
> > > That first value (1 vs 128) comes from QEMU (via cmos index 0x5f).
> > > That is, during smp init, SeaBIOS expects QEMU to tell it how many
> > > cpus are active, and SeaBIOS waits until that many CPUs check in from
> > > its SIPI request before proceeding.
> > >
> > > I wonder if QEMU reported only 1 active cpu via that cmos register,
> > > but more were actually active. If that was the case, it could
> > > certainly explain the failure - as multiple cpus could be running
> > > without the sipi trapoline in place.
> > >
> > > What does the log look like on a non-failure case?
> >
> > I had to drop down from 128 to get a working run with debug; here
> > are two runs with -smp 20 the first one worked, the second one
> > failed.
> [...]
> > =========== Working ===========
> >
> > SeaBIOS (version rel-1.8.0-0-g4c59f5d-20150219_092859-nilsson.home.kraxel.org)
> [...]
> > Found 20 cpu(s) max supported 20 cpu(s)
> [...]
> > =========== Broken ===========
> >
> > SeaBIOS (version rel-1.8.0-0-g4c59f5d-20150219_092859-nilsson.home.kraxel.org)
> [...]
> > Found 1 cpu(s) max supported 20 cpu(s)
>
> So, I couldn't get this to fail on my older AMD machine at all with
> the default SeaBIOS code. But, when I change the code with the patch
> below, it failed right away.
>
> KVM internal error. Suberror: 1
> emulation failure
> EAX=00000000 EBX=00000000 ECX=00000000 EDX=000fd2b8
> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
> EIP=000fd2c1 EFL=00000007 [-----PC] CPL=0 II=0 A20=1 SMM=0 HLT=0
> ES =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
> SS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> DS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> FS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> GS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
> TR =0000 00000000 0000ffff 00008300 DPL=0 TSS16-busy
> GDT= 000f6a50 00000037
> IDT= 000f6a8e 00000000
> CR0=60000011 CR2=00000000 CR3=00000000 CR4=00000000
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000000
> Code=66 ba b8 d2 0f 00 e9 a2 fe f3 90 f0 0f ba 2d 04 ff fb 3f 00 <72> f3 8b 25 00 ff fb 3f e8 d2 65 ff ff c7 05 04 ff fb 3f 00 00 00 00 f4 eb fd fa fc 66 b8
>
> And the failed debug output looks like:
>
> SeaBIOS (version rel-1.8.0-7-gd23eba6-dirty-20150311_121819-morn.localdomain)
> [...]
> cmos_smp_count0=20
> [...]
> cmos_smp_count=1
> cmos_smp_count2=1/20
> Found 1 cpu(s) max supported 20 cpu(s)
>
> I'm going to check the assembly for a compiler error, but is it
> possible QEMU is returning incorrect data in cmos index 0x5f?
>
> David, any chance you can recompile seabios and double check your
> output?
Done;
=========== Working ===========
SeaBIOS (version rel-1.8.0-0-g4c59f5d-dirty-20150311_164408-dgilbert-t530)
No Xen hypervisor found.
Running on QEMU (i440fx)
Running on KVM
RamSize: 0x40000000 [cmos]
Relocating init from 0x000df4e0 to 0x3ffaf570 (size 68048)
Found QEMU fw_cfg
RamBlock: addr 0x0000000000000000 len 0x0000000040000000 [e820]
Moving pm_base to 0x600
CPU Mhz=2116
cmos_smp_count0=20
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 6 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c04f
PCI: 32: 0000000080000000 - 00000000fec00000
PCI: map device bdf=00:03.0 bar 1, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:01.1 bar 4, addr 0000c040, size 00000010 [io]
PCI: map device bdf=00:03.0 bar 6, addr feb80000, size 00040000 [mem]
PCI: map device bdf=00:03.0 bar 0, addr febc0000, size 00020000 [mem]
PCI: map device bdf=00:02.0 bar 6, addr febe0000, size 00010000 [mem]
PCI: map device bdf=00:02.0 bar 2, addr febf0000, size 00001000 [mem]
PCI: map device bdf=00:02.0 bar 0, addr fd000000, size 01000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0x608
PCI: init bdf=00:02.0 id=1234:1111
PCI: init bdf=00:03.0 id=8086:100e
PCI: Using 00:02.0 for primary VGA
cmos_smp_count=20
handle_smp: apic_id=8
handle_smp: apic_id=4
handle_smp: apic_id=2
handle_smp: apic_id=15
handle_smp: apic_id=16
handle_smp: apic_id=10
handle_smp: apic_id=3
handle_smp: apic_id=12
handle_smp: apic_id=19
handle_smp: apic_id=14
handle_smp: apic_id=11
handle_smp: apic_id=7
handle_smp: apic_id=17
handle_smp: apic_id=6
handle_smp: apic_id=18
handle_smp: apic_id=13
handle_smp: apic_id=9
handle_smp: apic_id=5
handle_smp: apic_id=1
cmos_smp_count2=20/20
Found 20 cpu(s) max supported 20 cpu(s)
Copying PIR from 0x3ffbfc9c to 0x000f6720
Copying MPTABLE from 0x00006de8/3ffa64a0 to 0x000f64c0
Copying SMBIOS entry point from 0x00006de8 to 0x000f64a0
Scan for VGA option rom
Running option rom at c000:0003
Start SeaVGABIOS (version rel-1.8.0-0-g4c59f5d-20150219_092912-nilsson.home.kraxel.org)
enter vga_post:
a=00000010 b=0000ffff c=00000000 d=0000ffff ds=0000 es=f000 ss=0000
si=00000000 di=00006af0 bp=00000000 sp=00006d66 cs=f000 ip=d244 f=0000
VBE DISPI: bdf 00:02.0, bar 0
VBE DISPI: lfb_addr=fd000000, size 16 MB
Attempting to allocate VGA stack via pmm call to f000:d2f6
pmm call arg1=0
VGA stack allocated at ef430
Running option rom at c980:0003
Turning on vga text mode console
set VGA mode 3
SeaBIOS (version rel-1.8.0-0-g4c59f5d-dirty-20150311_164408-dgilbert-t530)
All threads complete.
Found 1 lpt ports
Found 1 serial ports
Searching bootorder for: /pci@i0cf8/isa@1/fdc@03f0/floppy@0
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
DVD/CD [ata1-0: QEMU DVD-ROM ATAPI-4 DVD/CD]
Searching bootorder for: /pci@i0cf8/*@1,1/drive@1/disk@0
PS2 keyboard initialized
All threads complete.
Scan for option roms
Running option rom at ca80:0003
pmm call arg1=1
pmm call arg1=0
pmm call arg1=1
pmm call arg1=0
Searching bootorder for: /pci@i0cf8/*@3
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
Running option rom at cb80:0003
Space available for UMB: ce000-ee800, f5fa0-f63c0
Returned 126976 bytes of ZoneHigh
e820 map has 7 items:
0: 0000000000000000 - 000000000009f800 = 1 RAM
1: 000000000009f800 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000003ffdf000 = 1 RAM
4: 000000003ffdf000 - 0000000040000000 = 2 RESERVED
5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
enter handle_19:
NULL
Booting from Hard Disk...
Boot failed: could not read the boot disk
enter handle_18:
NULL
Booting from Floppy...
floppy error: 40 00 00 00 00 01 02
Boot failed: could not read the boot disk
enter handle_18:
NULL
Booting from DVD/CD...
Device reports MEDIUM NOT PRESENT
scsi_is_ready returned -1
Boot failed: Could not read from CDROM (code 0003)
enter handle_18:
NULL
Booting from ROM...
Booting from ca80:0361
=========== Broken ===========
SeaBIOS (version rel-1.8.0-0-g4c59f5d-dirty-20150311_164408-dgilbert-t530)
No Xen hypervisor found.
Running on QEMU (i440fx)
Running on KVM
RamSize: 0x40000000 [cmos]
Relocating init from 0x000df4e0 to 0x3ffaf570 (size 68048)
Found QEMU fw_cfg
RamBlock: addr 0x0000000000000000 len 0x0000000040000000 [e820]
Moving pm_base to 0x600
CPU Mhz=2115
cmos_smp_count0=20
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 6 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c04f
PCI: 32: 0000000080000000 - 00000000fec00000
PCI: map device bdf=00:03.0 bar 1, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:01.1 bar 4, addr 0000c040, size 00000010 [io]
PCI: map device bdf=00:03.0 bar 6, addr feb80000, size 00040000 [mem]
PCI: map device bdf=00:03.0 bar 0, addr febc0000, size 00020000 [mem]
PCI: map device bdf=00:02.0 bar 6, addr febe0000, size 00010000 [mem]
PCI: map device bdf=00:02.0 bar 2, addr febf0000, size 00001000 [mem]
PCI: map device bdf=00:02.0 bar 0, addr fd000000, size 01000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0x608
PCI: init bdf=00:02.0 id=1234:1111
PCI: init bdf=00:03.0 id=8086:100e
PCI: Using 00:02.0 for primary VGA
cmos_smp_count=1
cmos_smp_count2=1/20
Found 1 cpu(s) max supported 20 cpu(s)
Copying PIR from 0x3ffbfc9c to 0x000f6720
Copying MPTABLE from 0x00006de8/3ffa64a0 to 0x000f64c0
Copying SMBIOS entry point from 0x00006de8 to 0x000f64a0
Scan for VGA option rom
Running option rom at c000:0003
Start SeaVGABIOS (version rel-1.8.0-0-g4c59f5d-20150219_092912-nilsson.home.kraxel.org)
enter vga_post:
a=00000010 b=0000ffff c=00000000 d=0000ffff ds=0000 es=f000 ss=0000
si=00000000 di=00006af0 bp=00000000 sp=00006d66 cs=f000 ip=d244 f=0000
VBE DISPI: bdf 00:02.0, bar 0
VBE DISPI: lfb_addr=fd000000, size 16 MB
Attempting to allocate VGA stack via pmm call to f000:d2f6
pmm call arg1=0
VGA stack allocated at ef430
Running option rom at c980:0003
Turning on vga text mode console
set VGA mode 3
SeaBIOS (version rel-1.8.0-0-g4c59f5d-dirty-20150311_164408-dgilbert-t530)
All threads complete.
Found 1 lpt ports
Found 1 serial ports
Searching bootorder for: /pci@i0cf8/isa@1/fdc@03f0/floppy@0
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
DVD/CD [ata1-0: QEMU DVD-ROM ATAPI-4 DVD/CD]
Searching bootorder for: /pci@i0cf8/*@1,1/drive@1/disk@0
PS2 keyboard initialized
All threads complete.
Scan for option roms
Running option rom at ca80:0003
pmm call arg1=1
pmm call arg1=0
pmm call arg1=1
pmm call arg1=0
Searching bootorder for: /pci@i0cf8/*@3
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
Running option rom at cb80:0003
Space available for UMB: ce000-ee800, f5fa0-f63c0
Returned 126976 bytes of ZoneHigh
e820 map has 7 items:
0: 0000000000000000 - 000000000009f800 = 1 RAM
1: 000000000009f800 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000003ffdf000 = 1 RAM
4: 000000003ffdf000 - 0000000040000000 = 2 RESERVED
5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
> -Kevin
>
>
> --- a/src/fw/smp.c
> +++ b/src/fw/smp.c
> @@ -128,6 +128,7 @@ smp_setup(void)
>
> // Wait for other CPUs to process the SIPI.
> u8 cmos_smp_count = rtc_read(CMOS_BIOS_SMP_COUNT) + 1;
> + dprintf(1, "cmos_smp_count=%d\n", cmos_smp_count);
> while (cmos_smp_count != CountCPUs)
> asm volatile(
> // Release lock and allow other processors to use the stack.
> @@ -140,6 +141,8 @@ smp_setup(void)
> : "+m" (SMPLock), "+m" (SMPStack)
> : : "cc", "memory");
> yield();
> + dprintf(1, "cmos_smp_count2=%d/%d\n", cmos_smp_count
> + , rtc_read(CMOS_BIOS_SMP_COUNT) + 1);
>
> // Restore memory.
> *(u64*)BUILD_AP_BOOT_ADDR = old;
> diff --git a/src/post.c b/src/post.c
> index 9ea5620..dc11c72 100644
> --- a/src/post.c
> +++ b/src/post.c
> @@ -170,6 +170,7 @@ platform_hardware_setup(void)
> clock_setup();
>
> // Platform specific setup
> + dprintf(1, "cmos_smp_count0=%d\n", rtc_read(CMOS_BIOS_SMP_COUNT) + 1);
> qemu_platform_setup();
> coreboot_platform_setup();
> }
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
next prev parent reply other threads:[~2015-03-11 16:52 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-03-05 22:14 [Qemu-devel] E5-2620v2 - emulation stop error Andrey Korolyov
2015-03-05 23:44 ` Andrey Korolyov
2015-03-06 16:57 ` Bandan Das
2015-03-07 0:00 ` Andrey Korolyov
2015-03-10 14:24 ` Andrey Korolyov
2015-03-10 16:57 ` Dr. David Alan Gilbert
2015-03-10 18:08 ` Andrey Korolyov
2015-03-10 18:16 ` Dr. David Alan Gilbert
2015-03-10 18:21 ` Andrey Korolyov
2015-03-10 19:30 ` Paolo Bonzini
2015-03-10 18:10 ` Paolo Bonzini
2015-03-10 18:21 ` Bandan Das
2015-03-10 19:25 ` Paolo Bonzini
2015-03-10 19:37 ` Dr. David Alan Gilbert
2015-03-10 20:29 ` Dr. David Alan Gilbert
2015-03-11 2:38 ` Bandan Das
2015-03-11 13:45 ` Dr. David Alan Gilbert
2015-03-11 15:42 ` Kevin O'Connor
2015-03-11 15:53 ` Dr. David Alan Gilbert
2015-03-11 16:37 ` Kevin O'Connor
2015-03-11 16:52 ` Dr. David Alan Gilbert [this message]
2015-03-11 17:37 ` Kevin O'Connor
2015-03-11 17:41 ` Paolo Bonzini
2015-03-11 17:59 ` Dr. David Alan Gilbert
2015-03-11 18:24 ` Bandan Das
2015-03-11 18:40 ` Kevin O'Connor
2015-03-11 18:45 ` Kevin O'Connor
2015-03-11 19:19 ` Kevin O'Connor
2015-03-11 19:33 ` Dr. David Alan Gilbert
2015-03-11 19:47 ` Bandan Das
2015-03-11 19:47 ` Andrey Korolyov
2015-03-11 19:59 ` Dr. David Alan Gilbert
2015-03-11 20:09 ` Andrey Korolyov
2015-03-12 9:59 ` Dr. David Alan Gilbert
2015-03-12 10:47 ` Andrey Korolyov
2015-03-16 19:17 ` Andrey Korolyov
2015-03-16 19:26 ` Dr. David Alan Gilbert
2015-03-25 20:43 ` Andrey Korolyov
2015-03-25 20:46 ` Andrey Korolyov
2015-03-25 20:54 ` Kevin O'Connor
2015-03-25 22:31 ` Andrey Korolyov
2015-03-25 23:02 ` Kevin O'Connor
2015-03-25 23:35 ` Andrey Korolyov
2015-03-26 0:05 ` Kevin O'Connor
2015-03-26 15:58 ` Radim Krčmář
2015-03-26 16:36 ` Kevin O'Connor
2015-03-26 16:48 ` Andrey Korolyov
2015-03-26 17:06 ` Kevin O'Connor
2015-03-26 17:08 ` Andrey Korolyov
2015-03-26 17:18 ` Kevin O'Connor
2015-03-26 17:33 ` Andrey Korolyov
2015-03-26 17:40 ` Radim Krčmář
2015-03-26 18:24 ` Andrey Korolyov
2015-03-26 20:40 ` Radim Krčmář
2015-03-26 21:03 ` Bandan Das
2015-03-27 10:16 ` Andrey Korolyov
2015-03-30 18:56 ` Radim Krčmář
2015-03-30 19:32 ` Andrey Korolyov
2015-03-31 13:45 ` Radim Krčmář
2015-03-31 14:56 ` Andrey Korolyov
2015-03-31 16:45 ` Radim Krčmář
2015-03-31 17:40 ` Andrey Korolyov
2015-03-31 18:01 ` Bandan Das
2015-03-31 18:04 ` Bandan Das
2015-03-31 18:23 ` Andrey Korolyov
2015-04-01 11:49 ` Radim Krčmář
2015-04-01 12:05 ` Paolo Bonzini
2015-04-01 12:26 ` Andrey Korolyov
2015-04-01 13:19 ` Paolo Bonzini
2015-04-01 15:37 ` Andrey Korolyov
2015-04-01 16:29 ` Andrey Korolyov
2015-04-01 22:58 ` Andrey Korolyov
2015-04-05 14:12 ` Andrey Korolyov
2015-03-27 11:54 ` Andrey Korolyov
2015-03-30 19:28 ` Radim Krčmář
2015-03-26 17:35 ` Radim Krčmář
2015-03-26 17:34 ` Radim Krčmář
2015-03-26 2:47 ` Bandan Das
2015-03-26 9:18 ` Andrey Korolyov
2015-03-26 15:05 ` Andrey Korolyov
2015-03-11 17:09 ` Bandan Das
2015-03-11 17:32 ` Kevin O'Connor
2015-03-11 18:01 ` Bandan Das
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150311165203.GL2334@work-vm \
--to=dgilbert@redhat.com \
--cc=andrey@xdel.ru \
--cc=bsd@redhat.com \
--cc=kevin@koconnor.net \
--cc=kraxel@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).