[Qemu-devel] pxe boot problems

[Qemu-devel] pxe boot problems

[Dietmar Maurer]

[-- Attachment #1: Type: text/plain, Size: 1372 bytes --]

pxe boot does not work with qemu 1.7 (also tested with latest code from master).

# kvm -m 1024 -net nic -net tap

simply hangs at:

iPXE (PCI 00:03.0) starting execution...

and I get the following output:

# kvm -m 1024 -net nic -net tap
KVM: unknown exit, hardware reason 80000021
EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa
ESI=c00e006a EDI=00098bf0 EBP=00000000 ESP=00007baa
EIP=00000215 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
CS =9c7c 0009c7e0 0000ffff 00009b00 DPL=0 CS16 [-RA]
SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA]
DS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
FS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
GS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 feffd000 00002088 00008b00 DPL=0 TSS32-busy
GDT=     0009cf40 00000037
IDT=     00000000 0000ffff
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=66 0f 01 16 10 00 66 0f 01 1e 48 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 00 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 4e 00 2e a1 ac 06 8e d8 8e c0 8e e0

this is using RHEL6 kernel 358.23.2.el6

any ideas?



[-- Attachment #2: Type: text/html, Size: 5235 bytes --]

Re: [Qemu-devel] pxe boot problems

[Dietmar Maurer]

A bisect points to the following patch:

# git bisect bad
c45e5b5b30ac1f5505725a7b36e68cedfce4f01f is the first bad commit
commit c45e5b5b30ac1f5505725a7b36e68cedfce4f01f
Author: Gerd Hoffmann <kraxel@redhat.com>
Date:   Tue Feb 26 17:46:11 2013 +0100

    Switch to efi-enabled nic roms by default
    
    All PCI nics are switched to EFI-enabled roms by default.  They are
    composed from three images (legacy, efi ia32 & efi x86), so classic
    pxe booting will continue to work.
    
    Exception: eepro100 is not switched, it uses a single rom for all
    emulated eepro100 variants, then goes patch the rom header on the
    fly with the correct PCI IDs.  I doubt that will work as-is with
    the efi roms.
    
    Keep old roms for 1.4+older machine types via compat properties,
    needed because the efi-enabled roms are larger so the pci rom bar
    size would change.
    
    Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>



> pxe boot does not work with qemu 1.7 (also tested with latest code from
> master).
> 
> # kvm -m 1024 -net nic -net tap
> 
> simply hangs at:
> 
> iPXE (PCI 00:03.0) starting execution.
> 
> and I get the following output:
> 
> # kvm -m 1024 -net nic -net tap
> KVM: unknown exit, hardware reason 80000021
> EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa
> ESI=c00e006a EDI=00098bf0 EBP=00000000 ESP=00007baa
> EIP=00000215 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
> ES =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
> CS =9c7c 0009c7e0 0000ffff 00009b00 DPL=0 CS16 [-RA]
> SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA]
> DS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
> FS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
> GS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
> TR =0000 feffd000 00002088 00008b00 DPL=0 TSS32-busy
> GDT=     0009cf40 00000037
> IDT=     00000000 0000ffff
> CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> DR3=0000000000000000
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000000
> Code=66 0f 01 16 10 00 66 0f 01 1e 48 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 00
> 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 4e 00 2e a1 ac 06 8e d8 8e c0 8e e0
> 
> this is using RHEL6 kernel 358.23.2.el6
> 
> any ideas?
> 

Re: [Qemu-devel] pxe boot problems

[Laszlo Ersek]

On 01/29/14 12:49, Dietmar Maurer wrote:
> A bisect points to the following patch:
> 
> # git bisect bad
> c45e5b5b30ac1f5505725a7b36e68cedfce4f01f is the first bad commit
> commit c45e5b5b30ac1f5505725a7b36e68cedfce4f01f
> Author: Gerd Hoffmann <kraxel@redhat.com>
> Date:   Tue Feb 26 17:46:11 2013 +0100
> 
>     Switch to efi-enabled nic roms by default
>     
>     All PCI nics are switched to EFI-enabled roms by default.  They are
>     composed from three images (legacy, efi ia32 & efi x86), so classic
>     pxe booting will continue to work.
>     
>     Exception: eepro100 is not switched, it uses a single rom for all
>     emulated eepro100 variants, then goes patch the rom header on the
>     fly with the correct PCI IDs.  I doubt that will work as-is with
>     the efi roms.
>     
>     Keep old roms for 1.4+older machine types via compat properties,
>     needed because the efi-enabled roms are larger so the pci rom bar
>     size would change.
>     
>     Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> 
> 
> 
>> pxe boot does not work with qemu 1.7 (also tested with latest code from
>> master).
>>
>> # kvm -m 1024 -net nic -net tap
>>
>> simply hangs at:
>>
>> iPXE (PCI 00:03.0) starting execution.
>>
>> and I get the following output:
>>
>> # kvm -m 1024 -net nic -net tap
>> KVM: unknown exit, hardware reason 80000021
>> EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa
>> ESI=c00e006a EDI=00098bf0 EBP=00000000 ESP=00007baa
>> EIP=00000215 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
>> ES =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>> CS =9c7c 0009c7e0 0000ffff 00009b00 DPL=0 CS16 [-RA]
>> SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA]
>> DS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>> FS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>> GS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
>> TR =0000 feffd000 00002088 00008b00 DPL=0 TSS32-busy
>> GDT=     0009cf40 00000037
>> IDT=     00000000 0000ffff
>> CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
>> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
>> DR3=0000000000000000
>> DR6=00000000ffff0ff0 DR7=0000000000000400
>> EFER=0000000000000000
>> Code=66 0f 01 16 10 00 66 0f 01 1e 48 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 00
>> 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 4e 00 2e a1 ac 06 8e d8 8e c0 8e e0
>>
>> this is using RHEL6 kernel 358.23.2.el6
>>
>> any ideas?

Does it work with TCG? If so, what host CPU do you have?

Also, can you try with a NIC model different from the default e1000?

Thanks
Laszlo

Re: [Qemu-devel] pxe boot problems

[Laszlo Ersek]

On 01/29/14 14:01, Laszlo Ersek wrote:
> On 01/29/14 12:49, Dietmar Maurer wrote:

>> A bisect points to the following patch:
>>
>> # git bisect bad
>> c45e5b5b30ac1f5505725a7b36e68cedfce4f01f is the first bad commit
>> commit c45e5b5b30ac1f5505725a7b36e68cedfce4f01f
>> Author: Gerd Hoffmann <kraxel@redhat.com>
>> Date:   Tue Feb 26 17:46:11 2013 +0100
>>
>>     Switch to efi-enabled nic roms by default
>>
>>     All PCI nics are switched to EFI-enabled roms by default.  They are
>>     composed from three images (legacy, efi ia32 & efi x86), so classic
>>     pxe booting will continue to work.
>>
>>     Exception: eepro100 is not switched, it uses a single rom for all
>>     emulated eepro100 variants, then goes patch the rom header on the
>>     fly with the correct PCI IDs.  I doubt that will work as-is with
>>     the efi roms.
>>
>>     Keep old roms for 1.4+older machine types via compat properties,
>>     needed because the efi-enabled roms are larger so the pci rom bar
>>     size would change.
>>
>>     Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>>
>>
>>
>>> pxe boot does not work with qemu 1.7 (also tested with latest code
>>> from master).
>>>
>>> # kvm -m 1024 -net nic -net tap
>>>
>>> simply hangs at:
>>>
>>> iPXE (PCI 00:03.0) starting execution.
>>>
>>> and I get the following output:
>>>
>>> # kvm -m 1024 -net nic -net tap
>>> KVM: unknown exit, hardware reason 80000021
>>> EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa
>>> ESI=c00e006a EDI=00098bf0 EBP=00000000 ESP=00007baa
>>> EIP=00000215 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
>>> ES =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> CS =9c7c 0009c7e0 0000ffff 00009b00 DPL=0 CS16 [-RA]
>>> SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA]
>>> DS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> FS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> GS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
>>> TR =0000 feffd000 00002088 00008b00 DPL=0 TSS32-busy
>>> GDT=     0009cf40 00000037
>>> IDT=     00000000 0000ffff
>>> CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
>>> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
>>> DR3=0000000000000000
>>> DR6=00000000ffff0ff0 DR7=0000000000000400
>>> EFER=0000000000000000
>>> Code=66 0f 01 16 10 00 66 0f 01 1e 48 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 00
>>> 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 4e 00 2e a1 ac 06 8e d8 8e c0 8e e0
>>>
>>> this is using RHEL6 kernel 358.23.2.el6
>>>
>>> any ideas?
>
> Does it work with TCG? If so, what host CPU do you have?
>
> Also, can you try with a NIC model different from the default e1000?

I can reproduce this.

- Qemu: 0169c511554cb0014a00290b0d3d26c31a49818f
- Host kernel: 2.6.32-431.3.1.el6.x86_64 (RHEL-6.5)
- Host CPU:
  - Intel(R) Xeon(R) CPU W3550
  - /sys/module/kvm_intel/parameters/unrestricted_guest == N

TCG works fine.

With KVM enabled, I tested all NIC models listed by
"-net nic,model=help":
- fails: ne2k_pci rtl8139 e1000 pcnet virtio
- works: i82551 i82557b i82559er

I rebuilt ipxe (albeit a different version from what Gerd checked in,
probably...) and tried to look for the offending insn stream in the
disassembly:

$ file bin/8086100e.mrom.tmp

  bin/8086100e.mrom.tmp: ELF 32-bit LSB executable, Intel 80386, version
  1 (SYSV), statically linked, not stripped

$ objdump -S bin/8086100e.mrom.tmp

>  3c8:   66 0f 01 1e             lidtw  (%esi)
>  * Note that the broadcast address is also a multicast address.
>  */
> static inline int is_multicast_ether_addr ( const void *addr ) {
>         const uint8_t *addr_bytes = addr;
>
>         return ( addr_bytes[0] & 0x01 );
>  3cc:   48                      dec    %eax
>  3cd:   00 0f                   add    %cl,(%edi)
>  *
>  * Check that the Ethernet address (MAC) is not 00:00:00:00:00:00, is
>  * not a multicast address, and is not ff:ff:ff:ff:ff:ff.
>  */
> static inline int is_valid_ether_addr ( const void *addr ) {
>         return ( ( ! is_multicast_ether_addr ( addr ) ) &&
>  3cf:   20 c0                   and    %al,%al
>  3d1:   0c 01                   or     $0x1,%al
>                 memcpy ( hw_addr, mac.raw, ETH_ALEN );
>                 return 0;
>         }
>
>         DBGC ( intel, "INTEL %p has no MAC address to use\n", intel );
>         return -ENOENT;
>  3d3:   0f 22 c0                mov    %eax,%cr0
>  3d6:   66 ea a4 00 00 00       ljmpw  $0x0,$0xa4

Here: ^^^^

>         return 0;
>
>         unregister_netdev ( netdev );
>  err_register_netdev:
>  err_fetch_mac:
>         intel_reset ( intel );
>  3dc:   08 00                   or     %al,(%eax)
>  3de:   0f 20 c0                mov    %cr0,%eax
>  *
>  * Drivers should call this method immediately before the final call
>  * to netdev_put().
>  */
> static inline void netdev_nullify ( struct net_device *netdev ) {
>         netdev->op = &null_netdev_operations;
>  3e1:   24 fe                   and    $0xfe,%al
>  3e3:   0f 22 c0                mov    %eax,%cr0
>  *
>  * @v netdev            Network device
>  */
> static inline __attribute__ (( always_inline )) void
> netdev_put ( struct net_device *netdev ) {
>         ref_put ( &netdev->refcnt );
>  3e6:   ff 2e                   ljmp   *(%esi)
>  3e8:   4e                      dec    %esi

KVM chokes on the LJMPW instruction. (It needs to emulate it on this
host CPU, but the emulation code fails to decode the instruction.)

The code seems to belong to intel_probe() in "src/drivers/net/intel.c":

intel_probe()
  intel_fetch_mac()
    is_valid_ether_addr()
      is_multicast_ether_addr()

I *guess* upstream Linux commit

    commit 414e6277fd148f6470261cef50a7fed0d88a2825
    Author: Gleb Natapov <gleb@redhat.com>
    Date:   Wed Apr 28 19:15:26 2010 +0300

        KVM: x86 emulator: handle "far address" source operand

        ljmp/lcall instruction operand contains address and segment.
        It can be 10 bytes long. Currently we decode it as two different
        operands. Fix it by introducing new kind of operand that can hold
        entire far address.

        Signed-off-by: Gleb Natapov <gleb@redhat.com>
        Signed-off-by: Avi Kivity <avi@redhat.com>

which had been first released in v2.6.36, should be ported to the RHEL-6
kernel.

... The pxe-e1000.rom binary seems to work. However, the last update to
*that* file is much older than the one to the efi-1000.rom binary.
Compare:

    commit 0099cd43ecf07710a608db5ca0945758514a14c2
    Author: Gerd Hoffmann <kraxel@redhat.com>
    Date:   Mon Mar 25 09:13:18 2013 +0100

        ipxe: update binaries

        Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

     pc-bios/efi-e1000.rom    | Bin 174080 -> 173568 bytes
     pc-bios/efi-eepro100.rom | Bin 175104 -> 174592 bytes
     pc-bios/efi-ne2k_pci.rom | Bin 173568 -> 173056 bytes
     pc-bios/efi-pcnet.rom    | Bin 173568 -> 173056 bytes
     pc-bios/efi-rtl8139.rom  | Bin 177152 -> 176640 bytes
     pc-bios/efi-virtio.rom   | Bin 171008 -> 171008 bytes
     6 files changed, 0 insertions(+), 0 deletions(-)

vs.

    commit 36d8d02dc8c45780cae74e2ba7a6135b95c16f81
    Author: Alex Williamson <alex.williamson@redhat.com>
    Date:   Mon Apr 18 11:46:41 2011 -0600

        PXE: Refresh all PXE ROMs from the ipxe submodule

        Add script to make this easy to repeat later.

        Signed-off-by: Alex Williamson <alex.williamson@redhat.com>

     pc-bios/README              |  17 ++++----
     pc-bios/pxe-e1000.rom       | Bin 72192 -> 67072 bytes
     pc-bios/pxe-eepro100.rom    | Bin 56832 -> 61440 bytes
     pc-bios/pxe-ne2k_pci.rom    | Bin 56320 -> 61440 bytes
     pc-bios/pxe-pcnet.rom       | Bin 56832 -> 61440 bytes
     pc-bios/pxe-rtl8139.rom     | Bin 56320 -> 61440 bytes
     pc-bios/pxe-virtio.rom      | Bin 56320 -> 60416 bytes
     scripts/refresh-pxe-roms.sh |  99 ++++++++++++++++++++++++++++++++++++++++++++
     8 files changed, 107 insertions(+), 9 deletions(-)

SeaBIOS has seen such problems before, I believe.

Laszlo

Re: [Qemu-devel] pxe boot problems

[Paolo Bonzini]

Il 29/01/2014 18:34, Laszlo Ersek ha scritto:
> On 01/29/14 14:01, Laszlo Ersek wrote:
>> On 01/29/14 12:49, Dietmar Maurer wrote:
>
>>> A bisect points to the following patch:
>>>
>>> # git bisect bad
>>> c45e5b5b30ac1f5505725a7b36e68cedfce4f01f is the first bad commit
>>> commit c45e5b5b30ac1f5505725a7b36e68cedfce4f01f
>>> Author: Gerd Hoffmann <kraxel@redhat.com>
>>> Date:   Tue Feb 26 17:46:11 2013 +0100
>>>
>>>     Switch to efi-enabled nic roms by default
>>>
>>>     All PCI nics are switched to EFI-enabled roms by default.  They are
>>>     composed from three images (legacy, efi ia32 & efi x86), so classic
>>>     pxe booting will continue to work.
>>>
>>>     Exception: eepro100 is not switched, it uses a single rom for all
>>>     emulated eepro100 variants, then goes patch the rom header on the
>>>     fly with the correct PCI IDs.  I doubt that will work as-is with
>>>     the efi roms.
>>>
>>>     Keep old roms for 1.4+older machine types via compat properties,
>>>     needed because the efi-enabled roms are larger so the pci rom bar
>>>     size would change.
>>>
>>>     Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>>>
>>>
>>>
>>>> pxe boot does not work with qemu 1.7 (also tested with latest code
>>>> from master).
>>>>
>>>> # kvm -m 1024 -net nic -net tap
>>>>
>>>> simply hangs at:
>>>>
>>>> iPXE (PCI 00:03.0) starting execution.
>>>>
>>>> and I get the following output:
>>>>
>>>> # kvm -m 1024 -net nic -net tap
>>>> KVM: unknown exit, hardware reason 80000021
>>>> EAX=00000011 EBX=00000000 ECX=00000030 EDX=00007baa
>>>> ESI=c00e006a EDI=00098bf0 EBP=00000000 ESP=00007baa
>>>> EIP=00000215 EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
>>>> ES =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>>> CS =9c7c 0009c7e0 0000ffff 00009b00 DPL=0 CS16 [-RA]
>>>> SS =0000 00000000 0000ffff 00009300 DPL=0 DS16 [-WA]
>>>> DS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>>> FS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>>> GS =0030 0009cf30 ffffffff 0000f300 DPL=3 DS16 [-WA]
>>>> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
>>>> TR =0000 feffd000 00002088 00008b00 DPL=0 TSS32-busy
>>>> GDT=     0009cf40 00000037
>>>> IDT=     00000000 0000ffff
>>>> CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
>>>> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
>>>> DR3=0000000000000000
>>>> DR6=00000000ffff0ff0 DR7=0000000000000400
>>>> EFER=0000000000000000
>>>> Code=66 0f 01 16 10 00 66 0f 01 1e 48 00 0f 20 c0 0c 01 0f 22 c0 <66> ea a4 00
>>>> 00 00 08 00 0f 20 c0 24 fe 0f 22 c0 ff 2e 4e 00 2e a1 ac 06 8e d8 8e c0 8e e0
>>>>
>>>> this is using RHEL6 kernel 358.23.2.el6
>>>>
>>>> any ideas?
>>
>> Does it work with TCG? If so, what host CPU do you have?
>>
>> Also, can you try with a NIC model different from the default e1000?
>
> I can reproduce this.
>
> - Qemu: 0169c511554cb0014a00290b0d3d26c31a49818f
> - Host kernel: 2.6.32-431.3.1.el6.x86_64 (RHEL-6.5)
> - Host CPU:
>   - Intel(R) Xeon(R) CPU W3550
>   - /sys/module/kvm_intel/parameters/unrestricted_guest == N
>
> TCG works fine.
>
> With KVM enabled, I tested all NIC models listed by
> "-net nic,model=help":
> - fails: ne2k_pci rtl8139 e1000 pcnet virtio
> - works: i82551 i82557b i82559er
>
> I rebuilt ipxe (albeit a different version from what Gerd checked in,
> probably...) and tried to look for the offending insn stream in the
> disassembly:
>
> $ file bin/8086100e.mrom.tmp
>
>   bin/8086100e.mrom.tmp: ELF 32-bit LSB executable, Intel 80386, version
>   1 (SYSV), statically linked, not stripped
>
> $ objdump -S bin/8086100e.mrom.tmp
>
>>  3c8:   66 0f 01 1e             lidtw  (%esi)
>>  * Note that the broadcast address is also a multicast address.
>>  */
>> static inline int is_multicast_ether_addr ( const void *addr ) {
>>         const uint8_t *addr_bytes = addr;
>>
>>         return ( addr_bytes[0] & 0x01 );
>>  3cc:   48                      dec    %eax
>>  3cd:   00 0f                   add    %cl,(%edi)
>>  *
>>  * Check that the Ethernet address (MAC) is not 00:00:00:00:00:00, is
>>  * not a multicast address, and is not ff:ff:ff:ff:ff:ff.
>>  */
>> static inline int is_valid_ether_addr ( const void *addr ) {
>>         return ( ( ! is_multicast_ether_addr ( addr ) ) &&
>>  3cf:   20 c0                   and    %al,%al
>>  3d1:   0c 01                   or     $0x1,%al
>>                 memcpy ( hw_addr, mac.raw, ETH_ALEN );
>>                 return 0;
>>         }
>>
>>         DBGC ( intel, "INTEL %p has no MAC address to use\n", intel );
>>         return -ENOENT;
>>  3d3:   0f 22 c0                mov    %eax,%cr0
>>  3d6:   66 ea a4 00 00 00       ljmpw  $0x0,$0xa4
>
> Here: ^^^^
>
>>         return 0;
>>
>>         unregister_netdev ( netdev );
>>  err_register_netdev:
>>  err_fetch_mac:
>>         intel_reset ( intel );
>>  3dc:   08 00                   or     %al,(%eax)
>>  3de:   0f 20 c0                mov    %cr0,%eax
>>  *
>>  * Drivers should call this method immediately before the final call
>>  * to netdev_put().
>>  */
>> static inline void netdev_nullify ( struct net_device *netdev ) {
>>         netdev->op = &null_netdev_operations;
>>  3e1:   24 fe                   and    $0xfe,%al
>>  3e3:   0f 22 c0                mov    %eax,%cr0
>>  *
>>  * @v netdev            Network device
>>  */
>> static inline __attribute__ (( always_inline )) void
>> netdev_put ( struct net_device *netdev ) {
>>         ref_put ( &netdev->refcnt );
>>  3e6:   ff 2e                   ljmp   *(%esi)
>>  3e8:   4e                      dec    %esi
>
> KVM chokes on the LJMPW instruction. (It needs to emulate it on this
> host CPU, but the emulation code fails to decode the instruction.)
>
> I *guess* upstream Linux commit
>
>     commit 414e6277fd148f6470261cef50a7fed0d88a2825
>     Author: Gleb Natapov <gleb@redhat.com>
>     Date:   Wed Apr 28 19:15:26 2010 +0300
>
>         KVM: x86 emulator: handle "far address" source operand
>
>         ljmp/lcall instruction operand contains address and segment.
>         It can be 10 bytes long. Currently we decode it as two different
>         operands. Fix it by introducing new kind of operand that can hold
>         entire far address.
>
>         Signed-off-by: Gleb Natapov <gleb@redhat.com>
>         Signed-off-by: Avi Kivity <avi@redhat.com>
>
> which had been first released in v2.6.36, should be ported to the RHEL-6
> kernel.

That's a candidate, but the commit does not say _what_ is being fixed 
exactly and the RHEL6 kernel does have code to decode 0xea.

Paolo

Re: [Qemu-devel] pxe boot problems

[Laszlo Ersek]

On 01/29/14 18:47, Paolo Bonzini wrote:
> Il 29/01/2014 18:34, Laszlo Ersek ha scritto:

>> KVM chokes on the LJMPW instruction. (It needs to emulate it on this
>> host CPU, but the emulation code fails to decode the instruction.)
>>
>> I *guess* upstream Linux commit
>>
>>     commit 414e6277fd148f6470261cef50a7fed0d88a2825
>>     Author: Gleb Natapov <gleb@redhat.com>
>>     Date:   Wed Apr 28 19:15:26 2010 +0300
>>
>>         KVM: x86 emulator: handle "far address" source operand
>>
>>         ljmp/lcall instruction operand contains address and segment.
>>         It can be 10 bytes long. Currently we decode it as two different
>>         operands. Fix it by introducing new kind of operand that can hold
>>         entire far address.
>>
>>         Signed-off-by: Gleb Natapov <gleb@redhat.com>
>>         Signed-off-by: Avi Kivity <avi@redhat.com>
>>
>> which had been first released in v2.6.36, should be ported to the RHEL-6
>> kernel.
> 
> That's a candidate, but the commit does not say _what_ is being fixed
> exactly and the RHEL6 kernel does have code to decode 0xea.

Yes, but as far I can see, the commit (which is not in RHEL-6) changes
*how* the operand of ljmp is decoded.

>From "opcode_table" in RHEL-6's "arch/x86/kvm/emulate.c":


        /* 0xE8 - 0xEF */
        SrcImm | Stack, SrcImm | ImplicitOps,
        SrcImmU | Src2Imm16 | No64, SrcImmByte | ImplicitOps,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^

and the patch changes that to

        SrcImmFAddr | No64

and adds new logic to fetch this source operand type.

... Which then seems to have an effect on what goes into
load_segment_descriptor() as segment selector, in the emulation of 0xea.

Of course I'm insufficiently equipped to debate this with you in earnest
:), but it seemed relevant to me.

Thanks,
Laszlo

Re: [Qemu-devel] pxe boot problems

[Paolo Bonzini]

Il 29/01/2014 19:09, Laszlo Ersek ha scritto:
> Yes, but as far I can see, the commit (which is not in RHEL-6) changes
> *how* the operand of ljmp is decoded.
>
> From "opcode_table" in RHEL-6's "arch/x86/kvm/emulate.c":
>
>
>         /* 0xE8 - 0xEF */
>         SrcImm | Stack, SrcImm | ImplicitOps,
>         SrcImmU | Src2Imm16 | No64, SrcImmByte | ImplicitOps,
>         ^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> and the patch changes that to
>
>         SrcImmFAddr | No64
>
> and adds new logic to fetch this source operand type.
>
> ... Which then seems to have an effect on what goes into
> load_segment_descriptor() as segment selector, in the emulation of 0xea.
>
> Of course I'm insufficiently equipped to debate this with you in earnest
> :), but it seemed relevant to me.

Yeah, it seems relevant to me too.

But before it was decoding two immediates, one after another, the first 
c->op_bytes long in c->src, and the second 2 bytes long in c->src2.  Now 
it's doing the same, but putting all c->op_bytes+2 bytes in c->src...

Though I guess the backport should be relatively easy if you want to try.

Paolo

Re: [Qemu-devel] pxe boot problems

[Laszlo Ersek]

On 01/29/14 19:13, Paolo Bonzini wrote:
> Il 29/01/2014 19:09, Laszlo Ersek ha scritto:
>> Yes, but as far I can see, the commit (which is not in RHEL-6) changes
>> *how* the operand of ljmp is decoded.
>>
>> From "opcode_table" in RHEL-6's "arch/x86/kvm/emulate.c":
>>
>>
>>         /* 0xE8 - 0xEF */
>>         SrcImm | Stack, SrcImm | ImplicitOps,
>>         SrcImmU | Src2Imm16 | No64, SrcImmByte | ImplicitOps,
>>         ^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> and the patch changes that to
>>
>>         SrcImmFAddr | No64
>>
>> and adds new logic to fetch this source operand type.
>>
>> ... Which then seems to have an effect on what goes into
>> load_segment_descriptor() as segment selector, in the emulation of 0xea.
>>
>> Of course I'm insufficiently equipped to debate this with you in earnest
>> :), but it seemed relevant to me.
> 
> Yeah, it seems relevant to me too.
> 
> But before it was decoding two immediates, one after another, the first
> c->op_bytes long in c->src, and the second 2 bytes long in c->src2.  Now
> it's doing the same, but putting all c->op_bytes+2 bytes in c->src...

You were right (what a surprise! :))

First (as I suspected) when unrestricted_guest is supported and enabled
on the host, everything works.

In case unrestricted_guest is either unsupported or disabled, the
symptom manifests itself. I added some debug messages to the emulation
code in KVM where I expected something to go wrong (near 0xea (jmp far)
and near Src2Imm16). Nothing was printed, indicating that the emulation
code never ran.

I looked up the hardware exit reason in the report (80000021) -- it's
EXIT_REASON_INVALID_STATE.

Thus I started browsing the KVM commit log for "unrestricted". Obviously
the commit I first found had to be

  commit daf727225b8abfdfe424716abac3d15a3ac5626a
  Author: Paolo Bonzini <pbonzini@redhat.com>
  Date:   Thu Oct 31 23:05:24 2013 +0100

      KVM: x86: fix emulation of "movzbl %bpl, %eax"

(by whom else :)), and the rest of the commit message taught me about
the "emulate_invalid_guest_state" module parameter (of module kvm-intel).

When setting this modparam to 1, the guest progresses a bit farther, but
then the following appears in the dmesg:

  emulation failed (emulation failure) rip 225 ff 2e 4e 00

Which seems to refer to

 3e6:   ff 2e                   ljmp   *(%esi)
 3e8:   4e                      dec    %esi

(also visible in the earlier disassembly).

Based on the upstream kernel, it looks like the RHEL-6 kernel misses
"Group5 / jmp_far" emulation:

  <http://thread.gmane.org/gmane.comp.emulators.kvm.devel/47489>

Patch 1:

  commit e35b7b9c9e7d8768ee34e5904fed4cb0f2c2cb5d
  Author: Gleb Natapov <gleb@redhat.com>
  Date:   Thu Feb 25 16:36:42 2010 +0200

      KVM: x86 emulator: Add decoding of 16bit second in memory argument

      Add decoding of Ep type of argument used by callf/jmpf.

      Signed-off-by: Gleb Natapov <gleb@redhat.com>
      Signed-off-by: Avi Kivity <avi@redhat.com>

Patch 2:

  commit ea79849d4c8461034b75acb19c8041b6fddee2a5
  Author: Gleb Natapov <gleb@redhat.com>
  Date:   Thu Feb 25 16:36:43 2010 +0200

      KVM: x86 emulator: Implement jmp far opcode ff/5

      Implement jmp far opcode ff/5. It is used by multiboot loader.

      Signed-off-by: Gleb Natapov <gleb@redhat.com>
      Signed-off-by: Avi Kivity <avi@redhat.com>


These were first released in v2.6.35, the RHEL-6 kernel lacks them, but
they are clean cherry-picks. They solve the problem for me.

I filed

  https://bugzilla.redhat.com/show_bug.cgi?id=1059496

Thanks!
Laszlo

Re: [Qemu-devel] pxe boot problems

[Dietmar Maurer]

> Does it work with TCG? 

It simply hangs a bit later if I use TCG, without any output on the console.

But It works perfectly when I switch back to the pxe-XX.rom files.

> Also, can you try with a NIC model different from the default e1000?

same behavior with e1000, rtl8139, pcnet

Re: [Qemu-devel] pxe boot problems

[Laszlo Ersek]

On 01/30/14 07:37, Dietmar Maurer wrote:
>> Does it work with TCG? 
> 
> It simply hangs a bit later if I use TCG, without any output on the console.

Strange. How recent qemu happens this with? The relevant emulation code
(under "ljmp Ev" in "target-i386/translate.c") has been changed as
recently as commit 78261634 (not in any release yet).

> But It works perfectly when I switch back to the pxe-XX.rom files.
> 
>> Also, can you try with a NIC model different from the default e1000?
> 
> same behavior with e1000, rtl8139, pcnet

These do match my results.

Please allow me to summarize the rest of the thread:

- New builds of iPXE contain funny jmp instructions.

- They are only present in the qemu tree in the efi-*.rom files,
  the pxe-*.rom builds date back to much earlier.

- When running the funny jmp instructions in a KVM guest, you either
  need "unrestricted_guest" support from the host CPU (check the
  "/sys/module/kvm_intel/parameters/unrestricted_guest" file when
  kvm-intel.ko is inserted), *or* you need to ask KVM to emulate invalid
  guest state, by passing "emulate_invalid_guest_state=1" to
  kvm-intel.ko -- check your module options under /etc/modprobe.d.
  (Note that you should rebuild the initramfs with dracut if you change
  those options.)

- In the latter case (ie. unrestricted_guest==0 &&
  emulate_invalid_guest_state==1), you will still run into an emulation
  problem on a current RHEL-6 host *later* (a different jmp insn in
  the iPXE builds). I filed RHBZ#1059496 for this and posted the
  backport last night. Gleb's upstream patches in question are e35b7b9c
  and ea79849d.

Laszlo