e1000 and PXE issues

e1000 and PXE issues

[Greg Kurtzer]

Hello,

I noticed some problems with the e1000 implementation in kvm >= 70. At
first glance it seemed liked a PXE problem as it would not acknowledge
the DHCP offer from the server. I tried several different Etherboot
ROM images and version 5.2.6 seemed to work. That version isn't PXE
compliant so I built an ELF image to boot, and it downloaded it very,
very, very, very slowly (as in about 10 minutes) but it did end up
working.

This all worked perfectly with version 69 and previous.

Please let me know if there is a need for any further information, or
anything additional to test.

Please note that I am not subscribed to this email list so please CC
me directly with any responses.

Thank you!

Greg
-- 
Greg Kurtzer
http://www.infiscale.com/
http://www.runlevelzero.net/
http://www.perceus.org/
http://www.caoslinux.org/

Re: e1000 and PXE issues

[H. Peter Anvin]

Greg Kurtzer wrote:
> Hello,
> 
> I noticed some problems with the e1000 implementation in kvm >= 70. At
> first glance it seemed liked a PXE problem as it would not acknowledge
> the DHCP offer from the server. I tried several different Etherboot
> ROM images and version 5.2.6 seemed to work. That version isn't PXE
> compliant so I built an ELF image to boot, and it downloaded it very,
> very, very, very slowly (as in about 10 minutes) but it did end up
> working.
> 
> This all worked perfectly with version 69 and previous.
> 
> Please let me know if there is a need for any further information, or
> anything additional to test.
> 
> Please note that I am not subscribed to this email list so please CC
> me directly with any responses.
> 

I think the e1000 driver in gPXE might have bitrotted.

	-hpa

Re: e1000 and PXE issues

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 773 bytes --]

Greg Kurtzer wrote:
> Hello,
>
> I noticed some problems with the e1000 implementation in kvm >= 70. At
> first glance it seemed liked a PXE problem as it would not acknowledge
> the DHCP offer from the server. I tried several different Etherboot
> ROM images and version 5.2.6 seemed to work. That version isn't PXE
> compliant so I built an ELF image to boot, and it downloaded it very,
> very, very, very slowly (as in about 10 minutes) but it did end up
> working.
>
> This all worked perfectly with version 69 and previous.
>   

There are two patches to e1000 in kvm-70; can you try backing them out 
(patch -Rp1 < test.patch) to see which one is guilty?

Candidates attached.


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


[-- Attachment #2: test1.patch --]
[-- Type: text/x-patch, Size: 1291 bytes --]

commit 2ec717a98c8b8e0c8f1ddc562115ca81d2dc024e
Author: Laurent Vivier <Laurent.Vivier@bull.net>
Date:   Fri May 30 16:07:31 2008 +0200

    kvm: qemu: coalesced MMIO support (e1000)
    
    This patch defines coalesced MMIO zones for e1000 ethernet card.
    
    Signed-off-by: Laurent Vivier <Laurent.Vivier@bull.net>
    Signed-off-by: Avi Kivity <avi@qumranet.com>

diff --git a/qemu/hw/e1000.c b/qemu/hw/e1000.c
index 01f8983..5b3a365 100644
--- a/qemu/hw/e1000.c
+++ b/qemu/hw/e1000.c
@@ -26,6 +26,7 @@
 #include "hw.h"
 #include "pci.h"
 #include "net.h"
+#include "qemu-kvm.h"
 
 #include "e1000_hw.h"
 
@@ -938,6 +939,18 @@ e1000_mmio_map(PCIDevice *pci_dev, int region_num,
 
     d->mmio_base = addr;
     cpu_register_physical_memory(addr, PNPMMIO_SIZE, d->mmio_index);
+
+    if (kvm_enabled()) {
+	int i;
+        uint32_t excluded_regs[] = {
+            E1000_MDIC, E1000_ICR, E1000_ICS, E1000_IMS,
+            E1000_IMC, E1000_TCTL, E1000_TDT, PNPMMIO_SIZE
+        };
+        qemu_kvm_register_coalesced_mmio(addr, excluded_regs[0]);
+        for (i = 0; excluded_regs[i] != PNPMMIO_SIZE; i++)
+            qemu_kvm_register_coalesced_mmio(addr + excluded_regs[i] + 4,
+                             excluded_regs[i + 1] - excluded_regs[i] - 4);
+    }
 }
 
 static int

[-- Attachment #3: test2.patch --]
[-- Type: text/x-patch, Size: 1018 bytes --]

commit 404ce95cb202ca6015beb26e9b870f91c0309ee0
Author: Anthony Liguori <aliguori@us.ibm.com>
Date:   Wed May 7 16:40:58 2008 -0500

    kvm: qemu: fix e1000 can_receive handler
    
    The current logic of the can_receive handler is to allow packets whenever the
    receiver is disabled or when there are descriptors available in the ring.
    
    I think the logic ought to be to allow packets whenever the receiver is enabled
    and there are descriptors available in the ring.
    
    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
    Signed-off-by: Avi Kivity <avi@qumranet.com>

diff --git a/qemu/hw/e1000.c b/qemu/hw/e1000.c
index 0728539..01f8983 100644
--- a/qemu/hw/e1000.c
+++ b/qemu/hw/e1000.c
@@ -520,8 +520,8 @@ e1000_can_receive(void *opaque)
 {
     E1000State *s = opaque;
 
-    return (!(s->mac_reg[RCTL] & E1000_RCTL_EN) ||
-            s->mac_reg[RDH] != s->mac_reg[RDT]);
+    return ((s->mac_reg[RCTL] & E1000_RCTL_EN) &&
+	    s->mac_reg[RDH] != s->mac_reg[RDT]);
 }
 
 static void

Re: e1000 and PXE issues

[Greg Kurtzer]

On Sat, Jul 26, 2008 at 1:12 AM, Avi Kivity <avi@qumranet.com> wrote:
> Greg Kurtzer wrote:
>>
>> Hello,
>>
>> I noticed some problems with the e1000 implementation in kvm >= 70. At
>> first glance it seemed liked a PXE problem as it would not acknowledge
>> the DHCP offer from the server. I tried several different Etherboot
>> ROM images and version 5.2.6 seemed to work. That version isn't PXE
>> compliant so I built an ELF image to boot, and it downloaded it very,
>> very, very, very slowly (as in about 10 minutes) but it did end up
>> working.
>>
>> This all worked perfectly with version 69 and previous.
>>
>
> There are two patches to e1000 in kvm-70; can you try backing them out
> (patch -Rp1 < test.patch) to see which one is guilty?
>
> Candidates attached.

Reverted both of these patches yet the problem remains. :(

This is easy to reproduce (at least on my build). if there is a DHCP
server on the network, just do a network boot on the e1000. It makes a
correct DHCPDISCOVER, but never responds to the DHCPOFFER (it should
do a DHCPREQUEST next). No packets are getting lost according tcpdump
on the master.

The console is showing:

Probing pci nic...
[e1000-82540em]Ethernet addr: 00:04:21:DE:99:55
Searching for server (DHCP)....No IP address
.No IP address
.No IP address
.No IP address
.No IP address
.No IP address
.No IP address
...

Thanks again for the help!

-- 
Greg Kurtzer
http://www.infiscale.com/
http://www.runlevelzero.net/
http://www.perceus.org/
http://www.caoslinux.org/

Re: e1000 and PXE issues

[Farkas Levente]

Greg Kurtzer wrote:
> On Sat, Jul 26, 2008 at 1:12 AM, Avi Kivity <avi@qumranet.com> wrote:
>> Greg Kurtzer wrote:
>>> Hello,
>>>
>>> I noticed some problems with the e1000 implementation in kvm >= 70. At
>>> first glance it seemed liked a PXE problem as it would not acknowledge
>>> the DHCP offer from the server. I tried several different Etherboot
>>> ROM images and version 5.2.6 seemed to work. That version isn't PXE
>>> compliant so I built an ELF image to boot, and it downloaded it very,
>>> very, very, very slowly (as in about 10 minutes) but it did end up
>>> working.
>>>
>>> This all worked perfectly with version 69 and previous.
>>>
>> There are two patches to e1000 in kvm-70; can you try backing them out
>> (patch -Rp1 < test.patch) to see which one is guilty?
>>
>> Candidates attached.
> 
> Reverted both of these patches yet the problem remains. :(
> 
> This is easy to reproduce (at least on my build). if there is a DHCP
> server on the network, just do a network boot on the e1000. It makes a
> correct DHCPDISCOVER, but never responds to the DHCPOFFER (it should
> do a DHCPREQUEST next). No packets are getting lost according tcpdump
> on the master.
> 
> The console is showing:
> 
> Probing pci nic...
> [e1000-82540em]Ethernet addr: 00:04:21:DE:99:55
> Searching for server (DHCP)....No IP address
> .No IP address
> .No IP address
> .No IP address
> .No IP address
> .No IP address
> .No IP address
> ...

the same happend with us both with e1000 and rtl8139 driver and the rh 
guy said it's not a virt-manager problem...
currently we're not able to pxe boot any kind of kvm guest with any 
network driver:-(

-- 
   Levente                               "Si vis pacem para bellum!"

Re: e1000 and PXE issues

[Greg Kurtzer]

The default rtl8139 seems to work for me, but granted it doesn't
respond right away. It takes it about 8-10 seconds to figure stuff
out. Also the rtl8139 seems to only come up as half duplex for me (not
sure if that is related, but worth mentioning).

I just tested kvm-72, and that doesn't fix the bug (not a big surprise
considering there was nothing mentioned in the changelog regarding PXE
fixups) but worth noting.

Thanks everyone!

Greg



On Sun, Jul 27, 2008 at 10:32 AM, Farkas Levente <lfarkas@lfarkas.org> wrote:
> Greg Kurtzer wrote:
>>
>> On Sat, Jul 26, 2008 at 1:12 AM, Avi Kivity <avi@qumranet.com> wrote:
>>>
>>> Greg Kurtzer wrote:
>>>>
>>>> Hello,
>>>>
>>>> I noticed some problems with the e1000 implementation in kvm >= 70. At
>>>> first glance it seemed liked a PXE problem as it would not acknowledge
>>>> the DHCP offer from the server. I tried several different Etherboot
>>>> ROM images and version 5.2.6 seemed to work. That version isn't PXE
>>>> compliant so I built an ELF image to boot, and it downloaded it very,
>>>> very, very, very slowly (as in about 10 minutes) but it did end up
>>>> working.
>>>>
>>>> This all worked perfectly with version 69 and previous.
>>>>
>>> There are two patches to e1000 in kvm-70; can you try backing them out
>>> (patch -Rp1 < test.patch) to see which one is guilty?
>>>
>>> Candidates attached.
>>
>> Reverted both of these patches yet the problem remains. :(
>>
>> This is easy to reproduce (at least on my build). if there is a DHCP
>> server on the network, just do a network boot on the e1000. It makes a
>> correct DHCPDISCOVER, but never responds to the DHCPOFFER (it should
>> do a DHCPREQUEST next). No packets are getting lost according tcpdump
>> on the master.
>>
>> The console is showing:
>>
>> Probing pci nic...
>> [e1000-82540em]Ethernet addr: 00:04:21:DE:99:55
>> Searching for server (DHCP)....No IP address
>> .No IP address
>> .No IP address
>> .No IP address
>> .No IP address
>> .No IP address
>> .No IP address
>> ...
>
> the same happend with us both with e1000 and rtl8139 driver and the rh guy
> said it's not a virt-manager problem...
> currently we're not able to pxe boot any kind of kvm guest with any network
> driver:-(
>
> --
>  Levente                               "Si vis pacem para bellum!"
>



-- 
Greg Kurtzer
http://www.infiscale.com/
http://www.runlevelzero.net/
http://www.perceus.org/
http://www.caoslinux.org/

Re: e1000 and PXE issues

[Amit Shah]

* On Sunday 27 Jul 2008 23:02:06 Farkas Levente wrote:
> Greg Kurtzer wrote:
> > On Sat, Jul 26, 2008 at 1:12 AM, Avi Kivity <avi@qumranet.com> wrote:
> >> Greg Kurtzer wrote:
> >>> Hello,
> >>>
> >>> I noticed some problems with the e1000 implementation in kvm >= 70. At
> >>> first glance it seemed liked a PXE problem as it would not acknowledge
> >>> the DHCP offer from the server. I tried several different Etherboot
> >>> ROM images and version 5.2.6 seemed to work. That version isn't PXE
> >>> compliant so I built an ELF image to boot, and it downloaded it very,
> >>> very, very, very slowly (as in about 10 minutes) but it did end up
> >>> working.
> >>>
> >>> This all worked perfectly with version 69 and previous.
> >>
> >> There are two patches to e1000 in kvm-70; can you try backing them out
> >> (patch -Rp1 < test.patch) to see which one is guilty?
> >>
> >> Candidates attached.
> >
> > Reverted both of these patches yet the problem remains. :(
> >
> > This is easy to reproduce (at least on my build). if there is a DHCP
> > server on the network, just do a network boot on the e1000. It makes a
> > correct DHCPDISCOVER, but never responds to the DHCPOFFER (it should
> > do a DHCPREQUEST next). No packets are getting lost according tcpdump
> > on the master.
> >
> > The console is showing:
> >
> > Probing pci nic...
> > [e1000-82540em]Ethernet addr: 00:04:21:DE:99:55
> > Searching for server (DHCP)....No IP address
> > .No IP address
> > .No IP address
> > .No IP address
> > .No IP address
> > .No IP address
> > .No IP address
> > ...
>
> the same happend with us both with e1000 and rtl8139 driver and the rh
> guy said it's not a virt-manager problem...
> currently we're not able to pxe boot any kind of kvm guest with any
> network driver:-(

Is it related to tap?

Re: e1000 and PXE issues

[Greg Kurtzer]

In my case, it is possible but seems unlikely as I can switch between
a working KVM-69 and KVM>=70 and replicate the bug (nothing related to
the TAP or bridge support or configuration on the system is changing
in my scenario). Still it could be a bug that just gets tickled by the
newer KVM's, but I can't say specifically.

I can provide a URL to an ISO of our Linux distribution that
consistently replicates this problem if it can't be replicated
upstream.

Thanks for the suggestion! :)

Greg



On Mon, Jul 28, 2008 at 1:16 AM, Amit Shah <amit.shah@qumranet.com> wrote:
> * On Sunday 27 Jul 2008 23:02:06 Farkas Levente wrote:
>> Greg Kurtzer wrote:
>> > On Sat, Jul 26, 2008 at 1:12 AM, Avi Kivity <avi@qumranet.com> wrote:
>> >> Greg Kurtzer wrote:
>> >>> Hello,
>> >>>
>> >>> I noticed some problems with the e1000 implementation in kvm >= 70. At
>> >>> first glance it seemed liked a PXE problem as it would not acknowledge
>> >>> the DHCP offer from the server. I tried several different Etherboot
>> >>> ROM images and version 5.2.6 seemed to work. That version isn't PXE
>> >>> compliant so I built an ELF image to boot, and it downloaded it very,
>> >>> very, very, very slowly (as in about 10 minutes) but it did end up
>> >>> working.
>> >>>
>> >>> This all worked perfectly with version 69 and previous.
>> >>
>> >> There are two patches to e1000 in kvm-70; can you try backing them out
>> >> (patch -Rp1 < test.patch) to see which one is guilty?
>> >>
>> >> Candidates attached.
>> >
>> > Reverted both of these patches yet the problem remains. :(
>> >
>> > This is easy to reproduce (at least on my build). if there is a DHCP
>> > server on the network, just do a network boot on the e1000. It makes a
>> > correct DHCPDISCOVER, but never responds to the DHCPOFFER (it should
>> > do a DHCPREQUEST next). No packets are getting lost according tcpdump
>> > on the master.
>> >
>> > The console is showing:
>> >
>> > Probing pci nic...
>> > [e1000-82540em]Ethernet addr: 00:04:21:DE:99:55
>> > Searching for server (DHCP)....No IP address
>> > .No IP address
>> > .No IP address
>> > .No IP address
>> > .No IP address
>> > .No IP address
>> > .No IP address
>> > ...
>>
>> the same happend with us both with e1000 and rtl8139 driver and the rh
>> guy said it's not a virt-manager problem...
>> currently we're not able to pxe boot any kind of kvm guest with any
>> network driver:-(
>
> Is it related to tap?
>



-- 
Greg Kurtzer
http://www.infiscale.com/
http://www.runlevelzero.net/
http://www.perceus.org/
http://www.caoslinux.org/

Re: e1000 and PXE issues

[Charles Duffy]

FYI,

I'm experiencing this as well, and note that the issue still occurs with 
-no-kvm, but not with current qemu-trunk (r4955); that said, 
qemu-trunk's behavior doesn't seem to be perfect either -- there are 
several "No IP address" responses before one is acknowledged.