Fwd: Why doesn't Intel e1000 NIC work correctly in Windows XP?

Fwd: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Flypen CloudMe]

Hello,


My Linux kernel is 2.6.32, and my machine has an Intel e1000 1Gbps
NIC. The Guest OS is Windows XP. I use PCI passthrough to pass the
e1000 NIC to the VM, so Windows XP can directly use this physical NIC
and should has higher performance.

I successfully install Intel PRO/1000 PT driver in XP. Then the NIC
can work, and the performance is not bad (about 700Mbps). However, if
I reboot XP, the strange thing happens. Just after reboot, the NIC can
be used. But if I run some other programs with some IO traffic, no
data can be transferred via this NIC from then on. There are no
abnormal information in Windows event logs, and there are no warnings
or errors for Intel PRO/1000 PT driver in Windows device manager. I
can't ping the VM from any machines. Why doesn't Intel e1000 NIC work
correctly after reboot?

Another strange thing. If I re-install PRO/1000 PT driver, the NIC can
be used again. No matter what applications I run in Guest OS, the NIC
can work well as long as I don't reboot it. But after I reboot the
Guest OS, I must re-install the NIC driver each time, otherwise the
NIC can't work.

Who knows the reason of this strange behavior for e1000 NIC in KVM
Windows environment? Is it driver related? I also use Linux as the
Guest OS, everything is well even I reboot the VM many times.

Thanks a lot!
flypen

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Alex Williamson]

On Mon, Jun 13, 2011 at 8:15 PM, Flypen CloudMe <flypen@gmail.com> wrote:
> Hello,
>
>
> My Linux kernel is 2.6.32, and my machine has an Intel e1000 1Gbps
> NIC. The Guest OS is Windows XP. I use PCI passthrough to pass the
> e1000 NIC to the VM, so Windows XP can directly use this physical NIC
> and should has higher performance.
>
> I successfully install Intel PRO/1000 PT driver in XP. Then the NIC
> can work, and the performance is not bad (about 700Mbps). However, if
> I reboot XP, the strange thing happens. Just after reboot, the NIC can
> be used. But if I run some other programs with some IO traffic, no
> data can be transferred via this NIC from then on. There are no
> abnormal information in Windows event logs, and there are no warnings
> or errors for Intel PRO/1000 PT driver in Windows device manager. I
> can't ping the VM from any machines. Why doesn't Intel e1000 NIC work
> correctly after reboot?
>
> Another strange thing. If I re-install PRO/1000 PT driver, the NIC can
> be used again. No matter what applications I run in Guest OS, the NIC
> can work well as long as I don't reboot it. But after I reboot the
> Guest OS, I must re-install the NIC driver each time, otherwise the
> NIC can't work.
>
> Who knows the reason of this strange behavior for e1000 NIC in KVM
> Windows environment? Is it driver related? I also use Linux as the
> Guest OS, everything is well even I reboot the VM many times.

What version of kvm are you using?  Thanks,

Alex

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Flypen CloudMe]

Hi,

I use Redhat Enterprise Linux 6, and use the KVM that is released by
Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.

It seems that the IRQs are conflicted after reboot. The NIC and the
SCSI controller have the same IRQ number. If I re-install the NIC
driver, the IRQ number of the NIC will be assigned another value, then
it can work normally. Do we have a way to let the NIC and the SCSI
controller have different IRQ number in VM?

Thanks,
flypen

On Tue, Jun 14, 2011 at 11:23 AM, Alex Williamson
<alex.williamson@redhat.com> wrote:
>
> On Mon, Jun 13, 2011 at 8:15 PM, Flypen CloudMe <flypen@gmail.com> wrote:
> > Hello,
> >
> >
> > My Linux kernel is 2.6.32, and my machine has an Intel e1000 1Gbps
> > NIC. The Guest OS is Windows XP. I use PCI passthrough to pass the
> > e1000 NIC to the VM, so Windows XP can directly use this physical NIC
> > and should has higher performance.
> >
> > I successfully install Intel PRO/1000 PT driver in XP. Then the NIC
> > can work, and the performance is not bad (about 700Mbps). However, if
> > I reboot XP, the strange thing happens. Just after reboot, the NIC can
> > be used. But if I run some other programs with some IO traffic, no
> > data can be transferred via this NIC from then on. There are no
> > abnormal information in Windows event logs, and there are no warnings
> > or errors for Intel PRO/1000 PT driver in Windows device manager. I
> > can't ping the VM from any machines. Why doesn't Intel e1000 NIC work
> > correctly after reboot?
> >
> > Another strange thing. If I re-install PRO/1000 PT driver, the NIC can
> > be used again. No matter what applications I run in Guest OS, the NIC
> > can work well as long as I don't reboot it. But after I reboot the
> > Guest OS, I must re-install the NIC driver each time, otherwise the
> > NIC can't work.
> >
> > Who knows the reason of this strange behavior for e1000 NIC in KVM
> > Windows environment? Is it driver related? I also use Linux as the
> > Guest OS, everything is well even I reboot the VM many times.
>
> What version of kvm are you using?  Thanks,
>
> Alex

RE: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Decker, Schorschi]

Is IRQ balancing enabled on at BIOS level?


Schorschi Decker

VP; Sr. Consultant Engineer
ECT&O Emerging Technologies / Virtualization Platform Engineering Team
Bank of America

Office 213-345-4714



-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Flypen CloudMe
Sent: Tuesday, 14 June, 2011 01:11
To: Alex Williamson
Cc: kvm@vger.kernel.org
Subject: Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

Hi,

I use Redhat Enterprise Linux 6, and use the KVM that is released by Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.

It seems that the IRQs are conflicted after reboot. The NIC and the SCSI controller have the same IRQ number. If I re-install the NIC driver, the IRQ number of the NIC will be assigned another value, then it can work normally. Do we have a way to let the NIC and the SCSI controller have different IRQ number in VM?

Thanks,
flypen

On Tue, Jun 14, 2011 at 11:23 AM, Alex Williamson <alex.williamson@redhat.com> wrote:
>
> On Mon, Jun 13, 2011 at 8:15 PM, Flypen CloudMe <flypen@gmail.com> wrote:
> > Hello,
> >
> >
> > My Linux kernel is 2.6.32, and my machine has an Intel e1000 1Gbps 
> > NIC. The Guest OS is Windows XP. I use PCI passthrough to pass the 
> > e1000 NIC to the VM, so Windows XP can directly use this physical 
> > NIC and should has higher performance.
> >
> > I successfully install Intel PRO/1000 PT driver in XP. Then the NIC 
> > can work, and the performance is not bad (about 700Mbps). However, 
> > if I reboot XP, the strange thing happens. Just after reboot, the 
> > NIC can be used. But if I run some other programs with some IO 
> > traffic, no data can be transferred via this NIC from then on. There 
> > are no abnormal information in Windows event logs, and there are no 
> > warnings or errors for Intel PRO/1000 PT driver in Windows device 
> > manager. I can't ping the VM from any machines. Why doesn't Intel 
> > e1000 NIC work correctly after reboot?
> >
> > Another strange thing. If I re-install PRO/1000 PT driver, the NIC 
> > can be used again. No matter what applications I run in Guest OS, 
> > the NIC can work well as long as I don't reboot it. But after I 
> > reboot the Guest OS, I must re-install the NIC driver each time, 
> > otherwise the NIC can't work.
> >
> > Who knows the reason of this strange behavior for e1000 NIC in KVM 
> > Windows environment? Is it driver related? I also use Linux as the 
> > Guest OS, everything is well even I reboot the VM many times.
>
> What version of kvm are you using?  Thanks,
>
> Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

----------------------------------------------------------------------
This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments, and be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. 
Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Sender. Subject to applicable law, Sender may intercept, monitor, review and retain e-communications (EC) traveling through its networks/systems and may produce any such EC to regulators, law enforcement, in litigation and as required by law. 
The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or free of errors or viruses. 

References to "Sender" are references to any subsidiary of Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this EC may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: 
http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you consent to the foregoing.

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Alex Williamson]

On Tue, 2011-06-14 at 16:11 +0800, Flypen CloudMe wrote:
> Hi,
> 
> I use Redhat Enterprise Linux 6, and use the KVM that is released by
> Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.
> 
> It seems that the IRQs are conflicted after reboot. The NIC and the
> SCSI controller have the same IRQ number. If I re-install the NIC
> driver, the IRQ number of the NIC will be assigned another value, then
> it can work normally. Do we have a way to let the NIC and the SCSI
> controller have different IRQ number in VM?

I'll see if I can reproduce and figure anything out.  Windows XP isn't a
guest we concentrate on, especially with device assignment.  Are you
using an AMD or Intel host system?  Does the same thing happen if you
run the XP guest on an IDE controller?  It would be helpful to post the
guest configuration, command line used or libvirt xml.  Also, you might
try latest upstream qemu-kvm to see if the problem still exists.
Thanks,

Alex

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Jan Kiszka]

On 2011-06-15 02:54, Alex Williamson wrote:
> On Tue, 2011-06-14 at 16:11 +0800, Flypen CloudMe wrote:
>> Hi,
>>
>> I use Redhat Enterprise Linux 6, and use the KVM that is released by
>> Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.
>>
>> It seems that the IRQs are conflicted after reboot. The NIC and the
>> SCSI controller have the same IRQ number. If I re-install the NIC
>> driver, the IRQ number of the NIC will be assigned another value, then
>> it can work normally. Do we have a way to let the NIC and the SCSI
>> controller have different IRQ number in VM?
> 
> I'll see if I can reproduce and figure anything out.  Windows XP isn't a
> guest we concentrate on, especially with device assignment.  Are you
> using an AMD or Intel host system?  Does the same thing happen if you
> run the XP guest on an IDE controller?  It would be helpful to post the
> guest configuration, command line used or libvirt xml.  Also, you might
> try latest upstream qemu-kvm to see if the problem still exists.

Maybe tracking of the INTx route across reset fails. Does this help?

diff --git a/hw/device-assignment.c b/hw/device-assignment.c
index 7eeecad..0693141 100644
--- a/hw/device-assignment.c
+++ b/hw/device-assignment.c
@@ -1719,6 +1719,8 @@ static void reset_assigned_device(DeviceState *dev)
      * disconnected from the PCI bus. This avoids further DMA transfers.
      */
     assigned_dev_pci_write_config(pci_dev, PCI_COMMAND, 0, 2);
+
+    assign_irq(adev);
 }
 
 static int assigned_initfn(struct PCIDevice *pci_dev)


Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Jan Kiszka]

On 2011-06-15 10:04, Jan Kiszka wrote:
> On 2011-06-15 02:54, Alex Williamson wrote:
>> On Tue, 2011-06-14 at 16:11 +0800, Flypen CloudMe wrote:
>>> Hi,
>>>
>>> I use Redhat Enterprise Linux 6, and use the KVM that is released by
>>> Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.
>>>
>>> It seems that the IRQs are conflicted after reboot. The NIC and the
>>> SCSI controller have the same IRQ number. If I re-install the NIC
>>> driver, the IRQ number of the NIC will be assigned another value, then
>>> it can work normally. Do we have a way to let the NIC and the SCSI
>>> controller have different IRQ number in VM?
>>
>> I'll see if I can reproduce and figure anything out.  Windows XP isn't a
>> guest we concentrate on, especially with device assignment.  Are you
>> using an AMD or Intel host system?  Does the same thing happen if you
>> run the XP guest on an IDE controller?  It would be helpful to post the
>> guest configuration, command line used or libvirt xml.  Also, you might
>> try latest upstream qemu-kvm to see if the problem still exists.
> 
> Maybe tracking of the INTx route across reset fails. Does this help?
> 
> diff --git a/hw/device-assignment.c b/hw/device-assignment.c
> index 7eeecad..0693141 100644
> --- a/hw/device-assignment.c
> +++ b/hw/device-assignment.c
> @@ -1719,6 +1719,8 @@ static void reset_assigned_device(DeviceState *dev)
>       * disconnected from the PCI bus. This avoids further DMA transfers.
>       */
>      assigned_dev_pci_write_config(pci_dev, PCI_COMMAND, 0, 2);
> +
> +    assign_irq(adev);
>  }
>  
>  static int assigned_initfn(struct PCIDevice *pci_dev)
> 

Nonsense, can't t make a difference as the PIIX3 resets the routing to
disable - which device-assignment does not deal with, but that's unrelated.

Try assigning a different slot to the passed-through adapter and the
lsi. For me it helped to put the lsi on one slot behind the
auto-assigned (-device lsi,addr=5).

I guess classic device assignment cannot support INTx sharing as the
kernel IRQ injection path does not inform user space about the device
state /wrt IRQs. Should be catch and reject this, or try to fix it up be
moving the assigned device around, Alex?

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Alex Williamson]

On Wed, 2011-06-15 at 11:31 +0200, Jan Kiszka wrote:
> On 2011-06-15 10:04, Jan Kiszka wrote:
> > On 2011-06-15 02:54, Alex Williamson wrote:
> >> On Tue, 2011-06-14 at 16:11 +0800, Flypen CloudMe wrote:
> >>> Hi,
> >>>
> >>> I use Redhat Enterprise Linux 6, and use the KVM that is released by
> >>> Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.
> >>>
> >>> It seems that the IRQs are conflicted after reboot. The NIC and the
> >>> SCSI controller have the same IRQ number. If I re-install the NIC
> >>> driver, the IRQ number of the NIC will be assigned another value, then
> >>> it can work normally. Do we have a way to let the NIC and the SCSI
> >>> controller have different IRQ number in VM?

Hmm, I'm still confused here.  I went back and double checked, and as I
thought, we disable the LSI SCSI controller in the RHEL6 KVM.  So I'm
curious what this device is.  Is it an assigned SCSI controller or is
there another one that we forgot to disable in RHEL or is this a
different version of KVM?  The config file or command line would be
handy here.

> >> I'll see if I can reproduce and figure anything out.  Windows XP isn't a
> >> guest we concentrate on, especially with device assignment.  Are you
> >> using an AMD or Intel host system?  Does the same thing happen if you
> >> run the XP guest on an IDE controller?  It would be helpful to post the
> >> guest configuration, command line used or libvirt xml.  Also, you might
> >> try latest upstream qemu-kvm to see if the problem still exists.

I tested with an 82578DM e1000e NIC on an Intel host system, and it
surprisingly worked just fine on the RHEL6.0 base.  This is with a 32bit
Windows XP SP3 install.  The device supports MSI, but windows only seems
to use it with INTx.  I did have to remove the emulated rtl8139 or else
I couldn't even boot due to BSODs in the guest.

> 
> Nonsense, can't t make a difference as the PIIX3 resets the routing to
> disable - which device-assignment does not deal with, but that's unrelated.

Yep, someone has to write it at some point and device assignment will
catch that.

> Try assigning a different slot to the passed-through adapter and the
> lsi. For me it helped to put the lsi on one slot behind the
> auto-assigned (-device lsi,addr=5).
> 
> I guess classic device assignment cannot support INTx sharing as the
> kernel IRQ injection path does not inform user space about the device
> state /wrt IRQs. Should be catch and reject this, or try to fix it up be
> moving the assigned device around, Alex?

I'm pretty sure we've looked at this path in the past and KVM will do
the right thing with respect to shared INTx in the guest (ie. KVM will
continue to assert the interrupt if it's been triggered by both
assignment and userspace until both de-assert).  Experimenting with a
RHEL guest I can make an assigned e1000e and emulated e1000 share an
interrupt.  It works, but performance is terrible and the assigned
device can get into a non-working state if the emulated device is
removed.  Perhaps there is still a path by which interrupts can get
turned off for the assigned device.

Alex

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Flypen CloudMe]

Hi,

Here are the command line:

/usr/bin/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048 -smp
2,sockets=1,cores=2,threads=1 \
-name winxp -uuid 23cd2751-8a30-dd34-db47-bfc8c76ccadb -nodefconfig
-nodefaults \
-chardev socket,id=monitor,path=/var/lib/libvirt/qemu/winxp.monitor,server,nowait
-mon chardev=monitor,mode=readline \
-rtc base=localtime -boot c -device lsi,id=scsi0,bus=pci.0,addr=0x5
-device lsi,id=scsi1,bus=pci.0,addr=0x6 \
-device lsi,id=scsi2,bus=pci.0,addr=0x7 -device
lsi,id=scsi3,bus=pci.0,addr=0x8 \
-drive file=/mnt/vmdisk/winxp.disk,if=none,id=drive-ide0-0-0,boot=on,format=raw,cache=none
\
-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
-drive file=/mnt/vmdisk/virtio-win-1.1.16.vfd,if=none,id=drive-fdc0-0-0,format=raw,cache=none\
-global isa-fdc.driveA=drive-fdc0-0-0 -drive
file=/dev/sd1,if=none,id=drive-scsi0-0-0,format=raw,cache=none \
-device scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 \
-drive file=/dev/sdb,if=none,id=drive-scsi0-0-1,format=raw,cache=none \
-device scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1 \
-drive file=/dev/sdc,if=none,id=drive-scsi0-0-2,format=raw,cache=none \
-device scsi-disk,bus=scsi0.0,scsi-id=2,drive=drive-scsi0-0-2,id=scsi0-0-2 \
-drive file=/dev/sdd,if=none,id=drive-scsi0-0-3,format=raw,cache=none \
-device scsi-disk,bus=scsi0.0,scsi-id=3,drive=drive-scsi0-0-3,id=scsi0-0-3 \
-drive file=/dev/sde,if=none,id=drive-scsi0-0-4,format=raw,cache=none \
-device scsi-disk,bus=scsi0.0,scsi-id=4,drive=drive-scsi0-0-4,id=scsi0-0-4 \
-drive file=/dev/sdf,if=none,id=drive-scsi3-0-0,format=raw,cache=none \
-device scsi-disk,bus=scsi3.0,scsi-id=0,drive=drive-scsi3-0-0,id=scsi3-0-0 \
-drive file=/mnt/vmdisk/D/1,if=none,id=drive-scsi0-0-6,format=raw,cache=none \
-device scsi-disk,bus=scsi0.0,scsi-id=6,drive=drive-scsi0-0-6,id=scsi0-0-6 \
-chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb \
-vnc 0.0.0.0:0 -k en-us -vga vmware -device
pci-assign,host=02:00.0,id=hostdev0,configfd=18,bus=pci.0,addr=0x3 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

The NIC and one SCSI controller (slot 7) has the same IRQ. The
performance in XP is really bad. When writing traffic to the drive,
the NIC can't be accessed, and ping will be also timeout.
If I let the NIC has the different IRQ number, then everything is OK.
Is it related to INTx model for XP?

We rebuild the QEMU, and add the LSI SCSI controller support. Why does
RHEL6 removes its support? Is this controller too old? Are there any
emulated SCSI devices to replace it?

Thanks,
flypen


On Thu, Jun 16, 2011 at 2:42 AM, Alex Williamson
<alex.williamson@redhat.com> wrote:
>
> On Wed, 2011-06-15 at 11:31 +0200, Jan Kiszka wrote:
> > On 2011-06-15 10:04, Jan Kiszka wrote:
> > > On 2011-06-15 02:54, Alex Williamson wrote:
> > >> On Tue, 2011-06-14 at 16:11 +0800, Flypen CloudMe wrote:
> > >>> Hi,
> > >>>
> > >>> I use Redhat Enterprise Linux 6, and use the KVM that is released by
> > >>> Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.
> > >>>
> > >>> It seems that the IRQs are conflicted after reboot. The NIC and the
> > >>> SCSI controller have the same IRQ number. If I re-install the NIC
> > >>> driver, the IRQ number of the NIC will be assigned another value, then
> > >>> it can work normally. Do we have a way to let the NIC and the SCSI
> > >>> controller have different IRQ number in VM?
>
> Hmm, I'm still confused here.  I went back and double checked, and as I
> thought, we disable the LSI SCSI controller in the RHEL6 KVM.  So I'm
> curious what this device is.  Is it an assigned SCSI controller or is
> there another one that we forgot to disable in RHEL or is this a
> different version of KVM?  The config file or command line would be
> handy here.
>
> > >> I'll see if I can reproduce and figure anything out.  Windows XP isn't a
> > >> guest we concentrate on, especially with device assignment.  Are you
> > >> using an AMD or Intel host system?  Does the same thing happen if you
> > >> run the XP guest on an IDE controller?  It would be helpful to post the
> > >> guest configuration, command line used or libvirt xml.  Also, you might
> > >> try latest upstream qemu-kvm to see if the problem still exists.
>
> I tested with an 82578DM e1000e NIC on an Intel host system, and it
> surprisingly worked just fine on the RHEL6.0 base.  This is with a 32bit
> Windows XP SP3 install.  The device supports MSI, but windows only seems
> to use it with INTx.  I did have to remove the emulated rtl8139 or else
> I couldn't even boot due to BSODs in the guest.
>
> >
> > Nonsense, can't t make a difference as the PIIX3 resets the routing to
> > disable - which device-assignment does not deal with, but that's unrelated.
>
> Yep, someone has to write it at some point and device assignment will
> catch that.
>
> > Try assigning a different slot to the passed-through adapter and the
> > lsi. For me it helped to put the lsi on one slot behind the
> > auto-assigned (-device lsi,addr=5).
> >
> > I guess classic device assignment cannot support INTx sharing as the
> > kernel IRQ injection path does not inform user space about the device
> > state /wrt IRQs. Should be catch and reject this, or try to fix it up be
> > moving the assigned device around, Alex?
>
> I'm pretty sure we've looked at this path in the past and KVM will do
> the right thing with respect to shared INTx in the guest (ie. KVM will
> continue to assert the interrupt if it's been triggered by both
> assignment and userspace until both de-assert).  Experimenting with a
> RHEL guest I can make an assigned e1000e and emulated e1000 share an
> interrupt.  It works, but performance is terrible and the assigned
> device can get into a non-working state if the emulated device is
> removed.  Perhaps there is still a path by which interrupts can get
> turned off for the assigned device.
>
> Alex
>
>
>

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Dor Laor]

Not sure it might help but IIRC the original e1000 driver for windows 
had some bugs that were fixed if you'll download the most recent driver 
from Intel site. This was the case for the fully emulated e1000 qemu 
device and might help here too.

On 06/19/2011 03:29 PM, Flypen CloudMe wrote:
> Hi,
>
> Here are the command line:
>
> /usr/bin/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048 -smp
> 2,sockets=1,cores=2,threads=1 \
> -name winxp -uuid 23cd2751-8a30-dd34-db47-bfc8c76ccadb -nodefconfig
> -nodefaults \
> -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/winxp.monitor,server,nowait
> -mon chardev=monitor,mode=readline \
> -rtc base=localtime -boot c -device lsi,id=scsi0,bus=pci.0,addr=0x5
> -device lsi,id=scsi1,bus=pci.0,addr=0x6 \
> -device lsi,id=scsi2,bus=pci.0,addr=0x7 -device
> lsi,id=scsi3,bus=pci.0,addr=0x8 \
> -drive file=/mnt/vmdisk/winxp.disk,if=none,id=drive-ide0-0-0,boot=on,format=raw,cache=none
> \
> -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
> -drive file=/mnt/vmdisk/virtio-win-1.1.16.vfd,if=none,id=drive-fdc0-0-0,format=raw,cache=none\
> -global isa-fdc.driveA=drive-fdc0-0-0 -drive
> file=/dev/sd1,if=none,id=drive-scsi0-0-0,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 \
> -drive file=/dev/sdb,if=none,id=drive-scsi0-0-1,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1 \
> -drive file=/dev/sdc,if=none,id=drive-scsi0-0-2,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=2,drive=drive-scsi0-0-2,id=scsi0-0-2 \
> -drive file=/dev/sdd,if=none,id=drive-scsi0-0-3,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=3,drive=drive-scsi0-0-3,id=scsi0-0-3 \
> -drive file=/dev/sde,if=none,id=drive-scsi0-0-4,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=4,drive=drive-scsi0-0-4,id=scsi0-0-4 \
> -drive file=/dev/sdf,if=none,id=drive-scsi3-0-0,format=raw,cache=none \
> -device scsi-disk,bus=scsi3.0,scsi-id=0,drive=drive-scsi3-0-0,id=scsi3-0-0 \
> -drive file=/mnt/vmdisk/D/1,if=none,id=drive-scsi0-0-6,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=6,drive=drive-scsi0-0-6,id=scsi0-0-6 \
> -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb \
> -vnc 0.0.0.0:0 -k en-us -vga vmware -device
> pci-assign,host=02:00.0,id=hostdev0,configfd=18,bus=pci.0,addr=0x3 \
> -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
>
> The NIC and one SCSI controller (slot 7) has the same IRQ. The
> performance in XP is really bad. When writing traffic to the drive,
> the NIC can't be accessed, and ping will be also timeout.
> If I let the NIC has the different IRQ number, then everything is OK.
> Is it related to INTx model for XP?
>
> We rebuild the QEMU, and add the LSI SCSI controller support. Why does
> RHEL6 removes its support? Is this controller too old? Are there any
> emulated SCSI devices to replace it?
>
> Thanks,
> flypen
>
>
> On Thu, Jun 16, 2011 at 2:42 AM, Alex Williamson
> <alex.williamson@redhat.com>  wrote:
>>
>> On Wed, 2011-06-15 at 11:31 +0200, Jan Kiszka wrote:
>>> On 2011-06-15 10:04, Jan Kiszka wrote:
>>>> On 2011-06-15 02:54, Alex Williamson wrote:
>>>>> On Tue, 2011-06-14 at 16:11 +0800, Flypen CloudMe wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I use Redhat Enterprise Linux 6, and use the KVM that is released by
>>>>>> Redhat officially. The kernel version is 2.6.32-71.el6.x86_64.
>>>>>>
>>>>>> It seems that the IRQs are conflicted after reboot. The NIC and the
>>>>>> SCSI controller have the same IRQ number. If I re-install the NIC
>>>>>> driver, the IRQ number of the NIC will be assigned another value, then
>>>>>> it can work normally. Do we have a way to let the NIC and the SCSI
>>>>>> controller have different IRQ number in VM?
>>
>> Hmm, I'm still confused here.  I went back and double checked, and as I
>> thought, we disable the LSI SCSI controller in the RHEL6 KVM.  So I'm
>> curious what this device is.  Is it an assigned SCSI controller or is
>> there another one that we forgot to disable in RHEL or is this a
>> different version of KVM?  The config file or command line would be
>> handy here.
>>
>>>>> I'll see if I can reproduce and figure anything out.  Windows XP isn't a
>>>>> guest we concentrate on, especially with device assignment.  Are you
>>>>> using an AMD or Intel host system?  Does the same thing happen if you
>>>>> run the XP guest on an IDE controller?  It would be helpful to post the
>>>>> guest configuration, command line used or libvirt xml.  Also, you might
>>>>> try latest upstream qemu-kvm to see if the problem still exists.
>>
>> I tested with an 82578DM e1000e NIC on an Intel host system, and it
>> surprisingly worked just fine on the RHEL6.0 base.  This is with a 32bit
>> Windows XP SP3 install.  The device supports MSI, but windows only seems
>> to use it with INTx.  I did have to remove the emulated rtl8139 or else
>> I couldn't even boot due to BSODs in the guest.
>>
>>>
>>> Nonsense, can't t make a difference as the PIIX3 resets the routing to
>>> disable - which device-assignment does not deal with, but that's unrelated.
>>
>> Yep, someone has to write it at some point and device assignment will
>> catch that.
>>
>>> Try assigning a different slot to the passed-through adapter and the
>>> lsi. For me it helped to put the lsi on one slot behind the
>>> auto-assigned (-device lsi,addr=5).
>>>
>>> I guess classic device assignment cannot support INTx sharing as the
>>> kernel IRQ injection path does not inform user space about the device
>>> state /wrt IRQs. Should be catch and reject this, or try to fix it up be
>>> moving the assigned device around, Alex?
>>
>> I'm pretty sure we've looked at this path in the past and KVM will do
>> the right thing with respect to shared INTx in the guest (ie. KVM will
>> continue to assert the interrupt if it's been triggered by both
>> assignment and userspace until both de-assert).  Experimenting with a
>> RHEL guest I can make an assigned e1000e and emulated e1000 share an
>> interrupt.  It works, but performance is terrible and the assigned
>> device can get into a non-working state if the emulated device is
>> removed.  Perhaps there is still a path by which interrupts can get
>> turned off for the assigned device.
>>
>> Alex
>>
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Alex Williamson]

On Sun, 2011-06-19 at 20:29 +0800, Flypen CloudMe wrote:
> Hi,
> 
> Here are the command line:
> 
> /usr/bin/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048 -smp
> 2,sockets=1,cores=2,threads=1 \
> -name winxp -uuid 23cd2751-8a30-dd34-db47-bfc8c76ccadb -nodefconfig
> -nodefaults \
> -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/winxp.monitor,server,nowait
> -mon chardev=monitor,mode=readline \
> -rtc base=localtime -boot c -device lsi,id=scsi0,bus=pci.0,addr=0x5
> -device lsi,id=scsi1,bus=pci.0,addr=0x6 \
> -device lsi,id=scsi2,bus=pci.0,addr=0x7 -device
> lsi,id=scsi3,bus=pci.0,addr=0x8 \
> -drive file=/mnt/vmdisk/winxp.disk,if=none,id=drive-ide0-0-0,boot=on,format=raw,cache=none
> \
> -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
> -drive file=/mnt/vmdisk/virtio-win-1.1.16.vfd,if=none,id=drive-fdc0-0-0,format=raw,cache=none\
> -global isa-fdc.driveA=drive-fdc0-0-0 -drive
> file=/dev/sd1,if=none,id=drive-scsi0-0-0,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 \
> -drive file=/dev/sdb,if=none,id=drive-scsi0-0-1,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1 \
> -drive file=/dev/sdc,if=none,id=drive-scsi0-0-2,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=2,drive=drive-scsi0-0-2,id=scsi0-0-2 \
> -drive file=/dev/sdd,if=none,id=drive-scsi0-0-3,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=3,drive=drive-scsi0-0-3,id=scsi0-0-3 \
> -drive file=/dev/sde,if=none,id=drive-scsi0-0-4,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=4,drive=drive-scsi0-0-4,id=scsi0-0-4 \
> -drive file=/dev/sdf,if=none,id=drive-scsi3-0-0,format=raw,cache=none \
> -device scsi-disk,bus=scsi3.0,scsi-id=0,drive=drive-scsi3-0-0,id=scsi3-0-0 \
> -drive file=/mnt/vmdisk/D/1,if=none,id=drive-scsi0-0-6,format=raw,cache=none \
> -device scsi-disk,bus=scsi0.0,scsi-id=6,drive=drive-scsi0-0-6,id=scsi0-0-6 \
> -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb \
> -vnc 0.0.0.0:0 -k en-us -vga vmware -device
> pci-assign,host=02:00.0,id=hostdev0,configfd=18,bus=pci.0,addr=0x3 \
> -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

That's a lot of SCSI controllers.  Why are you creating 4 separate lsi
SCSI controller devices, but only using 2 of them?  Can you reduce the
problem by just using 1?  If so, then you might be able to move the
assigned device and lsi device addr around so the guest will use
different INTx interrupts for these (or at least move them until the
assigned device gets an interrupt in the guest exclusively).  Is the
guest Windows XP 32bit or 64bit?  A 64bit Windows is probably more
likely to enable MSI interrupts (which hopefully your assigned device
supports), which would also eliminate INTx sharing problems.

> The NIC and one SCSI controller (slot 7) has the same IRQ. The
> performance in XP is really bad. When writing traffic to the drive,
> the NIC can't be accessed, and ping will be also timeout.
> If I let the NIC has the different IRQ number, then everything is OK.
> Is it related to INTx model for XP?

Maybe so.  Most of the guest/device combinations we test for device
assignment make use of MSI/X interrupts, which are more efficient, and
avoid these sorts of problems.

> We rebuild the QEMU, and add the LSI SCSI controller support. Why does
> RHEL6 removes its support? Is this controller too old? Are there any
> emulated SCSI devices to replace it?

We remove it because it's not well used or tested and we don't want to
support it.  Virtio-blk is the alternative we'd typically recommend for
guests with supported drivers.  Thanks,

Alex

Re: Why doesn't Intel e1000 NIC work correctly in Windows XP?

[Jan Kiszka]

On 2011-06-20 16:32, Alex Williamson wrote:
> On Sun, 2011-06-19 at 20:29 +0800, Flypen CloudMe wrote:
>> Hi,
>>
>> Here are the command line:
>>
>> /usr/bin/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048 -smp
>> 2,sockets=1,cores=2,threads=1 \
>> -name winxp -uuid 23cd2751-8a30-dd34-db47-bfc8c76ccadb -nodefconfig
>> -nodefaults \
>> -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/winxp.monitor,server,nowait
>> -mon chardev=monitor,mode=readline \
>> -rtc base=localtime -boot c -device lsi,id=scsi0,bus=pci.0,addr=0x5
>> -device lsi,id=scsi1,bus=pci.0,addr=0x6 \
>> -device lsi,id=scsi2,bus=pci.0,addr=0x7 -device
>> lsi,id=scsi3,bus=pci.0,addr=0x8 \
>> -drive file=/mnt/vmdisk/winxp.disk,if=none,id=drive-ide0-0-0,boot=on,format=raw,cache=none
>> \
>> -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
>> -drive file=/mnt/vmdisk/virtio-win-1.1.16.vfd,if=none,id=drive-fdc0-0-0,format=raw,cache=none\
>> -global isa-fdc.driveA=drive-fdc0-0-0 -drive
>> file=/dev/sd1,if=none,id=drive-scsi0-0-0,format=raw,cache=none \
>> -device scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0 \
>> -drive file=/dev/sdb,if=none,id=drive-scsi0-0-1,format=raw,cache=none \
>> -device scsi-disk,bus=scsi0.0,scsi-id=1,drive=drive-scsi0-0-1,id=scsi0-0-1 \
>> -drive file=/dev/sdc,if=none,id=drive-scsi0-0-2,format=raw,cache=none \
>> -device scsi-disk,bus=scsi0.0,scsi-id=2,drive=drive-scsi0-0-2,id=scsi0-0-2 \
>> -drive file=/dev/sdd,if=none,id=drive-scsi0-0-3,format=raw,cache=none \
>> -device scsi-disk,bus=scsi0.0,scsi-id=3,drive=drive-scsi0-0-3,id=scsi0-0-3 \
>> -drive file=/dev/sde,if=none,id=drive-scsi0-0-4,format=raw,cache=none \
>> -device scsi-disk,bus=scsi0.0,scsi-id=4,drive=drive-scsi0-0-4,id=scsi0-0-4 \
>> -drive file=/dev/sdf,if=none,id=drive-scsi3-0-0,format=raw,cache=none \
>> -device scsi-disk,bus=scsi3.0,scsi-id=0,drive=drive-scsi3-0-0,id=scsi3-0-0 \
>> -drive file=/mnt/vmdisk/D/1,if=none,id=drive-scsi0-0-6,format=raw,cache=none \
>> -device scsi-disk,bus=scsi0.0,scsi-id=6,drive=drive-scsi0-0-6,id=scsi0-0-6 \
>> -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb \
>> -vnc 0.0.0.0:0 -k en-us -vga vmware -device
>> pci-assign,host=02:00.0,id=hostdev0,configfd=18,bus=pci.0,addr=0x3 \
>> -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
> 
> That's a lot of SCSI controllers.  Why are you creating 4 separate lsi
> SCSI controller devices, but only using 2 of them?  Can you reduce the
> problem by just using 1?  If so, then you might be able to move the
> assigned device and lsi device addr around so the guest will use
> different INTx interrupts for these (or at least move them until the
> assigned device gets an interrupt in the guest exclusively).  Is the
> guest Windows XP 32bit or 64bit?  A 64bit Windows is probably more
> likely to enable MSI interrupts (which hopefully your assigned device
> supports), which would also eliminate INTx sharing problems.
> 

I tend to believe there is some problem with the IRQ routing information
provided to the BIOS or what the BIOS makes out of it. See how "info
pci" looks like on a "qemu-syste-x86_64 -device e1000 -device e1000" VM
after the BIOS is done:

[...]
  Bus  0, device   3, function 0:
    Ethernet controller: PCI device 8086:100e
      IRQ 11.
      BAR0: 32 bit memory at 0xf2020000 [0xf203ffff].
      BAR1: I/O at 0xc040 [0xc07f].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
      id ""
  Bus  0, device   4, function 0:
    Ethernet controller: PCI device 8086:100e
      IRQ 11.
      BAR0: 32 bit memory at 0xf2060000 [0xf207ffff].
      BAR1: I/O at 0xc080 [0xc0bf].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
      id ""
  Bus  0, device   5, function 0:
    Ethernet controller: PCI device 8086:100e
      IRQ 10.
      BAR0: 32 bit memory at 0xf20a0000 [0xf20bffff].
      BAR1: I/O at 0xc0c0 [0xc0ff].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
      id ""

Slot 3 & 4 on IRQ 11, but slot 5 on 10? That confuses Windows XP here -
at least until you reboot it after the device installation.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux