KVM x86_64 with SR-IOV..?

KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

Greetings KVM folks,

I wondering if any information exists for doing SR-IOV on the new VT-d
capable chipsets with KVM..?  From what I understand the patches for
doing this with KVM are floating around, but I have been unable to find
any user-level docs for actually making it all go against a upstream
v2.6.30-rc3 code..

So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre, and I
am really hoping to be able to jump to KVM for single-function and and
then multi-function SR-IOV.  I know that the VM migration stuff for IOV
in Xen is up and running,  and I assume it is being worked in for KVM
instance migration as well..?  This part is less important (at least for
me :-) than getting a stable SR-IOV setup running under the KVM
hypervisor..  Does anyone have any pointers for this..?

Any comments or suggestions are appreciated!

--nab

Re: KVM x86_64 with SR-IOV..?

[Sheng Yang]

On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> Greetings KVM folks,
> 
> I wondering if any information exists for doing SR-IOV on the new VT-d
> capable chipsets with KVM..?  From what I understand the patches for
> doing this with KVM are floating around, but I have been unable to find
> any user-level docs for actually making it all go against a upstream
> v2.6.30-rc3 code..
> 
> So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre, and I
> am really hoping to be able to jump to KVM for single-function and and
> then multi-function SR-IOV.  I know that the VM migration stuff for IOV
> in Xen is up and running,  and I assume it is being worked in for KVM
> instance migration as well..?  This part is less important (at least for
> me :-) than getting a stable SR-IOV setup running under the KVM
> hypervisor..  Does anyone have any pointers for this..?
> 
> Any comments or suggestions are appreciated!
> 

Hi Nicholas

The patches are not floating around now. As you know, SR-IOV for Linux have
been in 2.6.30, so then you can use upstream KVM and qemu-kvm(or recent
released kvm-85) with 2.6.30-rc3 as host kernel. And some time ago, there are
several SRIOV related patches for qemu-kvm, and now they all have been checked
in.

And for KVM, the extra document is not necessary, for you can simple assign a
VF to guest like any other devices. And how to create VF is specific for each
device driver. So just create a VF then assign it to KVM guest is fine.

-- 
regards
Yang, Sheng

Re: KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> > Greetings KVM folks,
> > 
> > I wondering if any information exists for doing SR-IOV on the new VT-d
> > capable chipsets with KVM..?  From what I understand the patches for
> > doing this with KVM are floating around, but I have been unable to find
> > any user-level docs for actually making it all go against a upstream
> > v2.6.30-rc3 code..
> > 
> > So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre, and I
> > am really hoping to be able to jump to KVM for single-function and and
> > then multi-function SR-IOV.  I know that the VM migration stuff for IOV
> > in Xen is up and running,  and I assume it is being worked in for KVM
> > instance migration as well..?  This part is less important (at least for
> > me :-) than getting a stable SR-IOV setup running under the KVM
> > hypervisor..  Does anyone have any pointers for this..?
> > 
> > Any comments or suggestions are appreciated!
> > 
> 
> Hi Nicholas
> 
> The patches are not floating around now. As you know, SR-IOV for Linux have
> been in 2.6.30, so then you can use upstream KVM and qemu-kvm(or recent
> released kvm-85) with 2.6.30-rc3 as host kernel. And some time ago, there are
> several SRIOV related patches for qemu-kvm, and now they all have been checked
> in.
> 
> And for KVM, the extra document is not necessary, for you can simple assign a
> VF to guest like any other devices. And how to create VF is specific for each
> device driver. So just create a VF then assign it to KVM guest is fine.
> 

Greetings Sheng,

So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
tell, the KVM_CAP_* defines from libkvm are enabled with building kvm-85
after './configure --kerneldir=/usr/src/linux-2.6.git' and the PCI
passthrough code is being enabled in kvm-85/qemu/hw/device-assignment.c
AFAICT..

>From there, I use the freshly installed qemu-x86_64-system binary to
start a Debian 5 x86_64 HVM (that previously had been moving network
packets under Xen for PCIe passthrough). I see the MSI-X interrupt
remapping working on the KVM host for the passed -pcidevice, and the
MMIO mappings from the qemu build that I also saw while using
Xen/qemu-dm built with PCI passthrough are there as well..

But while the KVM guest is booting, I see the following exception(s)
from qemu-x86_64-system for one of the VFs for a multi-function PCIe device:

BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)

I try with one of the on-board e1000e ports (02:00.0) and I see the same
exception along with some MSI-X exceptions from qemu-x86_64-system in
KVM guest.. However, I am still able to see the e1000e and the other
vxge multi-function device with lspci, but I am unable to dhcp or ping
with the e1000e and VF from multi-function device fails to register the
MSI-X interrupt in the guest..

Soooo, I enabled the debugging code in kvm-85/qemu/hw/device-assignment.c
and see the PAGE aligned MMIO memory for the passed PCIe device is being
released during the BUG exceptions above..  Is there something else I should
be looking at..?  I have pci-stub enabled, and I unbind 02:00.0
from /sys/bus/pci/drivers/e1000e/unbind successfully (just like with Xen
and pciback), but I am unable to do the 'echo -n 02:00.0
> /sys/bus/pci/drivers/pci-stub/bind' (it returns write error, no such
device, with no dmesg output) on the KVM host running v2.6.30-rc3.  Is
this supposed to happen on v2.6.30-rc3 with pci-stub..?  I am also using
the the kvm-85 source dist kvm_intel.ko and kvm.ko kernel modules.  Is
there something I am missing when building kvm-85 for SR-IOV passthrough..?

Also FYI, I am having to use pci=resource_alignment=
because the BIOS does not PAGE_SIZE align the MMIO BARs for my multi-function
devices..

Also, I tried with disabling the DMAR with the Intel IOMMU passthrough
from this patch:

https://lists.linux-foundation.org/pipermail/iommu/2009-April/001339.html

that did not make it into v2.6.30-rc3.  The patch logic was enabled but
still I saw the same kvm exceptions from qemu-system-x86_64.

Anyways, I am going to give it a shot with the Fedora 11 x86_64 Preview
and see if it works as expected with a IOH-5520 chipset with the AMI
BIOS on a Tyan S7010 with Xeon 5520s.  Hopefully this is just a kvm-85 build
and/or install issue I am seeing on my CentOS 5u3 install (that has a Xen PCIe
passthrough setup on it as well) with v2.6.30-rc3.  I will try on a fresh install
on a distro with the new KVM logic and see what happens.

:-)

Thanks for your comments!

--nab

Re: KVM x86_64 with SR-IOV..?

[Sheng Yang]

On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> > > Greetings KVM folks,
> > >
> > > I wondering if any information exists for doing SR-IOV on the new VT-d
> > > capable chipsets with KVM..?  From what I understand the patches for
> > > doing this with KVM are floating around, but I have been unable to find
> > > any user-level docs for actually making it all go against a upstream
> > > v2.6.30-rc3 code..
> > >
> > > So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre, and I
> > > am really hoping to be able to jump to KVM for single-function and and
> > > then multi-function SR-IOV.  I know that the VM migration stuff for IOV
> > > in Xen is up and running,  and I assume it is being worked in for KVM
> > > instance migration as well..?  This part is less important (at least
> > > for me :-) than getting a stable SR-IOV setup running under the KVM
> > > hypervisor..  Does anyone have any pointers for this..?
> > >
> > > Any comments or suggestions are appreciated!
> >
> > Hi Nicholas
> >
> > The patches are not floating around now. As you know, SR-IOV for Linux
> > have been in 2.6.30, so then you can use upstream KVM and qemu-kvm(or
> > recent released kvm-85) with 2.6.30-rc3 as host kernel. And some time
> > ago, there are several SRIOV related patches for qemu-kvm, and now they
> > all have been checked in.
> >
> > And for KVM, the extra document is not necessary, for you can simple
> > assign a VF to guest like any other devices. And how to create VF is
> > specific for each device driver. So just create a VF then assign it to
> > KVM guest is fine.
>
> Greetings Sheng,
>
> So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
> tell, the KVM_CAP_* defines from libkvm are enabled with building kvm-85
> after './configure --kerneldir=/usr/src/linux-2.6.git' and the PCI
> passthrough code is being enabled in kvm-85/qemu/hw/device-assignment.c
> AFAICT..
>
> >From there, I use the freshly installed qemu-x86_64-system binary to
>
> start a Debian 5 x86_64 HVM (that previously had been moving network
> packets under Xen for PCIe passthrough). I see the MSI-X interrupt
> remapping working on the KVM host for the passed -pcidevice, and the
> MMIO mappings from the qemu build that I also saw while using
> Xen/qemu-dm built with PCI passthrough are there as well..
>

Hi Nicholas

> But while the KVM guest is booting, I see the following exception(s)
> from qemu-x86_64-system for one of the VFs for a multi-function PCIe
> device:
>
> BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)

This one is mostly harmless.
>
> I try with one of the on-board e1000e ports (02:00.0) and I see the same
> exception along with some MSI-X exceptions from qemu-x86_64-system in
> KVM guest.. However, I am still able to see the e1000e and the other
> vxge multi-function device with lspci, but I am unable to dhcp or ping
> with the e1000e and VF from multi-function device fails to register the
> MSI-X interrupt in the guest..

Did you see the interrupt in the guest and host side? I think you can try on-
board e1000e for MSI-X first. And please ensure correlated driver have been 
loaded correctly. And what do you mean by "some MSI-X exceptions"? Better with 
the log.
>
> Soooo, I enabled the debugging code in kvm-85/qemu/hw/device-assignment.c
> and see the PAGE aligned MMIO memory for the passed PCIe device is being
> released during the BUG exceptions above..  Is there something else I
> should be looking at..?  

That part of memory should be released for trap MMIO for MSI-X table.

> I have pci-stub enabled, and I unbind 02:00.0
> from /sys/bus/pci/drivers/e1000e/unbind successfully (just like with Xen
> and pciback), but I am unable to do the 'echo -n 02:00.0
>
> > /sys/bus/pci/drivers/pci-stub/bind' (it returns write error, no such
>
> device, with no dmesg output) on the KVM host running v2.6.30-rc3.  Is
> this supposed to happen on v2.6.30-rc3 with pci-stub..?  

Maybe you need "echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind"? 

> I am also using
> the the kvm-85 source dist kvm_intel.ko and kvm.ko kernel modules.  Is
> there something I am missing when building kvm-85 for SR-IOV passthrough..?

I think the first thing is to confirm that device assignment work in your 
environment, using on-board card. You can also refer to 
http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM

And you can post debug_device_assignment=1 log and qemu log and the tail of 
dmesg as well.

Thanks!

-- 
regards
Yang, Sheng

>
> Also FYI, I am having to use pci=resource_alignment=
> because the BIOS does not PAGE_SIZE align the MMIO BARs for my
> multi-function devices..
>
> Also, I tried with disabling the DMAR with the Intel IOMMU passthrough
> from this patch:
>
> https://lists.linux-foundation.org/pipermail/iommu/2009-April/001339.html
>
> that did not make it into v2.6.30-rc3.  The patch logic was enabled but
> still I saw the same kvm exceptions from qemu-system-x86_64.
>
> Anyways, I am going to give it a shot with the Fedora 11 x86_64 Preview
> and see if it works as expected with a IOH-5520 chipset with the AMI
> BIOS on a Tyan S7010 with Xeon 5520s.  Hopefully this is just a kvm-85
> build and/or install issue I am seeing on my CentOS 5u3 install (that has a
> Xen PCIe passthrough setup on it as well) with v2.6.30-rc3.  I will try on
> a fresh install on a distro with the new KVM logic and see what happens.
>
> :-)
>
> Thanks for your comments!
>
> --nab

Re: KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> > > > Greetings KVM folks,
> > > >
> > > > I wondering if any information exists for doing SR-IOV on the new VT-d
> > > > capable chipsets with KVM..?  From what I understand the patches for
> > > > doing this with KVM are floating around, but I have been unable to find
> > > > any user-level docs for actually making it all go against a upstream
> > > > v2.6.30-rc3 code..
> > > >
> > > > So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre, and I
> > > > am really hoping to be able to jump to KVM for single-function and and
> > > > then multi-function SR-IOV.  I know that the VM migration stuff for IOV
> > > > in Xen is up and running,  and I assume it is being worked in for KVM
> > > > instance migration as well..?  This part is less important (at least
> > > > for me :-) than getting a stable SR-IOV setup running under the KVM
> > > > hypervisor..  Does anyone have any pointers for this..?
> > > >
> > > > Any comments or suggestions are appreciated!
> > >
> > > Hi Nicholas
> > >
> > > The patches are not floating around now. As you know, SR-IOV for Linux
> > > have been in 2.6.30, so then you can use upstream KVM and qemu-kvm(or
> > > recent released kvm-85) with 2.6.30-rc3 as host kernel. And some time
> > > ago, there are several SRIOV related patches for qemu-kvm, and now they
> > > all have been checked in.
> > >
> > > And for KVM, the extra document is not necessary, for you can simple
> > > assign a VF to guest like any other devices. And how to create VF is
> > > specific for each device driver. So just create a VF then assign it to
> > > KVM guest is fine.
> >
> > Greetings Sheng,
> >
> > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
> > tell, the KVM_CAP_* defines from libkvm are enabled with building kvm-85
> > after './configure --kerneldir=/usr/src/linux-2.6.git' and the PCI
> > passthrough code is being enabled in kvm-85/qemu/hw/device-assignment.c
> > AFAICT..
> >
> > >From there, I use the freshly installed qemu-x86_64-system binary to
> >
> > start a Debian 5 x86_64 HVM (that previously had been moving network
> > packets under Xen for PCIe passthrough). I see the MSI-X interrupt
> > remapping working on the KVM host for the passed -pcidevice, and the
> > MMIO mappings from the qemu build that I also saw while using
> > Xen/qemu-dm built with PCI passthrough are there as well..
> >
> 
> Hi Nicholas
> 
> > But while the KVM guest is booting, I see the following exception(s)
> > from qemu-x86_64-system for one of the VFs for a multi-function PCIe
> > device:
> >
> > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> 
> This one is mostly harmless.
> >

Ok, good to know..  :-)

> > I try with one of the on-board e1000e ports (02:00.0) and I see the same
> > exception along with some MSI-X exceptions from qemu-x86_64-system in
> > KVM guest.. However, I am still able to see the e1000e and the other
> > vxge multi-function device with lspci, but I am unable to dhcp or ping
> > with the e1000e and VF from multi-function device fails to register the
> > MSI-X interrupt in the guest..
> 
> Did you see the interrupt in the guest and host side?

Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
attaching the e1000e single-function device at 02:00.0 to pci-stub with:

   echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
   echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
   echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind 

I see the following the KVM host kernel ring buffer:

   e1000e 0000:02:00.0: PCI INT A disabled
   pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
   pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X

>  I think you can try on-
> board e1000e for MSI-X first. And please ensure correlated driver have been 
> loaded correctly.

<nod>..

>  And what do you mean by "some MSI-X exceptions"? Better with 
> the log.

Ok, with the Fedora 11 installed qemu-kemu, I see the expected
kvm_destroy_phys_mem() statements:

#kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0 lenny64guest1-orig.img 
BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)

However I still see the following in the KVM guest kernel ring buffer
running v2.6.30-rc in the HVM guest.

[    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
[    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
[    5.525710] e1000e 0000:00:05.0: setting latency timer to 64
[    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X interrupts.  Falling back to MSI interrupts.
[    5.527200] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts.  Falling back to legacy interrupts.
[    5.829988] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
[    5.830672] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
[    5.831240] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff

While doing dhcp, the e1000e throws a netdev watchdog transmit timeout..

Here is what lspci -v -s 00:05.0 looks like:

00:05.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
        Subsystem: Intel Corporation Device 0000
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 10
        Region 0: Memory at f2020000 (32-bit, non-prefetchable) [size=128K]
        Region 2: I/O ports at c220 [size=32]
        Region 3: Memory at f2040000 (32-bit, non-prefetchable) [size=16K]
        Kernel driver in use: e1000e
        Kernel modules: e1000e

I am going to double check my v2.6.30-rc3 KVM guest kernel build for the
PCI options.  Is there anything special I need to enable other than PCI
express support in the v2.6.30-rc3 guest under the PCI Bus options in
the kernel config..?  DMAR and Interrupt Remapping should be DISABLED in
the guest HVM kernels, right..?

Also just an observation, I noticed that in Xen HVM with SR-IOV
passthrough the PCIe devices appear as 05:00.0 after an 'xm pci-attach'
call.  Is there a reason that SR-IOV with KVM attaches said passthrough
devices under the 00.* PCI bus instead of it's own $NEXT_BUS_ID.00.0
value under the KVM guest.?  Does this have any effects on
functionality..?

Many thanks for your most valuable of time,

--nab

> >
> > Soooo, I enabled the debugging code in kvm-85/qemu/hw/device-assignment.c
> > and see the PAGE aligned MMIO memory for the passed PCIe device is being
> > released during the BUG exceptions above..  Is there something else I
> > should be looking at..?  
> 
> That part of memory should be released for trap MMIO for MSI-X table.
> 
> > I have pci-stub enabled, and I unbind 02:00.0
> > from /sys/bus/pci/drivers/e1000e/unbind successfully (just like with Xen
> > and pciback), but I am unable to do the 'echo -n 02:00.0
> >
> > > /sys/bus/pci/drivers/pci-stub/bind' (it returns write error, no such
> >
> > device, with no dmesg output) on the KVM host running v2.6.30-rc3.  Is
> > this supposed to happen on v2.6.30-rc3 with pci-stub..?  
> 
> Maybe you need "echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind"? 
> 
> > I am also using
> > the the kvm-85 source dist kvm_intel.ko and kvm.ko kernel modules.  Is
> > there something I am missing when building kvm-85 for SR-IOV passthrough..?
> 
> I think the first thing is to confirm that device assignment work in your 
> environment, using on-board card. You can also refer to 
> http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
> 
> And you can post debug_device_assignment=1 log and qemu log and the tail of 
> dmesg as well.
> 
> Thanks!
> 

Re: KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

On Sun, 2009-05-03 at 21:36 -0700, Nicholas A. Bellinger wrote:
> On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> > > > > Greetings KVM folks,
> > > > >
> > > > > I wondering if any information exists for doing SR-IOV on the new VT-d
> > > > > capable chipsets with KVM..?  From what I understand the patches for
> > > > > doing this with KVM are floating around, but I have been unable to find
> > > > > any user-level docs for actually making it all go against a upstream
> > > > > v2.6.30-rc3 code..
> > > > >
> > > > > So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre, and I
> > > > > am really hoping to be able to jump to KVM for single-function and and
> > > > > then multi-function SR-IOV.  I know that the VM migration stuff for IOV
> > > > > in Xen is up and running,  and I assume it is being worked in for KVM
> > > > > instance migration as well..?  This part is less important (at least
> > > > > for me :-) than getting a stable SR-IOV setup running under the KVM
> > > > > hypervisor..  Does anyone have any pointers for this..?
> > > > >
> > > > > Any comments or suggestions are appreciated!
> > > >
> > > > Hi Nicholas
> > > >
> > > > The patches are not floating around now. As you know, SR-IOV for Linux
> > > > have been in 2.6.30, so then you can use upstream KVM and qemu-kvm(or
> > > > recent released kvm-85) with 2.6.30-rc3 as host kernel. And some time
> > > > ago, there are several SRIOV related patches for qemu-kvm, and now they
> > > > all have been checked in.
> > > >
> > > > And for KVM, the extra document is not necessary, for you can simple
> > > > assign a VF to guest like any other devices. And how to create VF is
> > > > specific for each device driver. So just create a VF then assign it to
> > > > KVM guest is fine.
> > >
> > > Greetings Sheng,
> > >
> > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
> > > tell, the KVM_CAP_* defines from libkvm are enabled with building kvm-85
> > > after './configure --kerneldir=/usr/src/linux-2.6.git' and the PCI
> > > passthrough code is being enabled in kvm-85/qemu/hw/device-assignment.c
> > > AFAICT..
> > >
> > > >From there, I use the freshly installed qemu-x86_64-system binary to
> > >
> > > start a Debian 5 x86_64 HVM (that previously had been moving network
> > > packets under Xen for PCIe passthrough). I see the MSI-X interrupt
> > > remapping working on the KVM host for the passed -pcidevice, and the
> > > MMIO mappings from the qemu build that I also saw while using
> > > Xen/qemu-dm built with PCI passthrough are there as well..
> > >
> > 
> > Hi Nicholas
> > 
> > > But while the KVM guest is booting, I see the following exception(s)
> > > from qemu-x86_64-system for one of the VFs for a multi-function PCIe
> > > device:
> > >
> > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > 
> > This one is mostly harmless.
> > >
> 
> Ok, good to know..  :-)
> 
> > > I try with one of the on-board e1000e ports (02:00.0) and I see the same
> > > exception along with some MSI-X exceptions from qemu-x86_64-system in
> > > KVM guest.. However, I am still able to see the e1000e and the other
> > > vxge multi-function device with lspci, but I am unable to dhcp or ping
> > > with the e1000e and VF from multi-function device fails to register the
> > > MSI-X interrupt in the guest..
> > 
> > Did you see the interrupt in the guest and host side?
> 
> Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> attaching the e1000e single-function device at 02:00.0 to pci-stub with:
> 
>    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
>    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
>    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind 
> 
> I see the following the KVM host kernel ring buffer:
> 
>    e1000e 0000:02:00.0: PCI INT A disabled
>    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
>    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> 
> >  I think you can try on-
> > board e1000e for MSI-X first. And please ensure correlated driver have been 
> > loaded correctly.
> 
> <nod>..
> 
> >  And what do you mean by "some MSI-X exceptions"? Better with 
> > the log.
> 
> Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> kvm_destroy_phys_mem() statements:
> 
> #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0 lenny64guest1-orig.img 
> BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> 
> However I still see the following in the KVM guest kernel ring buffer
> running v2.6.30-rc in the HVM guest.
> 
> [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
> [    5.525710] e1000e 0000:00:05.0: setting latency timer to 64
> [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X interrupts.  Falling back to MSI interrupts.
> [    5.527200] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts.  Falling back to legacy interrupts.
> [    5.829988] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
> [    5.830672] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> [    5.831240] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> 
> While doing dhcp, the e1000e throws a netdev watchdog transmit timeout..
> 
> Here is what lspci -v -s 00:05.0 looks like:
> 
> 00:05.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
>         Subsystem: Intel Corporation Device 0000
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 0, Cache Line Size: 64 bytes
>         Interrupt: pin A routed to IRQ 10
>         Region 0: Memory at f2020000 (32-bit, non-prefetchable) [size=128K]
>         Region 2: I/O ports at c220 [size=32]
>         Region 3: Memory at f2040000 (32-bit, non-prefetchable) [size=16K]
>         Kernel driver in use: e1000e
>         Kernel modules: e1000e
> 

Hi Sheng,

Btw, this is what it looks like in KVM HVM guest running v2.6.30-rc3 after plugging
in the port and dhcp occuring..  The KVM HVM does not hard lock (cool :-), and I am
still able to access via the built-in qemu net-device.  Here are my .config options
for the v2.6.30-rc3 KVM guest running on top of 2.6.26.6-79.fc9.x86_64 Fedora 11
Preview KVM Host.  I am missing something in the v2.6.30-rc3 KVM guest config for
accessing an e1000e port SR-IOV below..?

Many thanks for your most valuable of time,

--nab

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
# CONFIG_DMAR is not set
# CONFIG_INTR_REMAP is not set
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=m
CONFIG_PCIEAER=y
# CONFIG_PCIEASPM is not set
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_LEGACY=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not set
# CONFIG_HT_IRQ is not set
# CONFIG_PCI_IOV is not set
CONFIG_ISA_DMA_API=y
CONFIG_K8_NB=y
# CONFIG_PCCARD is not set
CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_ACPI=m
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
CONFIG_HOTPLUG_PCI_CPCI=y
CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
CONFIG_HOTPLUG_PCI_SHPC=m


[   17.476125] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
[   19.969922] e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   30.250140] NET: Registered protocol family 10
[   30.251561] lo: Disabled Privacy Extensions
[   32.942145] 0000:00:05.0: eth1: Detected Tx Unit Hang:
[   32.942147]   TDH                  <1>
[   32.942148]   TDT                  <4>
[   32.942149]   next_to_use          <4>
[   32.942149]   next_to_clean        <0>
[   32.942150] buffer_info[next_to_clean]:
[   32.942151]   time_stamp           <fffef895>
[   32.942152]   next_to_watch        <0>
[   32.942153]   jiffies              <fffefb33>
[   32.942154]   next_to_watch.status <0>
[   34.804645] 0000:00:05.0: eth1: Detected Tx Unit Hang:
[   34.804647]   TDH                  <1>
[   34.804648]   TDT                  <4>
[   34.804649]   next_to_use          <4>
[   34.804650]   next_to_clean        <0>
[   34.804651] buffer_info[next_to_clean]:
[   34.804652]   time_stamp           <fffef895>
[   34.804653]   next_to_watch        <0>
[   34.804654]   jiffies              <fffefd05>
[   34.804655]   next_to_watch.status <0>
[   36.804621] 0000:00:05.0: eth1: Detected Tx Unit Hang:
[   36.804623]   TDH                  <1>
[   36.804624]   TDT                  <4>
[   36.804625]   next_to_use          <4>
[   36.804625]   next_to_clean        <0>
[   36.804626] buffer_info[next_to_clean]:
[   36.804627]   time_stamp           <fffef895>
[   36.804628]   next_to_watch        <0>
[   36.804629]   jiffies              <fffefef9>
[   36.804630]   next_to_watch.status <0>
[   38.804577] 0000:00:05.0: eth1: Detected Tx Unit Hang:
[   38.804579]   TDH                  <1>
[   38.804580]   TDT                  <4>
[   38.804581]   next_to_use          <4>
[   38.804591]   next_to_clean        <0>
[   38.804592] buffer_info[next_to_clean]:
[   38.804593]   time_stamp           <fffef895>
[   38.804594]   next_to_watch        <0>
[   38.804595]   jiffies              <ffff00ed>
[   38.804596]   next_to_watch.status <0>
[   39.804214] ------------[ cut here ]------------
[   39.804827] WARNING: at net/sched/sch_generic.c:226 dev_watchdog+0x11b/0x1bd()
[   39.805820] Hardware name:
[   39.806356] NETDEV WATCHDOG: eth1 (e1000e): transmit timed out
[   39.807003] Modules linked in: ipv6 loop serio_raw virtio_balloon pcspkr psmouse parport_pc button parport i2c_piix4 i2c_core processor evdev ext3 jbd mbcache ide_cd_mod cdrom ide_gd_mod ata_piix ata_generic libata scsi_mod virtio_pci virtio_ring virtio piix 8139cp ide_pci_generic 8139too e1000e ide_core mii floppy thermal fan thermal_sys
[   39.816257] Pid: 0, comm: swapper Not tainted 2.6.30-rc3 #7
[   39.816911] Call Trace:
[   39.817458]  <IRQ>  [<ffffffff80238caa>] ? warn_slowpath+0xd8/0x10a
[   39.818392]  [<ffffffff80343900>] ? cpumask_any_but+0x28/0x34
[   39.819036]  [<ffffffff80231aa1>] ? find_busiest_group+0x2dc/0x942
[   39.819697]  [<ffffffff8022d661>] ? enqueue_task_fair+0x24/0x6a
[   39.820436]  [<ffffffff8022aec9>] ? enqueue_task+0x5c/0x65
[   39.821118]  [<ffffffff8022aec9>] ? enqueue_task+0x5c/0x65
[   39.821762]  [<ffffffff8022afb9>] ? activate_task+0x20/0x26
[   39.822321]  [<ffffffff8023273b>] ? try_to_wake_up+0x212/0x224
[   39.822792]  [<ffffffff8024af2f>] ? autoremove_wake_function+0x9/0x2e
[   39.823273]  [<ffffffff80411a55>] ? dev_watchdog+0x11b/0x1bd
[   39.823723]  [<ffffffff8022bffa>] ? __wake_up+0x30/0x44
[   40.033557]  [<ffffffff8041193a>] ? dev_watchdog+0x0/0x1bd
[   40.034090]  [<ffffffff80241214>] ? run_timer_softirq+0x18c/0x202
[   40.034794]  [<ffffffff80251b82>] ? getnstimeofday+0x59/0xb3
[   40.035449]  [<ffffffff8023d772>] ? __do_softirq+0xa6/0x168
[   40.036174]  [<ffffffff8020ca7c>] ? call_softirq+0x1c/0x28
[   40.036861]  [<ffffffff8020e254>] ? do_softirq+0x2c/0x6c
[   40.037495]  [<ffffffff8023d474>] ? irq_exit+0x3f/0x7c
[   40.038125]  [<ffffffff8021be42>] ? smp_apic_timer_interrupt+0x87/0x94
[   40.038795]  [<ffffffff8020c493>] ? apic_timer_interrupt+0x13/0x20
[   40.039462]  <EOI>  [<ffffffff8021235c>] ? default_idle+0x5b/0x99
[   40.040370]  [<ffffffff8024e461>] ? notifier_call_chain+0x29/0x4c
[   40.041105]  [<ffffffff8020ad55>] ? cpu_idle+0x4a/0x8b
[   40.041723] ---[ end trace dc792b53566c049e ]---
[   40.484820] eth0: no IPv6 routers present
[   40.712073] eth1: no IPv6 routers present
[   43.489776] e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX

Re: KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

On Sun, 2009-05-03 at 22:28 -0700, Nicholas A. Bellinger wrote:
> On Sun, 2009-05-03 at 21:36 -0700, Nicholas A. Bellinger wrote:
> > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> > > > > > Greetings KVM folks,
> > > > > >
> > > > > > I wondering if any information exists for doing SR-IOV on the new VT-d
> > > > > > capable chipsets with KVM..?  From what I understand the patches for
> > > > > > doing this with KVM are floating around, but I have been unable to find
> > > > > > any user-level docs for actually making it all go against a upstream
> > > > > > v2.6.30-rc3 code..
> > > > > >
> > > > > > So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre, and I
> > > > > > am really hoping to be able to jump to KVM for single-function and and
> > > > > > then multi-function SR-IOV.  I know that the VM migration stuff for IOV
> > > > > > in Xen is up and running,  and I assume it is being worked in for KVM
> > > > > > instance migration as well..?  This part is less important (at least
> > > > > > for me :-) than getting a stable SR-IOV setup running under the KVM
> > > > > > hypervisor..  Does anyone have any pointers for this..?
> > > > > >
> > > > > > Any comments or suggestions are appreciated!
> > > > >
> > > > > Hi Nicholas
> > > > >
> > > > > The patches are not floating around now. As you know, SR-IOV for Linux
> > > > > have been in 2.6.30, so then you can use upstream KVM and qemu-kvm(or
> > > > > recent released kvm-85) with 2.6.30-rc3 as host kernel. And some time
> > > > > ago, there are several SRIOV related patches for qemu-kvm, and now they
> > > > > all have been checked in.
> > > > >
> > > > > And for KVM, the extra document is not necessary, for you can simple
> > > > > assign a VF to guest like any other devices. And how to create VF is
> > > > > specific for each device driver. So just create a VF then assign it to
> > > > > KVM guest is fine.
> > > >
> > > > Greetings Sheng,
> > > >
> > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
> > > > tell, the KVM_CAP_* defines from libkvm are enabled with building kvm-85
> > > > after './configure --kerneldir=/usr/src/linux-2.6.git' and the PCI
> > > > passthrough code is being enabled in kvm-85/qemu/hw/device-assignment.c
> > > > AFAICT..
> > > >
> > > > >From there, I use the freshly installed qemu-x86_64-system binary to
> > > >
> > > > start a Debian 5 x86_64 HVM (that previously had been moving network
> > > > packets under Xen for PCIe passthrough). I see the MSI-X interrupt
> > > > remapping working on the KVM host for the passed -pcidevice, and the
> > > > MMIO mappings from the qemu build that I also saw while using
> > > > Xen/qemu-dm built with PCI passthrough are there as well..
> > > >
> > > 
> > > Hi Nicholas
> > > 
> > > > But while the KVM guest is booting, I see the following exception(s)
> > > > from qemu-x86_64-system for one of the VFs for a multi-function PCIe
> > > > device:
> > > >
> > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > 
> > > This one is mostly harmless.
> > > >
> > 
> > Ok, good to know..  :-)
> > 
> > > > I try with one of the on-board e1000e ports (02:00.0) and I see the same
> > > > exception along with some MSI-X exceptions from qemu-x86_64-system in
> > > > KVM guest.. However, I am still able to see the e1000e and the other
> > > > vxge multi-function device with lspci, but I am unable to dhcp or ping
> > > > with the e1000e and VF from multi-function device fails to register the
> > > > MSI-X interrupt in the guest..
> > > 
> > > Did you see the interrupt in the guest and host side?
> > 
> > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > attaching the e1000e single-function device at 02:00.0 to pci-stub with:
> > 
> >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind 
> > 
> > I see the following the KVM host kernel ring buffer:
> > 
> >    e1000e 0000:02:00.0: PCI INT A disabled
> >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > 
> > >  I think you can try on-
> > > board e1000e for MSI-X first. And please ensure correlated driver have been 
> > > loaded correctly.
> > 
> > <nod>..
> > 
> > >  And what do you mean by "some MSI-X exceptions"? Better with 
> > > the log.
> > 
> > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > kvm_destroy_phys_mem() statements:
> > 
> > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0 lenny64guest1-orig.img 
> > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > 
> > However I still see the following in the KVM guest kernel ring buffer
> > running v2.6.30-rc in the HVM guest.
> > 
> > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
> > [    5.525710] e1000e 0000:00:05.0: setting latency timer to 64
> > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X interrupts.  Falling back to MSI interrupts.
> > [    5.527200] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts.  Falling back to legacy interrupts.
> > [    5.829988] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
> > [    5.830672] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> > [    5.831240] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > 
> > While doing dhcp, the e1000e throws a netdev watchdog transmit timeout..
> > 
> > Here is what lspci -v -s 00:05.0 looks like:
> > 
> > 00:05.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
> >         Subsystem: Intel Corporation Device 0000
> >         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
> >         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Latency: 0, Cache Line Size: 64 bytes
> >         Interrupt: pin A routed to IRQ 10
> >         Region 0: Memory at f2020000 (32-bit, non-prefetchable) [size=128K]
> >         Region 2: I/O ports at c220 [size=32]
> >         Region 3: Memory at f2040000 (32-bit, non-prefetchable) [size=16K]
> >         Kernel driver in use: e1000e
> >         Kernel modules: e1000e
> > 
> 
> Hi Sheng,
> 
> Btw, this is what it looks like in KVM HVM guest running v2.6.30-rc3 after plugging
> in the port and dhcp occuring..  The KVM HVM does not hard lock (cool :-), and I am
> still able to access via the built-in qemu net-device.  Here are my .config options
> for the v2.6.30-rc3 KVM guest running on top of 2.6.26.6-79.fc9.x86_64 Fedora 11
> Preview KVM Host. 

Whoops, that should have been 2.6.29.1-111.fc11.x86_64 for Fedora 11
Preview KVM Host kernel btw..

--nab

Re: KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

On Sun, 2009-05-03 at 22:28 -0700, Nicholas A. Bellinger wrote:
> > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > Greetings Sheng,
> > > >
> > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
> > > > tell, the KVM_CAP_* defines from libkvm are enabled with building kvm-85
> > > > after './configure --kerneldir=/usr/src/linux-2.6.git' and the PCI
> > > > passthrough code is being enabled in kvm-85/qemu/hw/device-assignment.c
> > > > AFAICT..
> > > >
> > > > >From there, I use the freshly installed qemu-x86_64-system binary to
> > > >
> > > > start a Debian 5 x86_64 HVM (that previously had been moving network
> > > > packets under Xen for PCIe passthrough). I see the MSI-X interrupt
> > > > remapping working on the KVM host for the passed -pcidevice, and the
> > > > MMIO mappings from the qemu build that I also saw while using
> > > > Xen/qemu-dm built with PCI passthrough are there as well..
> > > >
> > > 
> > > Hi Nicholas
> > > 
> > > > But while the KVM guest is booting, I see the following exception(s)
> > > > from qemu-x86_64-system for one of the VFs for a multi-function PCIe
> > > > device:
> > > >
> > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > 
> > > This one is mostly harmless.
> > > >
> > 
> > Ok, good to know..  :-)
> > 
> > > > I try with one of the on-board e1000e ports (02:00.0) and I see the same
> > > > exception along with some MSI-X exceptions from qemu-x86_64-system in
> > > > KVM guest.. However, I am still able to see the e1000e and the other
> > > > vxge multi-function device with lspci, but I am unable to dhcp or ping
> > > > with the e1000e and VF from multi-function device fails to register the
> > > > MSI-X interrupt in the guest..
> > > 
> > > Did you see the interrupt in the guest and host side?
> > 
> > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > attaching the e1000e single-function device at 02:00.0 to pci-stub with:
> > 
> >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind 
> > 
> > I see the following the KVM host kernel ring buffer:
> > 
> >    e1000e 0000:02:00.0: PCI INT A disabled
> >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > 

Ok, I also noticed the following output in /proc/interrupts on KVM host
with dual Intel E5520 processors (16 CPUs)

[root@barret ~]# cat /proc/interrupts | grep MSI
 55:         22          0          0          0          0          0          0          0          0         94     436974          0          0          0          0          0   PCI-MSI-edge      eth0-rx-0
 56:         27          0          0          0          0          0          0          0          0     613054          0          0      15253          0          0          0   PCI-MSI-edge      eth0-tx-0
 57:          3          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      eth0
 58:        521          0          0          0          5          0          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      kvm_assigned_msi_device

eth0 is the other e1000e port at 03:00.0 that is in use on the KVM host,
and it looks like the other e1000e port at 02:00.0 has been setup to
kvm_assigned_msi_device on irq 58.

I also noticed the following after starting a KVM guest in host's ring
buffer (not sure if this has anything to do with -pcidevice usage)

kvm: 3428: cpu6 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu5 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu9 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu1 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu8 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu2 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu3 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu4 unhandled wrmsr: 0xc0010117 data 0
kvm: 3428: cpu0 unhandled wrmsr: 0xc0010117 data 0


> > >  I think you can try on-
> > > board e1000e for MSI-X first. And please ensure correlated driver have been 
> > > loaded correctly.
> > 
> > <nod>..
> > 
> > >  And what do you mean by "some MSI-X exceptions"? Better with 
> > > the log.
> > 
> > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > kvm_destroy_phys_mem() statements:
> > 
> > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0 lenny64guest1-orig.img 
> > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > 
> > However I still see the following in the KVM guest kernel ring buffer
> > running v2.6.30-rc in the HVM guest.
> > 
> > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
> > [    5.525710] e1000e 0000:00:05.0: setting latency timer to 64
> > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X interrupts.  Falling back to MSI interrupts.
> > [    5.527200] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts.  Falling back to legacy interrupts.
> > [    5.829988] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
> > [    5.830672] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> > [    5.831240] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > 

>From there, I keep seeing the same MSI-X and MSI registration failures
from the e1000e driver from within the v2.6.30-rc3 KVM guest from above,
and will the exception will trigger the same netdev watchdog TX timeout
are observed.

I will try building a v2.6.29.y stable kernel in the guest HVM and see
if that makes any difference.  So far, it seems like it is a KVM guest
kernel issue with my v2.6.20-rc3 builds.  Is there anything else I
should be looking at on the FC11 KVM host to verify that everything from
the host side is working at expected..? 

Please let me know if you need any more information.

--nab

> > While doing dhcp, the e1000e throws a netdev watchdog transmit timeout..
> > 
> > Here is what lspci -v -s 00:05.0 looks like:
> > 
> > 00:05.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
> >         Subsystem: Intel Corporation Device 0000
> >         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
> >         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Latency: 0, Cache Line Size: 64 bytes
> >         Interrupt: pin A routed to IRQ 10
> >         Region 0: Memory at f2020000 (32-bit, non-prefetchable) [size=128K]
> >         Region 2: I/O ports at c220 [size=32]
> >         Region 3: Memory at f2040000 (32-bit, non-prefetchable) [size=16K]
> >         Kernel driver in use: e1000e
> >         Kernel modules: e1000e
> > 
> 
> Hi Sheng,
> 
> Btw, this is what it looks like in KVM HVM guest running v2.6.30-rc3 after plugging
> in the port and dhcp occuring..  The KVM HVM does not hard lock (cool :-), and I am
> still able to access via the built-in qemu net-device.  Here are my .config options
> for the v2.6.30-rc3 KVM guest running on top of 2.6.26.6-79.fc9.x86_64 Fedora 11
> Preview KVM Host.  I am missing something in the v2.6.30-rc3 KVM guest config for
> accessing an e1000e port SR-IOV below..?
> 
> Many thanks for your most valuable of time,
> 
> --nab
> 
> #
> # Bus options (PCI etc.)
> #
> CONFIG_PCI=y
> CONFIG_PCI_DIRECT=y
> CONFIG_PCI_MMCONFIG=y
> CONFIG_PCI_DOMAINS=y
> # CONFIG_DMAR is not set
> # CONFIG_INTR_REMAP is not set
> CONFIG_PCIEPORTBUS=y
> CONFIG_HOTPLUG_PCI_PCIE=m
> CONFIG_PCIEAER=y
> # CONFIG_PCIEASPM is not set
> CONFIG_ARCH_SUPPORTS_MSI=y
> CONFIG_PCI_MSI=y
> CONFIG_PCI_LEGACY=y
> # CONFIG_PCI_DEBUG is not set
> # CONFIG_PCI_STUB is not set
> # CONFIG_HT_IRQ is not set
> # CONFIG_PCI_IOV is not set
> CONFIG_ISA_DMA_API=y
> CONFIG_K8_NB=y
> # CONFIG_PCCARD is not set
> CONFIG_HOTPLUG_PCI=m
> CONFIG_HOTPLUG_PCI_FAKE=m
> CONFIG_HOTPLUG_PCI_ACPI=m
> CONFIG_HOTPLUG_PCI_ACPI_IBM=m
> CONFIG_HOTPLUG_PCI_CPCI=y
> CONFIG_HOTPLUG_PCI_CPCI_ZT5550=m
> CONFIG_HOTPLUG_PCI_CPCI_GENERIC=m
> CONFIG_HOTPLUG_PCI_SHPC=m
> 
> 
> [   17.476125] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
> [   19.969922] e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
> [   30.250140] NET: Registered protocol family 10
> [   30.251561] lo: Disabled Privacy Extensions
> [   32.942145] 0000:00:05.0: eth1: Detected Tx Unit Hang:
> [   32.942147]   TDH                  <1>
> [   32.942148]   TDT                  <4>
> [   32.942149]   next_to_use          <4>
> [   32.942149]   next_to_clean        <0>
> [   32.942150] buffer_info[next_to_clean]:
> [   32.942151]   time_stamp           <fffef895>
> [   32.942152]   next_to_watch        <0>
> [   32.942153]   jiffies              <fffefb33>
> [   32.942154]   next_to_watch.status <0>
> [   34.804645] 0000:00:05.0: eth1: Detected Tx Unit Hang:
> [   34.804647]   TDH                  <1>
> [   34.804648]   TDT                  <4>
> [   34.804649]   next_to_use          <4>
> [   34.804650]   next_to_clean        <0>
> [   34.804651] buffer_info[next_to_clean]:
> [   34.804652]   time_stamp           <fffef895>
> [   34.804653]   next_to_watch        <0>
> [   34.804654]   jiffies              <fffefd05>
> [   34.804655]   next_to_watch.status <0>
> [   36.804621] 0000:00:05.0: eth1: Detected Tx Unit Hang:
> [   36.804623]   TDH                  <1>
> [   36.804624]   TDT                  <4>
> [   36.804625]   next_to_use          <4>
> [   36.804625]   next_to_clean        <0>
> [   36.804626] buffer_info[next_to_clean]:
> [   36.804627]   time_stamp           <fffef895>
> [   36.804628]   next_to_watch        <0>
> [   36.804629]   jiffies              <fffefef9>
> [   36.804630]   next_to_watch.status <0>
> [   38.804577] 0000:00:05.0: eth1: Detected Tx Unit Hang:
> [   38.804579]   TDH                  <1>
> [   38.804580]   TDT                  <4>
> [   38.804581]   next_to_use          <4>
> [   38.804591]   next_to_clean        <0>
> [   38.804592] buffer_info[next_to_clean]:
> [   38.804593]   time_stamp           <fffef895>
> [   38.804594]   next_to_watch        <0>
> [   38.804595]   jiffies              <ffff00ed>
> [   38.804596]   next_to_watch.status <0>
> [   39.804214] ------------[ cut here ]------------
> [   39.804827] WARNING: at net/sched/sch_generic.c:226 dev_watchdog+0x11b/0x1bd()
> [   39.805820] Hardware name:
> [   39.806356] NETDEV WATCHDOG: eth1 (e1000e): transmit timed out
> [   39.807003] Modules linked in: ipv6 loop serio_raw virtio_balloon pcspkr psmouse parport_pc button parport i2c_piix4 i2c_core processor evdev ext3 jbd mbcache ide_cd_mod cdrom ide_gd_mod ata_piix ata_generic libata scsi_mod virtio_pci virtio_ring virtio piix 8139cp ide_pci_generic 8139too e1000e ide_core mii floppy thermal fan thermal_sys
> [   39.816257] Pid: 0, comm: swapper Not tainted 2.6.30-rc3 #7
> [   39.816911] Call Trace:
> [   39.817458]  <IRQ>  [<ffffffff80238caa>] ? warn_slowpath+0xd8/0x10a
> [   39.818392]  [<ffffffff80343900>] ? cpumask_any_but+0x28/0x34
> [   39.819036]  [<ffffffff80231aa1>] ? find_busiest_group+0x2dc/0x942
> [   39.819697]  [<ffffffff8022d661>] ? enqueue_task_fair+0x24/0x6a
> [   39.820436]  [<ffffffff8022aec9>] ? enqueue_task+0x5c/0x65
> [   39.821118]  [<ffffffff8022aec9>] ? enqueue_task+0x5c/0x65
> [   39.821762]  [<ffffffff8022afb9>] ? activate_task+0x20/0x26
> [   39.822321]  [<ffffffff8023273b>] ? try_to_wake_up+0x212/0x224
> [   39.822792]  [<ffffffff8024af2f>] ? autoremove_wake_function+0x9/0x2e
> [   39.823273]  [<ffffffff80411a55>] ? dev_watchdog+0x11b/0x1bd
> [   39.823723]  [<ffffffff8022bffa>] ? __wake_up+0x30/0x44
> [   40.033557]  [<ffffffff8041193a>] ? dev_watchdog+0x0/0x1bd
> [   40.034090]  [<ffffffff80241214>] ? run_timer_softirq+0x18c/0x202
> [   40.034794]  [<ffffffff80251b82>] ? getnstimeofday+0x59/0xb3
> [   40.035449]  [<ffffffff8023d772>] ? __do_softirq+0xa6/0x168
> [   40.036174]  [<ffffffff8020ca7c>] ? call_softirq+0x1c/0x28
> [   40.036861]  [<ffffffff8020e254>] ? do_softirq+0x2c/0x6c
> [   40.037495]  [<ffffffff8023d474>] ? irq_exit+0x3f/0x7c
> [   40.038125]  [<ffffffff8021be42>] ? smp_apic_timer_interrupt+0x87/0x94
> [   40.038795]  [<ffffffff8020c493>] ? apic_timer_interrupt+0x13/0x20
> [   40.039462]  <EOI>  [<ffffffff8021235c>] ? default_idle+0x5b/0x99
> [   40.040370]  [<ffffffff8024e461>] ? notifier_call_chain+0x29/0x4c
> [   40.041105]  [<ffffffff8020ad55>] ? cpu_idle+0x4a/0x8b
> [   40.041723] ---[ end trace dc792b53566c049e ]---
> [   40.484820] eth0: no IPv6 routers present
> [   40.712073] eth1: no IPv6 routers present
> [   43.489776] e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
> 
> 
>
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: KVM x86_64 with SR-IOV..?

[Sheng Yang]

On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> > > > > Greetings KVM folks,
> > > > >
> > > > > I wondering if any information exists for doing SR-IOV on the new
> > > > > VT-d capable chipsets with KVM..?  From what I understand the
> > > > > patches for doing this with KVM are floating around, but I have
> > > > > been unable to find any user-level docs for actually making it all
> > > > > go against a upstream v2.6.30-rc3 code..
> > > > >
> > > > > So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre,
> > > > > and I am really hoping to be able to jump to KVM for
> > > > > single-function and and then multi-function SR-IOV.  I know that
> > > > > the VM migration stuff for IOV in Xen is up and running,  and I
> > > > > assume it is being worked in for KVM instance migration as well..? 
> > > > > This part is less important (at least for me :-) than getting a
> > > > > stable SR-IOV setup running under the KVM hypervisor..  Does anyone
> > > > > have any pointers for this..?
> > > > >
> > > > > Any comments or suggestions are appreciated!
> > > >
> > > > Hi Nicholas
> > > >
> > > > The patches are not floating around now. As you know, SR-IOV for
> > > > Linux have been in 2.6.30, so then you can use upstream KVM and
> > > > qemu-kvm(or recent released kvm-85) with 2.6.30-rc3 as host kernel.
> > > > And some time ago, there are several SRIOV related patches for
> > > > qemu-kvm, and now they all have been checked in.
> > > >
> > > > And for KVM, the extra document is not necessary, for you can simple
> > > > assign a VF to guest like any other devices. And how to create VF is
> > > > specific for each device driver. So just create a VF then assign it
> > > > to KVM guest is fine.
> > >
> > > Greetings Sheng,
> > >
> > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
> > > tell, the KVM_CAP_* defines from libkvm are enabled with building
> > > kvm-85 after './configure --kerneldir=/usr/src/linux-2.6.git' and the
> > > PCI passthrough code is being enabled in
> > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > >
> > > >From there, I use the freshly installed qemu-x86_64-system binary to
> > >
> > > start a Debian 5 x86_64 HVM (that previously had been moving network
> > > packets under Xen for PCIe passthrough). I see the MSI-X interrupt
> > > remapping working on the KVM host for the passed -pcidevice, and the
> > > MMIO mappings from the qemu build that I also saw while using
> > > Xen/qemu-dm built with PCI passthrough are there as well..
> >
> > Hi Nicholas
> >
> > > But while the KVM guest is booting, I see the following exception(s)
> > > from qemu-x86_64-system for one of the VFs for a multi-function PCIe
> > > device:
> > >
> > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> >
> > This one is mostly harmless.
>
> Ok, good to know..  :-)
>
> > > I try with one of the on-board e1000e ports (02:00.0) and I see the
> > > same exception along with some MSI-X exceptions from qemu-x86_64-system
> > > in KVM guest.. However, I am still able to see the e1000e and the other
> > > vxge multi-function device with lspci, but I am unable to dhcp or ping
> > > with the e1000e and VF from multi-function device fails to register the
> > > MSI-X interrupt in the guest..
> >
> > Did you see the interrupt in the guest and host side?
>
> Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> attaching the e1000e single-function device at 02:00.0 to pci-stub with:
>
>    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
>    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
>    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
>
> I see the following the KVM host kernel ring buffer:
>
>    e1000e 0000:02:00.0: PCI INT A disabled
>    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
>    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
>
> >  I think you can try on-
> > board e1000e for MSI-X first. And please ensure correlated driver have
> > been loaded correctly.
>
> <nod>..
>
> >  And what do you mean by "some MSI-X exceptions"? Better with
> > the log.
>
> Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> kvm_destroy_phys_mem() statements:
>
> #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid parameters
> (slot=-1)
> BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
>
> However I still see the following in the KVM guest kernel ring buffer
> running v2.6.30-rc in the HVM guest.
>
> [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> (level, high) -> IRQ 10 [    5.525710] e1000e 0000:00:05.0: setting latency
> timer to 64
> [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X
> interrupts.  Falling back to MSI interrupts. [    5.527200] 0000:00:05.0:
> 0000:00:05.0: Failed to initialize MSI interrupts.  Falling back to legacy
> interrupts. [    5.829988] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width
> x1) 00:e0:81:c0:90:b2 [    5.830672] 0000:00:05.0: eth0: Intel(R) PRO/1000
> Network Connection [    5.831240] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA
> No: ffffff-0ff
>

Hi Nicholas

I think something need to be clarify:
1. For SRIOV, you need 2.6.30 as host kernel... But it's better to know if 
normal device assignment work in your environment at first.
2. The Fedora's userspace is even more old... You'd better try qemu-kvm 
upstream, which is more convenient for us to track the problem(and kvm-85 is 
also ok). And as you see above, your QEmu don't support MSI/MSIX...

So you can:
1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
2. Your host kernel is Fedora 11 Preview, that should be fine with device 
assignment at first(and let's solve it first, SRIOV the next step).
3. Your KVM version seems like kvm-85, you may provide some dmesg on host 
side(I think you didn't use the KVM come along with kernel).
4. Your guest kernel is somehow new, maybe you can change it to something like 
2.6.29.x which is more stable(but it may not matter). And you are right, no 
special setup needed for guest kernel (just suppose it's run in a no-VT 
capability machine).

And you can post your _host_ dmesg, as well as the output of debug for 
kvm-85/qemu/hw/device-assignment.c when you don't get response from dhcp.

Unhandled MSR doesn't matter - and I thought they have already been discard 
long ago?

I saw the interrupt of host assigned device seems fine, you can also check if 
guest have interrupt for that device.

Thanks.

-- 
regards
Yang, Sheng

Re: KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

[-- Attachment #1: Type: text/plain, Size: 28274 bytes --]

On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger wrote:
> > > > > > Greetings KVM folks,
> > > > > >
> > > > > > I wondering if any information exists for doing SR-IOV on the new
> > > > > > VT-d capable chipsets with KVM..?  From what I understand the
> > > > > > patches for doing this with KVM are floating around, but I have
> > > > > > been unable to find any user-level docs for actually making it all
> > > > > > go against a upstream v2.6.30-rc3 code..
> > > > > >
> > > > > > So far I have been doing IOV testing with Xen 3.3 and 3.4.0-pre,
> > > > > > and I am really hoping to be able to jump to KVM for
> > > > > > single-function and and then multi-function SR-IOV.  I know that
> > > > > > the VM migration stuff for IOV in Xen is up and running,  and I
> > > > > > assume it is being worked in for KVM instance migration as well..? 
> > > > > > This part is less important (at least for me :-) than getting a
> > > > > > stable SR-IOV setup running under the KVM hypervisor..  Does anyone
> > > > > > have any pointers for this..?
> > > > > >
> > > > > > Any comments or suggestions are appreciated!
> > > > >
> > > > > Hi Nicholas
> > > > >
> > > > > The patches are not floating around now. As you know, SR-IOV for
> > > > > Linux have been in 2.6.30, so then you can use upstream KVM and
> > > > > qemu-kvm(or recent released kvm-85) with 2.6.30-rc3 as host kernel.
> > > > > And some time ago, there are several SRIOV related patches for
> > > > > qemu-kvm, and now they all have been checked in.
> > > > >
> > > > > And for KVM, the extra document is not necessary, for you can simple
> > > > > assign a VF to guest like any other devices. And how to create VF is
> > > > > specific for each device driver. So just create a VF then assign it
> > > > > to KVM guest is fine.
> > > >
> > > > Greetings Sheng,
> > > >
> > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I can
> > > > tell, the KVM_CAP_* defines from libkvm are enabled with building
> > > > kvm-85 after './configure --kerneldir=/usr/src/linux-2.6.git' and the
> > > > PCI passthrough code is being enabled in
> > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > >
> > > > >From there, I use the freshly installed qemu-x86_64-system binary to
> > > >
> > > > start a Debian 5 x86_64 HVM (that previously had been moving network
> > > > packets under Xen for PCIe passthrough). I see the MSI-X interrupt
> > > > remapping working on the KVM host for the passed -pcidevice, and the
> > > > MMIO mappings from the qemu build that I also saw while using
> > > > Xen/qemu-dm built with PCI passthrough are there as well..
> > >
> > > Hi Nicholas
> > >
> > > > But while the KVM guest is booting, I see the following exception(s)
> > > > from qemu-x86_64-system for one of the VFs for a multi-function PCIe
> > > > device:
> > > >
> > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > >
> > > This one is mostly harmless.
> >
> > Ok, good to know..  :-)
> >
> > > > I try with one of the on-board e1000e ports (02:00.0) and I see the
> > > > same exception along with some MSI-X exceptions from qemu-x86_64-system
> > > > in KVM guest.. However, I am still able to see the e1000e and the other
> > > > vxge multi-function device with lspci, but I am unable to dhcp or ping
> > > > with the e1000e and VF from multi-function device fails to register the
> > > > MSI-X interrupt in the guest..
> > >
> > > Did you see the interrupt in the guest and host side?
> >
> > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > attaching the e1000e single-function device at 02:00.0 to pci-stub with:
> >
> >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> >
> > I see the following the KVM host kernel ring buffer:
> >
> >    e1000e 0000:02:00.0: PCI INT A disabled
> >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> >
> > >  I think you can try on-
> > > board e1000e for MSI-X first. And please ensure correlated driver have
> > > been loaded correctly.
> >
> > <nod>..
> >
> > >  And what do you mean by "some MSI-X exceptions"? Better with
> > > the log.
> >
> > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > kvm_destroy_phys_mem() statements:
> >
> > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid parameters
> > (slot=-1)
> > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> >
> > However I still see the following in the KVM guest kernel ring buffer
> > running v2.6.30-rc in the HVM guest.
> >
> > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > (level, high) -> IRQ 10 [    5.525710] e1000e 0000:00:05.0: setting latency
> > timer to 64
> > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X
> > interrupts.  Falling back to MSI interrupts. [    5.527200] 0000:00:05.0:
> > 0000:00:05.0: Failed to initialize MSI interrupts.  Falling back to legacy
> > interrupts. [    5.829988] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width
> > x1) 00:e0:81:c0:90:b2 [    5.830672] 0000:00:05.0: eth0: Intel(R) PRO/1000
> > Network Connection [    5.831240] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA
> > No: ffffff-0ff
> >
> 
> Hi Nicholas
> 
> I think something need to be clarify:
> 1. For SRIOV, you need 2.6.30 as host kernel... But it's better to know if 
> normal device assignment work in your environment at first.
> 2. The Fedora's userspace is even more old... You'd better try qemu-kvm 
> upstream, which is more convenient for us to track the problem(and kvm-85 is 
> also ok). And as you see above, your QEmu don't support MSI/MSIX...
> 

Ok, got it..

> So you can:
> 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.

Ok, I am now updated on in the FC 11 Host with kvm-85 kernel modules and
am using the built qemu-system-x86_64 from the kvm-85 source package:

loaded kvm module (kvm-85)
QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008 Fabrice Bellard

> 2. Your host kernel is Fedora 11 Preview, that should be fine with device 
> assignment at first(and let's solve it first, SRIOV the next step).

Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM host for
the momemt to get e1000e working.  But I will start building a
v2.6.30-rc3 kernel again for my fc11 host kernel as I do need SR-IOV at
some point... :-)

> 3. Your KVM version seems like kvm-85, you may provide some dmesg on host 
> side(I think you didn't use the KVM come along with kernel).

Ok, now within the KVM guest running v2.6.29.2, I see the following:

[    2.669243] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
[    2.672931] e1000e: Copyright (c) 1999-2008 Intel Corporation.
[    2.674932] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
[    2.675181] 8139too Fast Ethernet driver 0.9.28
[    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
(level, high) -> IRQ 10
[    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
[    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
[    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
[    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
[    2.698039] FDC 0 is a S82078B
[    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
00:e0:81:c0:90:b2
[    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
[    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff

And the folllowing from /proc/interrupts inside of the KVM guest:

 24:        117          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      eth1-rx-0
 25:          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      eth1-tx-0
 26:          2          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      eth1

ethtool eth1 reports that Link is detected, but I am still unable to get
a dhcp to work.

However, I do not see the netdev watchdog TX timeouts this time with
kvm-85, but still no ping to eth1 from KVM guest..

> 4. Your guest kernel is somehow new, maybe you can change it to something like 
> 2.6.29.x which is more stable(but it may not matter). And you are right, no 
> special setup needed for guest kernel (just suppose it's run in a no-VT 
> capability machine).

I will retest with the v2.6.20-rc3 guest and see if MSI-X is able to
successfully come-up and get dhcp.

> 
> And you can post your _host_ dmesg, as well as the output of debug for 

Attached is the dmesg from the 2.6.29.1-111.fc11.x86_64-iov-dmesg KVM
host machine.

> kvm-85/qemu/hw/device-assignment.c when you don't get response from dhcp.

[root@barret ~]#  qemu-system-x86_64 -m 2048 -smp 10 -pcidevice host=02:00.0 lenny64guest1-orig.img 
init_assigned_device: Registering real physical device 02:00.0 (bus=2 dev=0 func=0)
get_real_device: region 0 size 131072 start 0xfb5e0000 type 512 resource_fd 21
get_real_device: region 2 size 32 start 0xdc00 type 256 resource_fd 0
get_real_device: region 3 size 16384 start 0xfb5dc000 type 512 resource_fd 22
assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
assigned_dev_pci_read_config: (5.0): address=000a val=0x00000200 len=2
assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
assigned_dev_pci_write_config: (5.0): address=0010 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0010 val=0xfffe0000 len=4
assigned_dev_pci_read_config: (5.0): address=0010 val=0xfffe0000 len=4
assigned_dev_pci_write_config: (5.0): address=0010 val=0xf2020000 len=4
assigned_dev_iomem_map: e_phys=f2020000 r_virt=0x7f4a24018000 type=0 len=00020000 region_num=0 
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: NON BAR (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: (5.0): address=0014 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0018 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0018 val=0xffffffe1 len=4
assigned_dev_pci_read_config: (5.0): address=0018 val=0xffffffe1 len=4
assigned_dev_pci_write_config: (5.0): address=0018 val=0x0000c220 len=4
assigned_dev_ioport_map: e_phys=0xc220 r_baseport=dc00 type=0x1 len=32 region_num=2 
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: NON BAR (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: (5.0): address=001c val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=001c val=0xffffc000 len=4
assigned_dev_pci_read_config: (5.0): address=001c val=0xffffc000 len=4
assigned_dev_pci_write_config: (5.0): address=001c val=0xf2040000 len=4
assigned_dev_iomem_map: e_phys=f2040000 r_virt=0x7f4a24014000 type=0 len=00004000 region_num=3 
assigned_dev_iomem_map: munmap done, virt_base 0x0x7f4a24014000
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: NON BAR (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: (5.0): address=0020 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0024 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0030 val=0xffffffff len=4
assigned_dev_pci_write_config: NON BAR (5.0): address=0030 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0030 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=003d val=0x00000001 len=1
assigned_dev_pci_write_config: (5.0): address=003c val=0x0000000a len=1
assigned_dev_pci_read_config: (5.0): address=0000 val=0x10d38086 len=4
assigned_dev_pci_read_config: (5.0): address=000a val=0x00000200 len=2
assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
assigned_dev_pci_read_config: (5.0): address=000e val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0008 val=0x02000000 len=4
assigned_dev_pci_read_config: (5.0): address=000e val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0000 val=0x10d38086 len=4
assigned_dev_pci_read_config: (5.0): address=000e val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0008 val=0x02000000 len=4
assigned_dev_pci_read_config: (5.0): address=003d val=0x00000001 len=1
assigned_dev_pci_read_config: (5.0): address=003c val=0x0000000a len=1
assigned_dev_pci_read_config: (5.0): address=0010 val=0xf2020000 len=4
assigned_dev_pci_write_config: (5.0): address=0010 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0010 val=0xfffe0000 len=4
assigned_dev_pci_write_config: (5.0): address=0010 val=0xf2020000 len=4
assigned_dev_iomem_map: e_phys=f2020000 r_virt=0x7f4a24018000 type=0 len=00020000 region_num=0 
BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
assigned_dev_pci_read_config: (5.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0014 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=0018 val=0x0000c221 len=4
assigned_dev_pci_write_config: (5.0): address=0018 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0018 val=0xffffffe1 len=4
assigned_dev_pci_write_config: (5.0): address=0018 val=0x0000c221 len=4
assigned_dev_ioport_map: e_phys=0xc220 r_baseport=dc00 type=0x1 len=32 region_num=2 
assigned_dev_pci_read_config: (5.0): address=001c val=0xf2040000 len=4
assigned_dev_pci_write_config: (5.0): address=001c val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=001c val=0xffffc000 len=4
assigned_dev_pci_write_config: (5.0): address=001c val=0xf2040000 len=4
assigned_dev_iomem_map: e_phys=f2040000 r_virt=0x7f4a24014000 type=0 len=00004000 region_num=3 
BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
assigned_dev_iomem_map: munmap done, virt_base 0x0x7f4a24014000
assigned_dev_pci_read_config: (5.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0020 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0024 val=0xffffffff len=4
assigned_dev_pci_read_config: (5.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=0030 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0030 val=0xfffffffe len=4
assigned_dev_pci_write_config: NON BAR (5.0): address=0030 val=0xfffffffe len=4
assigned_dev_pci_read_config: (5.0): address=0030 val=0x00000000 len=4
assigned_dev_pci_write_config: (5.0): address=0030 val=0x00000000 len=4
assigned_dev_pci_write_config: NON BAR (5.0): address=0030 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=002c val=0x00008086 len=2
assigned_dev_pci_read_config: (5.0): address=002e val=0x00000000 len=2
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0010 val=0x00000000 len=2
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
assigned_dev_pci_write_config: (5.0): address=0004 val=0x00000407 len=2
assigned_dev_pci_write_config: NON BAR (5.0): address=0004 val=0x00000407 len=2
assigned_dev_pci_read_config: (5.0): address=000d val=0x00000000 len=1
assigned_dev_pci_write_config: (5.0): address=000d val=0x00000040 len=1
assigned_dev_pci_write_config: NON BAR (5.0): address=000d val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0000 val=0x10d38086 len=4
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00100407 len=4
assigned_dev_pci_read_config: (5.0): address=0008 val=0x02000000 len=4
assigned_dev_pci_read_config: (5.0): address=000c val=0x00000010 len=4
assigned_dev_pci_read_config: (5.0): address=0010 val=0xf2020000 len=4
assigned_dev_pci_read_config: (5.0): address=0014 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=0018 val=0x0000c221 len=4
assigned_dev_pci_read_config: (5.0): address=001c val=0xf2040000 len=4
assigned_dev_pci_read_config: (5.0): address=0020 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=0024 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=0028 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=002c val=0x00008086 len=4
assigned_dev_pci_read_config: (5.0): address=0030 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=4
assigned_dev_pci_read_config: (5.0): address=0038 val=0x00000000 len=4
assigned_dev_pci_read_config: (5.0): address=003c val=0x0000010a len=4
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0052 val=0x00000004 len=2
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0052 val=0x00000004 len=2
assigned_dev_pci_write_config: (5.0): address=0052 val=0x00000004 len=2
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0052 val=0x00000004 len=2
assigned_dev_pci_read_config: (5.0): address=0054 val=0x00000003 len=4
msix_mmio_writel: write to MSI-X entry table mmio offset 0x0, val 0xfee00000
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x8, val 0x4049
msix_mmio_writel: write to MSI-X entry table mmio offset 0x10, val 0xfee00000
msix_mmio_writel: write to MSI-X entry table mmio offset 0x14, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x18, val 0x4051
msix_mmio_writel: write to MSI-X entry table mmio offset 0x20, val 0xfee00000
msix_mmio_writel: write to MSI-X entry table mmio offset 0x24, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x28, val 0x4059
assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000007 len=2
assigned_dev_pci_write_config: (5.0): address=0004 val=0x00000407 len=2
assigned_dev_pci_write_config: NON BAR (5.0): address=0004 val=0x00000407 len=2
assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0052 val=0x00000004 len=2
assigned_dev_pci_write_config: (5.0): address=0052 val=0x00008004 len=2
assigned_dev_update_msix_mmio: MSI-X data 0x4049, MSI-X addr_lo 0xfee00000
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x18, entry 0
!assigned_dev_update_msix_mmio: MSI-X data 0x4051, MSI-X addr_lo 0xfee00000
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x19, entry 1
!assigned_dev_update_msix_mmio: MSI-X data 0x4059, MSI-X addr_lo 0xfee00000
!assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1a, entry 2
!assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
assigned_dev_pci_read_config: (5.0): address=000e val=0x00000000 len=2
msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x0, val 0xfee00000
msix_mmio_writel: write to MSI-X entry table mmio offset 0x4, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x8, val 0x4049
msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x10, val 0xfee00000
msix_mmio_writel: write to MSI-X entry table mmio offset 0x14, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x18, val 0x4051
msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x20, val 0xfee00000
msix_mmio_writel: write to MSI-X entry table mmio offset 0x24, val 0x0
msix_mmio_writel: write to MSI-X entry table mmio offset 0x28, val 0x4059


> Unhandled MSR doesn't matter - and I thought they have already been discard 
> long ago?
> 
> I saw the interrupt of host assigned device seems fine, you can also check if 
> guest have interrupt for that device.
> 
> Thanks.
> 

Thanks!

--nab


[-- Attachment #2: 2.6.29.1-111.fc11.x86_64-iov-dmesg --]
[-- Type: text/plain, Size: 70286 bytes --]

Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.29.1-111.fc11.x86_64 (mockbuild@x86-4.fedora.phx.redhat.com) (gcc version 4.4.0 20090423 (Red Hat 4.4.0-1) (GCC) ) #1 SMP Fri Apr 24 10:57:09 EDT 2009
Command line: ro root=/dev/mapper/vg_barret-lv_root rhgb pci=resource_alignment=06:00.0;06:00.1;06:00.2;06:00.3;06:00.4;06:00.5;06:00.6;06:00.7;07:00.0;07:00.1;07:00.2;07:00.3;07:00.4;07:00.5;07:00.6;07:00.7
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  Centaur CentaurHauls
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009ec00 (usable)
 BIOS-e820: 000000000009ec00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000bf780000 (usable)
 BIOS-e820: 00000000bf780000 - 00000000bf78e000 (ACPI data)
 BIOS-e820: 00000000bf78e000 - 00000000bf7d0000 (ACPI NVS)
 BIOS-e820: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
 BIOS-e820: 00000000bf7ec000 - 00000000c0000000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffa00000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000240000000 (usable)
PCI: Unknown option `resource_alignment=06:00.0;06:00.1;06:00.2;06:00.3;06:00.4;06:00.5;06:00.6;06:00.7;07:00.0;07:00.1;07:00.2;07:00.3;07:00.4;07:00.5;07:00.6;07:00.7'
DMI present.
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
last_pfn = 0x240000 max_arch_pfn = 0x100000000
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
original variable MTRRs
reg 0, base: 0GB, range: 8GB, type WB
reg 1, base: 8GB, range: 1GB, type WB
reg 2, base: 3GB, range: 1GB, type UC
reg 3, base: 3064MB, range: 8MB, type UC
total RAM coverred: 8184M
Found optimal setting for mtrr clean up
 gran_size: 64K 	chunk_size: 16M 	num_reg: 5  	lose cover RAM: 0G
New variable MTRRs
reg 0, base: 0GB, range: 2GB, type WB
reg 1, base: 2GB, range: 1GB, type WB
reg 2, base: 3064MB, range: 8MB, type UC
reg 3, base: 4GB, range: 4GB, type WB
reg 4, base: 8GB, range: 1GB, type WB
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
last_pfn = 0xbf780 max_arch_pfn = 0x100000000
init_memory_mapping: 0000000000000000-00000000bf780000
 0000000000 - 00bf600000 page 2M
 00bf600000 - 00bf780000 page 4k
kernel direct mapping tables up to bf780000 @ 10000-15000
last_map_addr: bf780000 end: bf780000
init_memory_mapping: 0000000100000000-0000000240000000
 0100000000 - 0240000000 page 2M
kernel direct mapping tables up to 240000000 @ 13000-1d000
last_map_addr: 240000000 end: 240000000
RAMDISK: 37ce4000 - 37fef167
ACPI: RSDP 000FA420, 0024 (r2 ACPIAM)
ACPI: XSDT BF780100, 0064 (r1 042109 XSDT1027 20090421 MSFT       97)
ACPI: FACP BF780290, 00F4 (r4 042109 FACP1027 20090421 MSFT       97)
ACPI Warning (tbfadt-0568): 32/64X length mismatch in Gpe0Block: 128/64 [20081204]
FADT: X_PM1a_EVT_BLK.bit_width (16) does not match PM1_EVT_LEN (4)
ACPI: DSDT BF7804B0, 5E8E (r2  7010_ 7010_001        1 INTL 20051117)
ACPI: FACS BF78E000, 0040
ACPI: APIC BF780390, 00DE (r2 042109 APIC1027 20090421 MSFT       97)
ACPI: MCFG BF780470, 003C (r1 042109 OEMMCFG  20090421 MSFT       97)
ACPI: OEMB BF78E040, 007A (r1 042109 OEMB1027 20090421 MSFT       97)
ACPI: HPET BF78A4B0, 0038 (r1 042109 OEMHPET  20090421 MSFT       97)
ACPI: SRAT BF78A4F0, 01D0 (r1 042109 OEMSRAT         1 INTL        1)
ACPI: DMAR BF78E0C0, 0128 (r1    AMI  OEMDMAR        1 MSFT       97)
ACPI: SSDT BF791E00, 0363 (r1 DpgPmm    CpuPm       12 INTL 20051117)
ACPI: Local APIC address 0xfee00000
SRAT: PXM 0 -> APIC 0 -> Node 0
SRAT: PXM 0 -> APIC 2 -> Node 0
SRAT: PXM 0 -> APIC 4 -> Node 0
SRAT: PXM 0 -> APIC 6 -> Node 0
SRAT: PXM 0 -> APIC 1 -> Node 0
SRAT: PXM 0 -> APIC 3 -> Node 0
SRAT: PXM 0 -> APIC 5 -> Node 0
SRAT: PXM 0 -> APIC 7 -> Node 0
SRAT: PXM 1 -> APIC 16 -> Node 1
SRAT: PXM 1 -> APIC 18 -> Node 1
SRAT: PXM 1 -> APIC 20 -> Node 1
SRAT: PXM 1 -> APIC 22 -> Node 1
SRAT: PXM 1 -> APIC 17 -> Node 1
SRAT: PXM 1 -> APIC 19 -> Node 1
SRAT: PXM 1 -> APIC 21 -> Node 1
SRAT: PXM 1 -> APIC 23 -> Node 1
SRAT: Node 0 PXM 0 0-a0000
SRAT: Node 0 PXM 0 100000-c0000000
SRAT: Node 0 PXM 0 100000000-140000000
SRAT: Node 2 PXM 257 140000000-240000000
NUMA: Allocated memnodemap from 18000 - 1c880
NUMA: Using 20 for the hash shift.
Bootmem setup node 0 0000000000000000-0000000140000000
  NODE_DATA [000000000001c880 - 000000000003187f]
  bootmap [0000000000032000 -  0000000000059fff] pages 28
(8 early reservations) ==> bootmem [0000000000 - 0140000000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
  #2 [0000200000 - 0000acb00c]    TEXT DATA BSS ==> [0000200000 - 0000acb00c]
  #3 [0037ce4000 - 0037fef167]          RAMDISK ==> [0037ce4000 - 0037fef167]
  #4 [000009ec00 - 0000100000]    BIOS reserved ==> [000009ec00 - 0000100000]
  #5 [0000010000 - 0000013000]          PGTABLE ==> [0000010000 - 0000013000]
  #6 [0000013000 - 0000018000]          PGTABLE ==> [0000013000 - 0000018000]
  #7 [0000018000 - 000001c880]       MEMNODEMAP ==> [0000018000 - 000001c880]
Bootmem setup node 2 0000000140000000-0000000240000000
  NODE_DATA [0000000140000000 - 0000000140014fff]
  bootmap [0000000140015000 -  0000000140034fff] pages 20
(8 early reservations) ==> bootmem [0140000000 - 0240000000]
  #0 [0000000000 - 0000001000]   BIOS data page
  #1 [0000006000 - 0000008000]       TRAMPOLINE
  #2 [0000200000 - 0000acb00c]    TEXT DATA BSS
  #3 [0037ce4000 - 0037fef167]          RAMDISK
  #4 [000009ec00 - 0000100000]    BIOS reserved
  #5 [0000010000 - 0000013000]          PGTABLE
  #6 [0000013000 - 0000018000]          PGTABLE
  #7 [0000018000 - 000001c880]       MEMNODEMAP
found SMP MP-table at [ffff8800000ff780] 000ff780
 [ffffe20000000000-ffffe200045fffff] PMD -> [ffff880028200000-ffff88002b9fffff] on node 0
 [ffffe20004600000-ffffe20007dfffff] PMD -> [ffff880140200000-ffff8801439fffff] on node 2
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x00240000
Movable zone start PFN for each node
early_node_map[4] active PFN ranges
    0: 0x00000010 -> 0x0000009e
    0: 0x00000100 -> 0x000bf780
    0: 0x00100000 -> 0x00140000
    2: 0x00140000 -> 0x00240000
On node 0 totalpages: 1046286
  DMA zone: 56 pages used for memmap
  DMA zone: 2365 pages reserved
  DMA zone: 1561 pages, LIFO batch:0
  DMA32 zone: 14280 pages used for memmap
  DMA32 zone: 765880 pages, LIFO batch:31
  Normal zone: 3584 pages used for memmap
  Normal zone: 258560 pages, LIFO batch:31
On node 2 totalpages: 1048576
  Normal zone: 14336 pages used for memmap
  Normal zone: 1034240 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x10] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x12] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x14] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x05] enabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x07] enabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x11] enabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x13] enabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x15] enabled)
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled)
ACPI: LAPIC_NMI (acpi_id[0xff] high dfl lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 0, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24])
IOAPIC[1]: apic_id 9, version 0, address 0xfec8a000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a301 base: 0xfed00000
SMP: Allowing 16 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 48
PM: Registered nosave memory: 000000000009e000 - 000000000009f000
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
PM: Registered nosave memory: 00000000bf780000 - 00000000bf78e000
PM: Registered nosave memory: 00000000bf78e000 - 00000000bf7d0000
PM: Registered nosave memory: 00000000bf7d0000 - 00000000bf7e0000
PM: Registered nosave memory: 00000000bf7e0000 - 00000000bf7ec000
PM: Registered nosave memory: 00000000bf7ec000 - 00000000c0000000
PM: Registered nosave memory: 00000000c0000000 - 00000000e0000000
PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
PM: Registered nosave memory: 00000000f0000000 - 00000000fee00000
PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
PM: Registered nosave memory: 00000000fee01000 - 00000000ffa00000
PM: Registered nosave memory: 00000000ffa00000 - 0000000100000000
Allocating PCI resources starting at c2000000 (gap: c0000000:20000000)
NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:16 nr_node_ids:3
PERCPU: Allocating 65536 bytes of per cpu data
Built 2 zonelists in Node order, mobility grouping on.  Total pages: 2060241
Policy zone: Normal
Kernel command line: ro root=/dev/mapper/vg_barret-lv_root rhgb pci=resource_alignment=06:00.0;06:00.1;06:00.2;06:00.3;06:00.4;06:00.5;06:00.6;06:00.7;07:00.0;07:00.1;07:00.2;07:00.3;07:00.4;07:00.5;07:00.6;07:00.7
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Extended CMOS year: 2000
Fast TSC calibration using PIT
Detected 2266.877 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
allocated 83886080 bytes of page_cgroup
please try cgroup_disable=memory option if you don't want
Checking aperture...
No AGP bridge found
Calgary: detecting Calgary via BIOS EBDA area
Calgary: Unable to locate Rio Grande table in EBDA - bailing!
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
software IO TLB at phys 0x20000000 - 0x24000000
Memory: 8103140k/9437184k available (3788k kernel code, 1057736k absent, 276308k reserved, 2295k data, 1312k init)
SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=3
hpet clockevent registered
HPET: 4 timers in total, 0 timers will be used for per-cpu timer
Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.75 BogoMIPS (lpj=2266877)
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Mount-cache hash table entries: 256
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 0/0x0 -> Node 0
CPU0: Thermal monitoring enabled (TM1)
using mwait in idle threads.
ACPI: Core revision 20081204
ftrace: converting mcount calls to 0f 1f 44 00 00
ftrace: allocating 18898 entries in 149 pages
Setting APIC routing to physical flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
APIC calibration not consistent with PM Timer: 119ms instead of 100ms
APIC delta adjusted to PM-Timer: 833336 (998607)
Booting processor 1 APIC 0x2 ip 0x6000
Initializing CPU#1
Calibrating delay using timer specific routine.. 4532.64 BogoMIPS (lpj=2266323)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 1/0x2 -> Node 0
CPU1: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 1, old 0x7040600070406, new 0x7010600070106
CPU1: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#1]: passed.
Booting processor 2 APIC 0x4 ip 0x6000
Initializing CPU#2
Calibrating delay using timer specific routine.. 4532.64 BogoMIPS (lpj=2266324)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 2/0x4 -> Node 0
CPU2: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 2, old 0x7040600070406, new 0x7010600070106
CPU2: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#2]: passed.
Booting processor 3 APIC 0x6 ip 0x6000
Initializing CPU#3
Calibrating delay using timer specific routine.. 4532.64 BogoMIPS (lpj=2266322)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 3/0x6 -> Node 0
CPU3: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 3, old 0x7040600070406, new 0x7010600070106
CPU3: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#3]: passed.
Booting processor 4 APIC 0x10 ip 0x6000
Initializing CPU#4
Calibrating delay using timer specific routine.. 4532.67 BogoMIPS (lpj=2266339)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 4/0x10 -> Node 0
CPU4: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 4, old 0x7040600070406, new 0x7010600070106
CPU4: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#4]: passed.
Booting processor 5 APIC 0x12 ip 0x6000
Initializing CPU#5
Calibrating delay using timer specific routine.. 4532.67 BogoMIPS (lpj=2266339)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 5/0x12 -> Node 0
CPU5: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 5, old 0x7040600070406, new 0x7010600070106
CPU5: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#5]: passed.
Booting processor 6 APIC 0x14 ip 0x6000
Initializing CPU#6
Calibrating delay using timer specific routine.. 4532.68 BogoMIPS (lpj=2266341)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 6/0x14 -> Node 0
CPU6: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 6, old 0x7040600070406, new 0x7010600070106
CPU6: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#6]: passed.
Booting processor 7 APIC 0x16 ip 0x6000
Initializing CPU#7
Calibrating delay using timer specific routine.. 4532.68 BogoMIPS (lpj=2266342)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 7/0x16 -> Node 0
CPU7: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 7, old 0x7040600070406, new 0x7010600070106
CPU7: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#7]: passed.
Booting processor 8 APIC 0x1 ip 0x6000
Initializing CPU#8
Calibrating delay using timer specific routine.. 4532.67 BogoMIPS (lpj=2266339)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 8/0x1 -> Node 0
CPU8: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 8, old 0x7040600070406, new 0x7010600070106
CPU8: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#8]: passed.
Booting processor 9 APIC 0x3 ip 0x6000
Initializing CPU#9
Calibrating delay using timer specific routine.. 4532.67 BogoMIPS (lpj=2266339)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 9/0x3 -> Node 0
CPU9: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 9, old 0x7040600070406, new 0x7010600070106
CPU9: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#9]: passed.
Booting processor 10 APIC 0x5 ip 0x6000
Initializing CPU#10
Calibrating delay using timer specific routine.. 4532.67 BogoMIPS (lpj=2266339)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 10/0x5 -> Node 0
CPU10: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 10, old 0x7040600070406, new 0x7010600070106
CPU10: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#10]: passed.
Booting processor 11 APIC 0x7 ip 0x6000
Initializing CPU#11
Calibrating delay using timer specific routine.. 4532.66 BogoMIPS (lpj=2266334)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 11/0x7 -> Node 0
CPU11: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 11, old 0x7040600070406, new 0x7010600070106
CPU11: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#11]: passed.
Booting processor 12 APIC 0x11 ip 0x6000
Initializing CPU#12
Calibrating delay using timer specific routine.. 16952.03 BogoMIPS (lpj=8476017)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 12/0x11 -> Node 0
CPU12: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 12, old 0x7040600070406, new 0x7010600070106
CPU12: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#12]: passed.
Booting processor 13 APIC 0x13 ip 0x6000
Initializing CPU#13
Calibrating delay using timer specific routine.. 4532.67 BogoMIPS (lpj=2266339)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 13/0x13 -> Node 0
CPU13: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 13, old 0x7040600070406, new 0x7010600070106
CPU13: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#13]: passed.
Booting processor 14 APIC 0x15 ip 0x6000
Initializing CPU#14
Calibrating delay using timer specific routine.. 4532.67 BogoMIPS (lpj=2266339)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 14/0x15 -> Node 0
CPU14: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 14, old 0x7040600070406, new 0x7010600070106
CPU14: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#14]: passed.
Booting processor 15 APIC 0x17 ip 0x6000
Initializing CPU#15
Calibrating delay using timer specific routine.. 4532.68 BogoMIPS (lpj=2266340)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU 15/0x17 -> Node 0
CPU15: Thermal monitoring enabled (TM1)
x86 PAT enabled: cpu 15, old 0x7040600070406, new 0x7010600070106
CPU15: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
checking TSC synchronization [CPU#0 -> CPU#15]: passed.
Brought up 16 CPUs
Total of 16 processors activated (84943.18 BogoMIPS).
sizeof(vma)=176 bytes
sizeof(page)=56 bytes
sizeof(inode)=560 bytes
sizeof(dentry)=192 bytes
sizeof(ext3inode)=760 bytes
sizeof(buffer_head)=104 bytes
sizeof(skbuff)=232 bytes
sizeof(task_struct)=5920 bytes
CPU0 attaching sched-domain:
 domain 0: span 0,8 level SIBLING
  groups: 0 8
  domain 1: span 0-3,8-11 level MC
   groups: 0,8 1,9 2,10 3,11
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU1 attaching sched-domain:
 domain 0: span 1,9 level SIBLING
  groups: 1 9
  domain 1: span 0-3,8-11 level MC
   groups: 1,9 2,10 3,11 0,8
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU2 attaching sched-domain:
 domain 0: span 2,10 level SIBLING
  groups: 2 10
  domain 1: span 0-3,8-11 level MC
   groups: 2,10 3,11 0,8 1,9
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU3 attaching sched-domain:
 domain 0: span 3,11 level SIBLING
  groups: 3 11
  domain 1: span 0-3,8-11 level MC
   groups: 3,11 0,8 1,9 2,10
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU4 attaching sched-domain:
 domain 0: span 4,12 level SIBLING
  groups: 4 12
  domain 1: span 4-7,12-15 level MC
   groups: 4,12 5,13 6,14 7,15
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU5 attaching sched-domain:
 domain 0: span 5,13 level SIBLING
  groups: 5 13
  domain 1: span 4-7,12-15 level MC
   groups: 5,13 6,14 7,15 4,12
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU6 attaching sched-domain:
 domain 0: span 6,14 level SIBLING
  groups: 6 14
  domain 1: span 4-7,12-15 level MC
   groups: 6,14 7,15 4,12 5,13
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU7 attaching sched-domain:
 domain 0: span 7,15 level SIBLING
  groups: 7 15
  domain 1: span 4-7,12-15 level MC
   groups: 7,15 4,12 5,13 6,14
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU8 attaching sched-domain:
 domain 0: span 0,8 level SIBLING
  groups: 8 0
  domain 1: span 0-3,8-11 level MC
   groups: 0,8 1,9 2,10 3,11
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU9 attaching sched-domain:
 domain 0: span 1,9 level SIBLING
  groups: 9 1
  domain 1: span 0-3,8-11 level MC
   groups: 1,9 2,10 3,11 0,8
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU10 attaching sched-domain:
 domain 0: span 2,10 level SIBLING
  groups: 10 2
  domain 1: span 0-3,8-11 level MC
   groups: 2,10 3,11 0,8 1,9
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU11 attaching sched-domain:
 domain 0: span 3,11 level SIBLING
  groups: 11 3
  domain 1: span 0-3,8-11 level MC
   groups: 3,11 0,8 1,9 2,10
   domain 2: span 0-15 level CPU
    groups: 0-3,8-11 4-7,12-15
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU12 attaching sched-domain:
 domain 0: span 4,12 level SIBLING
  groups: 12 4
  domain 1: span 4-7,12-15 level MC
   groups: 4,12 5,13 6,14 7,15
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU13 attaching sched-domain:
 domain 0: span 5,13 level SIBLING
  groups: 13 5
  domain 1: span 4-7,12-15 level MC
   groups: 5,13 6,14 7,15 4,12
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU14 attaching sched-domain:
 domain 0: span 6,14 level SIBLING
  groups: 14 6
  domain 1: span 4-7,12-15 level MC
   groups: 6,14 7,15 4,12 5,13
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
CPU15 attaching sched-domain:
 domain 0: span 7,15 level SIBLING
  groups: 15 7
  domain 1: span 4-7,12-15 level MC
   groups: 7,15 4,12 5,13 6,14
   domain 2: span 0-15 level CPU
    groups: 4-7,12-15 0-3,8-11
    domain 3: span 0-15 level NODE
     groups: 0-15
net_namespace: 1904 bytes
Booting paravirtualized kernel on bare hardware
regulator: core version 0.5
Time:  8:42:24  Date: 05/04/09
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
PCI: MCFG area at e0000000 reserved in E820
PCI: Using MMCONFIG at e0000000 - efffffff
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
ACPI: PCI Root Bridge [PCI0] (0000:00)
pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
pci 0000:00:00.0: PME# disabled
pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
pci 0000:00:01.0: PME# disabled
pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
pci 0000:00:03.0: PME# disabled
pci 0000:00:05.0: PME# supported from D0 D3hot D3cold
pci 0000:00:05.0: PME# disabled
pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
pci 0000:00:07.0: PME# disabled
pci 0000:00:13.0: reg 10 32bit mmio: [0xfec8a000-0xfec8afff]
pci 0000:00:13.0: PME# supported from D0 D3hot D3cold
pci 0000:00:13.0: PME# disabled
pci 0000:00:16.0: reg 10 64bit mmio: [0xfa6d4000-0xfa6d7fff]
pci 0000:00:16.1: reg 10 64bit mmio: [0xfa6d8000-0xfa6dbfff]
pci 0000:00:16.2: reg 10 64bit mmio: [0xfa6dc000-0xfa6dffff]
pci 0000:00:16.3: reg 10 64bit mmio: [0xfa6e0000-0xfa6e3fff]
pci 0000:00:16.4: reg 10 64bit mmio: [0xfa6e4000-0xfa6e7fff]
pci 0000:00:16.5: reg 10 64bit mmio: [0xfa6e8000-0xfa6ebfff]
pci 0000:00:16.6: reg 10 64bit mmio: [0xfa6ec000-0xfa6effff]
pci 0000:00:16.7: reg 10 64bit mmio: [0xfa6f0000-0xfa6f3fff]
pci 0000:00:1a.0: reg 20 io port: [0x9400-0x941f]
pci 0000:00:1a.1: reg 20 io port: [0x9480-0x949f]
pci 0000:00:1a.2: reg 20 io port: [0x9800-0x981f]
pci 0000:00:1a.7: reg 10 32bit mmio: [0xfa6f6000-0xfa6f63ff]
pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1a.7: PME# disabled
pci 0000:00:1b.0: reg 10 64bit mmio: [0xfa6f8000-0xfa6fbfff]
pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1b.0: PME# disabled
pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.0: PME# disabled
pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.4: PME# disabled
pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.5: PME# disabled
pci 0000:00:1d.0: reg 20 io port: [0x9880-0x989f]
pci 0000:00:1d.1: reg 20 io port: [0x9c00-0x9c1f]
pci 0000:00:1d.2: reg 20 io port: [0xa000-0xa01f]
pci 0000:00:1d.7: reg 10 32bit mmio: [0xfa6fc000-0xfa6fc3ff]
pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1d.7: PME# disabled
pci 0000:00:1f.0: quirk: region 0800-087f claimed by ICH6 ACPI/GPIO/TCO
pci 0000:00:1f.0: quirk: region 0500-053f claimed by ICH6 GPIO
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0a00 (mask 00ff)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 2 PIO at 0ca0 (mask 000f)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 3 PIO at f700 (mask 000f)
pci 0000:00:1f.2: reg 10 io port: [0xac00-0xac07]
pci 0000:00:1f.2: reg 14 io port: [0xa880-0xa883]
pci 0000:00:1f.2: reg 18 io port: [0xa800-0xa807]
pci 0000:00:1f.2: reg 1c io port: [0xa480-0xa483]
pci 0000:00:1f.2: reg 20 io port: [0xa400-0xa40f]
pci 0000:00:1f.2: reg 24 io port: [0xa080-0xa08f]
pci 0000:00:1f.3: reg 10 64bit mmio: [0xfa6fe000-0xfa6fe0ff]
pci 0000:00:1f.3: reg 20 io port: [0x400-0x41f]
pci 0000:00:1f.5: reg 10 io port: [0xbc00-0xbc07]
pci 0000:00:1f.5: reg 14 io port: [0xb880-0xb883]
pci 0000:00:1f.5: reg 18 io port: [0xb800-0xb807]
pci 0000:00:1f.5: reg 1c io port: [0xb480-0xb483]
pci 0000:00:1f.5: reg 20 io port: [0xb400-0xb40f]
pci 0000:00:1f.5: reg 24 io port: [0xb080-0xb08f]
pci 0000:07:00.0: reg 10 64bit mmio: [0xf2800000-0xf2ffffff]
pci 0000:07:00.0: reg 18 64bit mmio: [0xf27ff000-0xf27fffff]
pci 0000:07:00.0: reg 20 64bit mmio: [0xf27fec00-0xf27fecff]
pci 0000:07:00.0: reg 30 32bit mmio: [0xfbb00000-0xfbb7ffff]
pci 0000:07:00.0: supports D1 D2
pci 0000:07:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.0: PME# disabled
pci 0000:07:00.1: reg 10 64bit mmio: [0xf3800000-0xf3ffffff]
pci 0000:07:00.1: reg 18 64bit mmio: [0xf37ff000-0xf37fffff]
pci 0000:07:00.1: reg 20 64bit mmio: [0xf37fec00-0xf37fecff]
pci 0000:07:00.1: reg 30 32bit mmio: [0xfbb80000-0xfbbfffff]
pci 0000:07:00.1: supports D1 D2
pci 0000:07:00.1: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.1: PME# disabled
pci 0000:07:00.2: reg 10 64bit mmio: [0xf4800000-0xf4ffffff]
pci 0000:07:00.2: reg 18 64bit mmio: [0xf47ff000-0xf47fffff]
pci 0000:07:00.2: reg 20 64bit mmio: [0xf47fec00-0xf47fecff]
pci 0000:07:00.2: reg 30 32bit mmio: [0xfbc00000-0xfbc7ffff]
pci 0000:07:00.2: supports D1 D2
pci 0000:07:00.2: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.2: PME# disabled
pci 0000:07:00.3: reg 10 64bit mmio: [0xf5800000-0xf5ffffff]
pci 0000:07:00.3: reg 18 64bit mmio: [0xf57ff000-0xf57fffff]
pci 0000:07:00.3: reg 20 64bit mmio: [0xf57fec00-0xf57fecff]
pci 0000:07:00.3: reg 30 32bit mmio: [0xfbc80000-0xfbcfffff]
pci 0000:07:00.3: supports D1 D2
pci 0000:07:00.3: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.3: PME# disabled
pci 0000:07:00.4: reg 10 64bit mmio: [0xf6800000-0xf6ffffff]
pci 0000:07:00.4: reg 18 64bit mmio: [0xf67ff000-0xf67fffff]
pci 0000:07:00.4: reg 20 64bit mmio: [0xf67fec00-0xf67fecff]
pci 0000:07:00.4: reg 30 32bit mmio: [0xfbd00000-0xfbd7ffff]
pci 0000:07:00.4: supports D1 D2
pci 0000:07:00.4: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.4: PME# disabled
pci 0000:07:00.5: reg 10 64bit mmio: [0xf7800000-0xf7ffffff]
pci 0000:07:00.5: reg 18 64bit mmio: [0xf77ff000-0xf77fffff]
pci 0000:07:00.5: reg 20 64bit mmio: [0xf77fec00-0xf77fecff]
pci 0000:07:00.5: reg 30 32bit mmio: [0xfbd80000-0xfbdfffff]
pci 0000:07:00.5: supports D1 D2
pci 0000:07:00.5: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.5: PME# disabled
pci 0000:07:00.6: reg 10 64bit mmio: [0xf8800000-0xf8ffffff]
pci 0000:07:00.6: reg 18 64bit mmio: [0xf87ff000-0xf87fffff]
pci 0000:07:00.6: reg 20 64bit mmio: [0xf87fec00-0xf87fecff]
pci 0000:07:00.6: reg 30 32bit mmio: [0xfbe00000-0xfbe7ffff]
pci 0000:07:00.6: supports D1 D2
pci 0000:07:00.6: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.6: PME# disabled
pci 0000:07:00.7: reg 10 64bit mmio: [0xf9800000-0xf9ffffff]
pci 0000:07:00.7: reg 18 64bit mmio: [0xf97ff000-0xf97fffff]
pci 0000:07:00.7: reg 20 64bit mmio: [0xf97fec00-0xf97fecff]
pci 0000:07:00.7: reg 30 32bit mmio: [0xfbe80000-0xfbefffff]
pci 0000:07:00.7: supports D1 D2
pci 0000:07:00.7: PME# supported from D0 D1 D2 D3hot
pci 0000:07:00.7: PME# disabled
pci 0000:00:03.0: bridge 32bit mmio: [0xfbb00000-0xfbefffff]
pci 0000:00:03.0: bridge 64bit mmio pref: [0xf2700000-0xf9ffffff]
pci 0000:06:00.0: reg 10 64bit mmio: [0xd8800000-0xd8ffffff]
pci 0000:06:00.0: reg 18 64bit mmio: [0xd87ff000-0xd87fffff]
pci 0000:06:00.0: reg 20 64bit mmio: [0xd87fec00-0xd87fecff]
pci 0000:06:00.0: reg 30 32bit mmio: [0xfb700000-0xfb77ffff]
pci 0000:06:00.0: supports D1 D2
pci 0000:06:00.0: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.0: PME# disabled
pci 0000:06:00.1: reg 10 64bit mmio: [0xd9800000-0xd9ffffff]
pci 0000:06:00.1: reg 18 64bit mmio: [0xd97ff000-0xd97fffff]
pci 0000:06:00.1: reg 20 64bit mmio: [0xd97fec00-0xd97fecff]
pci 0000:06:00.1: reg 30 32bit mmio: [0xfb780000-0xfb7fffff]
pci 0000:06:00.1: supports D1 D2
pci 0000:06:00.1: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.1: PME# disabled
pci 0000:06:00.2: reg 10 64bit mmio: [0xda800000-0xdaffffff]
pci 0000:06:00.2: reg 18 64bit mmio: [0xda7ff000-0xda7fffff]
pci 0000:06:00.2: reg 20 64bit mmio: [0xda7fec00-0xda7fecff]
pci 0000:06:00.2: reg 30 32bit mmio: [0xfb800000-0xfb87ffff]
pci 0000:06:00.2: supports D1 D2
pci 0000:06:00.2: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.2: PME# disabled
pci 0000:06:00.3: reg 10 64bit mmio: [0xdb800000-0xdbffffff]
pci 0000:06:00.3: reg 18 64bit mmio: [0xdb7ff000-0xdb7fffff]
pci 0000:06:00.3: reg 20 64bit mmio: [0xdb7fec00-0xdb7fecff]
pci 0000:06:00.3: reg 30 32bit mmio: [0xfb880000-0xfb8fffff]
pci 0000:06:00.3: supports D1 D2
pci 0000:06:00.3: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.3: PME# disabled
pci 0000:06:00.4: reg 10 64bit mmio: [0xdc800000-0xdcffffff]
pci 0000:06:00.4: reg 18 64bit mmio: [0xdc7ff000-0xdc7fffff]
pci 0000:06:00.4: reg 20 64bit mmio: [0xdc7fec00-0xdc7fecff]
pci 0000:06:00.4: reg 30 32bit mmio: [0xfb900000-0xfb97ffff]
pci 0000:06:00.4: supports D1 D2
pci 0000:06:00.4: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.4: PME# disabled
pci 0000:06:00.5: reg 10 64bit mmio: [0xdd800000-0xddffffff]
pci 0000:06:00.5: reg 18 64bit mmio: [0xdd7ff000-0xdd7fffff]
pci 0000:06:00.5: reg 20 64bit mmio: [0xdd7fec00-0xdd7fecff]
pci 0000:06:00.5: reg 30 32bit mmio: [0xfb980000-0xfb9fffff]
pci 0000:06:00.5: supports D1 D2
pci 0000:06:00.5: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.5: PME# disabled
pci 0000:06:00.6: reg 10 64bit mmio: [0xde800000-0xdeffffff]
pci 0000:06:00.6: reg 18 64bit mmio: [0xde7ff000-0xde7fffff]
pci 0000:06:00.6: reg 20 64bit mmio: [0xde7fec00-0xde7fecff]
pci 0000:06:00.6: reg 30 32bit mmio: [0xfba00000-0xfba7ffff]
pci 0000:06:00.6: supports D1 D2
pci 0000:06:00.6: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.6: PME# disabled
pci 0000:06:00.7: reg 10 64bit mmio: [0xdf800000-0xdfffffff]
pci 0000:06:00.7: reg 18 64bit mmio: [0xdf7ff000-0xdf7fffff]
pci 0000:06:00.7: reg 20 64bit mmio: [0xdf7fec00-0xdf7fecff]
pci 0000:06:00.7: reg 30 32bit mmio: [0xfba80000-0xfbafffff]
pci 0000:06:00.7: supports D1 D2
pci 0000:06:00.7: PME# supported from D0 D1 D2 D3hot
pci 0000:06:00.7: PME# disabled
pci 0000:00:05.0: bridge 32bit mmio: [0xfb700000-0xfbafffff]
pci 0000:00:05.0: bridge 64bit mmio pref: [0xd8700000-0xdfffffff]
pci 0000:03:00.0: reg 10 32bit mmio: [0xfb6e0000-0xfb6fffff]
pci 0000:03:00.0: reg 18 io port: [0xec00-0xec1f]
pci 0000:03:00.0: reg 1c 32bit mmio: [0xfb6dc000-0xfb6dffff]
pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
pci 0000:03:00.0: PME# disabled
pci 0000:00:1c.4: bridge io port: [0xe000-0xefff]
pci 0000:00:1c.4: bridge 32bit mmio: [0xfb600000-0xfb6fffff]
pci 0000:02:00.0: reg 10 32bit mmio: [0xfb5e0000-0xfb5fffff]
pci 0000:02:00.0: reg 18 io port: [0xdc00-0xdc1f]
pci 0000:02:00.0: reg 1c 32bit mmio: [0xfb5dc000-0xfb5dffff]
pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
pci 0000:02:00.0: PME# disabled
pci 0000:00:1c.5: bridge io port: [0xd000-0xdfff]
pci 0000:00:1c.5: bridge 32bit mmio: [0xfb500000-0xfb5fffff]
pci 0000:01:00.0: reg 10 32bit mmio: [0xfa800000-0xfaffffff]
pci 0000:01:00.0: reg 14 32bit mmio: [0xfa7e0000-0xfa7fffff]
pci 0000:01:00.0: reg 18 io port: [0xc880-0xc8ff]
pci 0000:01:00.0: supports D1 D2
pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0000:01:00.0: PME# disabled
pci 0000:01:05.0: reg 10 32bit mmio: [0xfb4fe000-0xfb4fe7ff]
pci 0000:01:05.0: reg 14 io port: [0xcc00-0xcc7f]
pci 0000:01:05.0: supports D2
pci 0000:01:05.0: PME# supported from D2 D3hot D3cold
pci 0000:01:05.0: PME# disabled
pci 0000:00:1e.0: transparent bridge
pci 0000:00:1e.0: bridge io port: [0xc000-0xcfff]
pci 0000:00:1e.0: bridge 32bit mmio: [0xfa700000-0xfb4fffff]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE3._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE5._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE7._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P8._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P9._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs *5)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 *6 7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 6 *7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 6 7 10 11 12 *14 15)
ACPI Warning (tbutils-0242): Incorrect checksum in table [OEMB] - 9A, should be 97 [20081204]
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
DMAR:Host address width 40
DMAR:DRHD (flags: 0x00000001)base: 0x00000000fbffe000
DMAR:RMRR base: 0x00000000000ec000 end: 0x00000000000effff
DMAR:RMRR base: 0x00000000bf7ec000 end: 0x00000000bf7fffff
DMAR:Unknown DMAR structure type
IOAPIC id 8 under DRHD base 0xfbffe000
IOAPIC id 9 under DRHD base 0xfbffe000
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
hpet0: 4 comparators, 64-bit 14.318180 MHz counter
Switched to high resolution mode on CPU 0
Switched to high resolution mode on CPU 7
Switched to high resolution mode on CPU 5
Switched to high resolution mode on CPU 14
Switched to high resolution mode on CPU 8
Switched to high resolution mode on CPU 11
Switched to high resolution mode on CPU 2
Switched to high resolution mode on CPU 9
Switched to high resolution mode on CPU 10
Switched to high resolution mode on CPU 1
Switched to high resolution mode on CPU 3
Switched to high resolution mode on CPU 13
Switched to high resolution mode on CPU 12
Switched to high resolution mode on CPU 15
Switched to high resolution mode on CPU 4
Switched to high resolution mode on CPU 6
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 14 devices
ACPI: ACPI bus type pnp unregistered
system 00:01: iomem range 0xfbf00000-0xfbffffff has been reserved
system 00:01: iomem range 0xfc000000-0xfcffffff has been reserved
system 00:01: iomem range 0xfd000000-0xfdffffff has been reserved
system 00:01: iomem range 0xfe000000-0xfebfffff has been reserved
system 00:01: iomem range 0xfec8a000-0xfec8afff has been reserved
system 00:01: iomem range 0xfed10000-0xfed10fff has been reserved
system 00:08: ioport range 0xa00-0xa0f has been reserved
system 00:08: ioport range 0xa10-0xa1f has been reserved
system 00:09: ioport range 0x4d0-0x4d1 has been reserved
system 00:09: ioport range 0x800-0x87f has been reserved
system 00:09: ioport range 0x500-0x57f could not be reserved
system 00:09: iomem range 0xfed1c000-0xfed1ffff has been reserved
system 00:09: iomem range 0xfed20000-0xfed3ffff has been reserved
system 00:09: iomem range 0xfed40000-0xfed8ffff has been reserved
system 00:0b: iomem range 0xfec00000-0xfec00fff has been reserved
system 00:0b: iomem range 0xfee00000-0xfee00fff has been reserved
system 00:0c: iomem range 0xe0000000-0xefffffff has been reserved
system 00:0d: iomem range 0x0-0x9ffff could not be reserved
system 00:0d: iomem range 0xc0000-0xcffff has been reserved
system 00:0d: iomem range 0xe0000-0xfffff could not be reserved
system 00:0d: iomem range 0x100000-0xbfffffff could not be reserved
system 00:0d: iomem range 0xfed90000-0xffffffff could not be reserved
pci 0000:00:01.0: PCI bridge, secondary bus 0000:08
pci 0000:00:01.0:   IO window: disabled
pci 0000:00:01.0:   MEM window: disabled
pci 0000:00:01.0:   PREFETCH window: disabled
pci 0000:00:03.0: PCI bridge, secondary bus 0000:07
pci 0000:00:03.0:   IO window: disabled
pci 0000:00:03.0:   MEM window: 0xfbb00000-0xfbefffff
pci 0000:00:03.0:   PREFETCH window: 0x000000f2700000-0x000000f9ffffff
pci 0000:00:05.0: PCI bridge, secondary bus 0000:06
pci 0000:00:05.0:   IO window: disabled
pci 0000:00:05.0:   MEM window: 0xfb700000-0xfbafffff
pci 0000:00:05.0:   PREFETCH window: 0x000000d8700000-0x000000dfffffff
pci 0000:00:07.0: PCI bridge, secondary bus 0000:05
pci 0000:00:07.0:   IO window: disabled
pci 0000:00:07.0:   MEM window: disabled
pci 0000:00:07.0:   PREFETCH window: disabled
pci 0000:00:1c.0: PCI bridge, secondary bus 0000:04
pci 0000:00:1c.0:   IO window: disabled
pci 0000:00:1c.0:   MEM window: disabled
pci 0000:00:1c.0:   PREFETCH window: disabled
pci 0000:00:1c.4: PCI bridge, secondary bus 0000:03
pci 0000:00:1c.4:   IO window: 0xe000-0xefff
pci 0000:00:1c.4:   MEM window: 0xfb600000-0xfb6fffff
pci 0000:00:1c.4:   PREFETCH window: disabled
pci 0000:00:1c.5: PCI bridge, secondary bus 0000:02
pci 0000:00:1c.5:   IO window: 0xd000-0xdfff
pci 0000:00:1c.5:   MEM window: 0xfb500000-0xfb5fffff
pci 0000:00:1c.5:   PREFETCH window: disabled
pci 0000:00:1e.0: PCI bridge, secondary bus 0000:01
pci 0000:00:1e.0:   IO window: 0xc000-0xcfff
pci 0000:00:1e.0:   MEM window: 0xfa700000-0xfb4fffff
pci 0000:00:1e.0:   PREFETCH window: disabled
pci 0000:00:01.0: setting latency timer to 64
pci 0000:00:03.0: setting latency timer to 64
pci 0000:00:05.0: setting latency timer to 64
pci 0000:00:07.0: setting latency timer to 64
  alloc irq_desc for 17 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.0: setting latency timer to 64
pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.4: setting latency timer to 64
  alloc irq_desc for 16 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pci 0000:00:1c.5: PCI INT B -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.5: setting latency timer to 64
pci 0000:00:1e.0: setting latency timer to 64
pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
pci_bus 0000:08: resource 0 mem: [0x0-0x0]
pci_bus 0000:08: resource 1 mem: [0x0-0x0]
pci_bus 0000:08: resource 2 mem: [0x0-0x0]
pci_bus 0000:08: resource 3 mem: [0x0-0x0]
pci_bus 0000:07: resource 0 mem: [0x0-0x0]
pci_bus 0000:07: resource 1 mem: [0xfbb00000-0xfbefffff]
pci_bus 0000:07: resource 2 mem: [0xf2700000-0xf9ffffff]
pci_bus 0000:07: resource 3 mem: [0x0-0x0]
pci_bus 0000:06: resource 0 mem: [0x0-0x0]
pci_bus 0000:06: resource 1 mem: [0xfb700000-0xfbafffff]
pci_bus 0000:06: resource 2 mem: [0xd8700000-0xdfffffff]
pci_bus 0000:06: resource 3 mem: [0x0-0x0]
pci_bus 0000:05: resource 0 mem: [0x0-0x0]
pci_bus 0000:05: resource 1 mem: [0x0-0x0]
pci_bus 0000:05: resource 2 mem: [0x0-0x0]
pci_bus 0000:05: resource 3 mem: [0x0-0x0]
pci_bus 0000:04: resource 0 mem: [0x0-0x0]
pci_bus 0000:04: resource 1 mem: [0x0-0x0]
pci_bus 0000:04: resource 2 mem: [0x0-0x0]
pci_bus 0000:04: resource 3 mem: [0x0-0x0]
pci_bus 0000:03: resource 0 io:  [0xe000-0xefff]
pci_bus 0000:03: resource 1 mem: [0xfb600000-0xfb6fffff]
pci_bus 0000:03: resource 2 mem: [0x0-0x0]
pci_bus 0000:03: resource 3 mem: [0x0-0x0]
pci_bus 0000:02: resource 0 io:  [0xd000-0xdfff]
pci_bus 0000:02: resource 1 mem: [0xfb500000-0xfb5fffff]
pci_bus 0000:02: resource 2 mem: [0x0-0x0]
pci_bus 0000:02: resource 3 mem: [0x0-0x0]
pci_bus 0000:01: resource 0 io:  [0xc000-0xcfff]
pci_bus 0000:01: resource 1 mem: [0xfa700000-0xfb4fffff]
pci_bus 0000:01: resource 2 mem: [0x0-0x0]
pci_bus 0000:01: resource 3 io:  [0x00-0xffff]
pci_bus 0000:01: resource 4 mem: [0x000000-0xffffffffffffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
NET: Registered protocol family 1
checking if image is initramfs... it is
Freeing initrd memory: 3116k freed
audit: initializing netlink socket (disabled)
type=2000 audit(1241426543.677:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 15832
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci 0000:01:00.0: Boot video device
pcieport-driver 0000:00:01.0: setting latency timer to 64
  alloc irq_desc for 48 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pcieport-driver 0000:00:01.0: irq 48 for MSI/MSI-X
pcieport-driver 0000:00:03.0: setting latency timer to 64
  alloc irq_desc for 49 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pcieport-driver 0000:00:03.0: irq 49 for MSI/MSI-X
pcieport-driver 0000:00:05.0: setting latency timer to 64
  alloc irq_desc for 50 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pcieport-driver 0000:00:05.0: irq 50 for MSI/MSI-X
pcieport-driver 0000:00:07.0: setting latency timer to 64
  alloc irq_desc for 51 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pcieport-driver 0000:00:07.0: irq 51 for MSI/MSI-X
pcieport-driver 0000:00:1c.0: setting latency timer to 64
  alloc irq_desc for 52 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pcieport-driver 0000:00:1c.0: irq 52 for MSI/MSI-X
pcieport-driver 0000:00:1c.4: setting latency timer to 64
  alloc irq_desc for 53 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pcieport-driver 0000:00:1c.4: irq 53 for MSI/MSI-X
pcieport-driver 0000:00:1c.5: setting latency timer to 64
  alloc irq_desc for 54 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
pcieport-driver 0000:00:1c.5: irq 54 for MSI/MSI-X
aer 0000:00:01.0:pcie01: AER service couldn't init device: no _OSC support
aer 0000:00:03.0:pcie01: AER service couldn't init device: no _OSC support
aer 0000:00:05.0:pcie01: AER service couldn't init device: no _OSC support
aer 0000:00:07.0:pcie01: AER service couldn't init device: no _OSC support
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
input: Power Button (FF) as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
ACPI: Power Button (FF) [PWRF]
input: Power Button (CM) as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input1
ACPI: Power Button (CM) [PWRB]
ACPI: SSDT BF78E1F0, 2B94 (r1 DpgPmm  P001Ist       11 INTL 20051117)
ACPI: SSDT BF790D90, 0980 (r1  PmRef  P001Cst     3001 INTL 20051117)
ACPI: SSDT BF791710, 06E2 (r1  PmRef  Cpu0Tst     3000 INTL 20051117)
Monitor-Mwait will be used to enter C-1 state
Monitor-Mwait will be used to enter C-3 state
ACPI: CPU0 (power states: C1[C1] C2[C3])
processor ACPI_CPU:00: registered as cooling_device0
ACPI: Processor [P001] (supports 8 throttling states)
ACPI: CPU1 (power states: C1[C1] C2[C3])
processor ACPI_CPU:01: registered as cooling_device1
ACPI: Processor [P002] (supports 8 throttling states)
ACPI: CPU2 (power states: C1[C1] C2[C3])
processor ACPI_CPU:02: registered as cooling_device2
ACPI: Processor [P003] (supports 8 throttling states)
ACPI: CPU3 (power states: C1[C1] C2[C3])
processor ACPI_CPU:03: registered as cooling_device3
ACPI: Processor [P004] (supports 8 throttling states)
ACPI: CPU4 (power states: C1[C1] C2[C3])
processor ACPI_CPU:04: registered as cooling_device4
ACPI: Processor [P005] (supports 8 throttling states)
ACPI: CPU5 (power states: C1[C1] C2[C3])
processor ACPI_CPU:05: registered as cooling_device5
ACPI: Processor [P006] (supports 8 throttling states)
ACPI: CPU6 (power states: C1[C1] C2[C3])
processor ACPI_CPU:06: registered as cooling_device6
ACPI: Processor [P007] (supports 8 throttling states)
ACPI: CPU7 (power states: C1[C1] C2[C3])
processor ACPI_CPU:07: registered as cooling_device7
ACPI: Processor [P008] (supports 8 throttling states)
ACPI: CPU8 (power states: C1[C1] C2[C3])
processor ACPI_CPU:08: registered as cooling_device8
ACPI: Processor [P009] (supports 8 throttling states)
ACPI: CPU9 (power states: C1[C1] C2[C3])
processor ACPI_CPU:09: registered as cooling_device9
ACPI: Processor [P010] (supports 8 throttling states)
ACPI: CPU10 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0a: registered as cooling_device10
ACPI: Processor [P011] (supports 8 throttling states)
ACPI: CPU11 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0b: registered as cooling_device11
ACPI: Processor [P012] (supports 8 throttling states)
ACPI: CPU12 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0c: registered as cooling_device12
ACPI: Processor [P013] (supports 8 throttling states)
ACPI: CPU13 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0d: registered as cooling_device13
ACPI: Processor [P014] (supports 8 throttling states)
ACPI: CPU14 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0e: registered as cooling_device14
ACPI: Processor [P015] (supports 8 throttling states)
ACPI: CPU15 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0f: registered as cooling_device15
ACPI: Processor [P016] (supports 8 throttling states)
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:07: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
brd: module loaded
loop: module loaded
Fixed MDIO Bus: probed
input: Macintosh mouse button emulation as /devices/virtual/input/input2
Driver 'sd' needs updating - please use bus_type methods
Driver 'sr' needs updating - please use bus_type methods
ata_piix 0000:00:1f.2: version 2.12
  alloc irq_desc for 19 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
ata_piix 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
ata_piix 0000:00:1f.2: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: SATA max UDMA/133 cmd 0xac00 ctl 0xa880 bmdma 0xa400 irq 19
ata2: SATA max UDMA/133 cmd 0xa800 ctl 0xa480 bmdma 0xa408 irq 19
ata1.00: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.01: SATA link down (SStatus 0 SControl 300)
ata1.00: ATA-6: ST380817AS, 3.42, max UDMA/133
ata1.00: 156301488 sectors, multi 16: LBA48 NCQ (not used)
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      ST380817AS       3.42 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 156301488 512-byte hardware sectors: (80.0 GB/74.5 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:0:0: [sda] 156301488 512-byte hardware sectors: (80.0 GB/74.5 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
sd 0:0:0:0: Attached scsi generic sg0 type 0
ata2.00: SATA link down (SStatus 0 SControl 300)
ata2.01: SATA link down (SStatus 0 SControl 300)
ata_piix 0000:00:1f.5: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.5: MAP [ P0 -- P1 -- ]
ata_piix 0000:00:1f.5: setting latency timer to 64
scsi2 : ata_piix
scsi3 : ata_piix
ata3: SATA max UDMA/133 cmd 0xbc00 ctl 0xb880 bmdma 0xb400 irq 19
ata4: SATA max UDMA/133 cmd 0xb800 ctl 0xb480 bmdma 0xb408 irq 19
ata3: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
  alloc irq_desc for 18 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
ehci_hcd 0000:00:1a.7: PCI INT C -> GSI 18 (level, low) -> IRQ 18
ehci_hcd 0000:00:1a.7: setting latency timer to 64
ehci_hcd 0000:00:1a.7: EHCI Host Controller
ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1a.7: debug port 1
ehci_hcd 0000:00:1a.7: cache line size of 32 is not supported
ehci_hcd 0000:00:1a.7: irq 18, io mem 0xfa6f6000
ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 ehci_hcd
usb usb1: SerialNumber: 0000:00:1a.7
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
  alloc irq_desc for 23 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 23 (level, low) -> IRQ 23
ehci_hcd 0000:00:1d.7: setting latency timer to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:1d.7: debug port 1
ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
ehci_hcd 0000:00:1d.7: irq 23, io mem 0xfa6fc000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: EHCI Host Controller
usb usb2: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 ehci_hcd
usb usb2: SerialNumber: 0000:00:1d.7
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
uhci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
uhci_hcd 0000:00:1a.0: setting latency timer to 64
uhci_hcd 0000:00:1a.0: UHCI Host Controller
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.0: irq 16, io base 0x00009400
usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: UHCI Host Controller
usb usb3: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 uhci_hcd
usb usb3: SerialNumber: 0000:00:1a.0
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
  alloc irq_desc for 21 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
uhci_hcd 0000:00:1a.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
uhci_hcd 0000:00:1a.1: setting latency timer to 64
uhci_hcd 0000:00:1a.1: UHCI Host Controller
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1a.1: irq 21, io base 0x00009480
usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb4: Product: UHCI Host Controller
usb usb4: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 uhci_hcd
usb usb4: SerialNumber: 0000:00:1a.1
usb usb4: configuration #1 chosen from 1 choice
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
uhci_hcd 0000:00:1a.2: PCI INT D -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1a.2: setting latency timer to 64
uhci_hcd 0000:00:1a.2: UHCI Host Controller
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1a.2: irq 19, io base 0x00009800
usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb5: Product: UHCI Host Controller
usb usb5: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 uhci_hcd
usb usb5: SerialNumber: 0000:00:1a.2
usb usb5: configuration #1 chosen from 1 choice
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
uhci_hcd 0000:00:1d.0: setting latency timer to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.0: irq 23, io base 0x00009880
usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb6: Product: UHCI Host Controller
usb usb6: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 uhci_hcd
usb usb6: SerialNumber: 0000:00:1d.0
usb usb6: configuration #1 chosen from 1 choice
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1d.1: setting latency timer to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7
uhci_hcd 0000:00:1d.1: irq 19, io base 0x00009c00
usb usb7: New USB device found, idVendor=1d6b, idProduct=0001
usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb7: Product: UHCI Host Controller
usb usb7: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 uhci_hcd
usb usb7: SerialNumber: 0000:00:1d.1
usb usb7: configuration #1 chosen from 1 choice
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
uhci_hcd 0000:00:1d.2: setting latency timer to 64
uhci_hcd 0000:00:1d.2: UHCI Host Controller
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8
uhci_hcd 0000:00:1d.2: irq 18, io base 0x0000a000
usb usb8: New USB device found, idVendor=1d6b, idProduct=0001
usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb8: Product: UHCI Host Controller
usb usb8: Manufacturer: Linux 2.6.29.1-111.fc11.x86_64 uhci_hcd
usb usb8: SerialNumber: 0000:00:1d.2
usb usb8: configuration #1 chosen from 1 choice
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:03: RTC can wake from S4
rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: dm-devel@redhat.com
cpuidle: using governor ladder
cpuidle: using governor menu
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
nf_conntrack.acct=1 kernel paramater, acct=1 nf_conntrack module option or
sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
registered taskstats version 1
  Magic number: 9:822:724
block loop4: hash matches
Initalizing network drop monitor service
Freeing unused kernel memory: 1312k freed
Write protecting the kernel read-only data: 1844k
usb 2-5: new high speed USB device using ehci_hcd and address 2
usb 2-5: config 1 interface 0 altsetting 0 endpoint 0x81 has an invalid bInterval 255, changing to 11
usb 2-5: New USB device found, idVendor=046b, idProduct=ff01
usb 2-5: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 2-5: Product: Generic Hub
usb 2-5: Manufacturer: American Megatrends Inc.
usb 2-5: SerialNumber: serial
usb 2-5: configuration #1 chosen from 1 choice
hub 2-5:1.0: USB hub found
hub 2-5:1.0: 3 ports detected
EXT4-fs: barriers enabled
kjournald2 starting: pid 253, dev dm-0:8, commit interval 5 seconds
EXT4-fs: delayed allocation enabled
EXT4-fs: file extents enabled
EXT4-fs: mballoc enabled
EXT4-fs: mounted filesystem dm-0 with ordered data mode
SELinux:  Disabled at runtime.
SELinux:  Unregistering netfilter hooks
type=1404 audit(1241426546.983:2): selinux=0 auid=4294967295 ses=4294967295
usb 3-1: new full speed USB device using uhci_hcd and address 2
usb 3-1: New USB device found, idVendor=054c, idProduct=005d
usb 3-1: New USB device strings: Mfr=3, Product=1, SerialNumber=0
usb 3-1: Product: SCE USB Keyboard Hub
usb 3-1: Manufacturer: Sony Computer Entertainment Inc.
usb 3-1: configuration #1 chosen from 1 choice
hub 3-1:1.0: USB hub found
hub 3-1:1.0: 2 ports detected
usb 3-2: new low speed USB device using uhci_hcd and address 3
usb 3-2: New USB device found, idVendor=054c, idProduct=0061
usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 3-2: Product: SCE USB MOUSE
usb 3-2: Manufacturer: Sony Computer Entertainment Inc.
usb 3-2: configuration #1 chosen from 1 choice
input: Sony Computer Entertainment Inc. SCE USB MOUSE as /devices/pci0000:00/0000:00:1a.0/usb3/3-2/3-2:1.0/input/input3
generic-usb 0003:054C:0061.0001: input,hidraw0: USB HID v1.00 Mouse [Sony Computer Entertainment Inc. SCE USB MOUSE] on usb-0000:00:1a.0-2/input0
usb 2-5.1: new high speed USB device using ehci_hcd and address 3
usb 2-5.1: New USB device found, idVendor=046b, idProduct=ff10
usb 2-5.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 2-5.1: Product: Virtual Keyboard and Mouse
usb 2-5.1: Manufacturer: American Megatrends Inc.
usb 2-5.1: SerialNumber: serial
usb 2-5.1: configuration #1 chosen from 1 choice
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1d.7/usb2/2-5/2-5.1/2-5.1:1.0/input/input4
generic-usb 0003:046B:FF10.0002: input,hidraw1: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1d.7-5.1/input0
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1d.7/usb2/2-5/2-5.1/2-5.1:1.1/input/input5
generic-usb 0003:046B:FF10.0003: input,hidraw2: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1d.7-5.1/input1
usb 3-1.1: new full speed USB device using uhci_hcd and address 4
usb 3-1.1: New USB device found, idVendor=054c, idProduct=005c
usb 3-1.1: New USB device strings: Mfr=3, Product=2, SerialNumber=0
usb 3-1.1: Product: SCE USB Keyboard
usb 3-1.1: Manufacturer: Sony Computer Entertainment Inc.
usb 3-1.1: configuration #1 chosen from 1 choice
input: Sony Computer Entertainment Inc. SCE USB Keyboard as /devices/pci0000:00/0000:00:1a.0/usb3/3-1/3-1.1/3-1.1:1.0/input/input6
generic-usb 0003:054C:005C.0004: input,hidraw3: USB HID v1.00 Keyboard [Sony Computer Entertainment Inc. SCE USB Keyboard] on usb-0000:00:1a.0-1.1/input0
udev: starting version 141
input: PC Speaker as /devices/platform/pcspkr/input/input7
i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
iTCO_vendor_support: vendor-support=0
iTCO_wdt: Intel TCO WatchDog Timer Driver v1.05
iTCO_wdt: failed to reset NO_REBOOT flag, reboot disabled by hardware
iTCO_wdt: No card detected
firewire_ohci 0000:01:05.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
e1000e: Copyright (c) 1999-2008 Intel Corporation.
firewire_ohci: Added fw-ohci device 0000:01:05.0, OHCI version 1.10
  alloc irq_desc for 22 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
HDA Intel 0000:00:1b.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
HDA Intel 0000:00:1b.0: setting latency timer to 64
hda_codec: Unknown model for ALC262, trying auto-probe from BIOS...
ALSA sound/pci/hda/hda_codec.c:3503: autoconfig: line_outs=1 (0x14/0x0/0x0/0x0/0x0)
ALSA sound/pci/hda/hda_codec.c:3507:    speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
ALSA sound/pci/hda/hda_codec.c:3511:    hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
ALSA sound/pci/hda/hda_codec.c:3512:    mono: mono_out=0x0
ALSA sound/pci/hda/hda_codec.c:3520:    inputs: mic=0x18, fmic=0x19, line=0x1a, fline=0x0, cd=0x0, aux=0x0
e1000e 0000:03:00.0: Disabling L1 ASPM
e1000e 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
e1000e 0000:03:00.0: setting latency timer to 64
  alloc irq_desc for 55 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
e1000e 0000:03:00.0: irq 55 for MSI/MSI-X
  alloc irq_desc for 56 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
e1000e 0000:03:00.0: irq 56 for MSI/MSI-X
  alloc irq_desc for 57 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
e1000e 0000:03:00.0: irq 57 for MSI/MSI-X
0000:03:00.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b3
0000:03:00.0: eth0: Intel(R) PRO/1000 Network Connection
0000:03:00.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
e1000e 0000:02:00.0: Disabling L1 ASPM
e1000e 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
e1000e 0000:02:00.0: setting latency timer to 64
  alloc irq_desc for 58 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
e1000e 0000:02:00.0: irq 58 for MSI/MSI-X
  alloc irq_desc for 59 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
e1000e 0000:02:00.0: irq 59 for MSI/MSI-X
  alloc irq_desc for 60 on cpu 0 node 0
  alloc kstat_irqs on cpu 0 node 0
e1000e 0000:02:00.0: irq 60 for MSI/MSI-X
0000:02:00.0: eth1: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
0000:02:00.0: eth1: Intel(R) PRO/1000 Network Connection
0000:02:00.0: eth1: MAC: 3, PHY: 8, PBA No: ffffff-0ff
loaded kvm module (kvm-85)
firewire_core: created device fw0: GUID 00e0810000310bb8, S400
device-mapper: multipath: version 1.0.5 loaded
EXT4 FS on dm-0, internal journal on dm-0:8
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
swap_cgroup: uses 39680 bytes of vmalloc for pointer array space and 20316160 bytes to hold mem_cgroup pointers on swap
swap_cgroup can be disabled by noswapaccount boot option.
Adding 10158072k swap on /dev/mapper/vg_barret-lv_swap.  Priority:-1 extents:1 across:10158072k 
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
Microcode Update Driver: v2.00 removed.
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
ip6_tables: (C) 2000-2006 Netfilter Core Team
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
Bluetooth: Core ver 2.15
NET: Registered protocol family 31
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP ver 2.13
Bluetooth: L2CAP socket layer initialized
Bluetooth: BNEP (Ethernet Emulation) ver 1.3
Bluetooth: BNEP filters: protocol multicast
Bridge firewalling registered
Bluetooth: SCO (Voice Link) ver 0.6
Bluetooth: SCO socket layer initialized
virbr0: starting userspace STP failed, starting kernel STP
lo: Disabled Privacy Extensions
JBD: barrier-based sync failed on dm-0:8 - disabling barriers
ADDRCONF(NETDEV_UP): eth1: link is not ready
ADDRCONF(NETDEV_UP): eth0: link is not ready
e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
e1000e: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
virbr0: no IPv6 routers present
eth0: no IPv6 routers present
eth1: no IPv6 routers present
e1000e 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
qemu-system-x86:3361 freeing invalid memtype fb5dc000-fb5dd000
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
qemu-system-x86:3365 freeing invalid memtype fb5dd000-fb5e0000
qemu-system-x86:3365 freeing invalid memtype fb5dd000-fb5e0000
pci-stub 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
qemu-system-x86:3375 freeing invalid memtype fb5dc000-fb5dd000
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
qemu-system-x86:3380 freeing invalid memtype fb5dd000-fb5e0000
qemu-system-x86:3380 freeing invalid memtype fb5dd000-fb5e0000
pci-stub 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
qemu-system-x86:3388 freeing invalid memtype fb5dc000-fb5dd000
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
qemu-system-x86:3390 freeing invalid memtype fb5dd000-fb5e0000
qemu-system-x86:3390 freeing invalid memtype fb5dd000-fb5e0000
pci-stub 0000:02:00.0: PCI INT A disabled
Machine check events logged
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
qemu-system-x86:3413 freeing invalid memtype fb5dc000-fb5dd000
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
qemu-system-x86:3418 freeing invalid memtype fb5dd000-fb5e0000
qemu-system-x86:3418 freeing invalid memtype fb5dd000-fb5e0000
pci-stub 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
qemu-system-x86:3427 freeing invalid memtype fb5dc000-fb5dd000
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X

Re: KVM x86_64 with SR-IOV..?

[Sheng Yang]

On Monday 04 May 2009 17:11:59 Nicholas A. Bellinger wrote:
> On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> > On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger 
wrote:
> > > > > > > Greetings KVM folks,
> > > > > > >
> > > > > > > I wondering if any information exists for doing SR-IOV on the
> > > > > > > new VT-d capable chipsets with KVM..?  From what I understand
> > > > > > > the patches for doing this with KVM are floating around, but I
> > > > > > > have been unable to find any user-level docs for actually
> > > > > > > making it all go against a upstream v2.6.30-rc3 code..
> > > > > > >
> > > > > > > So far I have been doing IOV testing with Xen 3.3 and
> > > > > > > 3.4.0-pre, and I am really hoping to be able to jump to KVM for
> > > > > > > single-function and and then multi-function SR-IOV.  I know
> > > > > > > that the VM migration stuff for IOV in Xen is up and running, 
> > > > > > > and I assume it is being worked in for KVM instance migration
> > > > > > > as well..? This part is less important (at least for me :-)
> > > > > > > than getting a stable SR-IOV setup running under the KVM
> > > > > > > hypervisor..  Does anyone have any pointers for this..?
> > > > > > >
> > > > > > > Any comments or suggestions are appreciated!
> > > > > >
> > > > > > Hi Nicholas
> > > > > >
> > > > > > The patches are not floating around now. As you know, SR-IOV for
> > > > > > Linux have been in 2.6.30, so then you can use upstream KVM and
> > > > > > qemu-kvm(or recent released kvm-85) with 2.6.30-rc3 as host
> > > > > > kernel. And some time ago, there are several SRIOV related
> > > > > > patches for qemu-kvm, and now they all have been checked in.
> > > > > >
> > > > > > And for KVM, the extra document is not necessary, for you can
> > > > > > simple assign a VF to guest like any other devices. And how to
> > > > > > create VF is specific for each device driver. So just create a VF
> > > > > > then assign it to KVM guest is fine.
> > > > >
> > > > > Greetings Sheng,
> > > > >
> > > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I
> > > > > can tell, the KVM_CAP_* defines from libkvm are enabled with
> > > > > building kvm-85 after './configure
> > > > > --kerneldir=/usr/src/linux-2.6.git' and the PCI passthrough code is
> > > > > being enabled in
> > > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > > >
> > > > > >From there, I use the freshly installed qemu-x86_64-system binary
> > > > > > to
> > > > >
> > > > > start a Debian 5 x86_64 HVM (that previously had been moving
> > > > > network packets under Xen for PCIe passthrough). I see the MSI-X
> > > > > interrupt remapping working on the KVM host for the passed
> > > > > -pcidevice, and the MMIO mappings from the qemu build that I also
> > > > > saw while using Xen/qemu-dm built with PCI passthrough are there as
> > > > > well..
> > > >
> > > > Hi Nicholas
> > > >
> > > > > But while the KVM guest is booting, I see the following
> > > > > exception(s) from qemu-x86_64-system for one of the VFs for a
> > > > > multi-function PCIe device:
> > > > >
> > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > >
> > > > This one is mostly harmless.
> > >
> > > Ok, good to know..  :-)
> > >
> > > > > I try with one of the on-board e1000e ports (02:00.0) and I see the
> > > > > same exception along with some MSI-X exceptions from
> > > > > qemu-x86_64-system in KVM guest.. However, I am still able to see
> > > > > the e1000e and the other vxge multi-function device with lspci, but
> > > > > I am unable to dhcp or ping with the e1000e and VF from
> > > > > multi-function device fails to register the MSI-X interrupt in the
> > > > > guest..
> > > >
> > > > Did you see the interrupt in the guest and host side?
> > >
> > > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > > attaching the e1000e single-function device at 02:00.0 to pci-stub
> > > with:
> > >
> > >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> > >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> > >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> > >
> > > I see the following the KVM host kernel ring buffer:
> > >
> > >    e1000e 0000:02:00.0: PCI INT A disabled
> > >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> > >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > >
> > > >  I think you can try on-
> > > > board e1000e for MSI-X first. And please ensure correlated driver
> > > > have been loaded correctly.
> > >
> > > <nod>..
> > >
> > > >  And what do you mean by "some MSI-X exceptions"? Better with
> > > > the log.
> > >
> > > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > > kvm_destroy_phys_mem() statements:
> > >
> > > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid parameters
> > > (slot=-1)
> > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > >
> > > However I still see the following in the KVM guest kernel ring buffer
> > > running v2.6.30-rc in the HVM guest.
> > >
> > > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > (level, high) -> IRQ 10 [    5.525710] e1000e 0000:00:05.0: setting
> > > latency timer to 64
> > > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X
> > > interrupts.  Falling back to MSI interrupts. [    5.527200]
> > > 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts. 
> > > Falling back to legacy interrupts. [    5.829988] 0000:00:05.0: eth0:
> > > (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2 [    5.830672]
> > > 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection [    5.831240]
> > > 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> >
> > Hi Nicholas
> >
> > I think something need to be clarify:
> > 1. For SRIOV, you need 2.6.30 as host kernel... But it's better to know
> > if normal device assignment work in your environment at first.
> > 2. The Fedora's userspace is even more old... You'd better try qemu-kvm
> > upstream, which is more convenient for us to track the problem(and kvm-85
> > is also ok). And as you see above, your QEmu don't support MSI/MSIX...
>
> Ok, got it..
>
> > So you can:
> > 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
>
> Ok, I am now updated on in the FC 11 Host with kvm-85 kernel modules and
> am using the built qemu-system-x86_64 from the kvm-85 source package:
>
> loaded kvm module (kvm-85)
> QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008 Fabrice
> Bellard
>
> > 2. Your host kernel is Fedora 11 Preview, that should be fine with device
> > assignment at first(and let's solve it first, SRIOV the next step).
>
> Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM host for
> the momemt to get e1000e working.  But I will start building a
> v2.6.30-rc3 kernel again for my fc11 host kernel as I do need SR-IOV at
> some point... :-)
>
> > 3. Your KVM version seems like kvm-85, you may provide some dmesg on host
> > side(I think you didn't use the KVM come along with kernel).
>
> Ok, now within the KVM guest running v2.6.29.2, I see the following:
>
> [    2.669243] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
> [    2.672931] e1000e: Copyright (c) 1999-2008 Intel Corporation.
> [    2.674932] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> [    2.675181] 8139too Fast Ethernet driver 0.9.28
> [    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> (level, high) -> IRQ 10
> [    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
> [    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> [    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> [    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> [    2.698039] FDC 0 is a S82078B
> [    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> 00:e0:81:c0:90:b2
> [    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> [    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
>
> And the folllowing from /proc/interrupts inside of the KVM guest:
>
>  24:        117          0          0          0          0          0     
>     0          0          0          0   PCI-MSI-edge      eth1-rx-0 25:   
>       0          0          0          0          0          0          0  
>        0          0          0   PCI-MSI-edge      eth1-tx-0 26:          2
>          0          0          0          0          0          0         
> 0          0          0   PCI-MSI-edge      eth1
>
> ethtool eth1 reports that Link is detected, but I am still unable to get
> a dhcp to work.

It's a little strange that I checked all the log you posted, but can't find 
anything suspicious...(Except you got a MCE log in your dmesg, but I don't 
think it would relate to this).

You also already have interrupts in the guest for eth1-rx-0 and eth1, so at 
least part of interrupts can be delivered to the guest. 

You can try to connect the port to another NIC port directly. Set fixed ip for 
each, then ping each other.

You can also try to disable MSI-X capability in QEmu. Just using "#if 
0/#endif" to wrap "#ifdef KVM_CAP_DEVICE_MSIX/#endif" in 
hw/assigned_device_pci_cap_init(). Then the device would use MSI.

If I am lucky enough to find a 82574L card by hand, I would give it a try...

-- 
regards
Yang, Sheng

>
> However, I do not see the netdev watchdog TX timeouts this time with
> kvm-85, but still no ping to eth1 from KVM guest..
>
> > 4. Your guest kernel is somehow new, maybe you can change it to something
> > like 2.6.29.x which is more stable(but it may not matter). And you are
> > right, no special setup needed for guest kernel (just suppose it's run in
> > a no-VT capability machine).
>
> I will retest with the v2.6.20-rc3 guest and see if MSI-X is able to
> successfully come-up and get dhcp.
>
> > And you can post your _host_ dmesg, as well as the output of debug for
>
> Attached is the dmesg from the 2.6.29.1-111.fc11.x86_64-iov-dmesg KVM
> host machine.
>
> > kvm-85/qemu/hw/device-assignment.c when you don't get response from dhcp.
>
> [root@barret ~]#  qemu-system-x86_64 -m 2048 -smp 10 -pcidevice
> host=02:00.0 lenny64guest1-orig.img init_assigned_device: Registering real
> physical device 02:00.0 (bus=2 dev=0 func=0) get_real_device: region 0 size
> 131072 start 0xfb5e0000 type 512 resource_fd 21 get_real_device: region 2
> size 32 start 0xdc00 type 256 resource_fd 0 get_real_device: region 3 size
> 16384 start 0xfb5dc000 type 512 resource_fd 22
> assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
> assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
> assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
> assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
> assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
> assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
> assigned_dev_pci_read_config: (5.0): address=000a val=0x00000200 len=2
> assigned_dev_pci_read_config: (5.0): address=0000 val=0x00008086 len=2
> assigned_dev_pci_read_config: (5.0): address=0002 val=0x000010d3 len=2
> assigned_dev_pci_write_config: (5.0): address=0010 val=0xffffffff len=4
> assigned_dev_pci_read_config: (5.0): address=0010 val=0xfffe0000 len=4
> assigned_dev_pci_read_config: (5.0): address=0010 val=0xfffe0000 len=4
> assigned_dev_pci_write_config: (5.0): address=0010 val=0xf2020000 len=4
> assigned_dev_iomem_map: e_phys=f2020000 r_virt=0x7f4a24018000 type=0
> len=00020000 region_num=0 assigned_dev_pci_read_config: (5.0): address=0004
> val=0x00000403 len=2 assigned_dev_pci_write_config: (5.0): address=0004
> val=0x00000403 len=2 assigned_dev_pci_write_config: NON BAR (5.0):
> address=0004 val=0x00000403 len=2 assigned_dev_pci_write_config: (5.0):
> address=0014 val=0xffffffff len=4 assigned_dev_pci_read_config: (5.0):
> address=0014 val=0x00000000 len=4 assigned_dev_pci_write_config: (5.0):
> address=0018 val=0xffffffff len=4 assigned_dev_pci_read_config: (5.0):
> address=0018 val=0xffffffe1 len=4 assigned_dev_pci_read_config: (5.0):
> address=0018 val=0xffffffe1 len=4 assigned_dev_pci_write_config: (5.0):
> address=0018 val=0x0000c220 len=4 assigned_dev_ioport_map: e_phys=0xc220
> r_baseport=dc00 type=0x1 len=32 region_num=2 assigned_dev_pci_read_config:
> (5.0): address=0004 val=0x00000403 len=2 assigned_dev_pci_write_config:
> (5.0): address=0004 val=0x00000403 len=2 assigned_dev_pci_write_config: NON
> BAR (5.0): address=0004 val=0x00000403 len=2 assigned_dev_pci_write_config:
> (5.0): address=001c val=0xffffffff len=4 assigned_dev_pci_read_config:
> (5.0): address=001c val=0xffffc000 len=4 assigned_dev_pci_read_config:
> (5.0): address=001c val=0xffffc000 len=4 assigned_dev_pci_write_config:
> (5.0): address=001c val=0xf2040000 len=4 assigned_dev_iomem_map:
> e_phys=f2040000 r_virt=0x7f4a24014000 type=0 len=00004000 region_num=3
> assigned_dev_iomem_map: munmap done, virt_base 0x0x7f4a24014000
> assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000403 len=2
> assigned_dev_pci_write_config: (5.0): address=0004 val=0x00000403 len=2
> assigned_dev_pci_write_config: NON BAR (5.0): address=0004 val=0x00000403
> len=2 assigned_dev_pci_write_config: (5.0): address=0020 val=0xffffffff
> len=4 assigned_dev_pci_read_config: (5.0): address=0020 val=0x00000000
> len=4 assigned_dev_pci_write_config: (5.0): address=0024 val=0xffffffff
> len=4 assigned_dev_pci_read_config: (5.0): address=0024 val=0x00000000
> len=4 assigned_dev_pci_write_config: (5.0): address=0030 val=0xffffffff
> len=4 assigned_dev_pci_write_config: NON BAR (5.0): address=0030
> val=0xffffffff len=4 assigned_dev_pci_read_config: (5.0): address=0030
> val=0x00000000 len=4 assigned_dev_pci_read_config: (5.0): address=003d
> val=0x00000001 len=1 assigned_dev_pci_write_config: (5.0): address=003c
> val=0x0000000a len=1 assigned_dev_pci_read_config: (5.0): address=0000
> val=0x10d38086 len=4 assigned_dev_pci_read_config: (5.0): address=000a
> val=0x00000200 len=2 assigned_dev_pci_read_config: (5.0): address=0000
> val=0x00008086 len=2 assigned_dev_pci_read_config: (5.0): address=0002
> val=0x000010d3 len=2 assigned_dev_pci_read_config: (5.0): address=000e
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0008
> val=0x02000000 len=4 assigned_dev_pci_read_config: (5.0): address=000e
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0000
> val=0x10d38086 len=4 assigned_dev_pci_read_config: (5.0): address=000e
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0008
> val=0x02000000 len=4 assigned_dev_pci_read_config: (5.0): address=003d
> val=0x00000001 len=1 assigned_dev_pci_read_config: (5.0): address=003c
> val=0x0000000a len=1 assigned_dev_pci_read_config: (5.0): address=0010
> val=0xf2020000 len=4 assigned_dev_pci_write_config: (5.0): address=0010
> val=0xffffffff len=4 assigned_dev_pci_read_config: (5.0): address=0010
> val=0xfffe0000 len=4 assigned_dev_pci_write_config: (5.0): address=0010
> val=0xf2020000 len=4 assigned_dev_iomem_map: e_phys=f2020000
> r_virt=0x7f4a24018000 type=0 len=00020000 region_num=0 BUG:
> kvm_destroy_phys_mem: invalid parameters (slot=-1)
> assigned_dev_pci_read_config: (5.0): address=0014 val=0x00000000 len=4
> assigned_dev_pci_write_config: (5.0): address=0014 val=0xffffffff len=4
> assigned_dev_pci_read_config: (5.0): address=0014 val=0x00000000 len=4
> assigned_dev_pci_write_config: (5.0): address=0014 val=0x00000000 len=4
> assigned_dev_pci_read_config: (5.0): address=0018 val=0x0000c221 len=4
> assigned_dev_pci_write_config: (5.0): address=0018 val=0xffffffff len=4
> assigned_dev_pci_read_config: (5.0): address=0018 val=0xffffffe1 len=4
> assigned_dev_pci_write_config: (5.0): address=0018 val=0x0000c221 len=4
> assigned_dev_ioport_map: e_phys=0xc220 r_baseport=dc00 type=0x1 len=32
> region_num=2 assigned_dev_pci_read_config: (5.0): address=001c
> val=0xf2040000 len=4 assigned_dev_pci_write_config: (5.0): address=001c
> val=0xffffffff len=4 assigned_dev_pci_read_config: (5.0): address=001c
> val=0xffffc000 len=4 assigned_dev_pci_write_config: (5.0): address=001c
> val=0xf2040000 len=4 assigned_dev_iomem_map: e_phys=f2040000
> r_virt=0x7f4a24014000 type=0 len=00004000 region_num=3 BUG:
> kvm_destroy_phys_mem: invalid parameters (slot=-1)
> assigned_dev_iomem_map: munmap done, virt_base 0x0x7f4a24014000
> assigned_dev_pci_read_config: (5.0): address=0020 val=0x00000000 len=4
> assigned_dev_pci_write_config: (5.0): address=0020 val=0xffffffff len=4
> assigned_dev_pci_read_config: (5.0): address=0020 val=0x00000000 len=4
> assigned_dev_pci_write_config: (5.0): address=0020 val=0x00000000 len=4
> assigned_dev_pci_read_config: (5.0): address=0024 val=0x00000000 len=4
> assigned_dev_pci_write_config: (5.0): address=0024 val=0xffffffff len=4
> assigned_dev_pci_read_config: (5.0): address=0024 val=0x00000000 len=4
> assigned_dev_pci_write_config: (5.0): address=0024 val=0x00000000 len=4
> assigned_dev_pci_read_config: (5.0): address=0030 val=0x00000000 len=4
> assigned_dev_pci_write_config: (5.0): address=0030 val=0xfffffffe len=4
> assigned_dev_pci_write_config: NON BAR (5.0): address=0030 val=0xfffffffe
> len=4 assigned_dev_pci_read_config: (5.0): address=0030 val=0x00000000
> len=4 assigned_dev_pci_write_config: (5.0): address=0030 val=0x00000000
> len=4 assigned_dev_pci_write_config: NON BAR (5.0): address=0030
> val=0x00000000 len=4 assigned_dev_pci_read_config: (5.0): address=002c
> val=0x00008086 len=2 assigned_dev_pci_read_config: (5.0): address=002e
> val=0x00000000 len=2 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0004
> val=0x00000403 len=2 assigned_dev_pci_read_config: (5.0): address=0004
> val=0x00000403 len=2 assigned_dev_pci_read_config: (5.0): address=0006
> val=0x00000010 len=2 assigned_dev_pci_read_config: (5.0): address=0034
> val=0x00000040 len=1 assigned_dev_pci_read_config: (5.0): address=0040
> val=0x00000005 len=1 assigned_dev_pci_read_config: (5.0): address=0041
> val=0x00000050 len=1 assigned_dev_pci_read_config: (5.0): address=0050
> val=0x00000011 len=1 assigned_dev_pci_read_config: (5.0): address=0051
> val=0x00000000 len=1 assigned_dev_pci_read_config: (5.0): address=0010
> val=0x00000000 len=2 assigned_dev_pci_read_config: (5.0): address=0004
> val=0x00000403 len=2 assigned_dev_pci_read_config: (5.0): address=0004
> val=0x00000403 len=2 assigned_dev_pci_write_config: (5.0): address=0004
> val=0x00000407 len=2 assigned_dev_pci_write_config: NON BAR (5.0):
> address=0004 val=0x00000407 len=2 assigned_dev_pci_read_config: (5.0):
> address=000d val=0x00000000 len=1 assigned_dev_pci_write_config: (5.0):
> address=000d val=0x00000040 len=1 assigned_dev_pci_write_config: NON BAR
> (5.0): address=000d val=0x00000040 len=1 assigned_dev_pci_read_config:
> (5.0): address=0000 val=0x10d38086 len=4 assigned_dev_pci_read_config:
> (5.0): address=0004 val=0x00100407 len=4 assigned_dev_pci_read_config:
> (5.0): address=0008 val=0x02000000 len=4 assigned_dev_pci_read_config:
> (5.0): address=000c val=0x00000010 len=4 assigned_dev_pci_read_config:
> (5.0): address=0010 val=0xf2020000 len=4 assigned_dev_pci_read_config:
> (5.0): address=0014 val=0x00000000 len=4 assigned_dev_pci_read_config:
> (5.0): address=0018 val=0x0000c221 len=4 assigned_dev_pci_read_config:
> (5.0): address=001c val=0xf2040000 len=4 assigned_dev_pci_read_config:
> (5.0): address=0020 val=0x00000000 len=4 assigned_dev_pci_read_config:
> (5.0): address=0024 val=0x00000000 len=4 assigned_dev_pci_read_config:
> (5.0): address=0028 val=0x00000000 len=4 assigned_dev_pci_read_config:
> (5.0): address=002c val=0x00008086 len=4 assigned_dev_pci_read_config:
> (5.0): address=0030 val=0x00000000 len=4 assigned_dev_pci_read_config:
> (5.0): address=0034 val=0x00000040 len=4 assigned_dev_pci_read_config:
> (5.0): address=0038 val=0x00000000 len=4 assigned_dev_pci_read_config:
> (5.0): address=003c val=0x0000010a len=4 assigned_dev_pci_read_config:
> (5.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config:
> (5.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config:
> (5.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config:
> (5.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config:
> (5.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config:
> (5.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config:
> (5.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config:
> (5.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config:
> (5.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config:
> (5.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config:
> (5.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config:
> (5.0): address=0051 val=0x00000000 len=1 assigned_dev_pci_read_config:
> (5.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config:
> (5.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config:
> (5.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config:
> (5.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config:
> (5.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config:
> (5.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config:
> (5.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config:
> (5.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config:
> (5.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config:
> (5.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config:
> (5.0): address=0052 val=0x00000004 len=2 assigned_dev_pci_read_config:
> (5.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config:
> (5.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config:
> (5.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config:
> (5.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config:
> (5.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config:
> (5.0): address=0052 val=0x00000004 len=2 assigned_dev_pci_write_config:
> (5.0): address=0052 val=0x00000004 len=2 assigned_dev_pci_read_config:
> (5.0): address=0006 val=0x00000010 len=2 assigned_dev_pci_read_config:
> (5.0): address=0034 val=0x00000040 len=1 assigned_dev_pci_read_config:
> (5.0): address=0040 val=0x00000005 len=1 assigned_dev_pci_read_config:
> (5.0): address=0041 val=0x00000050 len=1 assigned_dev_pci_read_config:
> (5.0): address=0050 val=0x00000011 len=1 assigned_dev_pci_read_config:
> (5.0): address=0052 val=0x00000004 len=2 assigned_dev_pci_read_config:
> (5.0): address=0054 val=0x00000003 len=4 msix_mmio_writel: write to MSI-X
> entry table mmio offset 0x0, val 0xfee00000 msix_mmio_writel: write to
> MSI-X entry table mmio offset 0x4, val 0x0 msix_mmio_writel: write to MSI-X
> entry table mmio offset 0x8, val 0x4049 msix_mmio_writel: write to MSI-X
> entry table mmio offset 0x10, val 0xfee00000 msix_mmio_writel: write to
> MSI-X entry table mmio offset 0x14, val 0x0 msix_mmio_writel: write to
> MSI-X entry table mmio offset 0x18, val 0x4051 msix_mmio_writel: write to
> MSI-X entry table mmio offset 0x20, val 0xfee00000 msix_mmio_writel: write
> to MSI-X entry table mmio offset 0x24, val 0x0 msix_mmio_writel: write to
> MSI-X entry table mmio offset 0x28, val 0x4059
> assigned_dev_pci_read_config: (5.0): address=0004 val=0x00000007 len=2
> assigned_dev_pci_write_config: (5.0): address=0004 val=0x00000407 len=2
> assigned_dev_pci_write_config: NON BAR (5.0): address=0004 val=0x00000407
> len=2 assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010
> len=2 assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040
> len=1 assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005
> len=1 assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050
> len=1 assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011
> len=1 assigned_dev_pci_read_config: (5.0): address=0052 val=0x00000004
> len=2 assigned_dev_pci_write_config: (5.0): address=0052 val=0x00008004
> len=2 assigned_dev_update_msix_mmio: MSI-X data 0x4049, MSI-X addr_lo
> 0xfee00000 !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x18, entry 0
> !assigned_dev_update_msix_mmio: MSI-X data 0x4051, MSI-X addr_lo 0xfee00000
> !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x19, entry 1
> !assigned_dev_update_msix_mmio: MSI-X data 0x4059, MSI-X addr_lo 0xfee00000
> !assigned_dev_update_msix_mmio: MSI-X entry gsi 0x1a, entry 2
> !assigned_dev_pci_read_config: (5.0): address=0006 val=0x00000010 len=2
> assigned_dev_pci_read_config: (5.0): address=0034 val=0x00000040 len=1
> assigned_dev_pci_read_config: (5.0): address=0040 val=0x00000005 len=1
> assigned_dev_pci_read_config: (5.0): address=0041 val=0x00000050 len=1
> assigned_dev_pci_read_config: (5.0): address=0050 val=0x00000011 len=1
> assigned_dev_pci_read_config: (5.0): address=0051 val=0x00000000 len=1
> assigned_dev_pci_read_config: (5.0): address=000e val=0x00000000 len=2
> msix_mmio_writel: write to MSI-X entry table mmio offset 0xc, val 0x0
> msix_mmio_writel: write to MSI-X entry table mmio offset 0x0, val
> 0xfee00000 msix_mmio_writel: write to MSI-X entry table mmio offset 0x4,
> val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x8, val
> 0x4049 msix_mmio_writel: write to MSI-X entry table mmio offset 0x1c, val
> 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x10, val
> 0xfee00000 msix_mmio_writel: write to MSI-X entry table mmio offset 0x14,
> val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x18, val
> 0x4051 msix_mmio_writel: write to MSI-X entry table mmio offset 0x2c, val
> 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x20, val
> 0xfee00000 msix_mmio_writel: write to MSI-X entry table mmio offset 0x24,
> val 0x0 msix_mmio_writel: write to MSI-X entry table mmio offset 0x28, val
> 0x4059
>
> > Unhandled MSR doesn't matter - and I thought they have already been
> > discard long ago?
> >
> > I saw the interrupt of host assigned device seems fine, you can also
> > check if guest have interrupt for that device.
> >
> > Thanks.
>
> Thanks!
>
> --nab

Re: KVM x86_64 with SR-IOV..?

[Nicholas A. Bellinger]

On Mon, 2009-05-04 at 17:49 +0800, Sheng Yang wrote:
> On Monday 04 May 2009 17:11:59 Nicholas A. Bellinger wrote:
> > On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> > > On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > > > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger 
> wrote:
> > > > > > > > Greetings KVM folks,
> > > > > > > >
> > > > > > > > I wondering if any information exists for doing SR-IOV on the
> > > > > > > > new VT-d capable chipsets with KVM..?  From what I understand
> > > > > > > > the patches for doing this with KVM are floating around, but I
> > > > > > > > have been unable to find any user-level docs for actually
> > > > > > > > making it all go against a upstream v2.6.30-rc3 code..
> > > > > > > >
> > > > > > > > So far I have been doing IOV testing with Xen 3.3 and
> > > > > > > > 3.4.0-pre, and I am really hoping to be able to jump to KVM for
> > > > > > > > single-function and and then multi-function SR-IOV.  I know
> > > > > > > > that the VM migration stuff for IOV in Xen is up and running, 
> > > > > > > > and I assume it is being worked in for KVM instance migration
> > > > > > > > as well..? This part is less important (at least for me :-)
> > > > > > > > than getting a stable SR-IOV setup running under the KVM
> > > > > > > > hypervisor..  Does anyone have any pointers for this..?
> > > > > > > >
> > > > > > > > Any comments or suggestions are appreciated!
> > > > > > >
> > > > > > > Hi Nicholas
> > > > > > >
> > > > > > > The patches are not floating around now. As you know, SR-IOV for
> > > > > > > Linux have been in 2.6.30, so then you can use upstream KVM and
> > > > > > > qemu-kvm(or recent released kvm-85) with 2.6.30-rc3 as host
> > > > > > > kernel. And some time ago, there are several SRIOV related
> > > > > > > patches for qemu-kvm, and now they all have been checked in.
> > > > > > >
> > > > > > > And for KVM, the extra document is not necessary, for you can
> > > > > > > simple assign a VF to guest like any other devices. And how to
> > > > > > > create VF is specific for each device driver. So just create a VF
> > > > > > > then assign it to KVM guest is fine.
> > > > > >
> > > > > > Greetings Sheng,
> > > > > >
> > > > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I
> > > > > > can tell, the KVM_CAP_* defines from libkvm are enabled with
> > > > > > building kvm-85 after './configure
> > > > > > --kerneldir=/usr/src/linux-2.6.git' and the PCI passthrough code is
> > > > > > being enabled in
> > > > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > > > >
> > > > > > >From there, I use the freshly installed qemu-x86_64-system binary
> > > > > > > to
> > > > > >
> > > > > > start a Debian 5 x86_64 HVM (that previously had been moving
> > > > > > network packets under Xen for PCIe passthrough). I see the MSI-X
> > > > > > interrupt remapping working on the KVM host for the passed
> > > > > > -pcidevice, and the MMIO mappings from the qemu build that I also
> > > > > > saw while using Xen/qemu-dm built with PCI passthrough are there as
> > > > > > well..
> > > > >
> > > > > Hi Nicholas
> > > > >
> > > > > > But while the KVM guest is booting, I see the following
> > > > > > exception(s) from qemu-x86_64-system for one of the VFs for a
> > > > > > multi-function PCIe device:
> > > > > >
> > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > >
> > > > > This one is mostly harmless.
> > > >
> > > > Ok, good to know..  :-)
> > > >
> > > > > > I try with one of the on-board e1000e ports (02:00.0) and I see the
> > > > > > same exception along with some MSI-X exceptions from
> > > > > > qemu-x86_64-system in KVM guest.. However, I am still able to see
> > > > > > the e1000e and the other vxge multi-function device with lspci, but
> > > > > > I am unable to dhcp or ping with the e1000e and VF from
> > > > > > multi-function device fails to register the MSI-X interrupt in the
> > > > > > guest..
> > > > >
> > > > > Did you see the interrupt in the guest and host side?
> > > >
> > > > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > > > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > > > attaching the e1000e single-function device at 02:00.0 to pci-stub
> > > > with:
> > > >
> > > >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> > > >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> > > >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> > > >
> > > > I see the following the KVM host kernel ring buffer:
> > > >
> > > >    e1000e 0000:02:00.0: PCI INT A disabled
> > > >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> > > >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > > >
> > > > >  I think you can try on-
> > > > > board e1000e for MSI-X first. And please ensure correlated driver
> > > > > have been loaded correctly.
> > > >
> > > > <nod>..
> > > >
> > > > >  And what do you mean by "some MSI-X exceptions"? Better with
> > > > > the log.
> > > >
> > > > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > > > kvm_destroy_phys_mem() statements:
> > > >
> > > > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > > > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid parameters
> > > > (slot=-1)
> > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > >
> > > > However I still see the following in the KVM guest kernel ring buffer
> > > > running v2.6.30-rc in the HVM guest.
> > > >
> > > > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > > (level, high) -> IRQ 10 [    5.525710] e1000e 0000:00:05.0: setting
> > > > latency timer to 64
> > > > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X
> > > > interrupts.  Falling back to MSI interrupts. [    5.527200]
> > > > 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts. 
> > > > Falling back to legacy interrupts. [    5.829988] 0000:00:05.0: eth0:
> > > > (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2 [    5.830672]
> > > > 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection [    5.831240]
> > > > 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > >
> > > Hi Nicholas
> > >
> > > I think something need to be clarify:
> > > 1. For SRIOV, you need 2.6.30 as host kernel... But it's better to know
> > > if normal device assignment work in your environment at first.
> > > 2. The Fedora's userspace is even more old... You'd better try qemu-kvm
> > > upstream, which is more convenient for us to track the problem(and kvm-85
> > > is also ok). And as you see above, your QEmu don't support MSI/MSIX...
> >
> > Ok, got it..
> >
> > > So you can:
> > > 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
> >
> > Ok, I am now updated on in the FC 11 Host with kvm-85 kernel modules and
> > am using the built qemu-system-x86_64 from the kvm-85 source package:
> >
> > loaded kvm module (kvm-85)
> > QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008 Fabrice
> > Bellard
> >
> > > 2. Your host kernel is Fedora 11 Preview, that should be fine with device
> > > assignment at first(and let's solve it first, SRIOV the next step).
> >
> > Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM host for
> > the momemt to get e1000e working.  But I will start building a
> > v2.6.30-rc3 kernel again for my fc11 host kernel as I do need SR-IOV at
> > some point... :-)
> >
> > > 3. Your KVM version seems like kvm-85, you may provide some dmesg on host
> > > side(I think you didn't use the KVM come along with kernel).
> >
> > Ok, now within the KVM guest running v2.6.29.2, I see the following:
> >
> > [    2.669243] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
> > [    2.672931] e1000e: Copyright (c) 1999-2008 Intel Corporation.
> > [    2.674932] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > [    2.675181] 8139too Fast Ethernet driver 0.9.28
> > [    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > (level, high) -> IRQ 10
> > [    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
> > [    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> > [    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> > [    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> > [    2.698039] FDC 0 is a S82078B
> > [    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > 00:e0:81:c0:90:b2
> > [    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> > [    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> >
> > And the folllowing from /proc/interrupts inside of the KVM guest:
> >
> >  24:        117          0          0          0          0          0     
> >     0          0          0          0   PCI-MSI-edge      eth1-rx-0 25:   
> >       0          0          0          0          0          0          0  
> >        0          0          0   PCI-MSI-edge      eth1-tx-0 26:          2
> >          0          0          0          0          0          0         
> > 0          0          0   PCI-MSI-edge      eth1
> >
> > ethtool eth1 reports that Link is detected, but I am still unable to get
> > a dhcp to work.
> 
> It's a little strange that I checked all the log you posted, but can't find 
> anything suspicious...(Except you got a MCE log in your dmesg, but I don't 
> think it would relate to this).
> 
> You also already have interrupts in the guest for eth1-rx-0 and eth1, so at 
> least part of interrupts can be delivered to the guest. 
> 
> You can try to connect the port to another NIC port directly. Set fixed ip for 
> each, then ping each other.
> 
> You can also try to disable MSI-X capability in QEmu. Just using "#if 
> 0/#endif" to wrap "#ifdef KVM_CAP_DEVICE_MSIX/#endif" in 
> hw/assigned_device_pci_cap_init(). Then the device would use MSI.
> 
> If I am lucky enough to find a 82574L card by hand, I would give it a try...
> 
> -- 
> regards
> Yang, Sheng
> 

Greetings Sheng,

So I updated my FC11 Host to kernel v2.6.30-rc3 (and enabled ext4 of
course) and rebuilt the kvm-85 source kernel module and
qemu-system-x86_64 and I am now able to get dhcp and IP ops from the
02:00.0 device on my IOH-5520 board with the KVM guest using a v2.6.29.2
kernel!!  Everything is looking good with the v2.6.29.2, but after a
quick reboot back into my v2.6.30-rc3 KVM guest kernel build e1000e it
looks like I am unable to get dhcp.

Rebooting back into KVM Guest kernel v2.6.29.2 brings the pci-stub
assigned e1000e 82574L assigned with dhcp and everything looks
good!  :-)

I will keep poking at the v2.6.30-rc KVM guests (I am going to do a
complete rebuild) and see if it does not start move IP packets as well..

Thanks for all of your help in getting setup! 

--nab

Re: KVM x86_64 with SR-IOV..?

[Yu Zhao]

Hi,

The VF also works in the host if the VF driver is programed properly.
So it would be easier to develop the VF driver in the host and then
verify the VF driver in the guest.

BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
the CONFIG_PCI_IOV in the kernel .config?

Thanks,
Yu

On Mon, May 04, 2009 at 06:40:36PM +0800, Nicholas A. Bellinger wrote:
> On Mon, 2009-05-04 at 17:49 +0800, Sheng Yang wrote:
> > On Monday 04 May 2009 17:11:59 Nicholas A. Bellinger wrote:
> > > On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> > > > On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > > > > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger
> > wrote:
> > > > > > > > > Greetings KVM folks,
> > > > > > > > >
> > > > > > > > > I wondering if any information exists for doing SR-IOV on the
> > > > > > > > > new VT-d capable chipsets with KVM..?  From what I understand
> > > > > > > > > the patches for doing this with KVM are floating around, but I
> > > > > > > > > have been unable to find any user-level docs for actually
> > > > > > > > > making it all go against a upstream v2.6.30-rc3 code..
> > > > > > > > >
> > > > > > > > > So far I have been doing IOV testing with Xen 3.3 and
> > > > > > > > > 3.4.0-pre, and I am really hoping to be able to jump to KVM for
> > > > > > > > > single-function and and then multi-function SR-IOV.  I know
> > > > > > > > > that the VM migration stuff for IOV in Xen is up and running,
> > > > > > > > > and I assume it is being worked in for KVM instance migration
> > > > > > > > > as well..? This part is less important (at least for me :-)
> > > > > > > > > than getting a stable SR-IOV setup running under the KVM
> > > > > > > > > hypervisor..  Does anyone have any pointers for this..?
> > > > > > > > >
> > > > > > > > > Any comments or suggestions are appreciated!
> > > > > > > >
> > > > > > > > Hi Nicholas
> > > > > > > >
> > > > > > > > The patches are not floating around now. As you know, SR-IOV for
> > > > > > > > Linux have been in 2.6.30, so then you can use upstream KVM and
> > > > > > > > qemu-kvm(or recent released kvm-85) with 2.6.30-rc3 as host
> > > > > > > > kernel. And some time ago, there are several SRIOV related
> > > > > > > > patches for qemu-kvm, and now they all have been checked in.
> > > > > > > >
> > > > > > > > And for KVM, the extra document is not necessary, for you can
> > > > > > > > simple assign a VF to guest like any other devices. And how to
> > > > > > > > create VF is specific for each device driver. So just create a VF
> > > > > > > > then assign it to KVM guest is fine.
> > > > > > >
> > > > > > > Greetings Sheng,
> > > > > > >
> > > > > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I
> > > > > > > can tell, the KVM_CAP_* defines from libkvm are enabled with
> > > > > > > building kvm-85 after './configure
> > > > > > > --kerneldir=/usr/src/linux-2.6.git' and the PCI passthrough code is
> > > > > > > being enabled in
> > > > > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > > > > >
> > > > > > > >From there, I use the freshly installed qemu-x86_64-system binary
> > > > > > > > to
> > > > > > >
> > > > > > > start a Debian 5 x86_64 HVM (that previously had been moving
> > > > > > > network packets under Xen for PCIe passthrough). I see the MSI-X
> > > > > > > interrupt remapping working on the KVM host for the passed
> > > > > > > -pcidevice, and the MMIO mappings from the qemu build that I also
> > > > > > > saw while using Xen/qemu-dm built with PCI passthrough are there as
> > > > > > > well..
> > > > > >
> > > > > > Hi Nicholas
> > > > > >
> > > > > > > But while the KVM guest is booting, I see the following
> > > > > > > exception(s) from qemu-x86_64-system for one of the VFs for a
> > > > > > > multi-function PCIe device:
> > > > > > >
> > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > >
> > > > > > This one is mostly harmless.
> > > > >
> > > > > Ok, good to know..  :-)
> > > > >
> > > > > > > I try with one of the on-board e1000e ports (02:00.0) and I see the
> > > > > > > same exception along with some MSI-X exceptions from
> > > > > > > qemu-x86_64-system in KVM guest.. However, I am still able to see
> > > > > > > the e1000e and the other vxge multi-function device with lspci, but
> > > > > > > I am unable to dhcp or ping with the e1000e and VF from
> > > > > > > multi-function device fails to register the MSI-X interrupt in the
> > > > > > > guest..
> > > > > >
> > > > > > Did you see the interrupt in the guest and host side?
> > > > >
> > > > > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > > > > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > > > > attaching the e1000e single-function device at 02:00.0 to pci-stub
> > > > > with:
> > > > >
> > > > >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> > > > >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> > > > >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> > > > >
> > > > > I see the following the KVM host kernel ring buffer:
> > > > >
> > > > >    e1000e 0000:02:00.0: PCI INT A disabled
> > > > >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> > > > >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > > > >
> > > > > >  I think you can try on-
> > > > > > board e1000e for MSI-X first. And please ensure correlated driver
> > > > > > have been loaded correctly.
> > > > >
> > > > > <nod>..
> > > > >
> > > > > >  And what do you mean by "some MSI-X exceptions"? Better with
> > > > > > the log.
> > > > >
> > > > > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > > > > kvm_destroy_phys_mem() statements:
> > > > >
> > > > > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > > > > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid parameters
> > > > > (slot=-1)
> > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > >
> > > > > However I still see the following in the KVM guest kernel ring buffer
> > > > > running v2.6.30-rc in the HVM guest.
> > > > >
> > > > > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > > > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > > > (level, high) -> IRQ 10 [    5.525710] e1000e 0000:00:05.0: setting
> > > > > latency timer to 64
> > > > > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X
> > > > > interrupts.  Falling back to MSI interrupts. [    5.527200]
> > > > > 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts.
> > > > > Falling back to legacy interrupts. [    5.829988] 0000:00:05.0: eth0:
> > > > > (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2 [    5.830672]
> > > > > 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection [    5.831240]
> > > > > 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > > >
> > > > Hi Nicholas
> > > >
> > > > I think something need to be clarify:
> > > > 1. For SRIOV, you need 2.6.30 as host kernel... But it's better to know
> > > > if normal device assignment work in your environment at first.
> > > > 2. The Fedora's userspace is even more old... You'd better try qemu-kvm
> > > > upstream, which is more convenient for us to track the problem(and kvm-85
> > > > is also ok). And as you see above, your QEmu don't support MSI/MSIX...
> > >
> > > Ok, got it..
> > >
> > > > So you can:
> > > > 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
> > >
> > > Ok, I am now updated on in the FC 11 Host with kvm-85 kernel modules and
> > > am using the built qemu-system-x86_64 from the kvm-85 source package:
> > >
> > > loaded kvm module (kvm-85)
> > > QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008 Fabrice
> > > Bellard
> > >
> > > > 2. Your host kernel is Fedora 11 Preview, that should be fine with device
> > > > assignment at first(and let's solve it first, SRIOV the next step).
> > >
> > > Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM host for
> > > the momemt to get e1000e working.  But I will start building a
> > > v2.6.30-rc3 kernel again for my fc11 host kernel as I do need SR-IOV at
> > > some point... :-)
> > >
> > > > 3. Your KVM version seems like kvm-85, you may provide some dmesg on host
> > > > side(I think you didn't use the KVM come along with kernel).
> > >
> > > Ok, now within the KVM guest running v2.6.29.2, I see the following:
> > >
> > > [    2.669243] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
> > > [    2.672931] e1000e: Copyright (c) 1999-2008 Intel Corporation.
> > > [    2.674932] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > [    2.675181] 8139too Fast Ethernet driver 0.9.28
> > > [    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > (level, high) -> IRQ 10
> > > [    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
> > > [    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> > > [    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> > > [    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> > > [    2.698039] FDC 0 is a S82078B
> > > [    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > > 00:e0:81:c0:90:b2
> > > [    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> > > [    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > >
> > > And the folllowing from /proc/interrupts inside of the KVM guest:
> > >
> > >  24:        117          0          0          0          0          0
> > >     0          0          0          0   PCI-MSI-edge      eth1-rx-0 25:
> > >       0          0          0          0          0          0          0
> > >        0          0          0   PCI-MSI-edge      eth1-tx-0 26:          2
> > >          0          0          0          0          0          0
> > > 0          0          0   PCI-MSI-edge      eth1
> > >
> > > ethtool eth1 reports that Link is detected, but I am still unable to get
> > > a dhcp to work.
> >
> > It's a little strange that I checked all the log you posted, but can't find
> > anything suspicious...(Except you got a MCE log in your dmesg, but I don't
> > think it would relate to this).
> >
> > You also already have interrupts in the guest for eth1-rx-0 and eth1, so at
> > least part of interrupts can be delivered to the guest.
> >
> > You can try to connect the port to another NIC port directly. Set fixed ip for
> > each, then ping each other.
> >
> > You can also try to disable MSI-X capability in QEmu. Just using "#if
> > 0/#endif" to wrap "#ifdef KVM_CAP_DEVICE_MSIX/#endif" in
> > hw/assigned_device_pci_cap_init(). Then the device would use MSI.
> >
> > If I am lucky enough to find a 82574L card by hand, I would give it a try...
> >
> > --
> > regards
> > Yang, Sheng
> >
> 
> Greetings Sheng,
> 
> So I updated my FC11 Host to kernel v2.6.30-rc3 (and enabled ext4 of
> course) and rebuilt the kvm-85 source kernel module and
> qemu-system-x86_64 and I am now able to get dhcp and IP ops from the
> 02:00.0 device on my IOH-5520 board with the KVM guest using a v2.6.29.2
> kernel!!  Everything is looking good with the v2.6.29.2, but after a
> quick reboot back into my v2.6.30-rc3 KVM guest kernel build e1000e it
> looks like I am unable to get dhcp.
> 
> Rebooting back into KVM Guest kernel v2.6.29.2 brings the pci-stub
> assigned e1000e 82574L assigned with dhcp and everything looks
> good!  :-)
> 
> I will keep poking at the v2.6.30-rc KVM guests (I am going to do a
> complete rebuild) and see if it does not start move IP packets as well..
> 
> Thanks for all of your help in getting setup!
> 
> --nab
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: KVM x86_64 with SR-IOV..? (device passthrough with LIO-Target v3.0)

[Nicholas A. Bellinger]

[-- Attachment #1: Type: text/plain, Size: 18100 bytes --]

On Tue, 2009-05-05 at 09:42 +0800, Yu Zhao wrote:
> Hi,
> 
> The VF also works in the host if the VF driver is programed properly.
> So it would be easier to develop the VF driver in the host and then
> verify the VF driver in the guest.
> 
> BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
> the CONFIG_PCI_IOV in the kernel .config?
> 
> Thanks,
> Yu
> 

Greetings Yu and Sheng,

So the original attachment was for the v2.6.29-fc11 host kernel output,
I ended up jumping to v2.6.30-rc3 (and making sure CONFIG_PCI_IOV was
enabled) for KVM host with kvm-85 and now things are looking quite
stable for me.

So far I have been able to successfully push LIO-Target v3.0 traffic
*inside* a v2.6.29.2 KVM guest via the onboard e1000e (02:00.0) port
from another Linux/iSCSI Initiator machine using a Intel 1 Gb/sec port.
I am running badblocks tests to iSCSI Logical Units for RAMDISK_DR and
FILEIO storage objects (in the KVM Guest), and they are passing
validation and I am seeing ~500 Mb/sec of throughput and very low CPU
usage in the KVM guests.

One issue I did notice while using the pci-stub method of
device-assignment with same e1000 port (02:00.0) was while using an
iSCSI Initiator (Open-iSCSI) on the KVM Host machine and doing sustained
traffic into the LIO-Target KVM Guest on the same local KVM host to max
out traffic between the other onboard e1000e port (03.00.0), I see the
following:

pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
assign device: host bdf = 2:0:0
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
scsi4 : iSCSI Initiator over TCP/IP
scsi 4:0:0:0: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI: 5
sd 4:0:0:0: Attached scsi generic sg1 type 0
scsi 4:0:0:1: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI: 5
sd 4:0:0:1: Attached scsi generic sg2 type 0
sd 4:0:0:0: [sdb] 262144 512-byte hardware sectors: (134 MB/128 MiB)
sd 4:0:0:1: [sdc] 262144 512-byte hardware sectors: (134 MB/128 MiB)
sd 4:0:0:0: [sdb] Write Protect is off
sd 4:0:0:0: [sdb] Mode Sense: 2f 00 00 00
sd 4:0:0:1: [sdc] Write Protect is off
sd 4:0:0:1: [sdc] Mode Sense: 2f 00 00 00
sd 4:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 4:0:0:1: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sdb:<6> sdc: unknown partition table
sd 4:0:0:0: [sdb] Attached SCSI disk
 unknown partition table
sd 4:0:0:1: [sdc] Attached SCSI disk
------------[ cut here ]------------
WARNING: at kernel/irq/manage.c:260 enable_irq+0x36/0x50()
Hardware name: empty
Unbalanced enable for IRQ 59
Modules linked in: ipt_REJECT xt_tcpudp bridge stp sunrpc iptable_filter ip_tables xt_state nf_conntrack ip6table_filter ip6_tables x_tables ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table ext3 jbd loop dm_multipath scsi_dh kvm_intel kvm uinput i2c_i801 firewire_ohci joydev firewire_core sg i2c_core 8250_pnp crc_itu_t e1000e 8250 serial_core rtc_cmos pcspkr serio_raw rtc_core rtc_lib button sd_mod dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod uhci_hcd ohci_hcd ehci_hcd ata_piix libata scsi_mod [last unloaded: microcode]
Pid: 51, comm: events/0 Tainted: G        W  2.6.30-rc3 #11
Call Trace:
 [<ffffffff80235fee>] ? warn_slowpath+0xcb/0xe8
 [<ffffffff80253a7c>] ? generic_exec_single+0x6a/0x88
 [<ffffffff8022acec>] ? update_curr+0x67/0xeb
 [<ffffffffa0198748>] ? vcpu_kick_intr+0x0/0x1 [kvm]
 [<ffffffff8020a5d8>] ? __switch_to+0xb6/0x274
 [<ffffffff8022b70a>] ? __dequeue_entity+0x1b/0x2f
 [<ffffffffa01ac7e4>] ? kvm_irq_delivery_to_apic+0xb3/0xf7 [kvm]
 [<ffffffffa01aa4d4>] ? __apic_accept_irq+0x15a/0x173 [kvm]
 [<ffffffffa01ac883>] ? kvm_set_msi+0x5b/0x60 [kvm]
 [<ffffffff80266d97>] ? enable_irq+0x36/0x50
 [<ffffffffa0195ab5>] ? kvm_assigned_dev_interrupt_work_handler+0x6d/0xbc [kvm]
 [<ffffffff802449fa>] ? worker_thread+0x182/0x223
 [<ffffffff8024820b>] ? autoremove_wake_function+0x0/0x2a
 [<ffffffff80244878>] ? worker_thread+0x0/0x223
 [<ffffffff80244878>] ? worker_thread+0x0/0x223
 [<ffffffff80247e72>] ? kthread+0x54/0x7e
 [<ffffffff8020cb0a>] ? child_rip+0xa/0x20
 [<ffffffff804d0af5>] ? _spin_lock+0x5/0x8
 [<ffffffff80247e1e>] ? kthread+0x0/0x7e
 [<ffffffff8020cb00>] ? child_rip+0x0/0x20
---[ end trace 3fbc2dd20bf89ef1 ]---
 connection1:0: ping timeout of 5 secs expired, last rx 4295286327, last ping 4295285518, now 4295286768
 connection1:0: detected conn error (1011)

Attached are the v2.6.30-rc3 KVM host and v2.6.29.2 KVM guest dmesg
output.  When the 'Unbalanced enable for IRQ 59' happens on the KVM
host, I do not see any exceptions in KVM guest (other than the iSCSI
connections drop), but it requires a restart of KVM+qemu-system-x86_64
to get the e1000e port back up.

Other than that loopback scenario, things are looking good quite good
with this combination of kvm-85 kernel+guest so far for me.  I did end
up taking out the two 8x function 2x Path/Function PCIe IOV adapters for
now, as it seemed to have an effect on stability with all of MSI-X
interrupts enabled on the KVM host for 16 virtual adapters.

I will keep testing with e1000e ports and let the list know the
progress.  Thanks for your comments!

--nab

> On Mon, May 04, 2009 at 06:40:36PM +0800, Nicholas A. Bellinger wrote:
> > On Mon, 2009-05-04 at 17:49 +0800, Sheng Yang wrote:
> > > On Monday 04 May 2009 17:11:59 Nicholas A. Bellinger wrote:
> > > > On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> > > > > On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > > > > > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > > > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > > > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger
> > > wrote:
> > > > > > > > > > Greetings KVM folks,
> > > > > > > > > >
> > > > > > > > > > I wondering if any information exists for doing SR-IOV on the
> > > > > > > > > > new VT-d capable chipsets with KVM..?  From what I understand
> > > > > > > > > > the patches for doing this with KVM are floating around, but I
> > > > > > > > > > have been unable to find any user-level docs for actually
> > > > > > > > > > making it all go against a upstream v2.6.30-rc3 code..
> > > > > > > > > >
> > > > > > > > > > So far I have been doing IOV testing with Xen 3.3 and
> > > > > > > > > > 3.4.0-pre, and I am really hoping to be able to jump to KVM for
> > > > > > > > > > single-function and and then multi-function SR-IOV.  I know
> > > > > > > > > > that the VM migration stuff for IOV in Xen is up and running,
> > > > > > > > > > and I assume it is being worked in for KVM instance migration
> > > > > > > > > > as well..? This part is less important (at least for me :-)
> > > > > > > > > > than getting a stable SR-IOV setup running under the KVM
> > > > > > > > > > hypervisor..  Does anyone have any pointers for this..?
> > > > > > > > > >
> > > > > > > > > > Any comments or suggestions are appreciated!
> > > > > > > > >
> > > > > > > > > Hi Nicholas
> > > > > > > > >
> > > > > > > > > The patches are not floating around now. As you know, SR-IOV for
> > > > > > > > > Linux have been in 2.6.30, so then you can use upstream KVM and
> > > > > > > > > qemu-kvm(or recent released kvm-85) with 2.6.30-rc3 as host
> > > > > > > > > kernel. And some time ago, there are several SRIOV related
> > > > > > > > > patches for qemu-kvm, and now they all have been checked in.
> > > > > > > > >
> > > > > > > > > And for KVM, the extra document is not necessary, for you can
> > > > > > > > > simple assign a VF to guest like any other devices. And how to
> > > > > > > > > create VF is specific for each device driver. So just create a VF
> > > > > > > > > then assign it to KVM guest is fine.
> > > > > > > >
> > > > > > > > Greetings Sheng,
> > > > > > > >
> > > > > > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > > > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > > > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > > > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I
> > > > > > > > can tell, the KVM_CAP_* defines from libkvm are enabled with
> > > > > > > > building kvm-85 after './configure
> > > > > > > > --kerneldir=/usr/src/linux-2.6.git' and the PCI passthrough code is
> > > > > > > > being enabled in
> > > > > > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > > > > > >
> > > > > > > > >From there, I use the freshly installed qemu-x86_64-system binary
> > > > > > > > > to
> > > > > > > >
> > > > > > > > start a Debian 5 x86_64 HVM (that previously had been moving
> > > > > > > > network packets under Xen for PCIe passthrough). I see the MSI-X
> > > > > > > > interrupt remapping working on the KVM host for the passed
> > > > > > > > -pcidevice, and the MMIO mappings from the qemu build that I also
> > > > > > > > saw while using Xen/qemu-dm built with PCI passthrough are there as
> > > > > > > > well..
> > > > > > >
> > > > > > > Hi Nicholas
> > > > > > >
> > > > > > > > But while the KVM guest is booting, I see the following
> > > > > > > > exception(s) from qemu-x86_64-system for one of the VFs for a
> > > > > > > > multi-function PCIe device:
> > > > > > > >
> > > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > > >
> > > > > > > This one is mostly harmless.
> > > > > >
> > > > > > Ok, good to know..  :-)
> > > > > >
> > > > > > > > I try with one of the on-board e1000e ports (02:00.0) and I see the
> > > > > > > > same exception along with some MSI-X exceptions from
> > > > > > > > qemu-x86_64-system in KVM guest.. However, I am still able to see
> > > > > > > > the e1000e and the other vxge multi-function device with lspci, but
> > > > > > > > I am unable to dhcp or ping with the e1000e and VF from
> > > > > > > > multi-function device fails to register the MSI-X interrupt in the
> > > > > > > > guest..
> > > > > > >
> > > > > > > Did you see the interrupt in the guest and host side?
> > > > > >
> > > > > > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > > > > > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > > > > > attaching the e1000e single-function device at 02:00.0 to pci-stub
> > > > > > with:
> > > > > >
> > > > > >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> > > > > >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> > > > > >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> > > > > >
> > > > > > I see the following the KVM host kernel ring buffer:
> > > > > >
> > > > > >    e1000e 0000:02:00.0: PCI INT A disabled
> > > > > >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> > > > > >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > > > > >
> > > > > > >  I think you can try on-
> > > > > > > board e1000e for MSI-X first. And please ensure correlated driver
> > > > > > > have been loaded correctly.
> > > > > >
> > > > > > <nod>..
> > > > > >
> > > > > > >  And what do you mean by "some MSI-X exceptions"? Better with
> > > > > > > the log.
> > > > > >
> > > > > > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > > > > > kvm_destroy_phys_mem() statements:
> > > > > >
> > > > > > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > > > > > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid parameters
> > > > > > (slot=-1)
> > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > >
> > > > > > However I still see the following in the KVM guest kernel ring buffer
> > > > > > running v2.6.30-rc in the HVM guest.
> > > > > >
> > > > > > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > > > > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > > > > (level, high) -> IRQ 10 [    5.525710] e1000e 0000:00:05.0: setting
> > > > > > latency timer to 64
> > > > > > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X
> > > > > > interrupts.  Falling back to MSI interrupts. [    5.527200]
> > > > > > 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts.
> > > > > > Falling back to legacy interrupts. [    5.829988] 0000:00:05.0: eth0:
> > > > > > (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2 [    5.830672]
> > > > > > 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection [    5.831240]
> > > > > > 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > > > >
> > > > > Hi Nicholas
> > > > >
> > > > > I think something need to be clarify:
> > > > > 1. For SRIOV, you need 2.6.30 as host kernel... But it's better to know
> > > > > if normal device assignment work in your environment at first.
> > > > > 2. The Fedora's userspace is even more old... You'd better try qemu-kvm
> > > > > upstream, which is more convenient for us to track the problem(and kvm-85
> > > > > is also ok). And as you see above, your QEmu don't support MSI/MSIX...
> > > >
> > > > Ok, got it..
> > > >
> > > > > So you can:
> > > > > 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
> > > >
> > > > Ok, I am now updated on in the FC 11 Host with kvm-85 kernel modules and
> > > > am using the built qemu-system-x86_64 from the kvm-85 source package:
> > > >
> > > > loaded kvm module (kvm-85)
> > > > QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008 Fabrice
> > > > Bellard
> > > >
> > > > > 2. Your host kernel is Fedora 11 Preview, that should be fine with device
> > > > > assignment at first(and let's solve it first, SRIOV the next step).
> > > >
> > > > Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM host for
> > > > the momemt to get e1000e working.  But I will start building a
> > > > v2.6.30-rc3 kernel again for my fc11 host kernel as I do need SR-IOV at
> > > > some point... :-)
> > > >
> > > > > 3. Your KVM version seems like kvm-85, you may provide some dmesg on host
> > > > > side(I think you didn't use the KVM come along with kernel).
> > > >
> > > > Ok, now within the KVM guest running v2.6.29.2, I see the following:
> > > >
> > > > [    2.669243] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
> > > > [    2.672931] e1000e: Copyright (c) 1999-2008 Intel Corporation.
> > > > [    2.674932] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > > [    2.675181] 8139too Fast Ethernet driver 0.9.28
> > > > [    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > > (level, high) -> IRQ 10
> > > > [    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
> > > > [    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> > > > [    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> > > > [    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> > > > [    2.698039] FDC 0 is a S82078B
> > > > [    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > > > 00:e0:81:c0:90:b2
> > > > [    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> > > > [    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > > >
> > > > And the folllowing from /proc/interrupts inside of the KVM guest:
> > > >
> > > >  24:        117          0          0          0          0          0
> > > >     0          0          0          0   PCI-MSI-edge      eth1-rx-0 25:
> > > >       0          0          0          0          0          0          0
> > > >        0          0          0   PCI-MSI-edge      eth1-tx-0 26:          2
> > > >          0          0          0          0          0          0
> > > > 0          0          0   PCI-MSI-edge      eth1
> > > >
> > > > ethtool eth1 reports that Link is detected, but I am still unable to get
> > > > a dhcp to work.
> > >
> > > It's a little strange that I checked all the log you posted, but can't find
> > > anything suspicious...(Except you got a MCE log in your dmesg, but I don't
> > > think it would relate to this).
> > >
> > > You also already have interrupts in the guest for eth1-rx-0 and eth1, so at
> > > least part of interrupts can be delivered to the guest.
> > >
> > > You can try to connect the port to another NIC port directly. Set fixed ip for
> > > each, then ping each other.
> > >
> > > You can also try to disable MSI-X capability in QEmu. Just using "#if
> > > 0/#endif" to wrap "#ifdef KVM_CAP_DEVICE_MSIX/#endif" in
> > > hw/assigned_device_pci_cap_init(). Then the device would use MSI.
> > >
> > > If I am lucky enough to find a 82574L card by hand, I would give it a try...
> > >
> > > --
> > > regards
> > > Yang, Sheng
> > >
> > 
> > Greetings Sheng,
> > 
> > So I updated my FC11 Host to kernel v2.6.30-rc3 (and enabled ext4 of
> > course) and rebuilt the kvm-85 source kernel module and
> > qemu-system-x86_64 and I am now able to get dhcp and IP ops from the
> > 02:00.0 device on my IOH-5520 board with the KVM guest using a v2.6.29.2
> > kernel!!  Everything is looking good with the v2.6.29.2, but after a
> > quick reboot back into my v2.6.30-rc3 KVM guest kernel build e1000e it
> > looks like I am unable to get dhcp.
> > 
> > Rebooting back into KVM Guest kernel v2.6.29.2 brings the pci-stub
> > assigned e1000e 82574L assigned with dhcp and everything looks
> > good!  :-)
> > 
> > I will keep poking at the v2.6.30-rc KVM guests (I am going to do a
> > complete rebuild) and see if it does not start move IP packets as well..
> > 
> > Thanks for all of your help in getting setup!
> > 
> > --nab
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[-- Attachment #2: 2.6.30-rc3-kvm-host --]
[-- Type: text/plain, Size: 57308 bytes --]

Linux version 2.6.30-rc3 (root@barret.risingtidesystems.com) (gcc version 4.4.0 20090427 (Red Hat 4.4.0-3) (GCC) ) #11 SMP Mon May 4 03:01:10 PDT 2009
Command line: ro root=/dev/mapper/vg_barret-lv_root rhgb pci=resource_alignment=06:00.0;06:00.1;06:00.2;06:00.3;06:00.4;06:00.5;06:00.6;06:00.7;07:00.0;07:00.1;07:00.2;07:00.3;07:00.4;07:00.5;07:00.6;07:00.7
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  Centaur CentaurHauls
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009ec00 (usable)
 BIOS-e820: 000000000009ec00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000bf780000 (usable)
 BIOS-e820: 00000000bf780000 - 00000000bf78e000 (ACPI data)
 BIOS-e820: 00000000bf78e000 - 00000000bf7d0000 (ACPI NVS)
 BIOS-e820: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
 BIOS-e820: 00000000bf7ec000 - 00000000c0000000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffa00000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000240000000 (usable)
DMI present.
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
last_pfn = 0x240000 max_arch_pfn = 0x100000000
MTRR default type: uncachable
MTRR fixed ranges enabled:
  00000-9FFFF write-back
  A0000-BFFFF uncachable
  C0000-C7FFF write-protect
  C8000-DFFFF uncachable
  E0000-E3FFF write-protect
  E4000-EFFFF write-through
  F0000-FFFFF write-protect
MTRR variable ranges enabled:
  0 base 0000000000 mask FE00000000 write-back
  1 base 0200000000 mask FFC0000000 write-back
  2 base 00C0000000 mask FFC0000000 uncachable
  3 base 00BF800000 mask FFFF800000 uncachable
  4 disabled
  5 disabled
  6 disabled
  7 disabled
e820 update range: 00000000bf800000 - 0000000100000000 (usable) ==> (reserved)
last_pfn = 0xbf780 max_arch_pfn = 0x100000000
init_memory_mapping: 0000000000000000-00000000bf780000
 0000000000 - 00bf600000 page 2M
 00bf600000 - 00bf780000 page 4k
kernel direct mapping tables up to bf780000 @ 10000-15000
init_memory_mapping: 0000000100000000-0000000240000000
 0100000000 - 0240000000 page 2M
kernel direct mapping tables up to 240000000 @ 13000-1d000
RAMDISK: 37c91000 - 37fefb43
ACPI: RSDP 00000000000fa420 00024 (v02 ACPIAM)
ACPI: XSDT 00000000bf780100 00064 (v01 042109 XSDT1027 20090421 MSFT 00000097)
ACPI: FACP 00000000bf780290 000F4 (v04 042109 FACP1027 20090421 MSFT 00000097)
ACPI: DSDT 00000000bf7804b0 05E8E (v02  7010_ 7010_001 00000001 INTL 20051117)
ACPI: FACS 00000000bf78e000 00040
ACPI: APIC 00000000bf780390 000DE (v02 042109 APIC1027 20090421 MSFT 00000097)
ACPI: MCFG 00000000bf780470 0003C (v01 042109 OEMMCFG  20090421 MSFT 00000097)
ACPI: OEMB 00000000bf78e040 0007A (v01 042109 OEMB1027 20090421 MSFT 00000097)
ACPI: HPET 00000000bf78a4b0 00038 (v01 042109 OEMHPET  20090421 MSFT 00000097)
ACPI: SRAT 00000000bf78a4f0 001D0 (v01 042109 OEMSRAT  00000001 INTL 00000001)
ACPI: DMAR 00000000bf78e0c0 00128 (v01    AMI  OEMDMAR 00000001 MSFT 00000097)
ACPI: SSDT 00000000bf791e00 00363 (v01 DpgPmm    CpuPm 00000012 INTL 20051117)
ACPI: Local APIC address 0xfee00000
(8 early reservations) ==> bootmem [0000000000 - 0240000000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
  #2 [0000200000 - 0000d038f0]    TEXT DATA BSS ==> [0000200000 - 0000d038f0]
  #3 [0037c91000 - 0037fefb43]          RAMDISK ==> [0037c91000 - 0037fefb43]
  #4 [000009ec00 - 0000100000]    BIOS reserved ==> [000009ec00 - 0000100000]
  #5 [0000d04000 - 0000d04158]              BRK ==> [0000d04000 - 0000d04158]
  #6 [0000010000 - 0000013000]          PGTABLE ==> [0000010000 - 0000013000]
  #7 [0000013000 - 0000018000]          PGTABLE ==> [0000013000 - 0000018000]
 [ffffe20000000000-ffffe20007dfffff] PMD -> [ffff880028200000-ffff88002fffffff] on node 0
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x00240000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
    0: 0x00000010 -> 0x0000009e
    0: 0x00000100 -> 0x000bf780
    0: 0x00100000 -> 0x00240000
On node 0 totalpages: 2094862
  DMA zone: 56 pages used for memmap
  DMA zone: 2927 pages reserved
  DMA zone: 999 pages, LIFO batch:0
  DMA32 zone: 14280 pages used for memmap
  DMA32 zone: 765880 pages, LIFO batch:31
  Normal zone: 17920 pages used for memmap
  Normal zone: 1292800 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x10] enabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x12] enabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x14] enabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x16] enabled)
ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x05] enabled)
ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x07] enabled)
ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x11] enabled)
ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x13] enabled)
ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x15] enabled)
ACPI: LAPIC (acpi_id[0x10] lapic_id[0x17] enabled)
ACPI: LAPIC_NMI (acpi_id[0xff] high dfl lint[0x1])
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 0, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24])
IOAPIC[1]: apic_id 9, version 0, address 0xfec8a000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a301 base: 0xfed00000
SMP: Allowing 16 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 48
Allocating PCI resources starting at c2000000 (gap: c0000000:20000000)
NR_CPUS:255 nr_cpumask_bits:255 nr_cpu_ids:16 nr_node_ids:1
PERCPU: Embedded 24 pages at ffff880030000000, static data 68768 bytes
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2059679
Kernel command line: ro root=/dev/mapper/vg_barret-lv_root rhgb pci=resource_alignment=06:00.0;06:00.1;06:00.2;06:00.3;06:00.4;06:00.5;06:00.6;06:00.7;07:00.0;07:00.1;07:00.2;07:00.3;07:00.4;07:00.5;07:00.6;07:00.7
Initializing CPU#0
NR_IRQS:4352
PID hash table entries: 4096 (order: 12, 32768 bytes)
Extended CMOS year: 2000
Fast TSC calibration using PIT
Detected 2266.696 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Checking aperture...
No AGP bridge found
Memory: 8220660k/9437184k available (2906k kernel code, 1057736k absent, 157960k reserved, 1829k data, 452k init)
hpet clockevent registered
HPET: 4 timers in total, 0 timers will be used for per-cpu timer
Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.39 BogoMIPS (lpj=9066784)
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Mount-cache hash table entries: 256
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
using mwait in idle threads.
ACPI: Core revision 20090320
Setting APIC routing to physical flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Booting processor 1 APIC 0x2 ip 0x6000
Initializing CPU#1
Calibrating delay using timer specific routine.. 4533.20 BogoMIPS (lpj=9066400)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU1: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 2 APIC 0x4 ip 0x6000
Initializing CPU#2
Calibrating delay using timer specific routine.. 4533.19 BogoMIPS (lpj=9066384)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU2: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 3 APIC 0x6 ip 0x6000
Initializing CPU#3
Calibrating delay using timer specific routine.. 4533.20 BogoMIPS (lpj=9066404)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU3: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 4 APIC 0x10 ip 0x6000
Initializing CPU#4
Calibrating delay using timer specific routine.. 4533.24 BogoMIPS (lpj=9066490)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU4: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 5 APIC 0x12 ip 0x6000
Initializing CPU#5
Calibrating delay using timer specific routine.. 4533.24 BogoMIPS (lpj=9066488)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU5: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 6 APIC 0x14 ip 0x6000
Initializing CPU#6
Calibrating delay using timer specific routine.. 4533.24 BogoMIPS (lpj=9066485)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU6: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 7 APIC 0x16 ip 0x6000
Initializing CPU#7
Calibrating delay using timer specific routine.. 4533.24 BogoMIPS (lpj=9066483)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU7: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 8 APIC 0x1 ip 0x6000
Initializing CPU#8
Calibrating delay using timer specific routine.. 4533.23 BogoMIPS (lpj=9066471)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU8: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 9 APIC 0x3 ip 0x6000
Initializing CPU#9
Calibrating delay using timer specific routine.. 4533.23 BogoMIPS (lpj=9066474)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU9: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 10 APIC 0x5 ip 0x6000
Initializing CPU#10
Calibrating delay using timer specific routine.. 4533.23 BogoMIPS (lpj=9066470)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU10: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 11 APIC 0x7 ip 0x6000
Initializing CPU#11
Calibrating delay using timer specific routine.. 4533.23 BogoMIPS (lpj=9066476)
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU11: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 12 APIC 0x11 ip 0x6000
Initializing CPU#12
Calibrating delay using timer specific routine.. 4533.23 BogoMIPS (lpj=9066478)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 0
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU12: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 13 APIC 0x13 ip 0x6000
Initializing CPU#13
Calibrating delay using timer specific routine.. 4533.24 BogoMIPS (lpj=9066482)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU13: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 14 APIC 0x15 ip 0x6000
Initializing CPU#14
Calibrating delay using timer specific routine.. 4533.23 BogoMIPS (lpj=9066470)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 2
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU14: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Booting processor 15 APIC 0x17 ip 0x6000
Initializing CPU#15
Calibrating delay using timer specific routine.. 4533.23 BogoMIPS (lpj=9066474)
CPU: Physical Processor ID: 1
CPU: Processor Core ID: 3
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 256K
CPU: L3 cache: 8192K
CPU15: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz stepping 05
Skipping synchronization checks as TSC is reliable.
Brought up 16 CPUs
Total of 16 processors activated (72531.85 BogoMIPS).
CPU0 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 0 1 2 3 8 9 10 11
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU1 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 1 2 3 8 9 10 11 0
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU2 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 2 3 8 9 10 11 0 1
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU3 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 3 8 9 10 11 0 1 2
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU4 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 4 5 6 7 12 13 14 15
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
CPU5 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 5 6 7 12 13 14 15 4
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
CPU6 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 6 7 12 13 14 15 4 5
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
CPU7 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 7 12 13 14 15 4 5 6
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
CPU8 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 8 9 10 11 0 1 2 3
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU9 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 9 10 11 0 1 2 3 8
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU10 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 10 11 0 1 2 3 8 9
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU11 attaching sched-domain:
 domain 0: span 0-3,8-11 level MC
  groups: 11 0 1 2 3 8 9 10
  domain 1: span 0-15 level CPU
   groups: 0-3,8-11 4-7,12-15
CPU12 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 12 13 14 15 4 5 6 7
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
CPU13 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 13 14 15 4 5 6 7 12
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
CPU14 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 14 15 4 5 6 7 12 13
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
CPU15 attaching sched-domain:
 domain 0: span 4-7,12-15 level MC
  groups: 15 4 5 6 7 12 13 14
  domain 1: span 0-15 level CPU
   groups: 4-7,12-15 0-3,8-11
net_namespace: 1824 bytes
Booting paravirtualized kernel on bare hardware
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
PCI: MCFG area at e0000000 reserved in E820
PCI: Using MMCONFIG at e0000000 - efffffff
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S1 S3 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI Warning (tbutils-0246): Incorrect checksum in table [OEMB] - 9A, should be 97 [20090320]
ACPI: No dock devices found.
ACPI: PCI Root Bridge [PCI0] (0000:00)
pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
pci 0000:00:00.0: PME# disabled
pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
pci 0000:00:01.0: PME# disabled
pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
pci 0000:00:03.0: PME# disabled
pci 0000:00:05.0: PME# supported from D0 D3hot D3cold
pci 0000:00:05.0: PME# disabled
pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
pci 0000:00:07.0: PME# disabled
pci 0000:00:13.0: reg 10 32bit mmio: [0xfec8a000-0xfec8afff]
pci 0000:00:13.0: PME# supported from D0 D3hot D3cold
pci 0000:00:13.0: PME# disabled
pci 0000:00:16.0: reg 10 64bit mmio: [0xfaed4000-0xfaed7fff]
pci 0000:00:16.1: reg 10 64bit mmio: [0xfaed8000-0xfaedbfff]
pci 0000:00:16.2: reg 10 64bit mmio: [0xfaedc000-0xfaedffff]
pci 0000:00:16.3: reg 10 64bit mmio: [0xfaee0000-0xfaee3fff]
pci 0000:00:16.4: reg 10 64bit mmio: [0xfaee4000-0xfaee7fff]
pci 0000:00:16.5: reg 10 64bit mmio: [0xfaee8000-0xfaeebfff]
pci 0000:00:16.6: reg 10 64bit mmio: [0xfaeec000-0xfaeeffff]
pci 0000:00:16.7: reg 10 64bit mmio: [0xfaef0000-0xfaef3fff]
pci 0000:00:1a.0: reg 20 io port: [0x9400-0x941f]
pci 0000:00:1a.1: reg 20 io port: [0x9480-0x949f]
pci 0000:00:1a.2: reg 20 io port: [0x9800-0x981f]
pci 0000:00:1a.7: reg 10 32bit mmio: [0xfaef6000-0xfaef63ff]
pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1a.7: PME# disabled
pci 0000:00:1b.0: reg 10 64bit mmio: [0xfaef8000-0xfaefbfff]
pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1b.0: PME# disabled
pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.0: PME# disabled
pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.4: PME# disabled
pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.5: PME# disabled
pci 0000:00:1d.0: reg 20 io port: [0x9880-0x989f]
pci 0000:00:1d.1: reg 20 io port: [0x9c00-0x9c1f]
pci 0000:00:1d.2: reg 20 io port: [0xa000-0xa01f]
pci 0000:00:1d.7: reg 10 32bit mmio: [0xfaefc000-0xfaefc3ff]
pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1d.7: PME# disabled
pci 0000:00:1f.0: quirk: region 0800-087f claimed by ICH6 ACPI/GPIO/TCO
pci 0000:00:1f.0: quirk: region 0500-053f claimed by ICH6 GPIO
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0a00 (mask 00ff)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 2 PIO at 0ca0 (mask 000f)
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 3 PIO at f700 (mask 000f)
pci 0000:00:1f.2: reg 10 io port: [0xac00-0xac07]
pci 0000:00:1f.2: reg 14 io port: [0xa880-0xa883]
pci 0000:00:1f.2: reg 18 io port: [0xa800-0xa807]
pci 0000:00:1f.2: reg 1c io port: [0xa480-0xa483]
pci 0000:00:1f.2: reg 20 io port: [0xa400-0xa40f]
pci 0000:00:1f.2: reg 24 io port: [0xa080-0xa08f]
pci 0000:00:1f.3: reg 10 64bit mmio: [0xfaefe000-0xfaefe0ff]
pci 0000:00:1f.3: reg 20 io port: [0x400-0x41f]
pci 0000:00:1f.5: reg 10 io port: [0xbc00-0xbc07]
pci 0000:00:1f.5: reg 14 io port: [0xb880-0xb883]
pci 0000:00:1f.5: reg 18 io port: [0xb800-0xb807]
pci 0000:00:1f.5: reg 1c io port: [0xb480-0xb483]
pci 0000:00:1f.5: reg 20 io port: [0xb400-0xb40f]
pci 0000:00:1f.5: reg 24 io port: [0xb080-0xb08f]
pci 0000:03:00.0: reg 10 32bit mmio: [0xfbee0000-0xfbefffff]
pci 0000:03:00.0: reg 18 io port: [0xec00-0xec1f]
pci 0000:03:00.0: reg 1c 32bit mmio: [0xfbedc000-0xfbedffff]
pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
pci 0000:03:00.0: PME# disabled
pci 0000:00:1c.4: bridge io port: [0xe000-0xefff]
pci 0000:00:1c.4: bridge 32bit mmio: [0xfbe00000-0xfbefffff]
pci 0000:02:00.0: reg 10 32bit mmio: [0xfbde0000-0xfbdfffff]
pci 0000:02:00.0: reg 18 io port: [0xdc00-0xdc1f]
pci 0000:02:00.0: reg 1c 32bit mmio: [0xfbddc000-0xfbddffff]
pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
pci 0000:02:00.0: PME# disabled
pci 0000:00:1c.5: bridge io port: [0xd000-0xdfff]
pci 0000:00:1c.5: bridge 32bit mmio: [0xfbd00000-0xfbdfffff]
pci 0000:01:00.0: reg 10 32bit mmio: [0xfb000000-0xfb7fffff]
pci 0000:01:00.0: reg 14 32bit mmio: [0xfafe0000-0xfaffffff]
pci 0000:01:00.0: reg 18 io port: [0xc880-0xc8ff]
pci 0000:01:00.0: supports D1 D2
pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
pci 0000:01:00.0: PME# disabled
pci 0000:01:05.0: reg 10 32bit mmio: [0xfbcfe000-0xfbcfe7ff]
pci 0000:01:05.0: reg 14 io port: [0xcc00-0xcc7f]
pci 0000:01:05.0: supports D2
pci 0000:01:05.0: PME# supported from D2 D3hot D3cold
pci 0000:01:05.0: PME# disabled
pci 0000:00:1e.0: transparent bridge
pci 0000:00:1e.0: bridge io port: [0xc000-0xcfff]
pci 0000:00:1e.0: bridge 32bit mmio: [0xfaf00000-0xfbcfffff]
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE3._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE5._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.NPE7._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P8._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P9._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs *5)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 *6 7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 6 *7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 6 7 10 11 12 *14 15)
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
DMAR:Host address width 40
DMAR:DRHD (flags: 0x00000001)base: 0x00000000fbffe000
DMAR:RMRR base: 0x00000000000ec000 end: 0x00000000000effff
DMAR:RMRR base: 0x00000000bf7ec000 end: 0x00000000bf7fffff
DMAR:Unknown DMAR structure type
IOAPIC id 8 under DRHD base 0xfbffe000
IOAPIC id 9 under DRHD base 0xfbffe000
IOMMU 0xfbffe000: using Queued invalidation
IOMMU: Setting identity map for device 0000:00:1d.0 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xbf7ec000 - 0xbf800000]
IOMMU: Setting identity map for device 0000:00:1d.0 [0xec000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1d.1 [0xec000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1d.2 [0xec000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1d.7 [0xec000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1a.0 [0xec000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1a.1 [0xec000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1a.2 [0xec000 - 0xf0000]
IOMMU: Setting identity map for device 0000:00:1a.7 [0xec000 - 0xf0000]
IOMMU: gfx device 0000:01:00.0 1-1 mapping
IOMMU: Setting identity map for device 0000:01:00.0 [0x10000 - 0x9e000]
IOMMU: Setting identity map for device 0000:01:00.0 [0x100000 - 0xbf780000]
IOMMU: Setting identity map for device 0000:01:00.0 [0x100000000 - 0x240000000]
IOMMU: Prepare 0-16M unity mapping for LPC
IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0x1000000]
PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
Multi-level page-table translation for DMAR.
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
hpet0: 4 comparators, 64-bit 14.318180 MHz counter
pnp: PnP ACPI init
ACPI: bus type pnp registered
alloc irq_2_iommu on cpu 0 node 0
alloc irq_2_iommu on cpu 0 node 0
alloc irq_2_iommu on cpu 0 node 0
alloc irq_2_iommu on cpu 0 node 0
pnp: PnP ACPI: found 14 devices
ACPI: ACPI bus type pnp unregistered
system 00:01: iomem range 0xfbf00000-0xfbffffff has been reserved
system 00:01: iomem range 0xfc000000-0xfcffffff has been reserved
system 00:01: iomem range 0xfd000000-0xfdffffff has been reserved
system 00:01: iomem range 0xfe000000-0xfebfffff has been reserved
system 00:01: iomem range 0xfec8a000-0xfec8afff has been reserved
system 00:01: iomem range 0xfed10000-0xfed10fff has been reserved
system 00:08: ioport range 0xa00-0xa0f has been reserved
system 00:08: ioport range 0xa10-0xa1f has been reserved
system 00:09: ioport range 0x4d0-0x4d1 has been reserved
system 00:09: ioport range 0x800-0x87f has been reserved
system 00:09: ioport range 0x500-0x57f could not be reserved
system 00:09: iomem range 0xfed1c000-0xfed1ffff has been reserved
system 00:09: iomem range 0xfed20000-0xfed3ffff has been reserved
system 00:09: iomem range 0xfed40000-0xfed8ffff has been reserved
system 00:0b: iomem range 0xfec00000-0xfec00fff has been reserved
system 00:0b: iomem range 0xfee00000-0xfee00fff has been reserved
system 00:0c: iomem range 0xe0000000-0xefffffff has been reserved
system 00:0d: iomem range 0x0-0x9ffff could not be reserved
system 00:0d: iomem range 0xc0000-0xcffff has been reserved
system 00:0d: iomem range 0xe0000-0xfffff could not be reserved
system 00:0d: iomem range 0x100000-0xbfffffff could not be reserved
system 00:0d: iomem range 0xfed90000-0xffffffff could not be reserved
pci 0000:00:01.0: PCI bridge, secondary bus 0000:08
pci 0000:00:01.0:   IO window: disabled
pci 0000:00:01.0:   MEM window: disabled
pci 0000:00:01.0:   PREFETCH window: disabled
pci 0000:00:03.0: PCI bridge, secondary bus 0000:07
pci 0000:00:03.0:   IO window: disabled
pci 0000:00:03.0:   MEM window: disabled
pci 0000:00:03.0:   PREFETCH window: disabled
pci 0000:00:05.0: PCI bridge, secondary bus 0000:06
pci 0000:00:05.0:   IO window: disabled
pci 0000:00:05.0:   MEM window: disabled
pci 0000:00:05.0:   PREFETCH window: disabled
pci 0000:00:07.0: PCI bridge, secondary bus 0000:05
pci 0000:00:07.0:   IO window: disabled
pci 0000:00:07.0:   MEM window: disabled
pci 0000:00:07.0:   PREFETCH window: disabled
pci 0000:00:1c.0: PCI bridge, secondary bus 0000:04
pci 0000:00:1c.0:   IO window: disabled
pci 0000:00:1c.0:   MEM window: disabled
pci 0000:00:1c.0:   PREFETCH window: disabled
pci 0000:00:1c.4: PCI bridge, secondary bus 0000:03
pci 0000:00:1c.4:   IO window: 0xe000-0xefff
pci 0000:00:1c.4:   MEM window: 0xfbe00000-0xfbefffff
pci 0000:00:1c.4:   PREFETCH window: disabled
pci 0000:00:1c.5: PCI bridge, secondary bus 0000:02
pci 0000:00:1c.5:   IO window: 0xd000-0xdfff
pci 0000:00:1c.5:   MEM window: 0xfbd00000-0xfbdfffff
pci 0000:00:1c.5:   PREFETCH window: disabled
pci 0000:00:1e.0: PCI bridge, secondary bus 0000:01
pci 0000:00:1e.0:   IO window: 0xc000-0xcfff
pci 0000:00:1e.0:   MEM window: 0xfaf00000-0xfbcfffff
pci 0000:00:1e.0:   PREFETCH window: disabled
pci 0000:00:01.0: setting latency timer to 64
pci 0000:00:03.0: setting latency timer to 64
pci 0000:00:05.0: setting latency timer to 64
pci 0000:00:07.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.0: setting latency timer to 64
pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.4: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pci 0000:00:1c.5: PCI INT B -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.5: setting latency timer to 64
pci 0000:00:1e.0: setting latency timer to 64
pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
pci_bus 0000:08: resource 0 mem: [0x0-0x0]
pci_bus 0000:08: resource 1 mem: [0x0-0x0]
pci_bus 0000:08: resource 2 mem: [0x0-0x0]
pci_bus 0000:08: resource 3 mem: [0x0-0x0]
pci_bus 0000:07: resource 0 mem: [0x0-0x0]
pci_bus 0000:07: resource 1 mem: [0x0-0x0]
pci_bus 0000:07: resource 2 mem: [0x0-0x0]
pci_bus 0000:07: resource 3 mem: [0x0-0x0]
pci_bus 0000:06: resource 0 mem: [0x0-0x0]
pci_bus 0000:06: resource 1 mem: [0x0-0x0]
pci_bus 0000:06: resource 2 mem: [0x0-0x0]
pci_bus 0000:06: resource 3 mem: [0x0-0x0]
pci_bus 0000:05: resource 0 mem: [0x0-0x0]
pci_bus 0000:05: resource 1 mem: [0x0-0x0]
pci_bus 0000:05: resource 2 mem: [0x0-0x0]
pci_bus 0000:05: resource 3 mem: [0x0-0x0]
pci_bus 0000:04: resource 0 mem: [0x0-0x0]
pci_bus 0000:04: resource 1 mem: [0x0-0x0]
pci_bus 0000:04: resource 2 mem: [0x0-0x0]
pci_bus 0000:04: resource 3 mem: [0x0-0x0]
pci_bus 0000:03: resource 0 io:  [0xe000-0xefff]
pci_bus 0000:03: resource 1 mem: [0xfbe00000-0xfbefffff]
pci_bus 0000:03: resource 2 mem: [0x0-0x0]
pci_bus 0000:03: resource 3 mem: [0x0-0x0]
pci_bus 0000:02: resource 0 io:  [0xd000-0xdfff]
pci_bus 0000:02: resource 1 mem: [0xfbd00000-0xfbdfffff]
pci_bus 0000:02: resource 2 mem: [0x0-0x0]
pci_bus 0000:02: resource 3 mem: [0x0-0x0]
pci_bus 0000:01: resource 0 io:  [0xc000-0xcfff]
pci_bus 0000:01: resource 1 mem: [0xfaf00000-0xfbcfffff]
pci_bus 0000:01: resource 2 mem: [0x0-0x0]
pci_bus 0000:01: resource 3 io:  [0x00-0xffff]
pci_bus 0000:01: resource 4 mem: [0x000000-0xffffffffffffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
NET: Registered protocol family 1
checking if image is initramfs...
rootfs image is initramfs; unpacking...
Freeing initrd memory: 3450k freed
audit: initializing netlink socket (disabled)
type=2000 audit(1241513600.604:1): initialized
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
msgmni has been set to 16064
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci 0000:01:00.0: Boot video device
alloc irq_2_iommu on cpu 0 node 0
pcieport-driver 0000:00:01.0: irq 49 for MSI/MSI-X
pcieport-driver 0000:00:01.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pcieport-driver 0000:00:03.0: irq 50 for MSI/MSI-X
pcieport-driver 0000:00:03.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pcieport-driver 0000:00:05.0: irq 51 for MSI/MSI-X
pcieport-driver 0000:00:05.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pcieport-driver 0000:00:07.0: irq 52 for MSI/MSI-X
pcieport-driver 0000:00:07.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pcieport-driver 0000:00:1c.0: irq 53 for MSI/MSI-X
pcieport-driver 0000:00:1c.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pcieport-driver 0000:00:1c.4: irq 54 for MSI/MSI-X
pcieport-driver 0000:00:1c.4: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
pcieport-driver 0000:00:1c.5: irq 55 for MSI/MSI-X
pcieport-driver 0000:00:1c.5: setting latency timer to 64
aer 0000:00:01.0:pcie02: AER service couldn't init device: no _OSC support
aer 0000:00:03.0:pcie02: AER service couldn't init device: no _OSC support
aer 0000:00:05.0:pcie02: AER service couldn't init device: no _OSC support
aer 0000:00:07.0:pcie02: AER service couldn't init device: no _OSC support
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
ACPI: SSDT 00000000bf78e1f0 02B94 (v01 DpgPmm  P001Ist 00000011 INTL 20051117)
ACPI: SSDT 00000000bf790d90 00980 (v01  PmRef  P001Cst 00003001 INTL 20051117)
ACPI: SSDT 00000000bf791710 006E2 (v01  PmRef  Cpu0Tst 00003000 INTL 20051117)
Monitor-Mwait will be used to enter C-1 state
Monitor-Mwait will be used to enter C-3 state
ACPI: CPU0 (power states: C1[C1] C2[C3])
processor ACPI_CPU:00: registered as cooling_device0
ACPI: Processor [P001] (supports 8 throttling states)
ACPI: CPU1 (power states: C1[C1] C2[C3])
processor ACPI_CPU:01: registered as cooling_device1
ACPI: Processor [P002] (supports 8 throttling states)
ACPI: CPU2 (power states: C1[C1] C2[C3])
processor ACPI_CPU:02: registered as cooling_device2
ACPI: Processor [P003] (supports 8 throttling states)
ACPI: CPU3 (power states: C1[C1] C2[C3])
processor ACPI_CPU:03: registered as cooling_device3
ACPI: Processor [P004] (supports 8 throttling states)
ACPI: CPU4 (power states: C1[C1] C2[C3])
processor ACPI_CPU:04: registered as cooling_device4
ACPI: Processor [P005] (supports 8 throttling states)
ACPI: CPU5 (power states: C1[C1] C2[C3])
processor ACPI_CPU:05: registered as cooling_device5
ACPI: Processor [P006] (supports 8 throttling states)
ACPI: CPU6 (power states: C1[C1] C2[C3])
processor ACPI_CPU:06: registered as cooling_device6
ACPI: Processor [P007] (supports 8 throttling states)
ACPI: CPU7 (power states: C1[C1] C2[C3])
processor ACPI_CPU:07: registered as cooling_device7
ACPI: Processor [P008] (supports 8 throttling states)
ACPI: CPU8 (power states: C1[C1] C2[C3])
processor ACPI_CPU:08: registered as cooling_device8
ACPI: Processor [P009] (supports 8 throttling states)
ACPI: CPU9 (power states: C1[C1] C2[C3])
processor ACPI_CPU:09: registered as cooling_device9
ACPI: Processor [P010] (supports 8 throttling states)
ACPI: CPU10 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0a: registered as cooling_device10
ACPI: Processor [P011] (supports 8 throttling states)
ACPI: CPU11 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0b: registered as cooling_device11
ACPI: Processor [P012] (supports 8 throttling states)
ACPI: CPU12 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0c: registered as cooling_device12
ACPI: Processor [P013] (supports 8 throttling states)
ACPI: CPU13 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0d: registered as cooling_device13
ACPI: Processor [P014] (supports 8 throttling states)
ACPI: CPU14 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0e: registered as cooling_device14
ACPI: Processor [P015] (supports 8 throttling states)
ACPI: CPU15 (power states: C1[C1] C2[C3])
processor ACPI_CPU:0f: registered as cooling_device15
ACPI: Processor [P016] (supports 8 throttling states)
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
brd: module loaded
Uniform Multi-Platform E-IDE driver
ide_generic: please use "probe_mask=0x3f" module parameter for probing all legacy ISA IDE ports
Probing IDE interface ide0...
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
ide1 at 0x170-0x177,0x376 on irq 15
ide-gd driver 1.18
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
cpuidle: using governor ladder
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
TCP bic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
registered taskstats version 1
Freeing unused kernel memory: 452k freed
Write protecting the kernel read-only data: 4036k
SCSI subsystem initialized
libata version 3.00 loaded.
ata_piix 0000:00:1f.2: version 2.12
alloc irq_2_iommu on cpu 0 node 0
ata_piix 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
ata_piix 0000:00:1f.2: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: SATA max UDMA/133 cmd 0xac00 ctl 0xa880 bmdma 0xa400 irq 19
ata2: SATA max UDMA/133 cmd 0xa800 ctl 0xa480 bmdma 0xa408 irq 19
ata_piix 0000:00:1f.5: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.5: MAP [ P0 -- P1 -- ]
ata_piix 0000:00:1f.5: setting latency timer to 64
scsi2 : ata_piix
scsi3 : ata_piix
ata3: SATA max UDMA/133 cmd 0xbc00 ctl 0xb880 bmdma 0xb400 irq 19
ata4: SATA max UDMA/133 cmd 0xb800 ctl 0xb480 bmdma 0xb408 irq 19
ata3: SATA link down (SStatus 0 SControl 300)
ata1.00: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.01: SATA link down (SStatus 0 SControl 300)
ata1.00: ATA-6: ST380817AS, 3.42, max UDMA/133
ata1.00: 156301488 sectors, multi 16: LBA48 NCQ (not used)
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      ST380817AS       3.42 PQ: 0 ANSI: 5
ata2.00: SATA link down (SStatus 0 SControl 300)
ata2.01: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
alloc irq_2_iommu on cpu 0 node 0
ehci_hcd 0000:00:1a.7: PCI INT C -> GSI 18 (level, low) -> IRQ 18
ehci_hcd 0000:00:1a.7: setting latency timer to 64
ehci_hcd 0000:00:1a.7: EHCI Host Controller
ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1a.7: debug port 1
ehci_hcd 0000:00:1a.7: cache line size of 32 is not supported
ehci_hcd 0000:00:1a.7: irq 18, io mem 0xfaef6000
ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
alloc irq_2_iommu on cpu 0 node 0
ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 23 (level, low) -> IRQ 23
ehci_hcd 0000:00:1d.7: setting latency timer to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:1d.7: debug port 1
ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
ehci_hcd 0000:00:1d.7: irq 23, io mem 0xfaefc000
ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
uhci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
uhci_hcd 0000:00:1a.0: setting latency timer to 64
uhci_hcd 0000:00:1a.0: UHCI Host Controller
uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
uhci_hcd 0000:00:1a.0: irq 16, io base 0x00009400
usb usb3: configuration #1 chosen from 1 choice
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
alloc irq_2_iommu on cpu 0 node 0
uhci_hcd 0000:00:1a.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
uhci_hcd 0000:00:1a.1: setting latency timer to 64
uhci_hcd 0000:00:1a.1: UHCI Host Controller
uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4
uhci_hcd 0000:00:1a.1: irq 21, io base 0x00009480
usb usb4: configuration #1 chosen from 1 choice
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
uhci_hcd 0000:00:1a.2: PCI INT D -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1a.2: setting latency timer to 64
uhci_hcd 0000:00:1a.2: UHCI Host Controller
uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5
uhci_hcd 0000:00:1a.2: irq 19, io base 0x00009800
usb usb5: configuration #1 chosen from 1 choice
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
uhci_hcd 0000:00:1d.0: setting latency timer to 64
uhci_hcd 0000:00:1d.0: UHCI Host Controller
uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
uhci_hcd 0000:00:1d.0: irq 23, io base 0x00009880
usb usb6: configuration #1 chosen from 1 choice
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
uhci_hcd 0000:00:1d.1: setting latency timer to 64
uhci_hcd 0000:00:1d.1: UHCI Host Controller
uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7
uhci_hcd 0000:00:1d.1: irq 19, io base 0x00009c00
usb usb7: configuration #1 chosen from 1 choice
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
uhci_hcd 0000:00:1d.2: setting latency timer to 64
uhci_hcd 0000:00:1d.2: UHCI Host Controller
uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8
uhci_hcd 0000:00:1d.2: irq 18, io base 0x0000a000
usb usb8: configuration #1 chosen from 1 choice
hub 8-0:1.0: USB hub found
hub 8-0:1.0: 2 ports detected
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.14.0-ioctl (2008-04-23) initialised: dm-devel@redhat.com
Driver 'sd' needs updating - please use bus_type methods
sd 0:0:0:0: [sda] 156301488 512-byte hardware sectors: (80.0 GB/74.5 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
EXT4-fs: barriers enabled
kjournald2 starting: pid 883, dev dm-0:8, commit interval 5 seconds
EXT4-fs: delayed allocation enabled
EXT4-fs: file extents enabled
EXT4-fs: mballoc enabled
EXT4-fs: mounted filesystem dm-0 with ordered data mode
SELinux:  Disabled at runtime.
SELinux:  Unregistering netfilter hooks
type=1404 audit(1241513604.336:2): selinux=0 auid=4294967295 ses=4294967295
usb 2-5: new high speed USB device using ehci_hcd and address 2
usb 2-5: config 1 interface 0 altsetting 0 endpoint 0x81 has an invalid bInterval 255, changing to 11
usb 2-5: configuration #1 chosen from 1 choice
hub 2-5:1.0: USB hub found
hub 2-5:1.0: 3 ports detected
usb 3-1: new full speed USB device using uhci_hcd and address 2
usb 3-1: configuration #1 chosen from 1 choice
hub 3-1:1.0: USB hub found
hub 3-1:1.0: 2 ports detected
usb 3-2: new low speed USB device using uhci_hcd and address 3
usb 3-2: configuration #1 chosen from 1 choice
input: Sony Computer Entertainment Inc. SCE USB MOUSE as /class/input/input0
generic-usb 0003:054C:0061.0001: input: USB HID v1.00 Mouse [Sony Computer Entertainment Inc. SCE USB MOUSE] on usb-0000:00:1a.0-2/input0
usb 2-5.1: new high speed USB device using ehci_hcd and address 3
usb 2-5.1: configuration #1 chosen from 1 choice
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input1
generic-usb 0003:046B:FF10.0002: input: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1d.7-5.1/input0
input: American Megatrends Inc. Virtual Keyboard and Mouse as /class/input/input2
generic-usb 0003:046B:FF10.0003: input: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1d.7-5.1/input1
usb 3-1.1: new full speed USB device using uhci_hcd and address 4
usb 3-1.1: configuration #1 chosen from 1 choice
input: Sony Computer Entertainment Inc. SCE USB Keyboard as /class/input/input3
generic-usb 0003:054C:005C.0004: input: USB HID v1.00 Keyboard [Sony Computer Entertainment Inc. SCE USB Keyboard] on usb-0000:00:1a.0-1.1/input0
udev: starting version 141
udev: deprecated sysfs layout; update the kernel or disable CONFIG_SYSFS_DEPRECATED; some udev features will not work correctly
input: Power Button as /class/input/input4
ACPI: Power Button [PWRF]
input: Power Button as /class/input/input5
ACPI: Power Button [PWRB]
input: PC Speaker as /class/input/input6
rtc_cmos 00:03: RTC can wake from S4
rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.4-k4
e1000e: Copyright (c) 1999-2008 Intel Corporation.
e1000e 0000:03:00.0: Disabling L1 ASPM
e1000e 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
e1000e 0000:03:00.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
e1000e 0000:03:00.0: irq 56 for MSI/MSI-X
alloc irq_2_iommu on cpu 0 node 0
e1000e 0000:03:00.0: irq 57 for MSI/MSI-X
alloc irq_2_iommu on cpu 0 node 0
e1000e 0000:03:00.0: irq 58 for MSI/MSI-X
0000:03:00.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b3
0000:03:00.0: eth0: Intel(R) PRO/1000 Network Connection
0000:03:00.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
e1000e 0000:02:00.0: Disabling L1 ASPM
e1000e 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
e1000e 0000:02:00.0: setting latency timer to 64
alloc irq_2_iommu on cpu 0 node 0
e1000e 0000:02:00.0: irq 59 for MSI/MSI-X
alloc irq_2_iommu on cpu 0 node 0
e1000e 0000:02:00.0: irq 60 for MSI/MSI-X
alloc irq_2_iommu on cpu 0 node 0
e1000e 0000:02:00.0: irq 61 for MSI/MSI-X
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
0000:02:00.0: eth1: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
0000:02:00.0: eth1: Intel(R) PRO/1000 Network Connection
0000:02:00.0: eth1: MAC: 3, PHY: 8, PBA No: ffffff-0ff
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:07: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
sd 0:0:0:0: Attached scsi generic sg0 type 0
firewire_ohci 0000:01:05.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
firewire_ohci: Added fw-ohci device 0000:01:05.0, OHCI version 1.10
loaded kvm module (kvm-85)
firewire_core: created device fw0: GUID 00e0810000310bb8, S400
device-mapper: multipath: version 1.0.5 loaded
loop: module loaded
EXT4 FS on dm-0, internal journal on dm-0:8
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda1, internal journal
EXT3-fs: mounted filesystem with writeback data mode.
Adding 10158072k swap on /dev/mapper/vg_barret-lv_swap.  Priority:-1 extents:1 across:10158072k 
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
platform microcode: firmware: requesting intel-ucode/06-1a-05
Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
Microcode Update Driver: v2.00 removed.
Loading iSCSI transport class v2.0-870.
iscsi: registered transport (tcp)
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
iscsi: registered transport (iser)
ip6_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
nf_conntrack.acct=1 kernel paramater, acct=1 nf_conntrack module option or
sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
can't load conntrack support for proto=10
ip_tables: (C) 2000-2006 Netfilter Core Team
can't load conntrack support for proto=2
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
Bridge firewalling registered
virbr0: starting userspace STP failed, starting kernel STP
mtrr: type mismatch for fb000000,800000 old: write-back new: write-combining
can't load conntrack support for proto=2
ADDRCONF(NETDEV_UP): eth1: link is not ready
ADDRCONF(NETDEV_UP): eth0: link is not ready
e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
eth0: no IPv6 routers present
e1000e 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
assign device: host bdf = 2:0:0
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
pci-stub 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
assign device: host bdf = 2:0:0
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
------------[ cut here ]------------
WARNING: at kernel/irq/manage.c:260 enable_irq+0x36/0x50()
Hardware name: empty
Unbalanced enable for IRQ 59
Modules linked in: ipt_REJECT xt_tcpudp bridge stp sunrpc iptable_filter ip_tables xt_state nf_conntrack ip6table_filter ip6_tables x_tables ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table ext3 jbd loop dm_multipath scsi_dh kvm_intel kvm uinput i2c_i801 firewire_ohci joydev firewire_core sg i2c_core 8250_pnp crc_itu_t e1000e 8250 serial_core rtc_cmos pcspkr serio_raw rtc_core rtc_lib button sd_mod dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod uhci_hcd ohci_hcd ehci_hcd ata_piix libata scsi_mod [last unloaded: microcode]
Pid: 55, comm: events/4 Not tainted 2.6.30-rc3 #11
Call Trace:
 [<ffffffff80235fee>] ? warn_slowpath+0xcb/0xe8
 [<ffffffff80248214>] ? autoremove_wake_function+0x9/0x2a
 [<ffffffff80229392>] ? __wake_up_common+0x41/0x78
 [<ffffffff8022acec>] ? update_curr+0x67/0xeb
 [<ffffffff8020a5d8>] ? __switch_to+0xb6/0x274
 [<ffffffffa01ac7e4>] ? kvm_irq_delivery_to_apic+0xb3/0xf7 [kvm]
 [<ffffffffa01aa4d4>] ? __apic_accept_irq+0x15a/0x173 [kvm]
 [<ffffffffa01ac883>] ? kvm_set_msi+0x5b/0x60 [kvm]
 [<ffffffff80266d97>] ? enable_irq+0x36/0x50
 [<ffffffffa0195ab5>] ? kvm_assigned_dev_interrupt_work_handler+0x6d/0xbc [kvm]
 [<ffffffff802449fa>] ? worker_thread+0x182/0x223
 [<ffffffff8024820b>] ? autoremove_wake_function+0x0/0x2a
 [<ffffffff80244878>] ? worker_thread+0x0/0x223
 [<ffffffff80244878>] ? worker_thread+0x0/0x223
 [<ffffffff80247e72>] ? kthread+0x54/0x7e
 [<ffffffff8020cb0a>] ? child_rip+0xa/0x20
 [<ffffffff804d0af5>] ? _spin_lock+0x5/0x8
 [<ffffffff80247e1e>] ? kthread+0x0/0x7e
 [<ffffffff8020cb00>] ? child_rip+0x0/0x20
---[ end trace 3fbc2dd20bf89ef0 ]---
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
assign device: host bdf = 2:0:0
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
pci-stub 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
assign device: host bdf = 2:0:0
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
scsi4 : iSCSI Initiator over TCP/IP
scsi 4:0:0:0: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI: 5
sd 4:0:0:0: Attached scsi generic sg1 type 0
scsi 4:0:0:1: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI: 5
sd 4:0:0:1: Attached scsi generic sg2 type 0
sd 4:0:0:0: [sdb] 262144 512-byte hardware sectors: (134 MB/128 MiB)
sd 4:0:0:1: [sdc] 262144 512-byte hardware sectors: (134 MB/128 MiB)
sd 4:0:0:0: [sdb] Write Protect is off
sd 4:0:0:0: [sdb] Mode Sense: 2f 00 00 00
sd 4:0:0:1: [sdc] Write Protect is off
sd 4:0:0:1: [sdc] Mode Sense: 2f 00 00 00
sd 4:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
sd 4:0:0:1: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sdb:<6> sdc: unknown partition table
sd 4:0:0:0: [sdb] Attached SCSI disk
 unknown partition table
sd 4:0:0:1: [sdc] Attached SCSI disk
------------[ cut here ]------------
WARNING: at kernel/irq/manage.c:260 enable_irq+0x36/0x50()
Hardware name: empty
Unbalanced enable for IRQ 59
Modules linked in: ipt_REJECT xt_tcpudp bridge stp sunrpc iptable_filter ip_tables xt_state nf_conntrack ip6table_filter ip6_tables x_tables ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table ext3 jbd loop dm_multipath scsi_dh kvm_intel kvm uinput i2c_i801 firewire_ohci joydev firewire_core sg i2c_core 8250_pnp crc_itu_t e1000e 8250 serial_core rtc_cmos pcspkr serio_raw rtc_core rtc_lib button sd_mod dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod uhci_hcd ohci_hcd ehci_hcd ata_piix libata scsi_mod [last unloaded: microcode]
Pid: 51, comm: events/0 Tainted: G        W  2.6.30-rc3 #11
Call Trace:
 [<ffffffff80235fee>] ? warn_slowpath+0xcb/0xe8
 [<ffffffff80253a7c>] ? generic_exec_single+0x6a/0x88
 [<ffffffff8022acec>] ? update_curr+0x67/0xeb
 [<ffffffffa0198748>] ? vcpu_kick_intr+0x0/0x1 [kvm]
 [<ffffffff8020a5d8>] ? __switch_to+0xb6/0x274
 [<ffffffff8022b70a>] ? __dequeue_entity+0x1b/0x2f
 [<ffffffffa01ac7e4>] ? kvm_irq_delivery_to_apic+0xb3/0xf7 [kvm]
 [<ffffffffa01aa4d4>] ? __apic_accept_irq+0x15a/0x173 [kvm]
 [<ffffffffa01ac883>] ? kvm_set_msi+0x5b/0x60 [kvm]
 [<ffffffff80266d97>] ? enable_irq+0x36/0x50
 [<ffffffffa0195ab5>] ? kvm_assigned_dev_interrupt_work_handler+0x6d/0xbc [kvm]
 [<ffffffff802449fa>] ? worker_thread+0x182/0x223
 [<ffffffff8024820b>] ? autoremove_wake_function+0x0/0x2a
 [<ffffffff80244878>] ? worker_thread+0x0/0x223
 [<ffffffff80244878>] ? worker_thread+0x0/0x223
 [<ffffffff80247e72>] ? kthread+0x54/0x7e
 [<ffffffff8020cb0a>] ? child_rip+0xa/0x20
 [<ffffffff804d0af5>] ? _spin_lock+0x5/0x8
 [<ffffffff80247e1e>] ? kthread+0x0/0x7e
 [<ffffffff8020cb00>] ? child_rip+0x0/0x20
---[ end trace 3fbc2dd20bf89ef1 ]---
 connection1:0: ping timeout of 5 secs expired, last rx 4295286327, last ping 4295285518, now 4295286768
 connection1:0: detected conn error (1011)
 session1: session recovery timed out after 120 secs
sd 4:0:0:0: [sdb] Unhandled error code
sd 4:0:0:0: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK
end_request: I/O error, dev sdb, sector 138240
sd 4:0:0:0: [sdb] Unhandled error code
sd 4:0:0:0: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 1
Buffer I/O error on device sdb, logical block 2
Buffer I/O error on device sdb, logical block 3
sd 4:0:0:0: [sdb] Unhandled error code
sd 4:0:0:0: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
sd 4:0:0:0: [sdb] Unhandled error code
sd 4:0:0:0: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 1
Buffer I/O error on device sdb, logical block 2
Buffer I/O error on device sdb, logical block 3
sd 4:0:0:0: [sdb] Unhandled error code
sd 4:0:0:0: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
sd 4:0:0:0: [sdb] Unhandled error code
sd 4:0:0:0: [sdb] Result: hostbyte=DID_TRANSPORT_FAILFAST driverbyte=DRIVER_OK
end_request: I/O error, dev sdb, sector 0
pci-stub 0000:02:00.0: PCI INT A disabled
pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
assign device: host bdf = 2:0:0
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X

[-- Attachment #3: 2.6.29.2-kvm-guest-lio --]
[-- Type: text/plain, Size: 61929 bytes --]

May  5 02:28:32 lenny64guest1 kernel: imklog 3.18.6, log source = /proc/kmsg started.
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Initializing cgroup subsys cpuset
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Initializing cgroup subsys cpu
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Linux version 2.6.29.2 (root@lenny64guest1) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) #2 SMP Mon May 4 01:22:53 PDT 2009
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Command line: root=/dev/hda1 ro
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] KERNEL supported cpus:
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   Intel GenuineIntel
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   AMD AuthenticAMD
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   Centaur CentaurHauls
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] BIOS-provided physical RAM map:
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]  BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]  BIOS-e820: 000000007fff0000 - 0000000080000000 (ACPI data)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] DMI 2.4 present.
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] last_pfn = 0x7fff0 max_arch_pfn = 0x100000000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] init_memory_mapping: 0000000000000000-000000007fff0000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] last_map_addr: 7fff0000 end: 7fff0000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] RAMDISK: 379c8000 - 37fef480
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: RSDP 000FBE50, 0014 (r0 QEMU  )
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: RSDT 7FFF0000, 002C (r1 QEMU   QEMURSDT        1 QEMU        1)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: FACP 7FFF002C, 0074 (r1 QEMU   QEMUFACP        1 QEMU        1)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] FADT: X_PM1a_EVT_BLK.bit_width (16) does not match PM1_EVT_LEN (4)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: DSDT 7FFF0100, 2540 (r1   BXPC   BXDSDT        1 INTL 20061109)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: FACS 7FFF00C0, 0040
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: APIC 7FFF2640, 00E0 (r1 QEMU   QEMUAPIC        1 QEMU        1)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] No NUMA configuration found
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Faking a node at 0000000000000000-000000007fff0000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Bootmem setup node 0 0000000000000000-000000007fff0000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   NODE_DATA [0000000000001000 - 0000000000005fff]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   bootmap [000000000000a000 -  0000000000019fff] pages 10
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] (6 early reservations) ==> bootmem [0000000000 - 007fff0000]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   #2 [0000200000 - 000070493c]    TEXT DATA BSS ==> [0000200000 - 000070493c]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   #3 [00379c8000 - 0037fef480]          RAMDISK ==> [00379c8000 - 0037fef480]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   #4 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   #5 [0000008000 - 000000a000]          PGTABLE ==> [0000008000 - 000000a000]
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] found SMP MP-table at [ffff8800000fbd10] 000fbd10
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Zone PFN ranges:
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   DMA      0x00000000 -> 0x00001000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   DMA32    0x00001000 -> 0x00100000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]   Normal   0x00100000 -> 0x00100000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Movable zone start PFN for each node
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] early_node_map[2] active PFN ranges
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]     0: 0x00000000 -> 0x0000009f
May  5 02:28:32 lenny64guest1 kernel: [    0.000000]     0: 0x00000100 -> 0x0007fff0
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: PM-Timer IO Port: 0xb008
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] IOAPIC[0]: apic_id 2, version 0, address 0xfec00000, GSI 0-23
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Using ACPI (MADT) for SMP configuration information
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] SMP: Allowing 16 CPUs, 14 hotplug CPUs
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e8000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] PM: Registered nosave memory: 00000000000e8000 - 0000000000100000
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Allocating PCI resources starting at 88000000 (gap: 80000000:7ffbc000)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:16 nr_node_ids:1
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] PERCPU: Allocating 49152 bytes of per cpu data
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 515620
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Policy zone: DMA32
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Kernel command line: root=/dev/hda1 ro
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Initializing CPU#0
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes)
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Fast TSC calibration using PIT
May  5 02:28:32 lenny64guest1 kernel: [    0.000000] Detected 2266.606 MHz processor.
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] Console: colour VGA+ 80x25
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] console [tty0] enabled
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] Checking aperture...
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] No AGP bridge found
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] Memory: 2055644k/2097088k available (2463k kernel code, 388k absent, 41056k reserved, 1259k data, 464k init)
May  5 02:28:32 lenny64guest1 kernel: [    0.004014] Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.21 BogoMIPS (lpj=9066424)
May  5 02:28:32 lenny64guest1 kernel: [    0.004982] Security Framework initialized
May  5 02:28:32 lenny64guest1 kernel: [    0.005410] SELinux:  Disabled at boot.
May  5 02:28:32 lenny64guest1 kernel: [    0.005987] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
May  5 02:28:32 lenny64guest1 kernel: [    0.007713] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
May  5 02:28:32 lenny64guest1 kernel: [    0.008000] Mount-cache hash table entries: 256
May  5 02:28:32 lenny64guest1 kernel: [    0.008000] Initializing cgroup subsys ns
May  5 02:28:32 lenny64guest1 kernel: [    0.008000] Initializing cgroup subsys cpuacct
May  5 02:28:32 lenny64guest1 kernel: [    0.008010] Initializing cgroup subsys devices
May  5 02:28:32 lenny64guest1 kernel: [    0.008489] CPU: L1 I cache: 32K, L1 D cache: 32K
May  5 02:28:32 lenny64guest1 kernel: [    0.009040] CPU: L2 cache: 2048K
May  5 02:28:32 lenny64guest1 kernel: [    0.009426] CPU 0/0x0 -> Node 0
May  5 02:28:32 lenny64guest1 kernel: [    0.009860] ACPI: Core revision 20081204
May  5 02:28:32 lenny64guest1 kernel: [    0.011340] Setting APIC routing to flat
May  5 02:28:32 lenny64guest1 kernel: [    0.013152] ..TIMER: vector=0x30 apic1=0 pin1=0 apic2=-1 pin2=-1
May  5 02:28:32 lenny64guest1 kernel: [    0.053457] CPU0: Intel QEMU Virtual CPU version 0.10.0 stepping 03
May  5 02:28:32 lenny64guest1 kernel: [    0.063563] Booting processor 1 APIC 0x1 ip 0x6000
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] Initializing CPU#1
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] Calibrating delay using timer specific routine.. 4553.99 BogoMIPS (lpj=9107983)
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] CPU: L2 cache: 2048K
May  5 02:28:32 lenny64guest1 kernel: [    0.004000] CPU 1/0x1 -> Node 0
May  5 02:28:32 lenny64guest1 kernel: [    0.180153] CPU1: Intel QEMU Virtual CPU version 0.10.0 stepping 03
May  5 02:28:32 lenny64guest1 kernel: [    0.184027] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
May  5 02:28:32 lenny64guest1 kernel: [    0.192055] Brought up 2 CPUs
May  5 02:28:32 lenny64guest1 kernel: [    0.192480] Total of 2 processors activated (9087.20 BogoMIPS).
May  5 02:28:32 lenny64guest1 kernel: [    0.193288] net_namespace: 1888 bytes
May  5 02:28:32 lenny64guest1 kernel: [    0.193288] NET: Registered protocol family 16
May  5 02:28:32 lenny64guest1 kernel: [    0.196058] ACPI: bus type pci registered
May  5 02:28:32 lenny64guest1 kernel: [    0.196484] PCI: Using configuration type 1 for base access
May  5 02:28:32 lenny64guest1 kernel: [    0.200239] bio: create slab <bio-0> at 0
May  5 02:28:32 lenny64guest1 kernel: [    0.206341] ACPI: Interpreter enabled
May  5 02:28:32 lenny64guest1 kernel: [    0.206773] ACPI: (supports S0 S3 S4 S5)
May  5 02:28:32 lenny64guest1 kernel: [    0.208479] ACPI: Using IOAPIC for interrupt routing
May  5 02:28:32 lenny64guest1 kernel: [    0.212843] ACPI: No dock devices found.
May  5 02:28:32 lenny64guest1 kernel: [    0.212843] ACPI: PCI Root Bridge [PCI0] (0000:00)
May  5 02:28:32 lenny64guest1 kernel: [    0.216782] pci 0000:00:01.3: quirk: region b000-b03f claimed by PIIX4 ACPI
May  5 02:28:32 lenny64guest1 kernel: [    0.217309] pci 0000:00:01.3: quirk: region b100-b10f claimed by PIIX4 SMB
May  5 02:28:32 lenny64guest1 kernel: [    0.244182] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
May  5 02:28:32 lenny64guest1 kernel: [    0.246185] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
May  5 02:28:32 lenny64guest1 kernel: [    0.248084] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
May  5 02:28:32 lenny64guest1 kernel: [    0.249660] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
May  5 02:28:32 lenny64guest1 kernel: [    0.252165] usbcore: registered new interface driver usbfs
May  5 02:28:32 lenny64guest1 kernel: [    0.252715] usbcore: registered new interface driver hub
May  5 02:28:32 lenny64guest1 kernel: [    0.252751] usbcore: registered new device driver usb
May  5 02:28:32 lenny64guest1 kernel: [    0.256060] PCI: Using ACPI for IRQ routing
May  5 02:28:32 lenny64guest1 kernel: [    0.292029] pnp: PnP ACPI init
May  5 02:28:32 lenny64guest1 kernel: [    0.326776] ACPI: bus type pnp registered
May  5 02:28:32 lenny64guest1 kernel: [    0.329364] pnp: PnP ACPI: found 7 devices
May  5 02:28:32 lenny64guest1 kernel: [    0.329980] ACPI: ACPI bus type pnp unregistered
May  5 02:28:32 lenny64guest1 kernel: [    0.337062] NET: Registered protocol family 2
May  5 02:28:32 lenny64guest1 kernel: [    0.384400] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
May  5 02:28:32 lenny64guest1 kernel: [    0.424264] TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
May  5 02:28:32 lenny64guest1 kernel: [    0.428894] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
May  5 02:28:32 lenny64guest1 kernel: [    0.430439] TCP: Hash tables configured (established 262144 bind 65536)
May  5 02:28:32 lenny64guest1 kernel: [    0.431153] TCP reno registered
May  5 02:28:32 lenny64guest1 kernel: [    0.440570] NET: Registered protocol family 1
May  5 02:28:32 lenny64guest1 kernel: [    0.441388] checking if image is initramfs... it is
May  5 02:28:32 lenny64guest1 kernel: [    0.926395] Freeing initrd memory: 6301k freed
May  5 02:28:32 lenny64guest1 kernel: [    0.930547] audit: initializing netlink socket (disabled)
May  5 02:28:32 lenny64guest1 kernel: [    0.931023] type=2000 audit(1241515668.928:1): initialized
May  5 02:28:32 lenny64guest1 kernel: [    0.933622] HugeTLB registered 2 MB page size, pre-allocated 0 pages
May  5 02:28:32 lenny64guest1 kernel: [    0.934503] VFS: Disk quotas dquot_6.5.2
May  5 02:28:32 lenny64guest1 kernel: [    0.934991] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
May  5 02:28:32 lenny64guest1 kernel: [    0.935593] msgmni has been set to 4027
May  5 02:28:32 lenny64guest1 kernel: [    0.936416] alg: No test for stdrng (krng)
May  5 02:28:32 lenny64guest1 kernel: [    0.938660] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
May  5 02:28:32 lenny64guest1 kernel: [    0.939371] io scheduler noop registered
May  5 02:28:32 lenny64guest1 kernel: [    0.939797] io scheduler anticipatory registered
May  5 02:28:32 lenny64guest1 kernel: [    0.940286] io scheduler deadline registered
May  5 02:28:32 lenny64guest1 kernel: [    0.940780] io scheduler cfq registered (default)
May  5 02:28:32 lenny64guest1 kernel: [    0.941243] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
May  5 02:28:32 lenny64guest1 kernel: [    0.941726] pci 0000:00:01.0: PIIX3: Enabling Passive Release
May  5 02:28:32 lenny64guest1 kernel: [    0.942212] pci 0000:00:01.0: Activating ISA DMA hang workarounds
May  5 02:28:32 lenny64guest1 kernel: [    0.951852] Linux agpgart interface v0.103
May  5 02:28:32 lenny64guest1 kernel: [    0.952359] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
May  5 02:28:32 lenny64guest1 kernel: [    0.953144] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
May  5 02:28:32 lenny64guest1 kernel: [    0.954426] 00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
May  5 02:28:32 lenny64guest1 kernel: [    0.958929] brd: module loaded
May  5 02:28:32 lenny64guest1 kernel: [    0.959769] PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
May  5 02:28:32 lenny64guest1 kernel: [    0.981859] serio: i8042 KBD port at 0x60,0x64 irq 1
May  5 02:28:32 lenny64guest1 kernel: [    0.982333] serio: i8042 AUX port at 0x60,0x64 irq 12
May  5 02:28:32 lenny64guest1 kernel: [    0.982954] mice: PS/2 mouse device common for all mice
May  5 02:28:32 lenny64guest1 kernel: [    0.984137] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
May  5 02:28:32 lenny64guest1 kernel: [    0.984730] rtc0: alarms up to one day, 114 bytes nvram
May  5 02:28:32 lenny64guest1 kernel: [    0.985235] input: AT Translated Set 2 keyboard as /class/input/input0
May  5 02:28:32 lenny64guest1 kernel: [    0.985248] cpuidle: using governor ladder
May  5 02:28:32 lenny64guest1 kernel: [    0.985250] cpuidle: using governor menu
May  5 02:28:32 lenny64guest1 kernel: [    0.985258] No iBFT detected.
May  5 02:28:32 lenny64guest1 kernel: [    0.987658] TCP cubic registered
May  5 02:28:32 lenny64guest1 kernel: [    0.988031] NET: Registered protocol family 17
May  5 02:28:32 lenny64guest1 kernel: [    0.988871] registered taskstats version 1
May  5 02:28:32 lenny64guest1 kernel: [    0.989482] rtc_cmos 00:01: setting system clock to 2009-05-05 09:27:50 UTC (1241515670)
May  5 02:28:32 lenny64guest1 kernel: [    0.990244] Freeing unused kernel memory: 464k freed
May  5 02:28:32 lenny64guest1 kernel: [    1.065959] processor ACPI_CPU:00: registered as cooling_device0
May  5 02:28:32 lenny64guest1 kernel: [    1.066943] processor ACPI_CPU:01: registered as cooling_device1
May  5 02:28:32 lenny64guest1 kernel: [    1.325754] Uniform Multi-Platform E-IDE driver
May  5 02:28:32 lenny64guest1 kernel: [    1.330236] 8139too Fast Ethernet driver 0.9.28
May  5 02:28:32 lenny64guest1 kernel: [    1.330892] 8139too 0000:00:03.0: This (id 10ec:8139 rev 20) is an enhanced 8139C+ chip, use 8139cp
May  5 02:28:32 lenny64guest1 kernel: [    1.353440] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
May  5 02:28:32 lenny64guest1 kernel: [    1.354133] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11
May  5 02:28:32 lenny64guest1 kernel: [    1.357625] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
May  5 02:28:32 lenny64guest1 kernel: [    1.358629] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
May  5 02:28:32 lenny64guest1 kernel: [    1.359325] 8139cp 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 10 (level, high) -> IRQ 10
May  5 02:28:32 lenny64guest1 kernel: [    1.364546] eth0: RTL-8139C+ at 0xffffc2000095a000, 52:54:00:12:34:56, IRQ 10
May  5 02:28:32 lenny64guest1 kernel: [    1.376837] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
May  5 02:28:32 lenny64guest1 kernel: [    1.377531] e1000e: Copyright (c) 1999-2008 Intel Corporation.
May  5 02:28:32 lenny64guest1 kernel: [    1.379590] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
May  5 02:28:32 lenny64guest1 kernel: [    1.380312] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
May  5 02:28:32 lenny64guest1 kernel: [    1.412931] FDC 0 is a S82078B
May  5 02:28:32 lenny64guest1 kernel: [    1.501860] 0000:00:05.0: eth1: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
May  5 02:28:32 lenny64guest1 kernel: [    1.507930] 0000:00:05.0: eth1: Intel(R) PRO/1000 Network Connection
May  5 02:28:32 lenny64guest1 kernel: [    1.508792] 0000:00:05.0: eth1: MAC: 3, PHY: 8, PBA No: ffffff-0ff
May  5 02:28:32 lenny64guest1 kernel: [    1.509678] piix 0000:00:01.1: IDE controller (0x8086:0x7010 rev 0x00)
May  5 02:28:32 lenny64guest1 kernel: [    1.510771] piix 0000:00:01.1: not 100% native mode: will probe irqs later
May  5 02:28:32 lenny64guest1 kernel: [    1.511888]     ide0: BM-DMA at 0xc000-0xc007
May  5 02:28:32 lenny64guest1 kernel: [    1.512829]     ide1: BM-DMA at 0xc008-0xc00f
May  5 02:28:32 lenny64guest1 kernel: [    1.836303] hda: QEMU HARDDISK, ATA DISK drive
May  5 02:28:32 lenny64guest1 kernel: [    2.508387] hda: MWDMA2 mode selected
May  5 02:28:32 lenny64guest1 kernel: [    2.908184] hdc: QEMU DVD-ROM, ATAPI CD/DVD-ROM drive
May  5 02:28:32 lenny64guest1 kernel: [    3.580648] hdc: MWDMA2 mode selected
May  5 02:28:32 lenny64guest1 kernel: [    3.581712] ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
May  5 02:28:32 lenny64guest1 kernel: [    3.582555] isa bounce pool size: 16 pages
May  5 02:28:32 lenny64guest1 kernel: [    3.583217] ide1 at 0x170-0x177,0x376 on irq 15
May  5 02:28:32 lenny64guest1 kernel: [    3.598520] SCSI subsystem initialized
May  5 02:28:32 lenny64guest1 kernel: [    3.647764] ide-gd driver 1.18
May  5 02:28:32 lenny64guest1 kernel: [    3.648715] hda: max request size: 512KiB
May  5 02:28:32 lenny64guest1 kernel: [    3.649620] hda: 16384000 sectors (8388 MB) w/256KiB Cache, CHS=16253/255/63
May  5 02:28:32 lenny64guest1 kernel: [    3.654805] ide-cd driver 5.00
May  5 02:28:32 lenny64guest1 kernel: [    3.656393] hda: cache flushes supported
May  5 02:28:32 lenny64guest1 kernel: [    3.657128]  hda: hda1 hda2 < hda5 >
May  5 02:28:32 lenny64guest1 kernel: [    3.660108] ide-cd: hdc: ATAPI 4X CD-ROM drive, 512kB Cache
May  5 02:28:32 lenny64guest1 kernel: [    3.663306] Uniform CD-ROM driver Revision: 3.20
May  5 02:28:32 lenny64guest1 kernel: [    3.782707] PM: Starting manual resume from disk
May  5 02:28:32 lenny64guest1 kernel: [    3.803032] EXT3-fs: INFO: recovery required on readonly filesystem.
May  5 02:28:32 lenny64guest1 kernel: [    3.803751] EXT3-fs: write access will be enabled during recovery.
May  5 02:28:32 lenny64guest1 kernel: [    5.302058] kjournald starting.  Commit interval 5 seconds
May  5 02:28:32 lenny64guest1 kernel: [    5.302090] EXT3-fs: recovery complete.
May  5 02:28:32 lenny64guest1 kernel: [    5.308821] EXT3-fs: mounted filesystem with ordered data mode.
May  5 02:28:32 lenny64guest1 kernel: [    5.532342] udevd version 125 started
May  5 02:28:32 lenny64guest1 kernel: [    5.925537] input: Power Button (FF) as /class/input/input1
May  5 02:28:32 lenny64guest1 kernel: [    5.956326] ACPI: Power Button (FF) [PWRF]
May  5 02:28:32 lenny64guest1 kernel: [    5.966591] parport_pc 00:05: reported by Plug and Play ACPI
May  5 02:28:32 lenny64guest1 kernel: [    5.967403] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
May  5 02:28:32 lenny64guest1 kernel: [    6.090639] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
May  5 02:28:32 lenny64guest1 kernel: [    6.092050] input: PC Speaker as /class/input/input2
May  5 02:28:32 lenny64guest1 kernel: [    6.246113] input: ImExPS/2 Generic Explorer Mouse as /class/input/input3
May  5 02:28:32 lenny64guest1 kernel: [   37.729720] Adding 393552k swap on /dev/hda5.  Priority:-1 extents:1 across:393552k 
May  5 02:28:32 lenny64guest1 kernel: [   37.771561] EXT3 FS on hda1, internal journal
May  5 02:28:32 lenny64guest1 kernel: [   38.776656] loop: module loaded
May  5 02:28:32 lenny64guest1 kernel: [   39.898636] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
May  5 02:28:32 lenny64guest1 rsyslogd: [origin software="rsyslogd" swVersion="3.18.6" x-pid="1860" x-info="http://www.rsyslog.com"] restart
May  5 02:28:32 lenny64guest1 kernel: [   42.938673] NET: Registered protocol family 10
May  5 02:28:32 lenny64guest1 kernel: [   42.940835] lo: Disabled Privacy Extensions
May  5 02:28:32 lenny64guest1 /usr/sbin/gpm[2164]: *** info [daemon/startup.c(131)]: 
May  5 02:28:32 lenny64guest1 /usr/sbin/gpm[2164]: Started gpm successfully. Entered daemon mode.
May  5 02:28:51 lenny64guest1 kernel: [   62.496984] ADDRCONF(NETDEV_UP): eth1_rename: link is not ready
May  5 02:28:54 lenny64guest1 kernel: [   65.456923] e1000e: eth1_rename NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
May  5 02:28:54 lenny64guest1 kernel: [   65.458496] ADDRCONF(NETDEV_CHANGE): eth1_rename: link becomes ready
May  5 02:29:09 lenny64guest1 kernel: [   79.929586] TARGET_CORE[0]: Loading Generic Kernel Storage Engine: v3.0.0 on Linux/x86_64 on 2.6.29.2
May  5 02:29:09 lenny64guest1 kernel: [   79.931101] TARGET_CORE[0]: Initialized ConfigFS Fabric Infrastructure: v1.0.0 on Linux/x86_64 on 2.6.29.2
May  5 02:29:09 lenny64guest1 kernel: [   79.932245] SE_PC[0] - Registered Plugin Class: TRANSPORT
May  5 02:29:09 lenny64guest1 kernel: [   79.932912] PLUGIN_TRANSPORT[1] - pscsi registered
May  5 02:29:09 lenny64guest1 kernel: [   79.933553] PLUGIN_TRANSPORT[2] - stgt registered
May  5 02:29:09 lenny64guest1 kernel: [   79.934258] CORE_STGT[0]: Bus Initalization complete
May  5 02:29:09 lenny64guest1 kernel: [   79.934258] PLUGIN_TRANSPORT[4] - iblock registered
May  5 02:29:09 lenny64guest1 kernel: [   79.934258] PLUGIN_TRANSPORT[5] - rd_dr registered
May  5 02:29:09 lenny64guest1 kernel: [   79.937936] PLUGIN_TRANSPORT[6] - rd_mcp registered
May  5 02:29:09 lenny64guest1 kernel: [   79.938882] PLUGIN_TRANSPORT[7] - fileio registered
May  5 02:29:09 lenny64guest1 kernel: [   79.939821] SE_PC[1] - Registered Plugin Class: OBJ
May  5 02:29:09 lenny64guest1 kernel: [   79.940767] PLUGIN_OBJ[1] - dev registered
May  5 02:29:09 lenny64guest1 kernel: [   79.945344] Target_Core_ConfigFS: Located se_plugin: ffff88007c0998e0 plugin_name: iblock hba_type: 4 plugin_dep_id: 0
May  5 02:29:09 lenny64guest1 kernel: [   79.947103] CORE_HBA[0] - Linux-iSCSI.org iBlock HBA Driver 3.0 on Generic Target Core Stack v3.0
May  5 02:29:09 lenny64guest1 kernel: [   79.948871] CORE_HBA[0] - Attached iBlock HBA: 0 to Generic Target Core TCQ Depth: 512
May  5 02:29:09 lenny64guest1 kernel: [   79.950432] CORE_HBA[0] - Attached HBA to Generic Target Core
May  5 02:29:09 lenny64guest1 kernel: [   79.951431] IBLOCK: Allocated ib_dev for lvm_test0
May  5 02:29:09 lenny64guest1 kernel: [   79.952349] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff880037815000 se_dev_su_ptr: ffff88007d07c400
May  5 02:29:09 lenny64guest1 kernel: [   79.952849] Target_Core_ConfigFS: Set emulated VPD Unit Serial: a97e4ce21c0711de829b000c2943d57b
May  5 02:29:09 lenny64guest1 kernel: [   79.957608] Target_Core_ConfigFS: Located se_plugin: ffff88007c099918 plugin_name: rd_dr hba_type: 5 plugin_dep_id: 0
May  5 02:29:09 lenny64guest1 kernel: [   79.958789] CORE_HBA[1] - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
May  5 02:29:09 lenny64guest1 kernel: [   79.959887] CORE_HBA[1] - Attached Ramdisk HBA: 0 to Generic Target Core TCQ Depth: 256 MaxSectors: 1024
May  5 02:29:09 lenny64guest1 kernel: [   79.961070] CORE_HBA[1] - Attached HBA to Generic Target Core
May  5 02:29:09 lenny64guest1 kernel: [   79.961978] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007c9df000 se_dev_su_ptr: ffff88007bc0c6c0
May  5 02:29:09 lenny64guest1 kernel: [   79.963398] RAMDISK: Referencing Page Count: 32768
May  5 02:29:09 lenny64guest1 kernel: [   79.976051] CORE_RD[0] - Built Ramdisk Device ID: 0 space of 32768 pages in 17 tables
May  5 02:29:09 lenny64guest1 kernel: [   79.976051] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS emulation
May  5 02:29:09 lenny64guest1 kernel: [   79.976051] rd_dr: Enabling ALUA Emulation for SPC-3 device
May  5 02:29:09 lenny64guest1 kernel: [   79.980565] rd_dr: Adding to default ALUA LU Group: core/alua/lu_gps/default_lu_gp
May  5 02:29:09 lenny64guest1 kernel: [   79.982113] CORE_RD[0] - Activating Device with TCQ: 0 at Ramdisk Device ID: 0
May  5 02:29:09 lenny64guest1 kernel: [   79.983881]   Vendor: LIO-ORG   Model: RAMDISK-DR        Revision: 3.0 
May  5 02:29:09 lenny64guest1 kernel: [   79.991110]   Type:   Direct-Access                      ANSI SCSI revision: 05
May  5 02:29:09 lenny64guest1 kernel: [   79.993190] T10 VPD Unit Serial Number: 1234567890:1_0
May  5 02:29:09 lenny64guest1 kernel: [   79.994236] T10 VPD Page Length: 38
May  5 02:29:09 lenny64guest1 kernel: [   79.995091] T10 VPD Identifer Length: 34
May  5 02:29:09 lenny64guest1 kernel: [   79.995778] T10 VPD Identifier Association: addressed logical unit
May  5 02:29:09 lenny64guest1 kernel: [   79.996560] T10 VPD Identifier Type: T10 Vendor ID based
May  5 02:29:09 lenny64guest1 kernel: [   79.997212] T10 VPD ASCII Device Identifier: LIO-ORG
May  5 02:29:09 lenny64guest1 kernel: [   79.997920] CORE_RD[0] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768 pages in 17 tables, 134217728 total bytes
May  5 02:29:09 lenny64guest1 kernel: [   79.999089] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff88007d0a2c00
May  5 02:29:09 lenny64guest1 kernel: [   80.001332] Target_Core_ConfigFS: Located se_plugin: ffff88007c099918 plugin_name: rd_dr hba_type: 5 plugin_dep_id: 1
May  5 02:29:09 lenny64guest1 kernel: [   80.002518] CORE_HBA[2] - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
May  5 02:29:09 lenny64guest1 kernel: [   80.003617] CORE_HBA[2] - Attached Ramdisk HBA: 1 to Generic Target Core TCQ Depth: 256 MaxSectors: 1024
May  5 02:29:09 lenny64guest1 kernel: [   80.006041] CORE_HBA[2] - Attached HBA to Generic Target Core
May  5 02:29:09 lenny64guest1 kernel: [   80.007061] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007d1c4000 se_dev_su_ptr: ffff88007d098340
May  5 02:29:09 lenny64guest1 kernel: [   80.009412] RAMDISK: Referencing Page Count: 32768
May  5 02:29:09 lenny64guest1 kernel: [   80.020030] CORE_RD[1] - Built Ramdisk Device ID: 0 space of 32768 pages in 17 tables
May  5 02:29:09 lenny64guest1 kernel: [   80.020030] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS emulation
May  5 02:29:09 lenny64guest1 kernel: [   80.020030] rd_dr: Enabling ALUA Emulation for SPC-3 device
May  5 02:29:09 lenny64guest1 kernel: [   80.024538] rd_dr: Adding to default ALUA LU Group: core/alua/lu_gps/default_lu_gp
May  5 02:29:09 lenny64guest1 kernel: [   80.026042] CORE_RD[1] - Activating Device with TCQ: 0 at Ramdisk Device ID: 0
May  5 02:29:09 lenny64guest1 kernel: [   80.027331]   Vendor: LIO-ORG   Model: RAMDISK-DR        Revision: 3.0 
May  5 02:29:09 lenny64guest1 kernel: [   80.033340]   Type:   Direct-Access                      ANSI SCSI revision: 05
May  5 02:29:09 lenny64guest1 kernel: [   80.034768] T10 VPD Unit Serial Number: 1234567890:2_0
May  5 02:29:09 lenny64guest1 kernel: [   80.035481] T10 VPD Page Length: 38
May  5 02:29:09 lenny64guest1 kernel: [   80.036056] T10 VPD Identifer Length: 34
May  5 02:29:09 lenny64guest1 kernel: [   80.038286] T10 VPD Identifier Association: addressed logical unit
May  5 02:29:09 lenny64guest1 kernel: [   80.039292] T10 VPD Identifier Type: T10 Vendor ID based
May  5 02:29:09 lenny64guest1 kernel: [   80.040024] T10 VPD ASCII Device Identifier: LIO-ORG
May  5 02:29:09 lenny64guest1 kernel: [   80.040822] CORE_RD[1] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768 pages in 17 tables, 134217728 total bytes
May  5 02:29:09 lenny64guest1 kernel: [   80.041995] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff88007d0a2800
May  5 02:29:09 lenny64guest1 kernel: [   80.044313] Target_Core_ConfigFS: Located se_plugin: ffff88007c099988 plugin_name: fileio hba_type: 7 plugin_dep_id: 0
May  5 02:29:09 lenny64guest1 kernel: [   80.045498] CORE_HBA[3] - Linux-iSCSI.org FILEIO HBA Driver 3.0 on Generic Target Core Stack v3.0
May  5 02:29:09 lenny64guest1 kernel: [   80.046583] CORE_HBA[3] - Attached FILEIO HBA: 0 to Generic Target Core with TCQ Depth: 256 MaxSectors: 1024
May  5 02:29:09 lenny64guest1 kernel: [   80.047703] CORE_HBA[3] - Attached HBA to Generic Target Core
May  5 02:29:09 lenny64guest1 kernel: [   80.048833] FILEIO: Allocated fd_dev for ffff88007cd1cce0
May  5 02:29:09 lenny64guest1 kernel: [   80.049485] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007d14b800 se_dev_su_ptr: ffff88007d147a00
May  5 02:29:09 lenny64guest1 kernel: [   80.050974] FILEIO: Referencing Path: /tmp/fileio
May  5 02:29:09 lenny64guest1 kernel: [   80.051621] FILEIO: Referencing Size: 134217728 bytes
May  5 02:29:09 lenny64guest1 kernel: [   80.673775] fileio: Using SPC3_PERSISTENT_RESERVATIONS emulation
May  5 02:29:09 lenny64guest1 kernel: [   80.674485] fileio: Enabling ALUA Emulation for SPC-3 device
May  5 02:29:09 lenny64guest1 kernel: [   80.712769] fileio: Adding to default ALUA LU Group: core/alua/lu_gps/default_lu_gp
May  5 02:29:10 lenny64guest1 kernel: [   80.714302] CORE_FILE[0] - Activating Device with TCQ: 0 at FILEIO Device ID: 0
May  5 02:29:10 lenny64guest1 kernel: [   80.716272]   Vendor: LIO-ORG   Model: FILEIO            Revision: 3.0 
May  5 02:29:10 lenny64guest1 kernel: [   80.720275]   Type:   Direct-Access                      ANSI SCSI revision: 05
May  5 02:29:10 lenny64guest1 kernel: [   80.722177] T10 VPD Unit Serial Number: 1234567890:3_0
May  5 02:29:10 lenny64guest1 kernel: [   80.722949] T10 VPD Page Length: 34
May  5 02:29:10 lenny64guest1 kernel: [   80.723541] T10 VPD Identifer Length: 30
May  5 02:29:10 lenny64guest1 kernel: [   80.725526] T10 VPD Identifier Association: addressed logical unit
May  5 02:29:10 lenny64guest1 kernel: [   80.726558] T10 VPD Identifier Type: T10 Vendor ID based
May  5 02:29:10 lenny64guest1 kernel: [   80.726946] T10 VPD ASCII Device Identifier: LIO-ORG
May  5 02:29:10 lenny64guest1 kernel: [   80.728696] CORE_FILE[0] - Added LIO FILEIO Device ID: 0 at /tmp/fileio, 134217728 total bytes
May  5 02:29:10 lenny64guest1 kernel: [   80.730293] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff88007d0a2400
May  5 02:29:10 lenny64guest1 kernel: [   80.733419] Target_Core_ConfigFS: REGISTER -> group: ffffffffa026c100 name: iscsi
May  5 02:29:10 lenny64guest1 kernel: [   80.748954] Linux-iSCSI.org iSCSI Target Core Stack v3.0.0 on Linux/x86_64 on 2.6.29.2
May  5 02:29:10 lenny64guest1 kernel: [   80.750055] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
May  5 02:29:10 lenny64guest1 kernel: [   80.750791] Initialized struct target_fabric_configfs: ffff88007d147800 for iscsi
May  5 02:29:10 lenny64guest1 kernel: [   80.751827] <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
May  5 02:29:10 lenny64guest1 kernel: [   80.752601] LIO_TARGET[0] - Set fabric -> lio_target_fabric_configfs
May  5 02:29:10 lenny64guest1 kernel: [   80.753873] iscsi_allocate_thread_sets:195: ***OPS*** Spawned 4 thread set(s) (8 total threads).
May  5 02:29:10 lenny64guest1 kernel: [   80.755251] TARGET_CORE[iSCSI]: Allocated Discovery se_portal_group_t for endpoint: None, Portal Tag: 0
May  5 02:29:10 lenny64guest1 kernel: [   80.757870] CORE[0] - Allocated Discovery TPG
May  5 02:29:10 lenny64guest1 kernel: [   80.757870] Loading Complete.
May  5 02:29:10 lenny64guest1 kernel: [   80.759798] Target_Core_ConfigFS: REGISTER -> Located fabric: iscsi
May  5 02:29:10 lenny64guest1 kernel: [   80.760549] Target_Core_ConfigFS: REGISTER -> ffffffffa02a9bb0
May  5 02:29:10 lenny64guest1 kernel: [   80.761237] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: iscsi
May  5 02:29:10 lenny64guest1 kernel: [   80.761932] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for iscsi
May  5 02:29:10 lenny64guest1 kernel: [   80.762679] lio_target_call_coreaddtiqn(): name: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:10 lenny64guest1 kernel: [   80.763893] CORE[0] - Added iSCSI Target IQN: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:10 lenny64guest1 kernel: [   80.764541] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:10 lenny64guest1 kernel: [   80.764541] LIO_Target_ConfigFS: REGISTER -> Allocated Node: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:10 lenny64guest1 kernel: [   80.764541] lio_target_tiqn_addtpg() called: name tpgt_1
May  5 02:29:10 lenny64guest1 kernel: [   80.768137] lio_target_tiqn_addtpg() parent name: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:10 lenny64guest1 kernel: [   80.769307] lio_target_tiqn_addtpg() Using TPGT: 1
May  5 02:29:10 lenny64guest1 kernel: [   80.770009] TARGET_CORE[iSCSI]: Allocated Normal se_portal_group_t for endpoint: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6, Portal Tag: 1
May  5 02:29:10 lenny64guest1 kernel: [   80.771648] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[1] - Added iSCSI Target Portal Group
May  5 02:29:10 lenny64guest1 kernel: [   80.772875] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:10 lenny64guest1 kernel: [   80.774023] LIO_Target_ConfigFS: REGISTER -> Allocated TPG: tpgt_1
May  5 02:29:10 lenny64guest1 kernel: [   80.774757] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 PORTAL: 192.168.0.117:3260
May  5 02:29:10 lenny64guest1 kernel: [   80.776859] CORE[0] - Added Network Portal: 192.168.0.117:3260 on TCP on network device: None
May  5 02:29:10 lenny64guest1 kernel: [   80.776867] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] - Added Network Portal: 192.168.0.117:3260,1 on TCP on network device: None
May  5 02:29:10 lenny64guest1 kernel: [   80.776867] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[1] - Incremented np_exports to 1
May  5 02:29:10 lenny64guest1 kernel: [   80.780812] LIO_Target_ConfigFS: addnptotpg done!
May  5 02:29:10 lenny64guest1 kernel: [   80.850607] SCTP: Hash tables configured (established 65536 bind 65536)
May  5 02:29:10 lenny64guest1 kernel: [   80.941016] CORE[0] - Added Network Portal: 192.168.0.117:3260 on SCTP on network device: None
May  5 02:29:10 lenny64guest1 kernel: [   80.942266] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] - Added Network Portal: 192.168.0.117:3260,1 on SCTP on network device: None
May  5 02:29:10 lenny64guest1 kernel: [   80.943884] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[1] - Incremented np_exports to 1
May  5 02:29:10 lenny64guest1 kernel: [   80.946683] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 LUN: 0
May  5 02:29:10 lenny64guest1 kernel: [   80.949105] rd_dr/iSCSI: Adding to default ALUA Target Port Group: core/alua/tg_pt_gps/default_tg_pt_gp
May  5 02:29:10 lenny64guest1 kernel: [   80.950587] iSCSI_TPG[1]_LUN[0] - Activated iSCSI Logical Unit from CORE HBA: 1
May  5 02:29:10 lenny64guest1 kernel: [   80.951611] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_0
May  5 02:29:10 lenny64guest1 kernel: [   80.953594] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 LUN: 1
May  5 02:29:10 lenny64guest1 kernel: [   80.955813] rd_dr/iSCSI: Adding to default ALUA Target Port Group: core/alua/tg_pt_gps/default_tg_pt_gp
May  5 02:29:10 lenny64guest1 kernel: [   80.958695] iSCSI_TPG[1]_LUN[1] - Activated iSCSI Logical Unit from CORE HBA: 2
May  5 02:29:10 lenny64guest1 kernel: [   80.959736] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_1
May  5 02:29:10 lenny64guest1 kernel: [   80.960956] Disabling iSCSI Authentication Methods for TPG: 1.
May  5 02:29:10 lenny64guest1 kernel: [   80.962059] iSCSI_TPG[1] - Generate Initiator Portal Group ACLs: Enabled
May  5 02:29:10 lenny64guest1 kernel: [   80.962684] iSCSI_TPG[1] - Demo Mode Write Protect bit: OFF
May  5 02:29:10 lenny64guest1 kernel: [   80.962684] iSCSI_TPG[1] - Set Default CmdSN TCQ Depth to 4
May  5 02:29:10 lenny64guest1 kernel: [   80.964587] iSCSI_TPG[1] - Enabled iSCSI Target Portal Group
May  5 02:29:11 lenny64guest1 kernel: [   82.024298] Received iSCSI login request from 192.168.0.113 on TCP Network Portal 192.168.0.117:3260
May  5 02:29:11 lenny64guest1 kernel: [   82.025462] Located Storage Object: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:11 lenny64guest1 kernel: [   82.026588] Located Portal Group Object: 1
May  5 02:29:11 lenny64guest1 kernel: [   82.027224] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:29:11 lenny64guest1 kernel: [   82.028240] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:29:11 lenny64guest1 kernel: [   82.028240] iSCSI_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node: iqn.1994-05.com.fedora:8a8f1b7baf9b
May  5 02:29:11 lenny64guest1 kernel: [   82.028240] ------------------------------------------------------------------
May  5 02:29:11 lenny64guest1 kernel: [   82.028240] HeaderDigest:                 None
May  5 02:29:11 lenny64guest1 kernel: [   82.032492] DataDigest:                   None
May  5 02:29:11 lenny64guest1 kernel: [   82.033124] MaxRecvDataSegmentLength:     131072
May  5 02:29:11 lenny64guest1 kernel: [   82.033752] IFMarker:                     No
May  5 02:29:11 lenny64guest1 kernel: [   82.034368] OFMarker:                     No
May  5 02:29:11 lenny64guest1 kernel: [   82.034979] ------------------------------------------------------------------
May  5 02:29:11 lenny64guest1 kernel: [   82.036003] ------------------------------------------------------------------
May  5 02:29:11 lenny64guest1 kernel: [   82.036467] MaxConnections:               1
May  5 02:29:11 lenny64guest1 kernel: [   82.036467] TargetName:                   iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:11 lenny64guest1 kernel: [   82.036467] InitiatorName:                iqn.1994-05.com.fedora:8a8f1b7baf9b
May  5 02:29:11 lenny64guest1 kernel: [   82.036467] TargetAlias:                  LIO Target
May  5 02:29:11 lenny64guest1 kernel: [   82.041606] InitiatorAlias:               barret.risingtidesystems.com
May  5 02:29:11 lenny64guest1 kernel: [   82.042303] TargetPortalGroupTag:         1
May  5 02:29:11 lenny64guest1 kernel: [   82.042900] InitialR2T:                   Yes
May  5 02:29:11 lenny64guest1 kernel: [   82.043515] ImmediateData:                Yes
May  5 02:29:11 lenny64guest1 kernel: [   82.044127] MaxBurstLength:               262144
May  5 02:29:11 lenny64guest1 kernel: [   82.044755] FirstBurstLength:             65536
May  5 02:29:11 lenny64guest1 kernel: [   82.045387] DefaultTime2Wait:             2
May  5 02:29:11 lenny64guest1 kernel: [   82.045597] DefaultTime2Retain:           0
May  5 02:29:11 lenny64guest1 kernel: [   82.045597] MaxOutstandingR2T:            1
May  5 02:29:11 lenny64guest1 kernel: [   82.045597] DataPDUInOrder:               Yes
May  5 02:29:11 lenny64guest1 kernel: [   82.045597] DataSequenceInOrder:          Yes
May  5 02:29:11 lenny64guest1 kernel: [   82.048504] ErrorRecoveryLevel:           0
May  5 02:29:11 lenny64guest1 kernel: [   82.049117] SessionType:                  Normal
May  5 02:29:11 lenny64guest1 kernel: [   82.049745] ------------------------------------------------------------------
May  5 02:29:11 lenny64guest1 kernel: [   82.050770] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr: ffff8800684eae00
May  5 02:29:11 lenny64guest1 kernel: [   82.051506] iSCSI Login successful on CID: 0 from 192.168.0.113 to 192.168.0.117:3260,1
May  5 02:29:11 lenny64guest1 kernel: [   82.052497] Incremented iSCSI Connection count to 1 from node: iqn.1994-05.com.fedora:8a8f1b7baf9b
May  5 02:29:11 lenny64guest1 kernel: [   82.052497] Established iSCSI session from node: iqn.1994-05.com.fedora:8a8f1b7baf9b
May  5 02:29:11 lenny64guest1 kernel: [   82.052497] Incremented number of active iSCSI sessions to 1 on iSCSI Target Portal Group: 1
May  5 02:29:15 lenny64guest1 kernel: [   86.372863] Received iSCSI login request from 192.168.0.111 on TCP Network Portal 192.168.0.117:3260
May  5 02:29:15 lenny64guest1 kernel: [   86.375964] Located Storage Object: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:15 lenny64guest1 kernel: [   86.377124] Located Portal Group Object: 1
May  5 02:29:15 lenny64guest1 kernel: [   86.377746] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:29:15 lenny64guest1 kernel: [   86.378832] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:29:15 lenny64guest1 kernel: [   86.379914] iSCSI_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node: iqn.1993-08.org.debian:01:2dadf92d0ef
May  5 02:29:15 lenny64guest1 kernel: [   86.381340] ------------------------------------------------------------------
May  5 02:29:15 lenny64guest1 kernel: [   86.382383] HeaderDigest:                 None
May  5 02:29:15 lenny64guest1 kernel: [   86.383006] DataDigest:                   None
May  5 02:29:15 lenny64guest1 kernel: [   86.383631] MaxRecvDataSegmentLength:     131072
May  5 02:29:15 lenny64guest1 kernel: [   86.384299] IFMarker:                     No
May  5 02:29:15 lenny64guest1 kernel: [   86.386330] OFMarker:                     No
May  5 02:29:15 lenny64guest1 kernel: [   86.387210] ------------------------------------------------------------------
May  5 02:29:15 lenny64guest1 kernel: [   86.388855] ------------------------------------------------------------------
May  5 02:29:15 lenny64guest1 kernel: [   86.390364] MaxConnections:               1
May  5 02:29:15 lenny64guest1 kernel: [   86.391253] TargetName:                   iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:29:15 lenny64guest1 kernel: [   86.392956] InitiatorName:                iqn.1993-08.org.debian:01:2dadf92d0ef
May  5 02:29:15 lenny64guest1 kernel: [   86.394463] TargetAlias:                  LIO Target
May  5 02:29:15 lenny64guest1 kernel: [   86.395392] InitiatorAlias:               ubuntu
May  5 02:29:15 lenny64guest1 kernel: [   86.396342] TargetPortalGroupTag:         1
May  5 02:29:15 lenny64guest1 kernel: [   86.397233] InitialR2T:                   Yes
May  5 02:29:15 lenny64guest1 kernel: [   86.398123] ImmediateData:                Yes
May  5 02:29:15 lenny64guest1 kernel: [   86.399022] MaxBurstLength:               262144
May  5 02:29:15 lenny64guest1 kernel: [   86.399931] FirstBurstLength:             65536
May  5 02:29:15 lenny64guest1 kernel: [   86.400883] DefaultTime2Wait:             2
May  5 02:29:15 lenny64guest1 kernel: [   86.401765] DefaultTime2Retain:           0
May  5 02:29:15 lenny64guest1 kernel: [   86.402651] MaxOutstandingR2T:            1
May  5 02:29:15 lenny64guest1 kernel: [   86.403530] DataPDUInOrder:               Yes
May  5 02:29:15 lenny64guest1 kernel: [   86.404488] DataSequenceInOrder:          Yes
May  5 02:29:15 lenny64guest1 kernel: [   86.405396] ErrorRecoveryLevel:           0
May  5 02:29:15 lenny64guest1 kernel: [   86.406292] SessionType:                  Normal
May  5 02:29:15 lenny64guest1 kernel: [   86.407194] ------------------------------------------------------------------
May  5 02:29:15 lenny64guest1 kernel: [   86.408831] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr: ffff8800684ea000
May  5 02:29:15 lenny64guest1 kernel: [   86.409885] iSCSI Login successful on CID: 0 from 192.168.0.111 to 192.168.0.117:3260,1
May  5 02:29:15 lenny64guest1 kernel: [   86.411396] Incremented iSCSI Connection count to 1 from node: iqn.1993-08.org.debian:01:2dadf92d0ef
May  5 02:29:15 lenny64guest1 kernel: [   86.412540] Established iSCSI session from node: iqn.1993-08.org.debian:01:2dadf92d0ef
May  5 02:29:15 lenny64guest1 kernel: [   86.413606] Incremented number of active iSCSI sessions to 2 on iSCSI Target Portal Group: 1
May  5 02:36:08 lenny64guest1 kernel: [  499.471422] Received iSCSI login request from 192.168.0.116 on TCP Network Portal 192.168.0.117:3260
May  5 02:36:08 lenny64guest1 kernel: [  499.472848] ------------------------------------------------------------------
May  5 02:36:08 lenny64guest1 kernel: [  499.474097] HeaderDigest:                 None
May  5 02:36:08 lenny64guest1 kernel: [  499.474888] DataDigest:                   None
May  5 02:36:08 lenny64guest1 kernel: [  499.475521] MaxRecvDataSegmentLength:     8192
May  5 02:36:08 lenny64guest1 kernel: [  499.476154] IFMarker:                     No
May  5 02:36:08 lenny64guest1 kernel: [  499.476769] OFMarker:                     No
May  5 02:36:08 lenny64guest1 kernel: [  499.477515] ------------------------------------------------------------------
May  5 02:36:08 lenny64guest1 kernel: [  499.478703] ------------------------------------------------------------------
May  5 02:36:08 lenny64guest1 kernel: [  499.479745] InitiatorName:                iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:08 lenny64guest1 kernel: [  499.480845] TargetAlias:                  LIO Target
May  5 02:36:08 lenny64guest1 kernel: [  499.481612] InitiatorAlias:               subjekt
May  5 02:36:08 lenny64guest1 kernel: [  499.482249] TargetPortalGroupTag:         1
May  5 02:36:08 lenny64guest1 kernel: [  499.482870] DefaultTime2Wait:             2
May  5 02:36:08 lenny64guest1 kernel: [  499.483481] DefaultTime2Retain:           0
May  5 02:36:08 lenny64guest1 kernel: [  499.484122] ErrorRecoveryLevel:           0
May  5 02:36:08 lenny64guest1 kernel: [  499.484744] SessionType:                  Discovery
May  5 02:36:08 lenny64guest1 kernel: [  499.485461] ------------------------------------------------------------------
May  5 02:36:08 lenny64guest1 kernel: [  499.486714] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr: ffff8800684ea800
May  5 02:36:08 lenny64guest1 kernel: [  499.487451] iSCSI Login successful on CID: 0 from 192.168.0.116 to 192.168.0.117:3260,0
May  5 02:36:08 lenny64guest1 kernel: [  499.488654] Incremented iSCSI Connection count to 1 from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:08 lenny64guest1 kernel: [  499.489930] Established iSCSI session from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:08 lenny64guest1 kernel: [  499.491234] Incremented number of active iSCSI sessions to 1 on iSCSI Target Portal Group: 0
May  5 02:36:08 lenny64guest1 kernel: [  499.492724] iscsi_allocate_thread_sets:195: ***OPS*** Spawned 1 thread set(s) (2 total threads).
May  5 02:36:10 lenny64guest1 kernel: [  501.689089] Decremented iSCSI connection count to 0 from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:10 lenny64guest1 kernel: [  501.690948] TARGET_CORE[iSCSI]: Deregistered fabric_sess
May  5 02:36:10 lenny64guest1 kernel: [  501.691656] Released iSCSI session from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:10 lenny64guest1 kernel: [  501.692786] Decremented number of active iSCSI Sessions on iSCSI TPG: 0 to 0
May  5 02:36:30 lenny64guest1 kernel: [  521.627616] Received iSCSI login request from 192.168.0.116 on TCP Network Portal 192.168.0.117:3260
May  5 02:36:30 lenny64guest1 kernel: [  521.628783] Located Storage Object: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:36:30 lenny64guest1 kernel: [  521.629991] Located Portal Group Object: 1
May  5 02:36:30 lenny64guest1 kernel: [  521.630686] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:36:30 lenny64guest1 kernel: [  521.631786] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:36:30 lenny64guest1 kernel: [  521.632880] iSCSI_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:30 lenny64guest1 kernel: [  521.634266] ------------------------------------------------------------------
May  5 02:36:30 lenny64guest1 kernel: [  521.635322] HeaderDigest:                 None
May  5 02:36:30 lenny64guest1 kernel: [  521.635954] DataDigest:                   None
May  5 02:36:30 lenny64guest1 kernel: [  521.636584] MaxRecvDataSegmentLength:     131072
May  5 02:36:30 lenny64guest1 kernel: [  521.637222] IFMarker:                     No
May  5 02:36:30 lenny64guest1 kernel: [  521.637863] OFMarker:                     No
May  5 02:36:30 lenny64guest1 kernel: [  521.638562] ------------------------------------------------------------------
May  5 02:36:30 lenny64guest1 kernel: [  521.639665] ------------------------------------------------------------------
May  5 02:36:30 lenny64guest1 kernel: [  521.641312] MaxConnections:               1
May  5 02:36:30 lenny64guest1 kernel: [  521.643701] TargetName:                   iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:36:30 lenny64guest1 kernel: [  521.644866] InitiatorName:                iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:30 lenny64guest1 kernel: [  521.646013] TargetAlias:                  LIO Target
May  5 02:36:30 lenny64guest1 kernel: [  521.658853] InitiatorAlias:               subjekt
May  5 02:36:30 lenny64guest1 kernel: [  521.659506] TargetPortalGroupTag:         1
May  5 02:36:30 lenny64guest1 kernel: [  521.660109] InitialR2T:                   Yes
May  5 02:36:30 lenny64guest1 kernel: [  521.660733] ImmediateData:                Yes
May  5 02:36:30 lenny64guest1 kernel: [  521.661393] MaxBurstLength:               262144
May  5 02:36:30 lenny64guest1 kernel: [  521.662079] FirstBurstLength:             65536
May  5 02:36:30 lenny64guest1 kernel: [  521.662722] DefaultTime2Wait:             2
May  5 02:36:30 lenny64guest1 kernel: [  521.663354] DefaultTime2Retain:           0
May  5 02:36:30 lenny64guest1 kernel: [  521.663971] MaxOutstandingR2T:            1
May  5 02:36:30 lenny64guest1 kernel: [  521.664603] DataPDUInOrder:               Yes
May  5 02:36:30 lenny64guest1 kernel: [  521.665256] DataSequenceInOrder:          Yes
May  5 02:36:30 lenny64guest1 kernel: [  521.666121] ErrorRecoveryLevel:           0
May  5 02:36:30 lenny64guest1 kernel: [  521.666736] SessionType:                  Normal
May  5 02:36:30 lenny64guest1 kernel: [  521.667373] ------------------------------------------------------------------
May  5 02:36:30 lenny64guest1 kernel: [  521.668445] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr: ffff8800684ea200
May  5 02:36:30 lenny64guest1 kernel: [  521.669193] iSCSI Login successful on CID: 0 from 192.168.0.116 to 192.168.0.117:3260,1
May  5 02:36:30 lenny64guest1 kernel: [  521.670354] Incremented iSCSI Connection count to 1 from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:30 lenny64guest1 kernel: [  521.671467] Established iSCSI session from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:36:30 lenny64guest1 kernel: [  521.672538] Incremented number of active iSCSI sessions to 3 on iSCSI Target Portal Group: 1
May  5 02:48:48 lenny64guest1 kernel: [ 1259.452732] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 LUN: 2
May  5 02:49:01 lenny64guest1 kernel: [ 1272.273663] fileio/iSCSI: Adding to default ALUA Target Port Group: core/alua/tg_pt_gps/default_tg_pt_gp
May  5 02:49:01 lenny64guest1 kernel: [ 1272.274807] iSCSI_TPG[1]_LUN[2] - Activated iSCSI Logical Unit from CORE HBA: 3
May  5 02:49:01 lenny64guest1 kernel: [ 1272.275889] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.276846] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.276846] TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.276846] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.280397] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.281504] TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.282607] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.283740] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.284377] TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:01 lenny64guest1 kernel: [ 1272.284377] LIO_Target_ConfigFS: Created Port Symlink fileio -> lun_2
May  5 02:49:21 lenny64guest1 kernel: [ 1291.750986] Decremented iSCSI connection count to 0 from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:49:21 lenny64guest1 kernel: [ 1291.752197] TARGET_CORE[iSCSI]: Deregistered fabric_sess
May  5 02:49:21 lenny64guest1 kernel: [ 1291.752924] Released iSCSI session from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:49:21 lenny64guest1 kernel: [ 1291.753986] Decremented number of active iSCSI Sessions on iSCSI TPG: 1 to 2
May  5 02:49:22 lenny64guest1 kernel: [ 1293.680754] Received iSCSI login request from 192.168.0.116 on TCP Network Portal 192.168.0.117:3260
May  5 02:49:22 lenny64guest1 kernel: [ 1293.681922] Located Storage Object: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:49:22 lenny64guest1 kernel: [ 1293.683034] Located Portal Group Object: 1
May  5 02:49:22 lenny64guest1 kernel: [ 1293.683653] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:22 lenny64guest1 kernel: [ 1293.684800] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:22 lenny64guest1 kernel: [ 1293.685904] TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in Demo Mode
May  5 02:49:22 lenny64guest1 kernel: [ 1293.686994] iSCSI_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:49:22 lenny64guest1 kernel: [ 1293.688298] ------------------------------------------------------------------
May  5 02:49:22 lenny64guest1 kernel: [ 1293.689418] HeaderDigest:                 None
May  5 02:49:22 lenny64guest1 kernel: [ 1293.690047] DataDigest:                   None
May  5 02:49:22 lenny64guest1 kernel: [ 1293.690676] MaxRecvDataSegmentLength:     131072
May  5 02:49:22 lenny64guest1 kernel: [ 1293.691310] IFMarker:                     No
May  5 02:49:22 lenny64guest1 kernel: [ 1293.691931] OFMarker:                     No
May  5 02:49:22 lenny64guest1 kernel: [ 1293.692618] ------------------------------------------------------------------
May  5 02:49:22 lenny64guest1 kernel: [ 1293.693661] ------------------------------------------------------------------
May  5 02:49:22 lenny64guest1 kernel: [ 1293.694691] MaxConnections:               1
May  5 02:49:22 lenny64guest1 kernel: [ 1293.695303] TargetName:                   iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
May  5 02:49:22 lenny64guest1 kernel: [ 1293.696444] InitiatorName:                iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:49:22 lenny64guest1 kernel: [ 1293.697903] TargetAlias:                  LIO Target
May  5 02:49:22 lenny64guest1 kernel: [ 1293.698560] InitiatorAlias:               subjekt
May  5 02:49:22 lenny64guest1 kernel: [ 1293.699198] TargetPortalGroupTag:         1
May  5 02:49:22 lenny64guest1 kernel: [ 1293.699818] InitialR2T:                   Yes
May  5 02:49:22 lenny64guest1 kernel: [ 1293.712835] ImmediateData:                Yes
May  5 02:49:23 lenny64guest1 kernel: [ 1293.713596] MaxBurstLength:               262144
May  5 02:49:23 lenny64guest1 kernel: [ 1293.714227] FirstBurstLength:             65536
May  5 02:49:23 lenny64guest1 kernel: [ 1293.714859] DefaultTime2Wait:             2
May  5 02:49:23 lenny64guest1 kernel: [ 1293.715481] DefaultTime2Retain:           0
May  5 02:49:23 lenny64guest1 kernel: [ 1293.716105] MaxOutstandingR2T:            1
May  5 02:49:23 lenny64guest1 kernel: [ 1293.716801] DataPDUInOrder:               Yes
May  5 02:49:23 lenny64guest1 kernel: [ 1293.717559] DataSequenceInOrder:          Yes
May  5 02:49:23 lenny64guest1 kernel: [ 1293.718191] ErrorRecoveryLevel:           0
May  5 02:49:23 lenny64guest1 kernel: [ 1293.718812] SessionType:                  Normal
May  5 02:49:23 lenny64guest1 kernel: [ 1293.719448] ------------------------------------------------------------------
May  5 02:49:23 lenny64guest1 kernel: [ 1293.720574] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr: ffff8800684ea800
May  5 02:49:23 lenny64guest1 kernel: [ 1293.721456] iSCSI Login successful on CID: 0 from 192.168.0.116 to 192.168.0.117:3260,1
May  5 02:49:23 lenny64guest1 kernel: [ 1293.722523] Incremented iSCSI Connection count to 1 from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:49:23 lenny64guest1 kernel: [ 1293.723635] Established iSCSI session from node: iqn.1993-08.org.debian:01.2fed72c26a4c
May  5 02:49:23 lenny64guest1 kernel: [ 1293.724770] Incremented number of active iSCSI sessions to 3 on iSCSI Target Portal Group: 1
May  5 03:20:41 lenny64guest1 kernel: [ 3172.321826] Clocksource tsc unstable (delta = 4687268810 ns)

Re: KVM x86_64 with SR-IOV..? (device passthrough with LIO-Target v3.0)

[Nicholas A. Bellinger]

On Tue, 2009-05-05 at 03:43 -0700, Nicholas A. Bellinger wrote:
> On Tue, 2009-05-05 at 09:42 +0800, Yu Zhao wrote:
> > Hi,
> > 
> > The VF also works in the host if the VF driver is programed properly.
> > So it would be easier to develop the VF driver in the host and then
> > verify the VF driver in the guest.
> > 
> > BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
> > the CONFIG_PCI_IOV in the kernel .config?
> > 
> > Thanks,
> > Yu
> > 
> 
> Greetings Yu and Sheng,
> 
> So the original attachment was for the v2.6.29-fc11 host kernel output,
> I ended up jumping to v2.6.30-rc3 (and making sure CONFIG_PCI_IOV was
> enabled) for KVM host with kvm-85 and now things are looking quite
> stable for me.
> 
> So far I have been able to successfully push LIO-Target v3.0 traffic
> *inside* a v2.6.29.2 KVM guest via the onboard e1000e (02:00.0) port
> from another Linux/iSCSI Initiator machine using a Intel 1 Gb/sec port.
> I am running badblocks tests to iSCSI Logical Units for RAMDISK_DR and
> FILEIO storage objects (in the KVM Guest), and they are passing
> validation and I am seeing ~500 Mb/sec of throughput and very low CPU
> usage in the KVM guests.
> 

Ok I am seeing another issue with the e1000e port on 02:00.0..:

As i start to push multiple badblocks tests RAMDISK_DR iSCSI Logical
units into KVM Guest running LIO v2.6.29.2 from the external Linux/iSCSI
Initiator machine, after about 100 GB of iSCSI traffic, I see the
following exception in KVM host v2.6.30-rc3:

DRHD: handling fault status reg 2
DMAR:[DMA Write] Request device [02:00.0] fault addr 7fc958b010000 
DMAR:[fault reason 04] Access beyond MGAW
pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X

I am able to restart the LIO-Target KVM Guest and the Linux/iSCSI
Initiators are able to reconnect..  Wow, very cool..

Not sure if this is a bug in the target_core_mod RAMDISK_DR subsystem
plugin (mapping struct iovec to internally allocated struct page) or
what.  I will have to look at the DMAR code to understand what this
exception means..

--nab

> One issue I did notice while using the pci-stub method of
> device-assignment with same e1000 port (02:00.0) was while using an
> iSCSI Initiator (Open-iSCSI) on the KVM Host machine and doing sustained
> traffic into the LIO-Target KVM Guest on the same local KVM host to max
> out traffic between the other onboard e1000e port (03.00.0), I see the
> following:
> 
> pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> assign device: host bdf = 2:0:0
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
> scsi4 : iSCSI Initiator over TCP/IP
> scsi 4:0:0:0: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI: 5
> sd 4:0:0:0: Attached scsi generic sg1 type 0
> scsi 4:0:0:1: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI: 5
> sd 4:0:0:1: Attached scsi generic sg2 type 0
> sd 4:0:0:0: [sdb] 262144 512-byte hardware sectors: (134 MB/128 MiB)
> sd 4:0:0:1: [sdc] 262144 512-byte hardware sectors: (134 MB/128 MiB)
> sd 4:0:0:0: [sdb] Write Protect is off
> sd 4:0:0:0: [sdb] Mode Sense: 2f 00 00 00
> sd 4:0:0:1: [sdc] Write Protect is off
> sd 4:0:0:1: [sdc] Mode Sense: 2f 00 00 00
> sd 4:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
> sd 4:0:0:1: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
>  sdb:<6> sdc: unknown partition table
> sd 4:0:0:0: [sdb] Attached SCSI disk
>  unknown partition table
> sd 4:0:0:1: [sdc] Attached SCSI disk
> ------------[ cut here ]------------
> WARNING: at kernel/irq/manage.c:260 enable_irq+0x36/0x50()
> Hardware name: empty
> Unbalanced enable for IRQ 59
> Modules linked in: ipt_REJECT xt_tcpudp bridge stp sunrpc iptable_filter ip_tables xt_state nf_conntrack ip6table_filter ip6_tables x_tables ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table ext3 jbd loop dm_multipath scsi_dh kvm_intel kvm uinput i2c_i801 firewire_ohci joydev firewire_core sg i2c_core 8250_pnp crc_itu_t e1000e 8250 serial_core rtc_cmos pcspkr serio_raw rtc_core rtc_lib button sd_mod dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod uhci_hcd ohci_hcd ehci_hcd ata_piix libata scsi_mod [last unloaded: microcode]
> Pid: 51, comm: events/0 Tainted: G        W  2.6.30-rc3 #11
> Call Trace:
>  [<ffffffff80235fee>] ? warn_slowpath+0xcb/0xe8
>  [<ffffffff80253a7c>] ? generic_exec_single+0x6a/0x88
>  [<ffffffff8022acec>] ? update_curr+0x67/0xeb
>  [<ffffffffa0198748>] ? vcpu_kick_intr+0x0/0x1 [kvm]
>  [<ffffffff8020a5d8>] ? __switch_to+0xb6/0x274
>  [<ffffffff8022b70a>] ? __dequeue_entity+0x1b/0x2f
>  [<ffffffffa01ac7e4>] ? kvm_irq_delivery_to_apic+0xb3/0xf7 [kvm]
>  [<ffffffffa01aa4d4>] ? __apic_accept_irq+0x15a/0x173 [kvm]
>  [<ffffffffa01ac883>] ? kvm_set_msi+0x5b/0x60 [kvm]
>  [<ffffffff80266d97>] ? enable_irq+0x36/0x50
>  [<ffffffffa0195ab5>] ? kvm_assigned_dev_interrupt_work_handler+0x6d/0xbc [kvm]
>  [<ffffffff802449fa>] ? worker_thread+0x182/0x223
>  [<ffffffff8024820b>] ? autoremove_wake_function+0x0/0x2a
>  [<ffffffff80244878>] ? worker_thread+0x0/0x223
>  [<ffffffff80244878>] ? worker_thread+0x0/0x223
>  [<ffffffff80247e72>] ? kthread+0x54/0x7e
>  [<ffffffff8020cb0a>] ? child_rip+0xa/0x20
>  [<ffffffff804d0af5>] ? _spin_lock+0x5/0x8
>  [<ffffffff80247e1e>] ? kthread+0x0/0x7e
>  [<ffffffff8020cb00>] ? child_rip+0x0/0x20
> ---[ end trace 3fbc2dd20bf89ef1 ]---
>  connection1:0: ping timeout of 5 secs expired, last rx 4295286327, last ping 4295285518, now 4295286768
>  connection1:0: detected conn error (1011)
> 
> Attached are the v2.6.30-rc3 KVM host and v2.6.29.2 KVM guest dmesg
> output.  When the 'Unbalanced enable for IRQ 59' happens on the KVM
> host, I do not see any exceptions in KVM guest (other than the iSCSI
> connections drop), but it requires a restart of KVM+qemu-system-x86_64
> to get the e1000e port back up.
> 
> Other than that loopback scenario, things are looking good quite good
> with this combination of kvm-85 kernel+guest so far for me.  I did end
> up taking out the two 8x function 2x Path/Function PCIe IOV adapters for
> now, as it seemed to have an effect on stability with all of MSI-X
> interrupts enabled on the KVM host for 16 virtual adapters.
> 
> I will keep testing with e1000e ports and let the list know the
> progress.  Thanks for your comments!
> 
> --nab
> 
> > On Mon, May 04, 2009 at 06:40:36PM +0800, Nicholas A. Bellinger wrote:
> > > On Mon, 2009-05-04 at 17:49 +0800, Sheng Yang wrote:
> > > > On Monday 04 May 2009 17:11:59 Nicholas A. Bellinger wrote:
> > > > > On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> > > > > > On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > > > > > > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > > > > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > > > > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > > > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A. Bellinger
> > > > wrote:
> > > > > > > > > > > Greetings KVM folks,
> > > > > > > > > > >
> > > > > > > > > > > I wondering if any information exists for doing SR-IOV on the
> > > > > > > > > > > new VT-d capable chipsets with KVM..?  From what I understand
> > > > > > > > > > > the patches for doing this with KVM are floating around, but I
> > > > > > > > > > > have been unable to find any user-level docs for actually
> > > > > > > > > > > making it all go against a upstream v2.6.30-rc3 code..
> > > > > > > > > > >
> > > > > > > > > > > So far I have been doing IOV testing with Xen 3.3 and
> > > > > > > > > > > 3.4.0-pre, and I am really hoping to be able to jump to KVM for
> > > > > > > > > > > single-function and and then multi-function SR-IOV.  I know
> > > > > > > > > > > that the VM migration stuff for IOV in Xen is up and running,
> > > > > > > > > > > and I assume it is being worked in for KVM instance migration
> > > > > > > > > > > as well..? This part is less important (at least for me :-)
> > > > > > > > > > > than getting a stable SR-IOV setup running under the KVM
> > > > > > > > > > > hypervisor..  Does anyone have any pointers for this..?
> > > > > > > > > > >
> > > > > > > > > > > Any comments or suggestions are appreciated!
> > > > > > > > > >
> > > > > > > > > > Hi Nicholas
> > > > > > > > > >
> > > > > > > > > > The patches are not floating around now. As you know, SR-IOV for
> > > > > > > > > > Linux have been in 2.6.30, so then you can use upstream KVM and
> > > > > > > > > > qemu-kvm(or recent released kvm-85) with 2.6.30-rc3 as host
> > > > > > > > > > kernel. And some time ago, there are several SRIOV related
> > > > > > > > > > patches for qemu-kvm, and now they all have been checked in.
> > > > > > > > > >
> > > > > > > > > > And for KVM, the extra document is not necessary, for you can
> > > > > > > > > > simple assign a VF to guest like any other devices. And how to
> > > > > > > > > > create VF is specific for each device driver. So just create a VF
> > > > > > > > > > then assign it to KVM guest is fine.
> > > > > > > > >
> > > > > > > > > Greetings Sheng,
> > > > > > > > >
> > > > > > > > > So, I have been trying the latest kvm-85 release on a v2.6.30-rc3
> > > > > > > > > checkout from linux-2.6.git on a CentOS 5u3 x86_64 install on Intel
> > > > > > > > > IOH-5520 based dual socket Nehalem board.  I have enabled DMAR and
> > > > > > > > > Interrupt Remapping my KVM host using v2.6.30-rc3 and from what I
> > > > > > > > > can tell, the KVM_CAP_* defines from libkvm are enabled with
> > > > > > > > > building kvm-85 after './configure
> > > > > > > > > --kerneldir=/usr/src/linux-2.6.git' and the PCI passthrough code is
> > > > > > > > > being enabled in
> > > > > > > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > > > > > > >
> > > > > > > > > >From there, I use the freshly installed qemu-x86_64-system binary
> > > > > > > > > > to
> > > > > > > > >
> > > > > > > > > start a Debian 5 x86_64 HVM (that previously had been moving
> > > > > > > > > network packets under Xen for PCIe passthrough). I see the MSI-X
> > > > > > > > > interrupt remapping working on the KVM host for the passed
> > > > > > > > > -pcidevice, and the MMIO mappings from the qemu build that I also
> > > > > > > > > saw while using Xen/qemu-dm built with PCI passthrough are there as
> > > > > > > > > well..
> > > > > > > >
> > > > > > > > Hi Nicholas
> > > > > > > >
> > > > > > > > > But while the KVM guest is booting, I see the following
> > > > > > > > > exception(s) from qemu-x86_64-system for one of the VFs for a
> > > > > > > > > multi-function PCIe device:
> > > > > > > > >
> > > > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > > > >
> > > > > > > > This one is mostly harmless.
> > > > > > >
> > > > > > > Ok, good to know..  :-)
> > > > > > >
> > > > > > > > > I try with one of the on-board e1000e ports (02:00.0) and I see the
> > > > > > > > > same exception along with some MSI-X exceptions from
> > > > > > > > > qemu-x86_64-system in KVM guest.. However, I am still able to see
> > > > > > > > > the e1000e and the other vxge multi-function device with lspci, but
> > > > > > > > > I am unable to dhcp or ping with the e1000e and VF from
> > > > > > > > > multi-function device fails to register the MSI-X interrupt in the
> > > > > > > > > guest..
> > > > > > > >
> > > > > > > > Did you see the interrupt in the guest and host side?
> > > > > > >
> > > > > > > Ok, I am restarting the e1000e test with a fresh Fedora 11 install and
> > > > > > > KVM host kernel 2.6.29.1-111.fc11.x86_64.   After unbinding and
> > > > > > > attaching the e1000e single-function device at 02:00.0 to pci-stub
> > > > > > > with:
> > > > > > >
> > > > > > >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> > > > > > >    echo 0000:02:00.0 > /sys/bus/pci/devices/0000:02:00.0/driver/unbind
> > > > > > >    echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> > > > > > >
> > > > > > > I see the following the KVM host kernel ring buffer:
> > > > > > >
> > > > > > >    e1000e 0000:02:00.0: PCI INT A disabled
> > > > > > >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> > > > > > >    pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > > > > > >
> > > > > > > >  I think you can try on-
> > > > > > > > board e1000e for MSI-X first. And please ensure correlated driver
> > > > > > > > have been loaded correctly.
> > > > > > >
> > > > > > > <nod>..
> > > > > > >
> > > > > > > >  And what do you mean by "some MSI-X exceptions"? Better with
> > > > > > > > the log.
> > > > > > >
> > > > > > > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > > > > > > kvm_destroy_phys_mem() statements:
> > > > > > >
> > > > > > > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > > > > > > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid parameters
> > > > > > > (slot=-1)
> > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > > >
> > > > > > > However I still see the following in the KVM guest kernel ring buffer
> > > > > > > running v2.6.30-rc in the HVM guest.
> > > > > > >
> > > > > > > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > > > > > [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > > > > > (level, high) -> IRQ 10 [    5.525710] e1000e 0000:00:05.0: setting
> > > > > > > latency timer to 64
> > > > > > > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI-X
> > > > > > > interrupts.  Falling back to MSI interrupts. [    5.527200]
> > > > > > > 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI interrupts.
> > > > > > > Falling back to legacy interrupts. [    5.829988] 0000:00:05.0: eth0:
> > > > > > > (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2 [    5.830672]
> > > > > > > 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection [    5.831240]
> > > > > > > 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > > > > >
> > > > > > Hi Nicholas
> > > > > >
> > > > > > I think something need to be clarify:
> > > > > > 1. For SRIOV, you need 2.6.30 as host kernel... But it's better to know
> > > > > > if normal device assignment work in your environment at first.
> > > > > > 2. The Fedora's userspace is even more old... You'd better try qemu-kvm
> > > > > > upstream, which is more convenient for us to track the problem(and kvm-85
> > > > > > is also ok). And as you see above, your QEmu don't support MSI/MSIX...
> > > > >
> > > > > Ok, got it..
> > > > >
> > > > > > So you can:
> > > > > > 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
> > > > >
> > > > > Ok, I am now updated on in the FC 11 Host with kvm-85 kernel modules and
> > > > > am using the built qemu-system-x86_64 from the kvm-85 source package:
> > > > >
> > > > > loaded kvm module (kvm-85)
> > > > > QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008 Fabrice
> > > > > Bellard
> > > > >
> > > > > > 2. Your host kernel is Fedora 11 Preview, that should be fine with device
> > > > > > assignment at first(and let's solve it first, SRIOV the next step).
> > > > >
> > > > > Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM host for
> > > > > the momemt to get e1000e working.  But I will start building a
> > > > > v2.6.30-rc3 kernel again for my fc11 host kernel as I do need SR-IOV at
> > > > > some point... :-)
> > > > >
> > > > > > 3. Your KVM version seems like kvm-85, you may provide some dmesg on host
> > > > > > side(I think you didn't use the KVM come along with kernel).
> > > > >
> > > > > Ok, now within the KVM guest running v2.6.29.2, I see the following:
> > > > >
> > > > > [    2.669243] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
> > > > > [    2.672931] e1000e: Copyright (c) 1999-2008 Intel Corporation.
> > > > > [    2.674932] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > > > > [    2.675181] 8139too Fast Ethernet driver 0.9.28
> > > > > [    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > > > > (level, high) -> IRQ 10
> > > > > [    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
> > > > > [    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> > > > > [    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> > > > > [    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> > > > > [    2.698039] FDC 0 is a S82078B
> > > > > [    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > > > > 00:e0:81:c0:90:b2
> > > > > [    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
> > > > > [    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > > > >
> > > > > And the folllowing from /proc/interrupts inside of the KVM guest:
> > > > >
> > > > >  24:        117          0          0          0          0          0
> > > > >     0          0          0          0   PCI-MSI-edge      eth1-rx-0 25:
> > > > >       0          0          0          0          0          0          0
> > > > >        0          0          0   PCI-MSI-edge      eth1-tx-0 26:          2
> > > > >          0          0          0          0          0          0
> > > > > 0          0          0   PCI-MSI-edge      eth1
> > > > >
> > > > > ethtool eth1 reports that Link is detected, but I am still unable to get
> > > > > a dhcp to work.
> > > >
> > > > It's a little strange that I checked all the log you posted, but can't find
> > > > anything suspicious...(Except you got a MCE log in your dmesg, but I don't
> > > > think it would relate to this).
> > > >
> > > > You also already have interrupts in the guest for eth1-rx-0 and eth1, so at
> > > > least part of interrupts can be delivered to the guest.
> > > >
> > > > You can try to connect the port to another NIC port directly. Set fixed ip for
> > > > each, then ping each other.
> > > >
> > > > You can also try to disable MSI-X capability in QEmu. Just using "#if
> > > > 0/#endif" to wrap "#ifdef KVM_CAP_DEVICE_MSIX/#endif" in
> > > > hw/assigned_device_pci_cap_init(). Then the device would use MSI.
> > > >
> > > > If I am lucky enough to find a 82574L card by hand, I would give it a try...
> > > >
> > > > --
> > > > regards
> > > > Yang, Sheng
> > > >
> > > 
> > > Greetings Sheng,
> > > 
> > > So I updated my FC11 Host to kernel v2.6.30-rc3 (and enabled ext4 of
> > > course) and rebuilt the kvm-85 source kernel module and
> > > qemu-system-x86_64 and I am now able to get dhcp and IP ops from the
> > > 02:00.0 device on my IOH-5520 board with the KVM guest using a v2.6.29.2
> > > kernel!!  Everything is looking good with the v2.6.29.2, but after a
> > > quick reboot back into my v2.6.30-rc3 KVM guest kernel build e1000e it
> > > looks like I am unable to get dhcp.
> > > 
> > > Rebooting back into KVM Guest kernel v2.6.29.2 brings the pci-stub
> > > assigned e1000e 82574L assigned with dhcp and everything looks
> > > good!  :-)
> > > 
> > > I will keep poking at the v2.6.30-rc KVM guests (I am going to do a
> > > complete rebuild) and see if it does not start move IP packets as well..
> > > 
> > > Thanks for all of your help in getting setup!
> > > 
> > > --nab
> > > 
> > > 
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 

Re: KVM x86_64 with SR-IOV..? (device passthrough with LIO-Target v3.0)

[Nicholas A. Bellinger]

On Tue, 2009-05-05 at 04:28 -0700, Nicholas A. Bellinger wrote:
> On Tue, 2009-05-05 at 03:43 -0700, Nicholas A. Bellinger wrote:
> > On Tue, 2009-05-05 at 09:42 +0800, Yu Zhao wrote:
> > > Hi,
> > > 
> > > The VF also works in the host if the VF driver is programed properly.
> > > So it would be easier to develop the VF driver in the host and then
> > > verify the VF driver in the guest.
> > > 
> > > BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
> > > the CONFIG_PCI_IOV in the kernel .config?
> > > 
> > > Thanks,
> > > Yu
> > > 
> > 
> > Greetings Yu and Sheng,
> > 
> > So the original attachment was for the v2.6.29-fc11 host kernel output,
> > I ended up jumping to v2.6.30-rc3 (and making sure CONFIG_PCI_IOV was
> > enabled) for KVM host with kvm-85 and now things are looking quite
> > stable for me.
> > 
> > So far I have been able to successfully push LIO-Target v3.0 traffic
> > *inside* a v2.6.29.2 KVM guest via the onboard e1000e (02:00.0) port
> > from another Linux/iSCSI Initiator machine using a Intel 1 Gb/sec port.
> > I am running badblocks tests to iSCSI Logical Units for RAMDISK_DR and
> > FILEIO storage objects (in the KVM Guest), and they are passing
> > validation and I am seeing ~500 Mb/sec of throughput and very low CPU
> > usage in the KVM guests.
> > 
> 
> Ok I am seeing another issue with the e1000e port on 02:00.0..:
> 
> As i start to push multiple badblocks tests RAMDISK_DR iSCSI Logical
> units into KVM Guest running LIO v2.6.29.2 from the external Linux/iSCSI
> Initiator machine, after about 100 GB of iSCSI traffic, I see the
> following exception in KVM host v2.6.30-rc3:
> 
> DRHD: handling fault status reg 2
> DMAR:[DMA Write] Request device [02:00.0] fault addr 7fc958b010000 
> DMAR:[fault reason 04] Access beyond MGAW
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
> 
> I am able to restart the LIO-Target KVM Guest and the Linux/iSCSI
> Initiators are able to reconnect..  Wow, very cool..
> 
> Not sure if this is a bug in the target_core_mod RAMDISK_DR subsystem
> plugin (mapping struct iovec to internally allocated struct page) or
> what.  I will have to look at the DMAR code to understand what this
> exception means..
> 

Greetings Yu, Sheng and Co,

So I have been making progress this morning..  So far, I have hooked up
a LSI mpt-function PCIe SAS adapter into the KVM guest with a Sandisk
SATA SSD 32 GB drive.  It is using MSI interrupts (not MSI-X) and I am
able to push ~70 MB/sec from a 2nd Linux/iSCSI Initiator machine
(running Open-iSCSI) with the 1500 byte MTUs on e1000e ports from within
the KVM guest. 

The interesting thing is that I am having to use IBLOCK export (using
using submit_bio(), and complete emulation of SCSI control path) for
SATA SSD in order to get I/O running stable  Using the pSCSI export I am
getting immediate exceptions from scsi_execute_async() in the v2.6.29.2
KVM guest..  Using a 2nd SAS disk I am able to use target_core_mod/pSCSI
export and push badblocks and LTP disktest traffic however..

Here is a bit about the the setup looks, 

*) Linux/iSCSI Initiator node accessing KVM Guest LIO-Target v3.0
storage:

subjekt:~# lsscsi
[6:0:0:0]    disk    ATA      ST3250820AS      3.AA  /dev/sda
[10:0:0:0]   cd/dvd  PIONEER  DVD-ROM DVD-305  1.06  /dev/scd1
[18:0:0:0]   cd/dvd  TOSHIBA  DVD/HD  X807616  MC08  /dev/scd2
[32:0:0:0]   disk    LIO-ORG  RAMDISK-DR       3.0   /dev/sdb
[32:0:0:1]   disk    LIO-ORG  RAMDISK-DR       3.0   /dev/sdc
[32:0:0:2]   disk    LIO-ORG  FILEIO           3.0   /dev/sdd
[32:0:0:3]   disk    LIO-ORG  IBLOCK           3.0   /dev/sde

subjekt:~# sg_inq -i /dev/sde
VPD INQUIRY: Device Identification page
  Designation descriptor number 1, descriptor length: 20
    id_type: NAA,  code_set: Binary
    associated with the addressed logical unit
      NAA 6, IEEE Company_id: 0x1405
      Vendor Specific Identifier: 0xa97e4ce21
      Vendor Specific Identifier Extension: 0xc0711de829b000c2
      [0x6001405a97e4ce21c0711de829b000c2]
  Designation descriptor number 2, descriptor length: 52
    id_type: T10 vendor identification,  code_set: ASCII
    associated with the addressed logical unit
      vendor id: LIO-ORG
      vendor specific: IBLOCK:a97e4ce21c0711de829b000c2943d57b
  Designation descriptor number 3, descriptor length: 8
    transport: Internet SCSI (iSCSI)
    id_type: Relative target port,  code_set: Binary
    associated with the target port
      Relative target port: 0x1
  Designation descriptor number 4, descriptor length: 8
    transport: Internet SCSI (iSCSI)
    id_type: Target port group,  code_set: Binary
    associated with the target port
      Target port group: 0x0
  Designation descriptor number 5, descriptor length: 8
    id_type: Logical unit group,  code_set: Binary
    associated with the addressed logical unit
      Logical unit group: 0x0
  Designation descriptor number 6, descriptor length: 80
    transport: Internet SCSI (iSCSI)
    id_type: SCSI name string,  code_set: UTF-8
    associated with the target port
      SCSI name string:
iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6,t,0x0001

The SATA SSD is available as /dev/sde on the Linux/iSCSI Initiator node,
the 2nd SAS disk was removed from the dmesg below as having multiple
drives on the mpt-function card made it start to hiccup..

--nab

*) From with the LIO-Target KVM gest:

lenny64guest1:~# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 20)
00:04.0 RAM memory: Qumranet, Inc. Device 1002
00:05.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
00:06.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 02)


*) Dmesg from LIO-Target KVM guest:

[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.29.2 (root@lenny64guest1) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) #2 SMP Mon May 4 01:22:53 PDT 2009
[    0.000000] Command line: root=/dev/hda1 ro
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
[    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
[    0.000000]  BIOS-e820: 000000007fff0000 - 0000000080000000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
[    0.000000] DMI 2.4 present.
[    0.000000] last_pfn = 0x7fff0 max_arch_pfn = 0x100000000
[    0.000000] init_memory_mapping: 0000000000000000-000000007fff0000
[    0.000000]  0000000000 - 007fe00000 page 2M
[    0.000000]  007fe00000 - 007fff0000 page 4k
[    0.000000] kernel direct mapping tables up to 7fff0000 @ 8000-c000
[    0.000000] last_map_addr: 7fff0000 end: 7fff0000
[    0.000000] RAMDISK: 379c8000 - 37fef480
[    0.000000] ACPI: RSDP 000FBE50, 0014 (r0 QEMU  )
[    0.000000] ACPI: RSDT 7FFF0000, 002C (r1 QEMU   QEMURSDT        1 QEMU        1)
[    0.000000] ACPI: FACP 7FFF002C, 0074 (r1 QEMU   QEMUFACP        1 QEMU        1)
[    0.000000] FADT: X_PM1a_EVT_BLK.bit_width (16) does not match PM1_EVT_LEN (4)
[    0.000000] ACPI: DSDT 7FFF0100, 2540 (r1   BXPC   BXDSDT        1 INTL 20061109)
[    0.000000] ACPI: FACS 7FFF00C0, 0040
[    0.000000] ACPI: APIC 7FFF2640, 00E0 (r1 QEMU   QEMUAPIC        1 QEMU        1)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000007fff0000
[    0.000000] Bootmem setup node 0 0000000000000000-000000007fff0000
[    0.000000]   NODE_DATA [0000000000001000 - 0000000000005fff]
[    0.000000]   bootmap [000000000000a000 -  0000000000019fff] pages 10
[    0.000000] (6 early reservations) ==> bootmem [0000000000 - 007fff0000]
[    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
[    0.000000]   #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
[    0.000000]   #2 [0000200000 - 000070493c]    TEXT DATA BSS ==> [0000200000 - 000070493c]
[    0.000000]   #3 [00379c8000 - 0037fef480]          RAMDISK ==> [00379c8000 - 0037fef480]
[    0.000000]   #4 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
[    0.000000]   #5 [0000008000 - 000000a000]          PGTABLE ==> [0000008000 - 000000a000]
[    0.000000] found SMP MP-table at [ffff8800000fbd10] 000fbd10
[    0.000000]  [ffffe20000000000-ffffe20001bfffff] PMD -> [ffff880001200000-ffff880002dfffff] on node 0
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000000 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x00100000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000000 -> 0x0000009f
[    0.000000]     0: 0x00000100 -> 0x0007fff0
[    0.000000] On node 0 totalpages: 524175
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 1387 pages reserved
[    0.000000]   DMA zone: 2556 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 7112 pages used for memmap
[    0.000000]   DMA32 zone: 513064 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0xb008
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] disabled)
[    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 2, version 0, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.000000] ACPI: IRQ5 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] ACPI: IRQ10 used by override.
[    0.000000] ACPI: IRQ11 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] SMP: Allowing 16 CPUs, 14 hotplug CPUs
[    0.000000] nr_irqs_gsi: 24
[    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e8000
[    0.000000] PM: Registered nosave memory: 00000000000e8000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 88000000 (gap: 80000000:7ffbc000)
[    0.000000] NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:16 nr_node_ids:1
[    0.000000] PERCPU: Allocating 49152 bytes of per cpu data
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 515620
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: root=/dev/hda1 ro
[    0.000000] Initializing CPU#0
[    0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes)
[    0.000000] Fast TSC calibration using PIT
[    0.000000] Detected 2266.815 MHz processor.
[    0.004000] Console: colour VGA+ 80x25
[    0.004000] console [tty0] enabled
[    0.004000] Checking aperture...
[    0.004000] No AGP bridge found
[    0.004000] Calgary: detecting Calgary via BIOS EBDA area
[    0.004000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[    0.004000] Memory: 2055644k/2097088k available (2463k kernel code, 388k absent, 41056k reserved, 1259k data, 464k init)
[    0.004013] Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.63 BogoMIPS (lpj=9067260)
[    0.004985] Security Framework initialized
[    0.005422] SELinux:  Disabled at boot.
[    0.005995] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.007887] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.008603] Mount-cache hash table entries: 256
[    0.009309] Initializing cgroup subsys ns
[    0.009739] Initializing cgroup subsys cpuacct
[    0.010172] Initializing cgroup subsys devices
[    0.010655] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.011229] CPU: L2 cache: 2048K
[    0.011633] CPU 0/0x0 -> Node 0
[    0.012061] ACPI: Core revision 20081204
[    0.013529] Setting APIC routing to flat
[    0.015117] ..TIMER: vector=0x30 apic1=0 pin1=0 apic2=-1 pin2=-1
[    0.055398] CPU0: Intel QEMU Virtual CPU version 0.10.0 stepping 03
[    0.273970] Booting processor 1 APIC 0x1 ip 0x6000
[    0.004000] Initializing CPU#1
[    0.004000] Calibrating delay using timer specific routine.. 4533.28 BogoMIPS (lpj=9066565)
[    0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.004000] CPU: L2 cache: 2048K
[    0.004000] CPU 1/0x1 -> Node 0
[    0.364147] CPU1: Intel QEMU Virtual CPU version 0.10.0 stepping 03
[    0.367429] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
[    0.372036] Brought up 2 CPUs
[    0.372442] Total of 2 processors activated (9066.91 BogoMIPS).
[    0.373176] CPU0 attaching sched-domain:
[    0.373181]  domain 0: span 0-1 level CPU
[    0.373183]   groups: 0 1
[    0.373186] CPU1 attaching sched-domain:
[    0.373187]  domain 0: span 0-1 level CPU
[    0.373188]   groups: 1 0
[    0.373255] net_namespace: 1888 bytes
[    0.373255] NET: Registered protocol family 16
[    0.373269] ACPI: bus type pci registered
[    0.376082] PCI: Using configuration type 1 for base access
[    0.380281] bio: create slab <bio-0> at 0
[    0.381293] ACPI: EC: Look up EC in DSDT
[    0.383292] ACPI: Interpreter enabled
[    0.383693] ACPI: (supports S0 S3 S4 S5)
[    0.384763] ACPI: Using IOAPIC for interrupt routing
[    0.388691] ACPI: No dock devices found.
[    0.388729] ACPI: PCI Root Bridge [PCI0] (0000:00)
[    0.393074] pci 0000:00:01.1: reg 20 io port: [0xc000-0xc00f]
[    0.393406] pci 0000:00:01.3: quirk: region b000-b03f claimed by PIIX4 ACPI
[    0.393920] pci 0000:00:01.3: quirk: region b100-b10f claimed by PIIX4 SMB
[    0.395842] pci 0000:00:02.0: reg 10 32bit mmio: [0xf0000000-0xf1ffffff]
[    0.396508] pci 0000:00:02.0: reg 14 32bit mmio: [0xf2000000-0xf2000fff]
[    0.401611] pci 0000:00:03.0: reg 10 io port: [0xc100-0xc1ff]
[    0.401674] pci 0000:00:03.0: reg 14 32bit mmio: [0xf2001000-0xf20010ff]
[    0.401903] pci 0000:00:04.0: reg 10 io port: [0xc200-0xc21f]
[    0.403518] pci 0000:00:05.0: reg 10 32bit mmio: [0xf2020000-0xf203ffff]
[    0.404059] pci 0000:00:05.0: reg 18 io port: [0xc220-0xc23f]
[    0.404368] pci 0000:00:05.0: reg 1c 32bit mmio: [0xf2040000-0xf2043fff]
[    0.405176] pci 0000:00:06.0: reg 10 io port: [0xc300-0xc3ff]
[    0.405474] pci 0000:00:06.0: reg 14 32bit mmio: [0xf2044000-0xf2047fff]
[    0.405804] pci 0000:00:06.0: reg 1c 32bit mmio: [0xf2050000-0xf205ffff]
[    0.405948] pci 0000:00:06.0: reg 30 32bit mmio: [0x200000-0x3fffff]
[    0.406577] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    0.408281] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[    0.412098] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[    0.413393] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[    0.414571] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[    0.416142] usbcore: registered new interface driver usbfs
[    0.416504] usbcore: registered new interface driver hub
[    0.416504] usbcore: registered new device driver usb
[    0.416508] PCI: Using ACPI for IRQ routing
[    0.452044] pnp: PnP ACPI init
[    0.452515] ACPI: bus type pnp registered
[    0.454439] pnp: PnP ACPI: found 7 devices
[    0.454918] ACPI: ACPI bus type pnp unregistered
[    0.460804] pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
[    0.460807] pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
[    0.460969] NET: Registered protocol family 2
[    0.508246] IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.537526] TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
[    0.542048] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.543577] TCP: Hash tables configured (established 262144 bind 65536)
[    0.544332] TCP reno registered
[    0.556665] NET: Registered protocol family 1
[    0.557476] checking if image is initramfs... it is
[    0.980042] Switched to high resolution mode on CPU 1
[    0.983993] Switched to high resolution mode on CPU 0
[    1.028987] Freeing initrd memory: 6301k freed
[    1.032241] audit: initializing netlink socket (disabled)
[    1.032745] type=2000 audit(1241543237.032:1): initialized
[    1.035249] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    1.036055] VFS: Disk quotas dquot_6.5.2
[    1.036182] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.036182] msgmni has been set to 4027
[    1.036182] alg: No test for stdrng (krng)
[    1.036182] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    1.036182] io scheduler noop registered
[    1.036182] io scheduler anticipatory registered
[    1.036182] io scheduler deadline registered
[    1.040223] io scheduler cfq registered (default)
[    1.040692] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    1.041194] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    1.041681] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    1.042183] pci 0000:00:02.0: Boot video device
[    1.048876] Linux agpgart interface v0.103
[    1.049315] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    1.050082] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    1.051351] 00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    1.054962] brd: module loaded
[    1.055662] PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    1.111469] serio: i8042 KBD port at 0x60,0x64 irq 1
[    1.112044] serio: i8042 AUX port at 0x60,0x64 irq 12
[    1.112901] mice: PS/2 mouse device common for all mice
[    1.114216] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
[    1.114828] rtc0: alarms up to one day, 114 bytes nvram
[    1.115363] input: AT Translated Set 2 keyboard as /class/input/input0
[    1.115934] cpuidle: using governor ladder
[    1.116448] cpuidle: using governor menu
[    1.116926] No iBFT detected.
[    1.117634] TCP cubic registered
[    1.118088] NET: Registered protocol family 17
[    1.118873] registered taskstats version 1
[    1.119543] rtc_cmos 00:01: setting system clock to 2009-05-05 17:07:17 UTC (1241543237)
[    1.120418] Freeing unused kernel memory: 464k freed
[    1.193327] processor ACPI_CPU:00: registered as cooling_device0
[    1.194273] processor ACPI_CPU:01: registered as cooling_device1
[    1.392735] Uniform Multi-Platform E-IDE driver
[    1.413020] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
[    1.413531] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11
[    1.415182] 8139too Fast Ethernet driver 0.9.28
[    1.415673] 8139too 0000:00:03.0: This (id 10ec:8139 rev 20) is an enhanced 8139C+ chip, use 8139cp
[    1.418181] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
[    1.418680] e1000e: Copyright (c) 1999-2008 Intel Corporation.
[    1.419758] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
[    1.420328] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
[    1.421130] e1000e 0000:00:05.0: setting latency timer to 64
[    1.421963] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
[    1.421999] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
[    1.422033] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
[    1.431733] SCSI subsystem initialized
[    1.445624] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
[    1.453420] Fusion MPT base driver 3.04.07
[    1.453849] Copyright (c) 1999-2008 LSI Corporation
[    1.461332] Fusion MPT SAS Host driver 3.04.07
[    1.464878] FDC 0 is a S82078B
[    1.549697] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1) 00:e0:81:c0:90:b2
[    1.550752] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network Connection
[    1.551549] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
[    1.553336] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
[    1.553940] udev: renamed network interface eth0 to eth1
[    1.554683] 8139cp 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11
[    1.557146] eth0: RTL-8139C+ at 0xffffc2000096a000, 52:54:00:12:34:56, IRQ 11
[    1.560485] 8139cp 0000:00:03.0: setting latency timer to 64
[    1.560904] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
[    1.561607] mptsas 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 10 (level, high) -> IRQ 10
[    1.563379] mptbase: ioc0: Initiating bringup
[    1.634659] mptbase: ioc0: WARNING - Unexpected doorbell active!
[    5.343913] ioc0: LSISAS1068E B1: Capabilities={Initiator}
[    5.345428] mptsas 0000:00:06.0: setting latency timer to 64
[   22.874565] scsi0 : ioc0: LSISAS1068E B1, FwRev=010f0000h, Ports=1, MaxQ=511, IRQ=10
[   22.903281] scsi 0:0:0:0: Direct-Access     ATA      SanDisk SSD SATA 1.20 PQ: 0 ANSI: 5
[   22.906530] piix 0000:00:01.1: IDE controller (0x8086:0x7010 rev 0x00)
[   22.908623] piix 0000:00:01.1: not 100% native mode: will probe irqs later
[   22.909390] PIIX_IDE 0000:00:01.1: setting latency timer to 64
[   22.909410]     ide0: BM-DMA at 0xc000-0xc007
[   22.910045]     ide1: BM-DMA at 0xc008-0xc00f
[   22.910666] Probing IDE interface ide0...
[   22.933264] Driver 'sd' needs updating - please use bus_type methods
[   22.936186] sd 0:0:0:0: [sda] 62533296 512-byte hardware sectors: (32.0 GB/29.8 GiB)
[   22.938165] sd 0:0:0:0: [sda] Write Protect is off
[   22.938808] sd 0:0:0:0: [sda] Mode Sense: 73 00 00 08
[   22.940775] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   22.942194] sd 0:0:0:0: [sda] 62533296 512-byte hardware sectors: (32.0 GB/29.8 GiB)
[   22.944225] sd 0:0:0:0: [sda] Write Protect is off
[   22.944879] sd 0:0:0:0: [sda] Mode Sense: 73 00 00 08
[   22.946734] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   22.947886]  sda: unknown partition table
[   22.951811] sd 0:0:0:0: [sda] Attached SCSI disk
[   23.211126] hda: QEMU HARDDISK, ATA DISK drive
[   23.883187] hda: host max PIO4 wanted PIO255(auto-tune) selected PIO2
[   23.883289] hda: MWDMA2 mode selected
[   23.884023] Probing IDE interface ide1...
[   24.283042] hdc: QEMU DVD-ROM, ATAPI CD/DVD-ROM drive
[   24.955145] hdc: host max PIO4 wanted PIO255(auto-tune) selected PIO0
[   24.955199] hdc: MWDMA2 mode selected
[   24.955978] ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
[   24.956812] isa bounce pool size: 16 pages
[   24.957471] ide1 at 0x170-0x177,0x376 on irq 15
[   24.974924] libata version 3.00 loaded.
[   24.992633] ide-gd driver 1.18
[   24.993261] hda: max request size: 512KiB
[   24.993874] hda: 16384000 sectors (8388 MB) w/256KiB Cache, CHS=16253/255/63
[   25.000639] ide-cd driver 5.00
[   25.007111] hda: cache flushes supported
[   25.007753]  hda: hda1 hda2 < hda5 >
[   25.010488] ide-cd: hdc: ATAPI 4X CD-ROM drive, 512kB Cache
[   25.011858] Uniform CD-ROM driver Revision: 3.20
[   25.110990] PM: Starting manual resume from disk
[   25.129340] EXT3-fs: INFO: recovery required on readonly filesystem.
[   25.130052] EXT3-fs: write access will be enabled during recovery.
[   26.128769] kjournald starting.  Commit interval 5 seconds
[   26.129481] EXT3-fs: recovery complete.
[   26.130963] EXT3-fs: mounted filesystem with ordered data mode.
[   26.335166] udevd version 125 started
[   26.631975] input: Power Button (FF) as /class/input/input1
[   26.666993] ACPI: Power Button (FF) [PWRF]
[   26.681253] parport_pc 00:05: reported by Plug and Play ACPI
[   26.681824] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
[   26.723397] input: PC Speaker as /class/input/input2
[   26.724903] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
[   26.949233] input: ImExPS/2 Generic Explorer Mouse as /class/input/input3
[   58.566336] Adding 393552k swap on /dev/hda5.  Priority:-1 extents:1 across:393552k 
[   58.594239] EXT3 FS on hda1, internal journal
[   59.613114] loop: module loaded
[   60.681886] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
[   63.746408] NET: Registered protocol family 10
[   63.748013] lo: Disabled Privacy Extensions
[   73.968880] eth0: no IPv6 routers present
[  233.617434] ADDRCONF(NETDEV_UP): eth1_rename: link is not ready
[  236.587594] e1000e: eth1_rename NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[  236.595715] ADDRCONF(NETDEV_CHANGE): eth1_rename: link becomes ready
[  243.146435] TARGET_CORE[0]: Loading Generic Kernel Storage Engine: v3.0.0 on Linux/x86_64 on 2.6.29.2
[  243.147955] TARGET_CORE[0]: Initialized ConfigFS Fabric Infrastructure: v1.0.0 on Linux/x86_64 on 2.6.29.2
[  243.149107] SE_PC[0] - Registered Plugin Class: TRANSPORT
[  243.149830] PLUGIN_TRANSPORT[1] - pscsi registered
[  243.150492] PLUGIN_TRANSPORT[2] - stgt registered
[  243.151183] CORE_STGT[0]: Bus Initalization complete
[  243.151838] PLUGIN_TRANSPORT[4] - iblock registered
[  243.152483] PLUGIN_TRANSPORT[5] - rd_dr registered
[  243.153126] PLUGIN_TRANSPORT[6] - rd_mcp registered
[  243.155097] PLUGIN_TRANSPORT[7] - fileio registered
[  243.155743] SE_PC[1] - Registered Plugin Class: OBJ
[  243.156372] PLUGIN_OBJ[1] - dev registered
[  243.160265] Target_Core_ConfigFS: Located se_plugin: ffff88007c4318e0 plugin_name: iblock hba_type: 4 plugin_dep_id: 0
[  243.161454] CORE_HBA[0] - Linux-iSCSI.org iBlock HBA Driver 3.0 on Generic Target Core Stack v3.0
[  243.162630] CORE_HBA[0] - Attached iBlock HBA: 0 to Generic Target Core TCQ Depth: 512
[  243.163681] CORE_HBA[0] - Attached HBA to Generic Target Core
[  243.164496] IBLOCK: Allocated ib_dev for ssd_test0
[  243.165152] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007c022000 se_dev_su_ptr: ffff88007c444c00
[  243.166753] Target_Core_ConfigFS: Set emulated VPD Unit Serial: a97e4ce21c0711de829b000c2943d57b
[  243.167947] IBLOCK: Referencing Major: 8
[  243.168541] IBLOCK: Referencing Minor: 0
[  243.169225] IBLOCK: Claiming ffff88007c444c00 Major:Minor - 8:0
[  243.170021] bio: create slab <bio-1> at 1
[  243.170656] IBLOCK: Created bio_set() for major/minor: 8:0
[  243.171325] iblock: Using SPC3_PERSISTENT_RESERVATIONS emulation
[  243.172012] iblock: Enabling ALUA Emulation for SPC-3 device
[  243.172696] iblock: Adding to default ALUA LU Group: core/alua/lu_gps/default_lu_gp
[  243.173802] CORE_iBLOCK[0] - Activating Device with TCQ: 0 at Major: 8 Minor 0
[  243.175345]   Vendor: LIO-ORG   Model: IBLOCK            Revision: 3.0 
[  243.181019]   Type:   Direct-Access                      ANSI SCSI revision: 05
[  243.182592] T10 VPD Unit Serial Number: a97e4ce21c0711de829b000c2943d57b
[  243.183397] T10 VPD Page Length: 72
[  243.183983] T10 VPD Identifer Length: 16
[  243.184588] T10 VPD Identifier Association: addressed logical unit
[  243.185286] T10 VPD Identifier Type: NAA
[  243.186965] T10 VPD Binary Device Identifier: 36001405a97e4ce21c0711de829b000c2
[  243.187993] T10 VPD Identifer Length: 48
[  243.188602] T10 VPD Identifier Association: addressed logical unit
[  243.189303] T10 VPD Identifier Type: T10 Vendor ID based
[  243.190037] T10 VPD ASCII Device Identifier: LIO-ORG
[  243.191029] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff88007c5d1400
[  243.193246] Target_Core_ConfigFS: Located se_plugin: ffff88007c431838 plugin_name: pscsi hba_type: 1 plugin_dep_id: 0
[  243.194482] CORE_HBA[1] - Linux-iSCSI.org SCSI HBA Driver v3.0 on Generic Target Core Stack v3.0
[  243.195580] CORE_HBA[1] - MPT SPI Host
[  243.196170] CORE_HBA[1] - Attached SCSI HBA to Generic Target Core with TCQ Depth: 127 MaxSectors: 8192
[  243.197292] CORE_HBA[1] - Attached HBA to Generic Target Core
[  243.198125] PSCSI: Allocated pdv: ffff88007c42aa00 for sda
[  243.198803] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007c582000 se_dev_su_ptr: ffff88007c42aa00
[  243.201132] PSCSI: Allocated pdv: ffff88007d064140 for sdb
[  243.201952] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007c1d9800 se_dev_su_ptr: ffff88007d064140
[  243.203550] PSCSI[0]: Referencing SCSI Channel ID: 0
[  243.204200] PSCSI[0]: Referencing SCSI Target ID: 1
[  243.204841] PSCSI[0]: Referencing SCSI LUN ID: 0
[  243.205550] Unable to locate 0:0:1:0
[  243.208465] Target_Core_ConfigFS: Located se_plugin: ffff88007c431918 plugin_name: rd_dr hba_type: 5 plugin_dep_id: 0
[  243.209674] CORE_HBA[2] - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
[  243.210802] CORE_HBA[2] - Attached Ramdisk HBA: 0 to Generic Target Core TCQ Depth: 256 MaxSectors: 1024
[  243.211942] CORE_HBA[2] - Attached HBA to Generic Target Core
[  243.212774] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007c59c800 se_dev_su_ptr: ffff88007c4374c0
[  243.214315] RAMDISK: Referencing Page Count: 32768
[  243.224608] CORE_RD[0] - Built Ramdisk Device ID: 0 space of 32768 pages in 17 tables
[  243.225795] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS emulation
[  243.226490] rd_dr: Enabling ALUA Emulation for SPC-3 device
[  243.227165] rd_dr: Adding to default ALUA LU Group: core/alua/lu_gps/default_lu_gp
[  243.228212] CORE_RD[0] - Activating Device with TCQ: 0 at Ramdisk Device ID: 0
[  243.229792]   Vendor: LIO-ORG   Model: RAMDISK-DR        Revision: 3.0 
[  243.235413]   Type:   Direct-Access                      ANSI SCSI revision: 05
[  243.236861] T10 VPD Unit Serial Number: 1234567890:2_0
[  243.237599] T10 VPD Page Length: 38
[  243.238220] T10 VPD Identifer Length: 34
[  243.238860] T10 VPD Identifier Association: addressed logical unit
[  243.239540] T10 VPD Identifier Type: T10 Vendor ID based
[  243.240190] T10 VPD ASCII Device Identifier: LIO-ORG
[  243.241083] CORE_RD[0] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768 pages in 17 tables, 134217728 total bytes
[  243.242336] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff88007c5d1000
[  243.244650] Target_Core_ConfigFS: Located se_plugin: ffff88007c431918 plugin_name: rd_dr hba_type: 5 plugin_dep_id: 1
[  243.245895] CORE_HBA[3] - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
[  243.247006] CORE_HBA[3] - Attached Ramdisk HBA: 1 to Generic Target Core TCQ Depth: 256 MaxSectors: 1024
[  243.248131] CORE_HBA[3] - Attached HBA to Generic Target Core
[  243.248838] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007c18c000 se_dev_su_ptr: ffff88007c973940
[  243.252447] RAMDISK: Referencing Page Count: 32768
[  243.262801] CORE_RD[1] - Built Ramdisk Device ID: 0 space of 32768 pages in 17 tables
[  243.263870] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS emulation
[  243.264555] rd_dr: Enabling ALUA Emulation for SPC-3 device
[  243.265231] rd_dr: Adding to default ALUA LU Group: core/alua/lu_gps/default_lu_gp
[  243.266342] CORE_RD[1] - Activating Device with TCQ: 0 at Ramdisk Device ID: 0
[  243.267625]   Vendor: LIO-ORG   Model: RAMDISK-DR        Revision: 3.0 
[  243.273342]   Type:   Direct-Access                      ANSI SCSI revision: 05
[  243.274807] T10 VPD Unit Serial Number: 1234567890:3_0
[  243.275527] T10 VPD Page Length: 38
[  243.276125] T10 VPD Identifer Length: 34
[  243.276734] T10 VPD Identifier Association: addressed logical unit
[  243.277439] T10 VPD Identifier Type: T10 Vendor ID based
[  243.278173] T10 VPD ASCII Device Identifier: LIO-ORG
[  243.278891] CORE_RD[1] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768 pages in 17 tables, 134217728 total bytes
[  243.280067] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff880037943400
[  243.282480] Target_Core_ConfigFS: Located se_plugin: ffff88007c431988 plugin_name: fileio hba_type: 7 plugin_dep_id: 0
[  243.320107] CORE_HBA[4] - Linux-iSCSI.org FILEIO HBA Driver 3.0 on Generic Target Core Stack v3.0
[  243.321211] CORE_HBA[4] - Attached FILEIO HBA: 0 to Generic Target Core with TCQ Depth: 256 MaxSectors: 1024
[  243.322383] CORE_HBA[4] - Attached HBA to Generic Target Core
[  243.323284] FILEIO: Allocated fd_dev for ffff88007bd61ee0
[  243.323964] Target_Core_ConfigFS: Allocated se_subsystem_dev_t: ffff88007d1b6000 se_dev_su_ptr: ffff88007c444000
[  243.325456] Target_Core_ConfigFS: Set emulated VPD Unit Serial: 66e418b4e9ab11dd8f51000c2943d57b
[  243.340857] FILEIO: Referencing Path: /tmp/fileio
[  243.341504] FILEIO: Referencing Size: 134217728 bytes
[  243.342298] fileio: Using SPC3_PERSISTENT_RESERVATIONS emulation
[  243.342998] fileio: Enabling ALUA Emulation for SPC-3 device
[  243.343668] fileio: Adding to default ALUA LU Group: core/alua/lu_gps/default_lu_gp
[  243.344706] CORE_FILE[0] - Activating Device with TCQ: 0 at FILEIO Device ID: 0
[  243.346260]   Vendor: LIO-ORG   Model: FILEIO            Revision: 3.0 
[  243.351966]   Type:   Direct-Access                      ANSI SCSI revision: 05
[  243.353392] T10 VPD Unit Serial Number: 66e418b4e9ab11dd8f51000c2943d57b
[  243.354363] T10 VPD Page Length: 72
[  243.354961] T10 VPD Identifer Length: 16
[  243.355569] T10 VPD Identifier Association: addressed logical unit
[  243.356304] T10 VPD Identifier Type: NAA
[  243.356904] T10 VPD Binary Device Identifier: 3600140566e418b4e9ab11dd8f51000c2
[  243.358048] T10 VPD Identifer Length: 48
[  243.358649] T10 VPD Identifier Association: addressed logical unit
[  243.359351] T10 VPD Identifier Type: T10 Vendor ID based
[  243.360007] T10 VPD ASCII Device Identifier: LIO-ORG
[  243.360730] CORE_FILE[0] - Added LIO FILEIO Device ID: 0 at /tmp/fileio, 134217728 total bytes
[  243.361913] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff880037943c00
[  243.364168] Target_Core_ConfigFS: REGISTER -> group: ffffffffa02a0180 name: iscsi
[  243.446323] Linux-iSCSI.org iSCSI Target Core Stack v3.0.0 on Linux/x86_64 on 2.6.29.2
[  243.447405] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
[  243.448132] Initialized struct target_fabric_configfs: ffff88006b554000 for iscsi
[  243.449150] <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
[  243.449919] LIO_TARGET[0] - Set fabric -> lio_target_fabric_configfs
[  243.451519] iscsi_allocate_thread_sets:195: ***OPS*** Spawned 4 thread set(s) (8 total threads).
[  243.452909] TARGET_CORE[iSCSI]: Allocated Discovery se_portal_group_t for endpoint: None, Portal Tag: 0
[  243.454075] CORE[0] - Allocated Discovery TPG
[  243.454698] Loading Complete.
[  243.455569] Target_Core_ConfigFS: REGISTER -> Located fabric: iscsi
[  243.456279] Target_Core_ConfigFS: REGISTER -> ffffffffa02ddbb0
[  243.456959] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: iscsi
[  243.457695] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for iscsi
[  243.458610] lio_target_call_coreaddtiqn(): name: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  243.459839] CORE[0] - Added iSCSI Target IQN: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  243.461031] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  243.462213] LIO_Target_ConfigFS: REGISTER -> Allocated Node: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  243.463621] lio_target_tiqn_addtpg() called: name tpgt_1
[  243.464283] lio_target_tiqn_addtpg() parent name: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  243.465439] lio_target_tiqn_addtpg() Using TPGT: 1
[  243.466206] TARGET_CORE[iSCSI]: Allocated Normal se_portal_group_t for endpoint: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6, Portal Tag: 1
[  243.467824] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[1] - Added iSCSI Target Portal Group
[  243.469018] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  243.536626] LIO_Target_ConfigFS: REGISTER -> Allocated TPG: tpgt_1
[  243.537704] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 PORTAL: 192.168.0.117:3260
[  243.539129] CORE[0] - Added Network Portal: 192.168.0.117:3260 on TCP on network device: None
[  243.540238] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] - Added Network Portal: 192.168.0.117:3260,1 on TCP on network device: None
[  243.541925] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[1] - Incremented np_exports to 1
[  243.543104] LIO_Target_ConfigFS: addnptotpg done!
[  243.649897] SCTP: Hash tables configured (established 65536 bind 65536)
[  243.704216] CORE[0] - Added Network Portal: 192.168.0.117:3260 on SCTP on network device: None
[  243.733493] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] - Added Network Portal: 192.168.0.117:3260,1 on SCTP on network device: None
[  243.735142] CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[1] - Incremented np_exports to 1
[  243.737456] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 LUN: 0
[  243.739831] rd_dr/iSCSI: Adding to default ALUA Target Port Group: core/alua/tg_pt_gps/default_tg_pt_gp
[  243.740953] iSCSI_TPG[1]_LUN[0] - Activated iSCSI Logical Unit from CORE HBA: 2
[  243.742029] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_0
[  243.744038] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 LUN: 1
[  243.746304] rd_dr/iSCSI: Adding to default ALUA Target Port Group: core/alua/tg_pt_gps/default_tg_pt_gp
[  243.747436] iSCSI_TPG[1]_LUN[1] - Activated iSCSI Logical Unit from CORE HBA: 3
[  243.748468] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_1
[  243.750509] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 LUN: 2
[  243.752754] fileio/iSCSI: Adding to default ALUA Target Port Group: core/alua/tg_pt_gps/default_tg_pt_gp
[  243.753924] iSCSI_TPG[1]_LUN[2] - Activated iSCSI Logical Unit from CORE HBA: 4
[  243.754957] LIO_Target_ConfigFS: Created Port Symlink fileio -> lun_2
[  243.757089] LIO_Target_ConfigFS: REGISTER -> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1 LUN: 3
[  243.759370] iblock/iSCSI: Adding to default ALUA Target Port Group: core/alua/tg_pt_gps/default_tg_pt_gp
[  243.760507] iSCSI_TPG[1]_LUN[3] - Activated iSCSI Logical Unit from CORE HBA: 0
[  243.761525] LIO_Target_ConfigFS: Created Port Symlink ssd_test0 -> lun_3
[  243.762761] Disabling iSCSI Authentication Methods for TPG: 1.
[  243.763566] iSCSI_TPG[1] - Generate Initiator Portal Group ACLs: Enabled
[  243.764414] iSCSI_TPG[1] - Demo Mode Write Protect bit: OFF
[  243.765196] iSCSI_TPG[1] - Set Default CmdSN TCQ Depth to 4
[  243.832368] iSCSI_TPG[1] - Enabled iSCSI Target Portal Group
[  244.173472] Received iSCSI login request from 192.168.0.116 on TCP Network Portal 192.168.0.117:3260
[  244.174664] Located Storage Object: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  244.175776] Located Portal Group Object: 1
[  244.176405] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
[  244.177423] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
[  244.178656] TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in Demo Mode
[  244.179756] TARGET_CORE[iSCSI]->TPG[1]_LUN[3] - Adding READ-WRITE access for LUN in Demo Mode
[  244.180854] iSCSI_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node: iqn.1993-08.org.debian:01.2fed72c26a4c
[  244.182213] ------------------------------------------------------------------
[  244.183240] HeaderDigest:                 None
[  244.183860] DataDigest:                   None
[  244.184479] MaxRecvDataSegmentLength:     131072
[  244.185099] IFMarker:                     No
[  244.185779] OFMarker:                     No
[  244.186394] ------------------------------------------------------------------
[  244.187422] ------------------------------------------------------------------
[  244.188435] MaxConnections:               1
[  244.189056] TargetName:                   iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  244.191890] InitiatorName:                iqn.1993-08.org.debian:01.2fed72c26a4c
[  244.192934] TargetAlias:                  LIO Target
[  244.193622] InitiatorAlias:               subjekt
[  244.194272] TargetPortalGroupTag:         1
[  244.194878] InitialR2T:                   Yes
[  244.195485] ImmediateData:                Yes
[  244.196107] MaxBurstLength:               262144
[  244.196741] FirstBurstLength:             65536
[  244.197356] DefaultTime2Wait:             2
[  244.198038] DefaultTime2Retain:           0
[  244.198648] MaxOutstandingR2T:            1
[  244.199266] DataPDUInOrder:               Yes
[  244.199883] DataSequenceInOrder:          Yes
[  244.200499] ErrorRecoveryLevel:           0
[  244.201115] SessionType:                  Normal
[  244.201812] ------------------------------------------------------------------
[  244.202838] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr: ffff88006b597e00
[  244.203564] iSCSI Login successful on CID: 0 from 192.168.0.116 to 192.168.0.117:3260,1
[  244.204621] Incremented iSCSI Connection count to 1 from node: iqn.1993-08.org.debian:01.2fed72c26a4c
[  244.205800] Established iSCSI session from node: iqn.1993-08.org.debian:01.2fed72c26a4c
[  244.238865] Incremented number of active iSCSI sessions to 1 on iSCSI Target Portal Group: 1
[  247.041588] eth1_rename: no IPv6 routers present
[  248.739907] Received iSCSI login request from 192.168.0.111 on TCP Network Portal 192.168.0.117:3260
[  248.742799] Located Storage Object: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  248.743981] Located Portal Group Object: 1
[  248.744635] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
[  248.745818] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] - Adding READ-WRITE access for LUN in Demo Mode
[  248.746927] TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in Demo Mode
[  248.748017] TARGET_CORE[iSCSI]->TPG[1]_LUN[3] - Adding READ-WRITE access for LUN in Demo Mode
[  248.749122] iSCSI_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node: iqn.1993-08.org.debian:01:2dadf92d0ef
[  248.750565] ------------------------------------------------------------------
[  248.751597] HeaderDigest:                 None
[  248.752230] DataDigest:                   None
[  248.752861] MaxRecvDataSegmentLength:     131072
[  248.753543] IFMarker:                     No
[  248.754171] OFMarker:                     No
[  248.754785] ------------------------------------------------------------------
[  248.755822] ------------------------------------------------------------------
[  248.756853] MaxConnections:               1
[  248.757510] TargetName:                   iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6
[  248.758690] InitiatorName:                iqn.1993-08.org.debian:01:2dadf92d0ef
[  248.759742] TargetAlias:                  LIO Target
[  248.760397] InitiatorAlias:               ubuntu
[  248.761034] TargetPortalGroupTag:         1
[  248.761731] InitialR2T:                   Yes
[  248.762352] ImmediateData:                Yes
[  248.762972] MaxBurstLength:               262144
[  248.763604] FirstBurstLength:             65536
[  248.764233] DefaultTime2Wait:             2
[  248.764844] DefaultTime2Retain:           0
[  248.765453] MaxOutstandingR2T:            1
[  248.767078] DataPDUInOrder:               Yes
[  248.767698] DataSequenceInOrder:          Yes
[  248.768319] ErrorRecoveryLevel:           0
[  248.768926] SessionType:                  Normal
[  248.769681] ------------------------------------------------------------------
[  248.770703] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr: ffff88006b597000
[  248.771412] iSCSI Login successful on CID: 0 from 192.168.0.111 to 192.168.0.117:3260,1
[  248.772470] Incremented iSCSI Connection count to 1 from node: iqn.1993-08.org.debian:01:2dadf92d0ef
[  248.773644] Established iSCSI session from node: iqn.1993-08.org.debian:01:2dadf92d0ef
[  248.774694] Incremented number of active iSCSI sessions to 2 on iSCSI Target Portal Group: 1

Re: KVM x86_64 with SR-IOV..? (device passthrough with LIO-Target v3.0)

[Sheng Yang]

On Tuesday 05 May 2009 18:43:46 Nicholas A. Bellinger wrote:
> On Tue, 2009-05-05 at 09:42 +0800, Yu Zhao wrote:
> > Hi,
> >
> > The VF also works in the host if the VF driver is programed properly.
> > So it would be easier to develop the VF driver in the host and then
> > verify the VF driver in the guest.
> >
> > BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
> > the CONFIG_PCI_IOV in the kernel .config?
> >
> > Thanks,
> > Yu
>
> Greetings Yu and Sheng,
>
> So the original attachment was for the v2.6.29-fc11 host kernel output,
> I ended up jumping to v2.6.30-rc3 (and making sure CONFIG_PCI_IOV was
> enabled) for KVM host with kvm-85 and now things are looking quite
> stable for me.
>
> So far I have been able to successfully push LIO-Target v3.0 traffic
> *inside* a v2.6.29.2 KVM guest via the onboard e1000e (02:00.0) port
> from another Linux/iSCSI Initiator machine using a Intel 1 Gb/sec port.
> I am running badblocks tests to iSCSI Logical Units for RAMDISK_DR and
> FILEIO storage objects (in the KVM Guest), and they are passing
> validation and I am seeing ~500 Mb/sec of throughput and very low CPU
> usage in the KVM guests.
>
> One issue I did notice while using the pci-stub method of
> device-assignment with same e1000 port (02:00.0) was while using an
> iSCSI Initiator (Open-iSCSI) on the KVM Host machine and doing sustained
> traffic into the LIO-Target KVM Guest on the same local KVM host to max
> out traffic between the other onboard e1000e port (03.00.0), I see the
> following:
>
> pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> assign device: host bdf = 2:0:0
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
> scsi4 : iSCSI Initiator over TCP/IP
> scsi 4:0:0:0: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI:
> 5 sd 4:0:0:0: Attached scsi generic sg1 type 0
> scsi 4:0:0:1: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0 ANSI:
> 5 sd 4:0:0:1: Attached scsi generic sg2 type 0
> sd 4:0:0:0: [sdb] 262144 512-byte hardware sectors: (134 MB/128 MiB)
> sd 4:0:0:1: [sdc] 262144 512-byte hardware sectors: (134 MB/128 MiB)
> sd 4:0:0:0: [sdb] Write Protect is off
> sd 4:0:0:0: [sdb] Mode Sense: 2f 00 00 00
> sd 4:0:0:1: [sdc] Write Protect is off
> sd 4:0:0:1: [sdc] Mode Sense: 2f 00 00 00
> sd 4:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't
> support DPO or FUA sd 4:0:0:1: [sdc] Write cache: disabled, read cache:
> enabled, doesn't support DPO or FUA sdb:<6> sdc: unknown partition table
> sd 4:0:0:0: [sdb] Attached SCSI disk
>  unknown partition table
> sd 4:0:0:1: [sdc] Attached SCSI disk
> ------------[ cut here ]------------
> WARNING: at kernel/irq/manage.c:260 enable_irq+0x36/0x50()
> Hardware name: empty
> Unbalanced enable for IRQ 59
> Modules linked in: ipt_REJECT xt_tcpudp bridge stp sunrpc iptable_filter
> ip_tables xt_state nf_conntrack ip6table_filter ip6_tables x_tables ib_iser
> rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp
> libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq
> freq_table ext3 jbd loop dm_multipath scsi_dh kvm_intel kvm uinput i2c_i801
> firewire_ohci joydev firewire_core sg i2c_core 8250_pnp crc_itu_t e1000e
> 8250 serial_core rtc_cmos pcspkr serio_raw rtc_core rtc_lib button sd_mod
> dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod uhci_hcd
> ohci_hcd ehci_hcd ata_piix libata scsi_mod [last unloaded: microcode] Pid:
> 51, comm: events/0 Tainted: G        W  2.6.30-rc3 #11
> Call Trace:
>  [<ffffffff80235fee>] ? warn_slowpath+0xcb/0xe8
>  [<ffffffff80253a7c>] ? generic_exec_single+0x6a/0x88
>  [<ffffffff8022acec>] ? update_curr+0x67/0xeb
>  [<ffffffffa0198748>] ? vcpu_kick_intr+0x0/0x1 [kvm]
>  [<ffffffff8020a5d8>] ? __switch_to+0xb6/0x274
>  [<ffffffff8022b70a>] ? __dequeue_entity+0x1b/0x2f
>  [<ffffffffa01ac7e4>] ? kvm_irq_delivery_to_apic+0xb3/0xf7 [kvm]
>  [<ffffffffa01aa4d4>] ? __apic_accept_irq+0x15a/0x173 [kvm]
>  [<ffffffffa01ac883>] ? kvm_set_msi+0x5b/0x60 [kvm]
>  [<ffffffff80266d97>] ? enable_irq+0x36/0x50
>  [<ffffffffa0195ab5>] ? kvm_assigned_dev_interrupt_work_handler+0x6d/0xbc
> [kvm] [<ffffffff802449fa>] ? worker_thread+0x182/0x223
>  [<ffffffff8024820b>] ? autoremove_wake_function+0x0/0x2a
>  [<ffffffff80244878>] ? worker_thread+0x0/0x223
>  [<ffffffff80244878>] ? worker_thread+0x0/0x223
>  [<ffffffff80247e72>] ? kthread+0x54/0x7e
>  [<ffffffff8020cb0a>] ? child_rip+0xa/0x20
>  [<ffffffff804d0af5>] ? _spin_lock+0x5/0x8
>  [<ffffffff80247e1e>] ? kthread+0x0/0x7e
>  [<ffffffff8020cb00>] ? child_rip+0x0/0x20
> ---[ end trace 3fbc2dd20bf89ef1 ]---
>  connection1:0: ping timeout of 5 secs expired, last rx 4295286327, last
> ping 4295285518, now 4295286768 connection1:0: detected conn error (1011)
>
> Attached are the v2.6.30-rc3 KVM host and v2.6.29.2 KVM guest dmesg
> output.  When the 'Unbalanced enable for IRQ 59' happens on the KVM
> host, I do not see any exceptions in KVM guest (other than the iSCSI
> connections drop), but it requires a restart of KVM+qemu-system-x86_64
> to get the e1000e port back up.

Yeah, there is a bug. And Marcelo has fix for it, please wait a while for 
check in.

> Other than that loopback scenario, things are looking good quite good
> with this combination of kvm-85 kernel+guest so far for me.  I did end
> up taking out the two 8x function 2x Path/Function PCIe IOV adapters for
> now, as it seemed to have an effect on stability with all of MSI-X
> interrupts enabled on the KVM host for 16 virtual adapters.

That's great! :)

-- 
regards
Yang, Sheng

>
> I will keep testing with e1000e ports and let the list know the
> progress.  Thanks for your comments!
>
> --nab
>
> > On Mon, May 04, 2009 at 06:40:36PM +0800, Nicholas A. Bellinger wrote:
> > > On Mon, 2009-05-04 at 17:49 +0800, Sheng Yang wrote:
> > > > On Monday 04 May 2009 17:11:59 Nicholas A. Bellinger wrote:
> > > > > On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> > > > > > On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > > > > > > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > > > > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > > > > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > > > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A.
> > > > > > > > > > Bellinger
> > > >
> > > > wrote:
> > > > > > > > > > > Greetings KVM folks,
> > > > > > > > > > >
> > > > > > > > > > > I wondering if any information exists for doing SR-IOV
> > > > > > > > > > > on the new VT-d capable chipsets with KVM..?  From what
> > > > > > > > > > > I understand the patches for doing this with KVM are
> > > > > > > > > > > floating around, but I have been unable to find any
> > > > > > > > > > > user-level docs for actually making it all go against a
> > > > > > > > > > > upstream v2.6.30-rc3 code..
> > > > > > > > > > >
> > > > > > > > > > > So far I have been doing IOV testing with Xen 3.3 and
> > > > > > > > > > > 3.4.0-pre, and I am really hoping to be able to jump to
> > > > > > > > > > > KVM for single-function and and then multi-function
> > > > > > > > > > > SR-IOV.  I know that the VM migration stuff for IOV in
> > > > > > > > > > > Xen is up and running, and I assume it is being worked
> > > > > > > > > > > in for KVM instance migration as well..? This part is
> > > > > > > > > > > less important (at least for me :-) than getting a
> > > > > > > > > > > stable SR-IOV setup running under the KVM hypervisor.. 
> > > > > > > > > > > Does anyone have any pointers for this..?
> > > > > > > > > > >
> > > > > > > > > > > Any comments or suggestions are appreciated!
> > > > > > > > > >
> > > > > > > > > > Hi Nicholas
> > > > > > > > > >
> > > > > > > > > > The patches are not floating around now. As you know,
> > > > > > > > > > SR-IOV for Linux have been in 2.6.30, so then you can use
> > > > > > > > > > upstream KVM and qemu-kvm(or recent released kvm-85) with
> > > > > > > > > > 2.6.30-rc3 as host kernel. And some time ago, there are
> > > > > > > > > > several SRIOV related patches for qemu-kvm, and now they
> > > > > > > > > > all have been checked in.
> > > > > > > > > >
> > > > > > > > > > And for KVM, the extra document is not necessary, for you
> > > > > > > > > > can simple assign a VF to guest like any other devices.
> > > > > > > > > > And how to create VF is specific for each device driver.
> > > > > > > > > > So just create a VF then assign it to KVM guest is fine.
> > > > > > > > >
> > > > > > > > > Greetings Sheng,
> > > > > > > > >
> > > > > > > > > So, I have been trying the latest kvm-85 release on a
> > > > > > > > > v2.6.30-rc3 checkout from linux-2.6.git on a CentOS 5u3
> > > > > > > > > x86_64 install on Intel IOH-5520 based dual socket Nehalem
> > > > > > > > > board.  I have enabled DMAR and Interrupt Remapping my KVM
> > > > > > > > > host using v2.6.30-rc3 and from what I can tell, the
> > > > > > > > > KVM_CAP_* defines from libkvm are enabled with building
> > > > > > > > > kvm-85 after './configure
> > > > > > > > > --kerneldir=/usr/src/linux-2.6.git' and the PCI passthrough
> > > > > > > > > code is being enabled in
> > > > > > > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > > > > > > >
> > > > > > > > > >From there, I use the freshly installed qemu-x86_64-system
> > > > > > > > > > binary to
> > > > > > > > >
> > > > > > > > > start a Debian 5 x86_64 HVM (that previously had been
> > > > > > > > > moving network packets under Xen for PCIe passthrough). I
> > > > > > > > > see the MSI-X interrupt remapping working on the KVM host
> > > > > > > > > for the passed -pcidevice, and the MMIO mappings from the
> > > > > > > > > qemu build that I also saw while using Xen/qemu-dm built
> > > > > > > > > with PCI passthrough are there as well..
> > > > > > > >
> > > > > > > > Hi Nicholas
> > > > > > > >
> > > > > > > > > But while the KVM guest is booting, I see the following
> > > > > > > > > exception(s) from qemu-x86_64-system for one of the VFs for
> > > > > > > > > a multi-function PCIe device:
> > > > > > > > >
> > > > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > > > >
> > > > > > > > This one is mostly harmless.
> > > > > > >
> > > > > > > Ok, good to know..  :-)
> > > > > > >
> > > > > > > > > I try with one of the on-board e1000e ports (02:00.0) and I
> > > > > > > > > see the same exception along with some MSI-X exceptions
> > > > > > > > > from qemu-x86_64-system in KVM guest.. However, I am still
> > > > > > > > > able to see the e1000e and the other vxge multi-function
> > > > > > > > > device with lspci, but I am unable to dhcp or ping with the
> > > > > > > > > e1000e and VF from multi-function device fails to register
> > > > > > > > > the MSI-X interrupt in the guest..
> > > > > > > >
> > > > > > > > Did you see the interrupt in the guest and host side?
> > > > > > >
> > > > > > > Ok, I am restarting the e1000e test with a fresh Fedora 11
> > > > > > > install and KVM host kernel 2.6.29.1-111.fc11.x86_64.   After
> > > > > > > unbinding and attaching the e1000e single-function device at
> > > > > > > 02:00.0 to pci-stub with:
> > > > > > >
> > > > > > >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> > > > > > >    echo 0000:02:00.0 >
> > > > > > > /sys/bus/pci/devices/0000:02:00.0/driver/unbind echo
> > > > > > > 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> > > > > > >
> > > > > > > I see the following the KVM host kernel ring buffer:
> > > > > > >
> > > > > > >    e1000e 0000:02:00.0: PCI INT A disabled
> > > > > > >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) ->
> > > > > > > IRQ 17 pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > > > > > >
> > > > > > > >  I think you can try on-
> > > > > > > > board e1000e for MSI-X first. And please ensure correlated
> > > > > > > > driver have been loaded correctly.
> > > > > > >
> > > > > > > <nod>..
> > > > > > >
> > > > > > > >  And what do you mean by "some MSI-X exceptions"? Better with
> > > > > > > > the log.
> > > > > > >
> > > > > > > Ok, with the Fedora 11 installed qemu-kemu, I see the expected
> > > > > > > kvm_destroy_phys_mem() statements:
> > > > > > >
> > > > > > > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > > > > > > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid
> > > > > > > parameters (slot=-1)
> > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > > >
> > > > > > > However I still see the following in the KVM guest kernel ring
> > > > > > > buffer running v2.6.30-rc in the HVM guest.
> > > > > > >
> > > > > > > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ
> > > > > > > 10 [    5.524582] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA]
> > > > > > > -> GSI 10 (level, high) -> IRQ 10 [    5.525710] e1000e
> > > > > > > 0000:00:05.0: setting latency timer to 64
> > > > > > > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to initialize
> > > > > > > MSI-X interrupts.  Falling back to MSI interrupts. [   
> > > > > > > 5.527200] 0000:00:05.0: 0000:00:05.0: Failed to initialize MSI
> > > > > > > interrupts. Falling back to legacy interrupts. [    5.829988]
> > > > > > > 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > > > > > > 00:e0:81:c0:90:b2 [    5.830672] 0000:00:05.0: eth0: Intel(R)
> > > > > > > PRO/1000 Network Connection [    5.831240] 0000:00:05.0: eth0:
> > > > > > > MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > > > > >
> > > > > > Hi Nicholas
> > > > > >
> > > > > > I think something need to be clarify:
> > > > > > 1. For SRIOV, you need 2.6.30 as host kernel... But it's better
> > > > > > to know if normal device assignment work in your environment at
> > > > > > first. 2. The Fedora's userspace is even more old... You'd better
> > > > > > try qemu-kvm upstream, which is more convenient for us to track
> > > > > > the problem(and kvm-85 is also ok). And as you see above, your
> > > > > > QEmu don't support MSI/MSIX...
> > > > >
> > > > > Ok, got it..
> > > > >
> > > > > > So you can:
> > > > > > 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
> > > > >
> > > > > Ok, I am now updated on in the FC 11 Host with kvm-85 kernel
> > > > > modules and am using the built qemu-system-x86_64 from the kvm-85
> > > > > source package:
> > > > >
> > > > > loaded kvm module (kvm-85)
> > > > > QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008
> > > > > Fabrice Bellard
> > > > >
> > > > > > 2. Your host kernel is Fedora 11 Preview, that should be fine
> > > > > > with device assignment at first(and let's solve it first, SRIOV
> > > > > > the next step).
> > > > >
> > > > > Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM host
> > > > > for the momemt to get e1000e working.  But I will start building a
> > > > > v2.6.30-rc3 kernel again for my fc11 host kernel as I do need
> > > > > SR-IOV at some point... :-)
> > > > >
> > > > > > 3. Your KVM version seems like kvm-85, you may provide some dmesg
> > > > > > on host side(I think you didn't use the KVM come along with
> > > > > > kernel).
> > > > >
> > > > > Ok, now within the KVM guest running v2.6.29.2, I see the
> > > > > following:
> > > > >
> > > > > [    2.669243] e1000e: Intel(R) PRO/1000 Network Driver -
> > > > > 0.3.3.3-k6 [    2.672931] e1000e: Copyright (c) 1999-2008 Intel
> > > > > Corporation. [    2.674932] ACPI: PCI Interrupt Link [LNKA] enabled
> > > > > at IRQ 10 [    2.675181] 8139too Fast Ethernet driver 0.9.28
> > > > > [    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI
> > > > > 10 (level, high) -> IRQ 10
> > > > > [    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
> > > > > [    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> > > > > [    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> > > > > [    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> > > > > [    2.698039] FDC 0 is a S82078B
> > > > > [    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > > > > 00:e0:81:c0:90:b2
> > > > > [    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network
> > > > > Connection [    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA
> > > > > No: ffffff-0ff
> > > > >
> > > > > And the folllowing from /proc/interrupts inside of the KVM guest:
> > > > >
> > > > >  24:        117          0          0          0          0        
> > > > >  0 0          0          0          0   PCI-MSI-edge      eth1-rx-0
> > > > > 25: 0          0          0          0          0          0       
> > > > >   0 0          0          0   PCI-MSI-edge      eth1-tx-0 26:      
> > > > >    2 0          0          0          0          0          0 0    
> > > > >      0          0   PCI-MSI-edge      eth1
> > > > >
> > > > > ethtool eth1 reports that Link is detected, but I am still unable
> > > > > to get a dhcp to work.
> > > >
> > > > It's a little strange that I checked all the log you posted, but
> > > > can't find anything suspicious...(Except you got a MCE log in your
> > > > dmesg, but I don't think it would relate to this).
> > > >
> > > > You also already have interrupts in the guest for eth1-rx-0 and eth1,
> > > > so at least part of interrupts can be delivered to the guest.
> > > >
> > > > You can try to connect the port to another NIC port directly. Set
> > > > fixed ip for each, then ping each other.
> > > >
> > > > You can also try to disable MSI-X capability in QEmu. Just using "#if
> > > > 0/#endif" to wrap "#ifdef KVM_CAP_DEVICE_MSIX/#endif" in
> > > > hw/assigned_device_pci_cap_init(). Then the device would use MSI.
> > > >
> > > > If I am lucky enough to find a 82574L card by hand, I would give it a
> > > > try...
> > > >
> > > > --
> > > > regards
> > > > Yang, Sheng
> > >
> > > Greetings Sheng,
> > >
> > > So I updated my FC11 Host to kernel v2.6.30-rc3 (and enabled ext4 of
> > > course) and rebuilt the kvm-85 source kernel module and
> > > qemu-system-x86_64 and I am now able to get dhcp and IP ops from the
> > > 02:00.0 device on my IOH-5520 board with the KVM guest using a
> > > v2.6.29.2 kernel!!  Everything is looking good with the v2.6.29.2, but
> > > after a quick reboot back into my v2.6.30-rc3 KVM guest kernel build
> > > e1000e it looks like I am unable to get dhcp.
> > >
> > > Rebooting back into KVM Guest kernel v2.6.29.2 brings the pci-stub
> > > assigned e1000e 82574L assigned with dhcp and everything looks
> > > good!  :-)
> > >
> > > I will keep poking at the v2.6.30-rc KVM guests (I am going to do a
> > > complete rebuild) and see if it does not start move IP packets as
> > > well..
> > >
> > > Thanks for all of your help in getting setup!
> > >
> > > --nab
> > >
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: KVM x86_64 with SR-IOV..? (device passthrough with LIO-Target v3.0)

[Sheng Yang]

On Tuesday 05 May 2009 19:28:15 Nicholas A. Bellinger wrote:
> On Tue, 2009-05-05 at 03:43 -0700, Nicholas A. Bellinger wrote:
> > On Tue, 2009-05-05 at 09:42 +0800, Yu Zhao wrote:
> > > Hi,
> > >
> > > The VF also works in the host if the VF driver is programed properly.
> > > So it would be easier to develop the VF driver in the host and then
> > > verify the VF driver in the guest.
> > >
> > > BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
> > > the CONFIG_PCI_IOV in the kernel .config?
> > >
> > > Thanks,
> > > Yu
> >
> > Greetings Yu and Sheng,
> >
> > So the original attachment was for the v2.6.29-fc11 host kernel output,
> > I ended up jumping to v2.6.30-rc3 (and making sure CONFIG_PCI_IOV was
> > enabled) for KVM host with kvm-85 and now things are looking quite
> > stable for me.
> >
> > So far I have been able to successfully push LIO-Target v3.0 traffic
> > *inside* a v2.6.29.2 KVM guest via the onboard e1000e (02:00.0) port
> > from another Linux/iSCSI Initiator machine using a Intel 1 Gb/sec port.
> > I am running badblocks tests to iSCSI Logical Units for RAMDISK_DR and
> > FILEIO storage objects (in the KVM Guest), and they are passing
> > validation and I am seeing ~500 Mb/sec of throughput and very low CPU
> > usage in the KVM guests.
>
> Ok I am seeing another issue with the e1000e port on 02:00.0..:
>
> As i start to push multiple badblocks tests RAMDISK_DR iSCSI Logical
> units into KVM Guest running LIO v2.6.29.2 from the external Linux/iSCSI
> Initiator machine, after about 100 GB of iSCSI traffic, I see the
> following exception in KVM host v2.6.30-rc3:
>
> DRHD: handling fault status reg 2
> DMAR:[DMA Write] Request device [02:00.0] fault addr 7fc958b010000
> DMAR:[fault reason 04] Access beyond MGAW

This means the fault address is too big.... It's got 51 bits width which is 
far beyond the physical address limit of current IA32e(48 bits).

Don't know how you can get this...

-- 
regards
Yang, Sheng

> pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
> pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
>
> I am able to restart the LIO-Target KVM Guest and the Linux/iSCSI
> Initiators are able to reconnect..  Wow, very cool..
>
> Not sure if this is a bug in the target_core_mod RAMDISK_DR subsystem
> plugin (mapping struct iovec to internally allocated struct page) or
> what.  I will have to look at the DMAR code to understand what this
> exception means..
>
> --nab
>
> > One issue I did notice while using the pci-stub method of
> > device-assignment with same e1000 port (02:00.0) was while using an
> > iSCSI Initiator (Open-iSCSI) on the KVM Host machine and doing sustained
> > traffic into the LIO-Target KVM Guest on the same local KVM host to max
> > out traffic between the other onboard e1000e port (03.00.0), I see the
> > following:
> >
> > pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
> > assign device: host bdf = 2:0:0
> > pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
> > scsi4 : iSCSI Initiator over TCP/IP
> > scsi 4:0:0:0: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0
> > ANSI: 5 sd 4:0:0:0: Attached scsi generic sg1 type 0
> > scsi 4:0:0:1: Direct-Access     LIO-ORG  RAMDISK-DR       3.0  PQ: 0
> > ANSI: 5 sd 4:0:0:1: Attached scsi generic sg2 type 0
> > sd 4:0:0:0: [sdb] 262144 512-byte hardware sectors: (134 MB/128 MiB)
> > sd 4:0:0:1: [sdc] 262144 512-byte hardware sectors: (134 MB/128 MiB)
> > sd 4:0:0:0: [sdb] Write Protect is off
> > sd 4:0:0:0: [sdb] Mode Sense: 2f 00 00 00
> > sd 4:0:0:1: [sdc] Write Protect is off
> > sd 4:0:0:1: [sdc] Mode Sense: 2f 00 00 00
> > sd 4:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't
> > support DPO or FUA sd 4:0:0:1: [sdc] Write cache: disabled, read cache:
> > enabled, doesn't support DPO or FUA sdb:<6> sdc: unknown partition table
> > sd 4:0:0:0: [sdb] Attached SCSI disk
> >  unknown partition table
> > sd 4:0:0:1: [sdc] Attached SCSI disk
> > ------------[ cut here ]------------
> > WARNING: at kernel/irq/manage.c:260 enable_irq+0x36/0x50()
> > Hardware name: empty
> > Unbalanced enable for IRQ 59
> > Modules linked in: ipt_REJECT xt_tcpudp bridge stp sunrpc iptable_filter
> > ip_tables xt_state nf_conntrack ip6table_filter ip6_tables x_tables
> > ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp
> > libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq
> > freq_table ext3 jbd loop dm_multipath scsi_dh kvm_intel kvm uinput
> > i2c_i801 firewire_ohci joydev firewire_core sg i2c_core 8250_pnp
> > crc_itu_t e1000e 8250 serial_core rtc_cmos pcspkr serio_raw rtc_core
> > rtc_lib button sd_mod dm_snapshot dm_zero dm_mirror dm_region_hash dm_log
> > dm_mod uhci_hcd ohci_hcd ehci_hcd ata_piix libata scsi_mod [last
> > unloaded: microcode] Pid: 51, comm: events/0 Tainted: G        W 
> > 2.6.30-rc3 #11
> > Call Trace:
> >  [<ffffffff80235fee>] ? warn_slowpath+0xcb/0xe8
> >  [<ffffffff80253a7c>] ? generic_exec_single+0x6a/0x88
> >  [<ffffffff8022acec>] ? update_curr+0x67/0xeb
> >  [<ffffffffa0198748>] ? vcpu_kick_intr+0x0/0x1 [kvm]
> >  [<ffffffff8020a5d8>] ? __switch_to+0xb6/0x274
> >  [<ffffffff8022b70a>] ? __dequeue_entity+0x1b/0x2f
> >  [<ffffffffa01ac7e4>] ? kvm_irq_delivery_to_apic+0xb3/0xf7 [kvm]
> >  [<ffffffffa01aa4d4>] ? __apic_accept_irq+0x15a/0x173 [kvm]
> >  [<ffffffffa01ac883>] ? kvm_set_msi+0x5b/0x60 [kvm]
> >  [<ffffffff80266d97>] ? enable_irq+0x36/0x50
> >  [<ffffffffa0195ab5>] ? kvm_assigned_dev_interrupt_work_handler+0x6d/0xbc
> > [kvm] [<ffffffff802449fa>] ? worker_thread+0x182/0x223
> >  [<ffffffff8024820b>] ? autoremove_wake_function+0x0/0x2a
> >  [<ffffffff80244878>] ? worker_thread+0x0/0x223
> >  [<ffffffff80244878>] ? worker_thread+0x0/0x223
> >  [<ffffffff80247e72>] ? kthread+0x54/0x7e
> >  [<ffffffff8020cb0a>] ? child_rip+0xa/0x20
> >  [<ffffffff804d0af5>] ? _spin_lock+0x5/0x8
> >  [<ffffffff80247e1e>] ? kthread+0x0/0x7e
> >  [<ffffffff8020cb00>] ? child_rip+0x0/0x20
> > ---[ end trace 3fbc2dd20bf89ef1 ]---
> >  connection1:0: ping timeout of 5 secs expired, last rx 4295286327, last
> > ping 4295285518, now 4295286768 connection1:0: detected conn error (1011)
> >
> > Attached are the v2.6.30-rc3 KVM host and v2.6.29.2 KVM guest dmesg
> > output.  When the 'Unbalanced enable for IRQ 59' happens on the KVM
> > host, I do not see any exceptions in KVM guest (other than the iSCSI
> > connections drop), but it requires a restart of KVM+qemu-system-x86_64
> > to get the e1000e port back up.
> >
> > Other than that loopback scenario, things are looking good quite good
> > with this combination of kvm-85 kernel+guest so far for me.  I did end
> > up taking out the two 8x function 2x Path/Function PCIe IOV adapters for
> > now, as it seemed to have an effect on stability with all of MSI-X
> > interrupts enabled on the KVM host for 16 virtual adapters.
> >
> > I will keep testing with e1000e ports and let the list know the
> > progress.  Thanks for your comments!
> >
> > --nab
> >
> > > On Mon, May 04, 2009 at 06:40:36PM +0800, Nicholas A. Bellinger wrote:
> > > > On Mon, 2009-05-04 at 17:49 +0800, Sheng Yang wrote:
> > > > > On Monday 04 May 2009 17:11:59 Nicholas A. Bellinger wrote:
> > > > > > On Mon, 2009-05-04 at 16:20 +0800, Sheng Yang wrote:
> > > > > > > On Monday 04 May 2009 12:36:04 Nicholas A. Bellinger wrote:
> > > > > > > > On Mon, 2009-05-04 at 10:09 +0800, Sheng Yang wrote:
> > > > > > > > > On Monday 04 May 2009 08:53:07 Nicholas A. Bellinger wrote:
> > > > > > > > > > On Sat, 2009-05-02 at 18:22 +0800, Sheng Yang wrote:
> > > > > > > > > > > On Thu, Apr 30, 2009 at 01:22:54PM -0700, Nicholas A.
> > > > > > > > > > > Bellinger
> > > > >
> > > > > wrote:
> > > > > > > > > > > > Greetings KVM folks,
> > > > > > > > > > > >
> > > > > > > > > > > > I wondering if any information exists for doing
> > > > > > > > > > > > SR-IOV on the new VT-d capable chipsets with KVM..? 
> > > > > > > > > > > > From what I understand the patches for doing this
> > > > > > > > > > > > with KVM are floating around, but I have been unable
> > > > > > > > > > > > to find any user-level docs for actually making it
> > > > > > > > > > > > all go against a upstream v2.6.30-rc3 code..
> > > > > > > > > > > >
> > > > > > > > > > > > So far I have been doing IOV testing with Xen 3.3 and
> > > > > > > > > > > > 3.4.0-pre, and I am really hoping to be able to jump
> > > > > > > > > > > > to KVM for single-function and and then
> > > > > > > > > > > > multi-function SR-IOV.  I know that the VM migration
> > > > > > > > > > > > stuff for IOV in Xen is up and running, and I assume
> > > > > > > > > > > > it is being worked in for KVM instance migration as
> > > > > > > > > > > > well..? This part is less important (at least for me
> > > > > > > > > > > > :-) than getting a stable SR-IOV setup running under
> > > > > > > > > > > > the KVM hypervisor..  Does anyone have any pointers
> > > > > > > > > > > > for this..?
> > > > > > > > > > > >
> > > > > > > > > > > > Any comments or suggestions are appreciated!
> > > > > > > > > > >
> > > > > > > > > > > Hi Nicholas
> > > > > > > > > > >
> > > > > > > > > > > The patches are not floating around now. As you know,
> > > > > > > > > > > SR-IOV for Linux have been in 2.6.30, so then you can
> > > > > > > > > > > use upstream KVM and qemu-kvm(or recent released
> > > > > > > > > > > kvm-85) with 2.6.30-rc3 as host kernel. And some time
> > > > > > > > > > > ago, there are several SRIOV related patches for
> > > > > > > > > > > qemu-kvm, and now they all have been checked in.
> > > > > > > > > > >
> > > > > > > > > > > And for KVM, the extra document is not necessary, for
> > > > > > > > > > > you can simple assign a VF to guest like any other
> > > > > > > > > > > devices. And how to create VF is specific for each
> > > > > > > > > > > device driver. So just create a VF then assign it to
> > > > > > > > > > > KVM guest is fine.
> > > > > > > > > >
> > > > > > > > > > Greetings Sheng,
> > > > > > > > > >
> > > > > > > > > > So, I have been trying the latest kvm-85 release on a
> > > > > > > > > > v2.6.30-rc3 checkout from linux-2.6.git on a CentOS 5u3
> > > > > > > > > > x86_64 install on Intel IOH-5520 based dual socket
> > > > > > > > > > Nehalem board.  I have enabled DMAR and Interrupt
> > > > > > > > > > Remapping my KVM host using v2.6.30-rc3 and from what I
> > > > > > > > > > can tell, the KVM_CAP_* defines from libkvm are enabled
> > > > > > > > > > with building kvm-85 after './configure
> > > > > > > > > > --kerneldir=/usr/src/linux-2.6.git' and the PCI
> > > > > > > > > > passthrough code is being enabled in
> > > > > > > > > > kvm-85/qemu/hw/device-assignment.c AFAICT..
> > > > > > > > > >
> > > > > > > > > > >From there, I use the freshly installed
> > > > > > > > > > > qemu-x86_64-system binary to
> > > > > > > > > >
> > > > > > > > > > start a Debian 5 x86_64 HVM (that previously had been
> > > > > > > > > > moving network packets under Xen for PCIe passthrough). I
> > > > > > > > > > see the MSI-X interrupt remapping working on the KVM host
> > > > > > > > > > for the passed -pcidevice, and the MMIO mappings from the
> > > > > > > > > > qemu build that I also saw while using Xen/qemu-dm built
> > > > > > > > > > with PCI passthrough are there as well..
> > > > > > > > >
> > > > > > > > > Hi Nicholas
> > > > > > > > >
> > > > > > > > > > But while the KVM guest is booting, I see the following
> > > > > > > > > > exception(s) from qemu-x86_64-system for one of the VFs
> > > > > > > > > > for a multi-function PCIe device:
> > > > > > > > > >
> > > > > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > > > > >
> > > > > > > > > This one is mostly harmless.
> > > > > > > >
> > > > > > > > Ok, good to know..  :-)
> > > > > > > >
> > > > > > > > > > I try with one of the on-board e1000e ports (02:00.0) and
> > > > > > > > > > I see the same exception along with some MSI-X exceptions
> > > > > > > > > > from qemu-x86_64-system in KVM guest.. However, I am
> > > > > > > > > > still able to see the e1000e and the other vxge
> > > > > > > > > > multi-function device with lspci, but I am unable to dhcp
> > > > > > > > > > or ping with the e1000e and VF from multi-function device
> > > > > > > > > > fails to register the MSI-X interrupt in the guest..
> > > > > > > > >
> > > > > > > > > Did you see the interrupt in the guest and host side?
> > > > > > > >
> > > > > > > > Ok, I am restarting the e1000e test with a fresh Fedora 11
> > > > > > > > install and KVM host kernel 2.6.29.1-111.fc11.x86_64.   After
> > > > > > > > unbinding and attaching the e1000e single-function device at
> > > > > > > > 02:00.0 to pci-stub with:
> > > > > > > >
> > > > > > > >    echo "8086 10d3" > /sys/bus/pci/drivers/pci-stub/new_id
> > > > > > > >    echo 0000:02:00.0 >
> > > > > > > > /sys/bus/pci/devices/0000:02:00.0/driver/unbind echo
> > > > > > > > 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/bind
> > > > > > > >
> > > > > > > > I see the following the KVM host kernel ring buffer:
> > > > > > > >
> > > > > > > >    e1000e 0000:02:00.0: PCI INT A disabled
> > > > > > > >    pci-stub 0000:02:00.0: PCI INT A -> GSI 17 (level, low) ->
> > > > > > > > IRQ 17 pci-stub 0000:02:00.0: irq 58 for MSI/MSI-X
> > > > > > > >
> > > > > > > > >  I think you can try on-
> > > > > > > > > board e1000e for MSI-X first. And please ensure correlated
> > > > > > > > > driver have been loaded correctly.
> > > > > > > >
> > > > > > > > <nod>..
> > > > > > > >
> > > > > > > > >  And what do you mean by "some MSI-X exceptions"? Better
> > > > > > > > > with the log.
> > > > > > > >
> > > > > > > > Ok, with the Fedora 11 installed qemu-kemu, I see the
> > > > > > > > expected kvm_destroy_phys_mem() statements:
> > > > > > > >
> > > > > > > > #kvm-host qemu-kvm -m 2048 -smp 8 -pcidevice host=02:00.0
> > > > > > > > lenny64guest1-orig.img BUG: kvm_destroy_phys_mem: invalid
> > > > > > > > parameters (slot=-1)
> > > > > > > > BUG: kvm_destroy_phys_mem: invalid parameters (slot=-1)
> > > > > > > >
> > > > > > > > However I still see the following in the KVM guest kernel
> > > > > > > > ring buffer running v2.6.30-rc in the HVM guest.
> > > > > > > >
> > > > > > > > [    5.523790] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ
> > > > > > > > 10 [    5.524582] e1000e 0000:00:05.0: PCI INT A ->
> > > > > > > > Link[LNKA] -> GSI 10 (level, high) -> IRQ 10 [    5.525710]
> > > > > > > > e1000e 0000:00:05.0: setting latency timer to 64
> > > > > > > > [    5.526048] 0000:00:05.0: 0000:00:05.0: Failed to
> > > > > > > > initialize MSI-X interrupts.  Falling back to MSI interrupts.
> > > > > > > > [    5.527200] 0000:00:05.0: 0000:00:05.0: Failed to
> > > > > > > > initialize MSI interrupts. Falling back to legacy interrupts.
> > > > > > > > [    5.829988] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width
> > > > > > > > x1) 00:e0:81:c0:90:b2 [    5.830672] 0000:00:05.0: eth0:
> > > > > > > > Intel(R) PRO/1000 Network Connection [    5.831240]
> > > > > > > > 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA No: ffffff-0ff
> > > > > > >
> > > > > > > Hi Nicholas
> > > > > > >
> > > > > > > I think something need to be clarify:
> > > > > > > 1. For SRIOV, you need 2.6.30 as host kernel... But it's better
> > > > > > > to know if normal device assignment work in your environment at
> > > > > > > first. 2. The Fedora's userspace is even more old... You'd
> > > > > > > better try qemu-kvm upstream, which is more convenient for us
> > > > > > > to track the problem(and kvm-85 is also ok). And as you see
> > > > > > > above, your QEmu don't support MSI/MSIX...
> > > > > >
> > > > > > Ok, got it..
> > > > > >
> > > > > > > So you can:
> > > > > > > 1. Use latest qemu-kvm or kvm-85's QEmu. As well as latest KVM.
> > > > > >
> > > > > > Ok, I am now updated on in the FC 11 Host with kvm-85 kernel
> > > > > > modules and am using the built qemu-system-x86_64 from the kvm-85
> > > > > > source package:
> > > > > >
> > > > > > loaded kvm module (kvm-85)
> > > > > > QEMU PC emulator version 0.10.0 (kvm-85), Copyright (c) 2003-2008
> > > > > > Fabrice Bellard
> > > > > >
> > > > > > > 2. Your host kernel is Fedora 11 Preview, that should be fine
> > > > > > > with device assignment at first(and let's solve it first, SRIOV
> > > > > > > the next step).
> > > > > >
> > > > > > Ok, yeah I will stick with the v2.6.29 fc11 kernel on the KVM
> > > > > > host for the momemt to get e1000e working.  But I will start
> > > > > > building a v2.6.30-rc3 kernel again for my fc11 host kernel as I
> > > > > > do need SR-IOV at some point... :-)
> > > > > >
> > > > > > > 3. Your KVM version seems like kvm-85, you may provide some
> > > > > > > dmesg on host side(I think you didn't use the KVM come along
> > > > > > > with kernel).
> > > > > >
> > > > > > Ok, now within the KVM guest running v2.6.29.2, I see the
> > > > > > following:
> > > > > >
> > > > > > [    2.669243] e1000e: Intel(R) PRO/1000 Network Driver -
> > > > > > 0.3.3.3-k6 [    2.672931] e1000e: Copyright (c) 1999-2008 Intel
> > > > > > Corporation. [    2.674932] ACPI: PCI Interrupt Link [LNKA]
> > > > > > enabled at IRQ 10 [    2.675181] 8139too Fast Ethernet driver
> > > > > > 0.9.28
> > > > > > [    2.676783] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] ->
> > > > > > GSI 10 (level, high) -> IRQ 10
> > > > > > [    2.678143] e1000e 0000:00:05.0: setting latency timer to 64
> > > > > > [    2.679539] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> > > > > > [    2.679603] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> > > > > > [    2.679659] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> > > > > > [    2.698039] FDC 0 is a S82078B
> > > > > > [    2.801673] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > > > > > 00:e0:81:c0:90:b2
> > > > > > [    2.802811] 0000:00:05.0: eth0: Intel(R) PRO/1000 Network
> > > > > > Connection [    2.803697] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA
> > > > > > No: ffffff-0ff
> > > > > >
> > > > > > And the folllowing from /proc/interrupts inside of the KVM guest:
> > > > > >
> > > > > >  24:        117          0          0          0          0      
> > > > > >    0 0          0          0          0   PCI-MSI-edge     
> > > > > > eth1-rx-0 25: 0          0          0          0          0      
> > > > > >    0          0 0          0          0   PCI-MSI-edge     
> > > > > > eth1-tx-0 26:          2 0          0          0          0      
> > > > > >    0          0 0          0          0   PCI-MSI-edge      eth1
> > > > > >
> > > > > > ethtool eth1 reports that Link is detected, but I am still unable
> > > > > > to get a dhcp to work.
> > > > >
> > > > > It's a little strange that I checked all the log you posted, but
> > > > > can't find anything suspicious...(Except you got a MCE log in your
> > > > > dmesg, but I don't think it would relate to this).
> > > > >
> > > > > You also already have interrupts in the guest for eth1-rx-0 and
> > > > > eth1, so at least part of interrupts can be delivered to the guest.
> > > > >
> > > > > You can try to connect the port to another NIC port directly. Set
> > > > > fixed ip for each, then ping each other.
> > > > >
> > > > > You can also try to disable MSI-X capability in QEmu. Just using
> > > > > "#if 0/#endif" to wrap "#ifdef KVM_CAP_DEVICE_MSIX/#endif" in
> > > > > hw/assigned_device_pci_cap_init(). Then the device would use MSI.
> > > > >
> > > > > If I am lucky enough to find a 82574L card by hand, I would give it
> > > > > a try...
> > > > >
> > > > > --
> > > > > regards
> > > > > Yang, Sheng
> > > >
> > > > Greetings Sheng,
> > > >
> > > > So I updated my FC11 Host to kernel v2.6.30-rc3 (and enabled ext4 of
> > > > course) and rebuilt the kvm-85 source kernel module and
> > > > qemu-system-x86_64 and I am now able to get dhcp and IP ops from the
> > > > 02:00.0 device on my IOH-5520 board with the KVM guest using a
> > > > v2.6.29.2 kernel!!  Everything is looking good with the v2.6.29.2,
> > > > but after a quick reboot back into my v2.6.30-rc3 KVM guest kernel
> > > > build e1000e it looks like I am unable to get dhcp.
> > > >
> > > > Rebooting back into KVM Guest kernel v2.6.29.2 brings the pci-stub
> > > > assigned e1000e 82574L assigned with dhcp and everything looks
> > > > good!  :-)
> > > >
> > > > I will keep poking at the v2.6.30-rc KVM guests (I am going to do a
> > > > complete rebuild) and see if it does not start move IP packets as
> > > > well..
> > > >
> > > > Thanks for all of your help in getting setup!
> > > >
> > > > --nab
> > > >
> > > >
> > > > --
> > > > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > > > the body of a message to majordomo@vger.kernel.org
> > > > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: KVM x86_64 with SR-IOV..? (device passthrough with LIO-Target v3.0)

[Sheng Yang]

On Wednesday 06 May 2009 01:45:47 Nicholas A. Bellinger wrote:
> On Tue, 2009-05-05 at 04:28 -0700, Nicholas A. Bellinger wrote:
> > On Tue, 2009-05-05 at 03:43 -0700, Nicholas A. Bellinger wrote:
> > > On Tue, 2009-05-05 at 09:42 +0800, Yu Zhao wrote:
> > > > Hi,
> > > >
> > > > The VF also works in the host if the VF driver is programed properly.
> > > > So it would be easier to develop the VF driver in the host and then
> > > > verify the VF driver in the guest.
> > > >
> > > > BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
> > > > the CONFIG_PCI_IOV in the kernel .config?
> > > >
> > > > Thanks,
> > > > Yu
> > >
> > > Greetings Yu and Sheng,
> > >
> > > So the original attachment was for the v2.6.29-fc11 host kernel output,
> > > I ended up jumping to v2.6.30-rc3 (and making sure CONFIG_PCI_IOV was
> > > enabled) for KVM host with kvm-85 and now things are looking quite
> > > stable for me.
> > >
> > > So far I have been able to successfully push LIO-Target v3.0 traffic
> > > *inside* a v2.6.29.2 KVM guest via the onboard e1000e (02:00.0) port
> > > from another Linux/iSCSI Initiator machine using a Intel 1 Gb/sec port.
> > > I am running badblocks tests to iSCSI Logical Units for RAMDISK_DR and
> > > FILEIO storage objects (in the KVM Guest), and they are passing
> > > validation and I am seeing ~500 Mb/sec of throughput and very low CPU
> > > usage in the KVM guests.
> >
> > Ok I am seeing another issue with the e1000e port on 02:00.0..:
> >
> > As i start to push multiple badblocks tests RAMDISK_DR iSCSI Logical
> > units into KVM Guest running LIO v2.6.29.2 from the external Linux/iSCSI
> > Initiator machine, after about 100 GB of iSCSI traffic, I see the
> > following exception in KVM host v2.6.30-rc3:
> >
> > DRHD: handling fault status reg 2
> > DMAR:[DMA Write] Request device [02:00.0] fault addr 7fc958b010000
> > DMAR:[fault reason 04] Access beyond MGAW
> > pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
> > pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
> >
> > I am able to restart the LIO-Target KVM Guest and the Linux/iSCSI
> > Initiators are able to reconnect..  Wow, very cool..
> >
> > Not sure if this is a bug in the target_core_mod RAMDISK_DR subsystem
> > plugin (mapping struct iovec to internally allocated struct page) or
> > what.  I will have to look at the DMAR code to understand what this
> > exception means..
>
> Greetings Yu, Sheng and Co,
>
> So I have been making progress this morning..  So far, I have hooked up
> a LSI mpt-function PCIe SAS adapter into the KVM guest with a Sandisk
> SATA SSD 32 GB drive.  It is using MSI interrupts (not MSI-X) and I am
> able to push ~70 MB/sec from a 2nd Linux/iSCSI Initiator machine
> (running Open-iSCSI) with the 1500 byte MTUs on e1000e ports from within
> the KVM guest.

Is MSI-X can't be enabled or the device only have MSI capability? Just 
curious...

> The interesting thing is that I am having to use IBLOCK export (using
> using submit_bio(), and complete emulation of SCSI control path) for
> SATA SSD in order to get I/O running stable  Using the pSCSI export I am
> getting immediate exceptions from scsi_execute_async() in the v2.6.29.2
> KVM guest..  

Didn't see exception in the log below... (And buried with iscsi log I can't 
understand. Looking forward for the help from others...) Any thing notable 
show in the host side? I think the target to get pSCSI work well now?

BTW: Maybe you can try the patch from Marcelo titled "[patch 0/4] use 
smp_send_reschedule in vcpu_kick / assigned dev host intx race fix".

-- 
regards
Yang, Sheng


> Using a 2nd SAS disk I am able to use target_core_mod/pSCSI
> export and push badblocks and LTP disktest traffic however..

> Here is a bit about the the setup looks,
>
> *) Linux/iSCSI Initiator node accessing KVM Guest LIO-Target v3.0
> storage:
>
> subjekt:~# lsscsi
> [6:0:0:0]    disk    ATA      ST3250820AS      3.AA  /dev/sda
> [10:0:0:0]   cd/dvd  PIONEER  DVD-ROM DVD-305  1.06  /dev/scd1
> [18:0:0:0]   cd/dvd  TOSHIBA  DVD/HD  X807616  MC08  /dev/scd2
> [32:0:0:0]   disk    LIO-ORG  RAMDISK-DR       3.0   /dev/sdb
> [32:0:0:1]   disk    LIO-ORG  RAMDISK-DR       3.0   /dev/sdc
> [32:0:0:2]   disk    LIO-ORG  FILEIO           3.0   /dev/sdd
> [32:0:0:3]   disk    LIO-ORG  IBLOCK           3.0   /dev/sde
>
> subjekt:~# sg_inq -i /dev/sde
> VPD INQUIRY: Device Identification page
>   Designation descriptor number 1, descriptor length: 20
>     id_type: NAA,  code_set: Binary
>     associated with the addressed logical unit
>       NAA 6, IEEE Company_id: 0x1405
>       Vendor Specific Identifier: 0xa97e4ce21
>       Vendor Specific Identifier Extension: 0xc0711de829b000c2
>       [0x6001405a97e4ce21c0711de829b000c2]
>   Designation descriptor number 2, descriptor length: 52
>     id_type: T10 vendor identification,  code_set: ASCII
>     associated with the addressed logical unit
>       vendor id: LIO-ORG
>       vendor specific: IBLOCK:a97e4ce21c0711de829b000c2943d57b
>   Designation descriptor number 3, descriptor length: 8
>     transport: Internet SCSI (iSCSI)
>     id_type: Relative target port,  code_set: Binary
>     associated with the target port
>       Relative target port: 0x1
>   Designation descriptor number 4, descriptor length: 8
>     transport: Internet SCSI (iSCSI)
>     id_type: Target port group,  code_set: Binary
>     associated with the target port
>       Target port group: 0x0
>   Designation descriptor number 5, descriptor length: 8
>     id_type: Logical unit group,  code_set: Binary
>     associated with the addressed logical unit
>       Logical unit group: 0x0
>   Designation descriptor number 6, descriptor length: 80
>     transport: Internet SCSI (iSCSI)
>     id_type: SCSI name string,  code_set: UTF-8
>     associated with the target port
>       SCSI name string:
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6,t,0x0001
>
> The SATA SSD is available as /dev/sde on the Linux/iSCSI Initiator node,
> the 2nd SAS disk was removed from the dmesg below as having multiple
> drives on the mpt-function card made it start to hiccup..
>
> --nab
>
> *) From with the LIO-Target KVM gest:
>
> lenny64guest1:~# lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev
> 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton
> II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE
> [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4
> ACPI (rev 03) 00:02.0 VGA compatible controller: Cirrus Logic GD 5446
> 00:03.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> RTL-8139/8139C/8139C+ (rev 20) 00:04.0 RAM memory: Qumranet, Inc. Device
> 1002
> 00:05.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> Connection 00:06.0 SCSI storage controller: LSI Logic / Symbios Logic
> SAS1068E PCI-Express Fusion-MPT SAS (rev 02)
>
>
> *) Dmesg from LIO-Target KVM guest:
>
> [    0.000000] Initializing cgroup subsys cpuset
> [    0.000000] Initializing cgroup subsys cpu
> [    0.000000] Linux version 2.6.29.2 (root@lenny64guest1) (gcc version
> 4.3.2 (Debian 4.3.2-1.1) ) #2 SMP Mon May 4 01:22:53 PDT 2009 [   
> 0.000000] Command line: root=/dev/hda1 ro
> [    0.000000] KERNEL supported cpus:
> [    0.000000]   Intel GenuineIntel
> [    0.000000]   AMD AuthenticAMD
> [    0.000000]   Centaur CentaurHauls
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
> [    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
> [    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
> [    0.000000]  BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
> [    0.000000]  BIOS-e820: 000000007fff0000 - 0000000080000000 (ACPI data)
> [    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
> [    0.000000] DMI 2.4 present.
> [    0.000000] last_pfn = 0x7fff0 max_arch_pfn = 0x100000000
> [    0.000000] init_memory_mapping: 0000000000000000-000000007fff0000
> [    0.000000]  0000000000 - 007fe00000 page 2M
> [    0.000000]  007fe00000 - 007fff0000 page 4k
> [    0.000000] kernel direct mapping tables up to 7fff0000 @ 8000-c000
> [    0.000000] last_map_addr: 7fff0000 end: 7fff0000
> [    0.000000] RAMDISK: 379c8000 - 37fef480
> [    0.000000] ACPI: RSDP 000FBE50, 0014 (r0 QEMU  )
> [    0.000000] ACPI: RSDT 7FFF0000, 002C (r1 QEMU   QEMURSDT        1 QEMU 
>       1) [    0.000000] ACPI: FACP 7FFF002C, 0074 (r1 QEMU   QEMUFACP      
>  1 QEMU        1) [    0.000000] FADT: X_PM1a_EVT_BLK.bit_width (16) does
> not match PM1_EVT_LEN (4) [    0.000000] ACPI: DSDT 7FFF0100, 2540 (r1  
> BXPC   BXDSDT        1 INTL 20061109) [    0.000000] ACPI: FACS 7FFF00C0,
> 0040
> [    0.000000] ACPI: APIC 7FFF2640, 00E0 (r1 QEMU   QEMUAPIC        1 QEMU 
>       1) [    0.000000] ACPI: Local APIC address 0xfee00000
> [    0.000000] No NUMA configuration found
> [    0.000000] Faking a node at 0000000000000000-000000007fff0000
> [    0.000000] Bootmem setup node 0 0000000000000000-000000007fff0000
> [    0.000000]   NODE_DATA [0000000000001000 - 0000000000005fff]
> [    0.000000]   bootmap [000000000000a000 -  0000000000019fff] pages 10
> [    0.000000] (6 early reservations) ==> bootmem [0000000000 - 007fff0000]
> [    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==>
> [0000000000 - 0000001000] [    0.000000]   #1 [0000006000 - 0000008000]    
>   TRAMPOLINE ==> [0000006000 - 0000008000] [    0.000000]   #2 [0000200000
> - 000070493c]    TEXT DATA BSS ==> [0000200000 - 000070493c] [    0.000000]
>   #3 [00379c8000 - 0037fef480]          RAMDISK ==> [00379c8000 -
> 0037fef480] [    0.000000]   #4 [000009fc00 - 0000100000]    BIOS reserved
> ==> [000009fc00 - 0000100000] [    0.000000]   #5 [0000008000 - 000000a000]
>          PGTABLE ==> [0000008000 - 000000a000] [    0.000000] found SMP
> MP-table at [ffff8800000fbd10] 000fbd10
> [    0.000000]  [ffffe20000000000-ffffe20001bfffff] PMD ->
> [ffff880001200000-ffff880002dfffff] on node 0 [    0.000000] Zone PFN
> ranges:
> [    0.000000]   DMA      0x00000000 -> 0x00001000
> [    0.000000]   DMA32    0x00001000 -> 0x00100000
> [    0.000000]   Normal   0x00100000 -> 0x00100000
> [    0.000000] Movable zone start PFN for each node
> [    0.000000] early_node_map[2] active PFN ranges
> [    0.000000]     0: 0x00000000 -> 0x0000009f
> [    0.000000]     0: 0x00000100 -> 0x0007fff0
> [    0.000000] On node 0 totalpages: 524175
> [    0.000000]   DMA zone: 56 pages used for memmap
> [    0.000000]   DMA zone: 1387 pages reserved
> [    0.000000]   DMA zone: 2556 pages, LIFO batch:0
> [    0.000000]   DMA32 zone: 7112 pages used for memmap
> [    0.000000]   DMA32 zone: 513064 pages, LIFO batch:31
> [    0.000000] ACPI: PM-Timer IO Port: 0xb008
> [    0.000000] ACPI: Local APIC address 0xfee00000
> [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] disabled)
> [    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> [    0.000000] IOAPIC[0]: apic_id 2, version 0, address 0xfec00000, GSI
> 0-23 [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high
> level) [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high
> level) [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10
> high level) [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq
> 11 high level) [    0.000000] ACPI: IRQ5 used by override.
> [    0.000000] ACPI: IRQ9 used by override.
> [    0.000000] ACPI: IRQ10 used by override.
> [    0.000000] ACPI: IRQ11 used by override.
> [    0.000000] Using ACPI (MADT) for SMP configuration information
> [    0.000000] SMP: Allowing 16 CPUs, 14 hotplug CPUs
> [    0.000000] nr_irqs_gsi: 24
> [    0.000000] PM: Registered nosave memory: 000000000009f000 -
> 00000000000a0000 [    0.000000] PM: Registered nosave memory:
> 00000000000a0000 - 00000000000e8000 [    0.000000] PM: Registered nosave
> memory: 00000000000e8000 - 0000000000100000 [    0.000000] Allocating PCI
> resources starting at 88000000 (gap: 80000000:7ffbc000) [    0.000000]
> NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:16 nr_node_ids:1 [    0.000000]
> PERCPU: Allocating 49152 bytes of per cpu data
> [    0.000000] Built 1 zonelists in Node order, mobility grouping on. 
> Total pages: 515620 [    0.000000] Policy zone: DMA32
> [    0.000000] Kernel command line: root=/dev/hda1 ro
> [    0.000000] Initializing CPU#0
> [    0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes)
> [    0.000000] Fast TSC calibration using PIT
> [    0.000000] Detected 2266.815 MHz processor.
> [    0.004000] Console: colour VGA+ 80x25
> [    0.004000] console [tty0] enabled
> [    0.004000] Checking aperture...
> [    0.004000] No AGP bridge found
> [    0.004000] Calgary: detecting Calgary via BIOS EBDA area
> [    0.004000] Calgary: Unable to locate Rio Grande table in EBDA -
> bailing! [    0.004000] Memory: 2055644k/2097088k available (2463k kernel
> code, 388k absent, 41056k reserved, 1259k data, 464k init) [    0.004013]
> Calibrating delay loop (skipped), value calculated using timer frequency..
> 4533.63 BogoMIPS (lpj=9067260) [    0.004985] Security Framework
> initialized
> [    0.005422] SELinux:  Disabled at boot.
> [    0.005995] Dentry cache hash table entries: 262144 (order: 9, 2097152
> bytes) [    0.007887] Inode-cache hash table entries: 131072 (order: 8,
> 1048576 bytes) [    0.008603] Mount-cache hash table entries: 256
> [    0.009309] Initializing cgroup subsys ns
> [    0.009739] Initializing cgroup subsys cpuacct
> [    0.010172] Initializing cgroup subsys devices
> [    0.010655] CPU: L1 I cache: 32K, L1 D cache: 32K
> [    0.011229] CPU: L2 cache: 2048K
> [    0.011633] CPU 0/0x0 -> Node 0
> [    0.012061] ACPI: Core revision 20081204
> [    0.013529] Setting APIC routing to flat
> [    0.015117] ..TIMER: vector=0x30 apic1=0 pin1=0 apic2=-1 pin2=-1
> [    0.055398] CPU0: Intel QEMU Virtual CPU version 0.10.0 stepping 03
> [    0.273970] Booting processor 1 APIC 0x1 ip 0x6000
> [    0.004000] Initializing CPU#1
> [    0.004000] Calibrating delay using timer specific routine.. 4533.28
> BogoMIPS (lpj=9066565) [    0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
> [    0.004000] CPU: L2 cache: 2048K
> [    0.004000] CPU 1/0x1 -> Node 0
> [    0.364147] CPU1: Intel QEMU Virtual CPU version 0.10.0 stepping 03
> [    0.367429] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
> [    0.372036] Brought up 2 CPUs
> [    0.372442] Total of 2 processors activated (9066.91 BogoMIPS).
> [    0.373176] CPU0 attaching sched-domain:
> [    0.373181]  domain 0: span 0-1 level CPU
> [    0.373183]   groups: 0 1
> [    0.373186] CPU1 attaching sched-domain:
> [    0.373187]  domain 0: span 0-1 level CPU
> [    0.373188]   groups: 1 0
> [    0.373255] net_namespace: 1888 bytes
> [    0.373255] NET: Registered protocol family 16
> [    0.373269] ACPI: bus type pci registered
> [    0.376082] PCI: Using configuration type 1 for base access
> [    0.380281] bio: create slab <bio-0> at 0
> [    0.381293] ACPI: EC: Look up EC in DSDT
> [    0.383292] ACPI: Interpreter enabled
> [    0.383693] ACPI: (supports S0 S3 S4 S5)
> [    0.384763] ACPI: Using IOAPIC for interrupt routing
> [    0.388691] ACPI: No dock devices found.
> [    0.388729] ACPI: PCI Root Bridge [PCI0] (0000:00)
> [    0.393074] pci 0000:00:01.1: reg 20 io port: [0xc000-0xc00f]
> [    0.393406] pci 0000:00:01.3: quirk: region b000-b03f claimed by PIIX4
> ACPI [    0.393920] pci 0000:00:01.3: quirk: region b100-b10f claimed by
> PIIX4 SMB [    0.395842] pci 0000:00:02.0: reg 10 32bit mmio:
> [0xf0000000-0xf1ffffff] [    0.396508] pci 0000:00:02.0: reg 14 32bit mmio:
> [0xf2000000-0xf2000fff] [    0.401611] pci 0000:00:03.0: reg 10 io port:
> [0xc100-0xc1ff]
> [    0.401674] pci 0000:00:03.0: reg 14 32bit mmio: [0xf2001000-0xf20010ff]
> [    0.401903] pci 0000:00:04.0: reg 10 io port: [0xc200-0xc21f]
> [    0.403518] pci 0000:00:05.0: reg 10 32bit mmio: [0xf2020000-0xf203ffff]
> [    0.404059] pci 0000:00:05.0: reg 18 io port: [0xc220-0xc23f]
> [    0.404368] pci 0000:00:05.0: reg 1c 32bit mmio: [0xf2040000-0xf2043fff]
> [    0.405176] pci 0000:00:06.0: reg 10 io port: [0xc300-0xc3ff]
> [    0.405474] pci 0000:00:06.0: reg 14 32bit mmio: [0xf2044000-0xf2047fff]
> [    0.405804] pci 0000:00:06.0: reg 1c 32bit mmio: [0xf2050000-0xf205ffff]
> [    0.405948] pci 0000:00:06.0: reg 30 32bit mmio: [0x200000-0x3fffff]
> [    0.406577] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
> [    0.408281] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
> [    0.412098] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
> [    0.413393] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
> [    0.414571] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
> [    0.416142] usbcore: registered new interface driver usbfs
> [    0.416504] usbcore: registered new interface driver hub
> [    0.416504] usbcore: registered new device driver usb
> [    0.416508] PCI: Using ACPI for IRQ routing
> [    0.452044] pnp: PnP ACPI init
> [    0.452515] ACPI: bus type pnp registered
> [    0.454439] pnp: PnP ACPI: found 7 devices
> [    0.454918] ACPI: ACPI bus type pnp unregistered
> [    0.460804] pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
> [    0.460807] pci_bus 0000:00: resource 1 mem:
> [0x000000-0xffffffffffffffff] [    0.460969] NET: Registered protocol
> family 2
> [    0.508246] IP route cache hash table entries: 65536 (order: 7, 524288
> bytes) [    0.537526] TCP established hash table entries: 262144 (order:
> 10, 4194304 bytes) [    0.542048] TCP bind hash table entries: 65536
> (order: 8, 1048576 bytes) [    0.543577] TCP: Hash tables configured
> (established 262144 bind 65536) [    0.544332] TCP reno registered
> [    0.556665] NET: Registered protocol family 1
> [    0.557476] checking if image is initramfs... it is
> [    0.980042] Switched to high resolution mode on CPU 1
> [    0.983993] Switched to high resolution mode on CPU 0
> [    1.028987] Freeing initrd memory: 6301k freed
> [    1.032241] audit: initializing netlink socket (disabled)
> [    1.032745] type=2000 audit(1241543237.032:1): initialized
> [    1.035249] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [    1.036055] VFS: Disk quotas dquot_6.5.2
> [    1.036182] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> [    1.036182] msgmni has been set to 4027
> [    1.036182] alg: No test for stdrng (krng)
> [    1.036182] Block layer SCSI generic (bsg) driver version 0.4 loaded
> (major 253) [    1.036182] io scheduler noop registered
> [    1.036182] io scheduler anticipatory registered
> [    1.036182] io scheduler deadline registered
> [    1.040223] io scheduler cfq registered (default)
> [    1.040692] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
> [    1.041194] pci 0000:00:01.0: PIIX3: Enabling Passive Release
> [    1.041681] pci 0000:00:01.0: Activating ISA DMA hang workarounds
> [    1.042183] pci 0000:00:02.0: Boot video device
> [    1.048876] Linux agpgart interface v0.103
> [    1.049315] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [    1.050082] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> [    1.051351] 00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> [    1.054962] brd: module loaded
> [    1.055662] PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64
> irq 1,12 [    1.111469] serio: i8042 KBD port at 0x60,0x64 irq 1
> [    1.112044] serio: i8042 AUX port at 0x60,0x64 irq 12
> [    1.112901] mice: PS/2 mouse device common for all mice
> [    1.114216] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
> [    1.114828] rtc0: alarms up to one day, 114 bytes nvram
> [    1.115363] input: AT Translated Set 2 keyboard as /class/input/input0
> [    1.115934] cpuidle: using governor ladder
> [    1.116448] cpuidle: using governor menu
> [    1.116926] No iBFT detected.
> [    1.117634] TCP cubic registered
> [    1.118088] NET: Registered protocol family 17
> [    1.118873] registered taskstats version 1
> [    1.119543] rtc_cmos 00:01: setting system clock to 2009-05-05 17:07:17
> UTC (1241543237) [    1.120418] Freeing unused kernel memory: 464k freed
> [    1.193327] processor ACPI_CPU:00: registered as cooling_device0
> [    1.194273] processor ACPI_CPU:01: registered as cooling_device1
> [    1.392735] Uniform Multi-Platform E-IDE driver
> [    1.413020] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
> [    1.413531] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11
> (level, high) -> IRQ 11 [    1.415182] 8139too Fast Ethernet driver 0.9.28
> [    1.415673] 8139too 0000:00:03.0: This (id 10ec:8139 rev 20) is an
> enhanced 8139C+ chip, use 8139cp [    1.418181] e1000e: Intel(R) PRO/1000
> Network Driver - 0.3.3.3-k6 [    1.418680] e1000e: Copyright (c) 1999-2008
> Intel Corporation.
> [    1.419758] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> [    1.420328] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> (level, high) -> IRQ 10 [    1.421130] e1000e 0000:00:05.0: setting latency
> timer to 64
> [    1.421963] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> [    1.421999] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> [    1.422033] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> [    1.431733] SCSI subsystem initialized
> [    1.445624] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
> [    1.453420] Fusion MPT base driver 3.04.07
> [    1.453849] Copyright (c) 1999-2008 LSI Corporation
> [    1.461332] Fusion MPT SAS Host driver 3.04.07
> [    1.464878] FDC 0 is a S82078B
> [    1.549697] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> 00:e0:81:c0:90:b2 [    1.550752] 0000:00:05.0: eth0: Intel(R) PRO/1000
> Network Connection [    1.551549] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA
> No: ffffff-0ff [    1.553336] ACPI: PCI Interrupt Link [LNKC] enabled at
> IRQ 11
> [    1.553940] udev: renamed network interface eth0 to eth1
> [    1.554683] 8139cp 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11
> (level, high) -> IRQ 11 [    1.557146] eth0: RTL-8139C+ at
> 0xffffc2000096a000, 52:54:00:12:34:56, IRQ 11 [    1.560485] 8139cp
> 0000:00:03.0: setting latency timer to 64
> [    1.560904] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
> [    1.561607] mptsas 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 10
> (level, high) -> IRQ 10 [    1.563379] mptbase: ioc0: Initiating bringup
> [    1.634659] mptbase: ioc0: WARNING - Unexpected doorbell active!
> [    5.343913] ioc0: LSISAS1068E B1: Capabilities={Initiator}
> [    5.345428] mptsas 0000:00:06.0: setting latency timer to 64
> [   22.874565] scsi0 : ioc0: LSISAS1068E B1, FwRev=010f0000h, Ports=1,
> MaxQ=511, IRQ=10 [   22.903281] scsi 0:0:0:0: Direct-Access     ATA     
> SanDisk SSD SATA 1.20 PQ: 0 ANSI: 5 [   22.906530] piix 0000:00:01.1: IDE
> controller (0x8086:0x7010 rev 0x00) [   22.908623] piix 0000:00:01.1: not
> 100% native mode: will probe irqs later [   22.909390] PIIX_IDE
> 0000:00:01.1: setting latency timer to 64
> [   22.909410]     ide0: BM-DMA at 0xc000-0xc007
> [   22.910045]     ide1: BM-DMA at 0xc008-0xc00f
> [   22.910666] Probing IDE interface ide0...
> [   22.933264] Driver 'sd' needs updating - please use bus_type methods
> [   22.936186] sd 0:0:0:0: [sda] 62533296 512-byte hardware sectors: (32.0
> GB/29.8 GiB) [   22.938165] sd 0:0:0:0: [sda] Write Protect is off
> [   22.938808] sd 0:0:0:0: [sda] Mode Sense: 73 00 00 08
> [   22.940775] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled,
> doesn't support DPO or FUA [   22.942194] sd 0:0:0:0: [sda] 62533296
> 512-byte hardware sectors: (32.0 GB/29.8 GiB) [   22.944225] sd 0:0:0:0:
> [sda] Write Protect is off
> [   22.944879] sd 0:0:0:0: [sda] Mode Sense: 73 00 00 08
> [   22.946734] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled,
> doesn't support DPO or FUA [   22.947886]  sda: unknown partition table
> [   22.951811] sd 0:0:0:0: [sda] Attached SCSI disk
> [   23.211126] hda: QEMU HARDDISK, ATA DISK drive
> [   23.883187] hda: host max PIO4 wanted PIO255(auto-tune) selected PIO2
> [   23.883289] hda: MWDMA2 mode selected
> [   23.884023] Probing IDE interface ide1...
> [   24.283042] hdc: QEMU DVD-ROM, ATAPI CD/DVD-ROM drive
> [   24.955145] hdc: host max PIO4 wanted PIO255(auto-tune) selected PIO0
> [   24.955199] hdc: MWDMA2 mode selected
> [   24.955978] ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
> [   24.956812] isa bounce pool size: 16 pages
> [   24.957471] ide1 at 0x170-0x177,0x376 on irq 15
> [   24.974924] libata version 3.00 loaded.
> [   24.992633] ide-gd driver 1.18
> [   24.993261] hda: max request size: 512KiB
> [   24.993874] hda: 16384000 sectors (8388 MB) w/256KiB Cache,
> CHS=16253/255/63 [   25.000639] ide-cd driver 5.00
> [   25.007111] hda: cache flushes supported
> [   25.007753]  hda: hda1 hda2 < hda5 >
> [   25.010488] ide-cd: hdc: ATAPI 4X CD-ROM drive, 512kB Cache
> [   25.011858] Uniform CD-ROM driver Revision: 3.20
> [   25.110990] PM: Starting manual resume from disk
> [   25.129340] EXT3-fs: INFO: recovery required on readonly filesystem.
> [   25.130052] EXT3-fs: write access will be enabled during recovery.
> [   26.128769] kjournald starting.  Commit interval 5 seconds
> [   26.129481] EXT3-fs: recovery complete.
> [   26.130963] EXT3-fs: mounted filesystem with ordered data mode.
> [   26.335166] udevd version 125 started
> [   26.631975] input: Power Button (FF) as /class/input/input1
> [   26.666993] ACPI: Power Button (FF) [PWRF]
> [   26.681253] parport_pc 00:05: reported by Plug and Play ACPI
> [   26.681824] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
> [   26.723397] input: PC Speaker as /class/input/input2
> [   26.724903] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100,
> revision 0 [   26.949233] input: ImExPS/2 Generic Explorer Mouse as
> /class/input/input3 [   58.566336] Adding 393552k swap on /dev/hda5. 
> Priority:-1 extents:1 across:393552k [   58.594239] EXT3 FS on hda1,
> internal journal
> [   59.613114] loop: module loaded
> [   60.681886] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
> [   63.746408] NET: Registered protocol family 10
> [   63.748013] lo: Disabled Privacy Extensions
> [   73.968880] eth0: no IPv6 routers present
> [  233.617434] ADDRCONF(NETDEV_UP): eth1_rename: link is not ready
> [  236.587594] e1000e: eth1_rename NIC Link is Up 1000 Mbps Full Duplex,
> Flow Control: RX/TX [  236.595715] ADDRCONF(NETDEV_CHANGE): eth1_rename:
> link becomes ready [  243.146435] TARGET_CORE[0]: Loading Generic Kernel
> Storage Engine: v3.0.0 on Linux/x86_64 on 2.6.29.2 [  243.147955]
> TARGET_CORE[0]: Initialized ConfigFS Fabric Infrastructure: v1.0.0 on
> Linux/x86_64 on 2.6.29.2 [  243.149107] SE_PC[0] - Registered Plugin Class:
> TRANSPORT
> [  243.149830] PLUGIN_TRANSPORT[1] - pscsi registered
> [  243.150492] PLUGIN_TRANSPORT[2] - stgt registered
> [  243.151183] CORE_STGT[0]: Bus Initalization complete
> [  243.151838] PLUGIN_TRANSPORT[4] - iblock registered
> [  243.152483] PLUGIN_TRANSPORT[5] - rd_dr registered
> [  243.153126] PLUGIN_TRANSPORT[6] - rd_mcp registered
> [  243.155097] PLUGIN_TRANSPORT[7] - fileio registered
> [  243.155743] SE_PC[1] - Registered Plugin Class: OBJ
> [  243.156372] PLUGIN_OBJ[1] - dev registered
> [  243.160265] Target_Core_ConfigFS: Located se_plugin: ffff88007c4318e0
> plugin_name: iblock hba_type: 4 plugin_dep_id: 0 [  243.161454] CORE_HBA[0]
> - Linux-iSCSI.org iBlock HBA Driver 3.0 on Generic Target Core Stack v3.0 [
>  243.162630] CORE_HBA[0] - Attached iBlock HBA: 0 to Generic Target Core
> TCQ Depth: 512 [  243.163681] CORE_HBA[0] - Attached HBA to Generic Target
> Core
> [  243.164496] IBLOCK: Allocated ib_dev for ssd_test0
> [  243.165152] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> ffff88007c022000 se_dev_su_ptr: ffff88007c444c00 [  243.166753]
> Target_Core_ConfigFS: Set emulated VPD Unit Serial:
> a97e4ce21c0711de829b000c2943d57b [  243.167947] IBLOCK: Referencing Major:
> 8
> [  243.168541] IBLOCK: Referencing Minor: 0
> [  243.169225] IBLOCK: Claiming ffff88007c444c00 Major:Minor - 8:0
> [  243.170021] bio: create slab <bio-1> at 1
> [  243.170656] IBLOCK: Created bio_set() for major/minor: 8:0
> [  243.171325] iblock: Using SPC3_PERSISTENT_RESERVATIONS emulation
> [  243.172012] iblock: Enabling ALUA Emulation for SPC-3 device
> [  243.172696] iblock: Adding to default ALUA LU Group:
> core/alua/lu_gps/default_lu_gp [  243.173802] CORE_iBLOCK[0] - Activating
> Device with TCQ: 0 at Major: 8 Minor 0 [  243.175345]   Vendor: LIO-ORG  
> Model: IBLOCK            Revision: 3.0 [  243.181019]   Type:  
> Direct-Access                      ANSI SCSI revision: 05 [  243.182592]
> T10 VPD Unit Serial Number: a97e4ce21c0711de829b000c2943d57b [  243.183397]
> T10 VPD Page Length: 72
> [  243.183983] T10 VPD Identifer Length: 16
> [  243.184588] T10 VPD Identifier Association: addressed logical unit
> [  243.185286] T10 VPD Identifier Type: NAA
> [  243.186965] T10 VPD Binary Device Identifier:
> 36001405a97e4ce21c0711de829b000c2 [  243.187993] T10 VPD Identifer Length:
> 48
> [  243.188602] T10 VPD Identifier Association: addressed logical unit
> [  243.189303] T10 VPD Identifier Type: T10 Vendor ID based
> [  243.190037] T10 VPD ASCII Device Identifier: LIO-ORG
> [  243.191029] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr:
> ffff88007c5d1400 [  243.193246] Target_Core_ConfigFS: Located se_plugin:
> ffff88007c431838 plugin_name: pscsi hba_type: 1 plugin_dep_id: 0 [ 
> 243.194482] CORE_HBA[1] - Linux-iSCSI.org SCSI HBA Driver v3.0 on Generic
> Target Core Stack v3.0 [  243.195580] CORE_HBA[1] - MPT SPI Host
> [  243.196170] CORE_HBA[1] - Attached SCSI HBA to Generic Target Core with
> TCQ Depth: 127 MaxSectors: 8192 [  243.197292] CORE_HBA[1] - Attached HBA
> to Generic Target Core
> [  243.198125] PSCSI: Allocated pdv: ffff88007c42aa00 for sda
> [  243.198803] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> ffff88007c582000 se_dev_su_ptr: ffff88007c42aa00 [  243.201132] PSCSI:
> Allocated pdv: ffff88007d064140 for sdb
> [  243.201952] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> ffff88007c1d9800 se_dev_su_ptr: ffff88007d064140 [  243.203550] PSCSI[0]:
> Referencing SCSI Channel ID: 0
> [  243.204200] PSCSI[0]: Referencing SCSI Target ID: 1
> [  243.204841] PSCSI[0]: Referencing SCSI LUN ID: 0
> [  243.205550] Unable to locate 0:0:1:0
> [  243.208465] Target_Core_ConfigFS: Located se_plugin: ffff88007c431918
> plugin_name: rd_dr hba_type: 5 plugin_dep_id: 0 [  243.209674] CORE_HBA[2]
> - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
> [  243.210802] CORE_HBA[2] - Attached Ramdisk HBA: 0 to Generic Target Core
> TCQ Depth: 256 MaxSectors: 1024 [  243.211942] CORE_HBA[2] - Attached HBA
> to Generic Target Core
> [  243.212774] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> ffff88007c59c800 se_dev_su_ptr: ffff88007c4374c0 [  243.214315] RAMDISK:
> Referencing Page Count: 32768
> [  243.224608] CORE_RD[0] - Built Ramdisk Device ID: 0 space of 32768 pages
> in 17 tables [  243.225795] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS
> emulation [  243.226490] rd_dr: Enabling ALUA Emulation for SPC-3 device
> [  243.227165] rd_dr: Adding to default ALUA LU Group:
> core/alua/lu_gps/default_lu_gp [  243.228212] CORE_RD[0] - Activating
> Device with TCQ: 0 at Ramdisk Device ID: 0 [  243.229792]   Vendor: LIO-ORG
>   Model: RAMDISK-DR        Revision: 3.0 [  243.235413]   Type:  
> Direct-Access                      ANSI SCSI revision: 05 [  243.236861]
> T10 VPD Unit Serial Number: 1234567890:2_0
> [  243.237599] T10 VPD Page Length: 38
> [  243.238220] T10 VPD Identifer Length: 34
> [  243.238860] T10 VPD Identifier Association: addressed logical unit
> [  243.239540] T10 VPD Identifier Type: T10 Vendor ID based
> [  243.240190] T10 VPD ASCII Device Identifier: LIO-ORG
> [  243.241083] CORE_RD[0] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768
> pages in 17 tables, 134217728 total bytes [  243.242336]
> Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff88007c5d1000 [ 
> 243.244650] Target_Core_ConfigFS: Located se_plugin: ffff88007c431918
> plugin_name: rd_dr hba_type: 5 plugin_dep_id: 1 [  243.245895] CORE_HBA[3]
> - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
> [  243.247006] CORE_HBA[3] - Attached Ramdisk HBA: 1 to Generic Target Core
> TCQ Depth: 256 MaxSectors: 1024 [  243.248131] CORE_HBA[3] - Attached HBA
> to Generic Target Core
> [  243.248838] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> ffff88007c18c000 se_dev_su_ptr: ffff88007c973940 [  243.252447] RAMDISK:
> Referencing Page Count: 32768
> [  243.262801] CORE_RD[1] - Built Ramdisk Device ID: 0 space of 32768 pages
> in 17 tables [  243.263870] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS
> emulation [  243.264555] rd_dr: Enabling ALUA Emulation for SPC-3 device
> [  243.265231] rd_dr: Adding to default ALUA LU Group:
> core/alua/lu_gps/default_lu_gp [  243.266342] CORE_RD[1] - Activating
> Device with TCQ: 0 at Ramdisk Device ID: 0 [  243.267625]   Vendor: LIO-ORG
>   Model: RAMDISK-DR        Revision: 3.0 [  243.273342]   Type:  
> Direct-Access                      ANSI SCSI revision: 05 [  243.274807]
> T10 VPD Unit Serial Number: 1234567890:3_0
> [  243.275527] T10 VPD Page Length: 38
> [  243.276125] T10 VPD Identifer Length: 34
> [  243.276734] T10 VPD Identifier Association: addressed logical unit
> [  243.277439] T10 VPD Identifier Type: T10 Vendor ID based
> [  243.278173] T10 VPD ASCII Device Identifier: LIO-ORG
> [  243.278891] CORE_RD[1] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768
> pages in 17 tables, 134217728 total bytes [  243.280067]
> Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff880037943400 [ 
> 243.282480] Target_Core_ConfigFS: Located se_plugin: ffff88007c431988
> plugin_name: fileio hba_type: 7 plugin_dep_id: 0 [  243.320107] CORE_HBA[4]
> - Linux-iSCSI.org FILEIO HBA Driver 3.0 on Generic Target Core Stack v3.0 [
>  243.321211] CORE_HBA[4] - Attached FILEIO HBA: 0 to Generic Target Core
> with TCQ Depth: 256 MaxSectors: 1024 [  243.322383] CORE_HBA[4] - Attached
> HBA to Generic Target Core
> [  243.323284] FILEIO: Allocated fd_dev for ffff88007bd61ee0
> [  243.323964] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> ffff88007d1b6000 se_dev_su_ptr: ffff88007c444000 [  243.325456]
> Target_Core_ConfigFS: Set emulated VPD Unit Serial:
> 66e418b4e9ab11dd8f51000c2943d57b [  243.340857] FILEIO: Referencing Path:
> /tmp/fileio
> [  243.341504] FILEIO: Referencing Size: 134217728 bytes
> [  243.342298] fileio: Using SPC3_PERSISTENT_RESERVATIONS emulation
> [  243.342998] fileio: Enabling ALUA Emulation for SPC-3 device
> [  243.343668] fileio: Adding to default ALUA LU Group:
> core/alua/lu_gps/default_lu_gp [  243.344706] CORE_FILE[0] - Activating
> Device with TCQ: 0 at FILEIO Device ID: 0 [  243.346260]   Vendor: LIO-ORG 
>  Model: FILEIO            Revision: 3.0 [  243.351966]   Type:  
> Direct-Access                      ANSI SCSI revision: 05 [  243.353392]
> T10 VPD Unit Serial Number: 66e418b4e9ab11dd8f51000c2943d57b [  243.354363]
> T10 VPD Page Length: 72
> [  243.354961] T10 VPD Identifer Length: 16
> [  243.355569] T10 VPD Identifier Association: addressed logical unit
> [  243.356304] T10 VPD Identifier Type: NAA
> [  243.356904] T10 VPD Binary Device Identifier:
> 3600140566e418b4e9ab11dd8f51000c2 [  243.358048] T10 VPD Identifer Length:
> 48
> [  243.358649] T10 VPD Identifier Association: addressed logical unit
> [  243.359351] T10 VPD Identifier Type: T10 Vendor ID based
> [  243.360007] T10 VPD ASCII Device Identifier: LIO-ORG
> [  243.360730] CORE_FILE[0] - Added LIO FILEIO Device ID: 0 at /tmp/fileio,
> 134217728 total bytes [  243.361913] Target_Core_ConfigFS: Registered
> se_dev->se_dev_ptr: ffff880037943c00 [  243.364168] Target_Core_ConfigFS:
> REGISTER -> group: ffffffffa02a0180 name: iscsi [  243.446323]
> Linux-iSCSI.org iSCSI Target Core Stack v3.0.0 on Linux/x86_64 on 2.6.29.2
> [  243.447405] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API
> >>>>>>>>>>>>>>>>>>>>>> [  243.448132] Initialized struct
> target_fabric_configfs: ffff88006b554000 for iscsi [  243.449150]
> <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>> [  243.449919]
> LIO_TARGET[0] - Set fabric -> lio_target_fabric_configfs [  243.451519]
> iscsi_allocate_thread_sets:195: ***OPS*** Spawned 4 thread set(s) (8 total
> threads). [  243.452909] TARGET_CORE[iSCSI]: Allocated Discovery
> se_portal_group_t for endpoint: None, Portal Tag: 0 [  243.454075] CORE[0]
> - Allocated Discovery TPG
> [  243.454698] Loading Complete.
> [  243.455569] Target_Core_ConfigFS: REGISTER -> Located fabric: iscsi
> [  243.456279] Target_Core_ConfigFS: REGISTER -> ffffffffa02ddbb0
> [  243.456959] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: iscsi
> [  243.457695] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for
> iscsi [  243.458610] lio_target_call_coreaddtiqn(): name:
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 243.459839] CORE[0] - Added iSCSI Target IQN:
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 243.461031] LIO_Target_ConfigFS: REGISTER ->
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 243.462213] LIO_Target_ConfigFS: REGISTER -> Allocated Node:
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 243.463621] lio_target_tiqn_addtpg() called: name tpgt_1
> [  243.464283] lio_target_tiqn_addtpg() parent name:
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 243.465439] lio_target_tiqn_addtpg() Using TPGT: 1
> [  243.466206] TARGET_CORE[iSCSI]: Allocated Normal se_portal_group_t for
> endpoint: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6,
> Portal Tag: 1 [  243.467824]
> CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[
>1] - Added iSCSI Target Portal Group [  243.469018] LIO_Target_ConfigFS:
> REGISTER ->
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 243.536626] LIO_Target_ConfigFS: REGISTER -> Allocated TPG: tpgt_1 [ 
> 243.537704] LIO_Target_ConfigFS: REGISTER ->
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> PORTAL: 192.168.0.117:3260 [  243.539129] CORE[0] - Added Network Portal:
> 192.168.0.117:3260 on TCP on network device: None [  243.540238]
> CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] -
> Added Network Portal: 192.168.0.117:3260,1 on TCP on network device: None [
>  243.541925]
> CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[
>1] - Incremented np_exports to 1 [  243.543104] LIO_Target_ConfigFS:
> addnptotpg done!
> [  243.649897] SCTP: Hash tables configured (established 65536 bind 65536)
> [  243.704216] CORE[0] - Added Network Portal: 192.168.0.117:3260 on SCTP
> on network device: None [  243.733493]
> CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] -
> Added Network Portal: 192.168.0.117:3260,1 on SCTP on network device: None
> [  243.735142]
> CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[
>1] - Incremented np_exports to 1 [  243.737456] LIO_Target_ConfigFS:
> REGISTER ->
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> LUN: 0 [  243.739831] rd_dr/iSCSI: Adding to default ALUA Target Port
> Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.740953]
> iSCSI_TPG[1]_LUN[0] - Activated iSCSI Logical Unit from CORE HBA: 2 [ 
> 243.742029] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_0 [ 
> 243.744038] LIO_Target_ConfigFS: REGISTER ->
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> LUN: 1 [  243.746304] rd_dr/iSCSI: Adding to default ALUA Target Port
> Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.747436]
> iSCSI_TPG[1]_LUN[1] - Activated iSCSI Logical Unit from CORE HBA: 3 [ 
> 243.748468] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_1 [ 
> 243.750509] LIO_Target_ConfigFS: REGISTER ->
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> LUN: 2 [  243.752754] fileio/iSCSI: Adding to default ALUA Target Port
> Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.753924]
> iSCSI_TPG[1]_LUN[2] - Activated iSCSI Logical Unit from CORE HBA: 4 [ 
> 243.754957] LIO_Target_ConfigFS: Created Port Symlink fileio -> lun_2 [ 
> 243.757089] LIO_Target_ConfigFS: REGISTER ->
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> LUN: 3 [  243.759370] iblock/iSCSI: Adding to default ALUA Target Port
> Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.760507]
> iSCSI_TPG[1]_LUN[3] - Activated iSCSI Logical Unit from CORE HBA: 0 [ 
> 243.761525] LIO_Target_ConfigFS: Created Port Symlink ssd_test0 -> lun_3 [ 
> 243.762761] Disabling iSCSI Authentication Methods for TPG: 1.
> [  243.763566] iSCSI_TPG[1] - Generate Initiator Portal Group ACLs: Enabled
> [  243.764414] iSCSI_TPG[1] - Demo Mode Write Protect bit: OFF
> [  243.765196] iSCSI_TPG[1] - Set Default CmdSN TCQ Depth to 4
> [  243.832368] iSCSI_TPG[1] - Enabled iSCSI Target Portal Group
> [  244.173472] Received iSCSI login request from 192.168.0.116 on TCP
> Network Portal 192.168.0.117:3260 [  244.174664] Located Storage Object:
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 244.175776] Located Portal Group Object: 1
> [  244.176405] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access
> for LUN in Demo Mode [  244.177423] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] -
> Adding READ-WRITE access for LUN in Demo Mode [  244.178656]
> TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in
> Demo Mode [  244.179756] TARGET_CORE[iSCSI]->TPG[1]_LUN[3] - Adding
> READ-WRITE access for LUN in Demo Mode [  244.180854] iSCSI_TPG[1] - Added
> DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node:
> iqn.1993-08.org.debian:01.2fed72c26a4c [  244.182213]
> ------------------------------------------------------------------ [ 
> 244.183240] HeaderDigest:                 None
> [  244.183860] DataDigest:                   None
> [  244.184479] MaxRecvDataSegmentLength:     131072
> [  244.185099] IFMarker:                     No
> [  244.185779] OFMarker:                     No
> [  244.186394]
> ------------------------------------------------------------------ [ 
> 244.187422]
> ------------------------------------------------------------------ [ 
> 244.188435] MaxConnections:               1
> [  244.189056] TargetName:                  
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 244.191890] InitiatorName:               
> iqn.1993-08.org.debian:01.2fed72c26a4c [  244.192934] TargetAlias:         
>         LIO Target
> [  244.193622] InitiatorAlias:               subjekt
> [  244.194272] TargetPortalGroupTag:         1
> [  244.194878] InitialR2T:                   Yes
> [  244.195485] ImmediateData:                Yes
> [  244.196107] MaxBurstLength:               262144
> [  244.196741] FirstBurstLength:             65536
> [  244.197356] DefaultTime2Wait:             2
> [  244.198038] DefaultTime2Retain:           0
> [  244.198648] MaxOutstandingR2T:            1
> [  244.199266] DataPDUInOrder:               Yes
> [  244.199883] DataSequenceInOrder:          Yes
> [  244.200499] ErrorRecoveryLevel:           0
> [  244.201115] SessionType:                  Normal
> [  244.201812]
> ------------------------------------------------------------------ [ 
> 244.202838] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr:
> ffff88006b597e00 [  244.203564] iSCSI Login successful on CID: 0 from
> 192.168.0.116 to 192.168.0.117:3260,1 [  244.204621] Incremented iSCSI
> Connection count to 1 from node: iqn.1993-08.org.debian:01.2fed72c26a4c [ 
> 244.205800] Established iSCSI session from node:
> iqn.1993-08.org.debian:01.2fed72c26a4c [  244.238865] Incremented number of
> active iSCSI sessions to 1 on iSCSI Target Portal Group: 1 [  247.041588]
> eth1_rename: no IPv6 routers present
> [  248.739907] Received iSCSI login request from 192.168.0.111 on TCP
> Network Portal 192.168.0.117:3260 [  248.742799] Located Storage Object:
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 248.743981] Located Portal Group Object: 1
> [  248.744635] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access
> for LUN in Demo Mode [  248.745818] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] -
> Adding READ-WRITE access for LUN in Demo Mode [  248.746927]
> TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in
> Demo Mode [  248.748017] TARGET_CORE[iSCSI]->TPG[1]_LUN[3] - Adding
> READ-WRITE access for LUN in Demo Mode [  248.749122] iSCSI_TPG[1] - Added
> DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node:
> iqn.1993-08.org.debian:01:2dadf92d0ef [  248.750565]
> ------------------------------------------------------------------ [ 
> 248.751597] HeaderDigest:                 None
> [  248.752230] DataDigest:                   None
> [  248.752861] MaxRecvDataSegmentLength:     131072
> [  248.753543] IFMarker:                     No
> [  248.754171] OFMarker:                     No
> [  248.754785]
> ------------------------------------------------------------------ [ 
> 248.755822]
> ------------------------------------------------------------------ [ 
> 248.756853] MaxConnections:               1
> [  248.757510] TargetName:                  
> iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> 248.758690] InitiatorName:               
> iqn.1993-08.org.debian:01:2dadf92d0ef [  248.759742] TargetAlias:          
>        LIO Target
> [  248.760397] InitiatorAlias:               ubuntu
> [  248.761034] TargetPortalGroupTag:         1
> [  248.761731] InitialR2T:                   Yes
> [  248.762352] ImmediateData:                Yes
> [  248.762972] MaxBurstLength:               262144
> [  248.763604] FirstBurstLength:             65536
> [  248.764233] DefaultTime2Wait:             2
> [  248.764844] DefaultTime2Retain:           0
> [  248.765453] MaxOutstandingR2T:            1
> [  248.767078] DataPDUInOrder:               Yes
> [  248.767698] DataSequenceInOrder:          Yes
> [  248.768319] ErrorRecoveryLevel:           0
> [  248.768926] SessionType:                  Normal
> [  248.769681]
> ------------------------------------------------------------------ [ 
> 248.770703] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr:
> ffff88006b597000 [  248.771412] iSCSI Login successful on CID: 0 from
> 192.168.0.111 to 192.168.0.117:3260,1 [  248.772470] Incremented iSCSI
> Connection count to 1 from node: iqn.1993-08.org.debian:01:2dadf92d0ef [ 
> 248.773644] Established iSCSI session from node:
> iqn.1993-08.org.debian:01:2dadf92d0ef [  248.774694] Incremented number of
> active iSCSI sessions to 2 on iSCSI Target Portal Group: 1

Re: KVM x86_64 with SR-IOV..? (device passthrough with LIO-Target v3.0)

[Nicholas A. Bellinger]

On Wed, 2009-05-06 at 11:51 +0800, Sheng Yang wrote:
> On Wednesday 06 May 2009 01:45:47 Nicholas A. Bellinger wrote:
> > On Tue, 2009-05-05 at 04:28 -0700, Nicholas A. Bellinger wrote:
> > > On Tue, 2009-05-05 at 03:43 -0700, Nicholas A. Bellinger wrote:
> > > > On Tue, 2009-05-05 at 09:42 +0800, Yu Zhao wrote:
> > > > > Hi,
> > > > >
> > > > > The VF also works in the host if the VF driver is programed properly.
> > > > > So it would be easier to develop the VF driver in the host and then
> > > > > verify the VF driver in the guest.
> > > > >
> > > > > BTW, I didn't see the SR-IOV is enabled in your dmesg, did you select
> > > > > the CONFIG_PCI_IOV in the kernel .config?
> > > > >
> > > > > Thanks,
> > > > > Yu
> > > >
> > > > Greetings Yu and Sheng,
> > > >
> > > > So the original attachment was for the v2.6.29-fc11 host kernel output,
> > > > I ended up jumping to v2.6.30-rc3 (and making sure CONFIG_PCI_IOV was
> > > > enabled) for KVM host with kvm-85 and now things are looking quite
> > > > stable for me.
> > > >
> > > > So far I have been able to successfully push LIO-Target v3.0 traffic
> > > > *inside* a v2.6.29.2 KVM guest via the onboard e1000e (02:00.0) port
> > > > from another Linux/iSCSI Initiator machine using a Intel 1 Gb/sec port.
> > > > I am running badblocks tests to iSCSI Logical Units for RAMDISK_DR and
> > > > FILEIO storage objects (in the KVM Guest), and they are passing
> > > > validation and I am seeing ~500 Mb/sec of throughput and very low CPU
> > > > usage in the KVM guests.
> > >
> > > Ok I am seeing another issue with the e1000e port on 02:00.0..:
> > >
> > > As i start to push multiple badblocks tests RAMDISK_DR iSCSI Logical
> > > units into KVM Guest running LIO v2.6.29.2 from the external Linux/iSCSI
> > > Initiator machine, after about 100 GB of iSCSI traffic, I see the
> > > following exception in KVM host v2.6.30-rc3:
> > >
> > > DRHD: handling fault status reg 2
> > > DMAR:[DMA Write] Request device [02:00.0] fault addr 7fc958b010000
> > > DMAR:[fault reason 04] Access beyond MGAW
> > > pci-stub 0000:02:00.0: irq 59 for MSI/MSI-X
> > > pci-stub 0000:02:00.0: irq 60 for MSI/MSI-X
> > > pci-stub 0000:02:00.0: irq 61 for MSI/MSI-X
> > >
> > > I am able to restart the LIO-Target KVM Guest and the Linux/iSCSI
> > > Initiators are able to reconnect..  Wow, very cool..
> > >
> > > Not sure if this is a bug in the target_core_mod RAMDISK_DR subsystem
> > > plugin (mapping struct iovec to internally allocated struct page) or
> > > what.  I will have to look at the DMAR code to understand what this
> > > exception means..
> >
> > Greetings Yu, Sheng and Co,
> >
> > So I have been making progress this morning..  So far, I have hooked up
> > a LSI mpt-function PCIe SAS adapter into the KVM guest with a Sandisk
> > SATA SSD 32 GB drive.  It is using MSI interrupts (not MSI-X) and I am
> > able to push ~70 MB/sec from a 2nd Linux/iSCSI Initiator machine
> > (running Open-iSCSI) with the 1500 byte MTUs on e1000e ports from within
> > the KVM guest.
> 
> Is MSI-X can't be enabled or the device only have MSI capability? Just 
> curious...
> 

AFAICT MSI-X is not supported on the mpt-fusion SAS device when running
on the KVM host..

> > The interesting thing is that I am having to use IBLOCK export (using
> > using submit_bio(), and complete emulation of SCSI control path) for
> > SATA SSD in order to get I/O running stable  Using the pSCSI export I am
> > getting immediate exceptions from scsi_execute_async() in the v2.6.29.2
> > KVM guest..  
> 
> Didn't see exception in the log below... (And buried with iscsi log I can't 
> understand. Looking forward for the help from others...) Any thing notable 
> show in the host side? I think the target to get pSCSI work well now?
> 

Yeah, the log attached was using the SATA SSD with the IBLOCK export
(the one that worked..).  The exception that I was seeing from pSCSI
export from the same SATA SSD from within KVM Guest running v2.6.29.2
involved a non-zero return from
drivers/scsi/scsi_lib.c:scsi_execute_async() for basically all bulk
struct scatterlist I/O (the control path seemed OK however).

Unfortuately, scsi_execute_async() was (it has been removed for v2.6.30)
really bad at returning any useful value and hardcodes a return of
'DRIVER_ERROR << 24' in the error path, so I was unable to determine
anything useful for what was actually causing it to fail during my
initial testing..

I will keep poking at it and see if I can find anything interesting..

> BTW: Maybe you can try the patch from Marcelo titled "[patch 0/4] use 
> smp_send_reschedule in vcpu_kick / assigned dev host intx race fix".
> 

Thanks for the pointer, I will have a look.

Many thanks for your most valuable of time,

--nab

> -- 
> regards
> Yang, Sheng
> 
> 
> > Using a 2nd SAS disk I am able to use target_core_mod/pSCSI
> > export and push badblocks and LTP disktest traffic however..
> 
> > Here is a bit about the the setup looks,
> >
> > *) Linux/iSCSI Initiator node accessing KVM Guest LIO-Target v3.0
> > storage:
> >
> > subjekt:~# lsscsi
> > [6:0:0:0]    disk    ATA      ST3250820AS      3.AA  /dev/sda
> > [10:0:0:0]   cd/dvd  PIONEER  DVD-ROM DVD-305  1.06  /dev/scd1
> > [18:0:0:0]   cd/dvd  TOSHIBA  DVD/HD  X807616  MC08  /dev/scd2
> > [32:0:0:0]   disk    LIO-ORG  RAMDISK-DR       3.0   /dev/sdb
> > [32:0:0:1]   disk    LIO-ORG  RAMDISK-DR       3.0   /dev/sdc
> > [32:0:0:2]   disk    LIO-ORG  FILEIO           3.0   /dev/sdd
> > [32:0:0:3]   disk    LIO-ORG  IBLOCK           3.0   /dev/sde
> >
> > subjekt:~# sg_inq -i /dev/sde
> > VPD INQUIRY: Device Identification page
> >   Designation descriptor number 1, descriptor length: 20
> >     id_type: NAA,  code_set: Binary
> >     associated with the addressed logical unit
> >       NAA 6, IEEE Company_id: 0x1405
> >       Vendor Specific Identifier: 0xa97e4ce21
> >       Vendor Specific Identifier Extension: 0xc0711de829b000c2
> >       [0x6001405a97e4ce21c0711de829b000c2]
> >   Designation descriptor number 2, descriptor length: 52
> >     id_type: T10 vendor identification,  code_set: ASCII
> >     associated with the addressed logical unit
> >       vendor id: LIO-ORG
> >       vendor specific: IBLOCK:a97e4ce21c0711de829b000c2943d57b
> >   Designation descriptor number 3, descriptor length: 8
> >     transport: Internet SCSI (iSCSI)
> >     id_type: Relative target port,  code_set: Binary
> >     associated with the target port
> >       Relative target port: 0x1
> >   Designation descriptor number 4, descriptor length: 8
> >     transport: Internet SCSI (iSCSI)
> >     id_type: Target port group,  code_set: Binary
> >     associated with the target port
> >       Target port group: 0x0
> >   Designation descriptor number 5, descriptor length: 8
> >     id_type: Logical unit group,  code_set: Binary
> >     associated with the addressed logical unit
> >       Logical unit group: 0x0
> >   Designation descriptor number 6, descriptor length: 80
> >     transport: Internet SCSI (iSCSI)
> >     id_type: SCSI name string,  code_set: UTF-8
> >     associated with the target port
> >       SCSI name string:
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6,t,0x0001
> >
> > The SATA SSD is available as /dev/sde on the Linux/iSCSI Initiator node,
> > the 2nd SAS disk was removed from the dmesg below as having multiple
> > drives on the mpt-function card made it start to hiccup..
> >
> > --nab
> >
> > *) From with the LIO-Target KVM gest:
> >
> > lenny64guest1:~# lspci
> > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev
> > 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton
> > II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE
> > [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4
> > ACPI (rev 03) 00:02.0 VGA compatible controller: Cirrus Logic GD 5446
> > 00:03.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> > RTL-8139/8139C/8139C+ (rev 20) 00:04.0 RAM memory: Qumranet, Inc. Device
> > 1002
> > 00:05.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> > Connection 00:06.0 SCSI storage controller: LSI Logic / Symbios Logic
> > SAS1068E PCI-Express Fusion-MPT SAS (rev 02)
> >
> >
> > *) Dmesg from LIO-Target KVM guest:
> >
> > [    0.000000] Initializing cgroup subsys cpuset
> > [    0.000000] Initializing cgroup subsys cpu
> > [    0.000000] Linux version 2.6.29.2 (root@lenny64guest1) (gcc version
> > 4.3.2 (Debian 4.3.2-1.1) ) #2 SMP Mon May 4 01:22:53 PDT 2009 [   
> > 0.000000] Command line: root=/dev/hda1 ro
> > [    0.000000] KERNEL supported cpus:
> > [    0.000000]   Intel GenuineIntel
> > [    0.000000]   AMD AuthenticAMD
> > [    0.000000]   Centaur CentaurHauls
> > [    0.000000] BIOS-provided physical RAM map:
> > [    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
> > [    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
> > [    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
> > [    0.000000]  BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
> > [    0.000000]  BIOS-e820: 000000007fff0000 - 0000000080000000 (ACPI data)
> > [    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
> > [    0.000000] DMI 2.4 present.
> > [    0.000000] last_pfn = 0x7fff0 max_arch_pfn = 0x100000000
> > [    0.000000] init_memory_mapping: 0000000000000000-000000007fff0000
> > [    0.000000]  0000000000 - 007fe00000 page 2M
> > [    0.000000]  007fe00000 - 007fff0000 page 4k
> > [    0.000000] kernel direct mapping tables up to 7fff0000 @ 8000-c000
> > [    0.000000] last_map_addr: 7fff0000 end: 7fff0000
> > [    0.000000] RAMDISK: 379c8000 - 37fef480
> > [    0.000000] ACPI: RSDP 000FBE50, 0014 (r0 QEMU  )
> > [    0.000000] ACPI: RSDT 7FFF0000, 002C (r1 QEMU   QEMURSDT        1 QEMU 
> >       1) [    0.000000] ACPI: FACP 7FFF002C, 0074 (r1 QEMU   QEMUFACP      
> >  1 QEMU        1) [    0.000000] FADT: X_PM1a_EVT_BLK.bit_width (16) does
> > not match PM1_EVT_LEN (4) [    0.000000] ACPI: DSDT 7FFF0100, 2540 (r1  
> > BXPC   BXDSDT        1 INTL 20061109) [    0.000000] ACPI: FACS 7FFF00C0,
> > 0040
> > [    0.000000] ACPI: APIC 7FFF2640, 00E0 (r1 QEMU   QEMUAPIC        1 QEMU 
> >       1) [    0.000000] ACPI: Local APIC address 0xfee00000
> > [    0.000000] No NUMA configuration found
> > [    0.000000] Faking a node at 0000000000000000-000000007fff0000
> > [    0.000000] Bootmem setup node 0 0000000000000000-000000007fff0000
> > [    0.000000]   NODE_DATA [0000000000001000 - 0000000000005fff]
> > [    0.000000]   bootmap [000000000000a000 -  0000000000019fff] pages 10
> > [    0.000000] (6 early reservations) ==> bootmem [0000000000 - 007fff0000]
> > [    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==>
> > [0000000000 - 0000001000] [    0.000000]   #1 [0000006000 - 0000008000]    
> >   TRAMPOLINE ==> [0000006000 - 0000008000] [    0.000000]   #2 [0000200000
> > - 000070493c]    TEXT DATA BSS ==> [0000200000 - 000070493c] [    0.000000]
> >   #3 [00379c8000 - 0037fef480]          RAMDISK ==> [00379c8000 -
> > 0037fef480] [    0.000000]   #4 [000009fc00 - 0000100000]    BIOS reserved
> > ==> [000009fc00 - 0000100000] [    0.000000]   #5 [0000008000 - 000000a000]
> >          PGTABLE ==> [0000008000 - 000000a000] [    0.000000] found SMP
> > MP-table at [ffff8800000fbd10] 000fbd10
> > [    0.000000]  [ffffe20000000000-ffffe20001bfffff] PMD ->
> > [ffff880001200000-ffff880002dfffff] on node 0 [    0.000000] Zone PFN
> > ranges:
> > [    0.000000]   DMA      0x00000000 -> 0x00001000
> > [    0.000000]   DMA32    0x00001000 -> 0x00100000
> > [    0.000000]   Normal   0x00100000 -> 0x00100000
> > [    0.000000] Movable zone start PFN for each node
> > [    0.000000] early_node_map[2] active PFN ranges
> > [    0.000000]     0: 0x00000000 -> 0x0000009f
> > [    0.000000]     0: 0x00000100 -> 0x0007fff0
> > [    0.000000] On node 0 totalpages: 524175
> > [    0.000000]   DMA zone: 56 pages used for memmap
> > [    0.000000]   DMA zone: 1387 pages reserved
> > [    0.000000]   DMA zone: 2556 pages, LIFO batch:0
> > [    0.000000]   DMA32 zone: 7112 pages used for memmap
> > [    0.000000]   DMA32 zone: 513064 pages, LIFO batch:31
> > [    0.000000] ACPI: PM-Timer IO Port: 0xb008
> > [    0.000000] ACPI: Local APIC address 0xfee00000
> > [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] disabled)
> > [    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> > [    0.000000] IOAPIC[0]: apic_id 2, version 0, address 0xfec00000, GSI
> > 0-23 [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high
> > level) [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high
> > level) [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10
> > high level) [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq
> > 11 high level) [    0.000000] ACPI: IRQ5 used by override.
> > [    0.000000] ACPI: IRQ9 used by override.
> > [    0.000000] ACPI: IRQ10 used by override.
> > [    0.000000] ACPI: IRQ11 used by override.
> > [    0.000000] Using ACPI (MADT) for SMP configuration information
> > [    0.000000] SMP: Allowing 16 CPUs, 14 hotplug CPUs
> > [    0.000000] nr_irqs_gsi: 24
> > [    0.000000] PM: Registered nosave memory: 000000000009f000 -
> > 00000000000a0000 [    0.000000] PM: Registered nosave memory:
> > 00000000000a0000 - 00000000000e8000 [    0.000000] PM: Registered nosave
> > memory: 00000000000e8000 - 0000000000100000 [    0.000000] Allocating PCI
> > resources starting at 88000000 (gap: 80000000:7ffbc000) [    0.000000]
> > NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:16 nr_node_ids:1 [    0.000000]
> > PERCPU: Allocating 49152 bytes of per cpu data
> > [    0.000000] Built 1 zonelists in Node order, mobility grouping on. 
> > Total pages: 515620 [    0.000000] Policy zone: DMA32
> > [    0.000000] Kernel command line: root=/dev/hda1 ro
> > [    0.000000] Initializing CPU#0
> > [    0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes)
> > [    0.000000] Fast TSC calibration using PIT
> > [    0.000000] Detected 2266.815 MHz processor.
> > [    0.004000] Console: colour VGA+ 80x25
> > [    0.004000] console [tty0] enabled
> > [    0.004000] Checking aperture...
> > [    0.004000] No AGP bridge found
> > [    0.004000] Calgary: detecting Calgary via BIOS EBDA area
> > [    0.004000] Calgary: Unable to locate Rio Grande table in EBDA -
> > bailing! [    0.004000] Memory: 2055644k/2097088k available (2463k kernel
> > code, 388k absent, 41056k reserved, 1259k data, 464k init) [    0.004013]
> > Calibrating delay loop (skipped), value calculated using timer frequency..
> > 4533.63 BogoMIPS (lpj=9067260) [    0.004985] Security Framework
> > initialized
> > [    0.005422] SELinux:  Disabled at boot.
> > [    0.005995] Dentry cache hash table entries: 262144 (order: 9, 2097152
> > bytes) [    0.007887] Inode-cache hash table entries: 131072 (order: 8,
> > 1048576 bytes) [    0.008603] Mount-cache hash table entries: 256
> > [    0.009309] Initializing cgroup subsys ns
> > [    0.009739] Initializing cgroup subsys cpuacct
> > [    0.010172] Initializing cgroup subsys devices
> > [    0.010655] CPU: L1 I cache: 32K, L1 D cache: 32K
> > [    0.011229] CPU: L2 cache: 2048K
> > [    0.011633] CPU 0/0x0 -> Node 0
> > [    0.012061] ACPI: Core revision 20081204
> > [    0.013529] Setting APIC routing to flat
> > [    0.015117] ..TIMER: vector=0x30 apic1=0 pin1=0 apic2=-1 pin2=-1
> > [    0.055398] CPU0: Intel QEMU Virtual CPU version 0.10.0 stepping 03
> > [    0.273970] Booting processor 1 APIC 0x1 ip 0x6000
> > [    0.004000] Initializing CPU#1
> > [    0.004000] Calibrating delay using timer specific routine.. 4533.28
> > BogoMIPS (lpj=9066565) [    0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
> > [    0.004000] CPU: L2 cache: 2048K
> > [    0.004000] CPU 1/0x1 -> Node 0
> > [    0.364147] CPU1: Intel QEMU Virtual CPU version 0.10.0 stepping 03
> > [    0.367429] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
> > [    0.372036] Brought up 2 CPUs
> > [    0.372442] Total of 2 processors activated (9066.91 BogoMIPS).
> > [    0.373176] CPU0 attaching sched-domain:
> > [    0.373181]  domain 0: span 0-1 level CPU
> > [    0.373183]   groups: 0 1
> > [    0.373186] CPU1 attaching sched-domain:
> > [    0.373187]  domain 0: span 0-1 level CPU
> > [    0.373188]   groups: 1 0
> > [    0.373255] net_namespace: 1888 bytes
> > [    0.373255] NET: Registered protocol family 16
> > [    0.373269] ACPI: bus type pci registered
> > [    0.376082] PCI: Using configuration type 1 for base access
> > [    0.380281] bio: create slab <bio-0> at 0
> > [    0.381293] ACPI: EC: Look up EC in DSDT
> > [    0.383292] ACPI: Interpreter enabled
> > [    0.383693] ACPI: (supports S0 S3 S4 S5)
> > [    0.384763] ACPI: Using IOAPIC for interrupt routing
> > [    0.388691] ACPI: No dock devices found.
> > [    0.388729] ACPI: PCI Root Bridge [PCI0] (0000:00)
> > [    0.393074] pci 0000:00:01.1: reg 20 io port: [0xc000-0xc00f]
> > [    0.393406] pci 0000:00:01.3: quirk: region b000-b03f claimed by PIIX4
> > ACPI [    0.393920] pci 0000:00:01.3: quirk: region b100-b10f claimed by
> > PIIX4 SMB [    0.395842] pci 0000:00:02.0: reg 10 32bit mmio:
> > [0xf0000000-0xf1ffffff] [    0.396508] pci 0000:00:02.0: reg 14 32bit mmio:
> > [0xf2000000-0xf2000fff] [    0.401611] pci 0000:00:03.0: reg 10 io port:
> > [0xc100-0xc1ff]
> > [    0.401674] pci 0000:00:03.0: reg 14 32bit mmio: [0xf2001000-0xf20010ff]
> > [    0.401903] pci 0000:00:04.0: reg 10 io port: [0xc200-0xc21f]
> > [    0.403518] pci 0000:00:05.0: reg 10 32bit mmio: [0xf2020000-0xf203ffff]
> > [    0.404059] pci 0000:00:05.0: reg 18 io port: [0xc220-0xc23f]
> > [    0.404368] pci 0000:00:05.0: reg 1c 32bit mmio: [0xf2040000-0xf2043fff]
> > [    0.405176] pci 0000:00:06.0: reg 10 io port: [0xc300-0xc3ff]
> > [    0.405474] pci 0000:00:06.0: reg 14 32bit mmio: [0xf2044000-0xf2047fff]
> > [    0.405804] pci 0000:00:06.0: reg 1c 32bit mmio: [0xf2050000-0xf205ffff]
> > [    0.405948] pci 0000:00:06.0: reg 30 32bit mmio: [0x200000-0x3fffff]
> > [    0.406577] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
> > [    0.408281] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
> > [    0.412098] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
> > [    0.413393] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
> > [    0.414571] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
> > [    0.416142] usbcore: registered new interface driver usbfs
> > [    0.416504] usbcore: registered new interface driver hub
> > [    0.416504] usbcore: registered new device driver usb
> > [    0.416508] PCI: Using ACPI for IRQ routing
> > [    0.452044] pnp: PnP ACPI init
> > [    0.452515] ACPI: bus type pnp registered
> > [    0.454439] pnp: PnP ACPI: found 7 devices
> > [    0.454918] ACPI: ACPI bus type pnp unregistered
> > [    0.460804] pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
> > [    0.460807] pci_bus 0000:00: resource 1 mem:
> > [0x000000-0xffffffffffffffff] [    0.460969] NET: Registered protocol
> > family 2
> > [    0.508246] IP route cache hash table entries: 65536 (order: 7, 524288
> > bytes) [    0.537526] TCP established hash table entries: 262144 (order:
> > 10, 4194304 bytes) [    0.542048] TCP bind hash table entries: 65536
> > (order: 8, 1048576 bytes) [    0.543577] TCP: Hash tables configured
> > (established 262144 bind 65536) [    0.544332] TCP reno registered
> > [    0.556665] NET: Registered protocol family 1
> > [    0.557476] checking if image is initramfs... it is
> > [    0.980042] Switched to high resolution mode on CPU 1
> > [    0.983993] Switched to high resolution mode on CPU 0
> > [    1.028987] Freeing initrd memory: 6301k freed
> > [    1.032241] audit: initializing netlink socket (disabled)
> > [    1.032745] type=2000 audit(1241543237.032:1): initialized
> > [    1.035249] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> > [    1.036055] VFS: Disk quotas dquot_6.5.2
> > [    1.036182] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
> > [    1.036182] msgmni has been set to 4027
> > [    1.036182] alg: No test for stdrng (krng)
> > [    1.036182] Block layer SCSI generic (bsg) driver version 0.4 loaded
> > (major 253) [    1.036182] io scheduler noop registered
> > [    1.036182] io scheduler anticipatory registered
> > [    1.036182] io scheduler deadline registered
> > [    1.040223] io scheduler cfq registered (default)
> > [    1.040692] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
> > [    1.041194] pci 0000:00:01.0: PIIX3: Enabling Passive Release
> > [    1.041681] pci 0000:00:01.0: Activating ISA DMA hang workarounds
> > [    1.042183] pci 0000:00:02.0: Boot video device
> > [    1.048876] Linux agpgart interface v0.103
> > [    1.049315] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> > [    1.050082] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> > [    1.051351] 00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
> > [    1.054962] brd: module loaded
> > [    1.055662] PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64
> > irq 1,12 [    1.111469] serio: i8042 KBD port at 0x60,0x64 irq 1
> > [    1.112044] serio: i8042 AUX port at 0x60,0x64 irq 12
> > [    1.112901] mice: PS/2 mouse device common for all mice
> > [    1.114216] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
> > [    1.114828] rtc0: alarms up to one day, 114 bytes nvram
> > [    1.115363] input: AT Translated Set 2 keyboard as /class/input/input0
> > [    1.115934] cpuidle: using governor ladder
> > [    1.116448] cpuidle: using governor menu
> > [    1.116926] No iBFT detected.
> > [    1.117634] TCP cubic registered
> > [    1.118088] NET: Registered protocol family 17
> > [    1.118873] registered taskstats version 1
> > [    1.119543] rtc_cmos 00:01: setting system clock to 2009-05-05 17:07:17
> > UTC (1241543237) [    1.120418] Freeing unused kernel memory: 464k freed
> > [    1.193327] processor ACPI_CPU:00: registered as cooling_device0
> > [    1.194273] processor ACPI_CPU:01: registered as cooling_device1
> > [    1.392735] Uniform Multi-Platform E-IDE driver
> > [    1.413020] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
> > [    1.413531] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11
> > (level, high) -> IRQ 11 [    1.415182] 8139too Fast Ethernet driver 0.9.28
> > [    1.415673] 8139too 0000:00:03.0: This (id 10ec:8139 rev 20) is an
> > enhanced 8139C+ chip, use 8139cp [    1.418181] e1000e: Intel(R) PRO/1000
> > Network Driver - 0.3.3.3-k6 [    1.418680] e1000e: Copyright (c) 1999-2008
> > Intel Corporation.
> > [    1.419758] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
> > [    1.420328] e1000e 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10
> > (level, high) -> IRQ 10 [    1.421130] e1000e 0000:00:05.0: setting latency
> > timer to 64
> > [    1.421963] e1000e 0000:00:05.0: irq 24 for MSI/MSI-X
> > [    1.421999] e1000e 0000:00:05.0: irq 25 for MSI/MSI-X
> > [    1.422033] e1000e 0000:00:05.0: irq 26 for MSI/MSI-X
> > [    1.431733] SCSI subsystem initialized
> > [    1.445624] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
> > [    1.453420] Fusion MPT base driver 3.04.07
> > [    1.453849] Copyright (c) 1999-2008 LSI Corporation
> > [    1.461332] Fusion MPT SAS Host driver 3.04.07
> > [    1.464878] FDC 0 is a S82078B
> > [    1.549697] 0000:00:05.0: eth0: (PCI Express:2.5GB/s:Width x1)
> > 00:e0:81:c0:90:b2 [    1.550752] 0000:00:05.0: eth0: Intel(R) PRO/1000
> > Network Connection [    1.551549] 0000:00:05.0: eth0: MAC: 3, PHY: 8, PBA
> > No: ffffff-0ff [    1.553336] ACPI: PCI Interrupt Link [LNKC] enabled at
> > IRQ 11
> > [    1.553940] udev: renamed network interface eth0 to eth1
> > [    1.554683] 8139cp 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11
> > (level, high) -> IRQ 11 [    1.557146] eth0: RTL-8139C+ at
> > 0xffffc2000096a000, 52:54:00:12:34:56, IRQ 11 [    1.560485] 8139cp
> > 0000:00:03.0: setting latency timer to 64
> > [    1.560904] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
> > [    1.561607] mptsas 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 10
> > (level, high) -> IRQ 10 [    1.563379] mptbase: ioc0: Initiating bringup
> > [    1.634659] mptbase: ioc0: WARNING - Unexpected doorbell active!
> > [    5.343913] ioc0: LSISAS1068E B1: Capabilities={Initiator}
> > [    5.345428] mptsas 0000:00:06.0: setting latency timer to 64
> > [   22.874565] scsi0 : ioc0: LSISAS1068E B1, FwRev=010f0000h, Ports=1,
> > MaxQ=511, IRQ=10 [   22.903281] scsi 0:0:0:0: Direct-Access     ATA     
> > SanDisk SSD SATA 1.20 PQ: 0 ANSI: 5 [   22.906530] piix 0000:00:01.1: IDE
> > controller (0x8086:0x7010 rev 0x00) [   22.908623] piix 0000:00:01.1: not
> > 100% native mode: will probe irqs later [   22.909390] PIIX_IDE
> > 0000:00:01.1: setting latency timer to 64
> > [   22.909410]     ide0: BM-DMA at 0xc000-0xc007
> > [   22.910045]     ide1: BM-DMA at 0xc008-0xc00f
> > [   22.910666] Probing IDE interface ide0...
> > [   22.933264] Driver 'sd' needs updating - please use bus_type methods
> > [   22.936186] sd 0:0:0:0: [sda] 62533296 512-byte hardware sectors: (32.0
> > GB/29.8 GiB) [   22.938165] sd 0:0:0:0: [sda] Write Protect is off
> > [   22.938808] sd 0:0:0:0: [sda] Mode Sense: 73 00 00 08
> > [   22.940775] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled,
> > doesn't support DPO or FUA [   22.942194] sd 0:0:0:0: [sda] 62533296
> > 512-byte hardware sectors: (32.0 GB/29.8 GiB) [   22.944225] sd 0:0:0:0:
> > [sda] Write Protect is off
> > [   22.944879] sd 0:0:0:0: [sda] Mode Sense: 73 00 00 08
> > [   22.946734] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled,
> > doesn't support DPO or FUA [   22.947886]  sda: unknown partition table
> > [   22.951811] sd 0:0:0:0: [sda] Attached SCSI disk
> > [   23.211126] hda: QEMU HARDDISK, ATA DISK drive
> > [   23.883187] hda: host max PIO4 wanted PIO255(auto-tune) selected PIO2
> > [   23.883289] hda: MWDMA2 mode selected
> > [   23.884023] Probing IDE interface ide1...
> > [   24.283042] hdc: QEMU DVD-ROM, ATAPI CD/DVD-ROM drive
> > [   24.955145] hdc: host max PIO4 wanted PIO255(auto-tune) selected PIO0
> > [   24.955199] hdc: MWDMA2 mode selected
> > [   24.955978] ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
> > [   24.956812] isa bounce pool size: 16 pages
> > [   24.957471] ide1 at 0x170-0x177,0x376 on irq 15
> > [   24.974924] libata version 3.00 loaded.
> > [   24.992633] ide-gd driver 1.18
> > [   24.993261] hda: max request size: 512KiB
> > [   24.993874] hda: 16384000 sectors (8388 MB) w/256KiB Cache,
> > CHS=16253/255/63 [   25.000639] ide-cd driver 5.00
> > [   25.007111] hda: cache flushes supported
> > [   25.007753]  hda: hda1 hda2 < hda5 >
> > [   25.010488] ide-cd: hdc: ATAPI 4X CD-ROM drive, 512kB Cache
> > [   25.011858] Uniform CD-ROM driver Revision: 3.20
> > [   25.110990] PM: Starting manual resume from disk
> > [   25.129340] EXT3-fs: INFO: recovery required on readonly filesystem.
> > [   25.130052] EXT3-fs: write access will be enabled during recovery.
> > [   26.128769] kjournald starting.  Commit interval 5 seconds
> > [   26.129481] EXT3-fs: recovery complete.
> > [   26.130963] EXT3-fs: mounted filesystem with ordered data mode.
> > [   26.335166] udevd version 125 started
> > [   26.631975] input: Power Button (FF) as /class/input/input1
> > [   26.666993] ACPI: Power Button (FF) [PWRF]
> > [   26.681253] parport_pc 00:05: reported by Plug and Play ACPI
> > [   26.681824] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
> > [   26.723397] input: PC Speaker as /class/input/input2
> > [   26.724903] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100,
> > revision 0 [   26.949233] input: ImExPS/2 Generic Explorer Mouse as
> > /class/input/input3 [   58.566336] Adding 393552k swap on /dev/hda5. 
> > Priority:-1 extents:1 across:393552k [   58.594239] EXT3 FS on hda1,
> > internal journal
> > [   59.613114] loop: module loaded
> > [   60.681886] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
> > [   63.746408] NET: Registered protocol family 10
> > [   63.748013] lo: Disabled Privacy Extensions
> > [   73.968880] eth0: no IPv6 routers present
> > [  233.617434] ADDRCONF(NETDEV_UP): eth1_rename: link is not ready
> > [  236.587594] e1000e: eth1_rename NIC Link is Up 1000 Mbps Full Duplex,
> > Flow Control: RX/TX [  236.595715] ADDRCONF(NETDEV_CHANGE): eth1_rename:
> > link becomes ready [  243.146435] TARGET_CORE[0]: Loading Generic Kernel
> > Storage Engine: v3.0.0 on Linux/x86_64 on 2.6.29.2 [  243.147955]
> > TARGET_CORE[0]: Initialized ConfigFS Fabric Infrastructure: v1.0.0 on
> > Linux/x86_64 on 2.6.29.2 [  243.149107] SE_PC[0] - Registered Plugin Class:
> > TRANSPORT
> > [  243.149830] PLUGIN_TRANSPORT[1] - pscsi registered
> > [  243.150492] PLUGIN_TRANSPORT[2] - stgt registered
> > [  243.151183] CORE_STGT[0]: Bus Initalization complete
> > [  243.151838] PLUGIN_TRANSPORT[4] - iblock registered
> > [  243.152483] PLUGIN_TRANSPORT[5] - rd_dr registered
> > [  243.153126] PLUGIN_TRANSPORT[6] - rd_mcp registered
> > [  243.155097] PLUGIN_TRANSPORT[7] - fileio registered
> > [  243.155743] SE_PC[1] - Registered Plugin Class: OBJ
> > [  243.156372] PLUGIN_OBJ[1] - dev registered
> > [  243.160265] Target_Core_ConfigFS: Located se_plugin: ffff88007c4318e0
> > plugin_name: iblock hba_type: 4 plugin_dep_id: 0 [  243.161454] CORE_HBA[0]
> > - Linux-iSCSI.org iBlock HBA Driver 3.0 on Generic Target Core Stack v3.0 [
> >  243.162630] CORE_HBA[0] - Attached iBlock HBA: 0 to Generic Target Core
> > TCQ Depth: 512 [  243.163681] CORE_HBA[0] - Attached HBA to Generic Target
> > Core
> > [  243.164496] IBLOCK: Allocated ib_dev for ssd_test0
> > [  243.165152] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> > ffff88007c022000 se_dev_su_ptr: ffff88007c444c00 [  243.166753]
> > Target_Core_ConfigFS: Set emulated VPD Unit Serial:
> > a97e4ce21c0711de829b000c2943d57b [  243.167947] IBLOCK: Referencing Major:
> > 8
> > [  243.168541] IBLOCK: Referencing Minor: 0
> > [  243.169225] IBLOCK: Claiming ffff88007c444c00 Major:Minor - 8:0
> > [  243.170021] bio: create slab <bio-1> at 1
> > [  243.170656] IBLOCK: Created bio_set() for major/minor: 8:0
> > [  243.171325] iblock: Using SPC3_PERSISTENT_RESERVATIONS emulation
> > [  243.172012] iblock: Enabling ALUA Emulation for SPC-3 device
> > [  243.172696] iblock: Adding to default ALUA LU Group:
> > core/alua/lu_gps/default_lu_gp [  243.173802] CORE_iBLOCK[0] - Activating
> > Device with TCQ: 0 at Major: 8 Minor 0 [  243.175345]   Vendor: LIO-ORG  
> > Model: IBLOCK            Revision: 3.0 [  243.181019]   Type:  
> > Direct-Access                      ANSI SCSI revision: 05 [  243.182592]
> > T10 VPD Unit Serial Number: a97e4ce21c0711de829b000c2943d57b [  243.183397]
> > T10 VPD Page Length: 72
> > [  243.183983] T10 VPD Identifer Length: 16
> > [  243.184588] T10 VPD Identifier Association: addressed logical unit
> > [  243.185286] T10 VPD Identifier Type: NAA
> > [  243.186965] T10 VPD Binary Device Identifier:
> > 36001405a97e4ce21c0711de829b000c2 [  243.187993] T10 VPD Identifer Length:
> > 48
> > [  243.188602] T10 VPD Identifier Association: addressed logical unit
> > [  243.189303] T10 VPD Identifier Type: T10 Vendor ID based
> > [  243.190037] T10 VPD ASCII Device Identifier: LIO-ORG
> > [  243.191029] Target_Core_ConfigFS: Registered se_dev->se_dev_ptr:
> > ffff88007c5d1400 [  243.193246] Target_Core_ConfigFS: Located se_plugin:
> > ffff88007c431838 plugin_name: pscsi hba_type: 1 plugin_dep_id: 0 [ 
> > 243.194482] CORE_HBA[1] - Linux-iSCSI.org SCSI HBA Driver v3.0 on Generic
> > Target Core Stack v3.0 [  243.195580] CORE_HBA[1] - MPT SPI Host
> > [  243.196170] CORE_HBA[1] - Attached SCSI HBA to Generic Target Core with
> > TCQ Depth: 127 MaxSectors: 8192 [  243.197292] CORE_HBA[1] - Attached HBA
> > to Generic Target Core
> > [  243.198125] PSCSI: Allocated pdv: ffff88007c42aa00 for sda
> > [  243.198803] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> > ffff88007c582000 se_dev_su_ptr: ffff88007c42aa00 [  243.201132] PSCSI:
> > Allocated pdv: ffff88007d064140 for sdb
> > [  243.201952] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> > ffff88007c1d9800 se_dev_su_ptr: ffff88007d064140 [  243.203550] PSCSI[0]:
> > Referencing SCSI Channel ID: 0
> > [  243.204200] PSCSI[0]: Referencing SCSI Target ID: 1
> > [  243.204841] PSCSI[0]: Referencing SCSI LUN ID: 0
> > [  243.205550] Unable to locate 0:0:1:0
> > [  243.208465] Target_Core_ConfigFS: Located se_plugin: ffff88007c431918
> > plugin_name: rd_dr hba_type: 5 plugin_dep_id: 0 [  243.209674] CORE_HBA[2]
> > - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
> > [  243.210802] CORE_HBA[2] - Attached Ramdisk HBA: 0 to Generic Target Core
> > TCQ Depth: 256 MaxSectors: 1024 [  243.211942] CORE_HBA[2] - Attached HBA
> > to Generic Target Core
> > [  243.212774] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> > ffff88007c59c800 se_dev_su_ptr: ffff88007c4374c0 [  243.214315] RAMDISK:
> > Referencing Page Count: 32768
> > [  243.224608] CORE_RD[0] - Built Ramdisk Device ID: 0 space of 32768 pages
> > in 17 tables [  243.225795] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS
> > emulation [  243.226490] rd_dr: Enabling ALUA Emulation for SPC-3 device
> > [  243.227165] rd_dr: Adding to default ALUA LU Group:
> > core/alua/lu_gps/default_lu_gp [  243.228212] CORE_RD[0] - Activating
> > Device with TCQ: 0 at Ramdisk Device ID: 0 [  243.229792]   Vendor: LIO-ORG
> >   Model: RAMDISK-DR        Revision: 3.0 [  243.235413]   Type:  
> > Direct-Access                      ANSI SCSI revision: 05 [  243.236861]
> > T10 VPD Unit Serial Number: 1234567890:2_0
> > [  243.237599] T10 VPD Page Length: 38
> > [  243.238220] T10 VPD Identifer Length: 34
> > [  243.238860] T10 VPD Identifier Association: addressed logical unit
> > [  243.239540] T10 VPD Identifier Type: T10 Vendor ID based
> > [  243.240190] T10 VPD ASCII Device Identifier: LIO-ORG
> > [  243.241083] CORE_RD[0] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768
> > pages in 17 tables, 134217728 total bytes [  243.242336]
> > Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff88007c5d1000 [ 
> > 243.244650] Target_Core_ConfigFS: Located se_plugin: ffff88007c431918
> > plugin_name: rd_dr hba_type: 5 plugin_dep_id: 1 [  243.245895] CORE_HBA[3]
> > - Linux-iSCSI.org Ramdisk HBA Driver v3.0 on Generic Target Core Stack v3.0
> > [  243.247006] CORE_HBA[3] - Attached Ramdisk HBA: 1 to Generic Target Core
> > TCQ Depth: 256 MaxSectors: 1024 [  243.248131] CORE_HBA[3] - Attached HBA
> > to Generic Target Core
> > [  243.248838] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> > ffff88007c18c000 se_dev_su_ptr: ffff88007c973940 [  243.252447] RAMDISK:
> > Referencing Page Count: 32768
> > [  243.262801] CORE_RD[1] - Built Ramdisk Device ID: 0 space of 32768 pages
> > in 17 tables [  243.263870] rd_dr: Using SPC3_PERSISTENT_RESERVATIONS
> > emulation [  243.264555] rd_dr: Enabling ALUA Emulation for SPC-3 device
> > [  243.265231] rd_dr: Adding to default ALUA LU Group:
> > core/alua/lu_gps/default_lu_gp [  243.266342] CORE_RD[1] - Activating
> > Device with TCQ: 0 at Ramdisk Device ID: 0 [  243.267625]   Vendor: LIO-ORG
> >   Model: RAMDISK-DR        Revision: 3.0 [  243.273342]   Type:  
> > Direct-Access                      ANSI SCSI revision: 05 [  243.274807]
> > T10 VPD Unit Serial Number: 1234567890:3_0
> > [  243.275527] T10 VPD Page Length: 38
> > [  243.276125] T10 VPD Identifer Length: 34
> > [  243.276734] T10 VPD Identifier Association: addressed logical unit
> > [  243.277439] T10 VPD Identifier Type: T10 Vendor ID based
> > [  243.278173] T10 VPD ASCII Device Identifier: LIO-ORG
> > [  243.278891] CORE_RD[1] - Added LIO DIRECT Ramdisk Device ID: 0 of 32768
> > pages in 17 tables, 134217728 total bytes [  243.280067]
> > Target_Core_ConfigFS: Registered se_dev->se_dev_ptr: ffff880037943400 [ 
> > 243.282480] Target_Core_ConfigFS: Located se_plugin: ffff88007c431988
> > plugin_name: fileio hba_type: 7 plugin_dep_id: 0 [  243.320107] CORE_HBA[4]
> > - Linux-iSCSI.org FILEIO HBA Driver 3.0 on Generic Target Core Stack v3.0 [
> >  243.321211] CORE_HBA[4] - Attached FILEIO HBA: 0 to Generic Target Core
> > with TCQ Depth: 256 MaxSectors: 1024 [  243.322383] CORE_HBA[4] - Attached
> > HBA to Generic Target Core
> > [  243.323284] FILEIO: Allocated fd_dev for ffff88007bd61ee0
> > [  243.323964] Target_Core_ConfigFS: Allocated se_subsystem_dev_t:
> > ffff88007d1b6000 se_dev_su_ptr: ffff88007c444000 [  243.325456]
> > Target_Core_ConfigFS: Set emulated VPD Unit Serial:
> > 66e418b4e9ab11dd8f51000c2943d57b [  243.340857] FILEIO: Referencing Path:
> > /tmp/fileio
> > [  243.341504] FILEIO: Referencing Size: 134217728 bytes
> > [  243.342298] fileio: Using SPC3_PERSISTENT_RESERVATIONS emulation
> > [  243.342998] fileio: Enabling ALUA Emulation for SPC-3 device
> > [  243.343668] fileio: Adding to default ALUA LU Group:
> > core/alua/lu_gps/default_lu_gp [  243.344706] CORE_FILE[0] - Activating
> > Device with TCQ: 0 at FILEIO Device ID: 0 [  243.346260]   Vendor: LIO-ORG 
> >  Model: FILEIO            Revision: 3.0 [  243.351966]   Type:  
> > Direct-Access                      ANSI SCSI revision: 05 [  243.353392]
> > T10 VPD Unit Serial Number: 66e418b4e9ab11dd8f51000c2943d57b [  243.354363]
> > T10 VPD Page Length: 72
> > [  243.354961] T10 VPD Identifer Length: 16
> > [  243.355569] T10 VPD Identifier Association: addressed logical unit
> > [  243.356304] T10 VPD Identifier Type: NAA
> > [  243.356904] T10 VPD Binary Device Identifier:
> > 3600140566e418b4e9ab11dd8f51000c2 [  243.358048] T10 VPD Identifer Length:
> > 48
> > [  243.358649] T10 VPD Identifier Association: addressed logical unit
> > [  243.359351] T10 VPD Identifier Type: T10 Vendor ID based
> > [  243.360007] T10 VPD ASCII Device Identifier: LIO-ORG
> > [  243.360730] CORE_FILE[0] - Added LIO FILEIO Device ID: 0 at /tmp/fileio,
> > 134217728 total bytes [  243.361913] Target_Core_ConfigFS: Registered
> > se_dev->se_dev_ptr: ffff880037943c00 [  243.364168] Target_Core_ConfigFS:
> > REGISTER -> group: ffffffffa02a0180 name: iscsi [  243.446323]
> > Linux-iSCSI.org iSCSI Target Core Stack v3.0.0 on Linux/x86_64 on 2.6.29.2
> > [  243.447405] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API
> > >>>>>>>>>>>>>>>>>>>>>> [  243.448132] Initialized struct
> > target_fabric_configfs: ffff88006b554000 for iscsi [  243.449150]
> > <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>> [  243.449919]
> > LIO_TARGET[0] - Set fabric -> lio_target_fabric_configfs [  243.451519]
> > iscsi_allocate_thread_sets:195: ***OPS*** Spawned 4 thread set(s) (8 total
> > threads). [  243.452909] TARGET_CORE[iSCSI]: Allocated Discovery
> > se_portal_group_t for endpoint: None, Portal Tag: 0 [  243.454075] CORE[0]
> > - Allocated Discovery TPG
> > [  243.454698] Loading Complete.
> > [  243.455569] Target_Core_ConfigFS: REGISTER -> Located fabric: iscsi
> > [  243.456279] Target_Core_ConfigFS: REGISTER -> ffffffffa02ddbb0
> > [  243.456959] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: iscsi
> > [  243.457695] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for
> > iscsi [  243.458610] lio_target_call_coreaddtiqn(): name:
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 243.459839] CORE[0] - Added iSCSI Target IQN:
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 243.461031] LIO_Target_ConfigFS: REGISTER ->
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 243.462213] LIO_Target_ConfigFS: REGISTER -> Allocated Node:
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 243.463621] lio_target_tiqn_addtpg() called: name tpgt_1
> > [  243.464283] lio_target_tiqn_addtpg() parent name:
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 243.465439] lio_target_tiqn_addtpg() Using TPGT: 1
> > [  243.466206] TARGET_CORE[iSCSI]: Allocated Normal se_portal_group_t for
> > endpoint: iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6,
> > Portal Tag: 1 [  243.467824]
> > CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[
> >1] - Added iSCSI Target Portal Group [  243.469018] LIO_Target_ConfigFS:
> > REGISTER ->
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 243.536626] LIO_Target_ConfigFS: REGISTER -> Allocated TPG: tpgt_1 [ 
> > 243.537704] LIO_Target_ConfigFS: REGISTER ->
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> > PORTAL: 192.168.0.117:3260 [  243.539129] CORE[0] - Added Network Portal:
> > 192.168.0.117:3260 on TCP on network device: None [  243.540238]
> > CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] -
> > Added Network Portal: 192.168.0.117:3260,1 on TCP on network device: None [
> >  243.541925]
> > CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[
> >1] - Incremented np_exports to 1 [  243.543104] LIO_Target_ConfigFS:
> > addnptotpg done!
> > [  243.649897] SCTP: Hash tables configured (established 65536 bind 65536)
> > [  243.704216] CORE[0] - Added Network Portal: 192.168.0.117:3260 on SCTP
> > on network device: None [  243.733493]
> > CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6] -
> > Added Network Portal: 192.168.0.117:3260,1 on SCTP on network device: None
> > [  243.735142]
> > CORE[iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6]_TPG[
> >1] - Incremented np_exports to 1 [  243.737456] LIO_Target_ConfigFS:
> > REGISTER ->
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> > LUN: 0 [  243.739831] rd_dr/iSCSI: Adding to default ALUA Target Port
> > Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.740953]
> > iSCSI_TPG[1]_LUN[0] - Activated iSCSI Logical Unit from CORE HBA: 2 [ 
> > 243.742029] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_0 [ 
> > 243.744038] LIO_Target_ConfigFS: REGISTER ->
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> > LUN: 1 [  243.746304] rd_dr/iSCSI: Adding to default ALUA Target Port
> > Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.747436]
> > iSCSI_TPG[1]_LUN[1] - Activated iSCSI Logical Unit from CORE HBA: 3 [ 
> > 243.748468] LIO_Target_ConfigFS: Created Port Symlink ramdisk -> lun_1 [ 
> > 243.750509] LIO_Target_ConfigFS: REGISTER ->
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> > LUN: 2 [  243.752754] fileio/iSCSI: Adding to default ALUA Target Port
> > Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.753924]
> > iSCSI_TPG[1]_LUN[2] - Activated iSCSI Logical Unit from CORE HBA: 4 [ 
> > 243.754957] LIO_Target_ConfigFS: Created Port Symlink fileio -> lun_2 [ 
> > 243.757089] LIO_Target_ConfigFS: REGISTER ->
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 TPGT: 1
> > LUN: 3 [  243.759370] iblock/iSCSI: Adding to default ALUA Target Port
> > Group: core/alua/tg_pt_gps/default_tg_pt_gp [  243.760507]
> > iSCSI_TPG[1]_LUN[3] - Activated iSCSI Logical Unit from CORE HBA: 0 [ 
> > 243.761525] LIO_Target_ConfigFS: Created Port Symlink ssd_test0 -> lun_3 [ 
> > 243.762761] Disabling iSCSI Authentication Methods for TPG: 1.
> > [  243.763566] iSCSI_TPG[1] - Generate Initiator Portal Group ACLs: Enabled
> > [  243.764414] iSCSI_TPG[1] - Demo Mode Write Protect bit: OFF
> > [  243.765196] iSCSI_TPG[1] - Set Default CmdSN TCQ Depth to 4
> > [  243.832368] iSCSI_TPG[1] - Enabled iSCSI Target Portal Group
> > [  244.173472] Received iSCSI login request from 192.168.0.116 on TCP
> > Network Portal 192.168.0.117:3260 [  244.174664] Located Storage Object:
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 244.175776] Located Portal Group Object: 1
> > [  244.176405] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access
> > for LUN in Demo Mode [  244.177423] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] -
> > Adding READ-WRITE access for LUN in Demo Mode [  244.178656]
> > TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in
> > Demo Mode [  244.179756] TARGET_CORE[iSCSI]->TPG[1]_LUN[3] - Adding
> > READ-WRITE access for LUN in Demo Mode [  244.180854] iSCSI_TPG[1] - Added
> > DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node:
> > iqn.1993-08.org.debian:01.2fed72c26a4c [  244.182213]
> > ------------------------------------------------------------------ [ 
> > 244.183240] HeaderDigest:                 None
> > [  244.183860] DataDigest:                   None
> > [  244.184479] MaxRecvDataSegmentLength:     131072
> > [  244.185099] IFMarker:                     No
> > [  244.185779] OFMarker:                     No
> > [  244.186394]
> > ------------------------------------------------------------------ [ 
> > 244.187422]
> > ------------------------------------------------------------------ [ 
> > 244.188435] MaxConnections:               1
> > [  244.189056] TargetName:                  
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 244.191890] InitiatorName:               
> > iqn.1993-08.org.debian:01.2fed72c26a4c [  244.192934] TargetAlias:         
> >         LIO Target
> > [  244.193622] InitiatorAlias:               subjekt
> > [  244.194272] TargetPortalGroupTag:         1
> > [  244.194878] InitialR2T:                   Yes
> > [  244.195485] ImmediateData:                Yes
> > [  244.196107] MaxBurstLength:               262144
> > [  244.196741] FirstBurstLength:             65536
> > [  244.197356] DefaultTime2Wait:             2
> > [  244.198038] DefaultTime2Retain:           0
> > [  244.198648] MaxOutstandingR2T:            1
> > [  244.199266] DataPDUInOrder:               Yes
> > [  244.199883] DataSequenceInOrder:          Yes
> > [  244.200499] ErrorRecoveryLevel:           0
> > [  244.201115] SessionType:                  Normal
> > [  244.201812]
> > ------------------------------------------------------------------ [ 
> > 244.202838] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr:
> > ffff88006b597e00 [  244.203564] iSCSI Login successful on CID: 0 from
> > 192.168.0.116 to 192.168.0.117:3260,1 [  244.204621] Incremented iSCSI
> > Connection count to 1 from node: iqn.1993-08.org.debian:01.2fed72c26a4c [ 
> > 244.205800] Established iSCSI session from node:
> > iqn.1993-08.org.debian:01.2fed72c26a4c [  244.238865] Incremented number of
> > active iSCSI sessions to 1 on iSCSI Target Portal Group: 1 [  247.041588]
> > eth1_rename: no IPv6 routers present
> > [  248.739907] Received iSCSI login request from 192.168.0.111 on TCP
> > Network Portal 192.168.0.117:3260 [  248.742799] Located Storage Object:
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 248.743981] Located Portal Group Object: 1
> > [  248.744635] TARGET_CORE[iSCSI]->TPG[1]_LUN[0] - Adding READ-WRITE access
> > for LUN in Demo Mode [  248.745818] TARGET_CORE[iSCSI]->TPG[1]_LUN[1] -
> > Adding READ-WRITE access for LUN in Demo Mode [  248.746927]
> > TARGET_CORE[iSCSI]->TPG[1]_LUN[2] - Adding READ-WRITE access for LUN in
> > Demo Mode [  248.748017] TARGET_CORE[iSCSI]->TPG[1]_LUN[3] - Adding
> > READ-WRITE access for LUN in Demo Mode [  248.749122] iSCSI_TPG[1] - Added
> > DYNAMIC ACL with TCQ Depth: 4 for iSCSI Initiator Node:
> > iqn.1993-08.org.debian:01:2dadf92d0ef [  248.750565]
> > ------------------------------------------------------------------ [ 
> > 248.751597] HeaderDigest:                 None
> > [  248.752230] DataDigest:                   None
> > [  248.752861] MaxRecvDataSegmentLength:     131072
> > [  248.753543] IFMarker:                     No
> > [  248.754171] OFMarker:                     No
> > [  248.754785]
> > ------------------------------------------------------------------ [ 
> > 248.755822]
> > ------------------------------------------------------------------ [ 
> > 248.756853] MaxConnections:               1
> > [  248.757510] TargetName:                  
> > iqn.2003-01.org.linux-iscsi.lenny64guest1.x86_64:sn.35c620374de6 [ 
> > 248.758690] InitiatorName:               
> > iqn.1993-08.org.debian:01:2dadf92d0ef [  248.759742] TargetAlias:          
> >        LIO Target
> > [  248.760397] InitiatorAlias:               ubuntu
> > [  248.761034] TargetPortalGroupTag:         1
> > [  248.761731] InitialR2T:                   Yes
> > [  248.762352] ImmediateData:                Yes
> > [  248.762972] MaxBurstLength:               262144
> > [  248.763604] FirstBurstLength:             65536
> > [  248.764233] DefaultTime2Wait:             2
> > [  248.764844] DefaultTime2Retain:           0
> > [  248.765453] MaxOutstandingR2T:            1
> > [  248.767078] DataPDUInOrder:               Yes
> > [  248.767698] DataSequenceInOrder:          Yes
> > [  248.768319] ErrorRecoveryLevel:           0
> > [  248.768926] SessionType:                  Normal
> > [  248.769681]
> > ------------------------------------------------------------------ [ 
> > 248.770703] TARGET_CORE[iSCSI]: Registered fabric_sess_ptr:
> > ffff88006b597000 [  248.771412] iSCSI Login successful on CID: 0 from
> > 192.168.0.111 to 192.168.0.117:3260,1 [  248.772470] Incremented iSCSI
> > Connection count to 1 from node: iqn.1993-08.org.debian:01:2dadf92d0ef [ 
> > 248.773644] Established iSCSI session from node:
> > iqn.1993-08.org.debian:01:2dadf92d0ef [  248.774694] Incremented number of
> > active iSCSI sessions to 2 on iSCSI Target Portal Group: 1
> 
> 
>