[linux-linus test] 25677: regressions - FAIL

[linux-linus test] 25677: regressions - FAIL

[xen.org]

flight 25677 linux-linus real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/25677/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-winxpsp3 11 guest-saverestore.2 fail blocked in 12557

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl          10 migrate-support-check        fail   never pass
 test-amd64-amd64-pv           1 xen-build-check(1)           blocked  n/a
 test-amd64-amd64-xl-pcipt-intel  9 guest-start                 fail never pass
 test-amd64-i386-pv            1 xen-build-check(1)           blocked  n/a
 test-amd64-amd64-xl-winxpsp3 14 guest-stop                   fail   never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop                   fail  never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop             fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop              fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop                   fail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop         fail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop               fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop         fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop               fail never pass
 test-amd64-i386-xl-winxpsp3  14 guest-stop                   fail   never pass
 test-amd64-i386-xl-qemut-winxpsp3 14 guest-stop                fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop              fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 14 guest-stop             fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop               fail never pass

version targeted for testing:
 linux                f217c44ebd41ce7369d2df07622b2839479183b0
baseline version:
 linux                c16fa4f2ad19908a47c63d8fa436a1178438c7e7

------------------------------------------------------------
7133 people touched revisions under test,
not listing them all
------------------------------------------------------------

jobs:
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-i386-rhel6hvm-amd                                 pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-win7-amd64                               fail    
 test-amd64-i386-xl-win7-amd64                                fail    
 test-amd64-i386-xl-credit2                                   pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-xl-pcipt-intel                              fail    
 test-amd64-i386-rhel6hvm-intel                               pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-i386-xl-multivcpu                                 pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-xl-sedf-pin                                 pass    
 test-amd64-amd64-pv                                          blocked 
 test-amd64-i386-pv                                           blocked 
 test-amd64-amd64-xl-sedf                                     pass    
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1                     fail    
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1                     fail    
 test-amd64-i386-xl-winxpsp3-vcpus1                           fail    
 test-amd64-amd64-xl-qemut-winxpsp3                           fail    
 test-amd64-i386-xl-qemut-winxpsp3                            fail    
 test-amd64-amd64-xl-qemuu-winxpsp3                           fail    
 test-amd64-i386-xl-qemuu-winxpsp3                            fail    
 test-amd64-amd64-xl-winxpsp3                                 fail    
 test-amd64-i386-xl-winxpsp3                                  fail    


------------------------------------------------------------
sg-report-flight on osstest.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
    http://www.chiark.greenend.org.uk/~xensrcts/logs

Test harness code can be found at
    http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2409397 lines long.)

[linux-linus test] 25478: regressions - FAIL

[xen.org]

flight 25478 linux-linus real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557

Regressions which are regarded as allowable (not blocking):
 test-amd64-amd64-xl-sedf-pin  7 debian-install            fail REGR. vs. 12557
 test-amd64-amd64-xl-qemuu-win7-amd64  7 windows-install  fail blocked in 12557

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pcipt-intel  9 guest-start                 fail never pass
 test-armhf-armhf-xl          10 migrate-support-check        fail   never pass
 test-amd64-i386-xend-winxpsp3 17 leak-check/check             fail  never pass
 test-amd64-amd64-xl-winxpsp3 14 guest-stop                   fail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 14 guest-stop              fail never pass
 test-amd64-i386-xl-win7-amd64 14 guest-stop                   fail  never pass
 test-amd64-i386-xend-qemut-winxpsp3 17 leak-check/check        fail never pass
 test-amd64-amd64-xl-win7-amd64 14 guest-stop                   fail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 14 guest-stop         fail never pass
 test-amd64-amd64-xl-qemut-winxpsp3 14 guest-stop               fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 14 guest-stop               fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 14 guest-stop              fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 14 guest-stop             fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 14 guest-stop         fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 14 guest-stop               fail never pass

version targeted for testing:
 linux                c60f7d5a8e7c639de5d9dfe07e1e91d302d506e4
baseline version:
 linux                c16fa4f2ad19908a47c63d8fa436a1178438c7e7

------------------------------------------------------------
7112 people touched revisions under test,
not listing them all
------------------------------------------------------------

jobs:
 build-amd64-xend                                             pass    
 build-i386-xend                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 test-amd64-amd64-xl                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-amd64-i386-xl                                           pass    
 test-amd64-i386-rhel6hvm-amd                                 pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-win7-amd64                               fail    
 test-amd64-i386-xl-win7-amd64                                fail    
 test-amd64-i386-xl-credit2                                   pass    
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-amd64-xl-pcipt-intel                              fail    
 test-amd64-i386-rhel6hvm-intel                               pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-i386-xl-multivcpu                                 pass    
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-xl-sedf-pin                                 fail    
 test-amd64-amd64-pv                                          pass    
 test-amd64-i386-pv                                           pass    
 test-amd64-amd64-xl-sedf                                     pass    
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1                     fail    
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1                     fail    
 test-amd64-i386-xl-winxpsp3-vcpus1                           fail    
 test-amd64-i386-xend-qemut-winxpsp3                          fail    
 test-amd64-amd64-xl-qemut-winxpsp3                           fail    
 test-amd64-amd64-xl-qemuu-winxpsp3                           fail    
 test-amd64-i386-xend-winxpsp3                                fail    
 test-amd64-amd64-xl-winxpsp3                                 fail    


------------------------------------------------------------
sg-report-flight on osstest.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
    http://www.chiark.greenend.org.uk/~xensrcts/logs

Test harness code can be found at
    http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 2405737 lines long.)

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Fri, 2014-03-14 at 16:42 +0000, xen.org wrote:
> flight 25478 linux-linus real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557

Is anyone looking at these? Apparently this hasn't passed for 23 months:
http://xenbits.xen.org/gitweb/?p=linux-pvops.git;a=shortlog;h=refs/heads/tested/linux-linus

Looking through the recent failures this migration one seems quite
common but there seem to be a few others, search for "[linux-linux
test]" in http://lists.xen.org/archives/html/xen-devel/2014-03/ for some
examples.

The particular failure here is
http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/info.html and the console logs http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/serial-gall-mite.log  are full of
        Mar 14 13:24:27.592641 [ 1010.742462] mptsas 0000:03:00.0: swiotlb buffer is full

The migration itself times out after 5 minutes or so (for a 512M guest)

Ian.

Re: [linux-linus test] 25478: regressions - FAIL

[Konrad Rzeszutek Wilk]

On Fri, Mar 14, 2014 at 05:07:35PM +0000, Ian Campbell wrote:
> On Fri, 2014-03-14 at 16:42 +0000, xen.org wrote:
> > flight 25478 linux-linus real [real]
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/
> > 
> > Regressions :-(
> > 
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >  test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
> 
> Is anyone looking at these? Apparently this hasn't passed for 23 months:

I believe I asked to tweak somethings 23 months ago to troubleshoot this
but never heard back from you.

> http://xenbits.xen.org/gitweb/?p=linux-pvops.git;a=shortlog;h=refs/heads/tested/linux-linus
> 
> Looking through the recent failures this migration one seems quite
> common but there seem to be a few others, search for "[linux-linux
> test]" in http://lists.xen.org/archives/html/xen-devel/2014-03/ for some
> examples.
> 
> The particular failure here is
> http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/info.html and the console logs http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/serial-gall-mite.log  are full of

The 'info.html' you alluded - I just saw that happening with Xen 4.5, but
I don't see the SWIOTLB issue in my dom0.

>         Mar 14 13:24:27.592641 [ 1010.742462] mptsas 0000:03:00.0: swiotlb buffer is full
> 
> The migration itself times out after 5 minutes or so (for a 512M guest)

The more recent Linux kernel will also tell you what type of requests it was.
You might also want to try a larger SWIOTLB buffer, swiotlb=26422 for fun.

I think you are looking at two different issues.
> 
> Ian.
> 
> 

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Fri, 2014-03-14 at 14:23 -0400, Konrad Rzeszutek Wilk wrote:
> On Fri, Mar 14, 2014 at 05:07:35PM +0000, Ian Campbell wrote:
> > On Fri, 2014-03-14 at 16:42 +0000, xen.org wrote:
> > > flight 25478 linux-linus real [real]
> > > http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/
> > > 
> > > Regressions :-(
> > > 
> > > Tests which did not succeed and are blocking,
> > > including tests which could not be run:
> > >  test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
> > 
> > Is anyone looking at these? Apparently this hasn't passed for 23 months:
> 
> I believe I asked to tweak somethings 23 months ago to troubleshoot this
> but never heard back from you.

Was that me? I didn't think I had touched osstest at all that long ago
apart from occasionally pinging folks when things looked to be failing.
In any case sorry for letting it all through the cracks. Can you
remember what the tweaks were? (I'm a bit reluctant to play "tweak the
test case until it passes", but lets see what they are first).

What's weird is that the linux-3.4 and linux-3.10 stable branch flights
seem to be passing at least some of the time, although looking at the
history there they do seem to be hitting a failure in the same test
cases at least sometimes.

> > http://xenbits.xen.org/gitweb/?p=linux-pvops.git;a=shortlog;h=refs/heads/tested/linux-linus
> > 
> > Looking through the recent failures this migration one seems quite
> > common but there seem to be a few others, search for "[linux-linux
> > test]" in http://lists.xen.org/archives/html/xen-devel/2014-03/ for some
> > examples.
> > 
> > The particular failure here is
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/info.html and the console logs http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/serial-gall-mite.log  are full of
> 
> The 'info.html' you alluded - I just saw that happening with Xen 4.5, but
> I don't see the SWIOTLB issue in my dom0.
> 
> >         Mar 14 13:24:27.592641 [ 1010.742462] mptsas 0000:03:00.0: swiotlb buffer is full
> > 
> > The migration itself times out after 5 minutes or so (for a 512M guest)
> 
> The more recent Linux kernel

This should be the most recent kernel, it's testing Linus' master. Here
it is v3.14-rc6+ at changeset c60f7d5a8e7c639de5d9dfe07e1e91d302d506e4.

FWIW this happened again in 25558 over the weekend and
http://www.chiark.greenend.org.uk/~xensrcts/logs/25558/test-amd64-i386-pair/serial-gall-mite.log shows some different messages along with the ones quoted above:
        Mar 16 12:51:30.074927 [  845.982443] swiotlb_tbl_map_single: 269 callbacks suppressed
        [...]
        Mar 16 12:51:30.099339 [  845.982464] mptsas 0000:03:00.0: swiotlb buffer is full (sz: 4096 bytes)
AFAICS all of the latter are 4k sized.

I had to go back to
http://www.chiark.greenend.org.uk/~xensrcts/logs/25558/test-amd64-i386-pair/serial-gall-mite.log.0 to find the first such messages, there I see some bnx2 related ones as well e.g.
        Mar 16 12:46:48.315302 [  579.735844] bnx2 0000:02:00.0: swiotlb buffer is full (sz: 21024 bytes)

These all appear to start only after the guest is launched, but there is
no big smoking splat around that time like I was hoping for (i.e. around
"Mar 16 12:46:20.837409 (d1)").

The actual dom0 boot looks ok to me. FWIW:
        Mar 16 12:37:31.756322 [    0.000000] software IO TLB [mem 0x1ba93000-0x1fa93000] (64MB) mapped at [dba93000-dfa92fff]

Interesting that the issues seem to happen on the 64-bit side (this test
is a migration from 64- to 32-bit host). It's also strange that
test-amd64-amd64-xl and other amd64 based tests don't seem to be
affected.

>  will also tell you what type of requests it was.

Do you mean the size? It seems to print that only for certain requests.

> You might also want to try a larger SWIOTLB buffer, swiotlb=26422 for fun.

Any reason for that particular number?

> I think you are looking at two different issues.

You mean you think the swiotlb issue is unrelated to the slow migration
timeouts? I can believe that.

Ian.

Re: [linux-linus test] 25478: regressions - FAIL

[Konrad Rzeszutek Wilk]

On Mon, Mar 17, 2014 at 11:08:25AM +0000, Ian Campbell wrote:
> On Fri, 2014-03-14 at 14:23 -0400, Konrad Rzeszutek Wilk wrote:
> > On Fri, Mar 14, 2014 at 05:07:35PM +0000, Ian Campbell wrote:
> > > On Fri, 2014-03-14 at 16:42 +0000, xen.org wrote:
> > > > flight 25478 linux-linus real [real]
> > > > http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/
> > > > 
> > > > Regressions :-(
> > > > 
> > > > Tests which did not succeed and are blocking,
> > > > including tests which could not be run:
> > > >  test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
> > > 
> > > Is anyone looking at these? Apparently this hasn't passed for 23 months:
> > 
> > I believe I asked to tweak somethings 23 months ago to troubleshoot this
> > but never heard back from you.
> 
> Was that me? I didn't think I had touched osstest at all that long ago
> apart from occasionally pinging folks when things looked to be failing.
> In any case sorry for letting it all through the cracks. Can you
> remember what the tweaks were? (I'm a bit reluctant to play "tweak the
> test case until it passes", but lets see what they are first).
> 
> What's weird is that the linux-3.4 and linux-3.10 stable branch flights
> seem to be passing at least some of the time, although looking at the
> history there they do seem to be hitting a failure in the same test
> cases at least sometimes.
> 
> > > http://xenbits.xen.org/gitweb/?p=linux-pvops.git;a=shortlog;h=refs/heads/tested/linux-linus
> > > 
> > > Looking through the recent failures this migration one seems quite
> > > common but there seem to be a few others, search for "[linux-linux
> > > test]" in http://lists.xen.org/archives/html/xen-devel/2014-03/ for some
> > > examples.
> > > 
> > > The particular failure here is
> > > http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/info.html and the console logs http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/test-amd64-i386-pair/serial-gall-mite.log  are full of
> > 
> > The 'info.html' you alluded - I just saw that happening with Xen 4.5, but
> > I don't see the SWIOTLB issue in my dom0.
> > 
> > >         Mar 14 13:24:27.592641 [ 1010.742462] mptsas 0000:03:00.0: swiotlb buffer is full
> > > 
> > > The migration itself times out after 5 minutes or so (for a 512M guest)
> > 
> > The more recent Linux kernel
> 
> This should be the most recent kernel, it's testing Linus' master. Here
> it is v3.14-rc6+ at changeset c60f7d5a8e7c639de5d9dfe07e1e91d302d506e4.
> 
> FWIW this happened again in 25558 over the weekend and
> http://www.chiark.greenend.org.uk/~xensrcts/logs/25558/test-amd64-i386-pair/serial-gall-mite.log shows some different messages along with the ones quoted above:
>         Mar 16 12:51:30.074927 [  845.982443] swiotlb_tbl_map_single: 269 callbacks suppressed
>         [...]
>         Mar 16 12:51:30.099339 [  845.982464] mptsas 0000:03:00.0: swiotlb buffer is full (sz: 4096 bytes)
> AFAICS all of the latter are 4k sized.
> 
> I had to go back to
> http://www.chiark.greenend.org.uk/~xensrcts/logs/25558/test-amd64-i386-pair/serial-gall-mite.log.0 to find the first such messages, there I see some bnx2 related ones as well e.g.
>         Mar 16 12:46:48.315302 [  579.735844] bnx2 0000:02:00.0: swiotlb buffer is full (sz: 21024 bytes)
> 
> These all appear to start only after the guest is launched, but there is
> no big smoking splat around that time like I was hoping for (i.e. around
> "Mar 16 12:46:20.837409 (d1)").
> 
> The actual dom0 boot looks ok to me. FWIW:
>         Mar 16 12:37:31.756322 [    0.000000] software IO TLB [mem 0x1ba93000-0x1fa93000] (64MB) mapped at [dba93000-dfa92fff]
> 
> Interesting that the issues seem to happen on the 64-bit side (this test
> is a migration from 64- to 32-bit host). It's also strange that
> test-amd64-amd64-xl and other amd64 based tests don't seem to be
> affected.
> 
> >  will also tell you what type of requests it was.
> 
> Do you mean the size? It seems to print that only for certain requests.
> 

Right. They aren't that big - and only enough the mptsas is a 4KB one?

> > You might also want to try a larger SWIOTLB buffer, swiotlb=26422 for fun.
> 
> Any reason for that particular number?

It would allocate a larger SWIOTLB space - in case you are using at near its
capacity.
> 
> > I think you are looking at two different issues.
> 
> You mean you think the swiotlb issue is unrelated to the slow migration
> timeouts? I can believe that.
> 
> Ian.
> 

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Mon, 2014-03-17 at 15:36 -0400, Konrad Rzeszutek Wilk wrote:

> > >  will also tell you what type of requests it was.
> > 
> > Do you mean the size? It seems to print that only for certain requests.
> > 
> 
> Right. They aren't that big - and only enough the mptsas is a 4KB one?

s/only/oddly/?

FWIW across serial-gall-mite.log and .0 I see:
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 10072 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 12920 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 13032 bytes)
      2  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 1448 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 14688 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 15680 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 15928 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 21024 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 26064 bytes)
      3  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 2760 bytes)
     19  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 2896 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 31024 bytes)
      8  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 4344 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 8688 bytes)
  25209  mptsas 0000:03:00.0: swiotlb buffer is full
    878  mptsas 0000:03:00.0: swiotlb buffer is full (sz: 4096 bytes)
      1 serial-gall-mite.log:00:03:00.0: swiotlb buffer is full
      1 serial-gall-mite.log:Mar 16 12:53:55.385556 ptsas 0000:03:00.0: swiotlb buffer is full
      1 serial-gall-mite.log: mptsas 0000:03:00.0: swiotlb buffer is full
      2 serial-gall-mite.log:mptsas 0000:03:00.0: swiotlb buffer is full
      1 serial-gall-mite.log:sas 0000:03:00.0: swiotlb buffer is full

(where the first column is the number of times it was seen. From:
$ wget http://www.chiark.greenend.org.uk/~xensrcts/logs/25558/test-amd64-i386-pair/serial-gall-mite.log.0

        $ grep swiotlb.buffer.is.full serial-gall-mite.log* | cut -f 2 -d \] | sort | uniq -c

> 
> > > You might also want to try a larger SWIOTLB buffer, swiotlb=26422 for fun.
> > 
> > Any reason for that particular number?
> 
> It would allocate a larger SWIOTLB space - in case you are using at near its
> capacity.
> > 
> > > I think you are looking at two different issues.
> > 
> > You mean you think the swiotlb issue is unrelated to the slow migration
> > timeouts? I can believe that.
> > 
> > Ian.
> > 

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Tue, 2014-03-18 at 09:28 +0000, Ian Campbell wrote:

Ignore this one -- hit send too soon. The other reply is complete ;-)

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Mon, 2014-03-17 at 15:36 -0400, Konrad Rzeszutek Wilk wrote:

> > >  will also tell you what type of requests it was.
> > 
> > Do you mean the size? It seems to print that only for certain requests.
> > 
> 
> Right. They aren't that big - and only enough the mptsas is a 4KB one?

s/only/oddly/?

FWIW across serial-gall-mite.log and .0 I see:
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 10072 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 12920 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 13032 bytes)
      2  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 1448 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 14688 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 15680 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 15928 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 21024 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 26064 bytes)
      3  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 2760 bytes)
     19  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 2896 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 31024 bytes)
      8  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 4344 bytes)
      1  bnx2 0000:02:00.0: swiotlb buffer is full (sz: 8688 bytes)
  25209  mptsas 0000:03:00.0: swiotlb buffer is full
    878  mptsas 0000:03:00.0: swiotlb buffer is full (sz: 4096 bytes)
      1 serial-gall-mite.log:00:03:00.0: swiotlb buffer is full
      1 serial-gall-mite.log:Mar 16 12:53:55.385556 ptsas 0000:03:00.0: swiotlb buffer is full
      1 serial-gall-mite.log: mptsas 0000:03:00.0: swiotlb buffer is full
      2 serial-gall-mite.log:mptsas 0000:03:00.0: swiotlb buffer is full
      1 serial-gall-mite.log:sas 0000:03:00.0: swiotlb buffer is full

(where the first column is the number of times it was seen. From:
        $ wget http://www.chiark.greenend.org.uk/~xensrcts/logs/25558/test-amd64-i386-pair/serial-gall-mite.log
        $ wget http://www.chiark.greenend.org.uk/~xensrcts/logs/25558/test-amd64-i386-pair/serial-gall-mite.log.0
        $ grep swiotlb.buffer.is.full serial-gall-mite.log* | cut -f 2 -d \] | sort | uniq -c

> > > You might also want to try a larger SWIOTLB buffer, swiotlb=26422 for fun.
> > 
> > Any reason for that particular number?
> 
> It would allocate a larger SWIOTLB space

I know, but 26422 seems like an odd (non-2^x) number was all. What are
the units?

Ian.

Re: [linux-linus test] 25478: regressions - FAIL

[David Vrabel]

On 18/03/14 09:29, Ian Campbell wrote:
> On Mon, 2014-03-17 at 15:36 -0400, Konrad Rzeszutek Wilk wrote:
> 
>>>>  will also tell you what type of requests it was.
>>>
>>> Do you mean the size? It seems to print that only for certain requests.
>>>
>>
>> Right. They aren't that big - and only enough the mptsas is a 4KB one?
> 
> s/only/oddly/?

Does that card have 64-bit BARs?  It may also be possible that the
driver is setting the wrong DMA mask, basing it on the amount of RAM
instead of the card support.  I seem to recall at least one driver doing
this.

David

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Tue, 2014-03-18 at 10:55 +0000, David Vrabel wrote:
> On 18/03/14 09:29, Ian Campbell wrote:
> > On Mon, 2014-03-17 at 15:36 -0400, Konrad Rzeszutek Wilk wrote:
> > 
> >>>>  will also tell you what type of requests it was.
> >>>
> >>> Do you mean the size? It seems to print that only for certain requests.
> >>>
> >>
> >> Right. They aren't that big - and only enough the mptsas is a 4KB one?
> > 
> > s/only/oddly/?
> 
> Does that card have 64-bit BARs?  It may also be possible that the
> driver is setting the wrong DMA mask, basing it on the amount of RAM
> instead of the card support.  I seem to recall at least one driver doing
> this.

This would be evident in "lspci -vvv" I think? osstest doesn't collect
this right now. I've added it via the patch below for next time.

I thought I could figure it out from the serial logs of dom0 booting,
but I'm not seeing it. I'll see if I can borrow the machine and get back
to you.

BTW, I just noticed a shed load of 
        Mar 16 12:47:54.123819 [  630.035294] swiotlb_tbl_map_single: 359 callbacks suppressed^M

-----8<---------------------------

>From c06c9d212e0c3acad0edbfd993eb016ba2a90d66 Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@citrix.com>
Date: Tue, 18 Mar 2014 10:58:50 +0000
Subject: [OSSTEST PATCH] Capture "lspci -vvv" output.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 ts-logs-capture | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ts-logs-capture b/ts-logs-capture
index a0bd936..62b24d0 100755
--- a/ts-logs-capture
+++ b/ts-logs-capture
@@ -154,6 +154,7 @@ sub fetch_logs_host_guests () {
          'brctl show',
          'lvm lvs',
     'ps wwwaxf -eo pid,tty,stat,time,nice,psr,pcpu,pmem,nwchan,wchan:25,args',
+         'lspci -vvv',
          ) {
             try_cmd_output_save($cmd);
         }
-- 
1.8.5.2

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Tue, 2014-03-18 at 11:04 +0000, Ian Campbell wrote:
> On Tue, 2014-03-18 at 10:55 +0000, David Vrabel wrote:
> > On 18/03/14 09:29, Ian Campbell wrote:
> > > On Mon, 2014-03-17 at 15:36 -0400, Konrad Rzeszutek Wilk wrote:
> > > 
> > >>>>  will also tell you what type of requests it was.
> > >>>
> > >>> Do you mean the size? It seems to print that only for certain requests.
> > >>>
> > >>
> > >> Right. They aren't that big - and only enough the mptsas is a 4KB one?
> > > 
> > > s/only/oddly/?
> > 
> > Does that card have 64-bit BARs?  It may also be possible that the
> > driver is setting the wrong DMA mask, basing it on the amount of RAM
> > instead of the card support.  I seem to recall at least one driver doing
> > this.
> [...] I'll see if I can borrow the machine and get back
> to you.

Here we go. The device in question is:
                03:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
                	Subsystem: Dell SAS 6/iR Integrated Blades RAID Controller
                	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
                	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
                	Latency: 0, Cache Line Size: 64 bytes
                	Interrupt: pin A routed to IRQ 16
                	Region 0: I/O ports at fc00 [size=256]
                	Region 1: Memory at deaec000 (64-bit, non-prefetchable) [size=16K]
                	Region 3: Memory at deaf0000 (64-bit, non-prefetchable) [size=64K]
                	Expansion ROM at de900000 [disabled] [size=1M]
                	Capabilities: [50] Power Management version 2
                		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
                	Capabilities: [68] Express (v1) Endpoint, MSI 00
                		DevCap:	MaxPayload 4096 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
                		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
                			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                			MaxPayload 128 bytes, MaxReadReq 512 bytes
                		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                		LnkCap:	Port #0, Speed 2.5GT/s, Width x8, ASPM L0s L1, Latency L0 <64ns, L1 <1us
                			ClockPM- Surprise- LLActRep- BwNot-
                		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
                			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
                	Capabilities: [98] MSI: Enable- Count=1/1 Maskable- 64bit+
                		Address: 0000000000000000  Data: 0000
                	Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
                		Vector table: BAR=1 offset=00002000
                		PBA: BAR=1 offset=00003000
                	Capabilities: [100 v1] Advanced Error Reporting
                		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                		UESvrt:	DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
                		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                		CEMsk:	RxErr- BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr-
                		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
                	Kernel driver in use: mptsas

There are 64-bit bars but they don't have addresses >4G so I think they
don't count.

Any idea where I can find the DMA mask?

Full lspci below.

Ian.



00:00.0 Host bridge: Intel Corporation Core Processor DMI (rev 11)
	Subsystem: Dell Device 02a3
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Capabilities: [60] MSI: Enable- Count=1/2 Maskable+ 64bit-
		Address: 00000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [90] Express (v1) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
	Capabilities: [e0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol+
		UESvrt:	DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [150 v1] Access Control Services
		ACSCap:	SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
	Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>

00:03.0 PCI bridge: Intel Corporation Core Processor PCI Express Root Port 1 (rev 11) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: fff00000-000fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Subsystem: Dell Device 02a3
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
		Address: fee00000  Data: 40b8
		Masking: 00000002  Pending: 00000000
	Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled+ Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 25.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Off, PwrInd Off, Power+ Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BCD, TimeoutDis+ ARIFwd+
		DevCtl2: Completion Timeout: 1s to 3.5s, TimeoutDis- ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [e0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol+
		UESvrt:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 0e, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [150 v1] Access Control Services
		ACSCap:	SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
	Capabilities: [160 v0] Vendor Specific Information: ID=0002 Rev=0 Len=00c <?>
	Kernel driver in use: pcieport

00:05.0 PCI bridge: Intel Corporation Core Processor PCI Express Root Port 3 (rev 11) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=05, subordinate=08, sec-latency=0
	I/O behind bridge: 0000d000-0000efff
	Memory behind bridge: dd000000-de7fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Subsystem: Dell Device 02a3
	Capabilities: [60] MSI: Enable+ Count=1/2 Maskable+ 64bit-
		Address: fee00000  Data: 40c0
		Masking: 00000002  Pending: 00000000
	Capabilities: [90] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 256 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #3, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #2, PowerLimit 25.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Off, PwrInd Off, Power+ Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BCD, TimeoutDis+ ARIFwd+
		DevCtl2: Completion Timeout: 1s to 3.5s, TimeoutDis- ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [e0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol+
		UESvrt:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 0e, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [150 v1] Access Control Services
		ACSCap:	SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
	Kernel driver in use: pcieport

00:08.0 System peripheral: Intel Corporation Core Processor System Management Registers (rev 11)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0 unlimited, L1 unlimited
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v0] Vendor Specific Information: ID=0000 Rev=0 Len=000 <?>

00:08.1 System peripheral: Intel Corporation Core Processor Semaphore and Scratchpad Registers (rev 11)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0 unlimited, L1 unlimited
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v0] Vendor Specific Information: ID=0000 Rev=0 Len=000 <?>

00:08.2 System peripheral: Intel Corporation Core Processor System Control and Status Registers (rev 11)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Capabilities: [40] Express (v2) Root Complex Integrated Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed unknown, Width x0, ASPM L0s, Latency L0 unlimited, L1 unlimited
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed unknown, Width x0, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v0] Vendor Specific Information: ID=0000 Rev=0 Len=000 <?>

00:08.3 System peripheral: Intel Corporation Core Processor Miscellaneous Registers (rev 11)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

00:10.0 System peripheral: Intel Corporation Core Processor QPI Link (rev 11)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

00:10.1 System peripheral: Intel Corporation Core Processor QPI Routing and Protocol Registers (rev 11)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

00:1a.0 USB controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 05) (prog-if 20 [EHCI])
	Subsystem: Dell Device 02a3
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 22
	Region 0: Memory at de8fc000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ehci_hcd

00:1c.0 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 1 (rev 05) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
	I/O behind bridge: 0000f000-0000ffff
	Memory behind bridge: de900000-deafffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #1, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <1us, L1 <4us
			ClockPM- Surprise- LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+ ARIFwd-
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis- ARIFwd-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00000  Data: 40c8
	Capabilities: [90] Subsystem: Dell Device 02a3
	Capabilities: [a0] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pcieport

00:1c.4 PCI bridge: Intel Corporation 5 Series/3400 Series Chipset PCI Express Root Port 5 (rev 05) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: d8000000-dbffffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #5, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal+ ErrFatal+ PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+ ARIFwd-
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis- ARIFwd-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee00000  Data: 40d0
	Capabilities: [90] Subsystem: Dell Device 02a3
	Capabilities: [a0] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: pcieport

00:1d.0 USB controller: Intel Corporation 5 Series/3400 Series Chipset USB2 Enhanced Host Controller (rev 05) (prog-if 20 [EHCI])
	Subsystem: Dell Device 02a3
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 22
	Region 0: Memory at de8fe000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP+
	Kernel driver in use: ehci_hcd

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5) (prog-if 01 [Subtractive decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=32
	I/O behind bridge: 0000f000-00000fff
	Memory behind bridge: dc000000-dcffffff
	Prefetchable memory behind bridge: 00000000d7800000-00000000d7ffffff
	Secondary status: 66MHz- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA- VGA+ MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] Subsystem: Dell Device 02a3

00:1f.0 ISA bridge: Intel Corporation 3400 Series Chipset LPC Interface Controller (rev 05)
	Subsystem: Dell Device 02a3
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Capabilities: [e0] Vendor Specific Information: Len=10 <?>

00:1f.2 IDE interface: Intel Corporation 5 Series/3400 Series Chipset 4 port SATA IDE Controller (rev 05) (prog-if 8f [Master SecP SecO PriP PriO])
	Subsystem: Dell Device 02a3
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 20
	Region 0: I/O ports at cca0 [size=8]
	Region 1: I/O ports at cc90 [size=4]
	Region 2: I/O ports at cca8 [size=8]
	Region 3: I/O ports at cc94 [size=4]
	Region 4: I/O ports at ccc0 [size=16]
	Region 5: I/O ports at ccd0 [size=16]
	Capabilities: [70] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [b0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ata_piix

00:1f.5 IDE interface: Intel Corporation 5 Series/3400 Series Chipset 2 port SATA IDE Controller (rev 05) (prog-if 85 [Master SecO PriO])
	Subsystem: Dell Device 02a3
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 21
	Region 0: I/O ports at ccb0 [size=8]
	Region 1: I/O ports at cc98 [size=4]
	Region 2: I/O ports at ccb8 [size=8]
	Region 3: I/O ports at cc9c [size=4]
	Region 4: I/O ports at cce0 [size=16]
	Region 5: I/O ports at ccf0 [size=16]
	Capabilities: [70] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [b0] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: ata_piix

01:03.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a) (prog-if 00 [VGA controller])
	Subsystem: Dell Device 02a3
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32 (4000ns min, 8000ns max), Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 6
	Region 0: Memory at d7800000 (32-bit, prefetchable) [size=8M]
	Region 1: Memory at dc7fc000 (32-bit, non-prefetchable) [size=16K]
	Region 2: Memory at dc800000 (32-bit, non-prefetchable) [size=8M]
	[virtual] Expansion ROM at dc000000 [disabled] [size=64K]
	Capabilities: [dc] Power Management version 1
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20)
	Subsystem: Dell Device 02a3
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at d8000000 (64-bit, non-prefetchable) [size=32M]
	Capabilities: [48] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] Vital Product Data
		Product Name: Broadcom NetXtreme II Ethernet Controller
		Read-only fields:
			[PN] Part number: BCM95716C1
			[EC] Engineering changes: 220197-3
			[SN] Serial number: 0123456789
			[MN] Manufacture ID: 31 30 32 38
			[V0] Vendor specific: 5.2.7
			[RV] Reserved: checksum good, 23 byte(s) reserved
		End
	Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [a0] MSI-X: Enable+ Count=9 Masked-
		Vector table: BAR=0 offset=0000c000
		PBA: BAR=0 offset=0000e000
	Capabilities: [ac] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <4us, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <2us, L1 <2us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Device Serial Number a4-ba-db-ff-fe-4d-e3-a0
	Capabilities: [110 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [150 v1] Power Budgeting <?>
	Capabilities: [160 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Kernel driver in use: bnx2

02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20)
	Subsystem: Dell Device 02a3
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 17
	Region 0: Memory at da000000 (64-bit, non-prefetchable) [size=32M]
	Capabilities: [48] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] Vital Product Data
		Product Name: Broadcom NetXtreme II Ethernet Controller
		Read-only fields:
			[PN] Part number: BCM95716C1
			[EC] Engineering changes: 220197-3
			[SN] Serial number: 0123456789
			[MN] Manufacture ID: 31 30 32 38
			[V0] Vendor specific: 5.2.7
			[RV] Reserved: checksum good, 23 byte(s) reserved
		End
	Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [a0] MSI-X: Enable- Count=9 Masked-
		Vector table: BAR=0 offset=0000c000
		PBA: BAR=0 offset=0000e000
	Capabilities: [ac] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <4us, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <2us, L1 <2us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Device Serial Number a4-ba-db-ff-fe-4d-e3-a1
	Capabilities: [110 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [150 v1] Power Budgeting <?>
	Capabilities: [160 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Kernel driver in use: bnx2

03:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
	Subsystem: Dell SAS 6/iR Integrated Blades RAID Controller
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: I/O ports at fc00 [size=256]
	Region 1: Memory at deaec000 (64-bit, non-prefetchable) [size=16K]
	Region 3: Memory at deaf0000 (64-bit, non-prefetchable) [size=64K]
	Expansion ROM at de900000 [disabled] [size=1M]
	Capabilities: [50] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] Express (v1) Endpoint, MSI 00
		DevCap:	MaxPayload 4096 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x8, ASPM L0s L1, Latency L0 <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [98] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
		Vector table: BAR=1 offset=00002000
		PBA: BAR=1 offset=00003000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Kernel driver in use: mptsas

05:00.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=05, secondary=06, subordinate=08, sec-latency=0
	I/O behind bridge: 0000d000-0000efff
	Memory behind bridge: dd000000-de7fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v1) Upstream Port, MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-SlotPowerLimit 25.000W
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 256 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP+ FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=02 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32+ WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Kernel driver in use: pcieport

06:02.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=06, secondary=07, subordinate=07, sec-latency=0
	I/O behind bridge: 0000e000-0000efff
	Memory behind bridge: ddc00000-de7fffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v1) Downstream Port (Slot-), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 256 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee00000  Data: 403b
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP+ FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Kernel driver in use: pcieport

06:04.0 PCI bridge: Integrated Device Technology, Inc. PES12N3A PCI Express Switch (rev 0e) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=06, secondary=08, subordinate=08, sec-latency=0
	I/O behind bridge: 0000d000-0000dfff
	Memory behind bridge: dd000000-ddbfffff
	Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity+ SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v1) Downstream Port (Slot-), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 256 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <512ns, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee00000  Data: 40a3
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP+ FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Kernel driver in use: pcieport

07:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 18
	Region 0: Memory at ddfc0000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at de000000 (32-bit, non-prefetchable) [size=4M]
	Region 2: I/O ports at ecc0 [size=32]
	Region 3: Memory at ddfb8000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #2, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-9c-43-4c
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 1
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 00
		VF offset: 384, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000ddc00000 (64-bit, non-prefetchable)
		Region 3: Memory at 00000000ddc20000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb

07:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 19
	Region 0: Memory at ddfe0000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at de400000 (32-bit, non-prefetchable) [size=4M]
	Region 2: I/O ports at ece0 [size=32]
	Region 3: Memory at ddfbc000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #2, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-9c-43-4c
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 01
		VF offset: 384, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000ddc40000 (64-bit, non-prefetchable)
		Region 3: Memory at 00000000ddc60000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb

08:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at dd3c0000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at dd400000 (32-bit, non-prefetchable) [size=4M]
	Region 2: I/O ports at dcc0 [size=32]
	Region 3: Memory at dd3b8000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-9c-43-4e
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 1
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 00
		VF offset: 384, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000dd000000 (64-bit, non-prefetchable)
		Region 3: Memory at 00000000dd020000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb

08:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Intel Corporation Gigabit ET Quad Port Server Adapter
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 17
	Region 0: Memory at dd3e0000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at dd800000 (32-bit, non-prefetchable) [size=4M]
	Region 2: I/O ports at dce0 [size=32]
	Region 3: Memory at dd3bc000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable+ Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+
		DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-1b-21-ff-ff-9c-43-4e
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function Dependency Link: 01
		VF offset: 384, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000dd040000 (64-bit, non-prefetchable)
		Region 3: Memory at 00000000dd060000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb

ff:00.0 Host bridge: Intel Corporation Core Processor QuickPath Architecture Generic Non-Core Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:00.1 Host bridge: Intel Corporation Core Processor QuickPath Architecture System Address Decoder (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:02.0 Host bridge: Intel Corporation Core Processor QPI Link 0 (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:02.1 Host bridge: Intel Corporation Core Processor QPI Physical 0 (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:03.0 Host bridge: Intel Corporation Core Processor Integrated Memory Controller (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:03.1 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Target Address Decoder (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:03.2 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Test Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:03.4 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Test Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:04.0 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Control Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:04.1 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Address Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:04.2 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Rank Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:04.3 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 0 Thermal Control Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:05.0 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Control Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:05.1 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Address Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:05.2 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Rank Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

ff:05.3 Host bridge: Intel Corporation Core Processor Integrated Memory Controller Channel 1 Thermal Control Registers (rev 04)
	Subsystem: Intel Corporation Device 8086
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0

Re: [linux-linus test] 25478: regressions - FAIL

[David Vrabel]

On 18/03/14 11:10, Ian Campbell wrote:
> 
> There are 64-bit bars but they don't have addresses >4G so I think they
> don't count.

If a device only has 32-bit BARs it's generally not capable of handling
64-bit DMA addresses and we can point and laugh at the card.

mptsas is a driver that uses dma_get_required_mask() to determine a
"suitable" DMA mask -- but under Xen dma_get_required_mask() may return
the wrong mask since it only gets the physical RAM range and not machine
addresses.

David

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Tue, 2014-03-18 at 11:25 +0000, David Vrabel wrote:
> On 18/03/14 11:10, Ian Campbell wrote:
> > 
> > There are 64-bit bars but they don't have addresses >4G so I think they
> > don't count.
> 
> If a device only has 32-bit BARs it's generally not capable of handling
> 64-bit DMA addresses and we can point and laugh at the card.
> 
> mptsas is a driver that uses dma_get_required_mask() to determine a
> "suitable" DMA mask -- but under Xen dma_get_required_mask() may return
> the wrong mask since it only gets the physical RAM range and not machine
> addresses.

Is this the smoking gun then? Do we have any idea what to do about it?

Ian.

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[Ian Jackson]

xen.org writes ("[linux-linus test] 25677: regressions - FAIL"):
> flight 25677 linux-linus real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/25677/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-i386-pair 17 guest-migrate/src_host/dst_host fail REGR. vs. 12557

This is still failing with zillions of this message
  Mar 27 12:25:48.036598 [  850.054546] mptsas 0000:03:00.0: swiotlb buffer is full

Are we any closer to figuring out how to get a fix for this past the
x86 maintainers ?

Does the Linux kernel not have a "no regressions" policy ?  This is a
regression, after all...

Thanks,
Ian.

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[Ian Jackson]

Ian Jackson writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
> xen.org writes ("[linux-linus test] 25677: regressions - FAIL"):
> > flight 25677 linux-linus real [real]
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/25677/
> > 
> > Regressions :-(
> > 
> > Tests which did not succeed and are blocking,
> > including tests which could not be run:
> >  test-amd64-i386-pair 17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
> 
> This is still failing with zillions of this message
>   Mar 27 12:25:48.036598 [  850.054546] mptsas 0000:03:00.0: swiotlb buffer is full
> 
> Are we any closer to figuring out how to get a fix for this past the
> x86 maintainers ?
> 
> Does the Linux kernel not have a "no regressions" policy ?  This is a
> regression, after all...

Another week has gone by and Linux tip still fails this test.

Ian.

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[David Vrabel]

On 16/04/14 11:12, Ian Jackson wrote:
> Ian Jackson writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
>> xen.org writes ("[linux-linus test] 25677: regressions - FAIL"):
>>> flight 25677 linux-linus real [real]
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/25677/
>>>
>>> Regressions :-(
>>>
>>> Tests which did not succeed and are blocking,
>>> including tests which could not be run:
>>>  test-amd64-i386-pair 17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
>>
>> This is still failing with zillions of this message
>>   Mar 27 12:25:48.036598 [  850.054546] mptsas 0000:03:00.0: swiotlb buffer is full
>>
>> Are we any closer to figuring out how to get a fix for this past the
>> x86 maintainers ?
>>
>> Does the Linux kernel not have a "no regressions" policy ?  This is a
>> regression, after all...
> 
> Another week has gone by and Linux tip still fails this test.

Yes, You've not fixed it yet.

David

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[Ian Jackson]

David Vrabel writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
> On 16/04/14 11:12, Ian Jackson wrote:
> > Ian Jackson writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
> >> Are we any closer to figuring out how to get a fix for this past the
> >> x86 maintainers ?
> >>
> >> Does the Linux kernel not have a "no regressions" policy ?  This is a
> >> regression, after all...
> > 
> > Another week has gone by and Linux tip still fails this test.
> 
> Yes, You've not fixed it yet.

My understanding from private emails is that the problem is mostly
political rather than technical.

You wrote:
>> mptsas is a driver that uses dma_get_required_mask() to determine a
>> "suitable" DMA mask -- but under Xen dma_get_required_mask() may return
>> the wrong mask since it only gets the physical RAM range and not machine
>> addresses.

I'm not familiar with the Linux kernel's VM system.  However, if
someone would write a patch which provides an arch override for this
(which from the private emails seems like is what required and would
not be too hard for someone who knew what they were doing), then I can
try to do the political work of negotiating with the Linux community.

Ian.

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[David Vrabel]

On 16/04/14 12:47, Ian Jackson wrote:
> David Vrabel writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
>> On 16/04/14 11:12, Ian Jackson wrote:
>>> Ian Jackson writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
>>>> Are we any closer to figuring out how to get a fix for this past the
>>>> x86 maintainers ?
>>>>
>>>> Does the Linux kernel not have a "no regressions" policy ?  This is a
>>>> regression, after all...
>>>
>>> Another week has gone by and Linux tip still fails this test.
>>
>> Yes, You've not fixed it yet.
> 
> My understanding from private emails is that the problem is mostly
> political rather than technical.
> 
> You wrote:
>>> mptsas is a driver that uses dma_get_required_mask() to determine a
>>> "suitable" DMA mask -- but under Xen dma_get_required_mask() may return
>>> the wrong mask since it only gets the physical RAM range and not machine
>>> addresses.

For this specific problem with the mptsas driver, yes.  But there other
problems with swiotlb usage such as by skbs using compound pages.

We (XenServer) plan to fix all these by making use of the IOMMU. See
Malcolm's recent design doc.  This is still quite a ways off though.

> I'm not familiar with the Linux kernel's VM system.  However, if
> someone would write a patch which provides an arch override for this
> (which from the private emails seems like is what required and would
> not be too hard for someone who knew what they were doing), then I can
> try to do the political work of negotiating with the Linux community.

This fix doesn't require any particular knowledge of the VM subsystem
and there's already some infrastructure for providing an arch specific
implementation of dma_get_required_mask().

David

Re: [linux-linus test] 25478: regressions - FAIL

[David Vrabel]

On 14/03/14 17:07, Ian Campbell wrote:
> On Fri, 2014-03-14 at 16:42 +0000, xen.org wrote:
>> flight 25478 linux-linus real [real]
>> http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>>  test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
> 
> Is anyone looking at these? Apparently this hasn't passed for 23 months:
> http://xenbits.xen.org/gitweb/?p=linux-pvops.git;a=shortlog;h=refs/heads/tested/linux-linus
> 
> Looking through the recent failures this migration one seems quite
> common but there seem to be a few others, search for "[linux-linux
> test]" in http://lists.xen.org/archives/html/xen-devel/2014-03/ for some
> examples.

skb compound pages result in too much SWIOTLB usage.  In XenServer we
have the following to disable it.

net/core: Order-3 frag allocator causes SWIOTLB bouncing under Xen

From: Zoltan Kiss <zoltan.kiss@citrix.com>

THIS PATCH IS NOT INTENDED TO BE UPSTREAMED, IT HAS ONLY INFORMING PURPOSES!

I've noticed a performance regression with upstream kernels when used as
Dom0
under Xen. The classic kernel can utilize the whole bandwidth of a 10G NIC
(ca. 9.3 Gbps), but upstream can reach only ca. 7 Gbps. I found that it
happens because SWIOTLB has to do double buffering. The per task frag
allocator introduced in 5640f7 creates 32 kb frags, which are not contiguous
in mfn space.
This patch provides a workaround by going back to the old way. The possible
ideas came up to solve this:

* make sure Dom0 memory is contiguous: it sounds trivial, but doesn't
work with
driver domains, and there are lots of situations where this is not possible.
* use PVH Dom0: so we will have IOMMU. In the future sometime.
* use IOMMU with PV Dom0: this seems to happen earlier.

Signed-off-by: Zoltan Kiss <zoltan.kiss@citrix.com>

diff --git a/net/core/sock.c b/net/core/sock.c
index d6d024c..44614a5 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -1791,7 +1791,7 @@ struct sk_buff *sock_alloc_send_skb(struct sock
*sk, unsigned long size,
 EXPORT_SYMBOL(sock_alloc_send_skb);

 /* On 32bit arches, an skb frag is limited to 2^15 */
-#define SKB_FRAG_PAGE_ORDER	get_order(32768)
+#define SKB_FRAG_PAGE_ORDER	get_order(4096)

 bool sk_page_frag_refill(struct sock *sk, struct page_frag *pfrag)
 {

Re: [linux-linus test] 25478: regressions - FAIL

[Ian Campbell]

On Mon, 2014-03-17 at 12:01 +0000, David Vrabel wrote:
> On 14/03/14 17:07, Ian Campbell wrote:
> > On Fri, 2014-03-14 at 16:42 +0000, xen.org wrote:
> >> flight 25478 linux-linus real [real]
> >> http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/
> >>
> >> Regressions :-(
> >>
> >> Tests which did not succeed and are blocking,
> >> including tests which could not be run:
> >>  test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
> > 
> > Is anyone looking at these? Apparently this hasn't passed for 23 months:
> > http://xenbits.xen.org/gitweb/?p=linux-pvops.git;a=shortlog;h=refs/heads/tested/linux-linus
> > 
> > Looking through the recent failures this migration one seems quite
> > common but there seem to be a few others, search for "[linux-linux
> > test]" in http://lists.xen.org/archives/html/xen-devel/2014-03/ for some
> > examples.
> 
> skb compound pages result in too much SWIOTLB usage.  In XenServer we
> have the following to disable it.
> 
> net/core: Order-3 frag allocator causes SWIOTLB bouncing under Xen

Ah, I remember this issue (but not this symptom). I thought it had been
fixed but obviously not. (my own skanky patch was rejected but I thought
it had been solved some other way).

The host which is suffering this does have an IOMMU, so Zoltan's third
possibility for a proper fix ought to be plausible, but I suppose dom0
needs to somehow know whether or not to bounce the pages.

> 
> From: Zoltan Kiss <zoltan.kiss@citrix.com>
> 
> THIS PATCH IS NOT INTENDED TO BE UPSTREAMED, IT HAS ONLY INFORMING PURPOSES!
> 
> I've noticed a performance regression with upstream kernels when used as
> Dom0
> under Xen. The classic kernel can utilize the whole bandwidth of a 10G NIC
> (ca. 9.3 Gbps), but upstream can reach only ca. 7 Gbps. I found that it
> happens because SWIOTLB has to do double buffering. The per task frag
> allocator introduced in 5640f7 creates 32 kb frags, which are not contiguous
> in mfn space.
> This patch provides a workaround by going back to the old way. The possible
> ideas came up to solve this:
> 
> * make sure Dom0 memory is contiguous: it sounds trivial, but doesn't
> work with
> driver domains, and there are lots of situations where this is not possible.
> * use PVH Dom0: so we will have IOMMU. In the future sometime.
> * use IOMMU with PV Dom0: this seems to happen earlier.
> 
> Signed-off-by: Zoltan Kiss <zoltan.kiss@citrix.com>
> 
> diff --git a/net/core/sock.c b/net/core/sock.c
> index d6d024c..44614a5 100644
> --- a/net/core/sock.c
> +++ b/net/core/sock.c
> @@ -1791,7 +1791,7 @@ struct sk_buff *sock_alloc_send_skb(struct sock
> *sk, unsigned long size,
>  EXPORT_SYMBOL(sock_alloc_send_skb);
> 
>  /* On 32bit arches, an skb frag is limited to 2^15 */
> -#define SKB_FRAG_PAGE_ORDER	get_order(32768)
> +#define SKB_FRAG_PAGE_ORDER	get_order(4096)
> 
>  bool sk_page_frag_refill(struct sock *sk, struct page_frag *pfrag)
>  {

Re: [linux-linus test] 25478: regressions - FAIL

[Konrad Rzeszutek Wilk]

On Mon, Mar 17, 2014 at 12:01:18PM +0000, David Vrabel wrote:
> On 14/03/14 17:07, Ian Campbell wrote:
> > On Fri, 2014-03-14 at 16:42 +0000, xen.org wrote:
> >> flight 25478 linux-linus real [real]
> >> http://www.chiark.greenend.org.uk/~xensrcts/logs/25478/
> >>
> >> Regressions :-(
> >>
> >> Tests which did not succeed and are blocking,
> >> including tests which could not be run:
> >>  test-amd64-i386-pair   17 guest-migrate/src_host/dst_host fail REGR. vs. 12557
> > 
> > Is anyone looking at these? Apparently this hasn't passed for 23 months:
> > http://xenbits.xen.org/gitweb/?p=linux-pvops.git;a=shortlog;h=refs/heads/tested/linux-linus
> > 
> > Looking through the recent failures this migration one seems quite
> > common but there seem to be a few others, search for "[linux-linux
> > test]" in http://lists.xen.org/archives/html/xen-devel/2014-03/ for some
> > examples.
> 
> skb compound pages result in too much SWIOTLB usage.  In XenServer we
> have the following to disable it.
> 
> net/core: Order-3 frag allocator causes SWIOTLB bouncing under Xen
> 
> From: Zoltan Kiss <zoltan.kiss@citrix.com>
> 
> THIS PATCH IS NOT INTENDED TO BE UPSTREAMED, IT HAS ONLY INFORMING PURPOSES!
> 
> I've noticed a performance regression with upstream kernels when used as
> Dom0
> under Xen. The classic kernel can utilize the whole bandwidth of a 10G NIC
> (ca. 9.3 Gbps), but upstream can reach only ca. 7 Gbps. I found that it
> happens because SWIOTLB has to do double buffering. The per task frag
> allocator introduced in 5640f7 creates 32 kb frags, which are not contiguous
> in mfn space.
> This patch provides a workaround by going back to the old way. The possible
> ideas came up to solve this:
> 
> * make sure Dom0 memory is contiguous: it sounds trivial, but doesn't
> work with
> driver domains, and there are lots of situations where this is not possible.
> * use PVH Dom0: so we will have IOMMU. In the future sometime.
> * use IOMMU with PV Dom0: this seems to happen earlier.

Hm, any sign of the patches from Malcom?

> 
> Signed-off-by: Zoltan Kiss <zoltan.kiss@citrix.com>
> 
> diff --git a/net/core/sock.c b/net/core/sock.c
> index d6d024c..44614a5 100644
> --- a/net/core/sock.c
> +++ b/net/core/sock.c
> @@ -1791,7 +1791,7 @@ struct sk_buff *sock_alloc_send_skb(struct sock
> *sk, unsigned long size,
>  EXPORT_SYMBOL(sock_alloc_send_skb);
> 
>  /* On 32bit arches, an skb frag is limited to 2^15 */
> -#define SKB_FRAG_PAGE_ORDER	get_order(32768)
> +#define SKB_FRAG_PAGE_ORDER	get_order(4096)
> 
>  bool sk_page_frag_refill(struct sock *sk, struct page_frag *pfrag)
>  {

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[Konrad Rzeszutek Wilk]

On Apr 16, 2014 6:12 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
>
> Ian Jackson writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"): 
> > xen.org writes ("[linux-linus test] 25677: regressions - FAIL"): 
> > > flight 25677 linux-linus real [real] 
> > > http://www.chiark.greenend.org.uk/~xensrcts/logs/25677/ 
> > > 
> > > Regressions :-( 
> > > 
> > > Tests which did not succeed and are blocking, 
> > > including tests which could not be run: 
> > >  test-amd64-i386-pair 17 guest-migrate/src_host/dst_host fail REGR. vs. 12557 
> > 
> > This is still failing with zillions of this message 
> >   Mar 27 12:25:48.036598 [  850.054546] mptsas 0000:03:00.0: swiotlb buffer is full 
> > 
> > Are we any closer to figuring out how to get a fix for this past the 
> > x86 maintainers ? 
> > 
> > Does the Linux kernel not have a "no regressions" policy ?  This is a 
> > regression, after all... 
>
> Another week has gone by and Linux tip still fails this test.

Did you try the recommendation that David offered - boot with 4GB to dom0?

>
> Ian. 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[Ian Jackson]

Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
> On Apr 16, 2014 6:12 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
> > Another week has gone by and Linux tip still fails this test.
> 
> Did you try the recommendation that David offered - boot with 4GB to dom0?

That would be a workaround.  I can do that (as it happens these
machines have 8G of RAM so it is actually possible), but:

The purpose of running tests is to discover bugs (so that they can be
fixed).  It is not to generate a nice clean report by sweeping things
under the carpet.

Here the tests have discovered a bug in Linux.  AIUI it's a bug which
is visible when using this particular driver, but which is actually a
problem with the Xen integration into the Linux VM system in general.
So other it presumably affects other drivers too.  It should be fixed,
not worked around.

The downside of not working around this bug is that osstest's
failing-host-stickiness will cause an increasing proportion of the
tests to run on the affected hosts.  This might mask other bugs.

Conversely, working around this bug in the manner suggested will
presumably just make the bug disappear off our radar.

Ian.

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[Konrad Rzeszutek Wilk]

On Wed, Apr 16, 2014 at 01:06:06PM +0100, Ian Jackson wrote:
> Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"):
> > On Apr 16, 2014 6:12 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
> > > Another week has gone by and Linux tip still fails this test.
> > 
> > Did you try the recommendation that David offered - boot with 4GB to dom0?
> 
> That would be a workaround.  I can do that (as it happens these
> machines have 8G of RAM so it is actually possible), but:

Right.
> 
> The purpose of running tests is to discover bugs (so that they can be
> fixed).  It is not to generate a nice clean report by sweeping things
> under the carpet.

Right.
> 
> Here the tests have discovered a bug in Linux.  AIUI it's a bug which
> is visible when using this particular driver, but which is actually a
> problem with the Xen integration into the Linux VM system in general.
> So other it presumably affects other drivers too.  It should be fixed,
> not worked around.

I concur.
> 
> The downside of not working around this bug is that osstest's
> failing-host-stickiness will cause an increasing proportion of the
> tests to run on the affected hosts.  This might mask other bugs.
> 
> Conversely, working around this bug in the manner suggested will
> presumably just make the bug disappear off our radar.

There is a danger of that.

I am not going to be able to take a look at this bug in the next three 
weeks. If anybody else wants to take a stab at this - here is my hand-waving
idea of how it could be done:

1). Cleanup the ia64 usage of same code 'ia64_dma_get_required_mask'
    (it actually duplicates what the drivers/base has). Perhaps move it to
    lib/iommu-helper.c
    Call it 'generic_get_required_mask' or such. 
2). Look at other platforms that use a similar (or the same) code
    and see if they can re-use it now that it is in lib/iommu-helper.c.
3). Make the x86 dma_ops start using the extra "get_required_mask"
    and point it to the generic_get_required_mask.
4). Make the xen-swiotlb use its own version. 

> 
> Ian.

Re: [linux-linus test] 25478: regressions - FAIL [and 1 more messages]

[Konrad Rzeszutek Wilk]

On Apr 16, 2014 7:47 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
>
> David Vrabel writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"): 
> > On 16/04/14 11:12, Ian Jackson wrote: 
> > > Ian Jackson writes ("Re: [Xen-devel] [linux-linus test] 25478: regressions - FAIL [and 1 more messages]"): 
> > >> Are we any closer to figuring out how to get a fix for this past the 
> > >> x86 maintainers ? 
> > >> 
> > >> Does the Linux kernel not have a "no regressions" policy ?  This is a 
> > >> regression, after all... 
> > > 
> > > Another week has gone by and Linux tip still fails this test. 
> > 
> > Yes, You've not fixed it yet. 
>
> My understanding from private emails is that the problem is mostly 
> political rather than technical. 
>
> You wrote: 
> >> mptsas is a driver that uses dma_get_required_mask() to determine a 
> >> "suitable" DMA mask -- but under Xen dma_get_required_mask() may return 
> >> the wrong mask since it only gets the physical RAM range and not machine 
> >> addresses. 
>
> I'm not familiar with the Linux kernel's VM system.  However, if 
> someone would write a patch which provides an arch override for this 
> (which from the private emails seems like is what required and would 
> not be too hard for someone who knew what they were doing), then I can 
> try to do the political work of negotiating with the Linux community. 
>

Sure. But before we go that route - can you try the suggestion given by David to see if does indeed 'fix' the issue.

> Ian. 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel