[Bug 15362] New: MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[Bug 15362] New: MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362

           Summary: MPT Fusion SCSI drives no longer appear - suspect PCI
                    bus scan bug
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.30-1, 2.6.30-2, 2.6.32-trunk
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: linux-scsi@vger.kernel.org
        ReportedBy: pegasus@renegadetech.com
        Regression: Yes


Created an attachment (id=25130)
 --> (http://bugzilla.kernel.org/attachment.cgi?id=25130)
2.6.26 dmesg

Since upgrading to the 2.6.30 kernel on an AMD64 platform, drives attached to
my LSI/MPT SCSI controller are no longer visible. If I boot using the 2.6.26-2
kernel, it works fine. The SCSI controller doesn't even show up in lspci in
kernel versions above 2.6.26. (I do have the controller's BIOS disabled
however, but I understand that doesn't matter since the kernel will poll it
anyway, as 2.6.26 does. I tested with the controller's BIOS enabled too and it
doesn't make a difference.)

This has already been submitted to Debian as bug #543308, but evidence points
to a bug in the kernel PCI bus scanning code, since the following PCI devices
show up on 2.6.26-2 but not 2.6.30-1 and up:

0001:40:01.0 0604: 1022:7450 (rev 12)
0001:40:01.1 0800: 1022:7451 (rev 01)
0001:40:02.0 0604: 1022:7450 (rev 12)
0001:40:02.1 0800: 1022:7451 (rev 01)
0001:61:06.0 0100: 1000:0030 (rev 07)
0001:61:06.1 0100: 1000:0030 (rev 07)
0002:80:00.0 0580: 10de:005e (rev a3)
0002:80:01.0 0580: 10de:00d3 (rev a3)

-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362





--- Comment #1 from Sean M. Pappalardo <pegasus@renegadetech.com>  2010-02-20 15:05:37 ---
Created an attachment (id=25131)
 --> (http://bugzilla.kernel.org/attachment.cgi?id=25131)
lspci -n on 2.6.26

-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362





--- Comment #2 from Sean M. Pappalardo <pegasus@renegadetech.com>  2010-02-20 15:06:08 ---
Created an attachment (id=25132)
 --> (http://bugzilla.kernel.org/attachment.cgi?id=25132)
2.6.26 lspci -v

-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362





--- Comment #3 from Sean M. Pappalardo <pegasus@renegadetech.com>  2010-02-20 15:07:10 ---
Created an attachment (id=25133)
 --> (http://bugzilla.kernel.org/attachment.cgi?id=25133)
2.6.32 dmesg

-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362





--- Comment #4 from Sean M. Pappalardo <pegasus@renegadetech.com>  2010-02-20 15:07:48 ---
Created an attachment (id=25134)
 --> (http://bugzilla.kernel.org/attachment.cgi?id=25134)
2.6.32 lspci -n

-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362


Sean M. Pappalardo <pegasus@renegadetech.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #25131|lspci -n on 2.6.26          |2.6.26 lspci -n
        description|                            |




-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362





--- Comment #5 from Sean M. Pappalardo <pegasus@renegadetech.com>  2010-02-20 15:10:08 ---
I have dmesg and lspci -n and -v from 2.6.30 as well but they're similar to
2.6.32. Just let me know if you want them too.

-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[bugzilla-daemon]

http://bugzilla.kernel.org/show_bug.cgi?id=15362


Matthew Wilcox <matthew@wil.cx> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |matthew@wil.cx
          Component|SCSI                        |PCI
         AssignedTo|linux-scsi@vger.kernel.org  |drivers_pci@kernel-bugs.osd
                   |                            |l.org
            Product|IO/Storage                  |Drivers




--- Comment #6 from Matthew Wilcox <matthew@wil.cx>  2010-02-20 15:22:09 ---
OK, this is clearly a PCI issue, nothing to do with SCSI.

It looks like everything outside domain 0 is now not found.  This seems to be
due to the MMCONFIG access option not being used.

-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

Re: [Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[Bjorn Helgaas]

> I do still plan to work on bisecting this.

Can you also attach a dmesg log from a current kernel, e.g., 3.4 or
newer?  We now print a lot more information during PCI enumeration.

But I guess the problem is that 2.6.26 finds devices in domains 1 and
2, while 2.6.32 does not.  I think MMCONFIG is the only config access
method we have for domains other than 0.  That suggests that MMCONFIG
used to work but doesn't any more.  The dmesg logs claim that we're
not using MMCONFIG in either 2.6.26 or 2.6.32 though, so I don't know
why we found anything in 2.6.26.

Re: [Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[Yinghai Lu]

On Wed, Jun 27, 2012 at 7:32 AM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>> I do still plan to work on bisecting this.
>
> Can you also attach a dmesg log from a current kernel, e.g., 3.4 or
> newer?  We now print a lot more information during PCI enumeration.
>
> But I guess the problem is that 2.6.26 finds devices in domains 1 and
> 2, while 2.6.32 does not.  I think MMCONFIG is the only config access
> method we have for domains other than 0.  That suggests that MMCONFIG
> used to work but doesn't any more.  The dmesg logs claim that we're
> not using MMCONFIG in either 2.6.26 or 2.6.32 though, so I don't know
> why we found anything in 2.6.26.

in short: the bios is broken, it return wrong segment in DSDT.

in both case, only pci_conf1 is used.  CPU is not new enough.

after comparing the code 2.6.26 and 2.6.32.  2.6.26 is not checking
seg in pci_conf1_read. but 2.6.32 check that...

2.6.26:
static int pci_conf1_read(unsigned int seg, unsigned int bus,
                          unsigned int devfn, int reg, int len, u32 *value)
{
        unsigned long flags;

        if ((bus > 255) || (devfn > 255) || (reg > 255)) {
                *value = -1;
                return -EINVAL;
        }

2.6.32
20 static int pci_conf1_read(unsigned int seg, unsigned int bus,
21                           unsigned int devfn, int reg, int len, u32 *value)
22 {
23         unsigned long flags;
24
25         if (seg || (bus > 255) || (devfn > 255) || (reg > 4095)) {
26                 *value = -1;
27                 return -EINVAL;
28         }

so it happens to work on 2.6.26.

please get to get new BIOS from your vendor.
or you need to override your DSDT.

Thanks

Yinghai

Re: [Bug 15362] MPT Fusion SCSI drives no longer appear - suspect PCI bus scan bug

[Bjorn Helgaas]

> in short: the bios is broken, it return wrong segment in DSDT.

I *think* what Yinghai is saying is:

  - MMCONFIG is not used either in 2.6.26 or 2.6.32.
  - BIOS reports these host bridges via DSDT PNP0A08 devices:
        [PCI0] leading to segment 0000 bus 00
        [PCI1] leading to segment 0001 bus 40
        [PCI2] leading to segment 0002 bus 80
  - Buses 40 and 80 are actually in segment 0, not segments 1 and 2.
  - When we enumerate bus 40 and bus 80, we pass seg=1 and seg=2,
respectively, to pci_conf1_read(), but 2.6.26 ignores seg.  For
example, when we think we're reading 0001:40:01.0 config space, 2.6.26
actually reads 0000:40:01.0 config space instead.
  - In 2.6.32, instead of ignoring seg, we return an error if it is
not zero.  Therefore, we fail to find anything on bus 40 and bus 80.

Sean, what system and BIOS version is this?  (The 3.4.x dmesg log or
the "dmidecode" output will contain this information.)  I don't expect
HP to change the BIOS, and it wouldn't be reasonable to require users
to debug this issue and upgrade their BIOS in any case.

But I would like to read the release notes or help text that mentions
this issue.  If all the buses were in fact in segment 0, the DSDT
would typically not have any _SEG methods at all, because segment 0 is
the default.  Yinghai is assuming that HP went to the trouble to *add*
_SEG methods that returned incorrect values.  But the fact that HP was
aware of the issue and provided the BIOS "disable ACPI bus
segmentation" option makes it less likely that this is the case.

Also, the system was very likely tested with Windows, and the fact
that the BIOS option is to *disable* segmentation suggests that the
default is "segmentation enabled."  So my guess is that segmentation
does work with Windows.  Sean, can you confirm or deny that?  The
AIDA64 tool (free trial version at http://www.aida64.com/) generates a
report with useful information.

I agree with Jonathan's assertion here:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=543308#87 that the
BIOS switch is not adequate.  Neither is a patched DSDT.

I think it's likely that Windows works with segmentation, using
MMCONFIG, and that Linux is a bit too quick to disable MMCONFIG in
this case.