public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
       [not found]   ` <20060619005329.GA1425@greglaptop>
@ 2006-06-19  8:28     ` Andi Kleen
  2006-06-19 12:52       ` Brice Goglin
  2006-06-19 15:47       ` Greg Lindahl
  0 siblings, 2 replies; 25+ messages in thread
From: Andi Kleen @ 2006-06-19  8:28 UTC (permalink / raw)
  To: discuss, Brice Goglin, linux-kernel; +Cc: Greg Lindahl

[you destroyed the cc list. Don't do that]

On Monday 19 June 2006 02:53, Greg Lindahl wrote:
> On Sat, Jun 17, 2006 at 10:48:18AM -0400, Brice Goglin wrote:
> > IIRC, HT1000 is the southbridge part of the HT2000 chipset. We have been
> > told that MSI works on this chipset. And from what we've seen/tested, it
> > is true.
>
> We can also say this is true -- our InfiniPath HCA requires MSI, so we
> really notice this.

Isn't your HCA directly connected to HTX? If yes it will
likely not run into PCI bridge bugs.

We got a Serverworks based Supermicro system where the driver for the 
integrated tg3 NIC complains about MSI not working. So either that particular
system has a specific BIOS issue or it is broken for on motherboard
devices.
 
I was extrapolating from that.

It doesn't complain on another Supermicro system, but that seems
to be because that particular BCM570x silicon revision is blacklisted
for MSI in the tg3 driver.

>From that experience I certainly cannot say that MSI works 
very well on Serverworks so far. It might be safer to blacklist
until someone can explain what's going on on these systems.
The problem is that not all drivers do the MSI probing tg3 do,
so if you got a system where MSI doesn't work. but it tells
the kernel it does, and a driver turns on MSI things just
break.

For me so far it looks like MSI is another HPET or mmconfig - something
that looks nice in the specification, but hardware/BIOS developers
don't seem to pay particular attention to and is ridden with 
platform bugs.

-Andi

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-19  8:28     ` [discuss] " Andi Kleen
@ 2006-06-19 12:52       ` Brice Goglin
  2006-06-19 15:47       ` Greg Lindahl
  1 sibling, 0 replies; 25+ messages in thread
From: Brice Goglin @ 2006-06-19 12:52 UTC (permalink / raw)
  To: Andi Kleen; +Cc: discuss, linux-kernel, Greg Lindahl

Andi Kleen wrote:
> We got a Serverworks based Supermicro system where the driver for the 
> integrated tg3 NIC complains about MSI not working. So either that particular
> system has a specific BIOS issue or it is broken for on motherboard
> devices.
>  
> I was extrapolating from that.
>
> It doesn't complain on another Supermicro system, but that seems
> to be because that particular BCM570x silicon revision is blacklisted
> for MSI in the tg3 driver.
>
> From that experience I certainly cannot say that MSI works 
> very well on Serverworks so far. It might be safer to blacklist
> until someone can explain what's going on on these systems.
> The problem is that not all drivers do the MSI probing tg3 do,
> so if you got a system where MSI doesn't work. but it tells
> the kernel it does, and a driver turns on MSI things just
> break.
>   

Initially, we thought MSI did not work. But, we actually discovered that
it was caused by the MSI cap being disabled in the Hypertransport config
space. Enabling the MSI cap there seems to make it work.

My patches check the HT MSI cap in a quirk. So you will only get MSI for
your driver if the BIOS/chipset has been correctly initialized.

The next step would be to explicitely set the MSI cap in the
Hypertransport config space in my quirk. I have patches to do so, but
I'd rather wait until my current patches to are stabilized/merged.

Brice


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-19  8:28     ` [discuss] " Andi Kleen
  2006-06-19 12:52       ` Brice Goglin
@ 2006-06-19 15:47       ` Greg Lindahl
  1 sibling, 0 replies; 25+ messages in thread
From: Greg Lindahl @ 2006-06-19 15:47 UTC (permalink / raw)
  To: Andi Kleen; +Cc: discuss, Brice Goglin, linux-kernel

On Mon, Jun 19, 2006 at 10:28:50AM +0200, Andi Kleen wrote:

> Isn't your HCA directly connected to HTX? If yes it will
> likely not run into PCI bridge bugs.

I was referring to our new PCI Express HCA.

-- greg


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
       [not found] ` <fa.URgTUhhO9H/aLp98XyIN2gzSppk@ifi.uio.no>
@ 2006-06-20  5:42   ` Dave Olson
  2006-06-20  7:25     ` Andi Kleen
  0 siblings, 1 reply; 25+ messages in thread
From: Dave Olson @ 2006-06-20  5:42 UTC (permalink / raw)
  To: Andi Kleen; +Cc: discuss, Brice Goglin, linux-kernel, Greg Lindahl

On Mon, 19 Jun 2006, Andi Kleen wrote:

| [you destroyed the cc list. Don't do that]
| 
| On Monday 19 June 2006 02:53, Greg Lindahl wrote:
| > On Sat, Jun 17, 2006 at 10:48:18AM -0400, Brice Goglin wrote:
| > > IIRC, HT1000 is the southbridge part of the HT2000 chipset. We have been
| > > told that MSI works on this chipset. And from what we've seen/tested, it
| > > is true.
| >
| > We can also say this is true -- our InfiniPath HCA requires MSI, so we
| > really notice this.
| 
| Isn't your HCA directly connected to HTX? If yes it will
| likely not run into PCI bridge bugs.

The HyperTransport is directly attached, but it still uses parts of
the MSI infrastructure.   Mostly we've direct-coded it, but at some
point the MSI code should be expanded to support HTX devices as well.

| We got a Serverworks based Supermicro system where the driver for the 
| integrated tg3 NIC complains about MSI not working. So either that particular
| system has a specific BIOS issue or it is broken for on motherboard
| devices.
|  
| I was extrapolating from that.

I've used our PCIe card on the serverworks HT2000/ST2000 chipset with MSI, and it seems
to work fine (once you enable MSI in the config space, which the current
BIOS doesn't seem to do for some reason).  It may be that it doesn't work
well with PCI, but does with PCIe (different parts of the chipset, after all,
so it seems possible).

| For me so far it looks like MSI is another HPET or mmconfig - something
| that looks nice in the specification, but hardware/BIOS developers
| don't seem to pay particular attention to and is ridden with 
| platform bugs.

Sure, that's true of almost everything new.   It remains broken until people
start using it, complain, and get the bugs fixed.   Some of us have a vested
interest in having MSI work, since it's the only way we can deliver interrupts.
We've already worked with a few BIOS writers to get problems fixed, and we'll
keep doing so as we find problems.   If enough hardware vendors and consumers
do so, the broken stuff will get fixed, and stay fixed, because it will get
tested.


Dave Olson
olson@unixfolk.com
http://www.unixfolk.com/dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20  5:42   ` [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities Dave Olson
@ 2006-06-20  7:25     ` Andi Kleen
  2006-06-20  8:02       ` Jeff Garzik
                         ` (2 more replies)
  0 siblings, 3 replies; 25+ messages in thread
From: Andi Kleen @ 2006-06-20  7:25 UTC (permalink / raw)
  To: Dave Olson; +Cc: discuss, Brice Goglin, linux-kernel, Greg Lindahl, gregkh


> Sure, that's true of almost everything new.   It remains broken until people
> start using it, complain, and get the bugs fixed.   Some of us have a vested
> interest in having MSI work, since it's the only way we can deliver interrupts.
> We've already worked with a few BIOS writers to get problems fixed, and we'll
> keep doing so as we find problems.   If enough hardware vendors and consumers
> do so, the broken stuff will get fixed, and stay fixed, because it will get
> tested.

Sometimes there are new things that work relatively well and only break occassionally
and then there are things where it seems to be the other way round.

My point was basically that every time we tried to turn on such a banana green feature
without white listing it was a sea of pain to get it to work everywhere
and tended to cause far too many non boots

(and any non boot is far worse than whatever performance advantage you get
from it) 

So if there are any more MSI problems comming up IMHO it should be white list/disabled 
by default and only turn on after a long time when Windows uses it by default 
or something. Greg, do you agree?

-Andi

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20  7:25     ` Andi Kleen
@ 2006-06-20  8:02       ` Jeff Garzik
  2006-06-20  8:13         ` Andi Kleen
  2006-06-20 20:03       ` Greg Lindahl
  2006-06-20 21:29       ` Greg KH
  2 siblings, 1 reply; 25+ messages in thread
From: Jeff Garzik @ 2006-06-20  8:02 UTC (permalink / raw)
  To: Andi Kleen
  Cc: Dave Olson, discuss, Brice Goglin, linux-kernel, Greg Lindahl,
	gregkh

Andi Kleen wrote:
> So if there are any more MSI problems comming up IMHO it should be white list/disabled 
> by default and only turn on after a long time when Windows uses it by default 
> or something. Greg, do you agree?


We should be optimists, not pessimists.

MSI is useful enough that we should turn it on by default in newer systems.

	Jeff



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20  8:02       ` Jeff Garzik
@ 2006-06-20  8:13         ` Andi Kleen
  2006-06-20  8:27           ` Jeff Garzik
  0 siblings, 1 reply; 25+ messages in thread
From: Andi Kleen @ 2006-06-20  8:13 UTC (permalink / raw)
  To: discuss
  Cc: Jeff Garzik, Dave Olson, Brice Goglin, linux-kernel, Greg Lindahl,
	gregkh

On Tuesday 20 June 2006 10:02, Jeff Garzik wrote:
> Andi Kleen wrote:
> > So if there are any more MSI problems comming up IMHO it should be white list/disabled 
> > by default and only turn on after a long time when Windows uses it by default 
> > or something. Greg, do you agree?
> 
> 
> We should be optimists, not pessimists.

Yes, booting on all systems is overrated anyways, isn't it?

> 
> MSI is useful enough that we should turn it on by default in newer systems.

That is what we've tried so far and it seems to not work.

-Andi

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20  8:13         ` Andi Kleen
@ 2006-06-20  8:27           ` Jeff Garzik
  2006-06-20 22:44             ` Eric W. Biederman
  0 siblings, 1 reply; 25+ messages in thread
From: Jeff Garzik @ 2006-06-20  8:27 UTC (permalink / raw)
  To: Andi Kleen
  Cc: discuss, Dave Olson, Brice Goglin, linux-kernel, Greg Lindahl,
	gregkh

Andi Kleen wrote:
> On Tuesday 20 June 2006 10:02, Jeff Garzik wrote:
>> Andi Kleen wrote:
>>> So if there are any more MSI problems comming up IMHO it should be white list/disabled 
>>> by default and only turn on after a long time when Windows uses it by default 
>>> or something. Greg, do you agree?
>>
>> We should be optimists, not pessimists.
> 
> Yes, booting on all systems is overrated anyways, isn't it?

Don't be silly.  Whatever solution is arrived at will boot on all 
systems.  That's an obvious operational requirement.

This is how new technology always works in Linux.  We turn it on and see 
what works, and what doesn't.  And whether existing problems will 
disappear.  With MSI, I think we see them disappearing.

Newer systems seem to be doing better with MSI, in part because 
PCI-Express and other technologies trend towards MSI-style operation.

And the kernel's MSI code is finally getting cleaned up, and getting the 
attention it needs.


>> MSI is useful enough that we should turn it on by default in newer systems.
> 
> That is what we've tried so far and it seems to not work.

IMO that's an exaggeration.  On 50% of the x86-64 platforms (Intel), MSI 
has been working for quite some time.  On newer systems in the other 
half of the platforms, MSI seems be more usable than it has been in the 
past.

	Jeff



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20  7:25     ` Andi Kleen
  2006-06-20  8:02       ` Jeff Garzik
@ 2006-06-20 20:03       ` Greg Lindahl
  2006-06-20 20:20         ` Randy.Dunlap
  2006-06-20 20:23         ` Roland Dreier
  2006-06-20 21:29       ` Greg KH
  2 siblings, 2 replies; 25+ messages in thread
From: Greg Lindahl @ 2006-06-20 20:03 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Dave Olson, discuss, Brice Goglin, linux-kernel, gregkh

On Tue, Jun 20, 2006 at 09:25:30AM +0200, Andi Kleen wrote:

> So if there are any more MSI problems comming up IMHO it should be white list/disabled 
> by default and only turn on after a long time when Windows uses it by default 
> or something. Greg, do you agree?

You probably meant the other Greg, but the situation doesn't seem to
be that bad. Brice's proposed whitelist covers almost all Opteron PCI
Express servers.

What is the list of things which are known to have problems? All
PCI-X?  We can ask some more people with PCI-X MSI cards what works
for them, i.e.  Mellanox.

-- greg


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:03       ` Greg Lindahl
@ 2006-06-20 20:20         ` Randy.Dunlap
  2006-06-20 20:26           ` Brice Goglin
  2006-06-20 20:41           ` Greg Lindahl
  2006-06-20 20:23         ` Roland Dreier
  1 sibling, 2 replies; 25+ messages in thread
From: Randy.Dunlap @ 2006-06-20 20:20 UTC (permalink / raw)
  To: Greg Lindahl; +Cc: ak, olson, discuss, brice, linux-kernel, gregkh

On Tue, 20 Jun 2006 13:03:52 -0700 Greg Lindahl wrote:

> On Tue, Jun 20, 2006 at 09:25:30AM +0200, Andi Kleen wrote:
> 
> > So if there are any more MSI problems comming up IMHO it should be white list/disabled 
> > by default and only turn on after a long time when Windows uses it by default 
> > or something. Greg, do you agree?
> 
> You probably meant the other Greg, but the situation doesn't seem to
> be that bad. Brice's proposed whitelist covers almost all Opteron PCI
> Express servers.

Why "almost"?  What does a user do if his/hers is not covered?
Does it cover the 10 new models that are available tomorrow?
(hypothetical question)

> What is the list of things which are known to have problems? All
> PCI-X?  We can ask some more people with PCI-X MSI cards what works
> for them, i.e.  Mellanox.

---
~Randy

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:03       ` Greg Lindahl
  2006-06-20 20:20         ` Randy.Dunlap
@ 2006-06-20 20:23         ` Roland Dreier
  2006-06-20 20:54           ` Jeff Garzik
  1 sibling, 1 reply; 25+ messages in thread
From: Roland Dreier @ 2006-06-20 20:23 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Dave Olson, discuss, Brice Goglin, linux-kernel, gregkh

    Greg> What is the list of things which are known to have problems?
    Greg> All PCI-X?  We can ask some more people with PCI-X MSI cards
    Greg> what works for them, i.e.  Mellanox.

Actually most PCI-X works as well.  AMD 8131 PCI-X bridges don't work,
and it seems (based on the quirk in the kernel) that there are broken
Serverworks chipsets, but I've never actually seen motherboards with
that on there.

But for example I have an old Xeon motherboard which lspci says has

    PCI bridge: Intel Corporation 82870P2 P64H2 Hub PCI Bridge (rev 03)

and MSI works fine there.

There seem to be two issues here though.  First, MSI interrupts don't
always work, because of chipset bugs, BIOS bugs, etc.  This is fairly
manageable because the worst case is usually a single device not
generating interrupts.

However, the other issue is that CONFIG_PCI_MSI forces some other
changes to x86 interrupt handling, even if no devices will ever use
MSI.  And the changes are such that some systems can't even boot with
CONFIG_PCI_MSI enabled.  This is the more severe problem, which needs
to be handled if you want distros to turn on MSI.

 - R.

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:20         ` Randy.Dunlap
@ 2006-06-20 20:26           ` Brice Goglin
  2006-06-20 20:41           ` Greg Lindahl
  1 sibling, 0 replies; 25+ messages in thread
From: Brice Goglin @ 2006-06-20 20:26 UTC (permalink / raw)
  To: Randy.Dunlap; +Cc: Greg Lindahl, ak, olson, discuss, linux-kernel, gregkh

Randy.Dunlap wrote:
> On Tue, 20 Jun 2006 13:03:52 -0700 Greg Lindahl wrote:
>   
>> You probably meant the other Greg, but the situation doesn't seem to
>> be that bad. Brice's proposed whitelist covers almost all Opteron PCI
>> Express servers.
>>     
>
> Why "almost"?  What does a user do if his/hers is not covered?
> Does it cover the 10 new models that are available tomorrow?
> (hypothetical question)
>   

At least, it will not be worse than currently for new PCI-E chipsets (my
patch still enables MSI by default for these).

But, my quirks to check the MSI cap in the HT mapping might need to be
enabled for more chipsets (I only handle nVidia Ck804 and ServerWorks
HT2000).

Brice


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:20         ` Randy.Dunlap
  2006-06-20 20:26           ` Brice Goglin
@ 2006-06-20 20:41           ` Greg Lindahl
  2006-06-20 20:50             ` Brice Goglin
  1 sibling, 1 reply; 25+ messages in thread
From: Greg Lindahl @ 2006-06-20 20:41 UTC (permalink / raw)
  To: Randy.Dunlap; +Cc: ak, olson, discuss, brice, linux-kernel, gregkh

On Tue, Jun 20, 2006 at 01:20:49PM -0700, Randy.Dunlap wrote:

> Does it cover the 10 new models that are available tomorrow?
> (hypothetical question)

I see several people on this list who are aware of forthcoming PCI
Express server products, and they aren't objecting to broad
whitelisting of PCI Express.

Is there any known case of PCI Express and MSI not working on any
chipset? Andi, is the tg3 NIC that didn't work in a Supermicro system
on PCI-X or PCI Express?

-- greg



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:41           ` Greg Lindahl
@ 2006-06-20 20:50             ` Brice Goglin
  2006-06-20 20:53               ` Jeff Garzik
  2006-06-20 20:57               ` Dave Olson
  0 siblings, 2 replies; 25+ messages in thread
From: Brice Goglin @ 2006-06-20 20:50 UTC (permalink / raw)
  To: greg.lindahl; +Cc: Randy.Dunlap, ak, olson, discuss, linux-kernel, gregkh

Greg Lindahl wrote:
> Andi, is the tg3 NIC that didn't work in a Supermicro system
> on PCI-X or PCI Express?
>   

IIRC, Andi was talking about a Supermicro machine with a ServerWorks
HT2000 chipset. We have such a machine here. Its MSI is disabled in the
Hyper-transport mapping. But, MSI works once the HT capability is
enabled (and my quirk will detect it right).
For such machines, if people really want MSI, we'll need to enable the
HT cap in my quirk. But, as long as they just want IRQ to work,
detecting whether the HT cap is enabled or not should be enough.

Brice


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:50             ` Brice Goglin
@ 2006-06-20 20:53               ` Jeff Garzik
  2006-06-20 20:57               ` Dave Olson
  1 sibling, 0 replies; 25+ messages in thread
From: Jeff Garzik @ 2006-06-20 20:53 UTC (permalink / raw)
  To: Brice Goglin
  Cc: greg.lindahl, Randy.Dunlap, ak, olson, discuss, linux-kernel,
	gregkh

Brice Goglin wrote:
> Greg Lindahl wrote:
>> Andi, is the tg3 NIC that didn't work in a Supermicro system
>> on PCI-X or PCI Express?
>>   
> 
> IIRC, Andi was talking about a Supermicro machine with a ServerWorks
> HT2000 chipset. We have such a machine here. Its MSI is disabled in the
> Hyper-transport mapping. But, MSI works once the HT capability is
> enabled (and my quirk will detect it right).
> For such machines, if people really want MSI, we'll need to enable the
> HT cap in my quirk. But, as long as they just want IRQ to work,
> detecting whether the HT cap is enabled or not should be enough.

If it works reliably, we should definitely turn it on.

	Jeff, wishing his HT1000 did the same




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:23         ` Roland Dreier
@ 2006-06-20 20:54           ` Jeff Garzik
  0 siblings, 0 replies; 25+ messages in thread
From: Jeff Garzik @ 2006-06-20 20:54 UTC (permalink / raw)
  To: Roland Dreier
  Cc: Andi Kleen, Dave Olson, discuss, Brice Goglin, linux-kernel,
	gregkh

Roland Dreier wrote:
> However, the other issue is that CONFIG_PCI_MSI forces some other
> changes to x86 interrupt handling, even if no devices will ever use
> MSI.  And the changes are such that some systems can't even boot with
> CONFIG_PCI_MSI enabled.  This is the more severe problem, which needs
> to be handled if you want distros to turn on MSI.

Agreed.  We definitely want CONFIG_PCI_MSI-enabled kernels to work in 
all situations.

	Jeff



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 20:50             ` Brice Goglin
  2006-06-20 20:53               ` Jeff Garzik
@ 2006-06-20 20:57               ` Dave Olson
  1 sibling, 0 replies; 25+ messages in thread
From: Dave Olson @ 2006-06-20 20:57 UTC (permalink / raw)
  To: Brice Goglin
  Cc: greg.lindahl, Randy.Dunlap, ak, discuss, linux-kernel, gregkh

On Tue, 20 Jun 2006, Brice Goglin wrote:

| Greg Lindahl wrote:
| > Andi, is the tg3 NIC that didn't work in a Supermicro system
| > on PCI-X or PCI Express?
| >   
| 
| IIRC, Andi was talking about a Supermicro machine with a ServerWorks
| HT2000 chipset. We have such a machine here. Its MSI is disabled in the
| Hyper-transport mapping. But, MSI works once the HT capability is
| enabled (and my quirk will detect it right).
| For such machines, if people really want MSI, we'll need to enable the
| HT cap in my quirk. But, as long as they just want IRQ to work,
| detecting whether the HT cap is enabled or not should be enough.

We definitely want that chipset to be on the whitelist.   It's used in
supermicro systems, the Mac systems with the 970 chip, and some large
vendor systems as well.

The detection of MSI not being enabled in the config space is 
a good thing to do (and warn about), of course, no problem with that.

I've seen no MSI-related issues with the ST2000 on 3 different
platforms, so far.

Dave Olson
olson@unixfolk.com

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20  7:25     ` Andi Kleen
  2006-06-20  8:02       ` Jeff Garzik
  2006-06-20 20:03       ` Greg Lindahl
@ 2006-06-20 21:29       ` Greg KH
  2006-06-20 22:27         ` Brice Goglin
  2006-06-20 22:33         ` [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ Andi Kleen
  2 siblings, 2 replies; 25+ messages in thread
From: Greg KH @ 2006-06-20 21:29 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Dave Olson, discuss, Brice Goglin, linux-kernel, Greg Lindahl

On Tue, Jun 20, 2006 at 09:25:30AM +0200, Andi Kleen wrote:
> 
> > Sure, that's true of almost everything new.   It remains broken until people
> > start using it, complain, and get the bugs fixed.   Some of us have a vested
> > interest in having MSI work, since it's the only way we can deliver interrupts.
> > We've already worked with a few BIOS writers to get problems fixed, and we'll
> > keep doing so as we find problems.   If enough hardware vendors and consumers
> > do so, the broken stuff will get fixed, and stay fixed, because it will get
> > tested.
> 
> Sometimes there are new things that work relatively well and only break occassionally
> and then there are things where it seems to be the other way round.
> 
> My point was basically that every time we tried to turn on such a banana green feature
> without white listing it was a sea of pain to get it to work everywhere
> and tended to cause far too many non boots
> 
> (and any non boot is far worse than whatever performance advantage you get
> from it) 
> 
> So if there are any more MSI problems comming up IMHO it should be
> white list/disabled by default and only turn on after a long time when
> Windows uses it by default or something. Greg, do you agree?

No, I don't want a whitelist, as it will be hard to always keep adding
stuff to it (unless we can somehow figure out how to put a "cut-off"
date check in there).  Yes, we do have a number of systems today that
have MSI issues, but the newer ones all work properly, and we should
continue on with the way we have today (blasklist problem boards, as the
rest of the PCI subsystem works with the quirks).

thanks,

greg "the optimist" k-h

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 21:29       ` Greg KH
@ 2006-06-20 22:27         ` Brice Goglin
  2006-06-20 23:05           ` Greg KH
  2006-06-20 22:33         ` [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ Andi Kleen
  1 sibling, 1 reply; 25+ messages in thread
From: Brice Goglin @ 2006-06-20 22:27 UTC (permalink / raw)
  To: Greg KH; +Cc: Andi Kleen, Dave Olson, discuss, linux-kernel, Greg Lindahl

Greg KH wrote:
> No, I don't want a whitelist, as it will be hard to always keep adding
> stuff to it (unless we can somehow figure out how to put a "cut-off"
> date check in there).

My second patchset (Improve MSI detection v2) uses "PCI-E vs non-PCI-E"
as a cut-off "date". After reading all what people said in this thread,
I still think it is a good compromise (and very simple to implement) if
we blacklist PCI-E and whitelist non-PCI-E chipsets.

Brice


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ
  2006-06-20 21:29       ` Greg KH
  2006-06-20 22:27         ` Brice Goglin
@ 2006-06-20 22:33         ` Andi Kleen
  2006-06-20 22:46           ` Roland Dreier
  1 sibling, 1 reply; 25+ messages in thread
From: Andi Kleen @ 2006-06-20 22:33 UTC (permalink / raw)
  To: Greg KH; +Cc: Dave Olson, discuss, Brice Goglin, linux-kernel, Greg Lindahl

On Tuesday 20 June 2006 23:29, Greg KH wrote:
> On Tue, Jun 20, 2006 at 09:25:30AM +0200, Andi Kleen wrote:
> > 
> > > Sure, that's true of almost everything new.   It remains broken until people
> > > start using it, complain, and get the bugs fixed.   Some of us have a vested
> > > interest in having MSI work, since it's the only way we can deliver interrupts.
> > > We've already worked with a few BIOS writers to get problems fixed, and we'll
> > > keep doing so as we find problems.   If enough hardware vendors and consumers
> > > do so, the broken stuff will get fixed, and stay fixed, because it will get
> > > tested.
> > 
> > Sometimes there are new things that work relatively well and only break occassionally
> > and then there are things where it seems to be the other way round.
> > 
> > My point was basically that every time we tried to turn on such a banana green feature
> > without white listing it was a sea of pain to get it to work everywhere
> > and tended to cause far too many non boots
> > 
> > (and any non boot is far worse than whatever performance advantage you get
> > from it) 
> > 
> > So if there are any more MSI problems comming up IMHO it should be
> > white list/disabled by default and only turn on after a long time when
> > Windows uses it by default or something. Greg, do you agree?
> 
> No, I don't want a whitelist, as it will be hard to always keep adding
> stuff to it (unless we can somehow figure out how to put a "cut-off"
> date check in there).  Yes, we do have a number of systems today that
> have MSI issues, but the newer ones all work properly, and we should
> continue on with the way we have today (blasklist problem boards, as the
> rest of the PCI subsystem works with the quirks).

On Intel chipsets just enabling it is fine - i haven't heard of a MSI problem
there so far. Detecting Intel chipsets can be tricky though - there are
AMD systems with Intel PCI bridges now. I suppose any blacklist will be 
per bridge.

Just on AMD there seems to be no PCI-X bridge that actually works with MSI without
hacks and PCI-E seems to be quite spotty too (e.g. BCM at least partly broken) 

Brice claims BCM can be fixed with a quirk and Petr claimed AMD 8132 can be fixed
with a quirk, but my personal feeling is that it is very risky to do so because
if these bits are not enabled by default they are likely unvalidated and might break
in subtle ways under high load (past experiences with forcing hardware features
on against BIOS wishes were usually negative) 

The good message is that AMD without quirk they don't do anything so it would just
not be enabled and at least not break anything.

BCM seems to need a blacklist to force MSI off (or at least tg3 is complaining
that MSI doesn't work). I guess we can try to contact someone at BCM
and ask them if they actually tested it. If they did then enabling it would
be fine.

NForce4 PCI Express is an unknown - we'll see how that works.

-Andi

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20  8:27           ` Jeff Garzik
@ 2006-06-20 22:44             ` Eric W. Biederman
  0 siblings, 0 replies; 25+ messages in thread
From: Eric W. Biederman @ 2006-06-20 22:44 UTC (permalink / raw)
  To: Jeff Garzik
  Cc: Andi Kleen, discuss, Dave Olson, Brice Goglin, linux-kernel,
	Greg Lindahl, gregkh

Jeff Garzik <jeff@garzik.org> writes:

> Andi Kleen wrote:
>> On Tuesday 20 June 2006 10:02, Jeff Garzik wrote:
>>> Andi Kleen wrote:
>>>> So if there are any more MSI problems comming up IMHO it should be white
>>>> list/disabled by default and only turn on after a long time when Windows
>>>> uses it by default or something. Greg, do you agree?
>>>
>>> We should be optimists, not pessimists.
>> Yes, booting on all systems is overrated anyways, isn't it?
>
> Don't be silly.  Whatever solution is arrived at will boot on all systems.
> That's an obvious operational requirement.
>
> This is how new technology always works in Linux.  We turn it on and see what
> works, and what doesn't.  And whether existing problems will disappear.  With
> MSI, I think we see them disappearing.
>
> Newer systems seem to be doing better with MSI, in part because PCI-Express and
> other technologies trend towards MSI-style operation.
>
> And the kernel's MSI code is finally getting cleaned up, and getting the
> attention it needs.
>
>
>>> MSI is useful enough that we should turn it on by default in newer systems.
>> That is what we've tried so far and it seems to not work.
>
> IMO that's an exaggeration.  On 50% of the x86-64 platforms (Intel), MSI has
> been working for quite some time.  On newer systems in the other half of the
> platforms, MSI seems be more usable than it has been in the past.

It would help if the msi code as more than a 3 year old proof of concept
that has intel assumptions all through it.  Things are getting better
but there are still issues there.

I suspect that on x86 hypertransport systems if we directed our writes
to 0xFDF8000000 instead of at 0xfee0000 we would have quite a bit more luck.
Hopefully I can test and confirm that soon.  It is still a trick to get
a card with a working MSI implementation.

My gut feel is what we want is not a while list or a black list but instead
a MSI window driver.  The location and the meaning of the window is on the
edge of being an architectural definition.  But it really isn't and we are
treating it like one.

So I think part of the reason we are having MSI is we are making unwarranted
assumptions.  Once we take a good look at our side and fix that then we
can tell if something was broken.

Eric

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ
  2006-06-20 22:33         ` [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ Andi Kleen
@ 2006-06-20 22:46           ` Roland Dreier
  2006-06-21  6:19             ` Dave Olson
  0 siblings, 1 reply; 25+ messages in thread
From: Roland Dreier @ 2006-06-20 22:46 UTC (permalink / raw)
  To: Andi Kleen
  Cc: Greg KH, Dave Olson, discuss, Brice Goglin, linux-kernel,
	Greg Lindahl

 > NForce4 PCI Express is an unknown - we'll see how that works.

I have systems (HP DL145) with

    PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3)

and MSI-X works fine for me with Mellanox PCIe adapters (with no
quirks or anything -- the BIOS enables it by default):

    $ grep MSI-X /proc/interrupts
     66:     205792          0          0          0       PCI-MSI-X ib_mthca (comp)
     74:          1          0          0          0       PCI-MSI-X ib_mthca (async)
     82:       1343          0          0          0       PCI-MSI-X ib_mthca (cmd)

 - R.

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 22:27         ` Brice Goglin
@ 2006-06-20 23:05           ` Greg KH
  2006-06-20 23:16             ` Brice Goglin
  0 siblings, 1 reply; 25+ messages in thread
From: Greg KH @ 2006-06-20 23:05 UTC (permalink / raw)
  To: Brice Goglin; +Cc: Andi Kleen, Dave Olson, discuss, linux-kernel, Greg Lindahl

On Tue, Jun 20, 2006 at 06:27:45PM -0400, Brice Goglin wrote:
> Greg KH wrote:
> > No, I don't want a whitelist, as it will be hard to always keep adding
> > stuff to it (unless we can somehow figure out how to put a "cut-off"
> > date check in there).
> 
> My second patchset (Improve MSI detection v2) uses "PCI-E vs non-PCI-E"
> as a cut-off "date". After reading all what people said in this thread,
> I still think it is a good compromise (and very simple to implement) if
> we blacklist PCI-E and whitelist non-PCI-E chipsets.

No, that's not fair for those devices which do not have PCI-E and yet
have MSI (the original ones, that work just fine...)

Again, no "whitelist" please, just quirks to fix problems with ones that
we know we have problems with, just like all other PCI quirks...

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities
  2006-06-20 23:05           ` Greg KH
@ 2006-06-20 23:16             ` Brice Goglin
  0 siblings, 0 replies; 25+ messages in thread
From: Brice Goglin @ 2006-06-20 23:16 UTC (permalink / raw)
  To: Greg KH; +Cc: Andi Kleen, Dave Olson, discuss, linux-kernel, Greg Lindahl

Greg KH wrote:
> No, that's not fair for those devices which do not have PCI-E and yet
> have MSI (the original ones, that work just fine...)
>
> Again, no "whitelist" please, just quirks to fix problems with ones that
> we know we have problems with, just like all other PCI quirks...
>
> thanks,
>
> greg k-h
>   

Ok I will regenerate my patches to do so.

Brice


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ
  2006-06-20 22:46           ` Roland Dreier
@ 2006-06-21  6:19             ` Dave Olson
  0 siblings, 0 replies; 25+ messages in thread
From: Dave Olson @ 2006-06-21  6:19 UTC (permalink / raw)
  To: Roland Dreier
  Cc: Andi Kleen, Greg KH, discuss, Brice Goglin, linux-kernel,
	Greg Lindahl

On Tue, 20 Jun 2006, Roland Dreier wrote:

|  > NForce4 PCI Express is an unknown - we'll see how that works.
| 
| I have systems (HP DL145) with
| 
|     PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3)
| 
| and MSI-X works fine for me with Mellanox PCIe adapters (with no
| quirks or anything -- the BIOS enables it by default):

And for us, as well, for the InfiniPath PCIe cards (MSI, not MSI-X),
in multiple motherboards.

Dave Olson
olson@unixfolk.com
http://www.unixfolk.com/dave

^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2006-06-21  6:19 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <fa.5FgZbVFZIyOdjQ3utdNvbqTrUq0@ifi.uio.no>
     [not found] ` <fa.URgTUhhO9H/aLp98XyIN2gzSppk@ifi.uio.no>
2006-06-20  5:42   ` [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities Dave Olson
2006-06-20  7:25     ` Andi Kleen
2006-06-20  8:02       ` Jeff Garzik
2006-06-20  8:13         ` Andi Kleen
2006-06-20  8:27           ` Jeff Garzik
2006-06-20 22:44             ` Eric W. Biederman
2006-06-20 20:03       ` Greg Lindahl
2006-06-20 20:20         ` Randy.Dunlap
2006-06-20 20:26           ` Brice Goglin
2006-06-20 20:41           ` Greg Lindahl
2006-06-20 20:50             ` Brice Goglin
2006-06-20 20:53               ` Jeff Garzik
2006-06-20 20:57               ` Dave Olson
2006-06-20 20:23         ` Roland Dreier
2006-06-20 20:54           ` Jeff Garzik
2006-06-20 21:29       ` Greg KH
2006-06-20 22:27         ` Brice Goglin
2006-06-20 23:05           ` Greg KH
2006-06-20 23:16             ` Brice Goglin
2006-06-20 22:33         ` [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ Andi Kleen
2006-06-20 22:46           ` Roland Dreier
2006-06-21  6:19             ` Dave Olson
2006-06-17  3:01 [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities Brice Goglin
2006-06-17 14:48 ` Brice Goglin
     [not found]   ` <20060619005329.GA1425@greglaptop>
2006-06-19  8:28     ` [discuss] " Andi Kleen
2006-06-19 12:52       ` Brice Goglin
2006-06-19 15:47       ` Greg Lindahl

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox