RE: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

RE: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Allen Martin]

> 
> NForce4 PCI Express is an unknown - we'll see how that works.
> 

MSI is not officially supported on nForce4 and hasn't been fully tested.

However it should work ok for PCI-E.  I would definitely not recommend
enabling 
MSI for the internal MAC though.

-Allen
-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Andi Kleen]

"Allen Martin" <AMartin@nvidia.com> writes:

> > 
> > NForce4 PCI Express is an unknown - we'll see how that works.
> > 
> 
> MSI is not officially supported on nForce4 and hasn't been fully tested.

Ok thanks for the information. We should definitely disable it by default
then, maybe with an boot option so that the speed-over-stability crowd
can enable it (+ possibly an oops taint bit) 

-Andi

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Dave Olson]

On Wed, 21 Jun 2006, Andi Kleen wrote:

| "Allen Martin" <AMartin@nvidia.com> writes:
| 
| > > 
| > > NForce4 PCI Express is an unknown - we'll see how that works.
| > > 
| > 
| > MSI is not officially supported on nForce4 and hasn't been fully tested.
| 
| Ok thanks for the information. We should definitely disable it by default
| then, maybe with an boot option so that the speed-over-stability crowd
| can enable it (+ possibly an oops taint bit) 

Why disable it, when it's clearly working?   Disable it for the
onboard ethernet, perhaps, but as far as I know, nobody has reported
any MSI issues on nforce4?   I've been in the vendor position often
enough to know that "not supported" doesn't mean "known to not work".

Dave Olson
olson@unixfolk.com
http://www.unixfolk.com/dave

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Andi Kleen]

On Wednesday 21 June 2006 18:21, Dave Olson wrote:
> On Wed, 21 Jun 2006, Andi Kleen wrote:
> 
> | "Allen Martin" <AMartin@nvidia.com> writes:
> | 
> | > > 
> | > > NForce4 PCI Express is an unknown - we'll see how that works.
> | > > 
> | > 
> | > MSI is not officially supported on nForce4 and hasn't been fully tested.
> | 
> | Ok thanks for the information. We should definitely disable it by default
> | then, maybe with an boot option so that the speed-over-stability crowd
> | can enable it (+ possibly an oops taint bit) 
> 
> Why disable it, when it's clearly working?  

Because I don't think normal Linux users should be in the hardware validation
business. If the vendor says it's not tested we shouldn't enable it by default.

-Andi

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Greg Lindahl]

On Wed, Jun 21, 2006 at 06:37:30PM +0200, Andi Kleen wrote:

> Because I don't think normal Linux users should be in the hardware validation
> business. If the vendor says it's not tested we shouldn't enable it by default.

All you're going to do is piss off all the distros and hardare vendors
shipping systems with this chip in it. Nvidia is not the only "vendor"
involved, and is not the only one doing hardware validation. Sun, HP,
Newisys, and LNXI all have currently shipping Opteron products with a
CK804.  All 4 are probably going to want your employer's distro to not
disable MSI.

-- greg

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Naren (Narendra) Sankar]

> BCM seems to need a blacklist to force MSI off (or at least tg3 is
complaining
> that MSI doesn't work). I guess we can try to contact someone at BCM
> and ask them if they actually tested it. If they did then enabling it
would
> be fine. 

Hi Everyone

I just got forwarded this discussion on LKML. Unfortunately I am not
subscribed to the mailing list.

I want to clarify that all of Broadcom's AMD chipsets do support MSI and
have done so from the very beginning. The only issue is that we did not
enable the support by default in the hardware. However some platform
vendors are choosing for their own reasons not to use the BIOS required
programming to enable the MSI support in our chipsets. So the current
state of the market is that we have some systems out there that support
MSI on our chipsets and others that do not. We are trying to work with
every vendor to make sure this gets enabled.

MSI is fully tested on our chipsets and the kernel can safely turn it on
on any platform.

The BIOS/kernel programming requirement is that the HyperTransport MSI
mapping be enabled in all the PCIX and PCIe bridges of our chipsets.
Hence PCI register 0xa2 needs to be set to 1, in the standard PCI space
of all our Bridges. This is true for the HT2000, the HT2100 and the
HT1000.

Please cc me if there are any questions.

Thanks
 
Naren Sankar
System Architect
Chipset Product Line
Broadcom Corporation
2451 Mission College Blvd
Santa Clara, CA 95054
Office:  (408) 922-7143
Email:  nsankar@broadcom.com <mailto:nsankar@broadcom.com>  

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities

[Andi Kleen]

> Sure, that's true of almost everything new.   It remains broken until people
> start using it, complain, and get the bugs fixed.   Some of us have a vested
> interest in having MSI work, since it's the only way we can deliver interrupts.
> We've already worked with a few BIOS writers to get problems fixed, and we'll
> keep doing so as we find problems.   If enough hardware vendors and consumers
> do so, the broken stuff will get fixed, and stay fixed, because it will get
> tested.

Sometimes there are new things that work relatively well and only break occassionally
and then there are things where it seems to be the other way round.

My point was basically that every time we tried to turn on such a banana green feature
without white listing it was a sea of pain to get it to work everywhere
and tended to cause far too many non boots

(and any non boot is far worse than whatever performance advantage you get
from it) 

So if there are any more MSI problems comming up IMHO it should be white list/disabled 
by default and only turn on after a long time when Windows uses it by default 
or something. Greg, do you agree?

-Andi

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilities

[Greg KH]

On Tue, Jun 20, 2006 at 09:25:30AM +0200, Andi Kleen wrote:
> 
> > Sure, that's true of almost everything new.   It remains broken until people
> > start using it, complain, and get the bugs fixed.   Some of us have a vested
> > interest in having MSI work, since it's the only way we can deliver interrupts.
> > We've already worked with a few BIOS writers to get problems fixed, and we'll
> > keep doing so as we find problems.   If enough hardware vendors and consumers
> > do so, the broken stuff will get fixed, and stay fixed, because it will get
> > tested.
> 
> Sometimes there are new things that work relatively well and only break occassionally
> and then there are things where it seems to be the other way round.
> 
> My point was basically that every time we tried to turn on such a banana green feature
> without white listing it was a sea of pain to get it to work everywhere
> and tended to cause far too many non boots
> 
> (and any non boot is far worse than whatever performance advantage you get
> from it) 
> 
> So if there are any more MSI problems comming up IMHO it should be
> white list/disabled by default and only turn on after a long time when
> Windows uses it by default or something. Greg, do you agree?

No, I don't want a whitelist, as it will be hard to always keep adding
stuff to it (unless we can somehow figure out how to put a "cut-off"
date check in there).  Yes, we do have a number of systems today that
have MSI issues, but the newer ones all work properly, and we should
continue on with the way we have today (blasklist problem boards, as the
rest of the PCI subsystem works with the quirks).

thanks,

greg "the optimist" k-h

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Andi Kleen]

On Tuesday 20 June 2006 23:29, Greg KH wrote:
> On Tue, Jun 20, 2006 at 09:25:30AM +0200, Andi Kleen wrote:
> > 
> > > Sure, that's true of almost everything new.   It remains broken until people
> > > start using it, complain, and get the bugs fixed.   Some of us have a vested
> > > interest in having MSI work, since it's the only way we can deliver interrupts.
> > > We've already worked with a few BIOS writers to get problems fixed, and we'll
> > > keep doing so as we find problems.   If enough hardware vendors and consumers
> > > do so, the broken stuff will get fixed, and stay fixed, because it will get
> > > tested.
> > 
> > Sometimes there are new things that work relatively well and only break occassionally
> > and then there are things where it seems to be the other way round.
> > 
> > My point was basically that every time we tried to turn on such a banana green feature
> > without white listing it was a sea of pain to get it to work everywhere
> > and tended to cause far too many non boots
> > 
> > (and any non boot is far worse than whatever performance advantage you get
> > from it) 
> > 
> > So if there are any more MSI problems comming up IMHO it should be
> > white list/disabled by default and only turn on after a long time when
> > Windows uses it by default or something. Greg, do you agree?
> 
> No, I don't want a whitelist, as it will be hard to always keep adding
> stuff to it (unless we can somehow figure out how to put a "cut-off"
> date check in there).  Yes, we do have a number of systems today that
> have MSI issues, but the newer ones all work properly, and we should
> continue on with the way we have today (blasklist problem boards, as the
> rest of the PCI subsystem works with the quirks).

On Intel chipsets just enabling it is fine - i haven't heard of a MSI problem
there so far. Detecting Intel chipsets can be tricky though - there are
AMD systems with Intel PCI bridges now. I suppose any blacklist will be 
per bridge.

Just on AMD there seems to be no PCI-X bridge that actually works with MSI without
hacks and PCI-E seems to be quite spotty too (e.g. BCM at least partly broken) 

Brice claims BCM can be fixed with a quirk and Petr claimed AMD 8132 can be fixed
with a quirk, but my personal feeling is that it is very risky to do so because
if these bits are not enabled by default they are likely unvalidated and might break
in subtle ways under high load (past experiences with forcing hardware features
on against BIOS wishes were usually negative) 

The good message is that AMD without quirk they don't do anything so it would just
not be enabled and at least not break anything.

BCM seems to need a blacklist to force MSI off (or at least tg3 is complaining
that MSI doesn't work). I guess we can try to contact someone at BCM
and ask them if they actually tested it. If they did then enabling it would
be fine.

NForce4 PCI Express is an unknown - we'll see how that works.

-Andi

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Roland Dreier]

 > NForce4 PCI Express is an unknown - we'll see how that works.

I have systems (HP DL145) with

    PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3)

and MSI-X works fine for me with Mellanox PCIe adapters (with no
quirks or anything -- the BIOS enables it by default):

    $ grep MSI-X /proc/interrupts
     66:     205792          0          0          0       PCI-MSI-X ib_mthca (comp)
     74:          1          0          0          0       PCI-MSI-X ib_mthca (async)
     82:       1343          0          0          0       PCI-MSI-X ib_mthca (cmd)

 - R.

Re: [discuss] Re: [RFC] Whitelist chipsets supporting MSI and check Hyper-transport capabilitiesKJ

[Dave Olson]

On Tue, 20 Jun 2006, Roland Dreier wrote:

|  > NForce4 PCI Express is an unknown - we'll see how that works.
| 
| I have systems (HP DL145) with
| 
|     PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3)
| 
| and MSI-X works fine for me with Mellanox PCIe adapters (with no
| quirks or anything -- the BIOS enables it by default):

And for us, as well, for the InfiniPath PCIe cards (MSI, not MSI-X),
in multiple motherboards.

Dave Olson
olson@unixfolk.com
http://www.unixfolk.com/dave