[RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[Adrian Bunk]

We had during the last months have quite a few MSI bugs and even 
regressions due to:
- core kernel bugs,
- device driver bugs and
- hardware bugs

OTOH, MSI doesn't bring any real advantages for most users.

Let's therefore mark PCI_MSI as EXPERIMENTAL.

Signed-off-by: Adrian Bunk <bunk@stusta.de>

---
--- linux-2.6.21-rc4-mm1/drivers/pci/Kconfig.old	2007-03-26 21:04:11.000000000 +0200
+++ linux-2.6.21-rc4-mm1/drivers/pci/Kconfig	2007-03-26 21:04:29.000000000 +0200
@@ -2,8 +2,8 @@
 # PCI configuration
 #
 config PCI_MSI
-	bool "Message Signaled Interrupts (MSI and MSI-X)"
-	depends on PCI
+	bool "Message Signaled Interrupts (MSI and MSI-X)  (EXPERIMENTAL)"
+	depends on PCI && EXPERIMENTAL
 	depends on (X86_LOCAL_APIC && X86_IO_APIC) || IA64 || (PPC_MERGE && PPC_RTAS) || SPARC64
 	help
 	   This allows device drivers to enable MSI (Message Signaled

Re: [RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[David Miller]

From: Adrian Bunk <bunk@stusta.de>
Date: Tue, 27 Mar 2007 03:02:47 +0200

> We had during the last months have quite a few MSI bugs and even 
> regressions due to:
> - core kernel bugs,
> - device driver bugs and
> - hardware bugs
> 
> OTOH, MSI doesn't bring any real advantages for most users.
> 
> Let's therefore mark PCI_MSI as EXPERIMENTAL.
> 
> Signed-off-by: Adrian Bunk <bunk@stusta.de>

This is a good way to ensure that the code doesn't get tested
enough to ever fix the problems.

Re: [RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[Greg KH]

On Mon, Mar 26, 2007 at 06:24:22PM -0700, David Miller wrote:
> From: Adrian Bunk <bunk@stusta.de>
> Date: Tue, 27 Mar 2007 03:02:47 +0200
> 
> > We had during the last months have quite a few MSI bugs and even 
> > regressions due to:
> > - core kernel bugs,
> > - device driver bugs and
> > - hardware bugs
> > 
> > OTOH, MSI doesn't bring any real advantages for most users.
> > 
> > Let's therefore mark PCI_MSI as EXPERIMENTAL.
> > 
> > Signed-off-by: Adrian Bunk <bunk@stusta.de>
> 
> This is a good way to ensure that the code doesn't get tested
> enough to ever fix the problems.

I agree, we need to leave it as it is.  Now that Vista is out with
support for MSI, we will start to get more hardware that actually works
properly with it...

thanks,

greg k-h

Re: [RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[Eric W. Biederman]

David Miller <davem@davemloft.net> writes:

> From: Adrian Bunk <bunk@stusta.de>
> Date: Tue, 27 Mar 2007 03:02:47 +0200
>
>> We had during the last months have quite a few MSI bugs and even 
>> regressions due to:
>> - core kernel bugs,
>> - device driver bugs and
>> - hardware bugs
   - architecture bugs
   - MSI enabled on hardware that does not support it.

aka.  Drivers have started supporting MSI, People have started using
and testing MSI, and there has been MSI maintenance.  People care.

The most recent regressions involving MSI have been fixes propagating
their way through the kernel, and I can't think of a one of them
that was MSI specific.  Just that the bug didn't happen to show
up clearly without MSI enabled.

Finding that pci_save_state/pci_restore_state had serious resources
leaks was nasty.

Finding that pci_enable_device isn't suspend/resume safe as
implemented on x86 and ia64 is very nasty.  Currently on x86 it is
only really safe to call pci_enable_device exactly once.  But the bugs
are small enough we don't generally notice.

Personally I prefer glaring outstanding bugs to little subtle once
that only bite you on the second Tuesday of the month.

The recent MSI maintenance has shifted the code around enough that
problems became visible.  I'm not happy with this but I don't expect
this to be an on-going state of affairs.

>> OTOH, MSI doesn't bring any real advantages for most users.

So default it to off, although I suspect we are approaching the
point where it would actually be safe to default it to on.  We
need a kernel release that doesn't have msi issues yet.

>> Let's therefore mark PCI_MSI as EXPERIMENTAL.
>> 
>> Signed-off-by: Adrian Bunk <bunk@stusta.de>
>
> This is a good way to ensure that the code doesn't get tested
> enough to ever fix the problems.

Dave I agreed.  PCI_MSI is not the problem.

I think marking PCI_MSI as EXPERIMENTAL now would be closing the
barn door after the horses have fled.  I don't know of a core MSI
code path that hasn't been scrutinized lately.  I wouldn't say they
are perfect but they are junk either.  Especially given that the
code is not good enough where non x86 architectures can support
MSI.

There is one big remaining real world problem and that is we enable
MSI optimistically.  Resulting in it being enabled on chipsets that
don't support MSI.  We still need to change that behavior.  I have
been buried in the guts of things so I haven't had the free cycles to
worry about that yet, nor have there been enough people complaining
that it has crossed my pain threshold to just fix the thing.

I think where we are honestly at is that today MSI works on most new
chipsets.   MSI is supported by the hardware.  MSI is supported by the
OS.  With a little more maturity devices and device drivers will start
taking advantage of in ways that matter to users now that it works.

I can guess how they will take advantage of it and make it matter
to users but that would just be speculation on my part.

Eric

Re: [RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[Michael Ellerman]

[-- Attachment #1: Type: text/plain, Size: 3568 bytes --]

On Mon, 2007-03-26 at 21:13 -0600, Eric W. Biederman wrote:
> David Miller <davem@davemloft.net> writes:
> 
> > From: Adrian Bunk <bunk@stusta.de>
> > Date: Tue, 27 Mar 2007 03:02:47 +0200
> >
> >> We had during the last months have quite a few MSI bugs and even 
> >> regressions due to:
> >> - core kernel bugs,
> >> - device driver bugs and
> >> - hardware bugs
>    - architecture bugs
>    - MSI enabled on hardware that does not support it.
> 
> aka.  Drivers have started supporting MSI, People have started using
> and testing MSI, and there has been MSI maintenance.  People care.
> 
> The most recent regressions involving MSI have been fixes propagating
> their way through the kernel, and I can't think of a one of them
> that was MSI specific.  Just that the bug didn't happen to show
> up clearly without MSI enabled.
> 
> Finding that pci_save_state/pci_restore_state had serious resources
> leaks was nasty.
> 
> Finding that pci_enable_device isn't suspend/resume safe as
> implemented on x86 and ia64 is very nasty.  Currently on x86 it is
> only really safe to call pci_enable_device exactly once.  But the bugs
> are small enough we don't generally notice.
> 
> Personally I prefer glaring outstanding bugs to little subtle once
> that only bite you on the second Tuesday of the month.
> 
> The recent MSI maintenance has shifted the code around enough that
> problems became visible.  I'm not happy with this but I don't expect
> this to be an on-going state of affairs.
> 
> >> OTOH, MSI doesn't bring any real advantages for most users.
> 
> So default it to off, although I suspect we are approaching the
> point where it would actually be safe to default it to on.  We
> need a kernel release that doesn't have msi issues yet.
> 
> >> Let's therefore mark PCI_MSI as EXPERIMENTAL.
> >> 
> >> Signed-off-by: Adrian Bunk <bunk@stusta.de>
> >
> > This is a good way to ensure that the code doesn't get tested
> > enough to ever fix the problems.
> 
> Dave I agreed.  PCI_MSI is not the problem.
> 
> I think marking PCI_MSI as EXPERIMENTAL now would be closing the
> barn door after the horses have fled.  I don't know of a core MSI
> code path that hasn't been scrutinized lately.  I wouldn't say they
> are perfect but they are junk either.  Especially given that the
> code is not good enough where non x86 architectures can support
> MSI.
> 
> There is one big remaining real world problem and that is we enable
> MSI optimistically.  Resulting in it being enabled on chipsets that
> don't support MSI.  We still need to change that behavior.  I have
> been buried in the guts of things so I haven't had the free cycles to
> worry about that yet, nor have there been enough people complaining
> that it has crossed my pain threshold to just fix the thing.
> 
> I think where we are honestly at is that today MSI works on most new
> chipsets.   MSI is supported by the hardware.  MSI is supported by the
> OS.  With a little more maturity devices and device drivers will start
> taking advantage of in ways that matter to users now that it works.

.. and we will start to see more and more hardware that _only_ uses MSI.
So we need to get it fixed, rather than sweeping the bugs under the
carpet 'til later.

cheers

-- 
Michael Ellerman
OzLabs, IBM Australia Development Lab

wwweb: http://michael.ellerman.id.au
phone: +61 2 6212 1183 (tie line 70 21183)

We do not inherit the earth from our ancestors,
we borrow it from our children. - S.M.A.R.T Person

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[Jeff Garzik]

Eric W. Biederman wrote:
> aka.  Drivers have started supporting MSI, People have started using
> and testing MSI, and there has been MSI maintenance.  People care.

Agreed, well put.


> The most recent regressions involving MSI have been fixes propagating
> their way through the kernel, and I can't think of a one of them
> that was MSI specific.  Just that the bug didn't happen to show
> up clearly without MSI enabled.

Yep.


> So default it to off, although I suspect we are approaching the
> point where it would actually be safe to default it to on.  We
> need a kernel release that doesn't have msi issues yet.

Now that we are finally getting a handle on the MSI bumps-in-the-road, 
IMO it would be counterproductive to default it to 'off' now.

	Jeff

Re: [RFC: 2.6.21 patch] let PCI_MSI depend on EXPERIMENTAL

[Eric W. Biederman]

Jeff Garzik <jeff@garzik.org> writes:

> Eric W. Biederman wrote:
>> aka.  Drivers have started supporting MSI, People have started using
>> and testing MSI, and there has been MSI maintenance.  People care.
>
> Agreed, well put.
>
>
>> The most recent regressions involving MSI have been fixes propagating
>> their way through the kernel, and I can't think of a one of them
>> that was MSI specific.  Just that the bug didn't happen to show
>> up clearly without MSI enabled.
>
> Yep.
>
>
>> So default it to off, although I suspect we are approaching the
>> point where it would actually be safe to default it to on.  We
>> need a kernel release that doesn't have msi issues yet.
>
> Now that we are finally getting a handle on the MSI bumps-in-the-road, IMO it
> would be counterproductive to default it to 'off' now.

Let's see the Kconfig entry currently reads:

> config PCI_MSI
>         bool "Message Signaled Interrupts (MSI and MSI-X)"
>         depends on PCI
>         depends on (X86_LOCAL_APIC && X86_IO_APIC) || IA64 || SPARC64
>         help
>            This allows device drivers to enable MSI (Message Signaled
>            Interrupts).  Message Signaled Interrupts enable a device to
>            generate an interrupt using an inbound Memory Write on its
>            PCI bus instead of asserting a device IRQ pin.
> 
>            Use of PCI MSI interrupts can be disabled at kernel boot time
>            by using the 'pci=nomsi' option.  This disables MSI for the
>            entire system.
> 
>            If you don't know what to do here, say N.

So I don't know if we are defaulting it to off but we are advising it be
turned off.  So all of the bug reports are coming from people who presumably
know what they are doing and deliberately turn it on.

Once things quite down and we fix the chipset support detection then I think
we can default it to on.  Once that settles down we can remove the manual
option entirely except possibly under CONFIG_EMBEDDED.

We certainly don't have the question does this chipset support MSI handled
well enough to do better right now.  I think non-x86 might but we certainly
don't.

Eric