VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Michel Bouissou]

Hi there,

Natalie Protasevich and Alan Stern have worked a lot on helping me out with a 
VIA KT400 chipset / kernel 2.6.12 / IO-APIC / IRQ problem "irq 21: nobody 
cared!", which so far hasn't found its solution.

Research done with Alan shows that, on my system, the USB 2.0 controller seems 
to generate interrupts on the IRQ line attributed to the USB 1.1 controller, 
which isn't supposed to happen, and puzzles the system, when IO-APIC is 
enabled.

However, this didn't cause problems with 2.4 series kernels.

For the time being, there is no solution (Natalie is still investigating 
this), and it boils down to the following:

- If I boot with USB 2.0 enabled in BIOS, AND IO-APIC enabled in the kernel, 
then it badly breaks.

- If I either disable USB 2.0 in BIOS, or IO-APIC in the kernel, then it's OK.

I found today the thread between Bjorn Helgaas and Mathieu Bérard on LKML, 
where Mathieu reported the same problem, and Bjorn advised him to reverse a 
kernel patch (http://lkml.org/lkml/2005/6/21/243 ).

Mathieu (I don't have his email address, Bjorn, could you be so kind to 
forward this message to him) reports that it apparently solved this problem, 
so I tried to do the same, and reversed the same patch.

At first boot it seems to solve the issue, but when I rebooted again, it broke 
again, so this is not the solution -- the problem is not completely stable, 
sometimes it doesn't happen for some reason unknown to me... But most of the 
times it _does_ happen :-/

So this message is to inform different people who have suffered from this same 
problem, or are working for finding it a fix...

I'll be on travel for the coming week, and may or may not have occasional 
access to my email. (Please copy me on answers, as I'm not subscribed to the 
linux-kernel ML).

Cheers.

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Alan Stern]

On Sun, 17 Jul 2005, Michel Bouissou wrote:

> Hi there,
> 
> Natalie Protasevich and Alan Stern have worked a lot on helping me out with a 
> VIA KT400 chipset / kernel 2.6.12 / IO-APIC / IRQ problem "irq 21: nobody 
> cared!", which so far hasn't found its solution.
> 
> Research done with Alan shows that, on my system, the USB 2.0 controller seems 
> to generate interrupts on the IRQ line attributed to the USB 1.1 controller, 
> which isn't supposed to happen, and puzzles the system, when IO-APIC is 
> enabled.
> 
> However, this didn't cause problems with 2.4 series kernels.
> 
> For the time being, there is no solution (Natalie is still investigating 
> this), and it boils down to the following:
> 
> - If I boot with USB 2.0 enabled in BIOS, AND IO-APIC enabled in the kernel, 
> then it badly breaks.
> 
> - If I either disable USB 2.0 in BIOS, or IO-APIC in the kernel, then it's OK.
> 
> I found today the thread between Bjorn Helgaas and Mathieu Bérard on LKML, 
> where Mathieu reported the same problem, and Bjorn advised him to reverse a 
> kernel patch (http://lkml.org/lkml/2005/6/21/243 ).
> 
> Mathieu (I don't have his email address, Bjorn, could you be so kind to 
> forward this message to him) reports that it apparently solved this problem, 
> so I tried to do the same, and reversed the same patch.
> 
> At first boot it seems to solve the issue, but when I rebooted again, it broke 
> again, so this is not the solution -- the problem is not completely stable, 
> sometimes it doesn't happen for some reason unknown to me... But most of the 
> times it _does_ happen :-/
> 
> So this message is to inform different people who have suffered from this same 
> problem, or are working for finding it a fix...
> 
> I'll be on travel for the coming week, and may or may not have occasional 
> access to my email. (Please copy me on answers, as I'm not subscribed to the 
> linux-kernel ML).

Determining whether or not the system is working shouldn't be hit-or-miss.

To start out, try to determine whether each of the UHCI controllers really 
is mapped to IRQ 21.  Do this by booting with no USB devices plugged in, 
and then plugging a full- or low-speed device (like a USB mouse) into each 
of the ports in turn.  Check to make sure it works in each port and that 
that counts for IRQ 21 in /proc/interrupts increase each time.  To make 
this even more reliable you should run the test twice -- you don't have to 
reboot in between.  The first time, rmmod ehci-hcd before plugging in 
anything; the second time modprobe ehci-hcd before plugging.

The results you have reported make me a little suspicious.  The best way
to see whether the EHCI controller really is at fault is to plug in a
high-speed USB device.  I'm not totally familiar with the operation of
ehci-hcd, and it's quite possible that plugging in a low- or full-speed
device would not cause it to generate enough interrupts to be a problem --
even though you did trigger the fault by plugging in a low-speed mouse --
but plugging in a high-speed device ought to, provided ehci-hcd is loaded.  
Again, monitor the numbers in /proc/interrupts to see which is going up:  
IRQ 19 or IRQ 21.

The instability you mention is another cause for concern.  It may indicate 
that at some times, or under certain circumstances, the IRQs are routed 
wrongly whereas at others they are routed correctly.  And without any 
changes to the software!  If this is a random hardware fault it may be 
impossible to fix.  (But then why would it be so reliable under 2.4?)

Do you have a high-speed USB device?  Has it ever worked at high speed 
under 2.6?

Alan Stern

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Michel Bouissou]

Le Dimanche 17 Juillet 2005 22:36, vous avez écrit :
> Determining whether or not the system is working shouldn't be hit-or-miss.

Hum, yes, we're not using Windows ;-)

> To start out, try to determine whether each of the UHCI controllers really
> is mapped to IRQ 21.  Do this by booting with no USB devices plugged in,
> and then plugging a full- or low-speed device (like a USB mouse) into each
> of the ports in turn.  Check to make sure it works in each port and that
> that counts for IRQ 21 in /proc/interrupts increase each time.  To make
> this even more reliable you should run the test twice -- you don't have to
> reboot in between.  The first time, rmmod ehci-hcd before plugging in
> anything; the second time modprobe ehci-hcd before plugging.

I'm afraid I won't have time for this today. It's already more than 11 PM here 
and I'm leaving early tomorrow for travel...

But AFAIR, when I performed previous tests, I had tried about every USB socket 
on my computer (I have 6 of them...) to the same result.

> The results you have reported make me a little suspicious.  The best way
> to see whether the EHCI controller really is at fault is to plug in a
> high-speed USB device.  I'm not totally familiar with the operation of
> ehci-hcd, and it's quite possible that plugging in a low- or full-speed
> device would not cause it to generate enough interrupts to be a problem --
> even though you did trigger the fault by plugging in a low-speed mouse --
> but plugging in a high-speed device ought to, provided ehci-hcd is loaded.
>   Again, monitor the numbers in /proc/interrupts to see which is going up:
> IRQ 19 or IRQ 21.

Humm. I'm not sure about what you call a "full speed" device, for when I plug 
my USB scanner, my kernel reports it as a "full speed" USB device, and says 
it's managed by uhci (not ehci):

Jul 17 22:46:42 totor kernel: usb 3-2: new full speed USB device using 
uhci_hcd and address 3

I just tried an USB flashdisk that "used to work good with 2.4" and that I 
hadn't tried yet in 2.6. It's identified as "high speed" and ehci would like 
to manage it, but it seems I'm out of luck in some other aspect:

totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 25
totor kernel: usb 4-4: device not accepting address 25, error -71
totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 35
totor kernel: usb 4-4: device not accepting address 35, error -71
totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 36
totor kernel: usb 4-4: device not accepting address 36, error -71
totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 38
totor kernel: usb 4-4: device not accepting address 38, error -71
totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 48
totor kernel: usb 4-4: device not accepting address 48, error -71

...ad nauseam until I unplug the key...

Shhh... 

Doesn't like the front panel socket ? Let me try another USB socket... Just 
close to my mouse...

totor kernel: usb 4-2: new high speed USB device using ehci_hcd and address 16
totor kernel: SCSI subsystem initialized
totor kernel: Initializing USB Mass Storage driver...
totor kernel: scsi0 : SCSI emulation for USB Mass Storage devices
totor kernel: usbcore: registered new driver usb-storage
totor kernel: USB Mass Storage support registered.

Looks better, isn't it ?

Now, I checked that I can mount it and see its contents. That's OK.

I'm currently running with IO-APIC disabled, so my interrupts shows as:
[root@totor etc]# cat /proc/interrupts
           CPU0
  0:   12540579          XT-PIC  timer
  1:      22352          XT-PIC  i8042
  2:          0          XT-PIC  cascade
  4:      38647          XT-PIC  serial
  7:      18470          XT-PIC  parport0
 10:     863039          XT-PIC  uhci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb3, 
eth0, eth1, VIA8233, nvidia
 11:     155832          XT-PIC  ide0, ide1, ide2, ide3, ehci_hcd:usb4
 14:     112325          XT-PIC  ide4
 15:     112334          XT-PIC  ide5
NMI:          0
LOC:   12539533
ERR:        350


> The instability you mention is another cause for concern.  It may indicate
> that at some times, or under certain circumstances, the IRQs are routed
> wrongly whereas at others they are routed correctly.  And without any
> changes to the software!  If this is a random hardware fault it may be
> impossible to fix.  (But then why would it be so reliable under 2.4?)

I know that what I'm going to write will look crazy ;-) because it doesn't 
seem to make any sense, but I've noticed a pattern that tends to emerge from 
the different tests I've made with IO-APIC enabled and different 2.6.12 
kernels (patches, boot options, etc...) :

1/ When I'm testing a new kernel for the first time, I usually call it 
manually by typing the different relevant option manually from my grub 
(bootloader) commandline, and most of the times, it works without "losing IRQ 
21".
That's why I had thought, with your first suggestion of "usb-handoff" option, 
that my problem was solved.

Once I believe it works and want to test it again, I then put this as the 
default entry in my bootloader, then I reboot without touching anything (I 
let the bootloader select its default entry), and, usually, it then fails.

So I would say that a patterns looks emerging : When I have typed things on 
the keyboard at the bootloader stage, then loaded Linux, it may work. On the 
contrary, when I let the machine boot by itself without having touched 
anything, then I usually get these IRQ 21 losses.

Yes, I know this look completely silly ;-) but I mentioned it to be as 
complete as possible about what I noticed, and that may or may not be 
relevant...

Cheers.

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Michel Bouissou]

Le Dimanche 17 Juillet 2005 23:20, Michel Bouissou a écrit :
>
> I just tried an USB flashdisk that "used to work good with 2.4" and that I
> hadn't tried yet in 2.6. It's identified as "high speed" and ehci would
> like to manage it, but it seems I'm out of luck in some other aspect:
>
> totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address
> 25 totor kernel: usb 4-4: device not accepting address 25, error -71
> totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address
> 35 totor kernel: usb 4-4: device not accepting address 35, error -71
[...]
> ...ad nauseam until I unplug the key...
[...]
> Doesn't like the front panel socket ? Let me try another USB socket... Just
> close to my mouse...
>
> totor kernel: usb 4-2: new high speed USB device using ehci_hcd and address
> 16 totor kernel: SCSI subsystem initialized
> totor kernel: Initializing USB Mass Storage driver...

By the way, the front socket that dislikes the USB 2.0 flashdisk (ehci) feels 
perfectly happy if I plug and USB 1.1 flashdisk (uhci)... Feels good also if 
I plug my Digital Camera there... And I've plugged it there thousands of 
times.

Some posts I googled about this kind of errors tend to indicate this would be 
an IRQ mess ;-))

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Alan Stern]

On Sun, 17 Jul 2005, Michel Bouissou wrote:

> I'm afraid I won't have time for this today. It's already more than 11 PM here 
> and I'm leaving early tomorrow for travel...

I will be travelling this week also.  That's okay, there's no hurry.

> But AFAIR, when I performed previous tests, I had tried about every USB socket 
> on my computer (I have 6 of them...) to the same result.

But you didn't try these exact tests.

> Humm. I'm not sure about what you call a "full speed" device, for when I plug 
> my USB scanner, my kernel reports it as a "full speed" USB device, and says 
> it's managed by uhci (not ehci):
> 
> Jul 17 22:46:42 totor kernel: usb 3-2: new full speed USB device using 
> uhci_hcd and address 3

That's what I mean by a "full speed device".

> I just tried an USB flashdisk that "used to work good with 2.4" and that I 
> hadn't tried yet in 2.6. It's identified as "high speed" and ehci would like 
> to manage it, but it seems I'm out of luck in some other aspect:
> 
> totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 25
> totor kernel: usb 4-4: device not accepting address 25, error -71
> totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 35
> totor kernel: usb 4-4: device not accepting address 35, error -71
> totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 36
> totor kernel: usb 4-4: device not accepting address 36, error -71
> totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 38
> totor kernel: usb 4-4: device not accepting address 38, error -71
> totor kernel: usb 4-4: new high speed USB device using ehci_hcd and address 48
> totor kernel: usb 4-4: device not accepting address 48, error -71
> 
> ...ad nauseam until I unplug the key...

This could be the result of inadequate cabling inside the computer case
from the front panel socket to the motherboard.  Lots of other people have
seen that sort of thing.

> Shhh... 
> 
> Doesn't like the front panel socket ? Let me try another USB socket... Just 
> close to my mouse...
> 
> totor kernel: usb 4-2: new high speed USB device using ehci_hcd and address 16
> totor kernel: SCSI subsystem initialized
> totor kernel: Initializing USB Mass Storage driver...
> totor kernel: scsi0 : SCSI emulation for USB Mass Storage devices
> totor kernel: usbcore: registered new driver usb-storage
> totor kernel: USB Mass Storage support registered.
> 
> Looks better, isn't it ?
> 
> Now, I checked that I can mount it and see its contents. That's OK.
> 
> I'm currently running with IO-APIC disabled, so my interrupts shows as:

For the tests I described earlier, you will want to boot with IO-APIC 
enabled.  Otherwise there's nothing to test...


> I know that what I'm going to write will look crazy ;-) because it doesn't 
> seem to make any sense, but I've noticed a pattern that tends to emerge from 
> the different tests I've made with IO-APIC enabled and different 2.6.12 
> kernels (patches, boot options, etc...) :
> 
> 1/ When I'm testing a new kernel for the first time, I usually call it 
> manually by typing the different relevant option manually from my grub 
> (bootloader) commandline, and most of the times, it works without "losing IRQ 
> 21".
> That's why I had thought, with your first suggestion of "usb-handoff" option, 
> that my problem was solved.
> 
> Once I believe it works and want to test it again, I then put this as the 
> default entry in my bootloader, then I reboot without touching anything (I 
> let the bootloader select its default entry), and, usually, it then fails.
> 
> So I would say that a patterns looks emerging : When I have typed things on 
> the keyboard at the bootloader stage, then loaded Linux, it may work. On the 
> contrary, when I let the machine boot by itself without having touched 
> anything, then I usually get these IRQ 21 losses.
> 
> Yes, I know this look completely silly ;-) but I mentioned it to be as 
> complete as possible about what I noticed, and that may or may not be 
> relevant...

I would be very surprised if this turned out to mean anything.  But it 
doesn't hurt to try more tests, doing it both ways, and see what happens.


> By the way, the front socket that dislikes the USB 2.0 flashdisk (ehci) feels
> perfectly happy if I plug and USB 1.1 flashdisk (uhci)... Feels good also if 
> I plug my Digital Camera there... And I've plugged it there thousands of
> times.

That's to be expected from bad cabling.  Full-speed transmissions are much 
more tolerant of interference and noise than high-speed transmissions.

> Some posts I googled about this kind of errors tend to indicate this would be
> an IRQ mess ;-))

Sometimes it is.  And many of those posts are just guesses.  In your case
you know that the IRQs work correctly when you don't enable IO-APIC.

Alan Stern

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Michel Bouissou]

Le Dimanche 17 Juillet 2005 22:36, Alan Stern a écrit :
>
> To start out, try to determine whether each of the UHCI controllers really
> is mapped to IRQ 21.

I performed the tests you described, and here are the results I got.

First an exact description of my USB hardware.

My Gigabyte GA7-VAXP motherboard has 6 USB connectors.

- 2 integrated within the MB connectors plate (close to the PS/2 kbd/mouse 
connectors). Those have no cables and aren't prone to cable problems...

- 2 on a supplementary plate on the back. The plate and cables were provided 
with the MB.

- 2 on the front of my (good quality) Antec case, connectors and cables 
provided with the Antec case.

- The connectors on the supplementary plate and those on the case are 
connected to corresponding connectors on the motherboard.

All these connectors used to work perfectly in kernel 2.4, and I choose them 
only for ergonomic/position reasons, usually:

1/ The mouse (low-speed) connected to one of the MB integrated connectors

2/ The scanner (full-speed) connected to the supplementary plate on the back

3/ Removable devices (Camera, Flashdisks, full-speed or high-speed) usually 
connected to the case front-panel connectors.


Regarding USB, the output of "lspci -vv" says :

00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 
Controller (rev 80) (prog-if 00 [UHCI])
        Subsystem: Giga-byte Technology GA-7VAX Mainboard
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B-
        Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- 
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, cache line size 08
        Interrupt: pin A routed to IRQ 10
        Region 4: I/O ports at cc00 [size=32]
        Capabilities: [80] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA 
PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME+

00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 
Controller (rev 80) (prog-if 00 [UHCI])
        Subsystem: Giga-byte Technology GA-7VAX Mainboard
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B-
        Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- 
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, cache line size 08
        Interrupt: pin B routed to IRQ 10
        Region 4: I/O ports at d000 [size=32]
        Capabilities: [80] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA 
PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME+

00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 
Controller (rev 80) (prog-if 00 [UHCI])
        Subsystem: Giga-byte Technology GA-7VAX Mainboard
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B-
        Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- 
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, cache line size 08
        Interrupt: pin C routed to IRQ 10
        Region 4: I/O ports at d400 [size=32]
        Capabilities: [80] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA 
PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME+

00:10.3 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 82) (prog-if 20 
[EHCI])
        Subsystem: Giga-byte Technology GA-7VAX Mainboard
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- 
Stepping- SERR- FastB2B-
        Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- 
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, cache line size 10
        Interrupt: pin D routed to IRQ 11
        Region 0: Memory at e3009000 (32-bit, non-prefetchable) [size=256]
        Capabilities: [80] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA 
PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-


> Do this by booting with no USB devices plugged in, 
> and then plugging a full- or low-speed device (like a USB mouse) into each
> of the ports in turn.  Check to make sure it works in each port and that
> that counts for IRQ 21 in /proc/interrupts increase each time.  To make
> this even more reliable you should run the test twice -- you don't have to
> reboot in between.  The first time, rmmod ehci-hcd before plugging in
> anything; the second time modprobe ehci-hcd before plugging.

I've booted without any USB devices and did the following:

1/ rmmod ehci-hcd

2/ Plugged the mouse in each and every USB connector I have, in turn. The 
mouse was working good on each of them. IRQ 21 showed nicely incrementing 
each time I plugged / unplugged or moved the plugged mouse. System was happy 
and didn't log anything abnormal.

3/ Unplugged the mouse

4/ modprobe ehci-hcd

5/ Plugged the mouse. Immediately got "IRQ21: Nobody cared" and "Disabling IRQ 
21" messages.
=> Noticed IRQ21 count has suddenly been set to exactly 200000
After this, the mouse was now behaving slowly and erratically (USB polled 
without interrupts ?)

6/ Unplugged the mouse, then:
- rmmod ehci-hcd
- rmmod uhci-hcd
- modprobe ehci-hcd

7/ Plugged the mouse back. It was working happily again.

8/ Keeping the mouse plugged:
- modprobe ehci-hcd
=> Immediately got the IRQ21 insults again.
=> Noticed IRQ21 count has suddenly been set to exactly 400000
Mouse behaviour was slow and erratical again.

Repeated steps 6-8 using another USB socket, with the exact same results.

What do you think about this ?

Thanks again for your help.

Cheers.

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Alan Stern]

On Mon, 25 Jul 2005, Michel Bouissou wrote:

> 1/ rmmod ehci-hcd
> 
> 2/ Plugged the mouse in each and every USB connector I have, in turn. The 
> mouse was working good on each of them. IRQ 21 showed nicely incrementing 
> each time I plugged / unplugged or moved the plugged mouse. System was happy 
> and didn't log anything abnormal.

So it's definite that all three UHCI controllers use IRQ 21.

> 3/ Unplugged the mouse
> 
> 4/ modprobe ehci-hcd
> 
> 5/ Plugged the mouse. Immediately got "IRQ21: Nobody cared" and "Disabling IRQ 
> 21" messages.

I'm not aware under what circumstances the EHCI controller generates 
interrupt requests upon plugging in a new device, but apparently it did so 
for you.  Maybe because there were no other devices plugged in and the 
controller was suspended.

Clearly the EHCI controller also uses IRQ 21, even though the system 
thinks it is mapped to a different IRQ.

> => Noticed IRQ21 count has suddenly been set to exactly 200000

This is an artifact of the way the system detects and reports unhandled 
IRQs.

> After this, the mouse was now behaving slowly and erratically (USB polled 
> without interrupts ?)

Probably.

> 6/ Unplugged the mouse, then:
> - rmmod ehci-hcd
> - rmmod uhci-hcd
> - modprobe ehci-hcd

Do you really mean "modprobe ehci-hcd" here?  So the EHCI driver was
loaded and not the UHCI driver?  Or was that a typo?

> 7/ Plugged the mouse back. It was working happily again.

With no UHCI driver?

> 8/ Keeping the mouse plugged:
> - modprobe ehci-hcd

But you had just previously loaded ehci-hcd.  Unless the earlier line was 
a typo.

> => Immediately got the IRQ21 insults again.
> => Noticed IRQ21 count has suddenly been set to exactly 400000
> Mouse behaviour was slow and erratical again.
> 
> Repeated steps 6-8 using another USB socket, with the exact same results.
> 
> What do you think about this ?

It seems quite clear that the EHCI controller's IRQ line is causing the
problems.  Just out of curiousity, what happens if you really do remove
the UHCI driver, keeping only the EHCI driver, and then plug in the mouse?  
Off hand I would expect nothing much to happen -- maybe a line or two in
the system log, no change to the IRQ counters, and the mouse doesn't work
(not even erratically).

Alan Stern

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Michel Bouissou]

Le Lundi 25 Juillet 2005 21:18, Alan Stern a écrit :
>
> > 6/ Unplugged the mouse, then:
> > - rmmod ehci-hcd
> > - rmmod uhci-hcd
> > - modprobe ehci-hcd
>
> Do you really mean "modprobe ehci-hcd" here?  So the EHCI driver was
> loaded and not the UHCI driver?  Or was that a typo?

Sorry, typo, the copy/paste artifact syndrom :-(

I meant uhci, as you guessed...

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Michel Bouissou]

Le Lundi 25 Juillet 2005 21:18, Alan Stern a écrit :
>
> It seems quite clear that the EHCI controller's IRQ line is causing the
> problems.  Just out of curiousity, what happens if you really do remove
> the UHCI driver, keeping only the EHCI driver, and then plug in the mouse?
> Off hand I would expect nothing much to happen -- maybe a line or two in
> the system log, no change to the IRQ counters, and the mouse doesn't work
> (not even erratically).

As you expect, in such a condition (with only ehci loaded), absolutely nothing 
happens when plugging the mouse.

OTOH, a high-speed device is recognized, althouh it generates messages like:

totor kernel: usb 1-5: device not accepting address 3, error -71
totor kernel: usb 1-5: new high speed USB device using ehci_hcd and address 4
totor kernel: usb 1-5: device not accepting address 4, error -71
totor kernel: usb 1-5: new high speed USB device using ehci_hcd and address 5

If plugged to any USB socket, except the two integrated to the motherboard 
connectors plate. There only it fully succeeds without such errors.

Cheers.

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Alan Stern]

On Mon, 25 Jul 2005, Michel Bouissou wrote:

> Le Lundi 25 Juillet 2005 21:18, Alan Stern a écrit :
> >
> > It seems quite clear that the EHCI controller's IRQ line is causing the
> > problems.  Just out of curiousity, what happens if you really do remove
> > the UHCI driver, keeping only the EHCI driver, and then plug in the mouse?
> > Off hand I would expect nothing much to happen -- maybe a line or two in
> > the system log, no change to the IRQ counters, and the mouse doesn't work
> > (not even erratically).
> 
> As you expect, in such a condition (with only ehci loaded), absolutely nothing 
> happens when plugging the mouse.
> 
> OTOH, a high-speed device is recognized, althouh it generates messages like:
> 
> totor kernel: usb 1-5: device not accepting address 3, error -71
> totor kernel: usb 1-5: new high speed USB device using ehci_hcd and address 4
> totor kernel: usb 1-5: device not accepting address 4, error -71
> totor kernel: usb 1-5: new high speed USB device using ehci_hcd and address 5
> 
> If plugged to any USB socket, except the two integrated to the motherboard 
> connectors plate. There only it fully succeeds without such errors.

Now that's strange.  When you plug the high-speed device into the 
integrated ports, which IRQ counter changes?  Since nothing is using IRQ 
21, it should be disabled and its counter should remain constant.  Does 
this mean the interrupts show up on IRQ 19 (used by ehci-hcd), or do they 
not show up at all (i.e., is the USB connection just being polled)?

Those -71 errors do not indicate IRQ problems -- they indicate low-level 
errors in the USB data signals.  They could be caused by problems in the 
cabling from the motherboard to the ports, problems in the electrical 
terminations, or other things.

Alan Stern

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Michel Bouissou]

Le Lundi 25 Juillet 2005 22:44, Alan Stern a écrit :
>
> Now that's strange.  When you plug the high-speed device into the
> integrated ports, which IRQ counter changes?  Since nothing is using IRQ
> 21, it should be disabled and its counter should remain constant.  Does
> this mean the interrupts show up on IRQ 19 (used by ehci-hcd), or do they
> not show up at all (i.e., is the USB connection just being polled)?

I assume it's IRQ 19.

cat /proc/interrupts doesn't show IRQ21 at all when uhci isn't loaded.

IRQ 19 being shared with 4 IDE controllers that controls my hard drives, 
that's hard to isolate interrupts counts due to USB activity from interrupts 
counts due to disks activity...

-- 
Michel Bouissou <michel@bouissou.net> OpenPGP ID 0xDDE8AC6E

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Alan Stern]

On Tue, 26 Jul 2005, Michel Bouissou wrote:

> Le Lundi 25 Juillet 2005 22:44, Alan Stern a écrit :
> >
> > Now that's strange.  When you plug the high-speed device into the
> > integrated ports, which IRQ counter changes?  Since nothing is using IRQ
> > 21, it should be disabled and its counter should remain constant.  Does
> > this mean the interrupts show up on IRQ 19 (used by ehci-hcd), or do they
> > not show up at all (i.e., is the USB connection just being polled)?
> 
> I assume it's IRQ 19.
> 
> cat /proc/interrupts doesn't show IRQ21 at all when uhci isn't loaded.

As it shouldn't, since nothing is supposed to be using that IRQ.

> IRQ 19 being shared with 4 IDE controllers that controls my hard drives, 
> that's hard to isolate interrupts counts due to USB activity from interrupts 
> counts due to disks activity...

I was afraid you'd say that...

Natalie, that's all I can think of.  Now it's up to you to invent a patch
Michel can try out, to show just where the IO-APIC code is going wrong.

Alan Stern

Re: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Mathieu Bérard]

Michel Bouissou a écrit :

>Hi there,
>
>Natalie Protasevich and Alan Stern have worked a lot on helping me out with a 
>VIA KT400 chipset / kernel 2.6.12 / IO-APIC / IRQ problem "irq 21: nobody 
>cared!", which so far hasn't found its solution.
>
>Research done with Alan shows that, on my system, the USB 2.0 controller seems 
>to generate interrupts on the IRQ line attributed to the USB 1.1 controller, 
>which isn't supposed to happen, and puzzles the system, when IO-APIC is 
>enabled.
>
>However, this didn't cause problems with 2.4 series kernels.
>
>For the time being, there is no solution (Natalie is still investigating 
>this), and it boils down to the following:
>
>- If I boot with USB 2.0 enabled in BIOS, AND IO-APIC enabled in the kernel, 
>then it badly breaks.
>
>- If I either disable USB 2.0 in BIOS, or IO-APIC in the kernel, then it's OK.
>
>I found today the thread between Bjorn Helgaas and Mathieu Bérard on LKML, 
>where Mathieu reported the same problem, and Bjorn advised him to reverse a 
>kernel patch (http://lkml.org/lkml/2005/6/21/243 ).
>
>Mathieu (I don't have his email address, Bjorn, could you be so kind to 
>forward this message to him) reports that it apparently solved this problem, 
>so I tried to do the same, and reversed the same patch.
>
>
>
>
Hi,
yes I've encountered the same problem but my system
is a little bit different: It's a MSI mainboard with a VIA KT266A chipset
and no USB 2.0 controller (just 3 uhci).
IO-APIC is enabled.

With a 2.6.13-rc1-mm1 kernel, for example,
I got those error messages just after
the integrated sound card detection:

Jul  2 21:04:12 perenold kernel: irq 21: nobody cared (try booting with 
the "irqpoll" option)
Jul  2 21:04:12 perenold kernel: [<c0133c24>] __report_bad_irq+0x24/0x80
Jul  2 21:04:12 perenold kernel: [<c0133d22>] note_interrupt+0x72/0xc0
Jul  2 21:04:12 perenold kernel: [<c0133710>] __do_IRQ+0xe0/0xf0
Jul  2 21:04:12 perenold kernel: [<c0104f8e>] do_IRQ+0x3e/0x60
Jul  2 21:04:12 perenold kernel: =======================
Jul  2 21:04:12 perenold kernel: [<c0103502>] common_interrupt+0x1a/0x20
Jul  2 21:04:12 perenold kernel: [<c0142102>] zap_pte_range+0x82/0x1c0
Jul  2 21:04:12 perenold kernel: [<c01422bf>] unmap_page_range+0x7f/0xb0
Jul  2 21:04:12 perenold kernel: [<c01423f6>] unmap_vmas+0x106/0x210
Jul  2 21:04:12 perenold kernel: [<c01469c1>] exit_mmap+0x71/0x140
Jul  2 21:04:12 perenold kernel: [<c011547e>] mmput+0x2e/0xe0
Jul  2 21:04:12 perenold kernel: [<c015af3c>] exec_mmap+0xac/0x160
Jul  2 21:04:12 perenold kernel: [<c015b0a0>] flush_old_exec+0x70/0x700
Jul  2 21:04:12 perenold kernel: [<c0151475>] vfs_read+0xf5/0x160
Jul  2 21:04:12 perenold kernel: [<c015ae70>] kernel_read+0x40/0x60
Jul  2 21:04:12 perenold kernel: [<c0178f32>] load_elf_binary+0x252/0xd20
Jul  2 21:04:12 perenold kernel: [<c0138bf7>] buffered_rmqueue+0xb7/0x210
Jul  2 21:04:12 perenold kernel: [<c0138eeb>] __alloc_pages+0xdb/0x400
Jul  2 21:04:12 perenold kernel: [<c0104f95>] do_IRQ+0x45/0x60
Jul  2 21:04:12 perenold kernel: [<c01c2c6e>] __copy_from_user_ll+0x3e/0x70
Jul  2 21:04:12 perenold kernel: [<c015b92f>] 
search_binary_handler+0x4f/0x1d0
Jul  2 21:04:12 perenold kernel: [<c015bbfe>] do_execve+0x14e/0x200
Jul  2 21:04:12 perenold kernel: [<c010181f>] sys_execve+0x2f/0x70
Jul  2 21:04:12 perenold kernel: [<c0102aeb>] sysenter_past_esp+0x54/0x75
Jul  2 21:04:12 perenold kernel: handlers:
Jul  2 21:04:12 perenold kernel: [<e0c59450>] 
(snd_via82xx_interrupt+0x0/0xc0 [snd_via82xx])
Jul  2 21:04:12 perenold kernel: Disabling IRQ #21

and later:

Jul  2 21:04:37 perenold kernel: uhci_hcd 0000:00:11.3: Unlink after 
no-IRQ?  Controller is probably using the wrong IRQ.

IRQ 21 is then crazy with a rate of increasing of around 200000 per 
second in /proc/interrupts

All those error messages disappear if I revert the patch as I was 
advised to.

I have that in /proc/interrupts: (with an healthy kernel)
           CPU0
  0:  201893429    IO-APIC-edge  timer
  1:         21    IO-APIC-edge  i8042
  7:          2    IO-APIC-edge  parport0
  8:          0    IO-APIC-edge  rtc
  9:          0   IO-APIC-level  acpi
 14:    5448524    IO-APIC-edge  ide0
 15:   14583934    IO-APIC-edge  ide1
 16:     172543   IO-APIC-level  ide3
 17:     299810   IO-APIC-level  saa7134[0]
 18:   24973124   IO-APIC-level  eth0
 19:    5233000   IO-APIC-level  eth1
 20:         82   IO-APIC-level  uhci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb3
 21:          0   IO-APIC-level  VIA8233
NMI:          0
LOC:  201897508
ERR:          0
MIS:          0


So maybe, in my case, it's a mess between the IRQ of the uhci controllers
and the one of the integrated AC'97 sound ship.

But as it is mainly a server box nor the usb controllers nor the sound 
card are used very often,
so I don't know if those devices are actually working now. I am currently on
vacation 300 km away from that box so I can't really plug an USB key to
do some tests. But I will as soon as a can if that can help.
I will also try to reboot the box several times to see if the "IRQ 21 
nobody cared" error
reappears.

-- 
Mathieu

RE: VIA KT400 + Kernel 2.6.12 + IO-APIC + uhci_hcd = IRQ trouble

[Protasevich, Natalie]

> > Le Lundi 25 Juillet 2005 22:44, Alan Stern a écrit :
> > >
> > > Now that's strange.  When you plug the high-speed device into the 
> > > integrated ports, which IRQ counter changes?  Since 
> nothing is using 
> > > IRQ 21, it should be disabled and its counter should remain 
> > > constant.  Does this mean the interrupts show up on IRQ 
> 19 (used by 
> > > ehci-hcd), or do they not show up at all (i.e., is the 
> USB connection just being polled)?
> > 
> > I assume it's IRQ 19.
> > 
> > cat /proc/interrupts doesn't show IRQ21 at all when uhci 
> isn't loaded.
> 
> As it shouldn't, since nothing is supposed to be using that IRQ.
> 
> > IRQ 19 being shared with 4 IDE controllers that controls my hard 
> > drives, that's hard to isolate interrupts counts due to USB 
> activity 
> > from interrupts counts due to disks activity...
> 
> I was afraid you'd say that...
> 
> Natalie, that's all I can think of.  Now it's up to you to 
> invent a patch Michel can try out, to show just where the 
> IO-APIC code is going wrong.

I will sure try... I'm keeping an eye on your exchange don't worry :) 
just have to get done with urgent work piled up here while on my trip :< ...

--N
 
> Alan Stern
> 
>