[parisc-linux] Linux-2.6.6-rc1-pa3 and pdflush problems

[parisc-linux] Linux-2.6.6-rc1-pa3 and pdflush problems

[M. Grabert]

Hi,

I have experienced some problems with linux-2.6.6-rc1-pa0 and -pa3.
After running fine for around 10 hours, suddenly the
USB keyboard and mouse were not working anymore, and my friends
were not able to web access to the internet (which is routed through the
machine, with a transparent squid proxy set up that stores the
cache files on /dev/hda).

The LCD of my C3000 also showed some strange message instead of
'Linux-2.6.6-rc1-pa0' (I think I could read the letters HPMC somewhere,
but I didn't do a 'ser pim', since I forgot to clear it before, and
I didn't had my serial console cable handy).


I updated the kernel to -pa3, and now I could witness something strange
happening, again after about 11 hours running without problems:
Again the USB keyboard and mouse didn't work, the serial console
wouldn't respond to input, but printed out 'hda: lost interrupt',
'dma_mask=0x64' and 'retrying to recover interrupt' messages repeatedly.

This time I tried to ssh to the machine remotely and this worked,
and I was also able to ping to the internet, but - of course - the access
to the web wouldn't work, since the squid proxy would try to access /dev/hda.


However I was able to notice something rather odd:
wmcpuload showed 100% CPU usage, so I ran top and found
'pdflush' consuming all CPU ressources. The machine seemed to be running
fine (apart from no serial console, no /dev/hda access, no USB keyb/mouse),
and didn't seem to be slowed down despite pdflush burning all CPU cycles.
I pressed the power button, and the machine shut down properly (no HPMC
or similar).


So I guess the problem I experienced with -pa0 and -pa3 are somewhat
related (same symptoms), and I guess a HPMC would have resulted
if I just have waited long enough ...


Any ideas if this could be related to the recent changes in the cache
flush handling? Eg.
http://lists.parisc-linux.org/pipermail/parisc-linux-cvs/2004-April/033981.html


Unfortunately I just switched to the 2.6 kernel (for permanent use) recently,
so I'm not know whether this error would have happened with a
pre-2.6.6-rc1 kernel aswell.
Well, I have tested some 2.6.x kernels before, and most definitely also
for longer than 10 hours without problems, but basically running without
any load and without X (since I just got my Voodoo2 graphics card two
weeks ago).

I doubt this problem is related to the Voodoo2, since I guess some other
people would with a Voodoo graphics card would have posted similar problem
reports to the list. Furthermore the graphics card did work fine during
the problems (the screen was updated fine, I just couldn't use X anymore).


Sorry for so little info (System.map and .config will be supplied if
requested, but they haven't changed much since my last posts).

I'm gonna try to post a 'ser pim' when the problem reappears again,
and after the machine had a HPMC. Maybe somebody has some hints what
commands I could use to provide you with some more info if the problem
appears again ...


Greetings,
  Max

RE: [parisc-linux] Linux-2.6.6-rc1-pa3 and pdflush problems

[Joel Soete]

> Unfortunately I just switched to the 2.6 kernel (for permanent use) recently,
> so I'm not know whether this error would have happened with a
> pre-2.6.6-rc1 kernel aswell.

uhm the http://cvs.parisc-linux.org/download/linux-2.6/patch-2.6.5-pa8.gz
patch could always help you to rebuild this previous 2.6.6 kernel from upstream
src ;)

hth,
   Joel

PS: on my b2k (the brother model of c3k) I am running a 2.6.6-rc1-pa0 for
an uptime of 11days without any major pb. (well always some small hung of
some minutes because of scsi bus reset from time to time but no major issue).
But I have no additional gfx, no ide hd (just a cdrom drive) and I only use
serial console (so I couldn't help you more)

----------------------------------------------------------------------------------------
Tiscali ADSL: 35 €/mois, la meilleure offre du marché!
http://reg.tiscali.be/default.asp?lg=fr

RE: [parisc-linux] Linux-2.6.6-rc1-pa3 and pdflush problems

[M. Grabert]

On Tue, 27 Apr 2004, Joel Soete wrote:

> > Unfortunately I just switched to the 2.6 kernel (for permanent use) recently,
> > so I'm not know whether this error would have happened with a
> > pre-2.6.6-rc1 kernel aswell.
>
> uhm the http://cvs.parisc-linux.org/download/linux-2.6/patch-2.6.5-pa8.gz
> patch could always help you to rebuild this previous 2.6.6 kernel from upstream
> src ;)

Thanks, I'll give it a try ...

> PS: on my b2k (the brother model of c3k) I am running a 2.6.6-rc1-pa0 for
> an uptime of 11days without any major pb. (well always some small hung of
> some minutes because of scsi bus reset from time to time but no major issue).
> But I have no additional gfx, no ide hd (just a cdrom drive) and I only use
> serial console (so I couldn't help you more)

BTW,
This time it didn't take several hours, but merely a couple of minutes;
what I did was starting several downloads (onto a ext3 on /dev/hda1),
and then opening up a couple of websites in konqueror (which in turn uses
the proxy that also access /dev/hda1)


Again pdflush was consuming 100%, but this time I waited about 5mins and
pdflush behaved normal again ... just to start again later and disappear
again ...

I still have see the following messages on the serial console:

hda: lost interrupt
hda: dma_timer_expiry: dma status == 0x64
hda: DMA interrupt recovery

and also the same symptoms as before (network is working normal, USB
keyboard/mouse don't work). If you respond fast what commands I should
run, I can give you more info ...


Thanks,
   Max

RE: [parisc-linux] Linux-2.6.6-rc1-pa3 and pdflush problems

[Joel Soete]

> and also the same symptoms as before (network is working normal, USB
> keyboard/mouse don't work). If you respond fast what commands I should
> run, I can give you more info ...

hmm usb and ide seems to be on the same superio.

Joel

----------------------------------------------------------------------------------------
Tiscali ADSL: 35 €/mois, la meilleure offre du marché!
http://reg.tiscali.be/default.asp?lg=fr

RE: [parisc-linux] Linux-2.6.6-rc1-pa3 and Linux-2.6.6-rc2

[M. Grabert]

On Tue, 27 Apr 2004, Joel Soete wrote:

> > and also the same symptoms as before (network is working normal, USB
> > keyboard/mouse don't work). If you respond fast what commands I should
> > run, I can give you more info ...
>
> hmm usb and ide seems to be on the same superio.

Also something other interesting:

I killed squid and all other processes that are trying to access /dev/hda,
so the kernel messages 'hda: lost interrupt' stopped (mostly) for now.
Also pdflush doesn't consume any (significant) CPU time anymore.

The system is quite stable again, but still no access to the serial
console login prompt, nor the USB devices are working. BTW, a
/proc/bus/usb/devices results in the following kernel message:

usb 2-1: control timeout on ep0in

... and the process hangs.

Attaching the USB keyboard/mouse to my PCI USB2.0 card suddenly gives that
kernel output:

Badness in ohci_endpoint_disable at drivers/usb/host/ohci-hcd.c:355
Kernel addresses on the stack:
 [<10125b48>] printk+0x188/0x1c8
 [<101063b8>] dump_stack+0x18/0x24
 [<10267970>] ohci_endpoint_disable+0xd0/0x1a0
 [<10121f70>] __wake_up_common+0x60/0xa8
 [<10257cf4>] hcd_endpoint_disable+0x114/0x1b8
 [<10121fd8>] __wake_up+0x20/0x38
 [<102592c8>] usb_disable_endpoint+0x80/0x88
 [<102593d4>] usb_disable_device+0x94/0xb4
 [<102536c4>] usb_disconnect+0x90/0x140
 [<10255d54>] hub_port_connect_change+0x2dc/0x2e0
 [<1025604c>] hub_events+0x2f4/0x39c
 [<10256150>] hub_thread+0x5c/0x128
 [<1010bc5c>] ret_from_kernel_thread+0x1c/0x24


Another interesting thing is that I'm unable to kill -9 the X-Server.
Also I had some problems getting the Voodoo2 card working with custom
refresh rate settings; if I did specify a modeline in XF86Config-4
(generated with the fbset command), the machine would crash and print out
"bug in superio.c".
However if I use fbset after X has been started, it works perfectly.


Might be a Voodoo2 problem after all, who knows ...



And now for something completely different!



The Changelog to 2.6.6-rc2 contains the following:

=== snip ===

<dsaxena@plexity.net>
  [PATCH] PCI: Allow arch-specific pci_set_dma_mask and friends

  The patch provides the ability for architectures to have custom
  implementations of pci_set_dma_mask() and friends (dac_set_dma_mask
  and set_consistent_dma_mask). The reason I need this is b/c I have
  a chipset (Intel ARM IXP425) that has a broken PCI interface that
  only allows PCI dma to/from the bottom 64MB of system memory.  To get
  around this limitation, I trap a custom dma-mapping implementation that
  bounces buffers outside the 64MB window. At device discover time, my
  custom platform_notify() function gets called and it sets the dma_mask
  to (64MB-1) and in ARM's dma-mapping code, I check for dma_mask != 0xffffffff
  and if that is true, I call the special bounce helpers. This works great
  except that certain drivers (e100, ide-pci) call pci_set_dma_mask()
  with 0xffffffff and the generic implementation only allows for the
  architecture-defined pci_dma_supported() to return true or false. There
  is no method for the architecture to tell the PCI layer "I can't set
  the mask to 0xffffffff, but I can set it to this other value" and there
  is no way to pass that back to the driver. What this means is that if
  I have pci_set_dma_supported() return failure on full 32-bit DMA, the
  driver will not initialize the card; however, if I return true,
  pci_set_dma_mask() will set the dma mask to full 32-bits and I can no
  longer trap and will have buffers that are not dma-able and cause
  PCI master aborts.  Both of those are not acceptable.  IMHO, the
  driver shouldn't care if the architecture has to bounce DMA outside of
  64MB and since this is not something most architectures have to worry
  about, the easiest way to get around the issue is by allowing custom
  pci_set_dma_mask() for arches that need it but keeping the generic
  implementation for those that do not.  In my case, it simply returns
  0 to the driver but keeps the device mask set to 64MB-1 so I can trap.
=== snap ===

Since I still have problems to get onboard IDE DMA correctly working on my
C3000 (kernel and hdparm both say enabled, but just 3MB/s transfer rate
instead of >40 MB/s), and since any other PCI IDE card complains about
lost interrupts (on 2.6) or also behave like the onboard IDE (on 2.4),
I thought maybe this is related to the issue



Thanks for your help,
   Max

RE: [parisc-linux] Linux-2.6.6-rc1-pa3 and Linux-2.6.6-rc2

[Joel Soete]

> The Changelog to 2.6.6-rc2 contains the following:

Well, I don't know about Matthew merge project but yesterday I apply the
upstream patch-2.6.6-rc1-rc2 (http://www.kernel.org/pub/linux/kernel/v2.6/testing/incr/patch-2.6.6-rc1-rc2.bz2)
against our pa tree and it is quiet straighforward (just have to redo by
hand the hunk related to arch/parisc/kernel/cache.c, very short and easy).

I boot it with success on my b180 :) (not yet on my b2k :( ) but I trust
you could try it also?

That said, knowing that my serial console works fine, there are two additional
hw pieces that you can try to remove one by one to locate accurately the
pb, though I don't have usb kbd/mouse to test)

hth,
    Joel




----------------------------------------------------------------------------------------
Tiscali ADSL: 35 €/mois, la meilleure offre du marché!
http://reg.tiscali.be/default.asp?lg=fr

RE: [parisc-linux] Linux-2.6.6-rc1-pa3 and Linux-2.6.6-rc2

[M. Grabert]

On Tue, 27 Apr 2004, M. Grabert wrote:

> On Tue, 27 Apr 2004, Joel Soete wrote:
>
> > > and also the same symptoms as before (network is working normal, USB
> > > keyboard/mouse don't work). If you respond fast what commands I should
> > > run, I can give you more info ...
> >
> > hmm usb and ide seems to be on the same superio.

I found a (temporary) solution to my problem. At least I haven't
experienced any problems with the serial console lately.

BTW, the same happened also with a 2.4.26 kernel, so it's not only 2.6
related.

It appears that superio gets confused by my additional USB(2.0) PCI card
in my C3000. Whenever keyboard/mouse was plugged into the onboard USB
ports, simultaneous access to the IDE drive, keyboard and mouse could
cause a lock (and therefore neither USB, IDE and serial console were
working).

I resolved the issue by simply attaching the USB keyboard/mouse to my
USB PCI card. So far I hadn't any problems ....


BTW, using a Voodoo2 and a 2.6 kernel is not as stable as with 2.4.
Some XF86Config options can cause a crash (e.g "DontVTSwitch=true")
on 2.6.6-rc1-pa3, but not on 2.4.26-pa1 ...


greettings,
  Max