PREEMPT problems

PREEMPT problems

[Colin LEROY]

Hi,

This won't be really helpful, as I have no time to investigate, but I seem
to have problems with preempt (and latest main-tree BK). After a few hours
of uptime, I get hanging processes (sed, cvs (can't be network related as
the CVSROOT is local)) and gcc segfaults while compiling kernel.

Maybe it's not completely fixed yet...
--
Colin
  This message represents the official view of the voices
  in my head.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Benjamin Herrenschmidt]

On Fri, 2004-04-02 at 00:32, Colin LEROY wrote:
> Hi,
>
> This won't be really helpful, as I have no time to investigate, but I seem
> to have problems with preempt (and latest main-tree BK). After a few hours
> of uptime, I get hanging processes (sed, cvs (can't be network related as
> the CVSROOT is local)) and gcc segfaults while compiling kernel.
>
> Maybe it's not completely fixed yet...

Ah. Well, no problem here nor for other testers so far, weird...

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Benjamin Herrenschmidt]

On Fri, 2004-04-02 at 00:32, Colin LEROY wrote:
> Hi,
>
> This won't be really helpful, as I have no time to investigate, but I seem
> to have problems with preempt (and latest main-tree BK). After a few hours
> of uptime, I get hanging processes (sed, cvs (can't be network related as
> the CVSROOT is local)) and gcc segfaults while compiling kernel.

Oh, btw, finding out where those are beeing stuck may help. A good
way to do that is to use xmon "P" command, look at the KSP for one
of those processes (kernel stack pointer) and do a backtrace based
on it with "b <ksp value>"

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Michel Dänzer]

On Fri, 2004-04-02 at 02:56, Benjamin Herrenschmidt wrote:
> On Fri, 2004-04-02 at 00:32, Colin LEROY wrote:
> >
> > This won't be really helpful, as I have no time to investigate, but I seem
> > to have problems with preempt (and latest main-tree BK). After a few hours
> > of uptime, I get hanging processes (sed, cvs (can't be network related as
> > the CVSROOT is local)) and gcc segfaults while compiling kernel.

Same here, but at least some of the stuck processes continue after 'a
while'.

Note that the stuck processes only started with 2.6.5-rc3-ben0 here; the
last iteration of 2.6.5-rc2-ben0 is actually quite usable, albeit not
perfect either.

> Oh, btw, finding out where those are beeing stuck may help. A good
> way to do that is to use xmon "P" command, look at the KSP for one
> of those processes (kernel stack pointer) and do a backtrace based
> on it with "b <ksp value>"

How do I get into xmon again? Unfortunately, I didn't load the
System.map with this kernel...


--
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Michel Dänzer]

On Fri, 2004-04-02 at 18:00, Michel Dänzer wrote:
>
> Note that the stuck processes only started with 2.6.5-rc3-ben0 here; the
> last iteration of 2.6.5-rc2-ben0 is actually quite usable, albeit not
> perfect either.

I forgot to mention that the spurious segfaults also seemed much rarer
with 2.6.5-rc2-ben0.


--
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Benjamin Herrenschmidt]

On Sat, 2004-04-03 at 02:43, Michel Dänzer wrote:
> On Fri, 2004-04-02 at 18:00, Michel Dänzer wrote:
> >
> > Note that the stuck processes only started with 2.6.5-rc3-ben0 here; the
> > last iteration of 2.6.5-rc2-ben0 is actually quite usable, albeit not
> > perfect either.
>
> I forgot to mention that the spurious segfaults also seemed much rarer
> with 2.6.5-rc2-ben0.

I see no reason for that. I just added more protection ;)

I also added a WARN_ON(), do you see it triggered in dmesg ?

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Michel Dänzer]

On Sat, 2004-04-03 at 05:06, Benjamin Herrenschmidt wrote:
> On Sat, 2004-04-03 at 02:43, Michel Dänzer wrote:
> > On Fri, 2004-04-02 at 18:00, Michel Dänzer wrote:
> > >
> > > Note that the stuck processes only started with 2.6.5-rc3-ben0 here; the
> > > last iteration of 2.6.5-rc2-ben0 is actually quite usable, albeit not
> > > perfect either.
> >
> > I forgot to mention that the spurious segfaults also seemed much rarer
> > with 2.6.5-rc2-ben0.
>
> I see no reason for that. I just added more protection ;)

Sure enough, -rc2-ben0 is now behaving just as badly as -rc3-ben0.
Probably just fluctuation.

> I also added a WARN_ON(), do you see it triggered in dmesg ?

I'm not sure what those would look like, but

zgrep -i warn /var/log/kern.log*

doesn't seem to yield anything relevant.


--
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Benjamin Herrenschmidt]

> > I also added a WARN_ON(), do you see it triggered in dmesg ?
>
> I'm not sure what those would look like, but
>
> zgrep -i warn /var/log/kern.log*
>
> doesn't seem to yield anything relevant.

They look like bactraces ;)

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Benjamin Herrenschmidt]

> Sure enough, -rc2-ben0 is now behaving just as badly as -rc3-ben0.
> Probably just fluctuation.
>
> > I also added a WARN_ON(), do you see it triggered in dmesg ?

Still can't reproduce any problem here. You are using xfs right ?
Maybe it's a filesystem issue... or some driver (USB device ?) that
you are using that is allergic to preempt ?

It would be useful if you could spot one of those stuck processes
and get a backtrace from it. For that, use P<pid> in xmon to get the
ksp then b<ksp value> to get a backtrace.

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Michel Dänzer]

On Tue, 2004-04-06 at 02:51, Benjamin Herrenschmidt wrote:
> > Sure enough, -rc2-ben0 is now behaving just as badly as -rc3-ben0.
> > Probably just fluctuation.
> >
> > > I also added a WARN_ON(), do you see it triggered in dmesg ?
>
> Still can't reproduce any problem here. You are using xfs right ?

Yes. Colin, what about you?

> Maybe it's a filesystem issue... or some driver (USB device ?)

The only USB device I'm using regularly is a mouse.

> that you are using that is allergic to preempt ?

I don't think I'm using anything exotic... do you think ALSA could cause
such problems? The filesystem seems more likely to me, but what do I
know.


> It would be useful if you could spot one of those stuck processes
> and get a backtrace from it. For that, use P<pid> in xmon to get the
> ksp then b<ksp value> to get a backtrace.

I'm back to a non-preempt kernel for the time being, but I'll try to do
that if I ever feel adventurous again, and provided I find out how to
get into xmon.


--
Earthling Michel Dänzer      |     Debian (powerpc), X and DRI developer
Libre software enthusiast    |   http://svcs.affero.net/rm.php?r=daenzer


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Benjamin Herrenschmidt]

>
> I'm back to a non-preempt kernel for the time being, but I'll try to do
> that if I ever feel adventurous again, and provided I find out how to
> get into xmon.

Command-power (if CONFIG_XMON is enabled of course).

"P" -> list of processes, may wrap around though, "P<pid>" (all numbers
are in hex in xmon) to see a specific one

"b" -> backtrace current, "b<ksp>" -> bactrace from the stack at <ksp>

"x" -> exit xmon (you may have a stuck cmd key, just press it until it
goes back to normal).

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PREEMPT problems

[Wolfram Quester]

Hi!

On Thu, Apr 01, 2004 at 04:32:18PM +0200, Colin LEROY wrote:
>
> This won't be really helpful, as I have no time to investigate, but I
> seem to have problems with preempt (and latest main-tree BK). After
> a few hours of uptime, I get hanging processes (sed, cvs (can't be
> network related as the CVSROOT is local)) and gcc segfaults while
> compiling kernel.
>
> Maybe it's not completely fixed yet...

Hm, are you sure this is related to PREEMPT? I got this acouple of days
(30 Mar 2004) ago, when I tried to rcompile inkscape, a package I
maintain for debian.
I compiled the package and the recompiled in a chroot, to check
dependencies. sed crashed two times, attaching gdb to told me it was in
glibc-calls. To verify I compiled the same code out of the chroot and
got

if powerpc-linux-g++ -DHAVE_CONFIG_H -I. -I. -I../.. -I.. -I../../src
-DINKSCAPE_ICONS_DIR=\""/usr/share/inkscape"\"
-DINKSCAPE_PIXMAPDIR=\""/usr/share/inkscape"\" -I/usr/include/gtk-2.0
-I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0
-I/usr/include/pango-1.0 -I/usr/include/freetype2 -I/usr/X11R6/include
-I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include
-I/usr/include/libart-2.0 -I/usr/include/libxml2
-I/usr/lib/sigc++-1.2/include -I/usr/include/sigc++-1.2      -Wall -W
-Wpointer-arith -Wcast-align -Wsign-compare -Woverloaded-virtual
-Wswitch -Wstrict-prototypes -Wno-unused-parameter -g -O2 -MT
dialog-events.o -Molekular-Dynamik -MP -MF ".deps/dialog-events.Tpo" \
-c -o dialog-events.o `test -f 'dialog-events.cpp' || echo
'./'`dialog-events.cpp; \
then mv -f ".deps/dialog-events.Tpo" ".deps/dialog-events.Po"; \
else rm -f ".deps/dialog-events.Tpo"; exit 1; \
fi
/tmp/ccmgySoN.s: Assembler messages:
/tmp/ccmgySoN.s:17161: Error: value of 40696f11000a3ab8 too large for
field of 4 bytes at 0000000000007be8
make[4]: *** [dialog-events.o] Error 1
make[4]: Leaving directory
`/usr/local/src/packages/inkscape/temp/inkscape-0.37/src/dialogs'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory
`/usr/local/src/packages/inkscape/temp/inkscape-0.37/src'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory
`/usr/local/src/packages/inkscape/temp/inkscape-0.37'
make[1]: *** [all] Error 2
make[1]: Leaving directory
`/usr/local/src/packages/inkscape/temp/inkscape-0.37'
make: *** [build-stamp] Error 2

I had no explanation for this, except the update I did shortly before.
But IIRC I did not get an update of gcc or glibc. Later I went home and
recompiled there (same machine, PB 12", 1Ghz) and all was well. I didn't
have problems since then. Wierd.
My Kernel is still 2.4.24-ben1.

Best regards,

Wolfi

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

ppc4xx_find_bridges( ) hangs kernel on Xilinx ML300 (PCI)

[Mike Wellington]

Hi-
   I am trying to port the Xilinx PCI code to
an embedded linux kernel.  The code calls
ppc4xx_find_bridges( ) during the kernel initialization
and hangs the kernel.

   I built the GPL 2.4 kernel from the Open Source
archives and it hangs also.

   Then I copied ppc405_pci.c from the GPL source
to my kernel tree and copied  head_4xx.S from the
GPL source to my kernel tree and it still
hangs.

   I'm starting to wonder if there is something
wrong with the PCI support for the Xilinx OCP PCI
core.

   Does anybody have a working PCI device on a
Xilinx Virtex-II Pro using the Xilinx-supplied
PCI core?

-mike wellington
  wellington@lucent.com
  anyplatform@bithead.com

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

ppc4xx_find_bridges( ) hangs kernel on Xilinx ML300 (PCI)

[Mike Wellington]

Hi-
   I am trying to port the Xilinx PCI code to
an embedded linux kernel.  The code calls
ppc4xx_find_bridges( ) during the kernel initialization
and hangs the kernel.

   I built the GPL 2.4 kernel from the Open Source
archives and it hangs also.

   Then I copied ppc405_pci.c from the GPL source
to my kernel tree and copied  head_4xx.S from the
GPL source to my kernel tree and it still
hangs.

   I'm starting to wonder if there is something
wrong with the PCI support for the Xilinx OCP PCI
core.

   Does anybody have a working PCI device on a
Xilinx Virtex-II Pro using the Xilinx-supplied
PCI core?

-mike wellington
  wellington@lucent.com
  anyplatform@bithead.com


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: ppc4xx_find_bridges( ) hangs kernel on Xilinx ML300 (PCI)

[Peter Ryser]

The PCI support that is currently included in the Linux kernel tree is
for the V2PDK based OPB -> PCI bridge. It will not work with the EDK
based OPB -> PCI bridge, i.e. you will find exactly what you see.

Please download the EDK reference design for ML300 from
http://www.xilinx.com/ise/embedded/edk_examples.htm (design #6). The
design contains the V2PDK based bridge (including the pcore therefor)
and, thus, the Linux kernel will work fine with that design.

We are currently working on moving to the EDK based bridge and the Linux
kernel tree will be patched once this work is completed.

- Peter


Mike Wellington wrote:

>
> Hi-
>   I am trying to port the Xilinx PCI code to
> an embedded linux kernel.  The code calls
> ppc4xx_find_bridges( ) during the kernel initialization
> and hangs the kernel.
>
>   I built the GPL 2.4 kernel from the Open Source
> archives and it hangs also.
>
>   Then I copied ppc405_pci.c from the GPL source
> to my kernel tree and copied  head_4xx.S from the
> GPL source to my kernel tree and it still
> hangs.
>
>   I'm starting to wonder if there is something
> wrong with the PCI support for the Xilinx OCP PCI
> core.
>
>   Does anybody have a working PCI device on a
> Xilinx Virtex-II Pro using the Xilinx-supplied
> PCI core?
>
> -mike wellington
>  wellington@lucent.com
>  anyplatform@bithead.com
>
>
>
>


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: ppc4xx_find_bridges( ) hangs kernel on Xilinx ML300 (PCI)

[Mike Wellington]

Awesome!  You Rock!

-mikew



Peter Ryser wrote:
> The PCI support that is currently included in the Linux kernel tree is
> for the V2PDK based OPB -> PCI bridge. It will not work with the EDK
> based OPB -> PCI bridge, i.e. you will find exactly what you see.
>
> Please download the EDK reference design for ML300 from
> http://www.xilinx.com/ise/embedded/edk_examples.htm (design #6). The
> design contains the V2PDK based bridge (including the pcore therefor)
> and, thus, the Linux kernel will work fine with that design.
>
> We are currently working on moving to the EDK based bridge and the Linux
> kernel tree will be patched once this work is completed.
>
> - Peter
>
>
> Mike Wellington wrote:
>
>>
>> Hi-
>>   I am trying to port the Xilinx PCI code to
>> an embedded linux kernel.  The code calls
>> ppc4xx_find_bridges( ) during the kernel initialization
>> and hangs the kernel.
>>
>>   I built the GPL 2.4 kernel from the Open Source
>> archives and it hangs also.
>>
>>   Then I copied ppc405_pci.c from the GPL source
>> to my kernel tree and copied  head_4xx.S from the
>> GPL source to my kernel tree and it still
>> hangs.
>>
>>   I'm starting to wonder if there is something
>> wrong with the PCI support for the Xilinx OCP PCI
>> core.
>>
>>   Does anybody have a working PCI device on a
>> Xilinx Virtex-II Pro using the Xilinx-supplied
>> PCI core?
>>
>> -mike wellington
>>  wellington@lucent.com
>>  anyplatform@bithead.com
>>
>>
>>
>>


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

PCI - i *thought* the edk2 was what I was using!

[Mike Wellington]

Peter - I thought that the edk2 example from the link
you gave me was the pcore I was using.  I will start
over from scratch and try again.

My machine is busy doing some big  downloads/backups
so it will be tonite/tomorrow before I can verify
and report back.





Peter Ryser wrote:
> The PCI support that is currently included in the Linux kernel tree is
> for the V2PDK based OPB -> PCI bridge. It will not work with the EDK
> based OPB -> PCI bridge, i.e. you will find exactly what you see.
>
> Please download the EDK reference design for ML300 from
> http://www.xilinx.com/ise/embedded/edk_examples.htm (design #6). The
> design contains the V2PDK based bridge (including the pcore therefor)
> and, thus, the Linux kernel will work fine with that design.
>
> We are currently working on moving to the EDK based bridge and the Linux
> kernel tree will be patched once this work is completed.
>
> - Peter
>
>

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: PCI - i *thought* the edk2 was what I was using!

[Peter Ryser]

Mike,

the ml300_edk2.zip design does not have PCI at all. Only the
ml300_edk3.zip, i.e. the design to which the link currently points has
the pcore that works with Linux.

- Peter


Mike Wellington wrote:

> Peter - I thought that the edk2 example from the link
> you gave me was the pcore I was using.  I will start
> over from scratch and try again.
>
> My machine is busy doing some big  downloads/backups
> so it will be tonite/tomorrow before I can verify
> and report back.
>
>
>
>
>
> Peter Ryser wrote:
>
>> The PCI support that is currently included in the Linux kernel tree
>> is for the V2PDK based OPB -> PCI bridge. It will not work with the
>> EDK based OPB -> PCI bridge, i.e. you will find exactly what you see.
>>
>> Please download the EDK reference design for ML300 from
>> http://www.xilinx.com/ise/embedded/edk_examples.htm (design #6). The
>> design contains the V2PDK based bridge (including the pcore therefor)
>> and, thus, the Linux kernel will work fine with that design.
>>
>> We are currently working on moving to the EDK based bridge and the
>> Linux kernel tree will be patched once this work is completed.
>>
>> - Peter
>>
>>
>


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

did you give me the right ml300_edk*.zip link?

[Mike Wellington]

the link that you sent me was to ml300_edk2.zip ,
how can I download the ml300_edk3.zip ?

-mikew



eter Ryser wrote:
> Mike,
>
> the ml300_edk2.zip design does not have PCI at all. Only the
> ml300_edk3.zip, i.e. the design to which the link currently points has
> the pcore that works with Linux.
>
> - Peter
>
>
> Mike Wellington wrote:
>
>> Peter - I thought that the edk2 example from the link
>> you gave me was the pcore I was using.  I will start
>> over from scratch and try again.
>>
>> My machine is busy doing some big  downloads/backups
>> so it will be tonite/tomorrow before I can verify
>> and report back.
>>
>>
>>
>>
>>
>> Peter Ryser wrote:
>>
>>> The PCI support that is currently included in the Linux kernel tree
>>> is for the V2PDK based OPB -> PCI bridge. It will not work with the
>>> EDK based OPB -> PCI bridge, i.e. you will find exactly what you see.
>>>
>>> Please download the EDK reference design for ML300 from
>>> http://www.xilinx.com/ise/embedded/edk_examples.htm (design #6). The
>>> design contains the V2PDK based bridge (including the pcore therefor)
>>> and, thus, the Linux kernel will work fine with that design.
>>>
>>> We are currently working on moving to the EDK based bridge and the
>>> Linux kernel tree will be patched once this work is completed.
>>>
>>> - Peter
>>>
>>>
>>


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/