Floating point math in kernel interrupt -- am I doing this right? (repost)

Floating point math in kernel interrupt -- am I doing this right? (repost)

[Jeremy Friesner]

(Sorry if this is dupe, my first posting didn't seem to make it through)

Hi all,

Can someone who is well-versed in PowerPC kernel issues give me some advice?

First, a bit of background:  I'm working on an audio mixing/playback engine 
that runs on an embedded PowerPC chip (82xx) using the 2.4.23 kernel.  The 
audio output format is 48kHz floating point, and the mixing is done inside an 
interrupt routine that runs 6000 times per second, and calculates 8 samples 
per interrupt.

In order to do the necessary floating point math, without corrupting the state 
of any other processes that might be using the FPU, the interrupt routine 
first saves the state of the FPU registers to the stack, then does the math, 
then restores FPU registers again, before returning.

In general this seems to work fine, i.e. I get the audio output I expect and 
user processes aren't effected.  However, I've noticed that every so often I 
will get a kernel oops and my userland application will crash.  The oops 
looks like this:

<4>Oops: kernel access of bad area, sig: 11
<4>NIP: C001D408 XER: 20000000 LR: C001D3E0 SP: C6091E30 REGS: c6091d80 TRAP: 
0300    Not tainted
<4>MSR: 00001032 EE: 0 PR: 0 FP: 0 ME: 1 IR/DR: 11
<4>DAR: 3FE8228D, DSISR: 20000000
<4>TASK = c6090000[118] 'wtrxd' Last syscall: 178
<4>last math c6054000 last altivec 00000000
<4>GPR00: 0000001F C6091E30 C6090000 00000020 C6090398 C6055DE8 C017AB4C 
00000000
<4>GPR08: 408CC599 3FE8228D 0000001F 00000000 84022884 10510E98 105800BC 
308259E8
<4>GPR16: 1057302C 1058006C 00000000 00000000 00009032 06091F40 00000000 
C0005DE8
<4>GPR24: C0005B40 00000020 00008000 00000000 C6091F50 C6091F20 C6091E78 
00000020
<4>Call backtrace:
<4>00000400 C0009AB8 C0009CC0 C0009060 C0005B9C 0FC1BBA4 0FEF51D0
<4>0FEF6F58 0FEF3C3C 10366F34 10365C64 103572A0 103895D8 0FEF37EC
<4>0FCCDE2C

The frequency of the 'oops' occurrences appears to be proportional to the 
amount of code that is present in the FPU-enabled section of the interrupt 
routine.  If I enable all of the mixing code, they appear every 10-20 
minutes.  If I disable all of the mixing code except the saving and restoring 
of the FPU registers, they only occur about once every 6-12 hours.  If I 
disable the saving and restoring of the FPU registers entirely, they don't 
seem to occur at all (at least within the bounds of my testing so far -- 15+ 
hours).

So my question is, is there some minor detail that my FPU-state-save/restore 
code is missing, that would cause these sort of symptoms?  I just enough 
about kernel/assembly/PowerPC issues to be dangerous, so I thought maybe 
there was something obvious that I am missing.  The relevant code is shown 
below.

Many thanks for any help or hints you can provide,

Jeremy Friesner

--------------- begin code snippet ----------------------
[... in interrupt routine...]
   uint32 saved_msr;
   uint32 saved_fpr[32*2];  // temp storage space for the current floating 
point registers

   /* Set up floating point enabled mode -- NO FLOATING POINT OPS ALLOWED 
ABOVE THIS POINT! */
   {
      // Save existing MSR
      uint32 msr;
      __asm__ __volatile__ ("mfmsr %0" : "=r" (msr) : );
      saved_msr = msr;

      // Enable floating point
      msr |= MSR_FP;
      msr &= ~(MSR_FE0 | MSR_FE1);
      __asm__ __volatile__ ("mtmsr %0\n\tisync" : : "r" (msr));

      /* Save the floating point registers that the mix code uses, so that we 
won't screw up the user process */
#define SAVE_FPR(x) {uint32 * saveTo = &saved_fpr[x*2]; __asm__ __volatile__ 
("stfd " #x ", 0(%0)\n" : : "b" (saveTo));}
      SAVE_FPR(0);  SAVE_FPR(1);  SAVE_FPR(2); SAVE_FPR(3);  SAVE_FPR(4); 
      SAVE_FPR(5);  SAVE_FPR(6);  SAVE_FPR(7); SAVE_FPR(8);  SAVE_FPR(9);  
      SAVE_FPR(10); SAVE_FPR(11); SAVE_FPR(12); SAVE_FPR(13);
      /* Note that I only save/restore the first 13 FP registers, since gcc's 
generated code doesn't use the other ones anyway(!) */
   }

   [... the actual mixing routine is called here.  I compile the mix function 
in a separate file, without the -msoft-float flag ...]
   [... but note that I get crashes even if I comment out the call to the 
mixing routine, which suggests the problem is somewhere else...]
   DoMixing();

   /* Exit floating point enabled mode -- NO FLOATING POINT OPS ALLOWED BELOW 
THIS POINT! */
   {
      /* Restore the FP registers back to their original state */
#define RESTORE_FPR(x) {uint32 * restoreFrom = &saved_fpr[x*2]; __asm__ 
__volatile__ ("lfd  " #x ", 0(%0)\n" : : "b" (restoreFrom));}
      RESTORE_FPR(0);  RESTORE_FPR(1);  RESTORE_FPR(2); RESTORE_FPR(3);  
RESTORE_FPR(4);  
      RESTORE_FPR(5);  RESTORE_FPR(6);  RESTORE_FPR(7); RESTORE_FPR(8);  
RESTORE_FPR(9);  
      RESTORE_FPR(10); RESTORE_FPR(11); RESTORE_FPR(12); RESTORE_FPR(13);

      /* Restore MSR */
      __asm__ __volatile__ ("mtmsr %0\n" "isync" : : "r" (saved_msr));
   }

Re: Floating point math in kernel interrupt -- am I doing this right? (repost)

[Dan Malek]

On Feb 5, 2006, at 9:29 PM, Jeremy Friesner wrote:

> ....   and the mixing is done inside an
> interrupt routine that runs 6000 times per second, and calculates 8 
> samples
> per interrupt.

Bad news.  No Floating Point allowed in the kernel.

> In order to do the necessary floating point math, without corrupting 
> the state
> of any other processes that might be using the FPU, the interrupt 
> routine
> first saves the state of the FPU registers to the stack, then does the 
> math,
> then restores FPU registers again, before returning.

Not on 82xx.  All floating point is done in hardware.

> ....  The oops
> looks like this:

These aren't helpful without some supporting information, like
symbols, or learn to use ksymoops.

> So my question is, is there some minor detail that my 
> FPU-state-save/restore
> code is missing, that would cause these sort of symptoms?

Just get all of this out of the kernel.  If you used the standard ALSA
drivers and framework, lots of mixing features are available to you.
If you need some custom mixing, just write a plug-in to do it.
Your data rates are so low they don't justify making any hacks
like this.

Thanks.

	-- Dan

RE: Floating point math in kernel interrupt -- am I doing this right?(repost)

[Fillod Stephane]

Dan Malek wrote:
>> So my question is, is there some minor detail that my=20
>> FPU-state-save/restore
>> code is missing, that would cause these sort of symptoms?
>
>Just get all of this out of the kernel.  If you used the standard ALSA
>drivers and framework, lots of mixing features are available to you.
>If you need some custom mixing, just write a plug-in to do it.
>Your data rates are so low they don't justify making any hacks
>like this.

.. or if you *do* need hard-realtime feedback, you may consider
using Xenomai[1] (formerly known as RTAI/Fusion) instead of=20
a hack of the interrupt handler. Thanks to this nice project,=20
FPU on PQ2 works fine in kernel-land, and if need be, it will=20
let you move to user-land seamlessly while keeping the constraints.
Feel free to join the xenomai-help mailing list to get more help=20
on specific questions. Disclaimer: I'm not affiliated with Xenomai,
I am just a happy user :-)

[1] http://xenomai.org


Regards,
--=20
Stephane

Re: Floating point math in kernel interrupt -- am I doing this right? (repost)

[Jeremy Friesner]

> 
> On Feb 5, 2006, at 9:29 PM, Jeremy Friesner wrote:
> 
> > ....   and the mixing is done inside an
> > interrupt routine that runs 6000 times per second, and calculates 8 
> > samples
> > per interrupt.
> 
> Bad news.  No Floating Point allowed in the kernel.

I'm aware of that rule, and also why -- because the Linux kernel doesn't want to have to save and restore the state of the floating point registers on every context switch.  I'm also aware that it is nonetheless possible to use floating point in the kernel if you are careful to manually save and restore the proper state yourself, and that in fact floating point is sometimes used in the kernel as a means of doing 64-bit writes, as mentioned here:

http://www.tldp.org/LDP/cpg/html/x295.html

So it seems to me that it is possible to do floating point in the kernel, if you are careful to save and restore the proper context/state before and afterwards.  But if it isn't, can you explain to me why it can't be made to work=3F
 
> > In order to do the necessary floating point math, without corrupting 
> > the state
> > of any other processes that might be using the FPU, the interrupt 
> > routine
> > first saves the state of the FPU registers to the stack, then does the 
> > math,
> > then restores FPU registers again, before returning.
> 
> Not on 82xx.  All floating point is done in hardware.

Right; it is the hardware registers' state that I am saving to the stack, and then restoring afterwards.
 
> > So my question is, is there some minor detail that my 
> > FPU-state-save/restore
> > code is missing, that would cause these sort of symptoms=3F
> 
> Just get all of this out of the kernel.  If you used the standard ALSA
> drivers and framework, lots of mixing features are available to you.
> If you need some custom mixing, just write a plug-in to do it.
> Your data rates are so low they don't justify making any hacks
> like this.

I suppose that is an option, but I'd rather understand why my hack isn't working before I give up and redesign my application, which is already working the way I want it to, 99.9999% of the time.

Jeremy

RE: Floating point math in kernel interrupt -- am I doing this right?(repost)

[Joyeau Sylvain]

Jeremy,

What about kernel stack overflow ?
You mention that the occurrence of the crash is proportional to the =
amount of embraced code, which generally use more stack space too... but =
you aren't very accurate about the call context of your DoMix() =
function: it can be called depth enough to tickle the interrupted thread =
control block and issue indirect spurious behavior is userland.
Hope this help.

--
sj

> -----Original Message-----
> From: linuxppc-embedded-bounces@ozlabs.org =
[mailto:linuxppc-embedded-bounces@ozlabs.org] On Behalf Of Jeremy =
Friesner
> Sent: lundi 6 f=E9vrier 2006 17:55
> To: Dan Malek
> Cc: linuxppc-embedded@ozlabs.org
> Subject: Re: Floating point math in kernel interrupt -- am I doing =
this right?(repost)
>=20
> >
> > On Feb 5, 2006, at 9:29 PM, Jeremy Friesner wrote:
> >
> > > ....   and the mixing is done inside an
> > > interrupt routine that runs 6000 times per second, and calculates =
8
> > > samples
> > > per interrupt.
> >
> > Bad news.  No Floating Point allowed in the kernel.
>=20
> I'm aware of that rule, and also why -- because the Linux kernel =
doesn't want to have to save and restore the state of the
> floating point registers on every context switch.  I'm also aware that =
it is nonetheless possible to use floating point in
> the kernel if you are careful to manually save and restore the proper =
state yourself, and that in fact floating point is
> sometimes used in the kernel as a means of doing 64-bit writes, as =
mentioned here:
>=20
> http://www.tldp.org/LDP/cpg/html/x295.html
>=20
> So it seems to me that it is possible to do floating point in the =
kernel, if you are careful to save and restore the proper
> context/state before and afterwards.  But if it isn't, can you explain =
to me why it can't be made to work?
>=20
> > > In order to do the necessary floating point math, without =
corrupting
> > > the state
> > > of any other processes that might be using the FPU, the interrupt
> > > routine
> > > first saves the state of the FPU registers to the stack, then does =
the
> > > math,
> > > then restores FPU registers again, before returning.
> >
> > Not on 82xx.  All floating point is done in hardware.
>=20
> Right; it is the hardware registers' state that I am saving to the =
stack, and then restoring afterwards.
>=20
> > > So my question is, is there some minor detail that my
> > > FPU-state-save/restore
> > > code is missing, that would cause these sort of symptoms?
> >
> > Just get all of this out of the kernel.  If you used the standard =
ALSA
> > drivers and framework, lots of mixing features are available to you.
> > If you need some custom mixing, just write a plug-in to do it.
> > Your data rates are so low they don't justify making any hacks
> > like this.
>=20
> I suppose that is an option, but I'd rather understand why my hack =
isn't working before I give up and redesign my
> application, which is already working the way I want it to, 99.9999% =
of the time.
>=20
> Jeremy
>=20
>=20
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded

Voyager Video support color depths?

[Russell McGuire]

Everyone,

I had a question about the various video support that has been tested with
the 2.4.25 DENX Kernel.

To date I have had pretty good luck with by making sure I have compiled the
kernel with the Voyager.h file set at 32 BPP mode.

For speed optimization I wanted to cut it back to 16 BPP mode, however I
couldn't hardly get it to boot the machine after I did this change by
changing '#define SCREEN_BPP 16'. Most attempts the kernel would lock up
before a login prompt was available. When I tried SCREEN_BPP at 8 then it
booted fine, however QT/Embedded had terrible colors.

I was wondering if I should be looking for the trouble in some other code,
or perhaps hardware, or if this is a known bug in the SM501 / Voyager
driver, or perhaps the linux frame buffer driver.

Thanks all,

-Russ

RE: Floating point math in kernel interrupt -- am I doing this right?(repost)

[Jeremy Friesner]

Hi all,

I figured out what was wrong with my FPU save/restore code:  I was neglecting to save and restore the state of the FPSCR register.  Thanks to all who helped me figure it out!

For the sake of the list archives, I've pasted my fixed FP save/restore routines below.

-Jeremy

---------------------- snip -------------------------------

/* macros to save and restore floating point register state */
# define SAVE=5FFPR(x)    {=5F=5Fasm=5F=5F =5F=5Fvolatile=5F=5F ("stfd " #x ", " #x "*8(%0)\n" : : "b" (saved=5Ffpr));}
# define RESTORE=5FFPR(x) {=5F=5Fasm=5F=5F =5F=5Fvolatile=5F=5F ("lfd  " #x ", " #x "*8(%0)\n" : : "b" (saved=5Ffpr));}    

/* Set up floating-point-enabled-mode */
uint32 saved=5Fmsr;        /* space for the MSR register (32 bits) */
uint32 saved=5Ffpscr[1*2]; /* space for the FPSCR register (64 bits) */
uint32 saved=5Ffpr[14*2];  /* space for the 14 floating point registers I'll use (64 bits each) */
{
   uint32 msr;

   /* Save existing MSR for later */
   =5F=5Fasm=5F=5F =5F=5Fvolatile=5F=5F ("mfmsr %0" : "=3Dr" (msr) : );
   saved=5Fmsr =3D msr;

   /* Enable floating point */
   msr |=3D MSR=5FFP;
   msr &=3D ~(MSR=5FFE0 | MSR=5FFE1);
   =5F=5Fasm=5F=5F =5F=5Fvolatile=5F=5F ("mtmsr %0\n\tisync" : : "r" (msr));

   /* Save the floating point registers that will be used, so that we won't screw up user processes */
   SAVE=5FFPR(0);  SAVE=5FFPR(1);  SAVE=5FFPR(2);  SAVE=5FFPR(3);  SAVE=5FFPR(4);
   SAVE=5FFPR(5);  SAVE=5FFPR(6);  SAVE=5FFPR(7);  SAVE=5FFPR(8);  SAVE=5FFPR(9);
   SAVE=5FFPR(10); SAVE=5FFPR(11); SAVE=5FFPR(12); SAVE=5FFPR(13);

   /* Save existing FPSCR, by first retrieving it into FPR0, then storing FPR0 to the stack */
   =5F=5Fasm=5F=5F =5F=5Fvolatile=5F=5F ("mffs 0\n\tstfd 0, 0(%0)\n" : : "b" (saved=5Ffpscr));
}                

DoFloatingPointMathHere();

/* End floating-point-enabled-mode */
{
   /* Restore FPSCR, by first loading it into FPR0, then calling mtfsf to put it back into the register */
   =5F=5Fasm=5F=5F =5F=5Fvolatile=5F=5F ("lfd 0, 0(%0)\n\tmtfsf 0xFF,0" : : "b" (saved=5Ffpscr));

   /* Restore the FP registers that may have been munged */
   RESTORE=5FFPR(0);  RESTORE=5FFPR(1);  RESTORE=5FFPR(2);  RESTORE=5FFPR(3);  RESTORE=5FFPR(4);
   RESTORE=5FFPR(5);  RESTORE=5FFPR(6);  RESTORE=5FFPR(7);  RESTORE=5FFPR(8);  RESTORE=5FFPR(9);
   RESTORE=5FFPR(10); RESTORE=5FFPR(11); RESTORE=5FFPR(12); RESTORE=5FFPR(13);

   /* Restore MSR */
   =5F=5Fasm=5F=5F =5F=5Fvolatile=5F=5F ("mtmsr %0\n" "isync" : : "r" (saved=5Fmsr));
}