Re: RE: [PATCH] qe_ic: Do a sync when masking interrupts.

Re: RE: [PATCH] qe_ic: Do a sync when masking interrupts.

[Michael R. Zucca]

>From: Li Yang-r58472 <LeoLi@freescale.com>
>
>But an i/o read will be considerably slower than a sync, and it is in
>the critical path of interrupt.  I have tested the patch under
>relatively heavy Ethernet load, and there is no spurious interrupt.
>Maybe it is because the device is an SOC device and MMIO store completes
>faster.  I'm wondering if there is a standard test method to show if the
>faster approach is sufficient or not.

All a sync tells you is that an I/O made it out of the CPU. The problem is, there may be other places a write could get hung up. For instance, sometimes devices sit behind a bridge with a write FIFO. In such a scenario, you can't be sure a write has made it to the device until you do a read to flush the FIFO.

If you're trying to figure out the minimum thing to do (eieio, sync, read-back, etc.) you have to understand what your system is doing between the store and the bits going into the register.

It may be that a sync is enough, but you won't know until you fully understand the system's bus/bridge topolgy between the CPU and the device.

Re: [PATCH] qe_ic: Do a sync when masking interrupts.

[Segher Boessenkool]

> ll a sync tells you is that an I/O made it out of the CPU. The  
> problem is, there may be other places a write could get hung up.  
> For instance, sometimes devices sit behind a bridge with a write  
> FIFO. In such a scenario, you can't be sure a write has made it to  
> the device until you do a read to flush the FIFO.

It's not enough that a write made it to the device even -- you
have to make sure the device has acted on it.

> If you're trying to figure out the minimum thing to do (eieio,  
> sync, read-back, etc.) you have to understand what your system is  
> doing between the store and the bits going into the register.

What the system is doing, and also what exactly you want to
accomplish (what ordering and what completion you depend on).

> It may be that a sync is enough, but you won't know until you fully  
> understand the system's bus/bridge topolgy between the CPU and the  
> device.

If a sync after an MMIO write is enough, then (in almost all
cases) so is an eieio.


Segher

Re: [PATCH] qe_ic: Do a sync when masking interrupts.

[Scott Wood]

Segher Boessenkool wrote:
> If a sync after an MMIO write is enough, then (in almost all
> cases) so is an eieio.

In this case, the spurious interrupts still happen with eieio, but not 
with sync.  It's probably synchronizing the MMIO write with some 
unrelated load from memory (such as reading the stack frame to return 
from the mask function, or reading action->flags to determine whether to 
check for IRQ_DISABLED).

-Scott

Re: [PATCH] qe_ic: Do a sync when masking interrupts.

[Segher Boessenkool]

>> If a sync after an MMIO write is enough, then (in almost all
>> cases) so is an eieio.
>
> In this case, the spurious interrupts still happen with eieio, but  
> not with sync.  It's probably synchronizing the MMIO write with  
> some unrelated load from memory (such as reading the stack frame to  
> return from the mask function, or reading action->flags to  
> determine whether to check for IRQ_DISABLED).

Yes, it sure sounds like the only reason the sync insn helps
is that it causes a (small) delay.  If you document this (and
also that it isn't required for correctness) in the code, I
don't think anyone will complain / be curious about it anymore.


Segher

RE: [PATCH] qe_ic: Do a sync when masking interrupts.

[Li Yang-r58472]

> -----Original Message-----
> From: Scott Wood [mailto:scottwood@freescale.com]
> Sent: Tuesday, October 24, 2006 2:27 AM
> To: Segher Boessenkool
> Cc: mrz5149@acm.org; Li Yang-r58472; Paul Mackerras;
linuxppc-dev@ozlabs.org
> Subject: Re: [PATCH] qe_ic: Do a sync when masking interrupts.
>=20
> Segher Boessenkool wrote:
> > If a sync after an MMIO write is enough, then (in almost all
> > cases) so is an eieio.
>=20
> In this case, the spurious interrupts still happen with eieio, but not
> with sync. =20

eieio on e300 core is just a no-op.

- Leo

RE: [PATCH] qe_ic: Do a sync when masking interrupts.

[Benjamin Herrenschmidt]

On Tue, 2006-10-24 at 15:17 +0800, Li Yang-r58472 wrote:
> > -----Original Message-----
> > From: Scott Wood [mailto:scottwood@freescale.com]
> > Sent: Tuesday, October 24, 2006 2:27 AM
> > To: Segher Boessenkool
> > Cc: mrz5149@acm.org; Li Yang-r58472; Paul Mackerras;
> linuxppc-dev@ozlabs.org
> > Subject: Re: [PATCH] qe_ic: Do a sync when masking interrupts.
> > 
> > Segher Boessenkool wrote:
> > > If a sync after an MMIO write is enough, then (in almost all
> > > cases) so is an eieio.
> > 
> > In this case, the spurious interrupts still happen with eieio, but not
> > with sync.  
> 
> eieio on e300 core is just a no-op.

It's not sent to the bus on non-cahed storage (like the PCI bridge ?)
That's pretty bad ... there is quite a bit of code that assumes that
eieio's will prevent write combining...

Ben.

RE: [PATCH] qe_ic: Do a sync when masking interrupts.

[Liu Dave-r63238]

> > > Segher Boessenkool wrote:
> > > > If a sync after an MMIO write is enough, then (in almost all
> > > > cases) so is an eieio.
> > >=20
> > > In this case, the spurious interrupts still happen with=20
> eieio, but=20
> > > not with sync.
> >=20
> > eieio on e300 core is just a no-op.
>=20
> It's not sent to the bus on non-cahed storage (like the PCI=20
> bridge ?) That's pretty bad ... there is quite a bit of code=20
> that assumes that eieio's will prevent write combining...

Due to the LSU, the e300 cored doesn't reorder non-cached
stroage, and doesn't implementation of write commbining.
So, actually its behavior likes eieio.

-Dave

Re: RE: [PATCH] qe_ic: Do a sync when masking interrupts.

[Michael R. Zucca]

>From: Li Yang-r58472 <LeoLi@freescale.com>
>
>But an i/o read will be considerably slower than a sync, and it is in
>the critical path of interrupt.  I have tested the patch under
>relatively heavy Ethernet load, and there is no spurious interrupt.
>Maybe it is because the device is an SOC device and MMIO store completes
>faster.  I'm wondering if there is a standard test method to show if the
>faster approach is sufficient or not.

All a sync tells you is that an I/O made it out of the CPU. The problem is, there may be other places a write could get hung up. For instance, sometimes devices sit behind a bridge with a write FIFO. In such a scenario, you can't be sure a write has made it to the device until you do a read to flush the FIFO.

If you're trying to figure out the minimum thing to do (eieio, sync, read-back, etc.) you have to understand what your system is doing between the store and the bits going into the register.

It may be that a sync is enough, but you won't know until you fully understand the system's bus/bridge topolgy between the CPU and the device.