Preempt Real-time for ARM

Preempt Real-time for ARM

[Daniel Walker]

This is a release of Preempt Real-time for ARM . It includes everything
up to CONFIG_PREEMPT_RT , and all of the latency tracing except
interrupts off timing. The timing also excludes syscalls. This patch
includes only a port to OMAP boards. However, it should be straight
forward to get it working on other boards.

The biggest point of discussion relates to the interrupts in threads
implementation. It is largely identical to what is implemented in the
generic irq handling. However, ARM doesn't not implement generic irq
handling, and will not support it in the near future. I am not in
support of two different threaded interrupt implementations. 

I recently made a proposal to separate the threaded interrupt handling
from the generic irq handling, but I'm open to other ideas. 

The patch can be found at the following url

ftp://source.mvista.com/pub/realtime/arm/common_arm_realtime.patch


The patch order is as follows

http://www.kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.11-rc3.bz2
http://people.redhat.com/~mingo/realtime-preempt/realtime-preempt-2.6.11-rc3-V0.7.38-01
common_arm_realtime.patch

Enjoy!
Daniel Walker

Re: Preempt Real-time for ARM

[Thomas Gleixner]

On Sat, 2005-02-05 at 10:36 -0800, Daniel Walker wrote:

> The biggest point of discussion relates to the interrupts in threads
> implementation. It is largely identical to what is implemented in the
> generic irq handling. However, ARM doesn't not implement generic irq
> handling, and will not support it in the near future. I am not in
> support of two different threaded interrupt implementations. 

We have done the conversion to the generic irq handling and it works
fine on a couple of machines. 

I'm just waiting until the new SMP bits are there before I have another
go and clean up the missing SMP bits.

tglx

Re: Preempt Real-time for ARM

[Ingo Molnar]

* Thomas Gleixner <tglx@linutronix.de> wrote:

> On Sat, 2005-02-05 at 10:36 -0800, Daniel Walker wrote:
> 
> > The biggest point of discussion relates to the interrupts in threads
> > implementation. It is largely identical to what is implemented in the
> > generic irq handling. However, ARM doesn't not implement generic irq
> > handling, and will not support it in the near future. I am not in
> > support of two different threaded interrupt implementations. 
> 
> We have done the conversion to the generic irq handling and it works
> fine on a couple of machines. 

great - this would be a much preferred approach indeed.

> I'm just waiting until the new SMP bits are there before I have
> another go and clean up the missing SMP bits.

any chances for (most of) these bits going upstream as well? In any
case, the -RT tree can be a testbed for this.

	Ingo

Re: Preempt Real-time for ARM

[Thomas Gleixner]

On Wed, 2005-02-09 at 12:31 +0100, Ingo Molnar wrote:
> > I'm just waiting until the new SMP bits are there before I have
> > another go and clean up the missing SMP bits.
> 
> any chances for (most of) these bits going upstream as well? In any
> case, the -RT tree can be a testbed for this.

I guess this has to be discussed with Russell in detail. The UP bits are
quite simple and we can preserve the ARM oddities by tweaking the
generic layer a bit. The SMP stuff was/is work in progress, but what I
have seen until now is not too scary, but needs some careful thoughts.

tglx

Re: Preempt Real-time for ARM

[Russell King]

On Wed, Feb 09, 2005 at 12:31:40PM +0100, Ingo Molnar wrote:
> 
> * Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> > On Sat, 2005-02-05 at 10:36 -0800, Daniel Walker wrote:
> > 
> > > The biggest point of discussion relates to the interrupts in threads
> > > implementation. It is largely identical to what is implemented in the
> > > generic irq handling. However, ARM doesn't not implement generic irq
> > > handling, and will not support it in the near future. I am not in
> > > support of two different threaded interrupt implementations. 
> > 
> > We have done the conversion to the generic irq handling and it works
> > fine on a couple of machines. 
> 
> great - this would be a much preferred approach indeed.

Well, I remain unconvinced about the generic irq handling.

Back in 2.4 times, ARM used to use the x86 way to handle IRQs, and
it caused lots of dropped IRQs for CF cards and the like, particularly
in mixed level/edge triggered interrupt environments (where a mixture
of level and edge based outputs are connected to edge triggered inputs.)

The ARM IRQ code got completely rewritten during 2.5 with a clean
design, generated from the requirements of the machines.

This caused major changes throughout all the machine support files,
and I'm _NOT_, repeat _NOT_ going to consider going back to some
half baked approach which doesn't really fit the needs of the ARM
architecture, just because "oh, it's generic."  If it doesn't work
reliably, I'm not interested.

This is especially so when it impacts so many machines in ways
specific to each machine, and there's no way to get them tested
in one go.

If this is to be done, doing it in the middle of a stable kernel series
is NOT the time or place to do it.  I have recently had people complaining
about the "stability" of 2.6, particularly in relation to changes made by
other people affecting drivers.

Consider these questions in relation to the generic IRQ code:

1. Does it know the difference between handling level, edge-based and
   "simple" IRQs?  ("simple" IRQs are those which are cascaded, but
   don't have their own individual interrupt mask controls.)

2. Does the generic autoprobe code know which IRQs can be autoprobed
   and which can't?  (cascade interrupts are just one example of
   interrupts which must not be autoprobed.  There may be other
   reasons you wish to avoid probing other interrupts on a particular
   machine, which the machine support code knows about.)

3. Does the generic IRQ code know which IRQs can be claimed and
   which can't?  (IRQs 0 to NR_IRQS aren't always claimable, even
   when they appear to be available - iow, desc->handler != &no_irq_type.)

4. Does it allow per "hw type" retriggering of interrupts, even if the
   hardware itself is not capable of such an action?  (and running
   these interrupts at the next hardware interrupt?)

5. Does it allow control of interrupt wakeup sources?

6. Does it allow architectures to define their own irq_desc_t so that
   all the data for a particular IRQ is localised and contained within
   one data structure?

What you'll find is that the ARM interrupt structure is designed to
efficiently meet the requirements of our wide range of hardware interrupt
controllers, with chained interrupt controllers, with as low latency as
possible.

In essence, I'm opposed to completely rewriting the ARM interrupt
handling at this stage.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Re: Preempt Real-time for ARM

[Thomas Gleixner]

Russell,

On Wed, 2005-02-09 at 12:50 +0000, Russell King wrote:
> > > We have done the conversion to the generic irq handling and it works
> > > fine on a couple of machines. 
> > 
> > great - this would be a much preferred approach indeed.
> 
> Well, I remain unconvinced about the generic irq handling.
> 
> This caused major changes throughout all the machine support files,
> and I'm _NOT_, repeat _NOT_ going to consider going back to some
> half baked approach which doesn't really fit the needs of the ARM
> architecture, just because "oh, it's generic."  If it doesn't work
> reliably, I'm not interested.

Agreed. Doing it just for the sake of a generic solution is not the
point.

> This is especially so when it impacts so many machines in ways
> specific to each machine, and there's no way to get them tested
> in one go.

Ack.

> If this is to be done, doing it in the middle of a stable kernel series
> is NOT the time or place to do it.  I have recently had people complaining
> about the "stability" of 2.6, particularly in relation to changes made by
> other people affecting drivers.
> 
> Consider these questions in relation to the generic IRQ code:
> 
> 1. Does it know the difference between handling level, edge-based and
>    "simple" IRQs?  ("simple" IRQs are those which are cascaded, but
>    don't have their own individual interrupt mask controls.)

The changes we made to the generic layer in order to make it work on ARM
still call the level/edge/simple handlers.

> 2. Does the generic autoprobe code know which IRQs can be autoprobed
>    and which can't?  (cascade interrupts are just one example of
>    interrupts which must not be autoprobed.  There may be other
>    reasons you wish to avoid probing other interrupts on a particular
>    machine, which the machine support code knows about.)

Makes sense to add this to the generic code anyway.

> 3. Does the generic IRQ code know which IRQs can be claimed and
>    which can't?  (IRQs 0 to NR_IRQS aren't always claimable, even
>    when they appear to be available - iow, desc->handler != &no_irq_type.)

There's a check whether a particular irq has been exclusively allocated
or is available for driver use. Not sure, if it does exactly the same
what you are talking about. Needs to be checked.

> 4. Does it allow per "hw type" retriggering of interrupts, even if the
>    hardware itself is not capable of such an action?  (and running
>    these interrupts at the next hardware interrupt?)

I implemented this basically, but it's not fully tested/functional yet. 

> 5. Does it allow control of interrupt wakeup sources?

Are you talking about PM wakeup ? If yes, it is not there, but isn't
this functionality useful for other platforms than ARM too ?

> 6. Does it allow architectures to define their own irq_desc_t so that
>    all the data for a particular IRQ is localised and contained within
>    one data structure?

Having a generic irq_desc_t with an architecture specific member "struct
arch_irq_desc_t bla;" should do the job.

> In essence, I'm opposed to completely rewriting the ARM interrupt
> handling at this stage.

As expected :)
I accept your decision, but I hope that you are not completely opposed
to discuss a conversion at all, especially as other architectures will
benefit from the necessary changes to the generic code too. 

Thanks for the pointers so far.

tglx

Re: Preempt Real-time for ARM

[Daniel Walker]

On Wed, 2005-02-09 at 04:50, Russell King wrote:


> What you'll find is that the ARM interrupt structure is designed to
> efficiently meet the requirements of our wide range of hardware interrupt
> controllers, with chained interrupt controllers, with as low latency as
> possible.
> 
> In essence, I'm opposed to completely rewriting the ARM interrupt
> handling at this stage.


	Everyone wants this as the final solution, but all I want right now is
to have a clean patch for ARM RT .. It would be nice if ARM had the
generics, but It's not _my_ goal. 

	All I want to do is integrate the common IRQ threading code. To do that
I need things , from Russell, like per descriptor locks .. And I need
things , from Ingo, like pulling out the IRQ threading code..


Daniel

Re: Preempt Real-time for ARM

[Russell King]

On Wed, Feb 09, 2005 at 09:41:10AM -0800, Daniel Walker wrote:
> 	All I want to do is integrate the common IRQ threading code. To do that
> I need things , from Russell, like per descriptor locks .. And I need
> things , from Ingo, like pulling out the IRQ threading code..

I've said why per-IRQ locks are incorrect for the non-RT cases on ARM,
but unfortunately just repeating the reasons why it's wrong isn't
getting me anywhere either.  So shrug, all I can to is explain why
it's wrong, and if people choose not to listen there's nothing more
I can do.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Re: Preempt Real-time for ARM

[Eugeny S. Mints]

Russell King wrote:
> On Wed, Feb 09, 2005 at 09:41:10AM -0800, Daniel Walker wrote:
> 
>>	All I want to do is integrate the common IRQ threading code. To do that
>>I need things , from Russell, like per descriptor locks .. And I need
>>things , from Ingo, like pulling out the IRQ threading code..
> 
> 
> I've said why per-IRQ locks are incorrect for the non-RT cases on ARM,
> but unfortunately just repeating the reasons why it's wrong isn't
> getting me anywhere either.  So shrug, all I can to is explain why
> it's wrong, and if people choose not to listen there's nothing more
> I can do.

Lets summarize your main arguments from two threads - 
"irq_controller_lock" and this one:
(sorry, I summarized since I somehow accidently lost traack of 
"irq_controller_lock" thread and want to be sure I haven't missed anything)

1) if we drop ide_controller_lock we need to add per-chip lock due to 
read-modify-write issue

true

2) per-descriptor lock will not bring gains since
    a) SMP - almost nonexistent at the moment

As Daniel said - why not look to the future - did anybody expect 3 month 
  ago RT enchancement for Linux?! progress is too quick - and again from 
the perspective of SMP - irq_controller_lock is defective.

    b) lots of contention on request_irq/free_irq - rare
seems true
    c) multiple devices on the same interrupt line - rare
seems true

But in a whole it's not so unambiguously what outweighs - b)&c) against 
contra a)

3) per chip lock in combination with per descriptor lock
   a) decreases peformance

why not to lock per-chip lock only for chips indeed require this (i.e. 
with read/modify/write/) and drop the locking otherwise?!

   b) (quoatition):
      "Yes, and then audit that no one uses different irqchip structures
       covering the same register (consider a read-modify-write mask
       register where some IRQs are edge and others are level riggered.)"

such a register and a chip have ono-to-one relationship, do they? chip 
lock is something connected to _the_ chip. The above situation is 
definitly up to a developer and his own _fault_.

4) ARM IRQs are already "threaded"

As you said:  can we _please_ get the terminology right? As Nicolas 
pointed you were talking about completely different "threaded".


		Eugeny