Re: [PATCH] FRV: do_gettimeofday() should no longer use tickadj

Ingo Molnar
2006-09-09T05:54:50Z

* David Howells <dhowells@redhat.com> wrote:

> Ingo Molnar <mingo@elte.hu> wrote:
>
> > we'll get rid of that pt_regs thing centrally, from all drivers at once
> > - there's upstream buy-in for that already, and Thomas already generated
> > a test-patch for that a few months ago. But it's not a big issue right
> > now.
>
> Yay! Can you give me a pointer to the patch?

i cannot find Thomas' recent 2.6 one (Thomas, do you have a link to
it?), but i did one 5 years ago:

http://people.redhat.com/mingo/irq-rewrite-patches/irq-cleanup-2.4.15-B1.bz2

in general it's a large but otherwise pretty dumb patch.

> > this shouldnt be a big issue either - we use indirect jumps all around
> > the kernel.
>
> Yes, I know. I'm sometimes concerned at just how fast indirect jumps
> (and even direct calls) are proliferating. Look at the read syscall
> path for something like ext3 these days: it's like a pile of
> spaghetti. That seems particularly true of direct-IO where it seems
> to weave in and out of core code and the filesystem as it goes down.
> I'm also concerned about stack usage.

yeah - but unless you can suggest some low-maintainance-overhead
solution, not much can be done i suspect: being a few cycles slower is a
lot less of a problem than being less flexible in the design. In general
CPUs do optimize this quite well, but it is true that not every CPU
does.

> > CPUs are either smart enough to predict it
>
> I was told a while back (2002?) not to use indirect pointers for some
> stuff because CPUs _couldn't_ predict it. Maybe this has changed in
> modern CPUs.

indirect pointers are very common both in OSs and in applications,
especially in C++ based ones, where lots of execution goes off dynamic
objects which have function pointers associated to them. So _lots_ of
effort goes into branch prediction on the hardware side - and yes,
modern CPUs do quite well with indirect pointers too.

The worst-case scenario is when the indirect branch flip-flops between
multiple destination addresses - but that shouldnt be an issue for
genirq because most systems have _one_ preferred way of handling
interrupts that the majority of interrupt traffic uses. (for example on
i686 it's level-triggered PCI irqs)

But even if there's multiple destinations from the indirect jump, newest
CPUs (such as Core 2) can actually store _multiple_ branch history
targets and can prefetch all of them at once (if there's idle capacity
left).

(And i wouldnt be surprised if some modern CPUs already stored the
indirection register's index in the BHT, and used that for the
prediction. Most indirect calls happen off registers, and if the
compiler loads the register early enough (which it typically does) then
the branch target value is available to the CPU. Other context
information can be included in a BHT too.)

Also, in general, if something is arguably a smart thing to do in an OS
(and more design flexibility via function pointers is a smart thing for
which there is no viable alternative), we can expect CPUs to get
gradually better at handling them.

> > > (4) No account is taken of interrupt priority.
> >
> > hm, i'm not sure what you mean - could you be more specific?
>
> The FRV CPU, like many others, supports interrupt prioritisation. A
> particular interrupt level is set in the PSR, and any interrupt of a
> higher priority can interrupt. do_IRQ() can then do the interrupt
> processing in the interrupt level of the interrupt that invoked it,
> thus permitting higher priority interrupts to still happen.

ah, ok. For PREEMPT_HARDIRQS we thought about possibly utilizing
hw-level IRQ prioritization too - but it's quite inflexible in most IRQ
controller designs: the prioritization is rarely integrated with the CPU
and is often attached to the ACK/EOI-ing of the IRQ line (and an
unACK-ed IRQ can have side-effects).

So the thing we chose for PREEMPT_HARDIRQS was to do the prioritization
at the OS/scheduler level. And OS level handling of this is what we need
anyway: IRQ handlers are just the first, often tiny portion in a
critical workload that a system must perform. (we have softirqs,
signals, tasks, etc.) Nevertheless the door is open to utilize hw
capabilities of IRQ prioritization - we 'only' need standard driver and
/sys APIs to make use of them.

Ingo