[PATCH] ULI 2.6.12

[PATCH] ULI 2.6.12

[Michael Raymond]

    Here's a patch of the ULI code to Tony's pre-2.6.12 tree.
Signed-off-by: Michael A Raymond <mraymond@sgi.com>

    I was able to show that ULIs cannot be scheduled onto other CPUs.  For
every IRQ that a user requests it registers a standard handler.  All IRQ
handlers are called out of do_IRQ().  It's not until do_IRQ() is called that
ia64_eoi() is called to complete the ACK and allow lower priority interrupts
into the system.
    I've added a simple stats program that makes use of ULIs to the support
package on the web site (1).  We've also found a case where their low
perturbation will be useful for user level profiling. I'll be writing that
up next.
     	     					Thanks,
						       Michael
(1) http://oss.sgi.com/projects/uli/
-- 
Michael A. Raymond              Office: (651) 683-3434
Core OS Group                   Real-Time System Software

Re: [PATCH] ULI 2.6.12

[Peter Chubb]

>>>>> "Michael" = Michael Raymond <mraymond@sgi.com> writes:

Michael>     Here's a patch of the ULI code to Tony's pre-2.6.12 tree.
Michael> Signed-off-by: Michael A Raymond <mraymond@sgi.com>

Michael> All IRQ handlers are called out of
Michael> do_IRQ().  It's not until do_IRQ() is called that ia64_eoi()
Michael> is called to complete the ACK and allow lower priority
Michael> interrupts into the system.  

This is one thing that  bothers me most about this approach.  User
code is invoked before acknowledging the interrupt to the interrupt
controller.  As a general rule, kernel code should not trust user code
for its correct operation.  If the ULI does not clear the interrupt,
(because it is malicious, say) then there is a distinct possibility of
DOS here.


However, I think that the thing that bothers me *most* about SGI's ULI
approach is that a full context switch is not done.  The ULI runs as
if it were in the interrupted process's context.  `current' isn't
changed, so it runs with the privileges of the interrupted process.
The *way* it runs (CPU bound, presumably) will affect the scheduler's
decisions about how to run the interrupted process in the next
timeslice.  For most interrupt handlers this won't matter, but it'd be
relatively easy to construct a malicious one to slow particular
processes.

Running at interrupted process provilege doesn't matter very much,
because all system calls are mapped onto uli_syscall if you're running
as a user-level interrupt.  However, this severely limits what you can
do in a handler.  For example, you can't wake threads waiting on  a
futex (something we find very common in a user-level interrupt handler).


--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: [PATCH] ULI 2.6.12

[Christoph Hellwig]

On Mon, Apr 04, 2005 at 10:22:37AM +1000, Peter Chubb wrote:
> However, I think that the thing that bothers me *most* about SGI's ULI
> approach is that a full context switch is not done.  The ULI runs as
> if it were in the interrupted process's context.  `current' isn't
> changed, so it runs with the privileges of the interrupted process.
> The *way* it runs (CPU bound, presumably) will affect the scheduler's
> decisions about how to run the interrupted process in the next
> timeslice.  For most interrupt handlers this won't matter, but it'd be
> relatively easy to construct a malicious one to slow particular
> processes.

I agree.  Raymonds version looks like a hack for a very narrow special
case to me.   I really prefer your more portable and useful version, and
I don't think the performance differences matter given today hardware.

Re: [PATCH] ULI 2.6.12

[Michael Raymond]

    These are all valid concerns.

On Mon, Apr 04, 2005 at 10:22:37AM +1000, Peter Chubb wrote:
> This is one thing that  bothers me most about this approach.  User
> code is invoked before acknowledging the interrupt to the interrupt
> controller.  As a general rule, kernel code should not trust user code
> for its correct operation.  If the ULI does not clear the interrupt,
> (because it is malicious, say) then there is a distinct possibility of
> DOS here.
 
    The idea is that only trusted code by priviledged users will be run.
Let's assume for a moment that this isn't true.  My understanding of the
IA64 code is the following.

- Interrupt handling ends up in __do_IRQ()
- __do_IRQ() ACKs the interrupt with an infrastructure provided function
- control passes to handle_IRQ_event()
- Each registered handler is run.  These include traditional kernel drivers,
  ULIs, etc
- control returns through __do_IRQ() to ia64_handle_irq()
- ia64_eoi() is called to lower the blocked interrupt class and allow the
  interrupt to be handled again
 
> However, I think that the thing that bothers me *most* about SGI's ULI
> approach is that a full context switch is not done.  The ULI runs as
> if it were in the interrupted process's context.  `current' isn't
> changed, so it runs with the privileges of the interrupted process.
> The *way* it runs (CPU bound, presumably) will affect the scheduler's
> decisions about how to run the interrupted process in the next
> timeslice.  For most interrupt handlers this won't matter, but it'd be
> relatively easy to construct a malicious one to slow particular
> processes.

    Again the idea is that the handler function was registered by root or
some priviledged developer making a conscious decision.  This is currently
protected by permissions on the /dev/uli file.  If you'd like me to add
permission checks to the kernel functions I'd be happy to do it.
    Another help here is that handlers can only run for a certain amount of
time.  I think the arbitrarily chosen limit right now is ~10us.  A handler
that has to do anything more than simple work should wake up a thread /
process to do so.  I can make that time limit configurable if you'd like.
 
> Running at interrupted process provilege doesn't matter very much,
> because all system calls are mapped onto uli_syscall if you're running
> as a user-level interrupt.  However, this severely limits what you can
> do in a handler.  For example, you can't wake threads waiting on  a
> futex (something we find very common in a user-level interrupt handler).

    Currently its own semaphore approach is supported.  If you'd like me to
switch it to use futexes I can do that.
       	     	 	       	       		Thanks,
       	     	 	       	       			Michael
-- 
Michael A. Raymond              Office: (651) 683-3434
Core OS Group                   Real-Time System Software

Re: [PATCH] ULI 2.6.12

[Christoph Hellwig]

On Mon, Apr 04, 2005 at 08:31:44AM -0500, Michael Raymond wrote:
>     Again the idea is that the handler function was registered by root or
> some priviledged developer making a conscious decision.  This is currently
> protected by permissions on the /dev/uli file.  If you'd like me to add
> permission checks to the kernel functions I'd be happy to do it.

No, that's not the point.  User level interrupts are totally pointless if
they can cause the same problems on misbehaviour.