spinlocking between user context / tasklet / tophalf question

spinlocking between user context / tasklet / tophalf question

[Emmanuel Michon]

Hi,

I read various documents about spinlocks, including Linux device
drivers by A.Rubini 2nd edition, Unreliable guide to locking by P.R.Russel,
and the source code of mainly network device drivers.

I'm trying to achieve correct SMP synchronization on the Sigma Designs
EM84xx; this one involves an extra small hardware interrupt (let's call it tophalf),
only one tasklet scheduled at end of tophalf, and usual kernel side code of
ioctl() I call usercontext.

tophalf and tasklet are potentially writing the same data X

tasklet and usercontext are potentially writing the same data Y

So, my first guess was to use two spinlocks, X_lock and Y_lock, 

with

tophalf()
{
        spin_lock(&X_lock);
        write X
        spin_unlock(&X_lock);
}

tasklet()
{
        unsigned long flags;
        spin_lock_irqsave(&X_lock,flags);
        write X
        spin_lock(&Y_lock);
        write X, write Y
        spin_unlock(&Y_lock);
        write X
        spin_unlock_irqrestore(&X_lock,flags);
}

ioctl()
{
        spin_lock_bh(&Y_lock);
        write Y ... maybe copy_from_user/copy_to_user
        spin_unlock_bh(&Y_lock);
}

So far I get really hardcore freezes and I'm trying to handle this with kgdb

1. Should I use spin_lock(&Y_lock); or spin_lock_bh(&Y_lock); in the tasklet body?

2. What is the reality behind: ``things which sleep'', is it really a problem
to use copy_from_user/copy_to_user holding a spinlock?

3. Previous version used one semaphore to serialize usercontext access 
down_interruptible(&sem)/up(&sem)
and handle tasklet concurrency with:
down_trylock(&sem)/up(&sem)

That allowed to catch signals (^C) with the usual -ERESTARTSYS stuff. As
far as I understand, spinlocks allow the serialization but no way to interrupt
a dead system call --- should I keep the semaphore only for this purpose?

Sincerely yours,

-- 
Emmanuel Michon
Chef de projet
REALmagic France SAS
Mobile: 0614372733 GPGkeyID: D2997E42  

Re: spinlocking between user context / tasklet / tophalf question

[george anzinger]

Emmanuel Michon wrote:
> 
> Hi,
> 
> I read various documents about spinlocks, including Linux device
> drivers by A.Rubini 2nd edition, Unreliable guide to locking by P.R.Russel,
> and the source code of mainly network device drivers.
> 
> I'm trying to achieve correct SMP synchronization on the Sigma Designs
> EM84xx; this one involves an extra small hardware interrupt (let's call it tophalf),
> only one tasklet scheduled at end of tophalf, and usual kernel side code of
> ioctl() I call usercontext.
> 
> tophalf and tasklet are potentially writing the same data X
> 
> tasklet and usercontext are potentially writing the same data Y
> 
> So, my first guess was to use two spinlocks, X_lock and Y_lock,

Tasklets are run in interrupt context.  You need the irq versions of the
spinlock in kernel space.  In tasklet space a simple spinlock should be
enough as the tasklet can not be reentered.

-g
> 
> with
> 
> tophalf()
> {
>         spin_lock(&X_lock);
>         write X
>         spin_unlock(&X_lock);
> }
> 
> tasklet()
> {
>         unsigned long flags;
>         spin_lock_irqsave(&X_lock,flags);
>         write X
>         spin_lock(&Y_lock);
>         write X, write Y
>         spin_unlock(&Y_lock);
>         write X
>         spin_unlock_irqrestore(&X_lock,flags);
> }
> 
> ioctl()
> {
>         spin_lock_bh(&Y_lock);
>         write Y ... maybe copy_from_user/copy_to_user
>         spin_unlock_bh(&Y_lock);
> }
> 
> So far I get really hardcore freezes and I'm trying to handle this with kgdb
> 
> 1. Should I use spin_lock(&Y_lock); or spin_lock_bh(&Y_lock); in the tasklet body?
> 
> 2. What is the reality behind: ``things which sleep'', is it really a problem
> to use copy_from_user/copy_to_user holding a spinlock?
> 
> 3. Previous version used one semaphore to serialize usercontext access
> down_interruptible(&sem)/up(&sem)
> and handle tasklet concurrency with:
> down_trylock(&sem)/up(&sem)
> 
> That allowed to catch signals (^C) with the usual -ERESTARTSYS stuff. As
> far as I understand, spinlocks allow the serialization but no way to interrupt
> a dead system call --- should I keep the semaphore only for this purpose?
> 
> Sincerely yours,
> 
> --
> Emmanuel Michon
> Chef de projet
> REALmagic France SAS
> Mobile: 0614372733 GPGkeyID: D2997E42
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
George Anzinger   george@mvista.com
High-res-timers:  http://sourceforge.net/projects/high-res-timers/
Real time sched:  http://sourceforge.net/projects/rtsched/
Preemption patch: http://www.kernel.org/pub/linux/kernel/people/rml

Re: spinlocking between user context / tasklet / tophalf question

[Robert Love]

On Fri, 2002-04-26 at 04:52, Emmanuel Michon wrote:

> 1. Should I use spin_lock(&Y_lock); or spin_lock_bh(&Y_lock); in the tasklet
> body?

You would want to use spin_lock_bh to serialize against other softirqs. 
That does not seem to be a need, here.

> 2. What is the reality behind: ``things which sleep'', is it really a problem
> to use copy_from_user/copy_to_user holding a spinlock?

Yes, they sleep.

	Robert Love

Re: spinlocking between user context / tasklet / tophalf question

[Alan Cox]

> Tasklets are run in interrupt context.  You need the irq versions of the

Not always. Ksoftirqd...


> spinlock in kernel space.  In tasklet space a simple spinlock should be
> enough as the tasklet can not be reentered.

That depends what you are locking against.

Re: spinlocking between user context / tasklet / tophalf question

[Daniel Phillips]

Hi Robert,

On Friday 26 April 2002 21:19, Robert Love wrote:
> On Fri, 2002-04-26 at 04:52, Emmanuel Michon wrote:
> > 2. What is the reality behind: ``things which sleep'', is it really a problem
> > to use copy_from_user/copy_to_user holding a spinlock?
> 
> Yes, they sleep.

Well, I think he really wanted to know why it's bad.  Emmanuel, suppose process A
takes spinlock S then sleeps.  Process B is then scheduled and tries to acquire
S - bang, deadlock: Process A can't release the spinlock because it's sleeping.

Let alone the fact that another CPU trying to acquire S is going to end up
stalled potentially many milliseconds before process A wakes up again and
releases the spinlock.

-- 
Daniel