Re: Kernel rwlock design, Multicore and IGMP

[Eric Dumazet <eric.dumazet@gmail.com>]

Le jeudi 11 novembre 2010 à 16:23 +0100, Eric Dumazet a écrit :
> Le jeudi 11 novembre 2010 à 21:49 +0800, Cypher Wu a écrit :
> 
> Hi
> 
> CC netdev, since you ask questions about network stuff _and_ rwlock
> 
> 
> > I'm using TILEPro and its rwlock in kernel is a liitle different than
> > other platforms. It have a priority for write lock that when tried it
> > will block the following read lock even if read lock is hold by
> > others. Its code can be read in Linux Kernel 2.6.36 in
> > arch/tile/lib/spinlock_32.c.
> 
> This seems a bug to me.
> 
> read_lock() can be nested. We used such a schem in the past in iptables
> (it can re-enter itself),
> and we used instead a spinlock(), but with many discussions with lkml
> and Linus himself if I remember well.

I meant, a percpu spinlock, and extra logic to spin_lock() it one time,
even if nested.

static inline void xt_info_rdlock_bh(void)
{
        struct xt_info_lock *lock;

        local_bh_disable();
        lock = &__get_cpu_var(xt_info_locks);
        if (likely(!lock->readers++))
                spin_lock(&lock->lock);
}

static inline void xt_info_rdunlock_bh(void)
{
        struct xt_info_lock *lock = &__get_cpu_var(xt_info_locks);

        if (likely(!--lock->readers))
                spin_unlock(&lock->lock);
        local_bh_enable();
}

The write 'rwlock' side has to lock the percpu spinlock of all possible
cpus.

/*
 * The "writer" side needs to get exclusive access to the lock,
 * regardless of readers.  This must be called with bottom half
 * processing (and thus also preemption) disabled.
 */
static inline void xt_info_wrlock(unsigned int cpu)
{
        spin_lock(&per_cpu(xt_info_locks, cpu).lock);
}

static inline void xt_info_wrunlock(unsigned int cpu)
{
        spin_unlock(&per_cpu(xt_info_locks, cpu).lock);
}