Re: [PATCH net-next-2.6] inetpeer: do not use zero refcnt for freed entries

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Wed, Jun 16, 2010 at 04:45:24AM +0200, Eric Dumazet wrote:
> Le mardi 15 juin 2010 à 14:25 -0700, David Miller a écrit :
> > From: Eric Dumazet <eric.dumazet@gmail.com>
> > Date: Tue, 15 Jun 2010 20:23:14 +0200
> > 
> > > inetpeer currently uses an AVL tree protected by an rwlock.
> > > 
> > > It's possible to make most lookups use RCU
> >  ...
> > > Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> > 
> > Applied, nice work Eric.
> 
> Thanks David !
> 
> Re-reading patch I realize refcnt is expected to be 0 for unused entries
> (obviously), so we should use a different marker for 'about to be freed'
> ones.
> 
> Thanks
> 
> [PATCH net-next-2.6] inetpeer: do not use zero refcnt for freed entries
> 
> Followup of commit aa1039e73cc2 (inetpeer: RCU conversion)
> 
> Unused inet_peer entries have a null refcnt.
> 
> Using atomic_inc_not_zero() in rcu lookups is not going to work for
> them, and slow path is taken.
> 
> Fix this using -1 marker instead of 0 for deleted entries.

Based on this patch, looks good to me!  (I don't see lookup_rcu_bh() and
friends in the trees I have at hand.)

Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> ---
>  net/ipv4/inetpeer.c |   10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/net/ipv4/inetpeer.c b/net/ipv4/inetpeer.c
> index 58fbc7e..39a14ba 100644
> --- a/net/ipv4/inetpeer.c
> +++ b/net/ipv4/inetpeer.c
> @@ -187,7 +187,12 @@ static struct inet_peer *lookup_rcu_bh(__be32 daddr)
> 
>  	while (u != peer_avl_empty) {
>  		if (daddr == u->v4daddr) {
> -			if (unlikely(!atomic_inc_not_zero(&u->refcnt)))
> +			/* Before taking a reference, check if this entry was
> +			 * deleted, unlink_from_pool() sets refcnt=-1 to make
> +			 * distinction between an unused entry (refcnt=0) and
> +			 * a freed one.
> +			 */
> +			if (unlikely(!atomic_add_unless(&u->refcnt, 1, -1)))
>  				u = NULL;
>  			return u;
>  		}
> @@ -322,8 +327,9 @@ static void unlink_from_pool(struct inet_peer *p)
>  	 * in cleanup() function to prevent sudden disappearing.  If we can
>  	 * atomically (because of lockless readers) take this last reference,
>  	 * it's safe to remove the node and free it later.
> +	 * We use refcnt=-1 to alert lockless readers this entry is deleted.
>  	 */
> -	if (atomic_cmpxchg(&p->refcnt, 1, 0) == 1) {
> +	if (atomic_cmpxchg(&p->refcnt, 1, -1) == 1) {
>  		struct inet_peer **stack[PEER_MAXDEPTH];
>  		struct inet_peer ***stackptr, ***delp;
>  		if (lookup(p->v4daddr, stack) != p)
> 
>