brlock_is_locked()?

brlock_is_locked()?

[Brad Chapman]

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

Everyone,

	Is there a politically correct way to determine if a brlock
is in the locked or unlocked state, i.e. with something like this:

restart:
	if (brlock_is_locked(BR_NETPROTO_LOCK)) {
		CRITICAL_SECTION
		br_write_unlock_bh(BR_NETPROTO_LOCK);
	}
	else {
		/* Let's get dizzy */
		br_write_lock_bh(BR_NETPROTO_LOCK);
		goto restart;
	}

Brad

=====
Brad Chapman

Permanent e-mail: kakadu_croc@yahoo.com
Current e-mail: kakadu@adelphia.net

Reply to the address I used in the message to you,
please!

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: brlock_is_locked()?

[Ben LaHaise]

On Wed, 22 Aug 2001, Brad Chapman wrote:

> restart:
> 	if (brlock_is_locked(BR_NETPROTO_LOCK)) {
> 		CRITICAL_SECTION
> 		br_write_unlock_bh(BR_NETPROTO_LOCK);
> 	}
> 	else {
> 		/* Let's get dizzy */
> 		br_write_lock_bh(BR_NETPROTO_LOCK);
> 		goto restart;
> 	}

That code can never work.  None of the linux spinlocks track ownership, so
checking if a lock is locked tells you if your process or another has
ownership of the lock.  The above pseudo code is going to result in lots
of mangled data.

		-ben

Re: brlock_is_locked()?

[Brad Chapman]

[-- Attachment #1: Type: text/plain, Size: 1201 bytes --]

--- Ben LaHaise <bcrl@redhat.com> wrote:
> On Wed, 22 Aug 2001, Brad Chapman wrote:
> 
> > restart:
> > 	if (brlock_is_locked(BR_NETPROTO_LOCK)) {
> > 		CRITICAL_SECTION
> > 		br_write_unlock_bh(BR_NETPROTO_LOCK);
> > 	}
> > 	else {
> > 		/* Let's get dizzy */
> > 		br_write_lock_bh(BR_NETPROTO_LOCK);
> > 		goto restart;
> > 	}
> 
> That code can never work.  None of the linux spinlocks track ownership, so
> checking if a lock is locked tells you if your process or another has
> ownership of the lock.  The above pseudo code is going to result in lots
> of mangled data.
> 
> 		-ben
> 
Mr. LaHaise,

	Eeek! Sorry. What do you expect at 10:00 at night? ;-)

	I'm not talking about _who_ owns the lock, I'm talking about whether
the lock itself is locked. I don't care which process is using the lock;
I just want to know if _somebody_ is using it. Is this possible?

Brad


=====
Brad Chapman

Permanent e-mail: kakadu_croc@yahoo.com
Current e-mail: kakadu@adelphia.net

Reply to the address I used in the message to you,
please!

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: brlock_is_locked()?

[David S. Miller]

   From: Brad Chapman <kakadu_croc@yahoo.com>
   Date: Wed, 22 Aug 2001 11:00:56 -0700 (PDT)

   	I'm not talking about _who_ owns the lock, I'm talking about whether
   the lock itself is locked. I don't care which process is using the lock;
   I just want to know if _somebody_ is using it. Is this possible?

Show a valid use for such a boolean state, then
the discussion may proceed :-)

Later,
David S. Miller
davem@redhat.com

Re: brlock_is_locked()?

[Brad Chapman]

[-- Attachment #1: Type: text/plain, Size: 2039 bytes --]

--- "David S. Miller" <davem@redhat.com> wrote:
>    From: Brad Chapman <kakadu_croc@yahoo.com>
>    Date: Wed, 22 Aug 2001 11:00:56 -0700 (PDT)
> 
>    	I'm not talking about _who_ owns the lock, I'm talking about whether
>    the lock itself is locked. I don't care which process is using the lock;
>    I just want to know if _somebody_ is using it. Is this possible?
> 
> Show a valid use for such a boolean state, then
> the discussion may proceed :-)
> 
> Later,
> David S. Miller
> davem@redhat.com

Mr. Miller,

	I am debugging my port of the Netfilter IPv4 connection tracking
subsystem to IPv6. In some sections of the code, I had to split the code
into core functions, which did the actual work and could be used by people
with the appropriate locks, and wrapper functions, which grabbed the locks
themselves. 
	One such group of functions was the ip6_conntrack_protocol API.
To make life easier in the future, and to make third-party protocol
registration easier, I split the functions. In the unregistration function,
in order to be SMP-compliant we needed to grab BR_NETPROTO_LOCK to seal
the netfilter stack and prevent any packets from being scanned by the protocol
functions. When I split the unregistration function, I realized that there
was no way to detect if BR_NETPROTO_LOCK was locked, by _anyone_. Thus, if
someone  had BR_NETPROTO_LOCK and tried to call
ip6_conntrack_protocol_unregister() (the wrapper), we would deadlock.
	At this section of the code, it doesn't really matter who in the
netfilter/network stack has BR_NETPROTO_LOCK, just that we need to know if
it's already locked by someone else, or unlocked for us to use. Is what I'm
asking for physically possible?

Thanks,

Brad


=====
Brad Chapman

Permanent e-mail: kakadu_croc@yahoo.com
Current e-mail: kakadu@adelphia.net

Reply to the address I used in the message to you,
please!

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: brlock_is_locked()?

[David S. Miller]

   From: Brad Chapman <kakadu_croc@yahoo.com>
   Date: Wed, 22 Aug 2001 11:17:55 -0700 (PDT)

   	At this section of the code, it doesn't really matter who in the
   netfilter/network stack has BR_NETPROTO_LOCK, just that we need to know if
   it's already locked by someone else, or unlocked for us to use. Is what I'm
   asking for physically possible?

As a reader, the BR_NETPROTO_LOCK nests.  So no deadlock.

If the situation involves a writer, you must make sure that locking
rules are well defined and abided by.  No matter what you do, if you
try to conditionalize locking by testing if the lock is held, another
SMP can always get in there after you check it and corrupt your data.
The lock is basically pointless if you start testing it's locked
state.

Basically, "fix your code" ;-)

It's often easy to do this.  If you have two paths, one which is going
to already have the lock and one which won't, just make two functions
like so:

void __func(void)
{
	do_stuff();
}

void func(void)
{
	lock();
	__func();
	unlock();
}

I don't see why this is such a big problem.

Later,
David S. Miller
davem@redhat.com

Re: brlock_is_locked()?

[Brad Chapman]

[-- Attachment #1: Type: text/plain, Size: 2215 bytes --]

--- "David S. Miller" <davem@redhat.com> wrote:
>    From: Brad Chapman <kakadu_croc@yahoo.com>
>    Date: Wed, 22 Aug 2001 11:17:55 -0700 (PDT)
> 
>    	At this section of the code, it doesn't really matter who in the
>    netfilter/network stack has BR_NETPROTO_LOCK, just that we need to know if
>    it's already locked by someone else, or unlocked for us to use. Is what
> I'm
>    asking for physically possible?
> 
> As a reader, the BR_NETPROTO_LOCK nests.  So no deadlock.
> 
> If the situation involves a writer, you must make sure that locking
> rules are well defined and abided by.  No matter what you do, if you
> try to conditionalize locking by testing if the lock is held, another
> SMP can always get in there after you check it and corrupt your data.
> The lock is basically pointless if you start testing it's locked
> state.
> 
> Basically, "fix your code" ;-)
> 
> It's often easy to do this.  If you have two paths, one which is going
> to already have the lock and one which won't, just make two functions
> like so:
> 
> void __func(void)
> {
> 	do_stuff();
> }
> 
> void func(void)
> {
> 	lock();
> 	__func();
> 	unlock();
> }
> 
> I don't see why this is such a big problem.
> 
> Later,
> David S. Miller
> davem@redhat.com

Mr. Miller,

	It almost isn't. The problem starts when a third-party protocol
module grabs BR_NETPROTO_LOCK, unloads itself from the networking stack,
and then tries to call ip6_conntrack_protocol_unregister(). Deadlock.
The problem is that we need TWO locks: the brlock to seal the network stack,
and the conntrack rwlock to delete the protocol struct. Sure, you can
always share the rwlock and leave it at that, but if all you need it for
is to load/unload your protocol functions, then why bother polluting
the symbol tables?
	What do you think? Share the rwlock and make everybody who has
the brlock just use the core function?

Brad


=====
Brad Chapman

Permanent e-mail: kakadu_croc@yahoo.com
Current e-mail: kakadu@adelphia.net

Reply to the address I used in the message to you,
please!

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: brlock_is_locked()?

[David S. Miller]

   From: Brad Chapman <kakadu_croc@yahoo.com>
   Date: Wed, 22 Aug 2001 11:33:12 -0700 (PDT)
   
   	It almost isn't. The problem starts when a third-party protocol
   module grabs BR_NETPROTO_LOCK, unloads itself from the networking stack,
   and then tries to call ip6_conntrack_protocol_unregister(). Deadlock.
   The problem is that we need TWO locks: the brlock to seal the network stack,
   and the conntrack rwlock to delete the protocol struct. Sure, you can
   always share the rwlock and leave it at that, but if all you need it for
   is to load/unload your protocol functions, then why bother polluting
   the symbol tables?
   	What do you think? Share the rwlock and make everybody who has
   the brlock just use the core function?

You are only showing me that there is potential a deficiency in the
netfilter interfaces.  You ought to discuss with the netfilter people
a way to make the interfaces work better.

This is exactly what I said needed to be done.

Later,
David S. Miller
davem@redhat.com

   

Re: brlock_is_locked()?

[Brad Chapman]

[-- Attachment #1: Type: text/plain, Size: 2362 bytes --]

--- "David S. Miller" <davem@redhat.com> wrote:
>    From: Brad Chapman <kakadu_croc@yahoo.com>
>    Date: Wed, 22 Aug 2001 11:33:12 -0700 (PDT)
>    
>    	It almost isn't. The problem starts when a third-party protocol
>    module grabs BR_NETPROTO_LOCK, unloads itself from the networking stack,
>    and then tries to call ip6_conntrack_protocol_unregister(). Deadlock.
>    The problem is that we need TWO locks: the brlock to seal the network
> stack,
>    and the conntrack rwlock to delete the protocol struct. Sure, you can
>    always share the rwlock and leave it at that, but if all you need it for
>    is to load/unload your protocol functions, then why bother polluting
>    the symbol tables?
>    	What do you think? Share the rwlock and make everybody who has
>    the brlock just use the core function?
> 
> You are only showing me that there is potential a deficiency in the
> netfilter interfaces.  You ought to discuss with the netfilter people
> a way to make the interfaces work better.
> 
> This is exactly what I said needed to be done.
> 
> Later,
> David S. Miller
> davem@redhat.com

Mr. Miller,

	It's not really a deficiency. Rusty apparently decided that in
order to be SMP-compliant and to prevent Oopses, that the unregistration
function should grab the brlock so that all the packets would pass through
the protocol-handling functions. It doesn't necessarily _have_ to be done, 
because all the points which might make use of the protocol handlers are 
already locked or could be changed to use the lock, but if it should
be done, then you need a way to find out if the brlock is locked,
so that you can be fair to other people (I checked the brlock code and
didn't find any schedule()s; there's probably a reason for that).
	I suppose that if it really can't be done, then I'll remove the lock commands
for BR_NETPROTO_LOCK in the protocol API and just wait until people
report Evil Things(tm) happening when they unload. Besides, any netfilter
interface changes will have to wait until 2.5.

Brad   


=====
Brad Chapman

Permanent e-mail: kakadu_croc@yahoo.com
Current e-mail: kakadu@adelphia.net

Reply to the address I used in the message to you,
please!

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: brlock_is_locked()?

[David S. Miller]

   From: Brad Chapman <kakadu_croc@yahoo.com>
   Date: Wed, 22 Aug 2001 11:53:51 -0700 (PDT)

   	It's not really a deficiency. Rusty apparently decided that in
   order to be SMP-compliant and to prevent Oopses, that the unregistration
   function should grab the brlock so that all the packets would pass through
   the protocol-handling functions.

So arrange you code such that you aren't holding the netproto
lock when you call the unregistration function.

It is possible to shut down all references to whatever you
are unregistering, safely drop the lock, then call the
netfilter unregister routine.

   (I checked the brlock code and didn't find any schedule()s; there's
    probably a reason for that).

Ummm, this is SMP 101, you can't sleep with a lock held.
The global kernel lock is special in this regard, but all
other SMP locking primitives may not sleep.

Later,
David S. Miller
davem@redhat.com

Re: brlock_is_locked()?

[Brad Chapman]

[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]

Mr. Miller,

--- "David S. Miller" <davem@redhat.com> wrote:
>    From: Brad Chapman <kakadu_croc@yahoo.com>
>    Date: Wed, 22 Aug 2001 11:53:51 -0700 (PDT)
> 
>    	It's not really a deficiency. Rusty apparently decided that in
>    order to be SMP-compliant and to prevent Oopses, that the unregistration
>    function should grab the brlock so that all the packets would pass through
>    the protocol-handling functions.
> 
> So arrange you code such that you aren't holding the netproto
> lock when you call the unregistration function.
> 
> It is possible to shut down all references to whatever you
> are unregistering, safely drop the lock, then call the
> netfilter unregister routine.

	I understand that. What I'm afraid of is someone who writes a third-party
protocol module for ip_conntrack (or ip6_conntrack) and tries to call the wrapper
with BR_NETPROTO_LOCK held. LISB, IMHO the primary difficulty is not protecting
the protocol linked-list from packets inside the conntrack; it's protecting the
protocol linked-list from packets in the netfilter stack. FWICT, it should
be almost impossible for packets to move around in conntrack with the rwlock
but not the brlock; but strange things can happen......

> 
>    (I checked the brlock code and didn't find any schedule()s; there's
>     probably a reason for that).
> 
> Ummm, this is SMP 101, you can't sleep with a lock held.
> The global kernel lock is special in this regard, but all
> other SMP locking primitives may not sleep.

	Grrr....I read Rusty's Unreliable Guide to Kernel Locking (twice) and
still didn't remember that. Guess you have to schedule() yourself.

> 
> Later,
> David S. Miller
> davem@redhat.com

Brad


=====
Brad Chapman

Permanent e-mail: kakadu_croc@yahoo.com
Current e-mail: kakadu@adelphia.net

Reply to the address I used in the message to you,
please!

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: brlock_is_locked()?

[Jens Axboe]

On Wed, Aug 22 2001, Brad Chapman wrote:
> >    (I checked the brlock code and didn't find any schedule()s; there's
> >     probably a reason for that).
> > 
> > Ummm, this is SMP 101, you can't sleep with a lock held.
> > The global kernel lock is special in this regard, but all
> > other SMP locking primitives may not sleep.
> 
> 	Grrr....I read Rusty's Unreliable Guide to Kernel Locking (twice) and
> still didn't remember that. Guess you have to schedule() yourself.

Errr, like Dave said, _you cannot sleep while holding a lock_. It's not
just that the locking primitives themselves don't sleep, you must not
call schedule() (or any other function that may block/sleep) while
holding a lock. _That's_ SMP 101 :-)

-- 
Jens Axboe

Re: brlock_is_locked()?

[Brad Chapman]

--- Jens Axboe <axboe@suse.de> wrote:
> On Wed, Aug 22 2001, Brad Chapman wrote:
> > >    (I checked the brlock code and didn't find any schedule()s; there's
> > >     probably a reason for that).
> > > 
> > > Ummm, this is SMP 101, you can't sleep with a lock held.
> > > The global kernel lock is special in this regard, but all
> > > other SMP locking primitives may not sleep.
> > 
> > 	Grrr....I read Rusty's Unreliable Guide to Kernel Locking (twice) and
> > still didn't remember that. Guess you have to schedule() yourself.
> 
> Errr, like Dave said, _you cannot sleep while holding a lock_. It's not
> just that the locking primitives themselves don't sleep, you must not
> call schedule() (or any other function that may block/sleep) while
> holding a lock. _That's_ SMP 101 :-)
> 
> -- 
> Jens Axboe

Mr. Axboe,

	Sorry, I was unclear in my statement. If we fail to grab the lock, 
_then_ we sleep. If we succeed, we work our butts off, give the lock up,
and then schedule(). No sleeping on the lock job ;-)

Brad 


=====
Brad Chapman

Permanent e-mail: kakadu_croc@yahoo.com
Current e-mail: kakadu@adelphia.net
Alternate e-mail: kakadu@netscape.net

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/