"upping" a semaphore from interrupt context?

"upping" a semaphore from interrupt context?

[Florin Iucha]

[-- Attachment #1: Type: text/plain, Size: 686 bytes --]

Hello,

I am writing a USB driver for some custom hardware, and I need to
synchronize between the user-space and the USB subsystem.  Can I
create a semaphore and "down" it in the reader then "up" it in the
completion handler?

I know the completion handler runs in interrupt context so you are not
allowed to acquire any semaphores: but can you release them?  Will the
waiting tasks wake up after the handler and its caller returned - IOW
will the waking up task run in interrupt context as well?

This is with Linux 2.4 (if it makes a difference).

Thanks,
florin

-- 
Bruce Schneier expects the Spanish Inquisition.
      http://geekz.co.uk/schneierfacts/fact/163

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: "upping" a semaphore from interrupt context?

[Arnd Bergmann]

On Friday 22 June 2007, Florin Iucha wrote:
> I am writing a USB driver for some custom hardware, and I need to
> synchronize between the user-space and the USB subsystem.  Can I
> create a semaphore and "down" it in the reader then "up" it in the
> completion handler?

yes, but you should not. The use of semaphores is not recommended
for new code, it should be replaced with either a mutex or a
completion.

> I know the completion handler runs in interrupt context so you are not
> allowed to acquire any semaphores: but can you release them?  Will the
> waiting tasks wake up after the handler and its caller returned - IOW
> will the waking up task run in interrupt context as well?

What you should use is a 'struct completion', which you can wait for
in process context and complete from interrupt context.

	Arnd <><

Re: "upping" a semaphore from interrupt context?

[Robert P. J. Day]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 996 bytes --]

On Fri, 22 Jun 2007, Arnd Bergmann wrote:

> On Friday 22 June 2007, Florin Iucha wrote:
> > I am writing a USB driver for some custom hardware, and I need to
> > synchronize between the user-space and the USB subsystem.  Can I
> > create a semaphore and "down" it in the reader then "up" it in the
> > completion handler?
>
> yes, but you should not. The use of semaphores is not recommended
> for new code, it should be replaced with either a mutex or a
> completion.

can you clarify this?  it sounds like you're saying that the current
implementation of semaphores is entirely superfluous.  but surely it
isn't possible to replace all semaphores with either mutexes or
completions, is it?

rday
-- 
========================================================================
Robert P. J. Day
Linux Consulting, Training and Annoying Kernel Pedantry
Waterloo, Ontario, CANADA

http://fsdev.net/wiki/index.php?title=Main_Page
========================================================================

Re: "upping" a semaphore from interrupt context?

[Arnd Bergmann]

On Saturday 23 June 2007, Robert P. J. Day wrote:
> On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> >
> > yes, but you should not. The use of semaphores is not recommended
> > for new code, it should be replaced with either a mutex or a
> > completion.
> 
> can you clarify this?  it sounds like you're saying that the current
> implementation of semaphores is entirely superfluous.  but surely it
> isn't possible to replace all semaphores with either mutexes or
> completions, is it?

No, not all of them, but the vast majority. There are multiple
differences, the most important one being the 'counting' in
semaphores. You can e.g. define a semaphore that can be held
by N users at the same time, but not more. In a mutex, N is
by definition 1, so only one thread can hold a mutex.

There are other subtle differences in the implementation, e.g.
you cannot mutex_trylock at interrupt time.

	Arnd <><

Re: "upping" a semaphore from interrupt context?

[Robert P. J. Day]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1240 bytes --]

On Sat, 23 Jun 2007, Arnd Bergmann wrote:

> On Saturday 23 June 2007, Robert P. J. Day wrote:
> > On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> > >
> > > yes, but you should not. The use of semaphores is not
> > > recommended for new code, it should be replaced with either a
> > > mutex or a completion.
> >
> > can you clarify this?  it sounds like you're saying that the
> > current implementation of semaphores is entirely superfluous.  but
> > surely it isn't possible to replace all semaphores with either
> > mutexes or completions, is it?
>
> No, not all of them, but the vast majority. There are multiple
> differences, the most important one being the 'counting' in
> semaphores.

right, that was exactly the feature i was thinking of.  ok, i'm clear
on this now -- while the *majority* of semaphores can be more properly
replaced by mutexes or completions, there will always be a need for a
general-purpose counting semaphore.

rday
-- 
========================================================================
Robert P. J. Day
Linux Consulting, Training and Annoying Kernel Pedantry
Waterloo, Ontario, CANADA

http://fsdev.net/wiki/index.php?title=Main_Page
========================================================================

Re: "upping" a semaphore from interrupt context?

[Satyam Sharma]

Hi Robert, Arnd,

On 6/23/07, Arnd Bergmann <arnd@arndb.de> wrote:
> On Saturday 23 June 2007, Robert P. J. Day wrote:
> > On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> > >
> > > yes, but you should not. The use of semaphores is not recommended
> > > for new code, it should be replaced with either a mutex or a
> > > completion.
> >
> > can you clarify this? it sounds like you're saying that the current
> > implementation of semaphores is entirely superfluous. but surely it
> > isn't possible to replace all semaphores with either mutexes or
> > completions, is it?

Semaphores being used as completions are superfluous, obsoleted
by completion handlers. Semaphores that are not counted (hence
binary) are superfluous, obsoleted by struct mutex. It's not that using
semaphores for the above two usages would be "incorrect", it's just that
the other options are precisely implemented for their specific purpose
and hence are better (benchmark struct mutex against (binary) struct
semaphore yourself, for example). So there's no good reason why a
driver's design would want to use semaphores as above with the other
available options.

A simple way to detect users who are still {mis}using semaphores
as completions are those that will (thus) inevitably have to declare
(or initialize) them as locked, say most (all?) users of
DECLARE_MUTEX_LOCKED.

22-rc5 has 4 such users, one of which is dead code inside a #if 0,
one declared a spurious semaphore variable without using it
anywhere else in the code (deleted in -mm), one has been converted
to completions (in -mm) already, and the last one (libusual.c) is the
problematic one which still exists, because of the comment in that
file that it wants the _completions_ to be also *counted*, and thus
ostensibly wants to use semaphores instead of completions.
However, that comment is totally wrong, and doesn't seem to know
about the existence of the complete_all() function that precisely
serves the purpose that libusual wants.

In short: there are _no_ valid excuses to use DECLARE_MUTEX_LOCKED
(or semaphores as completions) ever.

[ libusual's {ab}use of that semaphore is *insane* for several other
reasons, but then that will be the subject matter of another thread :-) ]

> No, not all of them, but the vast majority. There are multiple
> differences, the most important one being the 'counting' in
> semaphores. You can e.g. define a semaphore that can be held
> by N users at the same time, but not more. In a mutex, N is
> by definition 1, so only one thread can hold a mutex.
>
> There are other subtle differences in the implementation, e.g.
> you cannot mutex_trylock at interrupt time.

IMHO, mutexes (or semaphores, be it binary ones or counted)
are simply *not* the correct locking primitive to use from interrupt
context in the first place. down_trylock() did work from interrupt
contexts earlier (without producing any __might_sleep-induced
warnings either) but mutex_trylock() does not work, and rightly so.
Somebody using a down_trylock() from interrupt context is a
sure-shot sign of design error (or massive ugliness) in the driver
somewhere (which could, and should, be resolved in design).

To elaborate: what is the (only) valid way in which such code
(that attempts to down_trylock() or mutex_trylock() from interrupt
context) can be written?

*****
err = down_trylock(...);
if (err) {
	/*
	 * Either:
	 * 1. proceed although someone else owns the lock anyway => insane
	 * 2. too bad, so ignore this interrupt just this once => WTF?
	 * 3. set up a timer and schedule another function to service the
	 *    interrupt / do what needs to be done then, hopefully the mutex
	 *    would be uncontended then => *gargh*
	 */
}
/* do what needs to be done with the shared data to service this interrupt */
*****

Option 2 is nonsense. Option 3 (believe me, I've seen code like that) is even
more so :-) [ note that timers run in softirq-context, so illegal to use
mutex_lock() or down() there again, which means we'd *again* have to use
_trylock() variant there, with the same ugly error handling I described above. ]

Satyam

Re: "upping" a semaphore from interrupt context?

[Oliver Neukum]

Am Samstag, 23. Juni 2007 schrieb Satyam Sharma:
>          * 3. set up a timer and schedule another function to service the
>          *    interrupt / do what needs to be done then, hopefully the mutex
>          *    would be uncontended then => *gargh*

You could use schedule_work(). However then why not use it always.
This would make sense if what you want to do is outright trivial.

	Regards
		Oliver

Re: "upping" a semaphore from interrupt context?

[Satyam Sharma]

On 6/23/07, Oliver Neukum <oliver@neukum.org> wrote:
> Am Samstag, 23. Juni 2007 schrieb Satyam Sharma:
> > * 3. set up a timer and schedule another function to service the
> > * interrupt / do what needs to be done then, hopefully the mutex
> > * would be uncontended then => *gargh*
>
> You could use schedule_work(). However then why not use it always.
> This would make sense if what you want to do is outright trivial.

If you use schedule_work() to pass off work from interrupt context
to process context, then you wouldn't be calling down_trylock()
from interrupt context in the first place (which is what is being
discussed here). You would simply pass off the entire code that
uses the shared data (and wraps a *proper* down() or mutex_lock()
around it, not the _trylock() variant) to the workqueue.

Also, that is precisely my point too. What I'm saying is that it is
generally poor design to be wanting to use the _trylock() variant
of semaphore / mutex in interrupt context. Workqueues _are_ the
preferred mechanism to use for (most) such cases where you need
to do something that may require you to sleep.

Satyam

Re: "upping" a semaphore from interrupt context?

[Robert P. J. Day]

On Sat, 23 Jun 2007, Satyam Sharma wrote:

> Hi Robert, Arnd,
>
> On 6/23/07, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Saturday 23 June 2007, Robert P. J. Day wrote:
> > > On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> > > >
> > > > yes, but you should not. The use of semaphores is not recommended
> > > > for new code, it should be replaced with either a mutex or a
> > > > completion.
> > >
> > > can you clarify this? it sounds like you're saying that the current
> > > implementation of semaphores is entirely superfluous. but surely it
> > > isn't possible to replace all semaphores with either mutexes or
> > > completions, is it?
>
> Semaphores being used as completions are superfluous, obsoleted by
> completion handlers. Semaphores that are not counted (hence binary)
> are superfluous, obsoleted by struct mutex.

hang on, how is that true?  as i read it, mutexes are more than just
binary semaphores -- they have additional restrictions that regular
semaphores don't.  so i'm not convinced that binary semaphores can
simply be replaced by mutexes, unless that's not what you meant here.

rday

-- 
========================================================================
Robert P. J. Day
Linux Consulting, Training and Annoying Kernel Pedantry
Waterloo, Ontario, CANADA

http://fsdev.net/wiki/index.php?title=Main_Page
========================================================================

Re: "upping" a semaphore from interrupt context?

[Satyam Sharma]

On 6/23/07, Robert P. J. Day <rpjday@mindspring.com> wrote:
> On Sat, 23 Jun 2007, Satyam Sharma wrote:
>
> > Hi Robert, Arnd,
> >
> > On 6/23/07, Arnd Bergmann <arnd@arndb.de> wrote:
> > > On Saturday 23 June 2007, Robert P. J. Day wrote:
> > > > On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> > > > >
> > > > > yes, but you should not. The use of semaphores is not recommended
> > > > > for new code, it should be replaced with either a mutex or a
> > > > > completion.
> > > >
> > > > can you clarify this? it sounds like you're saying that the current
> > > > implementation of semaphores is entirely superfluous. but surely it
> > > > isn't possible to replace all semaphores with either mutexes or
> > > > completions, is it?
> >
> > Semaphores being used as completions are superfluous, obsoleted by
> > completion handlers. Semaphores that are not counted (hence binary)
> > are superfluous, obsoleted by struct mutex.
>
> hang on, how is that true?  as i read it, mutexes are more than just
> binary semaphores -- they have additional restrictions that regular
> semaphores don't.

Yes, they do have additional restrictions (mutex_trylock() illegal from
contexts that cannot sleep, mutexes may only be unlocked by tasks
that took them in the first place). But note that these are
_implementation_ sanity checks that were introduced to catch
nonsensical usage, which was possible (and not explicitly being
guarded against, because of the generic-ness that was needed to
be maintained for the counted case too) with the "semaphore"s.

> so i'm not convinced that binary semaphores can
> simply be replaced by mutexes, unless that's not what you meant here.

I do mean precisely that. I really cannot think of any sensible / normal
usage case of binary semaphores that cannot be replaced with either
mutexes (if that's the kind of locking you actually want) or completion
handlers (if that's the kind of synchronization you actually want).

Satyam

Re: "upping" a semaphore from interrupt context?

[Robert P. J. Day]

On Sat, 23 Jun 2007, Satyam Sharma wrote:

> On 6/23/07, Robert P. J. Day <rpjday@mindspring.com> wrote:

> > hang on, how is that true?  as i read it, mutexes are more than
> > just binary semaphores -- they have additional restrictions that
> > regular semaphores don't.
>
> Yes, they do have additional restrictions (mutex_trylock() illegal
> from contexts that cannot sleep, mutexes may only be unlocked by
> tasks that took them in the first place). But note that these are
> _implementation_ sanity checks that were introduced to catch
> nonsensical usage, which was possible (and not explicitly being
> guarded against, because of the generic-ness that was needed to be
> maintained for the counted case too) with the "semaphore"s.
>
> > so i'm not convinced that binary semaphores can simply be replaced
> > by mutexes, unless that's not what you meant here.
>
> I do mean precisely that. I really cannot think of any sensible /
> normal usage case of binary semaphores that cannot be replaced with
> either mutexes (if that's the kind of locking you actually want) or
> completion handlers (if that's the kind of synchronization you
> actually want).

ok, i can see what you're getting at now.  i'm just going to have to
convince myself that, once you have mutexes and completions, you have
no further need for even general counting semaphores.

rday

Re: "upping" a semaphore from interrupt context?

[Satyam Sharma]

Hi Robert,

On 6/24/07, Robert P. J. Day <rpjday@mindspring.com> wrote:
> On Sat, 23 Jun 2007, Satyam Sharma wrote:
>
> > On 6/23/07, Robert P. J. Day <rpjday@mindspring.com> wrote:
>
> > > hang on, how is that true?  as i read it, mutexes are more than
> > > just binary semaphores -- they have additional restrictions that
> > > regular semaphores don't.
> >
> > Yes, they do have additional restrictions (mutex_trylock() illegal
> > from contexts that cannot sleep, mutexes may only be unlocked by
> > tasks that took them in the first place). But note that these are
> > _implementation_ sanity checks that were introduced to catch
> > nonsensical usage, which was possible (and not explicitly being
> > guarded against, because of the generic-ness that was needed to be
> > maintained for the counted case too) with the "semaphore"s.
> >
> > > so i'm not convinced that binary semaphores can simply be replaced
> > > by mutexes, unless that's not what you meant here.
> >
> > I do mean precisely that. I really cannot think of any sensible /
> > normal usage case of binary semaphores that cannot be replaced with
> > either mutexes (if that's the kind of locking you actually want) or
> > completion handlers (if that's the kind of synchronization you
> > actually want).
>
> ok, i can see what you're getting at now.  i'm just going to have to
> convince myself that, once you have mutexes and completions, you have
> no further need for even general counting semaphores.

Whoa, hold on. But I've been explicitly mentioning *binary* semaphores
all along!

Of course users who want / allow multiple tasks (but only upto a specific
maximum number, which is what counted semaphores are all about)
to be present in a given critical section simultaneously would still want
to use the _counted_ semaphores, which is why you won't see the old
"struct semaphores" dying anytime soon.

Satyam

Re: "upping" a semaphore from interrupt context?

[Robert P. J. Day]

On Sun, 24 Jun 2007, Satyam Sharma wrote:

> Whoa, hold on. But I've been explicitly mentioning *binary*
> semaphores all along!
>
> Of course users who want / allow multiple tasks (but only upto a
> specific maximum number, which is what counted semaphores are all
> about) to be present in a given critical section simultaneously
> would still want to use the _counted_ semaphores, which is why you
> won't see the old "struct semaphores" dying anytime soon.

ah, ok, i've been misreading all this, sorry.  i'm wondering, then,
if, given both mutexes and completions, whether the general semantics
of semaphores can be tightened up at all.  just curious.

rday
-- 
========================================================================
Robert P. J. Day
Linux Consulting, Training and Annoying Kernel Pedantry
Waterloo, Ontario, CANADA

http://fsdev.net/wiki/index.php?title=Main_Page
========================================================================

Re: "upping" a semaphore from interrupt context?

[Satyam Sharma]

Hi,

On 6/24/07, Robert P. J. Day <rpjday@mindspring.com> wrote:
> On Sun, 24 Jun 2007, Satyam Sharma wrote:
>
> > Whoa, hold on. But I've been explicitly mentioning *binary*
> > semaphores all along!
> >
> > Of course users who want / allow multiple tasks (but only upto a
> > specific maximum number, which is what counted semaphores are all
> > about) to be present in a given critical section simultaneously
> > would still want to use the _counted_ semaphores, which is why you
> > won't see the old "struct semaphores" dying anytime soon.
>
> ah, ok, i've been misreading all this, sorry.  i'm wondering, then,
> if, given both mutexes and completions, whether the general semantics
> of semaphores can be tightened up at all.  just curious.

[ Ok, I'll elaborate, just in case someone else interested regarding
this is watching or reading the archives ... but this may be waaaaay
too basic for you and others here, so those can safely ignore it. ]

First: By a semaphore being binary, I mean that the value of that
semaphore (counter) variable can only be 0 or 1. Say we follow the
rule that 1 == available (unlocked) and 0 == unavailable (locked).
So down == lock == test-and-decrement, up == unlock == increment.

Also, note that there is nothing (I'm _not_ talking of implementation
details here) that the "struct mutex" can do something that a *binary*
"struct semaphore" can't, if usage ensures that its value is always
in {0, 1}. So semantically speaking they are equivalent.

1. (Binary) semaphores used for locking:
========================================

Binary semaphores may be used to synchronize access to critical sections
that can (must) only be executed by _only_ *one* task at any given time.
That is, _no_ concurrent execution of that piece of code can be tolerated.
This is often the most common case / usage to implement lock-protection
of any shared data.

Now obviously, because only _one_ task can be in such critical sections
at any given time, who (and only who) can be allowed to unlock the
semaphore (i.e. increment the counter variable back to 1)?

The task that took this (binary) semaphore in the first place, isn't it?
So this precisely one of those extra sanity checks that the "struct
mutex" introduces. [ There are other sanity checks listed at the top of
include/linux/mutex.h, but this is one of the important ones, IMHO. ]

Why is this check _not_ applicable to *counting* semaphores? Because
_multiple_ tasks can be in that given critical section _concurrently_
so the task that _last_ down()'ed the semaphore *need not be* forced
to be same as the task that _next_ wants to up() it!

2. Semaphores used to indicate completion:
==========================================

Ok, this one's again purely an implementation thing -- in fact if you
look at the way "struct semaphore"s are currently implemented in the
kernel they _do_ use waitqueues, which is precisely what
"struct completion"s use as well. The difference is even less pronounced
because all down()'s do become (in the slowpath i.e. when the test failed
so we decide to schedule() ourselves out) _add_wait_queue_tail with
callers marking themselves as exclusive waiters), and all up()'s also do
call __wake_up_common with NR_EXCLUSIVE == 1 to ensure that at
least one exclusively-waiting task _is_ woken up when the lock is
released (i.e. counter incremented).

So this basically means that one _can_ use semaphores as completion
notification primitives, and also (to again give the technically
accurate answer to Florin's original question on this thread) "upping"
is not illegal per-se from interrupt/atomic contexts.

But wait_for_completion() and complete() are to be still preferred in
new code (although they have a very similar implementation) because they
are specifically for this purpose, plus they avoid all the extra
fastpath/lock-acquiring stuff that the semaphore implementation involves
(and one less member in the struct if nothing else). [ I have never
benchmarked, of course, but I'd expect use of the completion primitives
to be microscopically faster than using semaphores in this manner too. ]

Satyam

Re: "upping" a semaphore from interrupt context?

[Satyam Sharma]

Hi Florin,

On 6/22/07, Florin Iucha <florin@iucha.net> wrote:
> Hello,
>
> I am writing a USB driver for some custom hardware, and I need to
> synchronize between the user-space and the USB subsystem.  Can I
> create a semaphore and "down" it in the reader then "up" it in the
> completion handler?

It's not exactly clear from your description what you are
"synchronizing" exactly ... if there is some shared data you want
accessed from process as well as interrupt context, the (only) safe
primitives to use are spin_lock_irqsave / spin_unlock_irqrestore.

If you simply want the process context task to block _till_ it receives
some kind of notification that some other job has been completed
(seems to be from interrupt context in your case from your
description) than I suspect "struct completion" and associated
primitives are what you are looking for.

> I know the completion handler runs in interrupt context so you are not
> allowed to acquire any semaphores: but can you release them?  Will the
> waiting tasks wake up after the handler and its caller returned - IOW
> will the waking up task run in interrupt context as well?

The waiting task (say if it was blocked at wait_for_completion) would
continue to execute in the same context it was from that point onwards.

> This is with Linux 2.4 (if it makes a difference).

Whoa, it does, I would expect. I'm not sure completion handlers
exist in 2.4? If they do, they're the way to go. If not, sorry, I'm not
well versed with the 2.4 kernel at all.

Satyam