[Xenomai-core] Spinlock question.

[Xenomai-core] Spinlock question.

[Gilles Chanteperdrix]

Hi,

I am porting a network driver from Linux to RTnet, and I have a sudden
doubt about what I am doing: is it safe to call a function like
wake_up_interruptible, from a non real-time context, with CONFIG_PREEMPT
enabled, while holding an rthal spinlock ? Do not I risk a rescheduling
with the spinlock locked ?

-- 
                                                 Gilles Chanteperdrix

Re: [Xenomai-core] Spinlock question.

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

Gilles Chanteperdrix wrote:
> Hi,
> 
> I am porting a network driver from Linux to RTnet, and I have a sudden
> doubt about what I am doing: is it safe to call a function like
> wake_up_interruptible, from a non real-time context, with CONFIG_PREEMPT
> enabled, while holding an rthal spinlock ? Do not I risk a rescheduling
> with the spinlock locked ?

Mmh, doesn't stalling the primary domain (what rthal_spin_lock_irqsave
surely does) also make the root domain think it is running in interrupt
context, thus will prevent any Linux reschedule?

However, there is one big fat issue with this approach:
wake_up_interruptible takes some Linux spin lock. So you must never call
it from primary context on UP to avoid corruption. And an SMP, you
create a nice source for priority inversion: CPU2 holds wait_queue lock,
gets preempted by Xenomai and runs some longer low-prio RT job.
Meanwhile CPU1 tries to acquire wait_queue as well with IRQs disabled,
blocking any RT task on that CPU.

Better defer the Linux wake-up via a rtdm_nrtsig. Alternatively, make
the Linux waiter also a Xenomai task and use a rtdm_event e.g. (mmh,
maybe we should discuss the auto-shadow-any-Linux-task-on-demand feature
for Xenomai once again...).

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-core] Spinlock question.

[Dmitry Adamushko]

[-- Attachment #1: Type: text/plain, Size: 2184 bytes --]

On 26/03/07, Jan Kiszka <jan.kiszka@domain.hid> wrote:
>
> Gilles Chanteperdrix wrote:
> > Hi,
> >
> > I am porting a network driver from Linux to RTnet, and I have a sudden
> > doubt about what I am doing: is it safe to call a function like
> > wake_up_interruptible, from a non real-time context, with CONFIG_PREEMPT
> > enabled, while holding an rthal spinlock ? Do not I risk a rescheduling
> > with the spinlock locked ?
>
> Mmh, doesn't stalling the primary domain (what rthal_spin_lock_irqsave
> surely does) also make the root domain think it is running in interrupt
> context, thus will prevent any Linux reschedule?


I don't know the PREEMPT_RT code. Although, I do remember I asked Philippe
once and the subject was some code in nucleus/pipe.c

Just read the comments on PREEMPT_RT in xnpipe_wakeup_proc()... Philippe may
comment more on that.


However, there is one big fat issue with this approach:
> wake_up_interruptible takes some Linux spin lock. So you must never call
> it from primary context on UP to avoid corruption. And an SMP, you
> create a nice source for priority inversion: CPU2 holds wait_queue lock,
> gets preempted by Xenomai and runs some longer low-prio RT job.
> Meanwhile CPU1 tries to acquire wait_queue as well with IRQs disabled,
> blocking any RT task on that CPU.


Jan, Gilles is talking about calling this function from non-rt context :o)

"...is it safe to call a function like wake_up_interruptible, from a *non
real-time* context, with CONFIG_PREEMPT"

And for calling it from the real-time context, It's not only about a
spinlock but a list (of pending tasks) as well and, in general, any data
this routine may access and which can be /inconsistent/ at the moment. So
it's a no go.


Better defer the Linux wake-up via a rtdm_nrtsig. Alternatively, make
> the Linux waiter also a Xenomai task and use a rtdm_event e.g. (mmh,
> maybe we should discuss the auto-shadow-any-Linux-task-on-demand feature
> for Xenomai once again...).
>
> Jan
>
>
> _______________________________________________
> Xenomai-core mailing list
> Xenomai-core@domain.hid
> https://mail.gna.org/listinfo/xenomai-core
>
>
>


-- 
Best regards,
Dmitry Adamushko

[-- Attachment #2: Type: text/html, Size: 3038 bytes --]

Re: [Xenomai-core] Spinlock question.

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
> 
>>Hi,
>>
>>I am porting a network driver from Linux to RTnet, and I have a sudden
>>doubt about what I am doing: is it safe to call a function like
>>wake_up_interruptible, from a non real-time context, with CONFIG_PREEMPT
>>enabled, while holding an rthal spinlock ? Do not I risk a rescheduling
>>with the spinlock locked ?
> 
> 
> Mmh, doesn't stalling the primary domain (what rthal_spin_lock_irqsave
> surely does) also make the root domain think it is running in interrupt
> context, thus will prevent any Linux reschedule?

Is not it sufficient to add a call to preempt_disable/preempt_enable
around the spinlock functions ?


> However, there is one big fat issue with this approach:
> wake_up_interruptible takes some Linux spin lock. So you must never call
> it from primary context on UP to avoid corruption. And an SMP, you
> create a nice source for priority inversion: CPU2 holds wait_queue lock,
> gets preempted by Xenomai and runs some longer low-prio RT job.
> Meanwhile CPU1 tries to acquire wait_queue as well with IRQs disabled,
> blocking any RT task on that CPU.
> 
> Better defer the Linux wake-up via a rtdm_nrtsig. Alternatively, make
> the Linux waiter also a Xenomai task and use a rtdm_event e.g. (mmh,
> maybe we should discuss the auto-shadow-any-Linux-task-on-demand feature
> for Xenomai once again...).

I have other cases where the function that does the wakeup do it from
real-time context, so I will probably have to use the nrtsig approach
anyway, thanks.

-- 
                                                 Gilles Chanteperdrix

Re: [Xenomai-core] Spinlock question.

[Philippe Gerum]

On Mon, 2007-03-26 at 17:59 +0200, Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
> > Gilles Chanteperdrix wrote:
> > 
> >>Hi,
> >>
> >>I am porting a network driver from Linux to RTnet, and I have a sudden
> >>doubt about what I am doing: is it safe to call a function like
> >>wake_up_interruptible, from a non real-time context, with CONFIG_PREEMPT
> >>enabled, while holding an rthal spinlock ? Do not I risk a rescheduling
> >>with the spinlock locked ?
> > 
> > 
> > Mmh, doesn't stalling the primary domain (what rthal_spin_lock_irqsave
> > surely does) also make the root domain think it is running in interrupt
> > context, thus will prevent any Linux reschedule?
> 
> Is not it sufficient to add a call to preempt_disable/preempt_enable
> around the spinlock functions ?
> 

Yes, you could do that, and actually, you should do that, since rthal_*
macros manipulate raw spinlocks, not preempt-aware ones. This said, as
Jan pointed out, it would be pretty dangerous to hold such spinlock
right before going to sleep Linux-wise, due to priority inversion
issues.

The bottom-line is to never ever hold the nklock and anything that could
stall the high pipeline stage before being switched out by the Linux
scheduler.

> 
> > However, there is one big fat issue with this approach:
> > wake_up_interruptible takes some Linux spin lock. So you must never call
> > it from primary context on UP to avoid corruption. And an SMP, you
> > create a nice source for priority inversion: CPU2 holds wait_queue lock,
> > gets preempted by Xenomai and runs some longer low-prio RT job.
> > Meanwhile CPU1 tries to acquire wait_queue as well with IRQs disabled,
> > blocking any RT task on that CPU.
> > 
> > Better defer the Linux wake-up via a rtdm_nrtsig. Alternatively, make
> > the Linux waiter also a Xenomai task and use a rtdm_event e.g. (mmh,
> > maybe we should discuss the auto-shadow-any-Linux-task-on-demand feature
> > for Xenomai once again...).
> 
> I have other cases where the function that does the wakeup do it from
> real-time context, so I will probably have to use the nrtsig approach
> anyway, thanks.
> 
-- 
Philippe.