[Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.

Both service are NOT just root-only services. E.g., rtdm_irq_request
requires by specification support also over non-Linux contexts.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---

We run all critical code under an I-pipe spin lock, so this restriction
seems artificial. Nevertheless, I briefly checked if some arch has
specific needs but didn't find any trace in core-3.5.

Wolfgang, this already resolves the issue I mentioned regarding startup
of RTnet NICs.

 kernel/ipipe/core.c |    4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diff --git a/kernel/ipipe/core.c b/kernel/ipipe/core.c
index 9898b92..23a2146 100644
--- a/kernel/ipipe/core.c
+++ b/kernel/ipipe/core.c
@@ -846,8 +846,6 @@ int ipipe_request_irq(struct ipipe_domain *ipd,
 	unsigned long flags;
 	int ret = 0;
 
-	ipipe_root_only();
-
 	if (handler == NULL ||
 	    (irq >= IPIPE_NR_XIRQS && !ipipe_virtual_irq_p(irq)))
 		return -EINVAL;
@@ -881,8 +879,6 @@ void ipipe_free_irq(struct ipipe_domain *ipd,
 {
 	unsigned long flags;
 
-	ipipe_root_only();
-
 	spin_lock_irqsave(&__ipipe_lock, flags);
 
 	if (ipd->irqs[irq].handler == NULL)
-- 
1.7.3.4

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Philippe Gerum]

On 09/20/2012 12:37 PM, Jan Kiszka wrote:
> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
> 
> Both service are NOT just root-only services. E.g., rtdm_irq_request
> requires by specification support also over non-Linux contexts.

Nack. We can't run the enable code for MSIs over non-root, and
that code typically follows the irq request. Besides, we want to mask
the source upon irq free to handle the SMP case properly, which we could
not do from non-root with MSIs.

So either we have both request+enable and free usable over non-root, or
there is no point.

> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> We run all critical code under an I-pipe spin lock, so this restriction
> seems artificial. Nevertheless, I briefly checked if some arch has
> specific needs but didn't find any trace in core-3.5.
> 
> Wolfgang, this already resolves the issue I mentioned regarding startup
> of RTnet NICs.
> 
>  kernel/ipipe/core.c |    4 ----
>  1 files changed, 0 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/ipipe/core.c b/kernel/ipipe/core.c
> index 9898b92..23a2146 100644
> --- a/kernel/ipipe/core.c
> +++ b/kernel/ipipe/core.c
> @@ -846,8 +846,6 @@ int ipipe_request_irq(struct ipipe_domain *ipd,
>  	unsigned long flags;
>  	int ret = 0;
>  
> -	ipipe_root_only();
> -
>  	if (handler == NULL ||
>  	    (irq >= IPIPE_NR_XIRQS && !ipipe_virtual_irq_p(irq)))
>  		return -EINVAL;
> @@ -881,8 +879,6 @@ void ipipe_free_irq(struct ipipe_domain *ipd,
>  {
>  	unsigned long flags;
>  
> -	ipipe_root_only();
> -
>  	spin_lock_irqsave(&__ipipe_lock, flags);
>  
>  	if (ipd->irqs[irq].handler == NULL)
> 


-- 
Philippe.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 12:49, Philippe Gerum wrote:
> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>
>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>> requires by specification support also over non-Linux contexts.
> 
> Nack. We can't run the enable code for MSIs over non-root, and
> that code typically follows the irq request. Besides, we want to mask
> the source upon irq free to handle the SMP case properly, which we could
> not do from non-root with MSIs.
> 
> So either we have both request+enable and free usable over non-root, or
> there is no point.

OK, I get the point with legacy MSI. Then we have two other bugs to solve:
 - in I-pipe as it holds a hardened spin lock across enable/disable (of
   MSIs)
 - in Xenomai 2.6 (at least, didn't check forge) as it calls with a
   hardened spin lock held into ipipe_virtualize_irq

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 12:56, Jan Kiszka wrote:
> On 2012-09-20 12:49, Philippe Gerum wrote:
>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>
>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>> requires by specification support also over non-Linux contexts.
>>
>> Nack. We can't run the enable code for MSIs over non-root, and
>> that code typically follows the irq request. Besides, we want to mask
>> the source upon irq free to handle the SMP case properly, which we could
>> not do from non-root with MSIs.
>>
>> So either we have both request+enable and free usable over non-root, or
>> there is no point.
> 
> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>    MSIs)
>  - in Xenomai 2.6 (at least, didn't check forge) as it calls with a
>    hardened spin lock held into ipipe_virtualize_irq

Sorry, three bugs:
 - in the RTDM spec as it always allowed rtdm_irq_request over RT task
   contexts

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Philippe Gerum]

On 09/20/2012 12:57 PM, Jan Kiszka wrote:
> On 2012-09-20 12:56, Jan Kiszka wrote:
>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>
>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>> requires by specification support also over non-Linux contexts.
>>>
>>> Nack. We can't run the enable code for MSIs over non-root, and
>>> that code typically follows the irq request. Besides, we want to mask
>>> the source upon irq free to handle the SMP case properly, which we could
>>> not do from non-root with MSIs.
>>>
>>> So either we have both request+enable and free usable over non-root, or
>>> there is no point.
>>
>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>    MSIs)

Missed it. Where?

>>  - in Xenomai 2.6 (at least, didn't check forge) as it calls with a
>>    hardened spin lock held into ipipe_virtualize_irq
> 
> Sorry, three bugs:
>  - in the RTDM spec as it always allowed rtdm_irq_request over RT task
>    contexts
>

Yes. This was likely designed when ->enable/disable() handlers would not
traverse a whole pile of hairy code highly dependent on the PCI
infrastructure like today. And on the ppc side, this is not getting any
prettier.

> Jan
> 


-- 
Philippe.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 12:57, Jan Kiszka wrote:
> On 2012-09-20 12:56, Jan Kiszka wrote:
>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>
>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>> requires by specification support also over non-Linux contexts.
>>>
>>> Nack. We can't run the enable code for MSIs over non-root, and
>>> that code typically follows the irq request. Besides, we want to mask
>>> the source upon irq free to handle the SMP case properly, which we could
>>> not do from non-root with MSIs.
>>>
>>> So either we have both request+enable and free usable over non-root, or
>>> there is no point.
>>
>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>    MSIs)

I think this bug may only manifest over ARM as that arch does
enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
something prevents that enabling will ever happen for interrupts that
need Linux locks to work. Is that assured?

>>  - in Xenomai 2.6 (at least, didn't check forge) as it calls with a
>>    hardened spin lock held into ipipe_virtualize_irq

This problem is something I vaguely recall we discussed before already
in the past. I think there was no good solution for the Xenomai 2
architecture.

In this light, let's remove those checks nevertheless.
Enabling/disabling the IRQ are separate calls, and those should be
instrumented as those cause the restriction.

> 
> Sorry, three bugs:
>  - in the RTDM spec as it always allowed rtdm_irq_request over RT task
>    contexts

Changing the spec and adding a runtime check will likely be no issue. No
sane driver should have made use of that option. I will file a patch.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 13:15, Jan Kiszka wrote:
>> Sorry, three bugs:
>>  - in the RTDM spec as it always allowed rtdm_irq_request over RT task
>>    contexts
> 
> Changing the spec and adding a runtime check will likely be no issue. No
> sane driver should have made use of that option. I will file a patch.

Naturally, this also affects rtdm_irq_enable/disable, and some use cases
I'm aware off will bail out when we add assertions for non-RT context.

I would prefer to address it more selectively, allowing all those
services for IRQ types that support it (i.e. that can be atomically
enabled/disabled), keeping the freedom to reject or otherwise fail for
types that are in conflict.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Gilles Chanteperdrix]

On 09/20/2012 01:15 PM, Jan Kiszka wrote:

> On 2012-09-20 12:57, Jan Kiszka wrote:
>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>
>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>> requires by specification support also over non-Linux contexts.
>>>>
>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>> that code typically follows the irq request. Besides, we want to mask
>>>> the source upon irq free to handle the SMP case properly, which we could
>>>> not do from non-root with MSIs.
>>>>
>>>> So either we have both request+enable and free usable over non-root, or
>>>> there is no point.
>>>
>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>    MSIs)
> 
> I think this bug may only manifest over ARM as that arch does
> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
> something prevents that enabling will ever happen for interrupts that
> need Linux locks to work. Is that assured?


ARM does that because otherwise interrupts are not enabled after a call
to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work,
calling request_irq is required to get them working.

-- 
                                                                Gilles.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Philippe Gerum]

On 09/20/2012 01:15 PM, Jan Kiszka wrote:
> On 2012-09-20 12:57, Jan Kiszka wrote:
>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>
>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>> requires by specification support also over non-Linux contexts.
>>>>
>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>> that code typically follows the irq request. Besides, we want to mask
>>>> the source upon irq free to handle the SMP case properly, which we could
>>>> not do from non-root with MSIs.
>>>>
>>>> So either we have both request+enable and free usable over non-root, or
>>>> there is no point.
>>>
>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>    MSIs)
> 
> I think this bug may only manifest over ARM as that arch does
> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
> something prevents that enabling will ever happen for interrupts that
> need Linux locks to work. Is that assured?

I'm not referring to enable_irqdesc, but to the common programming
pattern of calling ipipe_request_irq from the same context than
ipipe_enable_irq (or directly the underlying irqchip handler for unmasking).

> 
>>>  - in Xenomai 2.6 (at least, didn't check forge) as it calls with a
>>>    hardened spin lock held into ipipe_virtualize_irq
> 
> This problem is something I vaguely recall we discussed before already
> in the past. I think there was no good solution for the Xenomai 2
> architecture.
> 

The main issue was about de-registering a handler, passing NULL. To
solve the SMP-specific issue of interrupt synchronization on all CPUs,
we would have to be able to disable the source, which may entail running
regular code, therefore restricting the valid calling context to root.

> In this light, let's remove those checks nevertheless.
> Enabling/disabling the IRQ are separate calls, and those should be
> instrumented as those cause the restriction.
> 

I don't see it this way, because we can't predict what will be the
constraints we might have for hooking irqs on all archs we will support.
Maybe we will have to run more mainline code in some cases. In any case,
we have to fold masking into the de-registration code for proper SMP
support - this fix was never finalized precisely because we could not
guarantee a root calling context in that case.

These checks are there to express the fact that calling from non-root is
inherently unsafe. We might find a (ugly) way to tag irq descriptors,
for knowing whether this is safe to call from non-root and test this
conditionally. But at the end of the day, we would still end up with
checking for arch-specific constraints in a generic API, which would be
wrong by design.

I put these checks when refactoring the pipeline API for the very same
reason than you agree to update the RTDM spec regarding
rtdm_request_irq: no sane code should have called ipipe_virtualize_irq()
from a non-root context. This is just about formalizing this fact.

>>
>> Sorry, three bugs:
>>  - in the RTDM spec as it always allowed rtdm_irq_request over RT task
>>    contexts
> 
> Changing the spec and adding a runtime check will likely be no issue. No
> sane driver should have made use of that option. I will file a patch.
> 

Ack.

> Jan
> 


-- 
Philippe.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 15:01, Gilles Chanteperdrix wrote:
> On 09/20/2012 01:15 PM, Jan Kiszka wrote:
> 
>> On 2012-09-20 12:57, Jan Kiszka wrote:
>>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>>
>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>>> requires by specification support also over non-Linux contexts.
>>>>>
>>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>>> that code typically follows the irq request. Besides, we want to mask
>>>>> the source upon irq free to handle the SMP case properly, which we could
>>>>> not do from non-root with MSIs.
>>>>>
>>>>> So either we have both request+enable and free usable over non-root, or
>>>>> there is no point.
>>>>
>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>>    MSIs)
>>
>> I think this bug may only manifest over ARM as that arch does
>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
>> something prevents that enabling will ever happen for interrupts that
>> need Linux locks to work. Is that assured?
> 
> 
> ARM does that because otherwise interrupts are not enabled after a call
> to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work,
> calling request_irq is required to get them working.

The generic pattern Xenomai currently uses is xnintr_attach +
xnintr_enable. The latter happens to be rtdm_irq_enable as well. So,
doesn't xnintr_enable work at all? Or is it only not suited as interrupt
startup? Then, is that a new issue or something core-3.x specific?
Finally, can't we address it in xnarch_enable_irq?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 15:10, Philippe Gerum wrote:
> On 09/20/2012 01:15 PM, Jan Kiszka wrote:
>> On 2012-09-20 12:57, Jan Kiszka wrote:
>>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>>
>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>>> requires by specification support also over non-Linux contexts.
>>>>>
>>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>>> that code typically follows the irq request. Besides, we want to mask
>>>>> the source upon irq free to handle the SMP case properly, which we could
>>>>> not do from non-root with MSIs.
>>>>>
>>>>> So either we have both request+enable and free usable over non-root, or
>>>>> there is no point.
>>>>
>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>>    MSIs)
>>
>> I think this bug may only manifest over ARM as that arch does
>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
>> something prevents that enabling will ever happen for interrupts that
>> need Linux locks to work. Is that assured?
> 
> I'm not referring to enable_irqdesc, but to the common programming
> pattern of calling ipipe_request_irq from the same context than
> ipipe_enable_irq (or directly the underlying irqchip handler for unmasking).

Patterns are better enforced at higher layers, not here.

> 
>>
>>>>  - in Xenomai 2.6 (at least, didn't check forge) as it calls with a
>>>>    hardened spin lock held into ipipe_virtualize_irq
>>
>> This problem is something I vaguely recall we discussed before already
>> in the past. I think there was no good solution for the Xenomai 2
>> architecture.
>>
> 
> The main issue was about de-registering a handler, passing NULL. To
> solve the SMP-specific issue of interrupt synchronization on all CPUs,
> we would have to be able to disable the source, which may entail running
> regular code, therefore restricting the valid calling context to root.

I'm not sure what the conclusion regarding this tricky topic was. But I
don't see how this can be addressed only at I-pipe level anyway.

> 
>> In this light, let's remove those checks nevertheless.
>> Enabling/disabling the IRQ are separate calls, and those should be
>> instrumented as those cause the restriction.
>>
> 
> I don't see it this way, because we can't predict what will be the
> constraints we might have for hooking irqs on all archs we will support.
> Maybe we will have to run more mainline code in some cases. In any case,
> we have to fold masking into the de-registration code for proper SMP
> support - this fix was never finalized precisely because we could not
> guarantee a root calling context in that case.
> 
> These checks are there to express the fact that calling from non-root is
> inherently unsafe. We might find a (ugly) way to tag irq descriptors,
> for knowing whether this is safe to call from non-root and test this
> conditionally. But at the end of the day, we would still end up with
> checking for arch-specific constraints in a generic API, which would be
> wrong by design.
> 
> I put these checks when refactoring the pipeline API for the very same
> reason than you agree to update the RTDM spec regarding
> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq()
> from a non-root context. This is just about formalizing this fact.

...at the wrong point. Plus it is breaking our instrumentation. Again:
this belongs where we can asses the problem better, i.e. in the higher
layer and in those functions that do break when called from RT context.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Philippe Gerum]

On 09/20/2012 03:54 PM, Jan Kiszka wrote:
> On 2012-09-20 15:10, Philippe Gerum wrote:
>> On 09/20/2012 01:15 PM, Jan Kiszka wrote:
>>> On 2012-09-20 12:57, Jan Kiszka wrote:
>>>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>>>
>>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>>>> requires by specification support also over non-Linux contexts.
>>>>>>
>>>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>>>> that code typically follows the irq request. Besides, we want to mask
>>>>>> the source upon irq free to handle the SMP case properly, which we could
>>>>>> not do from non-root with MSIs.
>>>>>>
>>>>>> So either we have both request+enable and free usable over non-root, or
>>>>>> there is no point.
>>>>>
>>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>>>    MSIs)
>>>
>>> I think this bug may only manifest over ARM as that arch does
>>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
>>> something prevents that enabling will ever happen for interrupts that
>>> need Linux locks to work. Is that assured?
>>
>> I'm not referring to enable_irqdesc, but to the common programming
>> pattern of calling ipipe_request_irq from the same context than
>> ipipe_enable_irq (or directly the underlying irqchip handler for unmasking).
> 
> Patterns are better enforced at higher layers, not here.
> 

The -core series is really about designing a Xenomai-specific real-time
enabler.

>>
>>>
>>>>>  - in Xenomai 2.6 (at least, didn't check forge) as it calls with a
>>>>>    hardened spin lock held into ipipe_virtualize_irq
>>>
>>> This problem is something I vaguely recall we discussed before already
>>> in the past. I think there was no good solution for the Xenomai 2
>>> architecture.
>>>
>>
>> The main issue was about de-registering a handler, passing NULL. To
>> solve the SMP-specific issue of interrupt synchronization on all CPUs,
>> we would have to be able to disable the source, which may entail running
>> regular code, therefore restricting the valid calling context to root.
> 
> I'm not sure what the conclusion regarding this tricky topic was. But I
> don't see how this can be addressed only at I-pipe level anyway.

The I-pipe knows about IRQs, and does very much know what it entails do
deal with the kernel infrastructure regarding this. It does abstract
this from the client code, enforcing rules to make sure the interface
remains applicable to all archs. These rules have an impact on client
code, we have to live with it.

> 
>>
>>> In this light, let's remove those checks nevertheless.
>>> Enabling/disabling the IRQ are separate calls, and those should be
>>> instrumented as those cause the restriction.
>>>
>>
>> I don't see it this way, because we can't predict what will be the
>> constraints we might have for hooking irqs on all archs we will support.
>> Maybe we will have to run more mainline code in some cases. In any case,
>> we have to fold masking into the de-registration code for proper SMP
>> support - this fix was never finalized precisely because we could not
>> guarantee a root calling context in that case.
>>
>> These checks are there to express the fact that calling from non-root is
>> inherently unsafe. We might find a (ugly) way to tag irq descriptors,
>> for knowing whether this is safe to call from non-root and test this
>> conditionally. But at the end of the day, we would still end up with
>> checking for arch-specific constraints in a generic API, which would be
>> wrong by design.
>>
>> I put these checks when refactoring the pipeline API for the very same
>> reason than you agree to update the RTDM spec regarding
>> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq()
>> from a non-root context. This is just about formalizing this fact.
> 
> ...at the wrong point. Plus it is breaking our instrumentation. Again:
> this belongs where we can asses the problem better, i.e. in the higher
> layer and in those functions that do break when called from RT context.

I don't think so.

> 
> Jan
> 


-- 
Philippe.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Gilles Chanteperdrix]

On 09/20/2012 03:15 PM, Jan Kiszka wrote:
> On 2012-09-20 15:01, Gilles Chanteperdrix wrote:
>> On 09/20/2012 01:15 PM, Jan Kiszka wrote:
>>
>>> On 2012-09-20 12:57, Jan Kiszka wrote:
>>>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>>>
>>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>>>> requires by specification support also over non-Linux contexts.
>>>>>>
>>>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>>>> that code typically follows the irq request. Besides, we want to mask
>>>>>> the source upon irq free to handle the SMP case properly, which we could
>>>>>> not do from non-root with MSIs.
>>>>>>
>>>>>> So either we have both request+enable and free usable over non-root, or
>>>>>> there is no point.
>>>>>
>>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>>>    MSIs)
>>>
>>> I think this bug may only manifest over ARM as that arch does
>>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
>>> something prevents that enabling will ever happen for interrupts that
>>> need Linux locks to work. Is that assured?
>>
>>
>> ARM does that because otherwise interrupts are not enabled after a call
>> to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work,
>> calling request_irq is required to get them working.
> 
> The generic pattern Xenomai currently uses is xnintr_attach +
> xnintr_enable. The latter happens to be rtdm_irq_enable as well. So,
> doesn't xnintr_enable work at all?

No. It has in fact, never worked before the I-pipe core. People had to
call request_irq before xnintr_attach.

> Or is it only not suited as interrupt
> startup? Then, is that a new issue or something core-3.x specific?
> Finally, can't we address it in xnarch_enable_irq?

This is a very old issue, that I had forgotten, and that bit me when I
tried to use a separate timer for Xenomai and Linux on omap3: because
the timer interrupt of a different timer has no corresponding linux
interrupt.

The problem is that irqs which are also used by Linux have to be treated
differently from irqs which are registered only for the linux domain.

-- 
					    Gilles.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 16:12, Gilles Chanteperdrix wrote:
> On 09/20/2012 03:15 PM, Jan Kiszka wrote:
>> On 2012-09-20 15:01, Gilles Chanteperdrix wrote:
>>> On 09/20/2012 01:15 PM, Jan Kiszka wrote:
>>>
>>>> On 2012-09-20 12:57, Jan Kiszka wrote:
>>>>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>>>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>>>>
>>>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>>>>> requires by specification support also over non-Linux contexts.
>>>>>>>
>>>>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>>>>> that code typically follows the irq request. Besides, we want to mask
>>>>>>> the source upon irq free to handle the SMP case properly, which we could
>>>>>>> not do from non-root with MSIs.
>>>>>>>
>>>>>>> So either we have both request+enable and free usable over non-root, or
>>>>>>> there is no point.
>>>>>>
>>>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>>>>    MSIs)
>>>>
>>>> I think this bug may only manifest over ARM as that arch does
>>>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
>>>> something prevents that enabling will ever happen for interrupts that
>>>> need Linux locks to work. Is that assured?
>>>
>>>
>>> ARM does that because otherwise interrupts are not enabled after a call
>>> to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work,
>>> calling request_irq is required to get them working.
>>
>> The generic pattern Xenomai currently uses is xnintr_attach +
>> xnintr_enable. The latter happens to be rtdm_irq_enable as well. So,
>> doesn't xnintr_enable work at all?
> 
> No. It has in fact, never worked before the I-pipe core. People had to
> call request_irq before xnintr_attach.
> 
>> Or is it only not suited as interrupt
>> startup? Then, is that a new issue or something core-3.x specific?
>> Finally, can't we address it in xnarch_enable_irq?
> 
> This is a very old issue, that I had forgotten, and that bit me when I
> tried to use a separate timer for Xenomai and Linux on omap3: because
> the timer interrupt of a different timer has no corresponding linux
> interrupt.
> 
> The problem is that irqs which are also used by Linux have to be treated
> differently from irqs which are registered only for the linux domain.

OK. Then we can safely assume that this initial enabling/disabling will
not cause excessive latencies or even acquire Linux locks on ARM, right?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 16:05, Philippe Gerum wrote:
>>>> In this light, let's remove those checks nevertheless.
>>>> Enabling/disabling the IRQ are separate calls, and those should be
>>>> instrumented as those cause the restriction.
>>>>
>>>
>>> I don't see it this way, because we can't predict what will be the
>>> constraints we might have for hooking irqs on all archs we will support.
>>> Maybe we will have to run more mainline code in some cases. In any case,
>>> we have to fold masking into the de-registration code for proper SMP
>>> support - this fix was never finalized precisely because we could not
>>> guarantee a root calling context in that case.
>>>
>>> These checks are there to express the fact that calling from non-root is
>>> inherently unsafe. We might find a (ugly) way to tag irq descriptors,
>>> for knowing whether this is safe to call from non-root and test this
>>> conditionally. But at the end of the day, we would still end up with
>>> checking for arch-specific constraints in a generic API, which would be
>>> wrong by design.
>>>
>>> I put these checks when refactoring the pipeline API for the very same
>>> reason than you agree to update the RTDM spec regarding
>>> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq()
>>> from a non-root context. This is just about formalizing this fact.
>>
>> ...at the wrong point. Plus it is breaking our instrumentation. Again:
>> this belongs where we can asses the problem better, i.e. in the higher
>> layer and in those functions that do break when called from RT context.
> 
> I don't think so.

I do agree that the caller of ipipe_request_irq should not be called
over RT context. However, ipipe_request_irq itself _is_ called under an
RT spin lock with hardirqs disabled, both under Xenomai 2.6 and upcoming
3. And this triggers CONFIG_IPIPE_DEBUG_CONTEXT, clearly showing that
things are still broken here. What do you suggest to fix all this?

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Gilles Chanteperdrix]

On 09/20/2012 05:00 PM, Jan Kiszka wrote:
> On 2012-09-20 16:12, Gilles Chanteperdrix wrote:
>> On 09/20/2012 03:15 PM, Jan Kiszka wrote:
>>> On 2012-09-20 15:01, Gilles Chanteperdrix wrote:
>>>> On 09/20/2012 01:15 PM, Jan Kiszka wrote:
>>>>
>>>>> On 2012-09-20 12:57, Jan Kiszka wrote:
>>>>>> On 2012-09-20 12:56, Jan Kiszka wrote:
>>>>>>> On 2012-09-20 12:49, Philippe Gerum wrote:
>>>>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote:
>>>>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672.
>>>>>>>>>
>>>>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request
>>>>>>>>> requires by specification support also over non-Linux contexts.
>>>>>>>>
>>>>>>>> Nack. We can't run the enable code for MSIs over non-root, and
>>>>>>>> that code typically follows the irq request. Besides, we want to mask
>>>>>>>> the source upon irq free to handle the SMP case properly, which we could
>>>>>>>> not do from non-root with MSIs.
>>>>>>>>
>>>>>>>> So either we have both request+enable and free usable over non-root, or
>>>>>>>> there is no point.
>>>>>>>
>>>>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve:
>>>>>>>  - in I-pipe as it holds a hardened spin lock across enable/disable (of
>>>>>>>    MSIs)
>>>>>
>>>>> I think this bug may only manifest over ARM as that arch does
>>>>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless
>>>>> something prevents that enabling will ever happen for interrupts that
>>>>> need Linux locks to work. Is that assured?
>>>>
>>>>
>>>> ARM does that because otherwise interrupts are not enabled after a call
>>>> to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work,
>>>> calling request_irq is required to get them working.
>>>
>>> The generic pattern Xenomai currently uses is xnintr_attach +
>>> xnintr_enable. The latter happens to be rtdm_irq_enable as well. So,
>>> doesn't xnintr_enable work at all?
>>
>> No. It has in fact, never worked before the I-pipe core. People had to
>> call request_irq before xnintr_attach.
>>
>>> Or is it only not suited as interrupt
>>> startup? Then, is that a new issue or something core-3.x specific?
>>> Finally, can't we address it in xnarch_enable_irq?
>>
>> This is a very old issue, that I had forgotten, and that bit me when I
>> tried to use a separate timer for Xenomai and Linux on omap3: because
>> the timer interrupt of a different timer has no corresponding linux
>> interrupt.
>>
>> The problem is that irqs which are also used by Linux have to be treated
>> differently from irqs which are registered only for the linux domain.
> 
> OK. Then we can safely assume that this initial enabling/disabling will
> not cause excessive latencies or even acquire Linux locks on ARM, right?

mmmm. enable_irq may take the descriptor spinlock, I have to check that.

-- 
					    Gilles.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Philippe Gerum]

On 09/20/2012 05:07 PM, Jan Kiszka wrote:
> On 2012-09-20 16:05, Philippe Gerum wrote:
>>>>> In this light, let's remove those checks nevertheless.
>>>>> Enabling/disabling the IRQ are separate calls, and those should be
>>>>> instrumented as those cause the restriction.
>>>>>
>>>>
>>>> I don't see it this way, because we can't predict what will be the
>>>> constraints we might have for hooking irqs on all archs we will support.
>>>> Maybe we will have to run more mainline code in some cases. In any case,
>>>> we have to fold masking into the de-registration code for proper SMP
>>>> support - this fix was never finalized precisely because we could not
>>>> guarantee a root calling context in that case.
>>>>
>>>> These checks are there to express the fact that calling from non-root is
>>>> inherently unsafe. We might find a (ugly) way to tag irq descriptors,
>>>> for knowing whether this is safe to call from non-root and test this
>>>> conditionally. But at the end of the day, we would still end up with
>>>> checking for arch-specific constraints in a generic API, which would be
>>>> wrong by design.
>>>>
>>>> I put these checks when refactoring the pipeline API for the very same
>>>> reason than you agree to update the RTDM spec regarding
>>>> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq()
>>>> from a non-root context. This is just about formalizing this fact.
>>>
>>> ...at the wrong point. Plus it is breaking our instrumentation. Again:
>>> this belongs where we can asses the problem better, i.e. in the higher
>>> layer and in those functions that do break when called from RT context.
>>
>> I don't think so.
> 
> I do agree that the caller of ipipe_request_irq should not be called
> over RT context. However, ipipe_request_irq itself _is_ called under an
> RT spin lock with hardirqs disabled, both under Xenomai 2.6 and upcoming
> 3. And this triggers CONFIG_IPIPE_DEBUG_CONTEXT, clearly showing that
> things are still broken here. What do you suggest to fix all this?
> 

Fix the callers in the upcoming Xenomai releases, and provide an open
coded implementation of ipipe_request_irq in ipipe_virtualize_irq when
CONFIG_IPIPE_LEGACY is enabled, which will take care of the broken legacy.

-- 
Philippe.

Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services"

[Jan Kiszka]

On 2012-09-20 17:16, Philippe Gerum wrote:
> On 09/20/2012 05:07 PM, Jan Kiszka wrote:
>> On 2012-09-20 16:05, Philippe Gerum wrote:
>>>>>> In this light, let's remove those checks nevertheless.
>>>>>> Enabling/disabling the IRQ are separate calls, and those should be
>>>>>> instrumented as those cause the restriction.
>>>>>>
>>>>>
>>>>> I don't see it this way, because we can't predict what will be the
>>>>> constraints we might have for hooking irqs on all archs we will support.
>>>>> Maybe we will have to run more mainline code in some cases. In any case,
>>>>> we have to fold masking into the de-registration code for proper SMP
>>>>> support - this fix was never finalized precisely because we could not
>>>>> guarantee a root calling context in that case.
>>>>>
>>>>> These checks are there to express the fact that calling from non-root is
>>>>> inherently unsafe. We might find a (ugly) way to tag irq descriptors,
>>>>> for knowing whether this is safe to call from non-root and test this
>>>>> conditionally. But at the end of the day, we would still end up with
>>>>> checking for arch-specific constraints in a generic API, which would be
>>>>> wrong by design.
>>>>>
>>>>> I put these checks when refactoring the pipeline API for the very same
>>>>> reason than you agree to update the RTDM spec regarding
>>>>> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq()
>>>>> from a non-root context. This is just about formalizing this fact.
>>>>
>>>> ...at the wrong point. Plus it is breaking our instrumentation. Again:
>>>> this belongs where we can asses the problem better, i.e. in the higher
>>>> layer and in those functions that do break when called from RT context.
>>>
>>> I don't think so.
>>
>> I do agree that the caller of ipipe_request_irq should not be called
>> over RT context. However, ipipe_request_irq itself _is_ called under an
>> RT spin lock with hardirqs disabled, both under Xenomai 2.6 and upcoming
>> 3. And this triggers CONFIG_IPIPE_DEBUG_CONTEXT, clearly showing that
>> things are still broken here. What do you suggest to fix all this?
>>
> 
> Fix the callers in the upcoming Xenomai releases, and provide an open
> coded implementation of ipipe_request_irq in ipipe_virtualize_irq when
> CONFIG_IPIPE_LEGACY is enabled, which will take care of the broken legacy.

Both callers have the same structure, so we can either fix both or none.
Fortunately, we appear to be fine by simply converting intrlock to a
plain Linux mutex. Will check twice later.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux