* [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" @ 2012-09-20 10:37 Jan Kiszka 2012-09-20 10:49 ` Philippe Gerum 0 siblings, 1 reply; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 10:37 UTC (permalink / raw) To: Philippe Gerum, Gilles Chanteperdrix; +Cc: Wolfgang Mauerer, Xenomai This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. Both service are NOT just root-only services. E.g., rtdm_irq_request requires by specification support also over non-Linux contexts. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> --- We run all critical code under an I-pipe spin lock, so this restriction seems artificial. Nevertheless, I briefly checked if some arch has specific needs but didn't find any trace in core-3.5. Wolfgang, this already resolves the issue I mentioned regarding startup of RTnet NICs. kernel/ipipe/core.c | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/kernel/ipipe/core.c b/kernel/ipipe/core.c index 9898b92..23a2146 100644 --- a/kernel/ipipe/core.c +++ b/kernel/ipipe/core.c @@ -846,8 +846,6 @@ int ipipe_request_irq(struct ipipe_domain *ipd, unsigned long flags; int ret = 0; - ipipe_root_only(); - if (handler == NULL || (irq >= IPIPE_NR_XIRQS && !ipipe_virtual_irq_p(irq))) return -EINVAL; @@ -881,8 +879,6 @@ void ipipe_free_irq(struct ipipe_domain *ipd, { unsigned long flags; - ipipe_root_only(); - spin_lock_irqsave(&__ipipe_lock, flags); if (ipd->irqs[irq].handler == NULL) -- 1.7.3.4 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 10:37 [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" Jan Kiszka @ 2012-09-20 10:49 ` Philippe Gerum 2012-09-20 10:56 ` Jan Kiszka 0 siblings, 1 reply; 18+ messages in thread From: Philippe Gerum @ 2012-09-20 10:49 UTC (permalink / raw) To: Jan Kiszka; +Cc: Wolfgang Mauerer, Xenomai On 09/20/2012 12:37 PM, Jan Kiszka wrote: > This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. > > Both service are NOT just root-only services. E.g., rtdm_irq_request > requires by specification support also over non-Linux contexts. Nack. We can't run the enable code for MSIs over non-root, and that code typically follows the irq request. Besides, we want to mask the source upon irq free to handle the SMP case properly, which we could not do from non-root with MSIs. So either we have both request+enable and free usable over non-root, or there is no point. > Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> > --- > > We run all critical code under an I-pipe spin lock, so this restriction > seems artificial. Nevertheless, I briefly checked if some arch has > specific needs but didn't find any trace in core-3.5. > > Wolfgang, this already resolves the issue I mentioned regarding startup > of RTnet NICs. > > kernel/ipipe/core.c | 4 ---- > 1 files changed, 0 insertions(+), 4 deletions(-) > > diff --git a/kernel/ipipe/core.c b/kernel/ipipe/core.c > index 9898b92..23a2146 100644 > --- a/kernel/ipipe/core.c > +++ b/kernel/ipipe/core.c > @@ -846,8 +846,6 @@ int ipipe_request_irq(struct ipipe_domain *ipd, > unsigned long flags; > int ret = 0; > > - ipipe_root_only(); > - > if (handler == NULL || > (irq >= IPIPE_NR_XIRQS && !ipipe_virtual_irq_p(irq))) > return -EINVAL; > @@ -881,8 +879,6 @@ void ipipe_free_irq(struct ipipe_domain *ipd, > { > unsigned long flags; > > - ipipe_root_only(); > - > spin_lock_irqsave(&__ipipe_lock, flags); > > if (ipd->irqs[irq].handler == NULL) > -- Philippe. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 10:49 ` Philippe Gerum @ 2012-09-20 10:56 ` Jan Kiszka 2012-09-20 10:57 ` Jan Kiszka 0 siblings, 1 reply; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 10:56 UTC (permalink / raw) To: Philippe Gerum; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 12:49, Philippe Gerum wrote: > On 09/20/2012 12:37 PM, Jan Kiszka wrote: >> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >> >> Both service are NOT just root-only services. E.g., rtdm_irq_request >> requires by specification support also over non-Linux contexts. > > Nack. We can't run the enable code for MSIs over non-root, and > that code typically follows the irq request. Besides, we want to mask > the source upon irq free to handle the SMP case properly, which we could > not do from non-root with MSIs. > > So either we have both request+enable and free usable over non-root, or > there is no point. OK, I get the point with legacy MSI. Then we have two other bugs to solve: - in I-pipe as it holds a hardened spin lock across enable/disable (of MSIs) - in Xenomai 2.6 (at least, didn't check forge) as it calls with a hardened spin lock held into ipipe_virtualize_irq Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 10:56 ` Jan Kiszka @ 2012-09-20 10:57 ` Jan Kiszka 2012-09-20 11:06 ` Philippe Gerum 2012-09-20 11:15 ` Jan Kiszka 0 siblings, 2 replies; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 10:57 UTC (permalink / raw) To: Philippe Gerum; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 12:56, Jan Kiszka wrote: > On 2012-09-20 12:49, Philippe Gerum wrote: >> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>> >>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>> requires by specification support also over non-Linux contexts. >> >> Nack. We can't run the enable code for MSIs over non-root, and >> that code typically follows the irq request. Besides, we want to mask >> the source upon irq free to handle the SMP case properly, which we could >> not do from non-root with MSIs. >> >> So either we have both request+enable and free usable over non-root, or >> there is no point. > > OK, I get the point with legacy MSI. Then we have two other bugs to solve: > - in I-pipe as it holds a hardened spin lock across enable/disable (of > MSIs) > - in Xenomai 2.6 (at least, didn't check forge) as it calls with a > hardened spin lock held into ipipe_virtualize_irq Sorry, three bugs: - in the RTDM spec as it always allowed rtdm_irq_request over RT task contexts Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 10:57 ` Jan Kiszka @ 2012-09-20 11:06 ` Philippe Gerum 2012-09-20 11:15 ` Jan Kiszka 1 sibling, 0 replies; 18+ messages in thread From: Philippe Gerum @ 2012-09-20 11:06 UTC (permalink / raw) To: Jan Kiszka; +Cc: Mauerer, Wolfgang, Xenomai On 09/20/2012 12:57 PM, Jan Kiszka wrote: > On 2012-09-20 12:56, Jan Kiszka wrote: >> On 2012-09-20 12:49, Philippe Gerum wrote: >>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>> >>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>> requires by specification support also over non-Linux contexts. >>> >>> Nack. We can't run the enable code for MSIs over non-root, and >>> that code typically follows the irq request. Besides, we want to mask >>> the source upon irq free to handle the SMP case properly, which we could >>> not do from non-root with MSIs. >>> >>> So either we have both request+enable and free usable over non-root, or >>> there is no point. >> >> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >> - in I-pipe as it holds a hardened spin lock across enable/disable (of >> MSIs) Missed it. Where? >> - in Xenomai 2.6 (at least, didn't check forge) as it calls with a >> hardened spin lock held into ipipe_virtualize_irq > > Sorry, three bugs: > - in the RTDM spec as it always allowed rtdm_irq_request over RT task > contexts > Yes. This was likely designed when ->enable/disable() handlers would not traverse a whole pile of hairy code highly dependent on the PCI infrastructure like today. And on the ppc side, this is not getting any prettier. > Jan > -- Philippe. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 10:57 ` Jan Kiszka 2012-09-20 11:06 ` Philippe Gerum @ 2012-09-20 11:15 ` Jan Kiszka 2012-09-20 11:27 ` Jan Kiszka ` (2 more replies) 1 sibling, 3 replies; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 11:15 UTC (permalink / raw) To: Philippe Gerum; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 12:57, Jan Kiszka wrote: > On 2012-09-20 12:56, Jan Kiszka wrote: >> On 2012-09-20 12:49, Philippe Gerum wrote: >>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>> >>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>> requires by specification support also over non-Linux contexts. >>> >>> Nack. We can't run the enable code for MSIs over non-root, and >>> that code typically follows the irq request. Besides, we want to mask >>> the source upon irq free to handle the SMP case properly, which we could >>> not do from non-root with MSIs. >>> >>> So either we have both request+enable and free usable over non-root, or >>> there is no point. >> >> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >> - in I-pipe as it holds a hardened spin lock across enable/disable (of >> MSIs) I think this bug may only manifest over ARM as that arch does enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless something prevents that enabling will ever happen for interrupts that need Linux locks to work. Is that assured? >> - in Xenomai 2.6 (at least, didn't check forge) as it calls with a >> hardened spin lock held into ipipe_virtualize_irq This problem is something I vaguely recall we discussed before already in the past. I think there was no good solution for the Xenomai 2 architecture. In this light, let's remove those checks nevertheless. Enabling/disabling the IRQ are separate calls, and those should be instrumented as those cause the restriction. > > Sorry, three bugs: > - in the RTDM spec as it always allowed rtdm_irq_request over RT task > contexts Changing the spec and adding a runtime check will likely be no issue. No sane driver should have made use of that option. I will file a patch. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 11:15 ` Jan Kiszka @ 2012-09-20 11:27 ` Jan Kiszka 2012-09-20 13:01 ` Gilles Chanteperdrix 2012-09-20 13:10 ` Philippe Gerum 2 siblings, 0 replies; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 11:27 UTC (permalink / raw) To: Philippe Gerum; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 13:15, Jan Kiszka wrote: >> Sorry, three bugs: >> - in the RTDM spec as it always allowed rtdm_irq_request over RT task >> contexts > > Changing the spec and adding a runtime check will likely be no issue. No > sane driver should have made use of that option. I will file a patch. Naturally, this also affects rtdm_irq_enable/disable, and some use cases I'm aware off will bail out when we add assertions for non-RT context. I would prefer to address it more selectively, allowing all those services for IRQ types that support it (i.e. that can be atomically enabled/disabled), keeping the freedom to reject or otherwise fail for types that are in conflict. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 11:15 ` Jan Kiszka 2012-09-20 11:27 ` Jan Kiszka @ 2012-09-20 13:01 ` Gilles Chanteperdrix 2012-09-20 13:15 ` Jan Kiszka 2012-09-20 13:10 ` Philippe Gerum 2 siblings, 1 reply; 18+ messages in thread From: Gilles Chanteperdrix @ 2012-09-20 13:01 UTC (permalink / raw) To: Jan Kiszka; +Cc: Mauerer, Wolfgang, Xenomai On 09/20/2012 01:15 PM, Jan Kiszka wrote: > On 2012-09-20 12:57, Jan Kiszka wrote: >> On 2012-09-20 12:56, Jan Kiszka wrote: >>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>> >>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>> requires by specification support also over non-Linux contexts. >>>> >>>> Nack. We can't run the enable code for MSIs over non-root, and >>>> that code typically follows the irq request. Besides, we want to mask >>>> the source upon irq free to handle the SMP case properly, which we could >>>> not do from non-root with MSIs. >>>> >>>> So either we have both request+enable and free usable over non-root, or >>>> there is no point. >>> >>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>> MSIs) > > I think this bug may only manifest over ARM as that arch does > enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless > something prevents that enabling will ever happen for interrupts that > need Linux locks to work. Is that assured? ARM does that because otherwise interrupts are not enabled after a call to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work, calling request_irq is required to get them working. -- Gilles. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 13:01 ` Gilles Chanteperdrix @ 2012-09-20 13:15 ` Jan Kiszka 2012-09-20 14:12 ` Gilles Chanteperdrix 0 siblings, 1 reply; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 13:15 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 15:01, Gilles Chanteperdrix wrote: > On 09/20/2012 01:15 PM, Jan Kiszka wrote: > >> On 2012-09-20 12:57, Jan Kiszka wrote: >>> On 2012-09-20 12:56, Jan Kiszka wrote: >>>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>>> >>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>>> requires by specification support also over non-Linux contexts. >>>>> >>>>> Nack. We can't run the enable code for MSIs over non-root, and >>>>> that code typically follows the irq request. Besides, we want to mask >>>>> the source upon irq free to handle the SMP case properly, which we could >>>>> not do from non-root with MSIs. >>>>> >>>>> So either we have both request+enable and free usable over non-root, or >>>>> there is no point. >>>> >>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>>> MSIs) >> >> I think this bug may only manifest over ARM as that arch does >> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless >> something prevents that enabling will ever happen for interrupts that >> need Linux locks to work. Is that assured? > > > ARM does that because otherwise interrupts are not enabled after a call > to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work, > calling request_irq is required to get them working. The generic pattern Xenomai currently uses is xnintr_attach + xnintr_enable. The latter happens to be rtdm_irq_enable as well. So, doesn't xnintr_enable work at all? Or is it only not suited as interrupt startup? Then, is that a new issue or something core-3.x specific? Finally, can't we address it in xnarch_enable_irq? Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 13:15 ` Jan Kiszka @ 2012-09-20 14:12 ` Gilles Chanteperdrix 2012-09-20 15:00 ` Jan Kiszka 0 siblings, 1 reply; 18+ messages in thread From: Gilles Chanteperdrix @ 2012-09-20 14:12 UTC (permalink / raw) To: Jan Kiszka; +Cc: Mauerer, Wolfgang, Xenomai On 09/20/2012 03:15 PM, Jan Kiszka wrote: > On 2012-09-20 15:01, Gilles Chanteperdrix wrote: >> On 09/20/2012 01:15 PM, Jan Kiszka wrote: >> >>> On 2012-09-20 12:57, Jan Kiszka wrote: >>>> On 2012-09-20 12:56, Jan Kiszka wrote: >>>>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>>>> >>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>>>> requires by specification support also over non-Linux contexts. >>>>>> >>>>>> Nack. We can't run the enable code for MSIs over non-root, and >>>>>> that code typically follows the irq request. Besides, we want to mask >>>>>> the source upon irq free to handle the SMP case properly, which we could >>>>>> not do from non-root with MSIs. >>>>>> >>>>>> So either we have both request+enable and free usable over non-root, or >>>>>> there is no point. >>>>> >>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>>>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>>>> MSIs) >>> >>> I think this bug may only manifest over ARM as that arch does >>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless >>> something prevents that enabling will ever happen for interrupts that >>> need Linux locks to work. Is that assured? >> >> >> ARM does that because otherwise interrupts are not enabled after a call >> to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work, >> calling request_irq is required to get them working. > > The generic pattern Xenomai currently uses is xnintr_attach + > xnintr_enable. The latter happens to be rtdm_irq_enable as well. So, > doesn't xnintr_enable work at all? No. It has in fact, never worked before the I-pipe core. People had to call request_irq before xnintr_attach. > Or is it only not suited as interrupt > startup? Then, is that a new issue or something core-3.x specific? > Finally, can't we address it in xnarch_enable_irq? This is a very old issue, that I had forgotten, and that bit me when I tried to use a separate timer for Xenomai and Linux on omap3: because the timer interrupt of a different timer has no corresponding linux interrupt. The problem is that irqs which are also used by Linux have to be treated differently from irqs which are registered only for the linux domain. -- Gilles. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 14:12 ` Gilles Chanteperdrix @ 2012-09-20 15:00 ` Jan Kiszka 2012-09-20 15:13 ` Gilles Chanteperdrix 0 siblings, 1 reply; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 15:00 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 16:12, Gilles Chanteperdrix wrote: > On 09/20/2012 03:15 PM, Jan Kiszka wrote: >> On 2012-09-20 15:01, Gilles Chanteperdrix wrote: >>> On 09/20/2012 01:15 PM, Jan Kiszka wrote: >>> >>>> On 2012-09-20 12:57, Jan Kiszka wrote: >>>>> On 2012-09-20 12:56, Jan Kiszka wrote: >>>>>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>>>>> >>>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>>>>> requires by specification support also over non-Linux contexts. >>>>>>> >>>>>>> Nack. We can't run the enable code for MSIs over non-root, and >>>>>>> that code typically follows the irq request. Besides, we want to mask >>>>>>> the source upon irq free to handle the SMP case properly, which we could >>>>>>> not do from non-root with MSIs. >>>>>>> >>>>>>> So either we have both request+enable and free usable over non-root, or >>>>>>> there is no point. >>>>>> >>>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>>>>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>>>>> MSIs) >>>> >>>> I think this bug may only manifest over ARM as that arch does >>>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless >>>> something prevents that enabling will ever happen for interrupts that >>>> need Linux locks to work. Is that assured? >>> >>> >>> ARM does that because otherwise interrupts are not enabled after a call >>> to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work, >>> calling request_irq is required to get them working. >> >> The generic pattern Xenomai currently uses is xnintr_attach + >> xnintr_enable. The latter happens to be rtdm_irq_enable as well. So, >> doesn't xnintr_enable work at all? > > No. It has in fact, never worked before the I-pipe core. People had to > call request_irq before xnintr_attach. > >> Or is it only not suited as interrupt >> startup? Then, is that a new issue or something core-3.x specific? >> Finally, can't we address it in xnarch_enable_irq? > > This is a very old issue, that I had forgotten, and that bit me when I > tried to use a separate timer for Xenomai and Linux on omap3: because > the timer interrupt of a different timer has no corresponding linux > interrupt. > > The problem is that irqs which are also used by Linux have to be treated > differently from irqs which are registered only for the linux domain. OK. Then we can safely assume that this initial enabling/disabling will not cause excessive latencies or even acquire Linux locks on ARM, right? Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 15:00 ` Jan Kiszka @ 2012-09-20 15:13 ` Gilles Chanteperdrix 0 siblings, 0 replies; 18+ messages in thread From: Gilles Chanteperdrix @ 2012-09-20 15:13 UTC (permalink / raw) To: Jan Kiszka; +Cc: Mauerer, Wolfgang, Xenomai On 09/20/2012 05:00 PM, Jan Kiszka wrote: > On 2012-09-20 16:12, Gilles Chanteperdrix wrote: >> On 09/20/2012 03:15 PM, Jan Kiszka wrote: >>> On 2012-09-20 15:01, Gilles Chanteperdrix wrote: >>>> On 09/20/2012 01:15 PM, Jan Kiszka wrote: >>>> >>>>> On 2012-09-20 12:57, Jan Kiszka wrote: >>>>>> On 2012-09-20 12:56, Jan Kiszka wrote: >>>>>>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>>>>>> >>>>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>>>>>> requires by specification support also over non-Linux contexts. >>>>>>>> >>>>>>>> Nack. We can't run the enable code for MSIs over non-root, and >>>>>>>> that code typically follows the irq request. Besides, we want to mask >>>>>>>> the source upon irq free to handle the SMP case properly, which we could >>>>>>>> not do from non-root with MSIs. >>>>>>>> >>>>>>>> So either we have both request+enable and free usable over non-root, or >>>>>>>> there is no point. >>>>>>> >>>>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>>>>>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>>>>>> MSIs) >>>>> >>>>> I think this bug may only manifest over ARM as that arch does >>>>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless >>>>> something prevents that enabling will ever happen for interrupts that >>>>> need Linux locks to work. Is that assured? >>>> >>>> >>>> ARM does that because otherwise interrupts are not enabled after a call >>>> to eg. rtdm_irq_request, and calling rtdm_irq_enable does not work, >>>> calling request_irq is required to get them working. >>> >>> The generic pattern Xenomai currently uses is xnintr_attach + >>> xnintr_enable. The latter happens to be rtdm_irq_enable as well. So, >>> doesn't xnintr_enable work at all? >> >> No. It has in fact, never worked before the I-pipe core. People had to >> call request_irq before xnintr_attach. >> >>> Or is it only not suited as interrupt >>> startup? Then, is that a new issue or something core-3.x specific? >>> Finally, can't we address it in xnarch_enable_irq? >> >> This is a very old issue, that I had forgotten, and that bit me when I >> tried to use a separate timer for Xenomai and Linux on omap3: because >> the timer interrupt of a different timer has no corresponding linux >> interrupt. >> >> The problem is that irqs which are also used by Linux have to be treated >> differently from irqs which are registered only for the linux domain. > > OK. Then we can safely assume that this initial enabling/disabling will > not cause excessive latencies or even acquire Linux locks on ARM, right? mmmm. enable_irq may take the descriptor spinlock, I have to check that. -- Gilles. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 11:15 ` Jan Kiszka 2012-09-20 11:27 ` Jan Kiszka 2012-09-20 13:01 ` Gilles Chanteperdrix @ 2012-09-20 13:10 ` Philippe Gerum 2012-09-20 13:54 ` Jan Kiszka 2 siblings, 1 reply; 18+ messages in thread From: Philippe Gerum @ 2012-09-20 13:10 UTC (permalink / raw) To: Jan Kiszka; +Cc: Mauerer, Wolfgang, Xenomai On 09/20/2012 01:15 PM, Jan Kiszka wrote: > On 2012-09-20 12:57, Jan Kiszka wrote: >> On 2012-09-20 12:56, Jan Kiszka wrote: >>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>> >>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>> requires by specification support also over non-Linux contexts. >>>> >>>> Nack. We can't run the enable code for MSIs over non-root, and >>>> that code typically follows the irq request. Besides, we want to mask >>>> the source upon irq free to handle the SMP case properly, which we could >>>> not do from non-root with MSIs. >>>> >>>> So either we have both request+enable and free usable over non-root, or >>>> there is no point. >>> >>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>> MSIs) > > I think this bug may only manifest over ARM as that arch does > enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless > something prevents that enabling will ever happen for interrupts that > need Linux locks to work. Is that assured? I'm not referring to enable_irqdesc, but to the common programming pattern of calling ipipe_request_irq from the same context than ipipe_enable_irq (or directly the underlying irqchip handler for unmasking). > >>> - in Xenomai 2.6 (at least, didn't check forge) as it calls with a >>> hardened spin lock held into ipipe_virtualize_irq > > This problem is something I vaguely recall we discussed before already > in the past. I think there was no good solution for the Xenomai 2 > architecture. > The main issue was about de-registering a handler, passing NULL. To solve the SMP-specific issue of interrupt synchronization on all CPUs, we would have to be able to disable the source, which may entail running regular code, therefore restricting the valid calling context to root. > In this light, let's remove those checks nevertheless. > Enabling/disabling the IRQ are separate calls, and those should be > instrumented as those cause the restriction. > I don't see it this way, because we can't predict what will be the constraints we might have for hooking irqs on all archs we will support. Maybe we will have to run more mainline code in some cases. In any case, we have to fold masking into the de-registration code for proper SMP support - this fix was never finalized precisely because we could not guarantee a root calling context in that case. These checks are there to express the fact that calling from non-root is inherently unsafe. We might find a (ugly) way to tag irq descriptors, for knowing whether this is safe to call from non-root and test this conditionally. But at the end of the day, we would still end up with checking for arch-specific constraints in a generic API, which would be wrong by design. I put these checks when refactoring the pipeline API for the very same reason than you agree to update the RTDM spec regarding rtdm_request_irq: no sane code should have called ipipe_virtualize_irq() from a non-root context. This is just about formalizing this fact. >> >> Sorry, three bugs: >> - in the RTDM spec as it always allowed rtdm_irq_request over RT task >> contexts > > Changing the spec and adding a runtime check will likely be no issue. No > sane driver should have made use of that option. I will file a patch. > Ack. > Jan > -- Philippe. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 13:10 ` Philippe Gerum @ 2012-09-20 13:54 ` Jan Kiszka 2012-09-20 14:05 ` Philippe Gerum 0 siblings, 1 reply; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 13:54 UTC (permalink / raw) To: Philippe Gerum; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 15:10, Philippe Gerum wrote: > On 09/20/2012 01:15 PM, Jan Kiszka wrote: >> On 2012-09-20 12:57, Jan Kiszka wrote: >>> On 2012-09-20 12:56, Jan Kiszka wrote: >>>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>>> >>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>>> requires by specification support also over non-Linux contexts. >>>>> >>>>> Nack. We can't run the enable code for MSIs over non-root, and >>>>> that code typically follows the irq request. Besides, we want to mask >>>>> the source upon irq free to handle the SMP case properly, which we could >>>>> not do from non-root with MSIs. >>>>> >>>>> So either we have both request+enable and free usable over non-root, or >>>>> there is no point. >>>> >>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>>> MSIs) >> >> I think this bug may only manifest over ARM as that arch does >> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless >> something prevents that enabling will ever happen for interrupts that >> need Linux locks to work. Is that assured? > > I'm not referring to enable_irqdesc, but to the common programming > pattern of calling ipipe_request_irq from the same context than > ipipe_enable_irq (or directly the underlying irqchip handler for unmasking). Patterns are better enforced at higher layers, not here. > >> >>>> - in Xenomai 2.6 (at least, didn't check forge) as it calls with a >>>> hardened spin lock held into ipipe_virtualize_irq >> >> This problem is something I vaguely recall we discussed before already >> in the past. I think there was no good solution for the Xenomai 2 >> architecture. >> > > The main issue was about de-registering a handler, passing NULL. To > solve the SMP-specific issue of interrupt synchronization on all CPUs, > we would have to be able to disable the source, which may entail running > regular code, therefore restricting the valid calling context to root. I'm not sure what the conclusion regarding this tricky topic was. But I don't see how this can be addressed only at I-pipe level anyway. > >> In this light, let's remove those checks nevertheless. >> Enabling/disabling the IRQ are separate calls, and those should be >> instrumented as those cause the restriction. >> > > I don't see it this way, because we can't predict what will be the > constraints we might have for hooking irqs on all archs we will support. > Maybe we will have to run more mainline code in some cases. In any case, > we have to fold masking into the de-registration code for proper SMP > support - this fix was never finalized precisely because we could not > guarantee a root calling context in that case. > > These checks are there to express the fact that calling from non-root is > inherently unsafe. We might find a (ugly) way to tag irq descriptors, > for knowing whether this is safe to call from non-root and test this > conditionally. But at the end of the day, we would still end up with > checking for arch-specific constraints in a generic API, which would be > wrong by design. > > I put these checks when refactoring the pipeline API for the very same > reason than you agree to update the RTDM spec regarding > rtdm_request_irq: no sane code should have called ipipe_virtualize_irq() > from a non-root context. This is just about formalizing this fact. ...at the wrong point. Plus it is breaking our instrumentation. Again: this belongs where we can asses the problem better, i.e. in the higher layer and in those functions that do break when called from RT context. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 13:54 ` Jan Kiszka @ 2012-09-20 14:05 ` Philippe Gerum 2012-09-20 15:07 ` Jan Kiszka 0 siblings, 1 reply; 18+ messages in thread From: Philippe Gerum @ 2012-09-20 14:05 UTC (permalink / raw) To: Jan Kiszka; +Cc: Mauerer, Wolfgang, Xenomai On 09/20/2012 03:54 PM, Jan Kiszka wrote: > On 2012-09-20 15:10, Philippe Gerum wrote: >> On 09/20/2012 01:15 PM, Jan Kiszka wrote: >>> On 2012-09-20 12:57, Jan Kiszka wrote: >>>> On 2012-09-20 12:56, Jan Kiszka wrote: >>>>> On 2012-09-20 12:49, Philippe Gerum wrote: >>>>>> On 09/20/2012 12:37 PM, Jan Kiszka wrote: >>>>>>> This reverts commit 073ff1e8045d0311b8cf390687c0ba3619681672. >>>>>>> >>>>>>> Both service are NOT just root-only services. E.g., rtdm_irq_request >>>>>>> requires by specification support also over non-Linux contexts. >>>>>> >>>>>> Nack. We can't run the enable code for MSIs over non-root, and >>>>>> that code typically follows the irq request. Besides, we want to mask >>>>>> the source upon irq free to handle the SMP case properly, which we could >>>>>> not do from non-root with MSIs. >>>>>> >>>>>> So either we have both request+enable and free usable over non-root, or >>>>>> there is no point. >>>>> >>>>> OK, I get the point with legacy MSI. Then we have two other bugs to solve: >>>>> - in I-pipe as it holds a hardened spin lock across enable/disable (of >>>>> MSIs) >>> >>> I think this bug may only manifest over ARM as that arch does >>> enable/disable_irq() inside __ipipe_enable/disable_irqdesc - unless >>> something prevents that enabling will ever happen for interrupts that >>> need Linux locks to work. Is that assured? >> >> I'm not referring to enable_irqdesc, but to the common programming >> pattern of calling ipipe_request_irq from the same context than >> ipipe_enable_irq (or directly the underlying irqchip handler for unmasking). > > Patterns are better enforced at higher layers, not here. > The -core series is really about designing a Xenomai-specific real-time enabler. >> >>> >>>>> - in Xenomai 2.6 (at least, didn't check forge) as it calls with a >>>>> hardened spin lock held into ipipe_virtualize_irq >>> >>> This problem is something I vaguely recall we discussed before already >>> in the past. I think there was no good solution for the Xenomai 2 >>> architecture. >>> >> >> The main issue was about de-registering a handler, passing NULL. To >> solve the SMP-specific issue of interrupt synchronization on all CPUs, >> we would have to be able to disable the source, which may entail running >> regular code, therefore restricting the valid calling context to root. > > I'm not sure what the conclusion regarding this tricky topic was. But I > don't see how this can be addressed only at I-pipe level anyway. The I-pipe knows about IRQs, and does very much know what it entails do deal with the kernel infrastructure regarding this. It does abstract this from the client code, enforcing rules to make sure the interface remains applicable to all archs. These rules have an impact on client code, we have to live with it. > >> >>> In this light, let's remove those checks nevertheless. >>> Enabling/disabling the IRQ are separate calls, and those should be >>> instrumented as those cause the restriction. >>> >> >> I don't see it this way, because we can't predict what will be the >> constraints we might have for hooking irqs on all archs we will support. >> Maybe we will have to run more mainline code in some cases. In any case, >> we have to fold masking into the de-registration code for proper SMP >> support - this fix was never finalized precisely because we could not >> guarantee a root calling context in that case. >> >> These checks are there to express the fact that calling from non-root is >> inherently unsafe. We might find a (ugly) way to tag irq descriptors, >> for knowing whether this is safe to call from non-root and test this >> conditionally. But at the end of the day, we would still end up with >> checking for arch-specific constraints in a generic API, which would be >> wrong by design. >> >> I put these checks when refactoring the pipeline API for the very same >> reason than you agree to update the RTDM spec regarding >> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq() >> from a non-root context. This is just about formalizing this fact. > > ...at the wrong point. Plus it is breaking our instrumentation. Again: > this belongs where we can asses the problem better, i.e. in the higher > layer and in those functions that do break when called from RT context. I don't think so. > > Jan > -- Philippe. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 14:05 ` Philippe Gerum @ 2012-09-20 15:07 ` Jan Kiszka 2012-09-20 15:16 ` Philippe Gerum 0 siblings, 1 reply; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 15:07 UTC (permalink / raw) To: Philippe Gerum; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 16:05, Philippe Gerum wrote: >>>> In this light, let's remove those checks nevertheless. >>>> Enabling/disabling the IRQ are separate calls, and those should be >>>> instrumented as those cause the restriction. >>>> >>> >>> I don't see it this way, because we can't predict what will be the >>> constraints we might have for hooking irqs on all archs we will support. >>> Maybe we will have to run more mainline code in some cases. In any case, >>> we have to fold masking into the de-registration code for proper SMP >>> support - this fix was never finalized precisely because we could not >>> guarantee a root calling context in that case. >>> >>> These checks are there to express the fact that calling from non-root is >>> inherently unsafe. We might find a (ugly) way to tag irq descriptors, >>> for knowing whether this is safe to call from non-root and test this >>> conditionally. But at the end of the day, we would still end up with >>> checking for arch-specific constraints in a generic API, which would be >>> wrong by design. >>> >>> I put these checks when refactoring the pipeline API for the very same >>> reason than you agree to update the RTDM spec regarding >>> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq() >>> from a non-root context. This is just about formalizing this fact. >> >> ...at the wrong point. Plus it is breaking our instrumentation. Again: >> this belongs where we can asses the problem better, i.e. in the higher >> layer and in those functions that do break when called from RT context. > > I don't think so. I do agree that the caller of ipipe_request_irq should not be called over RT context. However, ipipe_request_irq itself _is_ called under an RT spin lock with hardirqs disabled, both under Xenomai 2.6 and upcoming 3. And this triggers CONFIG_IPIPE_DEBUG_CONTEXT, clearly showing that things are still broken here. What do you suggest to fix all this? Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 15:07 ` Jan Kiszka @ 2012-09-20 15:16 ` Philippe Gerum 2012-09-20 15:45 ` Jan Kiszka 0 siblings, 1 reply; 18+ messages in thread From: Philippe Gerum @ 2012-09-20 15:16 UTC (permalink / raw) To: Jan Kiszka; +Cc: Mauerer, Wolfgang, Xenomai On 09/20/2012 05:07 PM, Jan Kiszka wrote: > On 2012-09-20 16:05, Philippe Gerum wrote: >>>>> In this light, let's remove those checks nevertheless. >>>>> Enabling/disabling the IRQ are separate calls, and those should be >>>>> instrumented as those cause the restriction. >>>>> >>>> >>>> I don't see it this way, because we can't predict what will be the >>>> constraints we might have for hooking irqs on all archs we will support. >>>> Maybe we will have to run more mainline code in some cases. In any case, >>>> we have to fold masking into the de-registration code for proper SMP >>>> support - this fix was never finalized precisely because we could not >>>> guarantee a root calling context in that case. >>>> >>>> These checks are there to express the fact that calling from non-root is >>>> inherently unsafe. We might find a (ugly) way to tag irq descriptors, >>>> for knowing whether this is safe to call from non-root and test this >>>> conditionally. But at the end of the day, we would still end up with >>>> checking for arch-specific constraints in a generic API, which would be >>>> wrong by design. >>>> >>>> I put these checks when refactoring the pipeline API for the very same >>>> reason than you agree to update the RTDM spec regarding >>>> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq() >>>> from a non-root context. This is just about formalizing this fact. >>> >>> ...at the wrong point. Plus it is breaking our instrumentation. Again: >>> this belongs where we can asses the problem better, i.e. in the higher >>> layer and in those functions that do break when called from RT context. >> >> I don't think so. > > I do agree that the caller of ipipe_request_irq should not be called > over RT context. However, ipipe_request_irq itself _is_ called under an > RT spin lock with hardirqs disabled, both under Xenomai 2.6 and upcoming > 3. And this triggers CONFIG_IPIPE_DEBUG_CONTEXT, clearly showing that > things are still broken here. What do you suggest to fix all this? > Fix the callers in the upcoming Xenomai releases, and provide an open coded implementation of ipipe_request_irq in ipipe_virtualize_irq when CONFIG_IPIPE_LEGACY is enabled, which will take care of the broken legacy. -- Philippe. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" 2012-09-20 15:16 ` Philippe Gerum @ 2012-09-20 15:45 ` Jan Kiszka 0 siblings, 0 replies; 18+ messages in thread From: Jan Kiszka @ 2012-09-20 15:45 UTC (permalink / raw) To: Philippe Gerum; +Cc: Mauerer, Wolfgang, Xenomai On 2012-09-20 17:16, Philippe Gerum wrote: > On 09/20/2012 05:07 PM, Jan Kiszka wrote: >> On 2012-09-20 16:05, Philippe Gerum wrote: >>>>>> In this light, let's remove those checks nevertheless. >>>>>> Enabling/disabling the IRQ are separate calls, and those should be >>>>>> instrumented as those cause the restriction. >>>>>> >>>>> >>>>> I don't see it this way, because we can't predict what will be the >>>>> constraints we might have for hooking irqs on all archs we will support. >>>>> Maybe we will have to run more mainline code in some cases. In any case, >>>>> we have to fold masking into the de-registration code for proper SMP >>>>> support - this fix was never finalized precisely because we could not >>>>> guarantee a root calling context in that case. >>>>> >>>>> These checks are there to express the fact that calling from non-root is >>>>> inherently unsafe. We might find a (ugly) way to tag irq descriptors, >>>>> for knowing whether this is safe to call from non-root and test this >>>>> conditionally. But at the end of the day, we would still end up with >>>>> checking for arch-specific constraints in a generic API, which would be >>>>> wrong by design. >>>>> >>>>> I put these checks when refactoring the pipeline API for the very same >>>>> reason than you agree to update the RTDM spec regarding >>>>> rtdm_request_irq: no sane code should have called ipipe_virtualize_irq() >>>>> from a non-root context. This is just about formalizing this fact. >>>> >>>> ...at the wrong point. Plus it is breaking our instrumentation. Again: >>>> this belongs where we can asses the problem better, i.e. in the higher >>>> layer and in those functions that do break when called from RT context. >>> >>> I don't think so. >> >> I do agree that the caller of ipipe_request_irq should not be called >> over RT context. However, ipipe_request_irq itself _is_ called under an >> RT spin lock with hardirqs disabled, both under Xenomai 2.6 and upcoming >> 3. And this triggers CONFIG_IPIPE_DEBUG_CONTEXT, clearly showing that >> things are still broken here. What do you suggest to fix all this? >> > > Fix the callers in the upcoming Xenomai releases, and provide an open > coded implementation of ipipe_request_irq in ipipe_virtualize_irq when > CONFIG_IPIPE_LEGACY is enabled, which will take care of the broken legacy. Both callers have the same structure, so we can either fix both or none. Fortunately, we appear to be fine by simply converting intrlock to a plain Linux mutex. Will check twice later. Jan -- Siemens AG, Corporate Technology, CT RTC ITP SDP-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2012-09-20 15:45 UTC | newest] Thread overview: 18+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-09-20 10:37 [Xenomai] [PATCH] Revert "ipipe: ipipe_request_irq(), ipipe_free_irq() are root-only services" Jan Kiszka 2012-09-20 10:49 ` Philippe Gerum 2012-09-20 10:56 ` Jan Kiszka 2012-09-20 10:57 ` Jan Kiszka 2012-09-20 11:06 ` Philippe Gerum 2012-09-20 11:15 ` Jan Kiszka 2012-09-20 11:27 ` Jan Kiszka 2012-09-20 13:01 ` Gilles Chanteperdrix 2012-09-20 13:15 ` Jan Kiszka 2012-09-20 14:12 ` Gilles Chanteperdrix 2012-09-20 15:00 ` Jan Kiszka 2012-09-20 15:13 ` Gilles Chanteperdrix 2012-09-20 13:10 ` Philippe Gerum 2012-09-20 13:54 ` Jan Kiszka 2012-09-20 14:05 ` Philippe Gerum 2012-09-20 15:07 ` Jan Kiszka 2012-09-20 15:16 ` Philippe Gerum 2012-09-20 15:45 ` Jan Kiszka
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.