From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ecfrec.frec.bull.fr (ecfrec.frec.bull.fr [129.183.4.8]) by ozlabs.org (Postfix) with ESMTP id 7E104DE2D1 for ; Fri, 1 Aug 2008 00:14:06 +1000 (EST) Date: Thu, 31 Jul 2008 16:14:30 +0200 From: Sebastien Dugue To: michael@ellerman.id.au Subject: Re: [PATCH] powerpc - Initialize the irq radix tree earlier Message-ID: <20080731161430.5de73ef8@bull.net> In-Reply-To: <1217511566.19050.26.camel@localhost> References: <1217497241-10685-1-git-send-email-sebastien.dugue@bull.net> <1217497241-10685-2-git-send-email-sebastien.dugue@bull.net> <1217504456.9817.22.camel@localhost> <20080731140002.31bbe4a0@bull.net> <1217509104.19050.11.camel@localhost> <1217509299.19050.15.camel@localhost> <20080731152606.5ae7c379@bull.net> <1217511566.19050.26.camel@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Cc: tinytim@us.ibm.com, linux-rt-users@vger.kernel.org, jean-pierre.dion@bull.net, rostedt@goodmis.org, linux-kernel@vger.kernel.org, linuxppc-dev@ozlabs.org, paulus@samba.org, gilles.carry@ext.bull.net, tglx@linutronix.de List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Thu, 31 Jul 2008 23:39:26 +1000 Michael Ellerman wrote: > On Thu, 2008-07-31 at 15:26 +0200, Sebastien Dugue wrote: > > On Thu, 31 Jul 2008 23:01:39 +1000 Michael Ellerman wrote: > >=20 > > > On Thu, 2008-07-31 at 22:58 +1000, Michael Ellerman wrote: > > > > On Thu, 2008-07-31 at 14:00 +0200, Sebastien Dugue wrote: > > > > > On Thu, 31 Jul 2008 21:40:56 +1000 Michael Ellerman wrote: > > > > > >=20 > > > > > > This boot ordering stuff is pretty hairy, so I might have missed > > > > > > something, but this is how the code is ordered AFAICT: > > > > > > =EF=BB=BF > > > > > > start_kernel() > > > > > > init_IRQ() > > > > > > ... > > > > > > local_irq_enable() > > > > > > ... > > > > > > rest_init() > > > > > > kernel_thread() > > > > > > kernel_init() > > > > > > smp_prepare_cpus() > > > > > > smp_xics_probe() (via smp_ops->probe()) > > > > > >=20 > > > > > >=20 > > > > > > What's stopping us from taking an irq between local_irq_enable(= ) and > > > > > > smp_xics_probe() ? Is it just that no one's request_irq()'ed t= hem yet? > > > > >=20 > > > > > It's hairy, I agree, but as you've mentioned no one has done a = request_irq() > > > > > at that point. The first one to do it is smp_xics_probe() for the= IPI. > > > >=20 > > > > Hmm, I don't think that's strong enough. I can trivially cause irqs= to > > > > fire during a kexec reboot just by mashing the keyboard. > > > >=20 > > > > And during a kdump boot all sorts of stuff could be firing. Even du= ring > > > > a clean boot, from firmware, I don't think we can guarantee that > > > > nothing's going to fire. > > > >=20 > > > > .. after a bit of testing .. > > > >=20 > > > > It seems it actually works (sort of).=20 > > > >=20 > > > > xics_remap_irq() calls irq_radix_revmap_lookup(), which calls: > > > >=20 > > > > ptr =3D radix_tree_lookup(&host->revmap_data.tree, hwirq); > > > >=20 > > > > And because =EF=BB=BFhost->revmap_data.tree was zalloc'ed we trip o= n the first > > > > check here: > > >=20 > > > @#$% ctrl-enter =3D=3D send! > > >=20 > > > Continuing ... > > >=20 > > > void *radix_tree_lookup(struct radix_tree_root *root, unsigned long i= ndex) > > > { > > > unsigned int height, shift; > > > struct radix_tree_node *node, **slot; > > >=20 > > > node =3D rcu_dereference(root->rnode); > > > if (node =3D=3D NULL) > > > return NULL; > > >=20 > > > Which means =EF=BB=BFirq_radix_revmap_lookup() will return NO_IRQ, wh= ich is cool. > >=20 > > Which is what I intended so that as long as no IRQ is registered we > > return NO_IRQ. > >=20 > > >=20 > > >=20 > > > So I think it can fly, as long as we're happy that we can't reverse m= ap > > > anything until smp_xics_probe() - and I think that's true, as any irq= we > > > take will be invalid. > >=20 > > That's true as no IRQs are registered before smp_xics_probe() and for= any > > interrupt we might get before that, irq_radix_revmap_lookup() will retu= rn > > NO_IRQ. >=20 > Cool, we agree :)=20 >=20 > My only worry is that we might be relying on on the particular radix > tree implementation a bit too much. Well maybe we could revert back to testing a flag just like we do for host->revmap_data.tree.gfp_mask !=3D 0. Dunno. > Is it documented somewhere that > the /very/ first check is for =EF=BB=BFroot->rnode !=3D NULL, and the res= t of the > root may be unintialised? Not in anything I could read except in looking at the code. >=20 > And I think it needs a big fat comment in the irq code saying that it's > safe because revmap_data is zalloc'ed, and that means the radix lookup > will fail (safely). Yep, right. Will advertise this properly for the next round if this remains the prefered solution. Thanks, Sebastien. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sebastien Dugue Subject: Re: [PATCH] powerpc - Initialize the irq radix tree earlier Date: Thu, 31 Jul 2008 16:14:30 +0200 Message-ID: <20080731161430.5de73ef8@bull.net> References: <1217497241-10685-1-git-send-email-sebastien.dugue@bull.net> <1217497241-10685-2-git-send-email-sebastien.dugue@bull.net> <1217504456.9817.22.camel@localhost> <20080731140002.31bbe4a0@bull.net> <1217509104.19050.11.camel@localhost> <1217509299.19050.15.camel@localhost> <20080731152606.5ae7c379@bull.net> <1217511566.19050.26.camel@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: linuxppc-dev@ozlabs.org, linux-kernel@vger.kernel.org, linux-rt-users@vger.kernel.org, benh@kernel.crashing.org, paulus@samba.org, jean-pierre.dion@bull.net, gilles.carry@ext.bull.net, tinytim@us.ibm.com, tglx@linutronix.de, rostedt@goodmis.org To: michael@ellerman.id.au Return-path: Received: from ecfrec.frec.bull.fr ([129.183.4.8]:54593 "EHLO ecfrec.frec.bull.fr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753818AbYGaOOG convert rfc822-to-8bit (ORCPT ); Thu, 31 Jul 2008 10:14:06 -0400 In-Reply-To: <1217511566.19050.26.camel@localhost> Sender: linux-rt-users-owner@vger.kernel.org List-ID: On Thu, 31 Jul 2008 23:39:26 +1000 Michael Ellerman wrote: > On Thu, 2008-07-31 at 15:26 +0200, Sebastien Dugue wrote: > > On Thu, 31 Jul 2008 23:01:39 +1000 Michael Ellerman wrote: > >=20 > > > On Thu, 2008-07-31 at 22:58 +1000, Michael Ellerman wrote: > > > > On Thu, 2008-07-31 at 14:00 +0200, Sebastien Dugue wrote: > > > > > On Thu, 31 Jul 2008 21:40:56 +1000 Michael Ellerman wrote: > > > > > >=20 > > > > > > This boot ordering stuff is pretty hairy, so I might have m= issed > > > > > > something, but this is how the code is ordered AFAICT: > > > > > > =EF=BB=BF > > > > > > start_kernel() > > > > > > init_IRQ() > > > > > > ... > > > > > > local_irq_enable() > > > > > > ... > > > > > > rest_init() > > > > > > kernel_thread() > > > > > > kernel_init() > > > > > > smp_prepare_cpus() > > > > > > smp_xics_probe() (via smp_ops->probe()) > > > > > >=20 > > > > > >=20 > > > > > > What's stopping us from taking an irq between local_irq_ena= ble() and > > > > > > smp_xics_probe() ? Is it just that no one's request_irq()'= ed them yet? > > > > >=20 > > > > > It's hairy, I agree, but as you've mentioned no one has don= e a request_irq() > > > > > at that point. The first one to do it is smp_xics_probe() for= the IPI. > > > >=20 > > > > Hmm, I don't think that's strong enough. I can trivially cause = irqs to > > > > fire during a kexec reboot just by mashing the keyboard. > > > >=20 > > > > And during a kdump boot all sorts of stuff could be firing. Eve= n during > > > > a clean boot, from firmware, I don't think we can guarantee tha= t > > > > nothing's going to fire. > > > >=20 > > > > .. after a bit of testing .. > > > >=20 > > > > It seems it actually works (sort of).=20 > > > >=20 > > > > xics_remap_irq() calls irq_radix_revmap_lookup(), which calls: > > > >=20 > > > > ptr =3D radix_tree_lookup(&host->revmap_data.tree, hwirq); > > > >=20 > > > > And because =EF=BB=BFhost->revmap_data.tree was zalloc'ed we tr= ip on the first > > > > check here: > > >=20 > > > @#$% ctrl-enter =3D=3D send! > > >=20 > > > Continuing ... > > >=20 > > > void *radix_tree_lookup(struct radix_tree_root *root, unsigned lo= ng index) > > > { > > > unsigned int height, shift; > > > struct radix_tree_node *node, **slot; > > >=20 > > > node =3D rcu_dereference(root->rnode); > > > if (node =3D=3D NULL) > > > return NULL; > > >=20 > > > Which means =EF=BB=BFirq_radix_revmap_lookup() will return NO_IRQ= , which is cool. > >=20 > > Which is what I intended so that as long as no IRQ is registered = we > > return NO_IRQ. > >=20 > > >=20 > > >=20 > > > So I think it can fly, as long as we're happy that we can't rever= se map > > > anything until smp_xics_probe() - and I think that's true, as any= irq we > > > take will be invalid. > >=20 > > That's true as no IRQs are registered before smp_xics_probe() and= for any > > interrupt we might get before that, irq_radix_revmap_lookup() will = return > > NO_IRQ. >=20 > Cool, we agree :)=20 >=20 > My only worry is that we might be relying on on the particular radix > tree implementation a bit too much. Well maybe we could revert back to testing a flag just like we do for host->revmap_data.tree.gfp_mask !=3D 0. Dunno. > Is it documented somewhere that > the /very/ first check is for =EF=BB=BFroot->rnode !=3D NULL, and the= rest of the > root may be unintialised? Not in anything I could read except in looking at the code. >=20 > And I think it needs a big fat comment in the irq code saying that it= 's > safe because revmap_data is zalloc'ed, and that means the radix looku= p > will fail (safely). Yep, right. Will advertise this properly for the next round if this remains the prefered solution. Thanks, Sebastien. -- To unsubscribe from this list: send the line "unsubscribe linux-rt-user= s" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756907AbYGaOOV (ORCPT ); Thu, 31 Jul 2008 10:14:21 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1753915AbYGaOOI (ORCPT ); Thu, 31 Jul 2008 10:14:08 -0400 Received: from ecfrec.frec.bull.fr ([129.183.4.8]:54593 "EHLO ecfrec.frec.bull.fr" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753818AbYGaOOG convert rfc822-to-8bit (ORCPT ); Thu, 31 Jul 2008 10:14:06 -0400 Date: Thu, 31 Jul 2008 16:14:30 +0200 From: Sebastien Dugue To: michael@ellerman.id.au Cc: linuxppc-dev@ozlabs.org, linux-kernel@vger.kernel.org, linux-rt-users@vger.kernel.org, benh@kernel.crashing.org, paulus@samba.org, jean-pierre.dion@bull.net, gilles.carry@ext.bull.net, tinytim@us.ibm.com, tglx@linutronix.de, rostedt@goodmis.org Subject: Re: [PATCH] powerpc - Initialize the irq radix tree earlier Message-ID: <20080731161430.5de73ef8@bull.net> In-Reply-To: <1217511566.19050.26.camel@localhost> References: <1217497241-10685-1-git-send-email-sebastien.dugue@bull.net> <1217497241-10685-2-git-send-email-sebastien.dugue@bull.net> <1217504456.9817.22.camel@localhost> <20080731140002.31bbe4a0@bull.net> <1217509104.19050.11.camel@localhost> <1217509299.19050.15.camel@localhost> <20080731152606.5ae7c379@bull.net> <1217511566.19050.26.camel@localhost> X-Mailer: Claws Mail 3.2.0 (GTK+ 2.12.2; i486-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 31 Jul 2008 23:39:26 +1000 Michael Ellerman wrote: > On Thu, 2008-07-31 at 15:26 +0200, Sebastien Dugue wrote: > > On Thu, 31 Jul 2008 23:01:39 +1000 Michael Ellerman wrote: > > > > > On Thu, 2008-07-31 at 22:58 +1000, Michael Ellerman wrote: > > > > On Thu, 2008-07-31 at 14:00 +0200, Sebastien Dugue wrote: > > > > > On Thu, 31 Jul 2008 21:40:56 +1000 Michael Ellerman wrote: > > > > > > > > > > > > This boot ordering stuff is pretty hairy, so I might have missed > > > > > > something, but this is how the code is ordered AFAICT: > > > > > >  > > > > > > start_kernel() > > > > > > init_IRQ() > > > > > > ... > > > > > > local_irq_enable() > > > > > > ... > > > > > > rest_init() > > > > > > kernel_thread() > > > > > > kernel_init() > > > > > > smp_prepare_cpus() > > > > > > smp_xics_probe() (via smp_ops->probe()) > > > > > > > > > > > > > > > > > > What's stopping us from taking an irq between local_irq_enable() and > > > > > > smp_xics_probe() ? Is it just that no one's request_irq()'ed them yet? > > > > > > > > > > It's hairy, I agree, but as you've mentioned no one has done a request_irq() > > > > > at that point. The first one to do it is smp_xics_probe() for the IPI. > > > > > > > > Hmm, I don't think that's strong enough. I can trivially cause irqs to > > > > fire during a kexec reboot just by mashing the keyboard. > > > > > > > > And during a kdump boot all sorts of stuff could be firing. Even during > > > > a clean boot, from firmware, I don't think we can guarantee that > > > > nothing's going to fire. > > > > > > > > .. after a bit of testing .. > > > > > > > > It seems it actually works (sort of). > > > > > > > > xics_remap_irq() calls irq_radix_revmap_lookup(), which calls: > > > > > > > > ptr = radix_tree_lookup(&host->revmap_data.tree, hwirq); > > > > > > > > And because host->revmap_data.tree was zalloc'ed we trip on the first > > > > check here: > > > > > > @#$% ctrl-enter == send! > > > > > > Continuing ... > > > > > > void *radix_tree_lookup(struct radix_tree_root *root, unsigned long index) > > > { > > > unsigned int height, shift; > > > struct radix_tree_node *node, **slot; > > > > > > node = rcu_dereference(root->rnode); > > > if (node == NULL) > > > return NULL; > > > > > > Which means irq_radix_revmap_lookup() will return NO_IRQ, which is cool. > > > > Which is what I intended so that as long as no IRQ is registered we > > return NO_IRQ. > > > > > > > > > > > So I think it can fly, as long as we're happy that we can't reverse map > > > anything until smp_xics_probe() - and I think that's true, as any irq we > > > take will be invalid. > > > > That's true as no IRQs are registered before smp_xics_probe() and for any > > interrupt we might get before that, irq_radix_revmap_lookup() will return > > NO_IRQ. > > Cool, we agree :) > > My only worry is that we might be relying on on the particular radix > tree implementation a bit too much. Well maybe we could revert back to testing a flag just like we do for host->revmap_data.tree.gfp_mask != 0. Dunno. > Is it documented somewhere that > the /very/ first check is for root->rnode != NULL, and the rest of the > root may be unintialised? Not in anything I could read except in looking at the code. > > And I think it needs a big fat comment in the irq code saying that it's > safe because revmap_data is zalloc'ed, and that means the radix lookup > will fail (safely). Yep, right. Will advertise this properly for the next round if this remains the prefered solution. Thanks, Sebastien.