From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932238AbdEJQWD (ORCPT ); Wed, 10 May 2017 12:22:03 -0400 Received: from mail-qt0-f195.google.com ([209.85.216.195]:34583 "EHLO mail-qt0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932080AbdEJQV7 (ORCPT ); Wed, 10 May 2017 12:21:59 -0400 Subject: Re: [PATCH] irq_bcm2836: Send event when onlining sleeping cores To: Phil Elwell , Marc Zyngier References: <87ziemm0fb.fsf@eliezer.anholt.net> <1e871525-3298-f16d-4f95-a44bc78068b5@arm.com> <87o9v1c37r.fsf@eliezer.anholt.net> <7f6f5e05-07b1-e64d-401c-a189623c22a0@arm.com> <3b9a0ec6-fc9e-5cd0-e40a-e91b5926f53e@raspberrypi.org> <93a69cf4-cbf6-e0a7-b7e3-d0e53040efe4@arm.com> <187815f0-1316-ee87-883d-11304064f8f9@raspberrypi.org> <06fad93c-a850-a99a-ae60-7306f9179032@arm.com> <87tw4thyzu.fsf@on-the-bus.cambridge.arm.com> <907820fc-c100-dad4-cf4f-95aa330667c5@arm.com> <1bbd3527-3328-b9c0-e9e0-d37092850a91@raspberrypi.org> Cc: Eric Anholt , Thomas Gleixner , Jason Cooper , Florian Fainelli , Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, linux-kernel@vger.kernel.org, linux-rpi-kernel@lists.infradead.org From: Florian Fainelli Message-ID: <817a2689-15df-f2e4-3b3d-e5831fba7fa3@gmail.com> Date: Wed, 10 May 2017 09:21:43 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <1bbd3527-3328-b9c0-e9e0-d37092850a91@raspberrypi.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/10/2017 03:31 AM, Phil Elwell wrote: > On 10/05/2017 11:09, Marc Zyngier wrote: >> On 10/05/17 10:05, Phil Elwell wrote: >>> On 10/05/2017 09:55, Marc Zyngier wrote: >>>> On Wed, May 10 2017 at 9:27:10 am BST, Phil Elwell wrote: >>>>> On 10/05/2017 08:42, Marc Zyngier wrote: >>>>>> On 09/05/17 20:02, Phil Elwell wrote: >>>>>>> On 09/05/2017 19:53, Marc Zyngier wrote: >>>>>>>> On 09/05/17 19:52, Phil Elwell wrote: >>>>>>>>> On 09/05/2017 19:14, Marc Zyngier wrote: >>>>>>>>>> On 09/05/17 19:08, Eric Anholt wrote: >>>>>>>>>>> Marc Zyngier writes: >>>>>>>>>>> >>>>>>>>>>>> On 09/05/17 17:59, Eric Anholt wrote: >>>>>>>>>>>>> Phil Elwell writes: >>>>>>>>>>>>> >>>>>>>>>>>>>> In order to reduce power consumption and bus traffic, it is sensible >>>>>>>>>>>>>> for secondary cores to enter a low-power idle state when waiting to >>>>>>>>>>>>>> be started. The wfe instruction causes a core to wait until an event >>>>>>>>>>>>>> or interrupt arrives before continuing to the next instruction. >>>>>>>>>>>>>> The sev instruction sends a wakeup event to the other cores, so call >>>>>>>>>>>>>> it from bcm2836_smp_boot_secondary, the function that wakes up the >>>>>>>>>>>>>> waiting cores during booting. >>>>>>>>>>>>>> >>>>>>>>>>>>>> It is harmless to use this patch without the corresponding change >>>>>>>>>>>>>> adding wfe to the ARMv7/ARMv8-32 stubs, but if the stubs are updated >>>>>>>>>>>>>> and this patch is not applied then the other cores will sleep forever. >>>>>>>>>>>>>> >>>>>>>>>>>>>> See: https://github.com/raspberrypi/linux/issues/1989 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Signed-off-by: Phil Elwell >>>>>>>>>>>>>> --- >>>>>>>>>>>>>> drivers/irqchip/irq-bcm2836.c | 3 +++ >>>>>>>>>>>>>> 1 file changed, 3 insertions(+) >>>>>>>>>>>>>> >>>>>>>>>>>>>> diff --git a/drivers/irqchip/irq-bcm2836.c b/drivers/irqchip/irq-bcm2836.c >>>>>>>>>>>>>> index e10597c..6dccdf9 100644 >>>>>>>>>>>>>> --- a/drivers/irqchip/irq-bcm2836.c >>>>>>>>>>>>>> +++ b/drivers/irqchip/irq-bcm2836.c >>>>>>>>>>>>>> @@ -248,6 +248,9 @@ static int __init bcm2836_smp_boot_secondary(unsigned int cpu, >>>>>>>>>>>>>> writel(secondary_startup_phys, >>>>>>>>>>>>>> intc.base + LOCAL_MAILBOX3_SET0 + 16 * cpu); >>>>>>>>>>>>>> >>>>>>>>>>>>>> + dsb(sy); /* Ensure write has completed before waking the other CPUs */ >>>>>>>>>>>>>> + sev(); >>>>>>>>>>>>>> + >>>>>>>>>>>>>> return 0; >>>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> This is also the behavior that the standard arm64 spin-table >>>>>>>>>>>>> method has, >>>>>>>>>>>>> which we unfortunately can't quite use. >>>>>>>>>>>> >>>>>>>>>>>> And why is that so? Why do you have to reinvent the wheel (and hide the >>>>>>>>>>>> cloned wheel in an interrupt controller driver)? >>>>>>>>>>>> >>>>>>>>>>>> That doesn't seem right to me. >>>>>>>>>>> >>>>>>>>>>> The armv8 stubs (firmware-supplied code in the low page that do the >>>>>>>>>>> spinning) do actually implement arm64's spin-table method. It's the >>>>>>>>>>> armv7 stubs that use these registers in the irqchip instead of plain >>>>>>>>>>> addresses in system memory. >>>>>>>>>> >>>>>>>>>> Let's put ARMv7 aside for the time being. If your firmware already >>>>>>>>>> implements spin-tables, why don't you simply use that at least on arm64? >>>>>>>>> >>>>>>>>> We do. >>>>>>>> >>>>>>>> Obviously not the way it is intended if you have to duplicate the core >>>>>>>> architectural code in the interrupt controller driver, which couldn't >>>>>>>> care less. >>>>>>> >>>>>>> If we were using this method on arm64 then the other cores would not start up >>>>>>> because armstub8.S has always included a wfe. Nothing in the commit mentions >>>>>>> arm64 - this is an ARCH=arm fix. >>>>>> >>>>>> Thanks for the clarification, which you could have added to the commit >>>>>> message. >>>>>> >>>>>> The question still remains: why do we have CPU bring-up code in an >>>>>> interrupt controller, instead of having it in the architecture code? >>>>>> >>>>>> The RPi-2 is the *only* platform to have its SMP bringup code outside of >>>>>> arch/arm, so the first course of action would be to move that code where >>>>>> it belongs. >>>>> >>>>> You were CC'd on the commit (41f4988cc287e5f836d3f6620c9f900bc9b560e9) that >>>>> introduced bcm2836_smp_boot_secondary - it seems strange to start objecting >>>>> now. >>>> >>>> Well, I'm far from being perfect. If I had noticed it, I'd have NACKed >>>> it. >>>> >>>>> Yes, I think it is odd that it didn't go into arch/arm/mach-bcm, but in >>>>> the interests of making changes in small, independent steps, do you have a >>>>> problem with this commit? >>>> >>>> On its own, no. I'm just not keen on adding more unrelated stuff to this >>>> file, so let's start with dealing with the original bug, and you can >>>> then add this fix on top. >>> >>> That's an interesting use of the word "bug". From Wikipedia: >>> >>> "A software bug is an error, flaw, failure or fault in a computer program or >>> system that causes it to produce an incorrect or unexpected result, or to >>> behave in unintended ways." >> >> Whatever. Should I call it "pile of crap dumped in unsuitable locations" >> instead? What does Wikipedia says about it? >> >>> Although your concerns are valid, the faults you are objecting to are not causing >>> a malfunction of any kind. If we were to update the RPi firmware before this >>> patch was merged then upstream users would be left with one wheel on their wagon. >> >> And that'd be your problem, not mine. Look, you can argue around this >> all day, or you can fix this mess. Your choice. > > Is that the opinion of all here? The choice of word here got largely out of the original topic and I surely did eat a ton of popcorn here. There are two things that need fixing, and the time line and process for fixing these is clear: - your bugfix (Phil) is something that should be applied now, and backported to -stable trees once the fix hits the irqchip tree (or Linus') - relocating the code that does the secondary boot out of drivers/irqchip/ into arch/arm/mach-bcm/ needs to happen (Marc), and this is 4.13 material, there is no urgency in doing this *right now*, but it needs to happen Does that work for everyone? -- Florian