From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752961AbcD1OsX (ORCPT ); Thu, 28 Apr 2016 10:48:23 -0400 Received: from foss.arm.com ([217.140.101.70]:44084 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751866AbcD1OsV (ORCPT ); Thu, 28 Apr 2016 10:48:21 -0400 Subject: Re: [PATCH 0/5] Partitioning per-cpu interrupts To: Thomas Gleixner , Jiang Liu , Jason Cooper , Will Deacon , Mark Rutland , Rob Herring References: <1460365075-7316-1-git-send-email-marc.zyngier@arm.com> Cc: linux-kernel@vger.kernel.org, devicetree@vger.kernel.org From: Marc Zyngier Organization: ARM Ltd Message-ID: <572222B0.8010608@arm.com> Date: Thu, 28 Apr 2016 15:48:16 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Icedove/38.7.0 MIME-Version: 1.0 In-Reply-To: <1460365075-7316-1-git-send-email-marc.zyngier@arm.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/04/16 09:57, Marc Zyngier wrote: > We've unfortunately started seeing a situation where percpu interrupts > are partitioned in the system: one arbitrary set of CPUs has an > interrupt connected to a type of device, while another disjoint set of > CPUs has the same interrupt connected to another type of device. > > This makes it impossible to have a device driver requesting this > interrupt using the current percpu-interrupt abstraction, as the same > interrupt number is now potentially claimed by at least two drivers, > and we forbid interrupt sharing on per-cpu interrupt. > > A potential solution to this has been proposed by Will Deacon, > expanding the handling in the core code: > > http://lists.infradead.org/pipermail/linux-arm-kernel/2015-November/388800.html > > followed by a counter-proposal from Thomas Gleixner, which Will tried > to implement, but ran into issues where the probing code was running > in preemptible context, making the percpu-ness of interrupts difficult > to guarantee. > > Another approach to this is to turn things upside down. Let's assume > that our system describes all the possible partitions for a given > interrupt, and give each of them a unique identifier. It is then > possible to create a namespace where the affinity identifier itself is > a form of interrupt number. At this point, it becomes easy to > implement a set of partitions as a cascaded irqchip, each affinity > identifier being the secondary HW irq, as outlined in the following > example: > > Aff-0: { cpu0 cpu3 } > Aff-1: { cpu1 cpu2 } > Aff-2: { cpu4 cpu5 cpu6 cpu7 } > > Let's assume that HW interrupt 1 is partitioned over these 3 > affinities. When HW interrupt 1 fires on a given CPU, all it takes is > to find out which affinity this CPU belongs to, which gives us a new > HW interrupt number. Bingo. Of course, this only works as long as you > don't have overlapping affinities (but if you do your system is broken > anyway). > > This allows us to keep a number of nice properties: > > - Each partition results in a separate percpu-interrupt (with a > restricted affinity), which keeps drivers happy. This alone > garantees that we do not have to change the programming model for > per-cpu interrupts. > > - Because the underlying interrupt is still per-cpu, the overhead of > the indirection can be kept pretty minimal. > > - The core code can ignore most of that crap. > > For that purpose, we implement a small library that deals with some of > the boilerplate code, relying on platform-specific drivers to provide > a description of the affinity sets and a set of callbacks. This also > relies on a small change in the irqdomain layer, and now offers a way > for the affinity of a percpu interrupt to be retrieved by a driver. > > As an example, the GICv3 driver has been adapted to use this new > feature. Patches on top of v4.6-r3, tested on an arm64 FVP model. Any comment on this? The Rockchip dudes have confirmed that this solves their problems (big-little system with PMUs using the same PPI). I've also posted a proof of concept patch for the ARM PMU over there: https://lkml.org/lkml/2016/4/25/227 Thanks, M. -- Jazz is not dead. It just smells funny...