From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EA21225397; Thu, 9 Oct 2025 17:05:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760029501; cv=none; b=ZShwPKWKzTjwejIUepog6gDBKbJXORJcoiFTgkxheEtYZ/fkI4RLYZJnZ6E6uNrgY6dQQreRMfoZjDW8XJqvR9P5To4EuoGQGuZZM7V9fL+VhTJYX6iEn7KE7gOROMQj8n08COjRRwCDKDnkp4763EP5k5uuMVve2uCFejE59JU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1760029501; c=relaxed/simple; bh=XwfqM5QHQpVM8Kdpu0a6NtJ56tZ8bqhomvImo9fS3uk=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=k9ajj3yGrANAQKzkGhyvSZ06smVNVWX1frD4gd7Fsh2SESlL2UeRoYygT/l0u7gk1D52F22Kr1W89Dk7gyeTJ0LXnz8IOeURd+yxFZTMmsx3Uq1e4Q+9wRB274s8CgGoEpSDg3TlKDwx8/rcaJt7HQ0C+6rxBw0GHqIVuvAyRKw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kEa/QA8O; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kEa/QA8O" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1FC1EC4CEE7; Thu, 9 Oct 2025 17:05:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1760029501; bh=XwfqM5QHQpVM8Kdpu0a6NtJ56tZ8bqhomvImo9fS3uk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=kEa/QA8OAig3Qw7EDRe8tDHK9DTEyg9YRFanDbVu/TifAOZdR70nR5TSrWZb0yRMC YCfwq5nRbbVc7M+8RLJfoi+QYfucS2KZWhxB9qYSBrqKKiU+MMhjzSeyOSfy3P+/oD 72zaq7VQXxsvKYtTmLbcxhDoq+zMKrqXe+3sAFmgbcnynw4CBbtm4XpekMTS4ihHcT y/XUz7p/FQ65RZ0sgla5djkU85Kd7xPiZpoTfLU34LAkowIUMCQdBur6o03hs/Vwzn 63AjUiee3ZdYEPlz0NGC++Bf9VGPYmMSWbbrO0ISrGJxMkyib9GUjqrwIDf2aq6FRJ 8rCzo7koEks7w== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1v6u4Z-0000000Chge-0GNv; Thu, 09 Oct 2025 17:04:59 +0000 Date: Thu, 09 Oct 2025 18:04:58 +0100 Message-ID: <86o6qgxayt.wl-maz@kernel.org> From: Marc Zyngier To: Thierry Reding Cc: Thomas Gleixner , linux-tegra@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: IRQ thread timeouts and affinity In-Reply-To: References: <86qzvcxi3j.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: thierry.reding@gmail.com, tglx@linutronix.de, linux-tegra@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Thu, 09 Oct 2025 17:05:15 +0100, Thierry Reding wrote: > > [1 ] > On Thu, Oct 09, 2025 at 03:30:56PM +0100, Marc Zyngier wrote: > > Hi Thierry, > > > > On Thu, 09 Oct 2025 12:38:55 +0100, > > Thierry Reding wrote: > > > > > > Which brings me to the actual question: what is the right way to solve > > > this? I had, maybe naively, assumed that the default CPU affinity, which > > > includes all available CPUs, would be sufficient to have interrupts > > > balanced across all of those CPUs, but that doesn't appear to be the > > > case. At least not with the GIC (v3) driver which selects one CPU (CPU 0 > > > in this particular case) from the affinity mask to set the "effective > > > affinity", which then dictates where IRQs are handled and where the > > > corresponding IRQ thread function is run. > > > > There's a (GIC-specific) answer to that, and that's the "1 of N" > > distribution model. The problem is that it is a massive headache (it > > completely breaks with per-CPU context). > > Heh, that started out as a very promising first paragraph but turned > ugly very quickly... =) > > > We could try and hack this in somehow, but defining a reasonable API > > is complicated. The set of CPUs receiving 1:N interrupts is a *global* > > set, which means you cannot have one interrupt targeting CPUs 0-1, and > > another targeting CPUs 2-3. You can only have a single set for all 1:N > > interrupts. How would you define such a set in a platform agnostic > > manner so that a random driver could use this? I definitely don't want > > to have a GIC-specific API. > > I see. I've been thinking that maybe the only way to solve this is using > some sort of policy. A very simple policy might be: use CPU 0 as the > "default" interrupt (much like it is now) because like you said there > might be assumptions built-in that break when the interrupt is scheduled > elsewhere. But then let individual drivers opt into the 1:N set, which > would perhaps span all available CPUs but the first one. From an API PoV > this would just be a flag that's passed to request_irq() (or one of its > derivatives). The $10k question is how do you pick the victim CPUs? I can't see how to do it in a reasonable way unless we decide that interrupts that have an affinity matching cpu_possible_mask are 1:N. And then we're left with wondering what to do about CPU hotplug. > > > Overall, there is quite a lot of work to be done in this space: the > > machine I'm typing this from doesn't have affinity control *at > > all*. Any interrupt can target any CPU, > > Well, that actually sounds pretty nice for the use-case that we have... > > > and if Linux doesn't expect > > that, tough. > > ... but yeah, it may also break things. Yeah. With GICv3, only SPIs can be 1:N, but on this (fruity) box, even MSIs can be arbitrarily moved from one CPU to another. This is a ticking bomb. I'll see if I can squeeze out some time to look into this -- no promises though. M. -- Without deviation from the norm, progress is not possible.