From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D694C433EF for ; Tue, 3 May 2022 09:33:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233449AbiECJgj (ORCPT ); Tue, 3 May 2022 05:36:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232146AbiECJgg (ORCPT ); Tue, 3 May 2022 05:36:36 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7BDF6289B7 for ; Tue, 3 May 2022 02:33:04 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 290BEB81D1B for ; Tue, 3 May 2022 09:33:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AE23AC385A4; Tue, 3 May 2022 09:33:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651570381; bh=3hJTw2o34sSVVc5erLnf4L9EiyUaTq6gZYXD782ef+I=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=BMM+LMqpUIvJ2q69Nfjb5EI+KtFimjuy5/MDGeT4KeuelSFkUTZPU2Ml/Rt75gSC9 QciiwvSan0443wO+GWP5qRZsFR3Bv5j++BaxepHbf/675SLiPHCmlN7do2a9G5Oef0 jZ2xCMPgzxZ1SjplHE/xmfhSTK/ETz5vP2qpIsh1fF5b81nBGMt4GxusM/jt1rJYsu bbRcj/gjcESIZKLlytzQhvU+xt7GhfcU3j/ywmgPbOLQVNtpQMRtixjkiI4W66uzvp Bblr2W2W2EjtO+CLc8Py++6yzqm6zFcy8l3Pwe9YeFAnKrEa6SYWU54DB9nUSktfls 3WYDj5hSyhyaw== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nlotv-008biW-3g; Tue, 03 May 2022 10:32:59 +0100 Date: Tue, 03 May 2022 10:32:58 +0100 Message-ID: <87fslr7ygl.wl-maz@kernel.org> From: Marc Zyngier To: Marek =?UTF-8?B?QmVow7pu?= Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , pali@kernel.org Subject: Re: irqdomain API: how to set affinity of parent irq of chained irqs? In-Reply-To: <20220502174559.78f5cbc0@dellmb> References: <20220502102137.764606ee@thinkpad> <87mtg0i8m8.wl-maz@kernel.org> <20220502174559.78f5cbc0@dellmb> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kabel@kernel.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, pali@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 02 May 2022 16:45:59 +0100, Marek Beh=C3=BAn wrote: >=20 > On Mon, 02 May 2022 10:31:11 +0100 > Marc Zyngier wrote: >=20 > > On Mon, 02 May 2022 09:21:37 +0100, > > Marek Beh=C3=BAn wrote: > > >=20 > > > Dear Marc, Thomas, > > >=20 > > > we have encountered the following problem that can hopefully be put > > > some light onto: What is the intended way to set affinity (and possib= ly > > > other irq attributes) of parent IRQ of chained IRQs, when using the > > > irqdomain API? =20 > >=20 > > Simples: you can't. What sense does it make to change the affinity of > > the parent interrupt, given that its fate is tied to *all* of the > > other interrupts that are muxed to it? >=20 > Dear Marc, >=20 > thank you for your answer. Still: >=20 > What about when we want to set the same affinity for all the chained > interrupts? >=20 > Example: on Armada 385 there are 4 PCIe controllers. Each controller > has one interrupt from which we trigger chained interrupts. We would > like to configure each controller to trigger interrupt (and thus all > chained interrupts in the domain) on different CPU core. >=20 > Moreover we would really like to do this in runtime, through sysfs, > depending on for example whether there are cards plugged in the PCIe > ports. >=20 > Maybe there should be some mechanism to allow to change affinity for > whole irqdomain, or something? Should? Maybe. But not for an irqdomain (which really doesn't have anything to do with interrupt affinity). What you may want is a new sysfs interface that would allow a parent interrupt affinity being changed, but also exposing to userspace all the interrupts this affects *at the same time*. something like: /sys/kernel/irq/42/smp_affinity_list /sys/kernel/irq/42/muxed_irqs/ /sys/kernel/irq/42/muxed_irqs/56 -> ../../56 /sys/kernel/irq/42/muxed_irqs/57 -> ../../57 The main issues are that: - we don't really track the muxing information in any of the data structures, so you can't just walk a short list and generate this information. You'd need to build the topology information at allocation time (or fish it out at runtime, but that's likely a pain). - sysfs doesn't deal with affinities at all. procfs does, but adding more crap there is frowned upon. - it *must* be a new interface. You can't repurpose the existing one, as something like irqbalance would be otherwise be massively confused by seeing interrupts moving around behind its back. - conversely, you'll need to teach irqbalance how to deal with this new interface. - this needs to be safe against CPU hotplug. It probably already is, but nobody ever tested it, given that userspace can't interact with these interrupts at the moment. M. --=20 Without deviation from the norm, progress is not possible.