From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64164C433EF for ; Thu, 23 Jun 2022 20:31:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229915AbiFWUbl (ORCPT ); Thu, 23 Jun 2022 16:31:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229995AbiFWUbk (ORCPT ); Thu, 23 Jun 2022 16:31:40 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E553654BFE; Thu, 23 Jun 2022 13:31:39 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7515261DAD; Thu, 23 Jun 2022 20:31:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9D15BC341C0; Thu, 23 Jun 2022 20:31:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656016298; bh=0j66pIjl2MVj7u0i9YOrsd99nin6Ggez1Qf11YPhGdU=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=gWXCiNaKTLf/70eWbnYFTEiKjUoEJzY8N0dADAS9dKEaoipN+Et4mQDKBBWNrqoCL zegD8ZZ9eyudiKSP3oux/v6LSKUaOqEPwwZCz+VRNQSwlN4WkgtG2G7kBGZTrcfwq8 QB+0dYgA4It8GtVO9vX7SPmWk+S2YiitvtHD2sdAyHzy0aDyiLYjSrFCKGMapZst3r D538EYDCELljgtalrE4C/3rQk8MFcYzhRcVsTHKHUJRb6HLcE4ElSkc7ARwjF2HdYS PKeUljkr7nD6AWst468BN6mMhl0IM9GfgUtshPpX5zaxyIjkUzcvVPfSGnyMu+CiwF e9mz1S51L4RxA== Received: from sofa.misterjones.org ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1o4TUG-002hiz-H6; Thu, 23 Jun 2022 21:31:36 +0100 Date: Thu, 23 Jun 2022 21:31:40 +0100 Message-ID: <874k0bf7f7.wl-maz@kernel.org> From: Marc Zyngier To: Bjorn Helgaas Cc: Pali =?UTF-8?B?Um9ow6Fy?= , Bjorn Helgaas , Thomas Petazzoni , Lorenzo Pieralisi , Rob Herring , Krzysztof =?UTF-8?B?V2lsY3p5xYRza2k=?= , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] PCI: mvebu: Use devm_request_irq() for registering interrupt handler In-Reply-To: <20220623164942.GA1457236@bhelgaas> References: <20220623163240.cu7cq3m7a2pjw62a@pali> <20220623164942.GA1457236@bhelgaas> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: helgaas@kernel.org, pali@kernel.org, bhelgaas@google.com, thomas.petazzoni@bootlin.com, lorenzo.pieralisi@arm.com, robh@kernel.org, kw@linux.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Hi Bjorn, On Thu, 23 Jun 2022 17:49:42 +0100, Bjorn Helgaas wrote: >=20 > [+cc Marc, IRQ affinity vs chained IRQ handlers] >=20 > On Thu, Jun 23, 2022 at 06:32:40PM +0200, Pali Roh=C3=A1r wrote: > > On Thursday 23 June 2022 11:27:47 Bjorn Helgaas wrote: > > > On Tue, May 24, 2022 at 02:28:17PM +0200, Pali Roh=C3=A1r wrote: > > > > Same as in commit a3b69dd0ad62 ("Revert "PCI: aardvark: Rewrite > > > > IRQ code to chained IRQ handler"") for pci-aardvark driver, use > > > > devm_request_irq() instead of chained IRQ handler in pci-mvebu.c > > > > driver. > > > > > > > > This change fixes affinity support and allows to pin interrupts > > > > from different PCIe controllers to different CPU cores. > > >=20 > > > Several other drivers use irq_set_chained_handler_and_data(). Do > > > any of them need similar changes? > >=20 > > I do not know. This needs testing on HW which use those other > > drivers. > >=20 > > > The commit log suggests that using chained IRQ handlers breaks > > > affinity support. But perhaps that's not the case and the real > > > culprit is some other difference between mvebu and the other > > > drivers. > >=20 > > It is possible. But similar patch (revert; linked below) was > > required for aardvark. I tested same approach on mvebu and it fixed > > affinity support. >=20 > This feels like something we should understand better. If > irq_set_chained_handler_and_data() is a problem for affinity, we > should fix it across the board in all the drivers at once. >=20 > If the real problem is something different, we should figure that out > and document it in the commit log. >=20 > I cc'd Marc in case he has time to educate us. Thanks for roping me in. The problem of changing affinities for chained (or multiplexing) interrupts is, to make it short, that it breaks the existing userspace ABI that a change in affinity affects only the interrupt userspace acts upon, and no other. Which is why we don't expose any affinity setting for such an interrupt, as by definition changing its affinity affects all the interrupts that are muxed onto it. By turning a chained interrupt into a normal handler, people work around the above rule and break the contract the kernel has with userspace. Do I think this is acceptable? No. Can something be done about this? Maybe. Marek asked this exact question last month[1], and I gave a detailed explanation of what could be done to improve matters, allowing this to happen as long as userspace is made aware of the effects, which means creating a new userspace ABI. I would rather see people work on a proper solution than add bad hacks that only work in environments where the userspace ABI can be safely ignored, such as on an closed, embedded device. HTH, M. [1] https://lore.kernel.org/all/20220502102137.764606ee@thinkpad/ --=20 Without deviation from the norm, progress is not possible.