From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4D8CC433EF for ; Sun, 17 Apr 2022 09:53:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233839AbiDQJ4Q (ORCPT ); Sun, 17 Apr 2022 05:56:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233831AbiDQJ4P (ORCPT ); Sun, 17 Apr 2022 05:56:15 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41073192A9; Sun, 17 Apr 2022 02:53:39 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8D5F56119F; Sun, 17 Apr 2022 09:53:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACD52C385A4; Sun, 17 Apr 2022 09:53:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1650189218; bh=FPGOhka1OfYKDPNYUqhjOQxntwkCKuERgN9/gaL5oSg=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=HgDNqPoSVC6yUBvDb277MAk2tXdnzWa6RDRmSgAORvgkQfuPFpApvZ0sgp/RrpPPB l/7G+BKz6qFBbS574Ug2iPK7+nZJkkIEtpSE0T9Syr+PnjYZTqrp0aQf9YGuw3Of+1 HKGD5zISkwWiOVzzSKG2+ENb9mxUY2QMNUDhxeXtTlEXlYdSAByXbS50tM7lRxwveW vt4xYdCdD4OnBY4RTpySsDELcQJB2IrP2Ejm5285ueuY5SVixjUypbLldxZDLSIN5q W+jqV12QHWnO5GKPMj/0S/0ebhdH2CPxFz9WRXh7tPSWbgmbsylbH5WCwuCWMUSiEP EGzwP8E8UWf6A== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ng1b6-004qUr-8n; Sun, 17 Apr 2022 10:53:36 +0100 Date: Sun, 17 Apr 2022 10:53:35 +0100 Message-ID: <87zgkk9gtc.wl-maz@kernel.org> From: Marc Zyngier To: Peter Geis Cc: Lorenzo Pieralisi , Rob Herring , Krzysztof =?UTF-8?B?V2lsY3p5xYRza2k=?= , Bjorn Helgaas , Heiko Stuebner , "open list:ARM/Rockchip SoC..." , PCI , devicetree , arm-mail-list , Linux Kernel Mailing List Subject: Re: [PATCH v7 2/4] PCI: dwc: rockchip: add legacy interrupt support In-Reply-To: References: <20220416110507.642398-1-pgwipeout@gmail.com> <20220416110507.642398-3-pgwipeout@gmail.com> <308e9c47197d4f7ae5a31cfcb5a10886@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: pgwipeout@gmail.com, lorenzo.pieralisi@arm.com, robh@kernel.org, kw@linux.com, bhelgaas@google.com, heiko@sntech.de, linux-rockchip@lists.infradead.org, linux-pci@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org On Sat, 16 Apr 2022 14:24:26 +0100, Peter Geis wrote: > > On Sat, Apr 16, 2022 at 8:54 AM Marc Zyngier wrote: > > > > Peter, > > > > May I suggest that you slow down on the number of versions you send? > > This is the 7th in 5 days, the 3rd today. > > > > At this stage, this is entirely counterproductive. > > Apologies, I'll be sure to be at least one cup of coffee in before > doing early morning code. Even with a steady intake of coffee, there is a pretty clear policy around the frequency of patch submission, see [1]. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst#n337 There is no hard enforcement of this process, but that should give you an idea of how to deal with it. In any case, 7 series in less than a week is a clear sign that this series should be *ignored*, as the author is likely to post yet another one in the next few hours. > > > > > On 2022-04-16 12:05, Peter Geis wrote: > > > The legacy interrupts on the rk356x pcie controller are handled by a > > > single muxed interrupt. Add irq domain support to the pcie-dw-rockchip > > > driver to support the virtual domain. > > > > > > Signed-off-by: Peter Geis > > > --- > > > drivers/pci/controller/dwc/pcie-dw-rockchip.c | 112 +++++++++++++++++- > > > 1 file changed, 110 insertions(+), 2 deletions(-) > > > > > > diff --git a/drivers/pci/controller/dwc/pcie-dw-rockchip.c > > > b/drivers/pci/controller/dwc/pcie-dw-rockchip.c > > > index c9b341e55cbb..863374604fb1 100644 > > > --- a/drivers/pci/controller/dwc/pcie-dw-rockchip.c > > > +++ b/drivers/pci/controller/dwc/pcie-dw-rockchip.c > > > @@ -10,9 +10,12 @@ > > > > > > #include > > > #include > > > +#include > > > +#include > > > #include > > > #include > > > #include > > > +#include > > > #include > > > #include > > > #include > > > @@ -36,10 +39,13 @@ > > > #define PCIE_LINKUP (PCIE_SMLH_LINKUP | PCIE_RDLH_LINKUP) > > > #define PCIE_L0S_ENTRY 0x11 > > > #define PCIE_CLIENT_GENERAL_CONTROL 0x0 > > > +#define PCIE_CLIENT_INTR_STATUS_LEGACY 0x8 > > > +#define PCIE_CLIENT_INTR_MASK_LEGACY 0x1c > > > #define PCIE_CLIENT_GENERAL_DEBUG 0x104 > > > -#define PCIE_CLIENT_HOT_RESET_CTRL 0x180 > > > +#define PCIE_CLIENT_HOT_RESET_CTRL 0x180 > > > #define PCIE_CLIENT_LTSSM_STATUS 0x300 > > > -#define PCIE_LTSSM_ENABLE_ENHANCE BIT(4) > > > +#define PCIE_LEGACY_INT_ENABLE GENMASK(3, 0) > > > +#define PCIE_LTSSM_ENABLE_ENHANCE BIT(4) > > > #define PCIE_LTSSM_STATUS_MASK GENMASK(5, 0) > > > > > > struct rockchip_pcie { > > > @@ -51,6 +57,8 @@ struct rockchip_pcie { > > > struct reset_control *rst; > > > struct gpio_desc *rst_gpio; > > > struct regulator *vpcie3v3; > > > + struct irq_domain *irq_domain; > > > + raw_spinlock_t irq_lock; > > > }; > > > > > > static int rockchip_pcie_readl_apb(struct rockchip_pcie *rockchip, > > > @@ -65,6 +73,94 @@ static void rockchip_pcie_writel_apb(struct > > > rockchip_pcie *rockchip, > > > writel_relaxed(val, rockchip->apb_base + reg); > > > } > > > > > > +static void rockchip_pcie_legacy_int_handler(struct irq_desc *desc) > > > +{ > > > + struct irq_chip *chip = irq_desc_get_chip(desc); > > > + struct rockchip_pcie *rockchip = irq_desc_get_handler_data(desc); > > > + unsigned long reg, hwirq; > > > + > > > + chained_irq_enter(chip, desc); > > > + > > > + reg = rockchip_pcie_readl_apb(rockchip, > > > PCIE_CLIENT_INTR_STATUS_LEGACY); > > > + > > > + for_each_set_bit(hwirq, ®, 8) > > > > 8? And yet: > > > > #define PCI_NUM_INTX 4 > > > > So whatever bits are set above bit 3, you are feeding garbage > > to the irqdomain code. > > There are 8 bits in total, the top four are for the TX interrupts, for > which EP mode is not yet supported by the driver. So why aren't they excluded from the set of bits that you look at? > I can constrain this further and let it be expanded when that support > is added, if that works for you? Well, you can't have INTx interrupts in EP mode (that's a TLP going out of the device, and not something that is signalled *to* the CPU). So the two should be mutually exclusive. > > > > > > + generic_handle_domain_irq(rockchip->irq_domain, hwirq); > > > + > > > + chained_irq_exit(chip, desc); > > > +} > > > + > > > +static void rockchip_intx_mask(struct irq_data *data) > > > +{ > > > + struct rockchip_pcie *rockchip = irq_data_get_irq_chip_data(data); > > > + unsigned long flags; > > > + u32 val; > > > + > > > + /* disable legacy interrupts */ > > > + raw_spin_lock_irqsave(&rockchip->irq_lock, flags); > > > + val = HIWORD_UPDATE_BIT(PCIE_LEGACY_INT_ENABLE); > > > + val |= PCIE_LEGACY_INT_ENABLE; > > > + rockchip_pcie_writel_apb(rockchip, val, > > > PCIE_CLIENT_INTR_MASK_LEGACY); > > > + raw_spin_unlock_irqrestore(&rockchip->irq_lock, flags); > > > > This is completely busted. INTx lines must be controlled individually. > > If I disable one device's INTx output, I don't want to see the > > interrupt firing because another one has had its own enabled. > > Okay, that makes sense. I'm hitting the entire block when it should be > the individual IRQ. > I also notice some drivers protect this with a spinlock while others > do not, how should this be handled? It obviously depends on how the HW. works. If this is a shared register using a RMW sequence, then you need some form of mutual exclusion in order to preserve the atomicity of the update. If the HW supports updating the masks using a set of hot bits (with separate clear/set registers), than there is no need for locking. In your case PCIE_CLIENT_INTR_MASK_LEGACY seems to support this odd "write-enable" feature which can probably be used to implement a lockless access, something like: void mask(struct irq_data *d) { u32 val = BIT(d->hwirq + 16) | BIT(d->hwirq); writel_relaxed(val, ...); } void mask(struct irq_data *d) { u32 val = BIT(d->hwirq + 16); writel_relaxed(val, ...); } Another thing is that it is completely unclear to me what initialises these interrupts the first place (INTR_MASK_LEGACY, INTR_EN_LEGACY). Are you relying on the firmware to do that for you? Thanks, M. -- Without deviation from the norm, progress is not possible.