From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7D6AC04A68 for ; Wed, 27 Jul 2022 15:34:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233858AbiG0Pel (ORCPT ); Wed, 27 Jul 2022 11:34:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41048 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231614AbiG0Pek (ORCPT ); Wed, 27 Jul 2022 11:34:40 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 700B13D58A; Wed, 27 Jul 2022 08:34:39 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D790361937; Wed, 27 Jul 2022 15:34:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 336FAC433C1; Wed, 27 Jul 2022 15:34:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658936078; bh=u5FpYXLGudkcqgj5As4ZvpQ7EARcTlBu0Fq/7zqid3U=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Xrr2hgE2YbAur0CpgHNFH6BLtKDSdwC5/OwvZJBkgV0AVQgh0JcajRJKX/GWql4BP 3SKmBeHF4NxAzsunG3GTlXxlc5zeYh/R63qgJ7yFej787PGH6fCQKXceGdXzdk/5B5 RBmrIDUhF2ZIv8VO8QbGxmoC+v0+3yNPgIPDiVEkfWAvUd0fZcsff94sM5XdCA0jtA eg/krYsEbWfdb8VU+WN57P+L7EGkEH+1ZGmrYRsRy26OFFMkzoFpT95jvpx5s3EPoq 12ZTLFZI7bSNHploGh/YsgCPmY+zvE4BPCb6wVVSFo1E8lFoEnZXvnbjKakzhQAem8 m47wBTFIz/L5g== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oGj3U-00AQSf-3V; Wed, 27 Jul 2022 16:34:36 +0100 Date: Wed, 27 Jul 2022 16:34:35 +0100 Message-ID: <871qu6y3g4.wl-maz@kernel.org> From: Marc Zyngier To: Frank Li Cc: "jdmason@kudzu.us" , "tglx@linutronix.de" , "robh+dt@kernel.org" , "krzysztof.kozlowski+dt@linaro.org" , "shawnguo@kernel.org" , "s.hauer@pengutronix.de" , "kw@linux.com" , "bhelgaas@google.com" , "kernel@vger.kernel.org" , "devicetree@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-pci@vger.kernel.org" , Peng Fan , Aisheng Dong , "kernel@pengutronix.de" , "festevam@gmail.com" , dl-linux-imx , "kishon@ti.com" , "lorenzo.pieralisi@arm.com" , "ntb@lists.linux.dev" Subject: Re: [EXT] Re: [PATCH v3 2/4] irqchip: imx mu worked as msi controller In-Reply-To: References: <20220720213036.1738628-1-Frank.Li@nxp.com> <20220720213036.1738628-3-Frank.Li@nxp.com> <874jza525l.wl-maz@kernel.org> <87wnc6xz6r.wl-maz@kernel.org> <877d3zx9su.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: frank.li@nxp.com, jdmason@kudzu.us, tglx@linutronix.de, robh+dt@kernel.org, krzysztof.kozlowski+dt@linaro.org, shawnguo@kernel.org, s.hauer@pengutronix.de, kw@linux.com, bhelgaas@google.com, kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, peng.fan@nxp.com, aisheng.dong@nxp.com, kernel@pengutronix.de, festevam@gmail.com, linux-imx@nxp.com, kishon@ti.com, lorenzo.pieralisi@arm.com, ntb@lists.linux.dev X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org On Wed, 27 Jul 2022 16:23:26 +0100, Frank Li wrote: > > > > > -----Original Message----- > > From: Marc Zyngier > > Sent: Wednesday, July 27, 2022 3:03 AM > > To: Frank Li > > Cc: jdmason@kudzu.us; tglx@linutronix.de; robh+dt@kernel.org; > > krzysztof.kozlowski+dt@linaro.org; shawnguo@kernel.org; > > s.hauer@pengutronix.de; kw@linux.com; bhelgaas@google.com; > > kernel@vger.kernel.org; devicetree@vger.kernel.org; linux-arm- > > kernel@lists.infradead.org; linux-pci@vger.kernel.org; Peng Fan > > ; Aisheng Dong ; > > kernel@pengutronix.de; festevam@gmail.com; dl-linux-imx > imx@nxp.com>; kishon@ti.com; lorenzo.pieralisi@arm.com; > > ntb@lists.linux.dev > > Subject: Re: [EXT] Re: [PATCH v3 2/4] irqchip: imx mu worked as msi > > controller > > > > Caution: EXT Email > > > > On Tue, 26 Jul 2022 22:48:32 +0100, > > Frank Li wrote: > > > > > > > > > > +static void imx_mu_msi_irq_handler(struct irq_desc *desc) > > > > > > > +{ > > > > > > > + struct imx_mu_msi *msi_data = > > irq_desc_get_handler_data(desc); > > > > > > > + u32 status; > > > > > > > + int i; > > > > > > > + > > > > > > > + status = imx_mu_read(msi_data, msi_data->cfg- > > > > >xSR[IMX_MU_RSR]); > > > > > > > + > > > > > > > + chained_irq_enter(irq_desc_get_chip(desc), desc); > > > > > > > + for (i = 0; i < IMX_MU_CHANS; i++) { > > > > > > > + if (status & IMX_MU_xSR_RFn(msi_data->cfg->type, i)) { > > > > > > > + imx_mu_read(msi_data, msi_data->cfg->xRR + i * 4); > > > > > > > + generic_handle_domain_irq(msi_data->parent, i); > > > > > > > > > > > > Why the parent? You must start at the top of the hierarchy. > > > > > > [Frank Li] Do you means that should be msi_data->msi_domain instead > > > of msi_data->parent? > > > > Indeed. you must *not* bypass the hierarchy, and the top level of the > > hierarchy has to implement whatever is required by the interrupt flow. > > > > [Frank Li] I see, just want to confirm msi_data->msi_domain should > be correct here? It should be leaf of irq hierarchy tree. Yes. > > > > > > > > > > > > > > > > > + } > > > > > > > + } > > > > > > > + chained_irq_exit(irq_desc_get_chip(desc), desc); > > > > > > > > > > > > If your MSIs are a chained interrupt, why do you even provide an > > > > > > affinity setting callback? > > > > > > > > > > [Frank Li] it will be crash if no affinity setting callback. > > > > > > > > Then you have to fix your driver. > > > > > > [Frank Li] After debug, msi_domain_set_affinity() have not did null check > > for (parent->chip->irq_set_affinity). > > > I think impact by using dummy set_affinity is minimized. > > > > > > int msi_domain_set_affinity(struct irq_data *irq_data, > > > const struct cpumask *mask, bool force) > > > { > > > struct irq_data *parent = irq_data->parent_data; > > > struct msi_msg msg[2] = { [1] = { }, }; > > > int ret; > > > > > > ret = parent->chip->irq_set_affinity(parent, mask, force); > > > if (ret >= 0 && ret != IRQ_SET_MASK_OK_DONE) { > > > BUG_ON(irq_chip_compose_msi_msg(irq_data, msg)); > > > msi_check_level(irq_data->domain, msg); > > > irq_chip_write_msi_msg(irq_data, msg); > > > } > > > > > > return ret; > > > } > > > > No. Changing the affinity of an interrupt must not affect the affinity > > of another. Given that this is a chained handler, you *cannot* satisfy > > this requirement. So you can't change the affinity at all. > > > > [Frank Li] I understand affinity can't be changed. > But system use set affinity to write msi msg. > > The call stack as > [ 25.508229] epf_ntb_write_msi_msg+0x78/0x90 > [ 25.512512] platform_msi_write_msg+0x2c/0x38 > [ 25.516882] msi_domain_set_affinity+0xb0/0xc0 > [ 25.521330] irq_do_set_affinity+0x174/0x220 > [ 25.525604] irq_setup_affinity+0xe0/0x188 > [ 25.529713] irq_startup+0x88/0x160 > [ 25.533214] __setup_irq+0x6c8/0x768 > > I have not found good place to hook a function to write msi msg. It is called at MSI activation time (msi_domain_activate). M. -- Without deviation from the norm, progress is not possible.