From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63E7AC4345F for ; Wed, 17 Apr 2024 15:33:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=k1ZZZVeTOeXtsqkOTOeXnDAG5YLzX+omDNgka8UX1Rc=; b=RNQa377xP9DUOG dLsju4k7xyibiSGKTV+/+uMfLBeC3IW9doJbKn6hSzwC2f5mqDH1RpohG07YIK0gi0AKWYoVTNtVi KXRFTsrfC5Hqz4uuh5BJ6GCaE89nWncW9xg6QK1AuctQQxM7mcmzG00e6ye97zey53EhNpLKjSg7X 1/VghVJbQioxpYFr9W2a0sLF02vvaTZLmJRHxPEqoGmrgmyK6rZ5b7RMV/qqLFGTjaGwms1yUFp2+ skZJOUCpDd7xPHeT/oMN1PaKl0QmyBjicF83sKhcYTy4QEsjPZMC/xiu4iJWhGcgNYVUYCx40Xj/l b0gUVVrAf8jBJA6wBt6g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rx7Ho-0000000Gbnn-2zbY; Wed, 17 Apr 2024 15:33:24 +0000 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rx7Hj-0000000Gbgr-1ss8 for linux-arm-kernel@lists.infradead.org; Wed, 17 Apr 2024 15:33:22 +0000 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1e3f17c64daso40342715ad.3 for ; Wed, 17 Apr 2024 08:33:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1713367993; x=1713972793; darn=lists.infradead.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Ci0/H2jmcb2D55FJA4q2NJz7BMdC0uHP2xAuYH6nUsQ=; b=GvrHdOlf8JGxZ6C6YjSo9yFDx+1/W6QeuoMMZ2Qh1lCInqUGnqxf6dVkk2lj8p5Apf 08bb+KBWpOFJhLu9iMtc4np2rvwxXDZ0HaJBhmIKlzR/6JVj7wzm93TZFzOiB/abdJyX CTWpg4bH+8M6YeTAP3rq1BsVWX30YeIeXl6k71bv4oECWiNki9hvEyBgmVpleAmZDoji Nye4LiM+exzJ1iF47Q26rvVxKGjUpfHULU3QZKPgl/Ebzpvc0nCgwWBlQqDMHv3JnMVd Co8kSjBQxsL1FnU91sdsIn3lxgu/x6KYW4OQNaZZKc2qT7O12X7vTg+2MvaaXd0ivPb/ 1gRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713367993; x=1713972793; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Ci0/H2jmcb2D55FJA4q2NJz7BMdC0uHP2xAuYH6nUsQ=; b=oy3cuI3W6hv2w+TTVHpVKCOTp4bB2EkTZv2kbShj78NyOVRuySCYjG9am+SvZmWZjt UP687nziNurvIvQzjLOjVdtbWqKuWCd0gbbK/4Xjzv7JANSUPRDyJ6Ss6mg2JhZ7rXii yKmx3fHJcpESDUtg4AMU/8uaxWAPzVthTEe2JzCX+DBfn9V5b6mkGMOV1NoN2il4i3Iv wAahoSS9WadPFzi0V/KhDXJNKgSdgK6hMSZDQsD41huKdnMFbLW1axOhx1zRzJKdLvKB 1mxcvM6aioJ8YgzgF+R5S2TVQfjIcvoXiEDP0zUCyM81+yj2SO78pQka50kVoYzkoAHD Il2w== X-Gm-Message-State: AOJu0YwdarPjZzxZNF9u+lnRxo8iNlCRircsNsmL83BRqNtWsQOjuOxK 15vw9z6Q7LsUx5SG+n0uZeJMESkJIr3ivWVjyfFvHUJldjYlX2Jsic6YlP80CSk= X-Google-Smtp-Source: AGHT+IHRZkCksWn0BOoecd+0pqZBwGMmL39xcXlgwLsVriOPR0oxo+w+nj8DW8s7r0j6AcJINnUn5A== X-Received: by 2002:a17:902:e5c1:b0:1e4:35b9:f150 with SMTP id u1-20020a170902e5c100b001e435b9f150mr19787192plf.11.1713367993089; Wed, 17 Apr 2024 08:33:13 -0700 (PDT) Received: from sunil-laptop ([106.51.187.230]) by smtp.gmail.com with ESMTPSA id j9-20020a170903024900b001e83a718d87sm1102769plh.19.2024.04.17.08.33.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Apr 2024 08:33:12 -0700 (PDT) Date: Wed, 17 Apr 2024 21:03:02 +0530 From: Sunil V L To: Bjorn Helgaas Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-acpi@vger.kernel.org, linux-pci@vger.kernel.org, acpica-devel@lists.linux.dev, Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , "Rafael J . Wysocki" , Len Brown , Bjorn Helgaas , Anup Patel , Thomas Gleixner , Samuel Holland , Robert Moore , Haibo1 Xu , Conor Dooley , Andrew Jones , Atish Kumar Patra , Andrei Warkentin , Marc Zyngier , =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= Subject: Re: [RFC PATCH v4 03/20] PCI: Make pci_create_root_bus() declare its reliance on MSI domains Message-ID: References: <20240416204653.GA164172@bhelgaas> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240416204653.GA164172@bhelgaas> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240417_083319_552120_EF5DE1C7 X-CRM114-Status: GOOD ( 47.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Apr 16, 2024 at 03:46:53PM -0500, Bjorn Helgaas wrote: > On Tue, Apr 16, 2024 at 01:54:04PM +0530, Sunil V L wrote: > > Hi Bjorn, > > > > On Mon, Apr 15, 2024 at 06:15:23PM -0500, Bjorn Helgaas wrote: > > > On Mon, Apr 15, 2024 at 10:30:56PM +0530, Sunil V L wrote: > > > > Similar to commit 9ec37efb8783 ("PCI/MSI: Make > > > > pci_host_common_probe() declare its reliance on MSI domains"), declare > > > > this dependency for PCI probe in ACPI based flow. > > > > > > > > This is required especially for RISC-V platforms where MSI controller > > > > can be absent. However, setting this for all architectures seem to cause > > > > issues on non RISC-V architectures [1]. Hence, enabled this only for > > > > RISC-V. > > > > > > > > [1] - https://lore.kernel.org/oe-lkp/202403041047.791cb18e-oliver.sang@intel.com > > > > > > > > Signed-off-by: Sunil V L > > > > --- > > > > drivers/pci/probe.c | 3 +++ > > > > 1 file changed, 3 insertions(+) > > > > > > > > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > > > > index 1325fbae2f28..e09915bee2ee 100644 > > > > --- a/drivers/pci/probe.c > > > > +++ b/drivers/pci/probe.c > > > > @@ -3048,6 +3048,9 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, > > > > bridge->sysdata = sysdata; > > > > bridge->busnr = bus; > > > > bridge->ops = ops; > > > > +#ifdef CONFIG_RISCV > > > > + bridge->msi_domain = true; > > > > +#endif > > > > > > Ugh. I looked at [1], but that's not a very good justification for > > > this #ifdef. The fault mentioned in [1] would need to be fixed, but > > > not this way. > > > > Thank you again for the feedback! > > > > I agree. This is due to my limitation with knowledge and resources to > > debug the issue happening on non-UEFI x86 system with some particular > > PCIe RC. Also, I was worried that we get into a rat hole of > > assumptions/quirks with various architecture/PCIe RC combinations. > > The problem is that adding #ifdefs like this leads to a rat hole > itself. We need to understand and fix the underlying issue instead. > Agree. Ideally, from my reading of code, this change should have worked across architectures. > > For ex: I think the issue is, somehow MSI domain is not set at the time > > of PCI host bridge registration in pci_register_host_bridge() causing > > PCI_BUS_FLAGS_NO_MSI to be set. This causes pci_alloc_irq_vectors() to > > fail. In portdrv.c, pcie_init_service_irqs() doesn't switch to INTx > > handling if MSI can not be used. It switches only if pcie_pme_no_msi() > > returns true. I couldn't find who actually sets up MSI domain bit late > > on this platform so that it somehow worked when we didn't set this flag. > > > > Unfortunately, I don't have system to root cause and fix this issue with > > confidence. Also, I don't know if any other architectures have similar > > issues which are not caught yet. Hence, I thought it may be better > > just restrict the change to RISC-V. > > It sounds like the above is a good start on analyzing the problem. > > I don't quite understand your statement that pcie_init_service_irqs() > doesn't fall back to INTx when MSI/MSI-X is not available. > > I'm looking at this: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/pcie/portdrv.c?id=v6.8#n177 > pcie_port_enable_irq_vec() attempts > pci_alloc_irq_vectors(PCI_IRQ_MSIX | PCI_IRQ_MSI) and returns 0 if > successful. If it returns failure, it looks like > pcie_init_service_irqs() *does* fall through to trying INTx > (PCI_IRQ_LEGACY). > You are right. Not sure what I was looking at :-(. I think fundamentally there are two issues here. 1) MSI domain should have been setup properly when pci_register_host_bridge() is called. I see that pci_arch_init() which is supposed to get called early calls x86_create_pci_msi_domain(). pci_register_host_bridge() also calls pci_set_bus_msi_domain() to setup the MSI domain which can walk up to host bridge to find. So, not sure why PCI_BUS_FLAGS_NO_MSI is getting set. Is there an issue in walking up the tree? 2) When it switches to legacy interrupt since MSI domain is not found, for some reason there is an interrupt enabled without a handler. I was suspecting PME since it was matching the IRQ#16 but it looks like PME handlers are present. I am unable to find anything suspicious from the log alone. I really don't know how to proceed further. With my limited understanding, I don't get any hint what is happening from the log. Thanks, Sunil _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel