From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F184C11F68 for ; Tue, 29 Jun 2021 23:14:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5825161D10 for ; Tue, 29 Jun 2021 23:14:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235199AbhF2XQk (ORCPT ); Tue, 29 Jun 2021 19:16:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:57582 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233329AbhF2XQk (ORCPT ); Tue, 29 Jun 2021 19:16:40 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 890A061D16; Tue, 29 Jun 2021 23:14:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1625008451; bh=B9niwO40e9W9wr3Oqskh655dcl5dlMNvAjpZ/8BRQaE=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=WKUu6rfYtfCck0isLyJxRq53hVBBHIe9UPk83ec+yu81yY0P1LjgQ4pm51qmoNI7m O2GDQ9/0MNb21bfDfrQzUo7TzG29mohPNwdYxoy5ZW385X+dWsEO/u68aXTphVSmHp LhqzUNqKofRqPONUNzBsHoH8cpB3DEBIW58R/fKRnMR9ebymoCe9vkrGUZE7Qm2fQV +La8XXcOObtRLcgqdDVIOvde+PjxFhOSyyHklgSTUp8wH/oPabZ7C/HDobWhdLsNEI IwX5wl09+G2kodABsPjHtSCH432NRzKSWyZXfCS0KUu9FqiBgaBWgfoqz/H0INYtz2 S2mwHscZmOjHg== Date: Tue, 29 Jun 2021 18:14:10 -0500 From: Bjorn Helgaas To: Robin Murphy Cc: Javier Martinez Canillas , linux-kernel@vger.kernel.org, Peter Robinson , Shawn Lin , Bjorn Helgaas , Heiko Stuebner , Lorenzo Pieralisi , Rob Herring , linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-rockchip@lists.infradead.org, Michal Simek , Ley Foon Tan , rfi@lists.rocketboards.org, Jingoo Han , Thierry Reding , Jonathan Hunter , linux-tegra@vger.kernel.org Subject: Re: [PATCH v2] PCI: rockchip: Avoid accessing PCIe registers with clocks gated Message-ID: <20210629231410.GA4097899@bjorn-Precision-5520> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3d5a983f-bfdd-d79b-4ec9-357ea26dd2c8@arm.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Tue, Jun 29, 2021 at 11:52:44AM +0100, Robin Murphy wrote: > On 2021-06-29 07:17, Javier Martinez Canillas wrote: > > On 6/29/21 2:38 AM, Bjorn Helgaas wrote: > > > On Thu, Jun 24, 2021 at 05:40:40PM -0500, Bjorn Helgaas wrote: > > > > [snip] > > > > > > > > > > > > So let's just move all the IRQ init before the pci_host_probe() call, that > > > > > will prevent issues like this and seems to be the correct thing to do too. > > > > > > > > Previously we registered rockchip_pcie_subsys_irq_handler() and > > > > rockchip_pcie_client_irq_handler() before the PCIe clocks were > > > > enabled. That's a problem because they depend on those clocks being > > > > enabled, and your patch fixes that. > > > > > > > > rockchip_pcie_legacy_int_handler() depends on rockchip->irq_domain, > > > > which isn't initialized until rockchip_pcie_init_irq_domain(). > > > > Previously we registered rockchip_pcie_legacy_int_handler() as the > > > > handler for the "legacy" IRQ before rockchip_pcie_init_irq_domain(). > > > > > > > > I think your patch *also* fixes that problem, right? > > > > > > The lack of consistency in how we use > > > irq_set_chained_handler_and_data() really bugs me. > > > > > > Your patch fixes the ordering issue where we installed > > > rockchip_pcie_legacy_int_handler() before initializing data > > > (rockchip->irq_domain) that it depends on. > > > > > > But AFAICT, rockchip still has the problem that we don't *unregister* > > > rockchip_pcie_legacy_int_handler() when the rockchip-pcie module is > > > removed. Doesn't this mean that if we unload the module, then receive > > > an interrupt from the device, we'll try to call a function that is no > > > longer present? > > > > Good question, I don't to be honest. I'll have to dig deeper on this but > > my experience is that the module removal (and device unbind) is not that > > well tested on ARM device drivers in general. > > Well, it does use devm_request_irq() so the handler should be unregistered > by devres *after* ->remove has finished, however that does still leave a > potential race window in which a pending IRQ could be taken during the later > part of rockchip_pcie_remove() after it has started turning off critical > things. Unless the clocks and regulators can also be delegated to devres, it > might be more robust to explicitly manage the IRQs as well. Mixing the two > schemes can be problematic when the exact order of both setup and teardown > matters. I don't understand the devm_request_irq() connection. I'm looking at this irq_set_chained_handler_and_data() call [1]: static int rockchip_pcie_setup_irq(struct rockchip_pcie *rockchip) { ... irq = platform_get_irq_byname(pdev, "legacy"); irq_set_chained_handler_and_data(irq, rockchip_pcie_legacy_int_handler, rockchip); irq = platform_get_irq_byname(pdev, "client"); ... We look up "irq", pass it to irq_set_chained_handler_and_data(), and throw it away without saving it anywhere. How would anything know how to unregister rockchip_pcie_legacy_int_handler()? I could imagine irq_set_chained_handler_and_data() saving what's needed for unregistration, but I would think that would require a device pointer, which we don't give it. I'm IRQ-illiterate, so please educate me! Bjorn [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pcie-rockchip-host.c?id=v5.13#n562