From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C1E6C4360F for ; Thu, 4 Apr 2019 21:30:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 58CBB21741 for ; Thu, 4 Apr 2019 21:30:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="OhTUw/ig" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729657AbfDDVar (ORCPT ); Thu, 4 Apr 2019 17:30:47 -0400 Received: from hqemgate15.nvidia.com ([216.228.121.64]:18414 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729501AbfDDVar (ORCPT ); Thu, 4 Apr 2019 17:30:47 -0400 Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Thu, 04 Apr 2019 14:30:33 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate102.nvidia.com (PGP Universal service); Thu, 04 Apr 2019 14:30:44 -0700 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Thu, 04 Apr 2019 14:30:44 -0700 Received: from [10.25.75.5] (172.20.13.39) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 4 Apr 2019 21:30:40 +0000 Subject: Re: [PATCH V2] PCI: tegra: Use the DMA-API to get the MSI address From: Vidya Sagar To: , CC: vidya sagar , , , , , , , , , NManikanta References: <1553004121-24606-1-git-send-email-vidyas@nvidia.com> X-Nvconfidentiality: public Message-ID: <9b6c7289-cc43-4fad-dcb0-97b4413ffd42@nvidia.com> Date: Fri, 5 Apr 2019 03:00:36 +0530 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: X-Originating-IP: [172.20.13.39] X-ClientProxiedBy: HQMAIL106.nvidia.com (172.18.146.12) To HQMAIL101.nvidia.com (172.20.187.10) Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1554413433; bh=qx87rYM84+xToigfohvAPpu6L6yIk1UJX7ckcAv8+r8=; h=X-PGP-Universal:Subject:From:To:CC:References:X-Nvconfidentiality: Message-ID:Date:User-Agent:MIME-Version:In-Reply-To: X-Originating-IP:X-ClientProxiedBy:Content-Type:Content-Language: Content-Transfer-Encoding; b=OhTUw/igy0x7rgP8KxyJZxmk2smxBkZoAGzh96S6M3MgvYqyKYnBc2+WUgZm/622u 799EDt7PPPvQTGaBdO0pwT0kzAaHfI0Xf4+yskEl6Y9TDNAPiL7mpUCmXHZI1ij/Gd fvyrQrQBnswSjhsrTVg1jbb0oNYSAucgySJNTRUmYnqJ3nh1UOq2ROovhnKPfzfC0d LvMYPkbG8cso7Og1ivk+IZKWH2oN+ne99IrhRC/KuAc+Zw8rZ1Y6tG5MZtMSDT8kqI NG5nEeqtotCxJkBDDdSxp95AFq3xbxoG4sCzgxewqHEk8rbIjTBBpO7kRdDJiEQsr0 5pK/KsYtSoPkw== Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On 4/1/2019 11:13 AM, Vidya Sagar wrote: Hi Bjorn / Lorenzo, Apologies for reminding you again. Could you please review this patch? Thanks, Vidya Sagar > Hi Bjorn / Lorenzo, > Can you please review this patch? >=20 > Thanks, > Vidya Sagar >=20 > On 3/27/2019 4:29 PM, vidya sagar wrote: >> Hi Bjorn/Lorenzo, >> Can you please review this patch? >> Thierry has reviewed it and I already took care=C2=A0of=C2=A0his comment= s. >> >> Thanks, >> Vidya Sagar >> >> On Tue, Mar 19, 2019 at 7:33 PM Vidya Sagar > wrote: >> >> =C2=A0=C2=A0=C2=A0 Since the upstream MSI memory writes are generated by= downstream devices, >> =C2=A0=C2=A0=C2=A0 it is logically correct to have MSI target memory com= ing from the DMA pool >> =C2=A0=C2=A0=C2=A0 reserved for PCIe than from the general memory pool r= eserved for CPU >> =C2=A0=C2=A0=C2=A0 access. This avoids PCIe DMA addresses coinciding wit= h MSI target address >> =C2=A0=C2=A0=C2=A0 thereby raising unwanted MSI interrupts. This patch a= lso enforces to limit >> =C2=A0=C2=A0=C2=A0 the MSI target address to 32-bits to make it work for= PCIe endponits that >> =C2=A0=C2=A0=C2=A0 support only 32-bit MSI target address and those endp= oints that support >> =C2=A0=C2=A0=C2=A0 64-bit MSI target address anyway work with 32-bit MSI= target address. >> >> =C2=A0=C2=A0=C2=A0 Signed-off-by: Vidya Sagar > >> =C2=A0=C2=A0=C2=A0 Reviewed-by: Thierry Reding > >> =C2=A0=C2=A0=C2=A0 Acked-by: Thierry Reding > >> =C2=A0=C2=A0=C2=A0 --- >> =C2=A0=C2=A0=C2=A0 v2: >> =C2=A0=C2=A0=C2=A0 * changed 'phys' type to 'dma_addr_t' from 'u64' >> =C2=A0=C2=A0=C2=A0 * added a comment on why DMA mask is set to 32-bit >> =C2=A0=C2=A0=C2=A0 * replaced 'dma' with 'DMA' >> >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0drivers/pci/controller/pci-tegra.c | 35 += +++++++++++++++++++++++++--------- >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A01 file changed, 26 insertions(+), 9 delet= ions(-) >> >> =C2=A0=C2=A0=C2=A0 diff --git a/drivers/pci/controller/pci-tegra.c b/dri= vers/pci/controller/pci-tegra.c >> =C2=A0=C2=A0=C2=A0 index f4f53d092e00..f8173a5e352d 100644 >> =C2=A0=C2=A0=C2=A0 --- a/drivers/pci/controller/pci-tegra.c >> =C2=A0=C2=A0=C2=A0 +++ b/drivers/pci/controller/pci-tegra.c >> =C2=A0=C2=A0=C2=A0 @@ -231,9 +231,9 @@ struct tegra_msi { >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct msi_controll= er chip; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 DECLARE_BITMAP(used= , INT_PCI_MSI_NR); >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct irq_domain *= domain; >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0unsigned long pages; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct mutex lock; >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0u64 phys; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0void *virt; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0dma_addr_t phys; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 int irq; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0}; >> >> =C2=A0=C2=A0=C2=A0 @@ -1536,7 +1536,7 @@ static int tegra_pcie_msi_setup= (struct tegra_pcie *pcie) >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 err =3D platform_ge= t_irq_byname(pdev, "msi"); >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (err < 0) { >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 dev_err(dev, "failed to get IRQ: %d\n", err); >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0goto err; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0goto free_irq_domain; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } >> >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 msi->irq =3D err; >> =C2=A0=C2=A0=C2=A0 @@ -1545,17 +1545,34 @@ static int tegra_pcie_msi_set= up(struct tegra_pcie *pcie) >> =C2=A0=C2=A0=C2=A0 tegra_msi_irq_chip.name , pcie); >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (err < 0) { >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 dev_err(dev, "failed to request IRQ: %d\n", err); >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0goto err; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0goto free_irq_domain; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0} >> =C2=A0=C2=A0=C2=A0 + >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Though the PCIe contro= ller can address >32-bit address space, to >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * facilitate endpoints t= hat support only 32-bit MSI target address, >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * the mask is set to 32-= bit to make sure that MSI target address is >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 * always a 32-bit addres= s >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 */ >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0err =3D dma_set_coherent_= mask(dev, DMA_BIT_MASK(32)); >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0if (err < 0) { >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0dev_err(dev, "failed to set DMA coherent mask: %d\n", err); >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0goto free_irq; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0} >> =C2=A0=C2=A0=C2=A0 + >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0msi->virt =3D dma_alloc_c= oherent(dev, PAGE_SIZE, &msi->phys, GFP_KERNEL); >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!msi->virt) { >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0dev_err(dev, "failed to allocate DMA memory for MSI\n"); >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0err =3D -ENOMEM; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0goto free_irq; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 } >> >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0/* setup AFI/FPCI range *= / >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0msi->pages =3D __get_free= _pages(GFP_KERNEL, 0); >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0msi->phys =3D virt_to_phy= s((void *)msi->pages); >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 host->msi =3D &msi-= >chip; >> >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return 0; >> >> =C2=A0=C2=A0=C2=A0 -err: >> =C2=A0=C2=A0=C2=A0 +free_irq: >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0free_irq(msi->irq, pcie); >> =C2=A0=C2=A0=C2=A0 +free_irq_domain: >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 irq_domain_remove(m= si->domain); >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return err; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0} >> =C2=A0=C2=A0=C2=A0 @@ -1592,7 +1609,7 @@ static void tegra_pcie_msi_tear= down(struct tegra_pcie *pcie) >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 struct tegra_msi *m= si =3D &pcie->msi; >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 unsigned int i, irq= ; >> >> =C2=A0=C2=A0=C2=A0 -=C2=A0 =C2=A0 =C2=A0 =C2=A0free_pages(msi->pages, 0)= ; >> =C2=A0=C2=A0=C2=A0 +=C2=A0 =C2=A0 =C2=A0 =C2=A0dma_free_coherent(pcie->d= ev, PAGE_SIZE, msi->virt, msi->phys); >> >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (msi->irq > 0) >> =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 free_irq(msi->irq, pcie); >> =C2=A0=C2=A0=C2=A0 -- =C2=A0=C2=A0=C2=A0 2.7.4 >> >=20