From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE02AC43387 for ; Thu, 3 Jan 2019 17:36:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C0018208E3 for ; Thu, 3 Jan 2019 17:36:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731097AbfACRgF (ORCPT ); Thu, 3 Jan 2019 12:36:05 -0500 Received: from mga11.intel.com ([192.55.52.93]:7052 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730250AbfACRgF (ORCPT ); Thu, 3 Jan 2019 12:36:05 -0500 X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Jan 2019 09:36:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,435,1539673200"; d="scan'208";a="123033583" Received: from smile.fi.intel.com (HELO smile) ([10.237.72.86]) by orsmga002.jf.intel.com with ESMTP; 03 Jan 2019 09:36:00 -0800 Received: from andy by smile with local (Exim 4.92-RC3) (envelope-from ) id 1gf6uV-00039S-4c; Thu, 03 Jan 2019 19:35:59 +0200 Date: Thu, 3 Jan 2019 19:35:59 +0200 From: Andy Shevchenko To: Gustavo Pimentel Cc: "linux-pci@vger.kernel.org" , "dmaengine@vger.kernel.org" , Vinod Koul , Eugeniy Paltsev , Lorenzo Pieralisi , Joao Pinto Subject: Re: [RFC 4/6] dma: Add Synopsys eDMA IP PCIe glue-logic Message-ID: <20190103173559.GW10650@smile.fi.intel.com> References: <20181212132528.GT10650@smile.fi.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Thu, Dec 13, 2018 at 02:40:40PM +0000, Gustavo Pimentel wrote: > On 12/12/2018 13:25, Andy Shevchenko wrote: > > On Wed, Dec 12, 2018 at 12:13:24PM +0100, Gustavo Pimentel wrote: > > It seems this driver somehow written as a copy-paste of existing pieces w/o good reasons to do such. > (1) > What do you mean? It's true I relied on existing drivers to develop this > solution, but is not it supposed to be so? See below. > >> +enum dw_edma_pcie_bar { > >> + BAR_0, > >> + BAR_1, > >> + BAR_2, > >> + BAR_3, > >> + BAR_4, > >> + BAR_5 > >> +}; > > > > Why do you need this at all? > > See my answer below. > > > > >> +static const struct dw_edma_pcie_data snps_edda_data = { > >> + // eDMA registers location > >> + .regs_bar = BAR_0, > >> + .regs_off = 0x1000, // 4 KBytes > >> + // eDMA memory linked list location > >> + .ll_bar = BAR_2, > >> + .ll_off = 0, // 0 KBytes > >> + .ll_sz = 0x20000, // 128 KBytes > >> + // Other > >> + .version = 0, > >> + .mode = EDMA_MODE_UNROLL, > >> +}; > > > > Huh? Isn't this > > The eDMA is a hardware block (with requires some configuration) accessible > through a PCIe EndPoint, unfortunately there isn't any way for now (at least) > through an PCIe capability for instance to detect those configurations > automatically, therefore the only way to pass that configuration is to associate > it to PCIe Vendor(0x16c3) and Device (0xedda) Id. > > To interact with eDMA hardware block the driver needs to access the eDMA > registers and the only way to do it is through PCIe mapping. In this those > registers will be available on BAR 0, with a offset of 4KB from the start. > > The driver also requires a memory space (RAM) to create a linked list > descriptors and pass the first descriptor address to the eDMA hardware block so > that it can start performing the transfer autonomously. Once again this memory > space is provide through PCIe on BAR 2. This linked list space is in the > beginning and it's restricted to a size of 128KB. > > Since this eDMA hardware block (more specifically eDMA registers) can evolve in > the future, I choosed to put here a version in order to have a driver that can > be easily adaptable and reusable without much effort. (2) > >> + err = pci_try_set_mwi(pdev); > >> + if (err) { > >> + dev_err(dev, "%s DMA memory write invalidate\n", > >> + pci_name(pdev)); > >> + return err; > >> + } > > > > Are you sure you need this? > > I'm not sure, probably not. I saw using this being use on /dma/dw/pci.c driver. > I'll test without it. That's exactly the point what I referred to in (1) above. Looks like here is cargo-cult programming done without understanding why. I dunno how many other cases in the driver like this. > >> + dev_info(dev, "Mode:\t%s\n", > >> + dw->mode == EDMA_MODE_LEGACY ? "Legacy" : "Unroll"); > >> + > > > > > >> + dev_info(dev, "Registers:\tBAR=%u, off=0x%.16llx B, addr=0x%.8lx\n", > >> + pdata->regs_bar, pdata->regs_off, > >> + (unsigned long) dw->regs); > > > > Oh, no, don't do casting when printing something. In only rare cases it's needed, not here. > > I've put the cast to remove the following warning: > > drivers/dma/dw-edma/dw-edma-pcie.c:138:15: warning: format ‘%lx’ expects > argument of type ‘long unsigned int’, but argument 6 has type ‘void *’ ...and as I said this is wrong. Please, find suitable specifiers for your case. > >> + if (pdev->msi_cap && pdev->msi_enabled) { > >> + cap_off = pdev->msi_cap + PCI_MSI_FLAGS; > >> + pci_read_config_word(pdev, cap_off, &flags); > >> + if (flags & PCI_MSI_FLAGS_ENABLE) { > >> + cap_off = pdev->msi_cap + PCI_MSI_ADDRESS_LO; > >> + pci_read_config_dword(pdev, cap_off, &addr_lo); > >> + > >> + if (flags & PCI_MSI_FLAGS_64BIT) { > >> + cap_off = pdev->msi_cap + PCI_MSI_ADDRESS_HI; > >> + pci_read_config_dword(pdev, cap_off, &addr_hi); > >> + cap_off = pdev->msi_cap + PCI_MSI_DATA_64; > >> + } else { > >> + addr_hi = 0; > >> + cap_off = pdev->msi_cap + PCI_MSI_DATA_32; > >> + } > >> + > >> + dw->msi_addr = addr_hi; > >> + dw->msi_addr <<= 32; > >> + dw->msi_addr |= addr_lo; > >> + > >> + pci_read_config_dword(pdev, cap_off, &(dw->msi_data)); > >> + dw->msi_data &= 0xffff; > >> + > >> + dev_info(dev, > >> + "MSI:\t\taddr=0x%.16llx, data=0x%.8x, nr=%d\n", > >> + dw->msi_addr, dw->msi_data, pdev->irq); > >> + } > >> + } > >> + > >> + if (pdev->msix_cap && pdev->msix_enabled) { > >> + u32 offset; > >> + u8 bir; > >> + > >> + cap_off = pdev->msix_cap + PCI_MSIX_FLAGS; > >> + pci_read_config_word(pdev, cap_off, &flags); > >> + > >> + if (flags & PCI_MSIX_FLAGS_ENABLE) { > >> + cap_off = pdev->msix_cap + PCI_MSIX_TABLE; > >> + pci_read_config_dword(pdev, cap_off, &offset); > >> + > >> + bir = offset & PCI_MSIX_TABLE_BIR; > >> + offset &= PCI_MSIX_TABLE_OFFSET; > >> + > >> + reg = pcim_iomap_table(pdev)[bir]; > >> + reg += offset; > >> + > >> + addr_lo = readl(reg + PCI_MSIX_ENTRY_LOWER_ADDR); > >> + addr_hi = readl(reg + PCI_MSIX_ENTRY_UPPER_ADDR); > >> + dw->msi_addr = addr_hi; > >> + dw->msi_addr <<= 32; > >> + dw->msi_addr |= addr_lo; > >> + > >> + dw->msi_data = readl(reg + PCI_MSIX_ENTRY_DATA); > >> + > >> + dev_info(dev, > >> + "MSI-X:\taddr=0x%.16llx, data=0x%.8x, nr=%d\n", > >> + dw->msi_addr, dw->msi_data, pdev->irq); > >> + } > >> + } > > > > What is this? Why? > > I need to find the PCIe interrupt vector address and value (either for MSI or > MSI-X, depends what the system has selected) in order to configure eDMA later to > generate the (done or abort) interruptions. It seems rather hackish way to do something which should be done by PCI/driver core. But perhaps Bjorn and others familiar with PCI endpoint facility can share more. Also this applies to (2) above. > >> + if (!pdev->msi_enabled && !pdev->msix_enabled) { > > I'm not finding it. Can you tell me what it is? > > Note: pci_msi_enabled() on msi.c returns if MSI has not been disabled by the > command-line option pci=nomsi Can you look a bit more carefully? For example, assume that 'dev' (sub)prefix in the name of function would tell something about scope of application. pci_dev_...() ? > > There is a helper from PCI core for this. -- With Best Regards, Andy Shevchenko