From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from e23smtp09.au.ibm.com (e23smtp09.au.ibm.com [202.81.31.142]) (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 8FAEA1A075F for ; Fri, 17 Jul 2015 10:29:40 +1000 (AEST) Received: from /spool/local by e23smtp09.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 17 Jul 2015 10:29:38 +1000 Received: from d23relay09.au.ibm.com (d23relay09.au.ibm.com [9.185.63.181]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id EDEFE2BB0047 for ; Fri, 17 Jul 2015 10:29:35 +1000 (EST) Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay09.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t6H0TR9j65142950 for ; Fri, 17 Jul 2015 10:29:35 +1000 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t6H0T3gD019982 for ; Fri, 17 Jul 2015 10:29:03 +1000 Date: Fri, 17 Jul 2015 10:28:39 +1000 From: Gavin Shan To: Gavin Shan Cc: linuxppc-dev@lists.ozlabs.org, benh@kernel.crashing.org, paulus@samba.org, aik@ozlabs.ru Subject: Re: [PATCH 2/2] powerpc/powernv: Double VF BAR size for compound PE Message-ID: <20150717002838.GA22191@gwshan> Reply-To: Gavin Shan References: <1437092083-20672-1-git-send-email-gwshan@linux.vnet.ibm.com> <1437092083-20672-3-git-send-email-gwshan@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <1437092083-20672-3-git-send-email-gwshan@linux.vnet.ibm.com> List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Fri, Jul 17, 2015 at 10:14:43AM +1000, Gavin Shan wrote: >When VF BAR size is equal to 128MB or bigger than that, we extend >the corresponding PF's IOV BAR to cover number of total VFs supported >by the PF. Otherwise, we extend the PF's IOV BAR to cover 256 VFs. >For the former case, we have to create compound PE, which includes >4 VFs. Those 4 VFs included in the compound PE can't be passed through >to different guests, which isn't good. > >The gate (128MB) was choosen based on the assumption that each PHB >supports 64GB M64 space and one PF's IOV BAR can be extended to be >as huge as 1/4 of that, which is 16GB. However, the IOV BAR can be >extended to half of PHB's M64 window when the PF seats behind the >root port. In that case, the gate can be enlarged to be 256MB to >avoid compound PE as we can. > >Signed-off-by: Gavin Shan >--- > arch/powerpc/platforms/powernv/pci-ioda.c | 21 ++++++++++++++++----- > 1 file changed, 16 insertions(+), 5 deletions(-) > >diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c >index 6ec62b9..5b2e88f 100644 >--- a/arch/powerpc/platforms/powernv/pci-ioda.c >+++ b/arch/powerpc/platforms/powernv/pci-ioda.c >@@ -2721,6 +2721,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) > struct resource *res; > int i; > resource_size_t size; >+ resource_size_t limit; > struct pci_dn *pdn; > int mul, total_vfs; > >@@ -2730,6 +2731,18 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) > hose = pci_bus_to_host(pdev->bus); > phb = hose->private_data; > >+ /* >+ * When the PF seats behind root port, the IOV BAR can >+ * consume half of the PHB's M64 window. Otherwise, >+ * 1/4 of the PHB's M64 window can be consumed to the >+ * maximal degree. >+ */ >+ if (!pci_is_root_bus(pdev->bus) && >+ pci_is_root_bus(pdev->bus->self->bus)) >+ limit = 128; >+ else >+ limit = 256; >+ I sent it too fast. The limit should be reversed: 256 when PF seats behind the root port. Otherwise, it should be 128. I will send follow-up v2 after waiting for couple of days in case there are some comments for this revision. > pdn = pci_get_pdn(pdev); > pdn->vfs_expanded = 0; > >@@ -2748,11 +2761,9 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) > } > > size = pci_iov_resource_size(pdev, i + PCI_IOV_RESOURCES); >- >- /* bigger than 64M */ >- if (size > (1 << 26)) { >- dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than 64M, roundup power2\n", >- i, res); >+ if (size >= (limit * 0x100000)) { >+ dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than %lldMB, roundup power2\n", >+ i, res, limit); > pdn->m64_per_iov = M64_PER_IOV; > mul = roundup_pow_of_two(total_vfs); > break; Thanks, Gavin