From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl0-x244.google.com (mail-pl0-x244.google.com [IPv6:2607:f8b0:400e:c01::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 418F4d44zDzDrbK for ; Mon, 18 Jun 2018 12:13:19 +1000 (AEST) Received: by mail-pl0-x244.google.com with SMTP id f1-v6so8152418plt.6 for ; Sun, 17 Jun 2018 19:13:19 -0700 (PDT) Date: Mon, 18 Jun 2018 12:13:07 +1000 From: Alexey Kardashevskiy To: Benjamin Herrenschmidt Cc: linuxppc-dev@lists.ozlabs.org, Alistair Popple , Russell Currey Subject: Re: [PATCH kernel 2/2] powerpc/powernv: Define PHB4 type and enable sketchy bypass on POWER9 Message-ID: <20180618121307.199cd998@aik.ozlabs.ibm.com> In-Reply-To: References: <20180601081028.29401-1-aik@ozlabs.ru> <20180601081028.29401-3-aik@ozlabs.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Sat, 16 Jun 2018 11:05:19 +1000 Benjamin Herrenschmidt wrote: > On Fri, 2018-06-01 at 18:10 +1000, Alexey Kardashevskiy wrote: > > These are found in POWER9 chips. Right now these PHBs have unknown type > > so changing it to PHB4 won't make much of a difference except enabling > > sketchy bypass for POWER9 as this does below. > > And that will break on multi-chip systems since P9 doesn't have the > memory contiguous (it has the chip ID in the top bits). This did not break mine and it is hard to see why it would break at all if we use 1G pages and the maximum we need to cover is 48 bits (this is what we are trying to support here - all these gpus, right?), or is it more now? If so, I have posted v2 of tce multilevel dynamic allocation which helps with enormous tce tables. > > Russell is working on a different implementation that should be much > more imune to the system physical memory layout. > > > Signed-off-by: Alexey Kardashevskiy > > --- > > arch/powerpc/platforms/powernv/pci.h | 1 + > > arch/powerpc/platforms/powernv/pci-ioda.c | 5 ++++- > > 2 files changed, 5 insertions(+), 1 deletion(-) > > > > diff --git a/arch/powerpc/platforms/powernv/pci.h b/arch/powerpc/platforms/powernv/pci.h > > index eada4b6..1408247 100644 > > --- a/arch/powerpc/platforms/powernv/pci.h > > +++ b/arch/powerpc/platforms/powernv/pci.h > > @@ -23,6 +23,7 @@ enum pnv_phb_model { > > PNV_PHB_MODEL_UNKNOWN, > > PNV_PHB_MODEL_P7IOC, > > PNV_PHB_MODEL_PHB3, > > + PNV_PHB_MODEL_PHB4, > > PNV_PHB_MODEL_NPU, > > PNV_PHB_MODEL_NPU2, > > }; > > diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c > > index 9239142..66c2804 100644 > > --- a/arch/powerpc/platforms/powernv/pci-ioda.c > > +++ b/arch/powerpc/platforms/powernv/pci-ioda.c > > @@ -1882,7 +1882,8 @@ static int pnv_pci_ioda_dma_set_mask(struct pci_dev *pdev, u64 dma_mask) > > if (dma_mask >> 32 && > > dma_mask > (memory_hotplug_max() + (1ULL << 32)) && > > pnv_pci_ioda_pe_single_vendor(pe) && > > - phb->model == PNV_PHB_MODEL_PHB3) { > > + (phb->model == PNV_PHB_MODEL_PHB3 || > > + phb->model == PNV_PHB_MODEL_PHB4)) { > > /* Configure the bypass mode */ > > rc = pnv_pci_ioda_dma_64bit_bypass(pe); > > if (rc) > > @@ -3930,6 +3931,8 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np, > > phb->model = PNV_PHB_MODEL_P7IOC; > > else if (of_device_is_compatible(np, "ibm,power8-pciex")) > > phb->model = PNV_PHB_MODEL_PHB3; > > + else if (of_device_is_compatible(np, "ibm,power9-pciex")) > > + phb->model = PNV_PHB_MODEL_PHB4; > > else if (of_device_is_compatible(np, "ibm,power8-npu-pciex")) > > phb->model = PNV_PHB_MODEL_NPU; > > else if (of_device_is_compatible(np, "ibm,power9-npu-pciex")) -- Alexey