* [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] @ 2020-02-06 3:17 Sam Bobroff 2020-02-06 4:23 ` Alexey Kardashevskiy 0 siblings, 1 reply; 4+ messages in thread From: Sam Bobroff @ 2020-02-06 3:17 UTC (permalink / raw) To: kvm, linuxppc-dev; +Cc: aik Older versions of skiboot only provide a single value in the device tree property "ibm,mmio-atsd", even when multiple Address Translation Shoot Down (ATSD) registers are present. This prevents NVLink2 devices (other than the first) from being used with vfio-pci because vfio-pci expects to be able to assign a dedicated ATSD register to each NVLink2 device. However, ATSD registers can be shared among devices. This change allows vfio-pci to fall back to sharing the register at index 0 if necessary. Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com> --- drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c index f2983f0f84be..851ba673882b 100644 --- a/drivers/vfio/pci/vfio_pci_nvlink2.c +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev) if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index, &mmio_atsd)) { - dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); - mmio_atsd = 0; + dev_warn(&vdev->pdev->dev, + "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n", + nvlink_index); + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0, + &mmio_atsd)) { + dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); + mmio_atsd = 0; + } else { + dev_warn(&vdev->pdev->dev, + "Using fallback ibm,mmio-atsd[0] for ATSD.\n"); + } } if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) { -- 2.22.0.216.g00a2a96fc9 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] 2020-02-06 3:17 [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] Sam Bobroff @ 2020-02-06 4:23 ` Alexey Kardashevskiy 2020-02-07 2:39 ` Sam Bobroff 0 siblings, 1 reply; 4+ messages in thread From: Alexey Kardashevskiy @ 2020-02-06 4:23 UTC (permalink / raw) To: Sam Bobroff, kvm, linuxppc-dev, Alex Williamson On 06/02/2020 14:17, Sam Bobroff wrote: > Older versions of skiboot only provide a single value in the device > tree property "ibm,mmio-atsd", even when multiple Address Translation > Shoot Down (ATSD) registers are present. This prevents NVLink2 devices > (other than the first) from being used with vfio-pci because vfio-pci > expects to be able to assign a dedicated ATSD register to each NVLink2 > device. > > However, ATSD registers can be shared among devices. This change > allows vfio-pci to fall back to sharing the register at index 0 if > necessary. > > Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com> > --- > drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++-- > 1 file changed, 11 insertions(+), 2 deletions(-) > > diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c > index f2983f0f84be..851ba673882b 100644 > --- a/drivers/vfio/pci/vfio_pci_nvlink2.c > +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c > @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev) > > if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index, > &mmio_atsd)) { > - dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); > - mmio_atsd = 0; > + dev_warn(&vdev->pdev->dev, > + "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n", > + nvlink_index); We do not really need this warning (nvlink_index doesn't matter that much, we can work out from the device tree what happened), warnings below are enough (if you really want, you can print nvlink_index there). Either way, Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru> > + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0, > + &mmio_atsd)) { > + dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); > + mmio_atsd = 0; > + } else { > + dev_warn(&vdev->pdev->dev, > + "Using fallback ibm,mmio-atsd[0] for ATSD.\n"); > + } > } > > if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) { > -- Alexey ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] 2020-02-06 4:23 ` Alexey Kardashevskiy @ 2020-02-07 2:39 ` Sam Bobroff 2020-02-07 4:43 ` Sam Bobroff 0 siblings, 1 reply; 4+ messages in thread From: Sam Bobroff @ 2020-02-07 2:39 UTC (permalink / raw) To: Alexey Kardashevskiy; +Cc: Alex Williamson, linuxppc-dev, kvm [-- Attachment #1: Type: text/plain, Size: 2363 bytes --] On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote: > > > On 06/02/2020 14:17, Sam Bobroff wrote: > > Older versions of skiboot only provide a single value in the device > > tree property "ibm,mmio-atsd", even when multiple Address Translation > > Shoot Down (ATSD) registers are present. This prevents NVLink2 devices > > (other than the first) from being used with vfio-pci because vfio-pci > > expects to be able to assign a dedicated ATSD register to each NVLink2 > > device. > > > > However, ATSD registers can be shared among devices. This change > > allows vfio-pci to fall back to sharing the register at index 0 if > > necessary. > > > > Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com> > > --- > > drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++-- > > 1 file changed, 11 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c > > index f2983f0f84be..851ba673882b 100644 > > --- a/drivers/vfio/pci/vfio_pci_nvlink2.c > > +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c > > @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev) > > > > if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index, > > &mmio_atsd)) { > > - dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); > > - mmio_atsd = 0; > > + dev_warn(&vdev->pdev->dev, > > + "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n", > > + nvlink_index); > > > We do not really need this warning (nvlink_index doesn't matter that > much, we can work out from the device tree what happened), warnings > below are enough (if you really want, you can print nvlink_index there). > > Either way, > > Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru> > Thanks, I'll change it if there's some reason to do another version but otherwise leave it as is. Sam. > > > > > + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0, > > + &mmio_atsd)) { > > + dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); > > + mmio_atsd = 0; > > + } else { > > + dev_warn(&vdev->pdev->dev, > > + "Using fallback ibm,mmio-atsd[0] for ATSD.\n"); > > + } > > } > > > > if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) { > > > > -- > Alexey [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] 2020-02-07 2:39 ` Sam Bobroff @ 2020-02-07 4:43 ` Sam Bobroff 0 siblings, 0 replies; 4+ messages in thread From: Sam Bobroff @ 2020-02-07 4:43 UTC (permalink / raw) To: Alexey Kardashevskiy; +Cc: Alex Williamson, linuxppc-dev, kvm [-- Attachment #1: Type: text/plain, Size: 2710 bytes --] On Fri, Feb 07, 2020 at 01:39:14PM +1100, Sam Bobroff wrote: > On Thu, Feb 06, 2020 at 03:23:03PM +1100, Alexey Kardashevskiy wrote: > > > > > > On 06/02/2020 14:17, Sam Bobroff wrote: > > > Older versions of skiboot only provide a single value in the device > > > tree property "ibm,mmio-atsd", even when multiple Address Translation > > > Shoot Down (ATSD) registers are present. This prevents NVLink2 devices > > > (other than the first) from being used with vfio-pci because vfio-pci > > > expects to be able to assign a dedicated ATSD register to each NVLink2 > > > device. > > > > > > However, ATSD registers can be shared among devices. This change > > > allows vfio-pci to fall back to sharing the register at index 0 if > > > necessary. > > > > > > Signed-off-by: Sam Bobroff <sbobroff@linux.ibm.com> > > > --- > > > drivers/vfio/pci/vfio_pci_nvlink2.c | 13 +++++++++++-- > > > 1 file changed, 11 insertions(+), 2 deletions(-) > > > > > > diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c > > > index f2983f0f84be..851ba673882b 100644 > > > --- a/drivers/vfio/pci/vfio_pci_nvlink2.c > > > +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c > > > @@ -420,8 +420,17 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev) > > > > > > if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index, > > > &mmio_atsd)) { > > > - dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); > > > - mmio_atsd = 0; > > > + dev_warn(&vdev->pdev->dev, > > > + "No ibm,mmio-atsd[%d] found: trying ibm,mmio-atsd[0]\n", > > > + nvlink_index); > > > > > > We do not really need this warning (nvlink_index doesn't matter that > > much, we can work out from the device tree what happened), warnings > > below are enough (if you really want, you can print nvlink_index there). > > > > Either way, > > > > Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru> > > > > Thanks, > > I'll change it if there's some reason to do another version but > otherwise leave it as is. > > Sam. Oh! I almost forgot: maybe this should carry a fixes tag? Fixes: 7f92891778df ("vfio_pci: Add NVIDIA GV100GL [Tesla V100 SXM2] subdriver") > > > > > > > > > + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0, > > > + &mmio_atsd)) { > > > + dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); > > > + mmio_atsd = 0; > > > + } else { > > > + dev_warn(&vdev->pdev->dev, > > > + "Using fallback ibm,mmio-atsd[0] for ATSD.\n"); > > > + } > > > } > > > > > > if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) { > > > > > > > -- > > Alexey [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-02-07 4:44 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-02-06 3:17 [PATCH 1/1] vfio-pci/nvlink2: Allow fallback to ibm,mmio-atsd[0] Sam Bobroff 2020-02-06 4:23 ` Alexey Kardashevskiy 2020-02-07 2:39 ` Sam Bobroff 2020-02-07 4:43 ` Sam Bobroff
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).