From: Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
To: "Christian König" <christian.koenig-5C7GfCeVMHo@public.gmane.org>,
"Bjorn Helgaas" <helgaas-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
"Alex Williamson"
<alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: "Jens Axboe" <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>,
"Keith Busch"
<keith.busch-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>,
linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org,
linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
"Jérôme Glisse" <jglisse-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
"Jason Gunthorpe" <jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>,
"Benjamin Herrenschmidt"
<benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org>,
"Bjorn Helgaas"
<bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
"Max Gurtovoy" <maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>,
"Christoph Hellwig" <hch-jcswGhMUV9g@public.gmane.org>
Subject: Re: [PATCH v4 04/14] PCI/P2PDMA: Clear ACS P2P flags for all devices behind switches
Date: Tue, 8 May 2018 10:27:56 -0600 [thread overview]
Message-ID: <0b4183ef-e720-204b-9e85-b9eaf7a4136a@deltatee.com> (raw)
In-Reply-To: <e58e8d27-42a5-ca69-3b68-c409a2922cd1-5C7GfCeVMHo@public.gmane.org>
On 08/05/18 01:17 AM, Christian König wrote:
> AMD APUs mandatory need the ACS flag set for the GPU integrated in the
> CPU when IOMMU is enabled or otherwise you will break SVM.
Well, given that the current set only disables ACS bits on bridges
(previous versions were only on switches) this shouldn't be an issue for
integrated devices. We do not disable ACS flags globally.
> And what exactly is the problem here? I'm currently testing P2P with
> GPUs in different IOMMU domains and at least with AMD IOMMUs that works
> perfectly fine.
In addition to Stephen's comments, seeing we've established a general
need to avoid the root complex (until we have a whitelist at least) we
must have ACS disabled along the path between the devices. Otherwise,
all TLPs will go through the root complex and if there is no support it
will fail.
If the consensus is we want a command line option, then so be it. But
we'll have to deny pretty much all P2P transactions unless the user
correctly disables ACS along the path using the command line option and
this is really annoying for users of this functionality to understand
how to do that correctly.
Logan
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
next prev parent reply other threads:[~2018-05-08 16:27 UTC|newest]
Thread overview: 103+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-23 23:30 [PATCH v4 00/14] Copy Offload in NVMe Fabrics with P2P PCI Memory Logan Gunthorpe
[not found] ` <20180423233046.21476-1-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-04-23 23:30 ` [PATCH v4 01/14] PCI/P2PDMA: Support peer-to-peer memory Logan Gunthorpe
[not found] ` <20180423233046.21476-2-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-07 23:00 ` Bjorn Helgaas
[not found] ` <20180507230034.GE161390-1RhO1Y9PlrlHTL0Zs8A6p5iNqAH0jzoTYJqu5kTmcBRl57MIdRCFDg@public.gmane.org>
2018-05-07 23:09 ` Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 02/14] PCI/P2PDMA: Add sysfs group to display p2pmem stats Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 03/14] PCI/P2PDMA: Add PCI p2pmem dma mappings to adjust the bus offset Logan Gunthorpe
[not found] ` <20180423233046.21476-4-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-07 23:02 ` Bjorn Helgaas
2018-04-23 23:30 ` [PATCH v4 04/14] PCI/P2PDMA: Clear ACS P2P flags for all devices behind switches Logan Gunthorpe
[not found] ` <20180423233046.21476-5-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-04-24 3:33 ` Randy Dunlap
2018-05-07 23:13 ` Bjorn Helgaas
[not found] ` <20180507231306.GG161390-1RhO1Y9PlrlHTL0Zs8A6p5iNqAH0jzoTYJqu5kTmcBRl57MIdRCFDg@public.gmane.org>
2018-05-08 7:17 ` Christian König
[not found] ` <e58e8d27-42a5-ca69-3b68-c409a2922cd1-5C7GfCeVMHo@public.gmane.org>
2018-05-08 14:25 ` Stephen Bates
[not found] ` <479F2F93-903B-4A8A-8989-A4BD0E5E1471-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-08 16:37 ` Christian König
2018-05-08 16:27 ` Logan Gunthorpe [this message]
[not found] ` <0b4183ef-e720-204b-9e85-b9eaf7a4136a-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-08 16:50 ` Christian König
[not found] ` <a943a92d-15cd-fb2d-21c3-5e5cfccd9eff-5C7GfCeVMHo@public.gmane.org>
2018-05-08 19:13 ` Logan Gunthorpe
2018-05-08 19:34 ` Alex Williamson
[not found] ` <20180508133407.57a46902-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 19:45 ` Logan Gunthorpe
[not found] ` <5fc9b1c1-9208-06cc-0ec5-1f54c2520494-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-08 20:13 ` Alex Williamson
[not found] ` <20180508141331.7cd737cb-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 20:19 ` Logan Gunthorpe
[not found] ` <d70cf286-f7b7-99a5-1042-022506298fed-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-08 20:43 ` Alex Williamson
[not found] ` <20180508144341.0441b676-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 20:49 ` Logan Gunthorpe
[not found] ` <a524127a-3557-3dff-971c-dfe94ba5a01c-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-08 21:26 ` Alex Williamson
[not found] ` <20180508152631.50fd583c-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 21:42 ` Stephen Bates
[not found] ` <354F7407-0DC7-470C-B9AA-74FDF9C46B08-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-08 22:03 ` Alex Williamson
[not found] ` <20180508160336.0935ddde-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 22:10 ` Logan Gunthorpe
[not found] ` <20905682-9440-7d4b-0260-99d3dc794c3d-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-08 22:25 ` Stephen Bates
[not found] ` <C3CEE4DB-A001-4E4C-B9B9-4B160E4C5367-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-08 23:11 ` Alex Williamson
[not found] ` <20180508171150.0e8fd291-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 23:31 ` Logan Gunthorpe
[not found] ` <0a0a1782-04b2-c571-63c7-bfff68312946-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-09 0:17 ` Alex Williamson
2018-05-08 22:32 ` Alex Williamson
[not found] ` <20180508163206.7d3bf383-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 23:00 ` Dan Williams
[not found] ` <CAPcyv4i1ksiu1pExF=EE21P2Wz1a9LaDaG247oG_WSbYgcR93g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-05-08 23:15 ` Logan Gunthorpe
[not found] ` <2ce7908d-1422-55f8-aeba-94ccce2cb484-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-09 12:38 ` Stephen Bates
2018-05-08 22:21 ` Don Dutile
[not found] ` <11819753-af26-4632-8580-d4a47127a3b2-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-09 12:44 ` Stephen Bates
[not found] ` <C6D0BD3A-FA82-425A-899C-C2DBDCBC7EEC-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-09 15:58 ` Don Dutile
2018-05-08 20:50 ` Jerome Glisse
[not found] ` <20180508205005.GC15608-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-08 21:35 ` Stephen Bates
2018-05-09 13:12 ` Stephen Bates
[not found] ` <7FFB9603-DF9F-4441-82E9-46037CB6C0DE-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-09 13:40 ` Christian König
[not found] ` <a1a36f88-fbfb-b5ee-6dcb-5c5bb96132dd-5C7GfCeVMHo@public.gmane.org>
2018-05-09 15:41 ` Stephen Bates
[not found] ` <1775CC56-4651-422F-953A-18E024D3717C-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-09 16:07 ` Jerome Glisse
[not found] ` <20180509160722.GB4140-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-09 16:30 ` Stephen Bates
[not found] ` <366A8132-B88A-40F7-BDE3-DA542E45FC0C-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-09 17:49 ` Jerome Glisse
[not found] ` <20180509174952.GC4140-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-10 14:20 ` Stephen Bates
[not found] ` <405531AE-8315-4A4F-9B0C-8DBE49BFCAB4-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-10 14:29 ` Christian König
[not found] ` <73b6a454-bf84-1640-0b5e-137d03c0ad8c-5C7GfCeVMHo@public.gmane.org>
2018-05-10 14:59 ` Jerome Glisse
[not found] ` <20180510145946.GB3652-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-10 18:44 ` Stephen Bates
2018-05-09 16:45 ` Logan Gunthorpe
[not found] ` <4e0d0b96-ab02-2662-adf3-fa956efd294c-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-10 12:52 ` Christian König
[not found] ` <2fc61d29-9eb4-d168-a3e5-955c36e5d821-5C7GfCeVMHo@public.gmane.org>
2018-05-10 14:16 ` Stephen Bates
2018-05-10 14:41 ` Jerome Glisse
[not found] ` <20180510144137.GA3652-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-10 18:41 ` Stephen Bates
[not found] ` <B525870A-B25E-46BB-B501-DAA39F1741DC-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-10 18:59 ` Logan Gunthorpe
2018-05-10 19:10 ` Alex Williamson
[not found] ` <20180510131015.4ad59477-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-10 19:24 ` Jerome Glisse
[not found] ` <94C8FE12-7FC3-48BD-9DCA-E6A427E71810-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-10 16:32 ` Logan Gunthorpe
[not found] ` <f0101988-cf82-7ffb-2aca-d3b8d2f4b504-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-10 17:11 ` Stephen Bates
[not found] ` <868B49CE-4F0E-4A48-BE78-12149F85F1A4-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-10 17:15 ` Logan Gunthorpe
[not found] ` <8113cba8-62b9-1801-7a77-f82be223b183-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-11 8:52 ` Christian König
[not found] ` <cf2b92c9-7c5b-f19c-b5b1-2f2d435296d1-5C7GfCeVMHo@public.gmane.org>
2018-05-11 15:48 ` Logan Gunthorpe
[not found] ` <53fa38ce-40dd-6bc1-9b83-c6ccf228b755-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-11 21:50 ` Stephen Bates
[not found] ` <A50EA568-81E7-4AD7-8C1B-1029FEA92F49-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-11 22:24 ` Stephen Bates
2018-05-11 22:55 ` Logan Gunthorpe
2018-05-08 14:31 ` Dan Williams
[not found] ` <CAPcyv4g+5N_n03J-VhmJfrr60Sqk95ZapLKBkwSsGvV4nQ+ZRw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-05-08 14:44 ` Stephen Bates
[not found] ` <64C231F5-DE36-415F-B308-3A423B0BBACB-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-08 21:04 ` Don Dutile
[not found] ` <15433946-f7f5-f610-4e80-380fb59920e5-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-08 21:27 ` Stephen Bates
[not found] ` <3C9FB262-A93C-4C8F-B1E0-85C6D6F78BC2-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-08 23:06 ` Don Dutile
[not found] ` <c5f30ced-7bbf-9420-db1a-72519fc78b3a-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-09 0:01 ` Alex Williamson
[not found] ` <20180508180157.7c7b393f-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-09 12:35 ` Stephen Bates
[not found] ` <B5BFCF39-6BB4-4E45-B805-C8FCC8B79EF3-pv7U853sEMVWk0Htik3J/w@public.gmane.org>
2018-05-09 14:44 ` Alex Williamson
[not found] ` <20180509084445.626d2732-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-09 15:52 ` Don Dutile
2018-05-09 15:47 ` Don Dutile
2018-05-09 15:53 ` Don Dutile
2018-04-23 23:30 ` [PATCH v4 05/14] docs-rst: Add a new directory for PCI documentation Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 06/14] PCI/P2PDMA: Add P2P DMA driver writer's documentation Logan Gunthorpe
[not found] ` <20180423233046.21476-7-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-07 23:20 ` Bjorn Helgaas
2018-05-22 21:24 ` Randy Dunlap
[not found] ` <59b7762e-6d38-7746-20f3-22fd762ecfe7-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2018-05-22 21:28 ` Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 07/14] block: Introduce PCI P2P flags for request and request queue Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 08/14] IB/core: Ensure we map P2P memory correctly in rdma_rw_ctx_[init|destroy]() Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 09/14] nvme-pci: Use PCI p2pmem subsystem to manage the CMB Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 10/14] nvme-pci: Add support for P2P memory in requests Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 11/14] nvme-pci: Add a quirk for a pseudo CMB Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 12/14] nvmet: Introduce helper functions to allocate and free request SGLs Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 13/14] nvmet-rdma: Use new SGL alloc/free helper for requests Logan Gunthorpe
2018-04-23 23:30 ` [PATCH v4 14/14] nvmet: Optionally use PCI P2P memory Logan Gunthorpe
2018-05-02 11:51 ` [PATCH v4 00/14] Copy Offload in NVMe Fabrics with P2P PCI Memory Christian König
[not found] ` <b8acf835-df94-9967-2327-8f0e39d88511-5C7GfCeVMHo@public.gmane.org>
2018-05-02 15:56 ` Logan Gunthorpe
[not found] ` <805645c1-ea40-2e57-88eb-5dd34e579b2e-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-03 9:05 ` Christian König
[not found] ` <3e4e0126-f444-8d88-6793-b5eb97c61f76-5C7GfCeVMHo@public.gmane.org>
2018-05-03 15:59 ` Logan Gunthorpe
[not found] ` <ce2e5351-f32a-bd80-1c77-b7fc38842175-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-03 17:29 ` Christian König
[not found] ` <38d866cf-f7b4-7118-d737-5a5dcd9f3784-5C7GfCeVMHo@public.gmane.org>
2018-05-03 18:43 ` Logan Gunthorpe
[not found] ` <2d59aa02-f2fa-bd88-1b6c-923117a6ad28-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
2018-05-04 14:27 ` Christian König
[not found] ` <17e7bc28-f499-c7f7-f91a-9f6778de5fd3-5C7GfCeVMHo@public.gmane.org>
2018-05-04 15:52 ` Logan Gunthorpe
2018-05-07 23:23 ` Bjorn Helgaas
[not found] ` <20180507232346.GI161390-1RhO1Y9PlrlHTL0Zs8A6p5iNqAH0jzoTYJqu5kTmcBRl57MIdRCFDg@public.gmane.org>
2018-05-07 23:34 ` Logan Gunthorpe
2018-05-08 16:57 ` Alex Williamson
[not found] ` <20180508105759.7dbaa8fe-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org>
2018-05-08 19:14 ` Logan Gunthorpe
2018-05-08 21:25 ` Don Dutile
[not found] ` <c16ecc49-a0f3-152d-785b-a646a3dab60d-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-05-08 21:40 ` Alex Williamson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b4183ef-e720-204b-9e85-b9eaf7a4136a@deltatee.com \
--to=logang-otvngxwrz7hwk0htik3j/w@public.gmane.org \
--cc=alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
--cc=axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org \
--cc=benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org \
--cc=bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
--cc=christian.koenig-5C7GfCeVMHo@public.gmane.org \
--cc=hch-jcswGhMUV9g@public.gmane.org \
--cc=helgaas-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
--cc=jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
--cc=jglisse-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
--cc=keith.busch-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
--cc=linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org \
--cc=linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \
--cc=linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=maxg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox