* Re: [PATCH RFC 1/1] Add support for ZONE_DEVICE IO memory with struct pages.
[not found] <1457979277-26791-1-git-send-email-stephen.bates@pmcs.com>
@ 2016-03-14 21:23 ` Matthew Wilcox
0 siblings, 0 replies; only message in thread
From: Matthew Wilcox @ 2016-03-14 21:23 UTC (permalink / raw)
To: Stephen Bates
Cc: haggaie, javier, linux-rdma, linux-nvdimm, sagig, linux-mm,
artemyko, hch, leonro, jgunthorpe
On Mon, Mar 14, 2016 at 12:14:37PM -0600, Stephen Bates wrote:
> 3. Coherency Issues. When IOMEM is written from both the CPU and a PCIe
> peer there is potential for coherency issues and for writes to occur out
> of order. This is something that users of this feature need to be
> cognizant of and may necessitate the use of CONFIG_EXPERT. Though really,
> this isn't much different than the existing situation with RDMA: if
> userspace sets up an MR for remote use, they need to be careful about
> using that memory region themselves.
There's more to the coherency problem than this. As I understand it, on
x86, memory in a PCI BAR does not participate in the coherency protocol.
So you can get a situation where CPU A stores 4 bytes to offset 8 in a
cacheline, then CPU B stores 4 bytes to offset 16 in the same cacheline,
and CPU A's write mysteriously goes missing.
I may have misunderstood the exact details when this was explained to me a
few years ago, but the details were horrible enough to run away screaming.
Pretending PCI BARs are real memory? Just Say No.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
^ permalink raw reply [flat|nested] only message in thread