Re: [RFC net-next 0/7] Provide an ism layer

[Andrew Lunn <andrew@lunn.ch>]

> Conceptually kind of but the existing s390 specific ISM device is a bit
> special. But let me start with some background. On s390 aka Mainframes
> OSs including Linux runs in so called logical partitions (LPARs) which
> are machine hypervisor VMs which use partitioned non-paging memory. The
> fact that memory is partitioned is important because this means LPARs
> can not share physical memory by mapping it.
> 
> Now at a high level an ISM device allows communication between two such
> Linux LPARs on the same machine. The device is discovered as a PCI
> device and allows Linux to take a buffer called a DMB map that in the
> IOMMU and generate a token specific to another LPAR which also sees an
> ISM device sharing the same virtual channel identifier (VCHID). This
> token can then be transferred out of band (e.g. as part of an extended
> TCP handshake in SMC-D) to that other system. With the token the other
> system can use its ISM device to securely (authenticated by the token,
> LPAR identity and the IOMMU mapping) write into the original systems
> DMB at throughput and latency similar to doing a memcpy() via a
> syscall.
> 
> On the implementation level the ISM device is actually a piece of
> firmware and the write to a remote DMB is a special case of our PCI
> Store Block instruction (no real MMIO on s390, instead there are
> special instructions). Sadly there are a few more quirks but in
> principle you can think of it as redirecting writes to a part of the
> ISM PCI devices' BAR to the DMB in the peer system if that makes sense.
> There's of course also a mechanism to cause an interrupt on the
> receiver as the write completes.

So the s390 details are interesting, but as you say, it is
special. Ideally, all the special should be hidden away inside the
driver.

So please take a step back. What is the abstract model?

Can the abstract model be mapped onto CLX? Could it be used with a GPU
vRAM? SoC with real shared memory between a pool of CPUs.

	Andrew