[RFC] Proposed PCI ECR for description of Advanced Peer to Peer Capabilities

[Jonathan Cameron <jonathan.cameron@huawei.com>]

Hi All,

This is perhaps a somewhat unusual RFC as it's to ask for feedback on an ECR
that is currently under discussion in the PCI SIG.  That ECR, at least partly,
came out of the issues highlighted during the discussion of P2PDMA.  In
that patch set, p2pdma is only allowed if there is a switch between the
devices involved ensuring the path does not pass through a root port.
Put simply there is no way to know if a root complex will allow p2p or not.

So we are asking for feedback at a stage where modifications to the
proposal can be made. Eric has agreed to this being shared with mailing list
to hopefully get such feedback.  Note this is under active discussion within
the SIG so inputs in that forum also welcome.

I would recommend reading the ECR and presentation for more details.
Anyone who can't get to Google Drive, email me and I'll send a copy directly.

ECR: https://drive.google.com/open?id=1X08pfyjkNwSfpTVBuiut90ajxBZd7BXg
Slides: https://drive.google.com/open?id=11d3egA8x91d99-QqfYGZ-MxLGrHpJHCK

My summary probably won't be great, but I'll give it a go to persuade people
to look at the real docs!

Problem Statement:

Linux does not generally enable Peer to Peer because (amongst other reasons)
it does not know which Root Complexes support Peer to Peer.

* Some Root Complexes do not support Peer to Peer
* Some Root Complexes only support Posted Peer to Peer
* Some Root Complexes only support Peer to Peer within one PCIe Segment
* Some Root Complexes only support Address Decode Peer to Peer (no ID routing)
* Some Root Complexes may not support Peer to Peer from Integrated End Points

Related other issue
* MPS must be programmed the same across all devices supporting P2P
* Hot plugged devices might require a smaller MPS than was programmed.

Solution:
* New extended capability and control (though only one bit in the control)
* Advertise what a Peer to Peer Root Port and Switch Down Stream ports can do.
* Define packet splitting scheme for MPS mismatches.
	- Rules on how this must be done if supported.

All feedback welcome.  Let's make sure we get this right.

Thanks,

Jonathan Cameron
Huawei