Re: [PATCH 2/3] pci: Clamp pcie_set_readrq() when using "performance" settings

Benjamin LaHaise
2011-10-04T14:42:16Z

On Mon, Oct 03, 2011 at 04:55:48PM -0500, Jon Mason wrote:
> From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
>
> When configuring the PCIe settings for "performance", we allow parents
> to have a larger Max Payload Size than children and rely on children
> Max Read Request Size to not be larger than their own MPS to avoid
> having the host bridge generate responses they can't cope with.

I'm pretty sure that simply will not work, and is an incorrect understanding
of how PCIe bridges and devices interact with regards to transaction size
limits. Here's why: I am actually implementing a PCIe nic on an FPGA at
present, and have just been in the process of tuning how memory read
requests are issued and processed. It is perfectly valid for a PCIe
endpoint to issue a read request for an entire 4KB block (assuming it
respects the no 4KB boundary crossings rule), even when the MPS setting
is only 64 or 128 bytes. However, the root complex or PCIe bridge *must
not* exceed the Maximum Payload Size for any completions with data or
posted writes. Multiple completions are okay and expected for read
requests. If the MPS on the bridge is set to a larger value than
what all of the endpoints connected to it, the bridge or root complex will
happily send read completions exceeding the endpoint's MPS. This can and
will lead to failure on the parts of endpoints.

-ben