Re: about mpss with pcie_bus_perf

[Yijing Wang <wangyijing@huawei.com>]

>>> Not a lot of verbiage in here, but I believe this is the expected
>>> behavior for the "pcie_bus_perf" kernel boot parm.  With it, each pci
>>> device sets its MPS to the max of the parent
>>
>> Yes, it's the expected behavior for the "pcie_bus_per".
>> Pcie_write_mrrs() will additionally set mrrs to largest supported value for safe.
>>
>>>
>>> >From the commit log:
>>>
>>>     - A more optimal way is possible, if it falls within a couple of
>>>       constraints:
>>>     * The top-level host bridge will never generate packets larger than the
>>>       smallest TLP (or if it can be controlled independently from its MPS at
>>>       least)
>>>     * The device will never generate packets larger than MPS (which can be
>>>       configured via MRRS)
>>>     * No support of direct PCI-E <-> PCI-E transfers between devices without
>>>       some additional code to specifically deal with that case
>>>
>>>     Then we can use an approach that basically ignores downstream requests
>>>     and focuses exclusively on upstream requests. In that case, all we need
>>
>> Hi Jon, I do not quite understand why we can ignores downstream here , as a model like Yinghai's pcie topo:
>>
>> mps 256           mps 256               mps 256
>> root port ------Switch port(UP) -------Switch port(DP) A   --------PCIe Endpoint Device ( mps = 128)
>>                                |                          <-------Read Request to upstream is safe because MRRS is set to properly value.
>>                                |                          <-------TLP payload won't excess (mps=128) as a transmitter, so this is also safe.
>>                                |                          -------->Downstream TLP like read completion and some other TLP write to PCIe EP device
>>                                |                                   My question is here, how can we ensure Downstream is safe?
>>                                |
>>                                |
>>                                |-------Switch port(DP) B
>>
>> Sorry to disturb you, I would be appreciate if you can me any advice. Thanks!
> 
> If all inter-device communication is removed, then the only
> communication is CPU, Endpoint, and switches in-between.  Going from
> CPU to Endpoint, the MPS is actually going to be the Cache Line size.
> Since the Cache line size is 64B on x86 and most other architectures,
> there is no worry that the endpoint will get a PCIE packet larger than
> the MPS.  Also, using the MRRS to clamp down the endpoint to the MPS
> of the switches should ensure no reads larger than the MPS.  Going
> from Endpoint to CPU, we must ensure that all switches have a MPSS
> large enough for any device under them.  If not, then we must clamp
> down the Endpoint MPS.
> 
> If all of this works, then we can ensure a much larger MPS for all of
> the PCI devices under a switch and not be bound by the smallest MPSS
> of an endpoint on the switch.
> 

Hi Jon, I got it, Thanks for your explanation.


Thanks!
Yijing.



>>
>>>     to care about is that a device MPS is no larger than its parent MPS,
>>>     which allows us to keep all switches/bridges to the max MPS supported by
>>>     their parent and eventually the PHB.
>>>
>>>
>>> If this is not behaving as described (which I can't tell from the log
>>> above), then feel free to assign the bug to me.
>>>
>>> Thanks,
>>> Jon
>>>
>>>
>>>
>>>>
>>>> Bjorn
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> .
>>>
>>
>>
>> --
>> Thanks!
>> Yijing
>>
> 
> .
> 


-- 
Thanks!
Yijing