Re: [PATCH v7 3/3] cxl: Add documentation to explain the shared link bandwidth calculation

Dave Jiang
2024-08-28T15:35:09Z

On 8/28/24 2:00 AM, Jonathan Cameron wrote:
> On Tue, 27 Aug 2024 15:38:27 -0700
> Dave Jiang <dave.jiang@intel.com> wrote:
>
>> On 8/27/24 9:06 AM, Jonathan Cameron wrote:
>>> On Wed, 10 Jul 2024 15:24:02 -0700
>>> Dave Jiang <dave.jiang@intel.com> wrote:
>>>
>>>> Create a kernel documentation to describe how the CXL shared upstream
>>>> link bandwidth is calculated.
>>>>
>>>> Suggested-by: Dan Williams <dan.j.williams@intel.com>
>>>> Signed-off-by: Dave Jiang <dave.jiang@intel.com>
>>> Oops. Missed this previously. A few minor things inline.
>>>
>>> J
>>>> ---
>>>> .../driver-api/cxl/access-coordinates.rst | 90 +++++++++++++++++++
>>>> Documentation/driver-api/cxl/index.rst | 1 +
>>>> MAINTAINERS | 1 +
>>>> 3 files changed, 92 insertions(+)
>>>> create mode 100644 Documentation/driver-api/cxl/access-coordinates.rst
>>>>
>>>> diff --git a/Documentation/driver-api/cxl/access-coordinates.rst b/Documentation/driver-api/cxl/access-coordinates.rst
>>>> new file mode 100644
>>>> index 000000000000..973e63872f06
>>>> --- /dev/null
>>>> +++ b/Documentation/driver-api/cxl/access-coordinates.rst
>>>> @@ -0,0 +1,90 @@
>>>> +.. SPDX-License-Identifier: GPL-2.0
>>>> +.. include:: <isonum.txt>
>>>> +
>>>> +==================================
>>>> +CXL Access Coordinates Computation
>>>> +==================================
>>>> +
>>>> +Shared Upstream Link Calculation
>>>> +================================
>>>> +For certain CXL region construction with endpoints behind CXL switches (SW) or
>>>> +Root Ports (RP), there is the possibility of the total bandwdith for all
>>> spell check. bandwidth
>>>> +the endpoints behind a switch being more than the switch upstream link.
>>>
>>> Could also be the Generic Port bit of the topology. Mind you I'm still failing
>>> to come up with text for the qemu GP Documentation that the reviewers can follow
>>> so maybe that's just too hard to describe here. Could use weasel words.
>>>
>>> "A similar situation can occur within the host, upstream of the root ports."
>>>
>>>
>>>> +The CXL driver performs an additional pass after all the targets have
>>>> +arrived for a region in order to recalculate the bandwidths with possible
>>>> +upstream link being a limiting factor in mind.
>>>> +
>>>> +The algorithm assumes the configuration is a symmetric topology as that
>>>> +maximizes performance. When asymmetric topology is detected, the calculation
>>>> +is aborted when such topology is detected. An asymmetric topology is detected
>>>
>>> is detected is duplicated.
>>
>> I don't follow here.
>>
>> + When asymmetric topology is detected, the calculation
> ^^^^^^^^^^^
>> +is aborted when such topology is detected.
> ^^^^^^^^^^^^
> Delete the second one.

Thanks. The brain does funny tricks sometimes when reading the stuff you write yourself.

>
>>>
>>>> +during topology walk where the number of RPs detected as a grandparent is not
>>>> +equal to the number of devices iterated in the same iteration loop.
>>>
>>> Maybe make the point that asymmetric in terms of only properties of devices
>>> is not detected. It just uses the first one I think?
>>
>> I also don't follow here.
>>
>> I also wonder if I should use "unbalanced" topology rather than "unsymmetric"?
>
> Difference between detecting a topology with different number of links and one
> where say one type 3 device in an interleave set has lower bandwidth in it's CDAT.
>
> IIRC this does the first, but not the second.

I think here we attempt to detect whether one root port has different level of switch(s) vs another. For example a topology configuration like RP0 is direct attached and RP1 has a switch underneath.

>
>>
>> DJ
>