Re: [PATCH v7 3/3] cxl: Add documentation to explain the shared link bandwidth calculation

[Jonathan Cameron <Jonathan.Cameron@Huawei.com>]

On Wed, 28 Aug 2024 08:35:07 -0700
Dave Jiang <dave.jiang@intel.com> wrote:

> On 8/28/24 2:00 AM, Jonathan Cameron wrote:
> > On Tue, 27 Aug 2024 15:38:27 -0700
> > Dave Jiang <dave.jiang@intel.com> wrote:
> >   
> >> On 8/27/24 9:06 AM, Jonathan Cameron wrote:  
> >>> On Wed, 10 Jul 2024 15:24:02 -0700
> >>> Dave Jiang <dave.jiang@intel.com> wrote:
> >>>     
> >>>> Create a kernel documentation to describe how the CXL shared upstream
> >>>> link bandwidth is calculated.
> >>>>
> >>>> Suggested-by: Dan Williams <dan.j.williams@intel.com>
> >>>> Signed-off-by: Dave Jiang <dave.jiang@intel.com>    
> >>> Oops. Missed this previously.  A few minor things inline.
> >>>
> >>> J    
> >>>> ---
> >>>>  .../driver-api/cxl/access-coordinates.rst     | 90 +++++++++++++++++++
> >>>>  Documentation/driver-api/cxl/index.rst        |  1 +
> >>>>  MAINTAINERS                                   |  1 +
> >>>>  3 files changed, 92 insertions(+)
> >>>>  create mode 100644 Documentation/driver-api/cxl/access-coordinates.rst
> >>>>
> >>>> diff --git a/Documentation/driver-api/cxl/access-coordinates.rst b/Documentation/driver-api/cxl/access-coordinates.rst
> >>>> new file mode 100644
> >>>> index 000000000000..973e63872f06
> >>>> --- /dev/null
> >>>> +++ b/Documentation/driver-api/cxl/access-coordinates.rst
> >>>> @@ -0,0 +1,90 @@
> >>>> +.. SPDX-License-Identifier: GPL-2.0
> >>>> +.. include:: <isonum.txt>
> >>>> +
> >>>> +==================================
> >>>> +CXL Access Coordinates Computation
> >>>> +==================================
> >>>> +
> >>>> +Shared Upstream Link Calculation
> >>>> +================================
> >>>> +For certain CXL region construction with endpoints behind CXL switches (SW) or
> >>>> +Root Ports (RP), there is the possibility of the total bandwdith for all    
> >>> spell check. bandwidth    
> >>>> +the endpoints behind a switch being more than the switch upstream link.    
> >>>
> >>> Could also be the Generic Port bit of the topology. Mind you I'm still failing
> >>> to come up with text for the qemu GP Documentation that the reviewers can follow
> >>> so maybe that's just too hard to describe here.  Could use weasel words.
> >>>
> >>> "A similar situation can occur within the host, upstream of the root ports."
> >>>
> >>>     
> >>>> +The CXL driver performs an additional pass after all the targets have
> >>>> +arrived for a region in order to recalculate the bandwidths with possible
> >>>> +upstream link being a limiting factor in mind.
> >>>> +
> >>>> +The algorithm assumes the configuration is a symmetric topology as that
> >>>> +maximizes performance. When asymmetric topology is detected, the calculation
> >>>> +is aborted when such topology is detected. An asymmetric topology is detected    
> >>>
> >>> is detected is duplicated.    
> >>
> >> I don't follow here.
> >>
> >> + When asymmetric topology is detected, the calculation  
> >                              ^^^^^^^^^^^  
> >> +is aborted when such topology is detected.  
> >                                 ^^^^^^^^^^^^
> > Delete the second one.  
> 
> Thanks. The brain does funny tricks sometimes when reading the stuff you write yourself. 
> 
> >   
> >>>     
> >>>> +during topology walk where the number of RPs detected as a grandparent is not
> >>>> +equal to the number of devices iterated in the same iteration loop.    
> >>>
> >>> Maybe make the point that asymmetric in terms of only properties of devices
> >>> is not detected. It just uses the first one I think?    
> >>
> >> I also don't follow here.
> >>
> >> I also wonder if I should use "unbalanced" topology rather than "unsymmetric"?  
> > 
> > Difference between detecting a topology with different number of links and one
> > where say one type 3 device in an interleave set has lower bandwidth in it's CDAT.
> > 
> > IIRC this does the first, but not the second.  
> 
> I think here we attempt to detect whether one root port has different level of switch(s) vs another. For example a topology configuration like RP0 is direct attached and RP1 has a switch underneath. 

I'd just add a line to say that we assume that
subtle asymmetry in properties doesn't happen, so compute
bandwidth on assumption all paths to EPs are equivalent.

Jonathan

> 
> >   
> >>
> >> DJ  
> >   
>