Re: [PATCH] cxl: Move register block enumeration to core

[Ben Widawsky <ben.widawsky@intel.com>]

On 21-09-21 11:42:55, Dan Williams wrote:
> On Tue, Sep 21, 2021 at 9:44 AM Ben Widawsky <ben.widawsky@intel.com> wrote:
> >
> > On 21-09-21 07:07:13, Dan Williams wrote:
> > > On Mon, Sep 20, 2021 at 3:57 PM Ben Widawsky <ben.widawsky@intel.com> wrote:
> > > >
> > > > CXL drivers or cxl_core itself will require the ability to map component
> > > > registers in order to map HDM decoder resources amongst other things.
> > > > Much of the register mapping code has already moved into cxl_core. The
> > > > code to pull the BAR number and block office remained within cxl_pci
> >
> > s/office/offset - before anyone else notices...
> >
> > > > because there was no need to move it. Upcoming work will require this
> > > > functionality to be available outside of cxl_pci.
> > > >
> > > > There are two intentional functional changes:
> > > > 1. cxl_pci: If there is more than 1 component, or device register block,
> > > >    only the first one (of each) is checked. Previous logic checked all
> > > >    duplicate register blocks and additionally attempted to map unused
> > > >    register blocks if present.
> > > > 2. cxl_pci: No more dev_dbg for unused register blocks
> > >
> > > Why not break these out into separate patches before moving the code?
> > > It makes it easier to review, and it increases the precision of future
> > > Fixes: patches if necessary.
> > >
> >
> > I can. Indeed it was my instinct to do so. I went against my instinct...
> >
> > What are you thinking (something like...)?
> > 1. Change register locator identifiers to enum
> > 2. refactor cxl_pci to use the new find() function.
> 
> In order to ease the coordination pressure perhaps you could define a
> __weak copy of this helper in the CXL core with a comment of:
> 
> /* TODO: Delete once this same function is available from the PCI core */
> 
> ...and then separately send the refactor series to all the stakeholders.

You mean so I can work on the other drivers without being blocked on this? I
think as long as you generally agree with the final outcome, I'll be okay to
just keep working on top of this.

> 
> > 3. Remove map.type
> 
> Inject a patch for:
> 
> "No more dev_dbg() for unused register blocks"
> 
> ...here?
> 

Sure.

> > 4. move required functionality to core
> >
> > > >
> > > > Cc: Ira Weiny <ira.weiny@intel.com>
> > > > Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
> > > > ---
> > > >  drivers/cxl/core/regs.c |  89 ++++++++++++++++++++++++++++++++++
> > > >  drivers/cxl/cxl.h       |   3 ++
> > > >  drivers/cxl/pci.c       | 104 +++++++---------------------------------
> > > >  drivers/cxl/pci.h       |  14 +++---
> > > >  4 files changed, 116 insertions(+), 94 deletions(-)
> > > >
> > > > diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c
> > > > index 41de4a136ecd..ef6164ef449f 100644
> > > > --- a/drivers/cxl/core/regs.c
> > > > +++ b/drivers/cxl/core/regs.c
> > > > @@ -5,6 +5,7 @@
> > > >  #include <linux/slab.h>
> > > >  #include <linux/pci.h>
> > > >  #include <cxlmem.h>
> > > > +#include <pci.h>
> > > >
> > > >  /**
> > > >   * DOC: cxl registers
> > > > @@ -247,3 +248,91 @@ int cxl_map_device_regs(struct pci_dev *pdev,
> > > >         return 0;
> > > >  }
> > > >  EXPORT_SYMBOL_GPL(cxl_map_device_regs);
> > > > +
> > > > +static void cxl_decode_register_block(u32 reg_lo, u32 reg_hi, u8 *bar,
> > > > +                                     u64 *offset, u8 *reg_type)
> > > > +{
> > > > +       *offset = ((u64)reg_hi << 32) | (reg_lo & CXL_REGLOC_ADDR_MASK);
> > > > +       *bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo);
> > > > +       *reg_type = FIELD_GET(CXL_REGLOC_RBI_MASK, reg_lo);
> > > > +}
> > > > +
> > > > +static int cxl_pci_dvsec(struct pci_dev *pdev, int dvsec)
> > > > +{
> > > > +       int pos;
> > > > +
> > > > +       pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DVSEC);
> > > > +       if (!pos)
> > > > +               return 0;
> > > > +
> > > > +       while (pos) {
> > > > +               u16 vendor, id;
> > > > +
> > > > +               pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER1, &vendor);
> > > > +               pci_read_config_word(pdev, pos + PCI_DVSEC_HEADER2, &id);
> > > > +               if (vendor == PCI_DVSEC_VENDOR_ID_CXL && dvsec == id)
> > > > +                       return pos;
> > > > +
> > > > +               pos = pci_find_next_ext_capability(pdev, pos,
> > > > +                                                  PCI_EXT_CAP_ID_DVSEC);
> > > > +       }
> > >
> > > We punted on refactoring this for the initial driver submission
> > > because it was difficult to coordinate. Now that cxl.git is an
> > > established tree, instead of moving this it seems time to address that
> > > refactor that Bjorn asked about. Bjorn, would you be willing to carry
> > > a non-rebasing branch with such a cleanup that CXL could pull from?
> > >
> >
> > Also here:
> > https://lore.kernel.org/linux-pci/20210913190131.xiiszmno46qie7v5@intel.com/
> 
> Looks good, although I notice that find_dvsec_from_pos() from
> arch/powerpc/platforms/powernv/ocxl.c wants to start searching for the
> dvsec starting from an initial offset.
> 
> >
> > > > +
> > > > +       return 0;
> > > > +}
> > > > +
> > > > +/**
> > > > + * cxl_get_register_block() - Find the CXL register block by identifier
> > > > + * @pdev: The PCI device implementing the registers
> > > > + * @type: Type of register block to find
> > > > + * @map: (Output) parameters used to map the regiseter block
> > >
> > > s/regiseter/register/
> > >
> > > > + *
> > > > + * If register block is found, 0 is returned and @map is populated; else returns
> > > > + * negative error code.
> > > > + */
> > > > +int cxl_get_register_block(struct pci_dev *pdev, enum cxl_regloc_type type,
> > >
> > > Seeing this broken out again it looks like a 'find' operation rather
> > > than a 'get', but not a big deal.
> > >
> >
> > I'm okay to rename it. It also makes me realize map.type becomes a useless
> > field.
> 
> True.