Re: [PATCH v9 02/15] cxl/regs: Prepare for multiple users of register mappings

[Jonathan Cameron <Jonathan.Cameron@Huawei.com>]

On Thu, 31 Aug 2023 14:43:53 +0200
Robert Richter <rrichter@amd.com> wrote:

> On 29.08.23 14:52:54, Jonathan Cameron wrote:
> > On Fri, 25 Aug 2023 18:31:58 -0500
> > Terry Bowman <terry.bowman@amd.com> wrote:
> >   
> > > From: Robert Richter <rrichter@amd.com>
> > > 
> > > The function devm_cxl_iomap_block() is used to map register mappings
> > > of CXL component or device registers. A @dev is used to unmap the IO
> > > regions during device removal.
> > > 
> > > Now, there are multiple devices using the register mappings. E.g. the
> > > RAS cap of the Component Registers is used by cxl_pci, the HDM cap
> > > used in cxl_mem. This could cause IO blocks not being freed and a
> > > subsequent reinitialization to fail if the same device is used for
> > > both.
> > > 
> > > To prevent that, expand cxl_map_component_regs() to pass a @dev to be
> > > used with devm to IO unmap. This allows to pass the device that
> > > actually is creating and using the IO region.
> > > 
> > > For symmetry also change the function i/f of cxl_map_device_regs().
> > > 
> > > Signed-off-by: Robert Richter <rrichter@amd.com>
> > > Signed-off-by: Terry Bowman <terry.bowman@amd.com>  
> > 
> > I'm not sure we should leave map->dev around after this change
> > as it's not obvious where it is valid to use and where it isn't.
> > 
> > Perhaps we just need to pass the device into the few calls
> > that use it other than the ones you have here.  
> 
> I have checked that and it turned out we would need to pass @dev
> through multiple functions (see cxl_setup_regs()). So I left it in as
> devm is a special case (mis)using it.
> 
> > 
> > This patch itself looks fine to me.  
> 
> I will take it as a Reviewed-by:?
> 
No.  I'm not convinced on the map->dev question even though I appreciate
that passing it around is a mess. To me it's really non obvious that
in some cases map->dev is appropriate and in others a different dev
is needed.  Maybe some naming change would make it clear what each one is for?

Jonathan

> Thanks,
> 
> -Robert
> 
> > 
> > Jonathan
> >   
> > > ---
> > >  drivers/cxl/core/hdm.c  | 3 ++-
> > >  drivers/cxl/core/regs.c | 4 ++--
> > >  drivers/cxl/cxl.h       | 2 ++
> > >  drivers/cxl/pci.c       | 4 ++--
> > >  4 files changed, 8 insertions(+), 5 deletions(-)
> > > 
> > > diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c
> > > index 4449b34a80cc..17c8ba8c75e0 100644
> > > --- a/drivers/cxl/core/hdm.c
> > > +++ b/drivers/cxl/core/hdm.c
> > > @@ -98,7 +98,8 @@ static int map_hdm_decoder_regs(struct cxl_port *port, void __iomem *crb,
> > >  		return -ENODEV;
> > >  	}
> > >  
> > > -	return cxl_map_component_regs(&map, regs, BIT(CXL_CM_CAP_CAP_ID_HDM));
> > > +	return cxl_map_component_regs(&map, &port->dev, regs,
> > > +				      BIT(CXL_CM_CAP_CAP_ID_HDM));
> > >  }
> > >  
> > >  static bool should_emulate_decoders(struct cxl_endpoint_dvsec_info *info)
> > > diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c
> > > index 6281127b3e9d..dfc3e272e7d8 100644
> > > --- a/drivers/cxl/core/regs.c
> > > +++ b/drivers/cxl/core/regs.c
> > > @@ -201,10 +201,10 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr,
> > >  }
> > >  
> > >  int cxl_map_component_regs(const struct cxl_register_map *map,
> > > +			   struct device *dev,
> > >  			   struct cxl_component_regs *regs,
> > >  			   unsigned long map_mask)
> > >  {
> > > -	struct device *dev = map->dev;
> > >  	struct mapinfo {
> > >  		const struct cxl_reg_map *rmap;
> > >  		void __iomem **addr;
> > > @@ -235,9 +235,9 @@ int cxl_map_component_regs(const struct cxl_register_map *map,
> > >  EXPORT_SYMBOL_NS_GPL(cxl_map_component_regs, CXL);
> > >  
> > >  int cxl_map_device_regs(const struct cxl_register_map *map,
> > > +			struct device *dev,
> > >  			struct cxl_device_regs *regs)
> > >  {
> > > -	struct device *dev = map->dev;
> > >  	resource_size_t phys_addr = map->resource;
> > >  	struct mapinfo {
> > >  		const struct cxl_reg_map *rmap;
> > > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> > > index 76d92561af29..ec8ba9ebcf64 100644
> > > --- a/drivers/cxl/cxl.h
> > > +++ b/drivers/cxl/cxl.h
> > > @@ -274,9 +274,11 @@ void cxl_probe_component_regs(struct device *dev, void __iomem *base,
> > >  void cxl_probe_device_regs(struct device *dev, void __iomem *base,
> > >  			   struct cxl_device_reg_map *map);
> > >  int cxl_map_component_regs(const struct cxl_register_map *map,
> > > +			   struct device *dev,
> > >  			   struct cxl_component_regs *regs,
> > >  			   unsigned long map_mask);
> > >  int cxl_map_device_regs(const struct cxl_register_map *map,
> > > +			struct device *dev,
> > >  			struct cxl_device_regs *regs);
> > >  int cxl_map_pmu_regs(struct pci_dev *pdev, struct cxl_pmu_regs *regs,
> > >  		     struct cxl_register_map *map);
> > > diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
> > > index 48f88d96029d..88ddcff8a0b2 100644
> > > --- a/drivers/cxl/pci.c
> > > +++ b/drivers/cxl/pci.c
> > > @@ -827,7 +827,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > >  	if (rc)
> > >  		return rc;
> > >  
> > > -	rc = cxl_map_device_regs(&map, &cxlds->regs.device_regs);
> > > +	rc = cxl_map_device_regs(&map, cxlds->dev, &cxlds->regs.device_regs);
> > >  	if (rc)
> > >  		return rc;
> > >  
> > > @@ -844,7 +844,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > >  
> > >  	cxlds->component_reg_phys = map.resource;
> > >  
> > > -	rc = cxl_map_component_regs(&map, &cxlds->regs.component,
> > > +	rc = cxl_map_component_regs(&map, cxlds->dev, &cxlds->regs.component,
> > >  				    BIT(CXL_CM_CAP_CAP_ID_RAS));
> > >  	if (rc)
> > >  		dev_dbg(&pdev->dev, "Failed to map RAS capability.\n");  
> >