Re: [RFC PATCH 2/2] cxl: Force a CAPP reset when unloading CXL module

[Frederic Barrat <fbarrat@linux.ibm.com>]

Le 25/01/2019 à 06:11, Vaibhav Jain a écrit :
> This patch forces shutdown of CAPP when CXL module is unloaded. This
> is accomplished via a call to pnv_phb_to_cxl_mode() with mode ==
> OPAL_PHB_CAPI_MODE_PCIE.
> 
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---
>   drivers/misc/cxl/cxl.h  |  1 +
>   drivers/misc/cxl/main.c |  3 +++
>   drivers/misc/cxl/pci.c  | 25 ++++++++++++++++++++++++-
>   3 files changed, 28 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/misc/cxl/cxl.h b/drivers/misc/cxl/cxl.h
> index d1d927ccb589..e545c2b81faf 100644
> --- a/drivers/misc/cxl/cxl.h
> +++ b/drivers/misc/cxl/cxl.h
> @@ -1136,4 +1136,5 @@ void cxl_context_mm_count_get(struct cxl_context *ctx);
>   /* Decrements the reference count to "struct mm_struct" */
>   void cxl_context_mm_count_put(struct cxl_context *ctx);
>   
> +void cxl_pci_shutdown_capp(void);
>   #endif
> diff --git a/drivers/misc/cxl/main.c b/drivers/misc/cxl/main.c
> index f35406be465a..f14ff0dcf231 100644
> --- a/drivers/misc/cxl/main.c
> +++ b/drivers/misc/cxl/main.c
> @@ -372,6 +372,9 @@ static void exit_cxl(void)
>   	if (cxl_is_power8())
>   		unregister_cxl_calls(&cxl_calls);
>   	idr_destroy(&cxl_adapter_idr);
> +
> +	if (cpu_has_feature(CPU_FTR_HVMODE))
> +		cxl_pci_shutdown_capp();
>   }
>   
>   module_init(init_cxl);
> diff --git a/drivers/misc/cxl/pci.c b/drivers/misc/cxl/pci.c
> index c79ba1c699ad..01be2e2d1069 100644
> --- a/drivers/misc/cxl/pci.c
> +++ b/drivers/misc/cxl/pci.c
> @@ -25,7 +25,7 @@
>   
>   #include "cxl.h"
>   #include <misc/cxl.h>
> -
> +#include <misc/cxllib.h>
>   
>   #define CXL_PCI_VSEC_ID	0x1280
>   #define CXL_VSEC_MIN_SIZE 0x80
> @@ -2065,6 +2065,29 @@ static void cxl_pci_resume(struct pci_dev *pdev)
>   	}
>   }
>   
> +void cxl_pci_shutdown_capp(void)
> +{
> +	struct pci_dev *pdev;
> +	struct pci_bus *root_bus;
> +	int rc;
> +
> +	/* Iterate over all CAPP supported PHB's and force them to PCI mode */
> +	list_for_each_entry(root_bus, &pci_root_buses, node) {
> +		for_each_pci_bridge(pdev, root_bus) {
> +
> +			if (!cxllib_slot_is_supported(pdev, 0))
> +				continue;
> +
> +			rc = pnv_phb_to_cxl_mode(pdev,
> +						 OPAL_PHB_CAPI_MODE_PCIE);
> +			if (rc)
> +				dev_err(&pdev->dev,
> +					"cxl: Error resetting CAPP. Err=%d\n",
> +					rc);
> +		}


That's the part I don't like. We're iterating over quite a few PCI 
devices, we basically don't know the ones we need to reset.
If we have a per-adapter property on /sys to activate the 
reset-on-unload, then we could move the call to 
pnv_phb_to_cxl_mode(OPAL_PHB_CAPI_MODE_PCIE) on the cxl_remove() 
callback, and only do it for the adapters we've been asked.

   Fred



> +	}
> +}
> +
>   static const struct pci_error_handlers cxl_err_handler = {
>   	.error_detected = cxl_pci_error_detected,
>   	.slot_reset = cxl_pci_slot_reset,
>