[PATCH v2] PCI:vmd: add the module param to adjust msi mode

[PATCH v2] PCI:vmd: add the module param to adjust msi mode

[korantwork]

From: Xinghui Li <korantli@tencent.com>

In the legacy, the vmd msi-mode can only be adjusted by configing
vmd_ids table.This patch adds another way to adjust msi mode by
adjusting module param, which allow users easier to adjust the vmd
according to the I/O scenario without rebuilding driver.There are two
params could be recognized: on, off. The default param is "NULL",
the goal is not to affect the existing settings of the device.

Signed-off-by: Xinghui Li <korantli@tencent.com>
Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>
---
 drivers/pci/controller/vmd.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 990630ec57c6..8b42b2c1d949 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -34,6 +34,20 @@
 #define MB2_SHADOW_OFFSET	0x2000
 #define MB2_SHADOW_SIZE		16
 
+/*
+ * The VMD msi_remap module parameter provides the alternative way
+ * to adjust msi mode when loading vmd.ko other than vmd_ids table.
+ * There are two params could be recognized:
+ *
+ * 1-off
+ * 2-on
+ *
+ * The default param is "NULL", the goal is not to affect the existing
+ * settings of the device.
+ */
+char *msi_remap = "NULL";
+module_param(msi_remap, charp, 0444);
+
 enum vmd_features {
 	/*
 	 * Device may contain registers which hint the physical location of the
@@ -875,6 +889,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 			return ret;
 
 		vmd_set_msi_remapping(vmd, true);
+		dev_info(&vmd->dev->dev, "init vmd with remapping msi-x\n");
 
 		ret = vmd_create_irq_domain(vmd);
 		if (ret)
@@ -887,6 +902,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 		irq_domain_update_bus_token(vmd->irq_domain, DOMAIN_BUS_VMD_MSI);
 	} else {
 		vmd_set_msi_remapping(vmd, false);
+		dev_info(&vmd->dev->dev, "init vmd with bypass msi-x\n");
 	}
 
 	pci_add_resource(&resources, &vmd->resources[0]);
@@ -955,6 +971,14 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 	return 0;
 }
 
+static void vmd_config_msi_remap_param(unsigned long *features)
+{
+	if (strcmp(msi_remap, "on") == 0)
+		*features &= ~(VMD_FEAT_CAN_BYPASS_MSI_REMAP);
+	else if (strcmp(msi_remap, "off") == 0)
+		*features |= VMD_FEAT_CAN_BYPASS_MSI_REMAP;
+}
+
 static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
 {
 	unsigned long features = (unsigned long) id->driver_data;
@@ -984,6 +1008,8 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
 	if (err < 0)
 		goto out_release_instance;
 
+	vmd_config_msi_remap_param(&features);
+
 	vmd->cfgbar = pcim_iomap(dev, VMD_CFGBAR, 0);
 	if (!vmd->cfgbar) {
 		err = -ENOMEM;
-- 
2.31.1

Re: [PATCH v2] PCI:vmd: add the module param to adjust msi mode

[Bjorn Helgaas]

Please adjust the subject line to match previous history, e.g.,

  PCI: vmd: Add ... MSI ...

On Tue, Mar 07, 2023 at 04:35:59PM +0800, korantwork@gmail.com wrote:
> From: Xinghui Li <korantli@tencent.com>
> 
> In the legacy, the vmd msi-mode can only be adjusted by configing
> vmd_ids table.This patch adds another way to adjust msi mode by
> adjusting module param, which allow users easier to adjust the vmd
> according to the I/O scenario without rebuilding driver.There are two
> params could be recognized: on, off. The default param is "NULL",
> the goal is not to affect the existing settings of the device.

Please add a space after the period that ends each sentence.
Capitalize "MSI" to match usage in spec.

> Signed-off-by: Xinghui Li <korantli@tencent.com>
> Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>

I didn't see a response from Nirmal on the mailing list with the
Reviewed-by.  I think it's better if Nirmal responds to the patch
directly on the mailing list with the Reviewed-by, and whoever applies
the patch can incorporate it.  Otherwise we have no visibility into
any interaction between you and Nirmal.

> ---
>  drivers/pci/controller/vmd.c | 26 ++++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index 990630ec57c6..8b42b2c1d949 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -34,6 +34,20 @@
>  #define MB2_SHADOW_OFFSET	0x2000
>  #define MB2_SHADOW_SIZE		16
>  
> +/*
> + * The VMD msi_remap module parameter provides the alternative way
> + * to adjust msi mode when loading vmd.ko other than vmd_ids table.
> + * There are two params could be recognized:
> + *
> + * 1-off
> + * 2-on

It looks like your code matches either "on" or "off", not "1" or "2".

> + * The default param is "NULL", the goal is not to affect the existing
> + * settings of the device.
> + */
> +char *msi_remap = "NULL";

Looks like this should be static?  And using "NULL" (as opposed to
something like the empty string "") suggests some intrinsic meaning of
"NULL", but I think there is no intrinsic meaning and the only point
is that "NULL" doesn't match either "on" or "off".

> +module_param(msi_remap, charp, 0444);
> +
>  enum vmd_features {
>  	/*
>  	 * Device may contain registers which hint the physical location of the
> @@ -875,6 +889,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
>  			return ret;
>  
>  		vmd_set_msi_remapping(vmd, true);
> +		dev_info(&vmd->dev->dev, "init vmd with remapping msi-x\n");
>  
>  		ret = vmd_create_irq_domain(vmd);
>  		if (ret)
> @@ -887,6 +902,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
>  		irq_domain_update_bus_token(vmd->irq_domain, DOMAIN_BUS_VMD_MSI);
>  	} else {
>  		vmd_set_msi_remapping(vmd, false);
> +		dev_info(&vmd->dev->dev, "init vmd with bypass msi-x\n");
>  	}
>  
>  	pci_add_resource(&resources, &vmd->resources[0]);
> @@ -955,6 +971,14 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
>  	return 0;
>  }
>  
> +static void vmd_config_msi_remap_param(unsigned long *features)
> +{
> +	if (strcmp(msi_remap, "on") == 0)
> +		*features &= ~(VMD_FEAT_CAN_BYPASS_MSI_REMAP);
> +	else if (strcmp(msi_remap, "off") == 0)
> +		*features |= VMD_FEAT_CAN_BYPASS_MSI_REMAP;
> +}
> +
>  static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
>  {
>  	unsigned long features = (unsigned long) id->driver_data;
> @@ -984,6 +1008,8 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
>  	if (err < 0)
>  		goto out_release_instance;
>  
> +	vmd_config_msi_remap_param(&features);
> +
>  	vmd->cfgbar = pcim_iomap(dev, VMD_CFGBAR, 0);
>  	if (!vmd->cfgbar) {
>  		err = -ENOMEM;
> -- 
> 2.31.1
> 

Re: [PATCH v2] PCI:vmd: add the module param to adjust msi mode

[Xinghui Li]

Bjorn Helgaas <helgaas@kernel.org> 于2023年3月9日周四 06:57写道：
>
> Please adjust the subject line to match previous history, e.g.,
>
>   PCI: vmd: Add ... MSI ...
>
OK. I will fix it. Sorry for ignoring the subject format.

> > In the legacy, the vmd msi-mode can only be adjusted by configing
> > vmd_ids table.This patch adds another way to adjust msi mode by
> > adjusting module param, which allow users easier to adjust the vmd
> > according to the I/O scenario without rebuilding driver.There are two
> > params could be recognized: on, off. The default param is "NULL",
> > the goal is not to affect the existing settings of the device.
>
> Please add a space after the period that ends each sentence.
> Capitalize "MSI" to match usage in spec.
>
Sorry for the format issue. I neglected them. My bad~

> > Signed-off-by: Xinghui Li <korantli@tencent.com>
> > Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>
>
> I didn't see a response from Nirmal on the mailing list with the
> Reviewed-by.  I think it's better if Nirmal responds to the patch
> directly on the mailing list with the Reviewed-by, and whoever applies
> the patch can incorporate it.  Otherwise we have no visibility into
> any interaction between you and Nirmal.
>
I pinged Nirmal to reply to this patch, It seems he forgot to cc the
mail list in the previous version's discussion.

> > +/*
> > + * The VMD msi_remap module parameter provides the alternative way
> > + * to adjust msi mode when loading vmd.ko other than vmd_ids table.
> > + * There are two params could be recognized:
> > + *
> > + * 1-off
> > + * 2-on
>
> It looks like your code matches either "on" or "off", not "1" or "2".
>
I will change the comment. It does mislead the reader. I mean the No.1
param is "on" and the No.2 param is "off"

> > + * The default param is "NULL", the goal is not to affect the existing
> > + * settings of the device.
> > + */
> > +char *msi_remap = "NULL";
>
> Looks like this should be static?  And using "NULL" (as opposed to
> something like the empty string "") suggests some intrinsic meaning of
> "NULL", but I think there is no intrinsic meaning and the only point
> is that "NULL" doesn't match either "on" or "off".
>
The "static" one is better, I will add it.
Initial parameters with "NULL" just aim to mismatch "on" or "off". Do
you prefer to init it without the default string?

Re: [PATCH v2] PCI:vmd: add the module param to adjust msi mode

[Patel, Nirmal]

On 3/9/2023 5:31 AM, Xinghui Li wrote:
> Bjorn Helgaas <helgaas@kernel.org> 于2023年3月9日周四 06:57写道：
>> Please adjust the subject line to match previous history, e.g.,
>>
>>   PCI: vmd: Add ... MSI ...
>>
> OK. I will fix it. Sorry for ignoring the subject format.
>
>>> In the legacy, the vmd msi-mode can only be adjusted by configing
>>> vmd_ids table.This patch adds another way to adjust msi mode by
>>> adjusting module param, which allow users easier to adjust the vmd
>>> according to the I/O scenario without rebuilding driver.There are two
>>> params could be recognized: on, off. The default param is "NULL",
>>> the goal is not to affect the existing settings of the device.
>> Please add a space after the period that ends each sentence.
>> Capitalize "MSI" to match usage in spec.
>>
> Sorry for the format issue. I neglected them. My bad~
>
>>> Signed-off-by: Xinghui Li <korantli@tencent.com>
>>> Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>
>> I didn't see a response from Nirmal on the mailing list with the
>> Reviewed-by.  I think it's better if Nirmal responds to the patch
>> directly on the mailing list with the Reviewed-by, and whoever applies
>> the patch can incorporate it.  Otherwise we have no visibility into
>> any interaction between you and Nirmal.
>>
> I pinged Nirmal to reply to this patch, It seems he forgot to cc the
> mail list in the previous version's discussion.
>
>>> +/*
>>> + * The VMD msi_remap module parameter provides the alternative way
>>> + * to adjust msi mode when loading vmd.ko other than vmd_ids table.
>>> + * There are two params could be recognized:
>>> + *
>>> + * 1-off
>>> + * 2-on
>> It looks like your code matches either "on" or "off", not "1" or "2".
>>
> I will change the comment. It does mislead the reader. I mean the No.1
> param is "on" and the No.2 param is "off"
>
>>> + * The default param is "NULL", the goal is not to affect the existing
>>> + * settings of the device.
>>> + */
>>> +char *msi_remap = "NULL";
>> Looks like this should be static?  And using "NULL" (as opposed to
>> something like the empty string "") suggests some intrinsic meaning of
>> "NULL", but I think there is no intrinsic meaning and the only point
>> is that "NULL" doesn't match either "on" or "off".
>>
> The "static" one is better, I will add it.
> Initial parameters with "NULL" just aim to mismatch "on" or "off". Do
> you prefer to init it without the default string?

Please address Bjorn's comments. Thank you.

Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>