[PATCH v4] PCI: vmd: Make vmd_dev::cfg_lock a raw_spinlock_t.

[PATCH v4] PCI: vmd: Make vmd_dev::cfg_lock a raw_spinlock_t.

[Sebastian Andrzej Siewior]

From: Ryo Takakura <ryotkkr98@gmail.com>

The access to the PCI config space via pci_ops::read and pci_ops::write
is a low-level hardware access. The functions can be accessed with
disabled interrupts even on PREEMPT_RT. The pci_lock has been made a
raw_spinlock_t for this purpose. A spinlock_t becomes a sleeping lock on
PREEMPT_RT can not be acquired with disabled interrupts.
The vmd_dev::cfg_lock is accessed in the same context as the pci_lock.

Make vmd_dev::cfg_lock a raw_spinlock_t.

[bigeasy: Reword commit message.]

Signed-off-by: Ryo Takakura <ryotkkr98@gmail.com>
Tested-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
Acked-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
Changes since v3 https://lore.kernel.org/all/20241219014549.24427-1-ryotkkr98@gmail.com/
- Repost with updated changelog.

Changes since v2 https://lore.kernel.org/lkml/20241218115951.83062-1-ryotkkr98@gmail.com/
- In case if CONFIG_PCI_LOCKLESS_CONFIG is set, vmd_pci_read/write()
  still neeed cfg_lock for their serialization. So instead of removing
  it, convert it to raw spinlock.

v1: https://lore.kernel.org/lkml/20241215141321.383144-1-ryotkkr98@gmail.com/

 drivers/pci/controller/vmd.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 9d9596947350f..94ceec50a2b94 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -125,7 +125,7 @@ struct vmd_irq_list {
 struct vmd_dev {
 	struct pci_dev		*dev;
 
-	spinlock_t		cfg_lock;
+	raw_spinlock_t		cfg_lock;
 	void __iomem		*cfgbar;
 
 	int msix_count;
@@ -391,7 +391,7 @@ static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
 	if (!addr)
 		return -EFAULT;
 
-	spin_lock_irqsave(&vmd->cfg_lock, flags);
+	raw_spin_lock_irqsave(&vmd->cfg_lock, flags);
 	switch (len) {
 	case 1:
 		*value = readb(addr);
@@ -406,7 +406,7 @@ static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
 		ret = -EINVAL;
 		break;
 	}
-	spin_unlock_irqrestore(&vmd->cfg_lock, flags);
+	raw_spin_unlock_irqrestore(&vmd->cfg_lock, flags);
 	return ret;
 }
 
@@ -426,7 +426,7 @@ static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
 	if (!addr)
 		return -EFAULT;
 
-	spin_lock_irqsave(&vmd->cfg_lock, flags);
+	raw_spin_lock_irqsave(&vmd->cfg_lock, flags);
 	switch (len) {
 	case 1:
 		writeb(value, addr);
@@ -444,7 +444,7 @@ static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
 		ret = -EINVAL;
 		break;
 	}
-	spin_unlock_irqrestore(&vmd->cfg_lock, flags);
+	raw_spin_unlock_irqrestore(&vmd->cfg_lock, flags);
 	return ret;
 }
 
@@ -1009,7 +1009,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
 	if (features & VMD_FEAT_OFFSET_FIRST_VECTOR)
 		vmd->first_vec = 1;
 
-	spin_lock_init(&vmd->cfg_lock);
+	raw_spin_lock_init(&vmd->cfg_lock);
 	pci_set_drvdata(dev, vmd);
 	err = vmd_enable_domain(vmd, features);
 	if (err)
-- 
2.47.2

Re: [PATCH v4] PCI: vmd: Make vmd_dev::cfg_lock a raw_spinlock_t.

[Krzysztof Wilczyński]

Hello,

> The access to the PCI config space via pci_ops::read and pci_ops::write
> is a low-level hardware access. The functions can be accessed with
> disabled interrupts even on PREEMPT_RT. The pci_lock has been made a
> raw_spinlock_t for this purpose. A spinlock_t becomes a sleeping lock on
> PREEMPT_RT can not be acquired with disabled interrupts.
> The vmd_dev::cfg_lock is accessed in the same context as the pci_lock.
> 
> Make vmd_dev::cfg_lock a raw_spinlock_t.

Applied to controller/vmd, thank you!

	Krzysztof

Re: [PATCH v4] PCI: vmd: Make vmd_dev::cfg_lock a raw_spinlock_t.

[Bjorn Helgaas]

On Tue, Feb 18, 2025 at 09:08:30AM +0100, Sebastian Andrzej Siewior wrote:
> From: Ryo Takakura <ryotkkr98@gmail.com>
> 
> The access to the PCI config space via pci_ops::read and pci_ops::write
> is a low-level hardware access. The functions can be accessed with
> disabled interrupts even on PREEMPT_RT. The pci_lock has been made a
> raw_spinlock_t for this purpose. A spinlock_t becomes a sleeping lock on
> PREEMPT_RT can not be acquired with disabled interrupts.

I think this is missing a word or two and should say:

  A spinlock_t becomes a sleeping lock on PREEMPT_RT, so it cannot be
  acquired with disabled interrupts.

pci_ops.read() is called while holding the non-sleepable
raw_spinlock_t pci_lock, so pci_ops.read() must also be non-sleepable.

It's not really relevant that "pci_ops.read() can be called with
disabled interrupts even on PREEMPT_RT".  It can *always* be called
with interrupts disabled.

> The vmd_dev::cfg_lock is accessed in the same context as the pci_lock.
> 
> Make vmd_dev::cfg_lock a raw_spinlock_t.
> 
> [bigeasy: Reword commit message.]
> 
> Signed-off-by: Ryo Takakura <ryotkkr98@gmail.com>
> Tested-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
> Acked-by: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> Changes since v3 https://lore.kernel.org/all/20241219014549.24427-1-ryotkkr98@gmail.com/
> - Repost with updated changelog.
> 
> Changes since v2 https://lore.kernel.org/lkml/20241218115951.83062-1-ryotkkr98@gmail.com/
> - In case if CONFIG_PCI_LOCKLESS_CONFIG is set, vmd_pci_read/write()
>   still neeed cfg_lock for their serialization. So instead of removing
>   it, convert it to raw spinlock.
> 
> v1: https://lore.kernel.org/lkml/20241215141321.383144-1-ryotkkr98@gmail.com/
> 
>  drivers/pci/controller/vmd.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index 9d9596947350f..94ceec50a2b94 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -125,7 +125,7 @@ struct vmd_irq_list {
>  struct vmd_dev {
>  	struct pci_dev		*dev;
>  
> -	spinlock_t		cfg_lock;
> +	raw_spinlock_t		cfg_lock;
>  	void __iomem		*cfgbar;
>  
>  	int msix_count;
> @@ -391,7 +391,7 @@ static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
>  	if (!addr)
>  		return -EFAULT;
>  
> -	spin_lock_irqsave(&vmd->cfg_lock, flags);
> +	raw_spin_lock_irqsave(&vmd->cfg_lock, flags);
>  	switch (len) {
>  	case 1:
>  		*value = readb(addr);
> @@ -406,7 +406,7 @@ static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
>  		ret = -EINVAL;
>  		break;
>  	}
> -	spin_unlock_irqrestore(&vmd->cfg_lock, flags);
> +	raw_spin_unlock_irqrestore(&vmd->cfg_lock, flags);
>  	return ret;
>  }
>  
> @@ -426,7 +426,7 @@ static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
>  	if (!addr)
>  		return -EFAULT;
>  
> -	spin_lock_irqsave(&vmd->cfg_lock, flags);
> +	raw_spin_lock_irqsave(&vmd->cfg_lock, flags);
>  	switch (len) {
>  	case 1:
>  		writeb(value, addr);
> @@ -444,7 +444,7 @@ static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
>  		ret = -EINVAL;
>  		break;
>  	}
> -	spin_unlock_irqrestore(&vmd->cfg_lock, flags);
> +	raw_spin_unlock_irqrestore(&vmd->cfg_lock, flags);
>  	return ret;
>  }
>  
> @@ -1009,7 +1009,7 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
>  	if (features & VMD_FEAT_OFFSET_FIRST_VECTOR)
>  		vmd->first_vec = 1;
>  
> -	spin_lock_init(&vmd->cfg_lock);
> +	raw_spin_lock_init(&vmd->cfg_lock);
>  	pci_set_drvdata(dev, vmd);
>  	err = vmd_enable_domain(vmd, features);
>  	if (err)
> -- 
> 2.47.2
> 

Re: [PATCH v4] PCI: vmd: Make vmd_dev::cfg_lock a raw_spinlock_t.

[Krzysztof Wilczyński]

Hello,

> > The access to the PCI config space via pci_ops::read and pci_ops::write
> > is a low-level hardware access. The functions can be accessed with
> > disabled interrupts even on PREEMPT_RT. The pci_lock has been made a
> > raw_spinlock_t for this purpose. A spinlock_t becomes a sleeping lock on
> > PREEMPT_RT can not be acquired with disabled interrupts.
> 
> I think this is missing a word or two and should say:
> 
>   A spinlock_t becomes a sleeping lock on PREEMPT_RT, so it cannot be
>   acquired with disabled interrupts.

I changed the commit log directly on the relevant branch.  Thank you!

	Krzysztof