[Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Alex Williamson]

When using VFIO to assign a device to a guest, we want to make sure
the device is quiesced on VM reset to stop all DMA within the guest
mapped memory.  Add an ioctl which just calls pci_reset_function()
and returns whether it succeeds.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---

We've recently needed to add this functionality for current KVM
based device assignment, VFIO should provide a way to do this too.
An example of it being used in the Qemu VFIO driver can be found
here:

https://github.com/awilliam/qemu-vfio/blob/vfio/hw/vfio.c

 drivers/vfio/vfio_main.c |    4 ++++
 include/linux/vfio.h     |    3 +++
 2 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
index 7e427fc..b9bb692 100644
--- a/drivers/vfio/vfio_main.c
+++ b/drivers/vfio/vfio_main.c
@@ -490,6 +490,10 @@ static long vfio_unl_ioctl(struct file *filep,
 		ret = vfio_irq_eoi_eventfd(vdev, fd);
 		break;
 
+	case VFIO_RESET_FUNCTION:
+		ret = pci_reset_function(vdev->pdev);
+		break;
+
 	default:
 		return -EINVAL;
 	}
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index f07d8fe..47d9bb9 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -215,6 +215,9 @@ struct vfio_dma_map {
 /* Re-enable INTx via eventfd */
 #define	VFIO_IRQ_EOI_EVENTFD	_IOW(';', 110, int)
 
+/* Reset PCI function */
+#define VFIO_RESET_FUNCTION	_IO(';', 111)
+
 /*
  * Reads, writes, and mmaps determine which PCI BAR (or config space)
  * from the high level bits of the file offset

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Randy Dunlap]

On Tue, 19 Apr 2011 14:32:59 -0600 Alex Williamson wrote:

> When using VFIO to assign a device to a guest, we want to make sure
> the device is quiesced on VM reset to stop all DMA within the guest
> mapped memory.  Add an ioctl which just calls pci_reset_function()
> and returns whether it succeeds.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> ---
> 
> We've recently needed to add this functionality for current KVM
> based device assignment, VFIO should provide a way to do this too.
> An example of it being used in the Qemu VFIO driver can be found
> here:
> 
> https://github.com/awilliam/qemu-vfio/blob/vfio/hw/vfio.c
> 
>  drivers/vfio/vfio_main.c |    4 ++++
>  include/linux/vfio.h     |    3 +++
>  2 files changed, 7 insertions(+), 0 deletions(-)
> 
> diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
> index 7e427fc..b9bb692 100644
> --- a/drivers/vfio/vfio_main.c
> +++ b/drivers/vfio/vfio_main.c
> @@ -490,6 +490,10 @@ static long vfio_unl_ioctl(struct file *filep,
>  		ret = vfio_irq_eoi_eventfd(vdev, fd);
>  		break;
>  
> +	case VFIO_RESET_FUNCTION:
> +		ret = pci_reset_function(vdev->pdev);
> +		break;
> +
>  	default:
>  		return -EINVAL;
>  	}
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index f07d8fe..47d9bb9 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -215,6 +215,9 @@ struct vfio_dma_map {
>  /* Re-enable INTx via eventfd */
>  #define	VFIO_IRQ_EOI_EVENTFD	_IOW(';', 110, int)
>  
> +/* Reset PCI function */
> +#define VFIO_RESET_FUNCTION	_IO(';', 111)
> +
>  /*
>   * Reads, writes, and mmaps determine which PCI BAR (or config space)
>   * from the high level bits of the file offset

I can't find include/linux/vfio.h in linux-next or mainline git, but
ioctls need to be documented in Documentation/ioctl/ioctl-number.txt....

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Chris Wright]

* Randy Dunlap (rdunlap@xenotime.net) wrote:
> I can't find include/linux/vfio.h in linux-next or mainline git, but
> ioctls need to be documented in Documentation/ioctl/ioctl-number.txt....

It is in the full patchset: https://github.com/pugs/vfio-linux-2.6

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Chris Wright]

* Alex Williamson (alex.williamson@redhat.com) wrote:
> When using VFIO to assign a device to a guest, we want to make sure
> the device is quiesced on VM reset to stop all DMA within the guest
> mapped memory.  Add an ioctl which just calls pci_reset_function()
> and returns whether it succeeds.

Shouldn't there be a reset when binding/unbinding vfio to/from a pci
device?

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Alex Williamson]

On Tue, 2011-04-19 at 15:07 -0700, Chris Wright wrote:
> * Alex Williamson (alex.williamson@redhat.com) wrote:
> > When using VFIO to assign a device to a guest, we want to make sure
> > the device is quiesced on VM reset to stop all DMA within the guest
> > mapped memory.  Add an ioctl which just calls pci_reset_function()
> > and returns whether it succeeds.
> 
> Shouldn't there be a reset when binding/unbinding vfio to/from a pci
> device?

There's already one when the /dev/vfioX file is opened, we should add
another on release, and probably add the same PCI save state store/load
that I'm proposing for KVM across those.  Thanks,

Alex

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Chris Wright]

* Alex Williamson (alex.williamson@redhat.com) wrote:
> On Tue, 2011-04-19 at 15:07 -0700, Chris Wright wrote:
> > * Alex Williamson (alex.williamson@redhat.com) wrote:
> > > When using VFIO to assign a device to a guest, we want to make sure
> > > the device is quiesced on VM reset to stop all DMA within the guest
> > > mapped memory.  Add an ioctl which just calls pci_reset_function()
> > > and returns whether it succeeds.
> > 
> > Shouldn't there be a reset when binding/unbinding vfio to/from a pci
> > device?
> 
> There's already one when the /dev/vfioX file is opened, we should add
> another on release, and probably add the same PCI save state store/load
> that I'm proposing for KVM across those.  Thanks,

Hmm, I looked and didn't see it, hence the question.

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Alex Williamson]

On Tue, 2011-04-19 at 15:26 -0700, Chris Wright wrote:
> * Alex Williamson (alex.williamson@redhat.com) wrote:
> > On Tue, 2011-04-19 at 15:07 -0700, Chris Wright wrote:
> > > * Alex Williamson (alex.williamson@redhat.com) wrote:
> > > > When using VFIO to assign a device to a guest, we want to make sure
> > > > the device is quiesced on VM reset to stop all DMA within the guest
> > > > mapped memory.  Add an ioctl which just calls pci_reset_function()
> > > > and returns whether it succeeds.
> > > 
> > > Shouldn't there be a reset when binding/unbinding vfio to/from a pci
> > > device?
> > 
> > There's already one when the /dev/vfioX file is opened, we should add
> > another on release, and probably add the same PCI save state store/load
> > that I'm proposing for KVM across those.  Thanks,
> 
> Hmm, I looked and didn't see it, hence the question.

vfio_open() -> pci_reset_function()
https://github.com/pugs/vfio-linux-2.6/blob/vfio/drivers/vfio/vfio_main.c

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Chris Wright]

* Alex Williamson (alex.williamson@redhat.com) wrote:
> On Tue, 2011-04-19 at 15:26 -0700, Chris Wright wrote:
> > * Alex Williamson (alex.williamson@redhat.com) wrote:
> > > On Tue, 2011-04-19 at 15:07 -0700, Chris Wright wrote:
> > > > * Alex Williamson (alex.williamson@redhat.com) wrote:
> > > > > When using VFIO to assign a device to a guest, we want to make sure
> > > > > the device is quiesced on VM reset to stop all DMA within the guest
> > > > > mapped memory.  Add an ioctl which just calls pci_reset_function()
> > > > > and returns whether it succeeds.
> > > > 
> > > > Shouldn't there be a reset when binding/unbinding vfio to/from a pci
> > > > device?
> > > 
> > > There's already one when the /dev/vfioX file is opened, we should add
> > > another on release, and probably add the same PCI save state store/load
> > > that I'm proposing for KVM across those.  Thanks,
> > 
> > Hmm, I looked and didn't see it, hence the question.
> 
> vfio_open() -> pci_reset_function()
> https://github.com/pugs/vfio-linux-2.6/blob/vfio/drivers/vfio/vfio_main.c

Got it, thanks Alex.

Re: [Qemu-devel] [PATCH] vfio: Add an ioctl to reset the device

[Tom Lyon]

On Tuesday, April 19, 2011 01:32:59 pm Alex Williamson wrote:
> When using VFIO to assign a device to a guest, we want to make sure
> the device is quiesced on VM reset to stop all DMA within the guest
> mapped memory.  Add an ioctl which just calls pci_reset_function()
> and returns whether it succeeds.
> 
> Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> ---
> 
> We've recently needed to add this functionality for current KVM
> based device assignment, VFIO should provide a way to do this too.
> An example of it being used in the Qemu VFIO driver can be found
> here:
> 
> https://github.com/awilliam/qemu-vfio/blob/vfio/hw/vfio.c
> 
>  drivers/vfio/vfio_main.c |    4 ++++
>  include/linux/vfio.h     |    3 +++
>  2 files changed, 7 insertions(+), 0 deletions(-)
> 
> diff --git a/drivers/vfio/vfio_main.c b/drivers/vfio/vfio_main.c
> index 7e427fc..b9bb692 100644
> --- a/drivers/vfio/vfio_main.c
> +++ b/drivers/vfio/vfio_main.c
> @@ -490,6 +490,10 @@ static long vfio_unl_ioctl(struct file *filep,
>  		ret = vfio_irq_eoi_eventfd(vdev, fd);
>  		break;
> 
> +	case VFIO_RESET_FUNCTION:
> +		ret = pci_reset_function(vdev->pdev);
> +		break;
> +
>  	default:
>  		return -EINVAL;
>  	}
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index f07d8fe..47d9bb9 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -215,6 +215,9 @@ struct vfio_dma_map {
>  /* Re-enable INTx via eventfd */
>  #define	VFIO_IRQ_EOI_EVENTFD	_IOW(';', 110, int)
> 
> +/* Reset PCI function */
> +#define VFIO_RESET_FUNCTION	_IO(';', 111)
> +
>  /*
>   * Reads, writes, and mmaps determine which PCI BAR (or config space)
>   * from the high level bits of the file offset

Applied.