* [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id @ 2018-10-08 20:10 Kirti Wankhede 2018-10-10 20:38 ` Alex Williamson 2018-10-17 9:05 ` Christoph Hellwig 0 siblings, 2 replies; 6+ messages in thread From: Kirti Wankhede @ 2018-10-08 20:10 UTC (permalink / raw) To: alex.williamson, eskultet, kvm, libvir-list; +Cc: Kirti Wankhede, cjia Generally a single instance of mdev device, a share of physical device, is assigned to user space application or a VM. There are cases when multiple instances of mdev devices of same or different types are required by user space application or VM. For example in case of vGPU, multiple mdev devices of type which represents whole GPU can be assigned to one instance of application or VM. All types of mdev devices may not support assigning multiple mdev devices to a user space application. In that case vendor driver can fail open() call of mdev device. But there is no way to know User space application to about the configuration supported by vendor driver. To expose supported configuration, vendor driver should add 'single_usage_restriction' attribute to type-id directory. Returning Y for this attribute indicates vendor driver has restriction of single mdev device of particular <type-id> assigned to one user space application. Returning N indicates that multiple mdev devices of particular <type-id> can be assigned to one user space application. User space application should read if 'single_usage_restriction' attibute is present in <type-id> directory of all mdev devices which are going to be used. If all read N then user space application can proceed with multiple mdev devices. This is optional and readonly attribute. Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com> Reviewed-by: Neo Jia <cjia@nvidia.com> --- Documentation/ABI/testing/sysfs-bus-vfio-mdev | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-vfio-mdev b/Documentation/ABI/testing/sysfs-bus-vfio-mdev index 452dbe39270e..3aca352a70e5 100644 --- a/Documentation/ABI/testing/sysfs-bus-vfio-mdev +++ b/Documentation/ABI/testing/sysfs-bus-vfio-mdev @@ -85,6 +85,22 @@ Users: a particular <type-id> that can help in understanding the features provided by that type of mediated device. +What: /sys/.../mdev_supported_types/<type-id>/single_usage_restriction +Date: October 2018 +Contact: Kirti Wankhede <kwankhede@nvidia.com> +Description: + Reading this attribute will return Y or N. Returning Y indicates + vendor driver has restriction of single mdev device of this + particular <type-id> assigned to one user space application. + Returning N indicates that multiple mdev devices of particular + <type-id> can be assigned to one user space application. + This is optional and readonly attribute. +Users: + User space application should read if 'single_usage_restriction' + attibute is present in <type-id> directory of all mdev devices + which are going to be used. If all read N then user space + application can proceed with multiple mdev devices. + What: /sys/.../<device>/<UUID>/ Date: October 2016 Contact: Kirti Wankhede <kwankhede@nvidia.com> -- 2.7.0 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id 2018-10-08 20:10 [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id Kirti Wankhede @ 2018-10-10 20:38 ` Alex Williamson 2018-10-10 23:22 ` Tian, Kevin 2018-10-17 9:05 ` Christoph Hellwig 1 sibling, 1 reply; 6+ messages in thread From: Alex Williamson @ 2018-10-10 20:38 UTC (permalink / raw) To: Kirti Wankhede; +Cc: libvir-list, intel-gvt-dev, cjia, kvm, eskultet On Tue, 9 Oct 2018 01:40:17 +0530 Kirti Wankhede <kwankhede@nvidia.com> wrote: > Generally a single instance of mdev device, a share of physical device, is > assigned to user space application or a VM. There are cases when multiple > instances of mdev devices of same or different types are required by user > space application or VM. For example in case of vGPU, multiple mdev devices > of type which represents whole GPU can be assigned to one instance of > application or VM. > > All types of mdev devices may not support assigning multiple mdev devices > to a user space application. In that case vendor driver can fail open() > call of mdev device. But there is no way to know User space application to > about the configuration supported by vendor driver. > > To expose supported configuration, vendor driver should add > 'single_usage_restriction' attribute to type-id directory. Returning Y for > this attribute indicates vendor driver has restriction of single mdev > device of particular <type-id> assigned to one user space application. > Returning N indicates that multiple mdev devices of particular <type-id> > can be assigned to one user space application. > > User space application should read if 'single_usage_restriction' attibute > is present in <type-id> directory of all mdev devices which are going to be > used. If all read N then user space application can proceed with multiple > mdev devices. > > This is optional and readonly attribute. > > Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com> > Reviewed-by: Neo Jia <cjia@nvidia.com> > --- > Documentation/ABI/testing/sysfs-bus-vfio-mdev | 16 ++++++++++++++++ > 1 file changed, 16 insertions(+) > > diff --git a/Documentation/ABI/testing/sysfs-bus-vfio-mdev b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > index 452dbe39270e..3aca352a70e5 100644 > --- a/Documentation/ABI/testing/sysfs-bus-vfio-mdev > +++ b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > @@ -85,6 +85,22 @@ Users: > a particular <type-id> that can help in understanding the > features provided by that type of mediated device. > > +What: /sys/.../mdev_supported_types/<type-id>/single_usage_restriction > +Date: October 2018 > +Contact: Kirti Wankhede <kwankhede@nvidia.com> > +Description: > + Reading this attribute will return Y or N. Returning Y indicates > + vendor driver has restriction of single mdev device of this > + particular <type-id> assigned to one user space application. > + Returning N indicates that multiple mdev devices of particular > + <type-id> can be assigned to one user space application. > + This is optional and readonly attribute. > +Users: > + User space application should read if 'single_usage_restriction' > + attibute is present in <type-id> directory of all mdev devices > + which are going to be used. If all read N then user space > + application can proceed with multiple mdev devices. But we don't say what userspace should do when this optional attribute is not present. Do we know of any cases other than the NVIDIA GRID vGPU drivers that have this restriction? Intel folks, are multiple GVT-g mdevs currently allowed in a VM? I don't think the libvirt algorithm is going to be as simple as suggested here and we should probably understand what it really needs to be. There's also a scope issue that's unclear here, the verbiage above suggests that I can't combine a 'single_usage_restriction=Y' mdev with any other mdev, but clearly there's no dependency between adding both an NVIDIA and Intel vGPU to the same VM, right? Or NVIDIA and any of the mdev sample drivers. The restriction is across mdev types and parent devices in this case, but I think it stops at the vendor driver, right? How do we make that more clear, both in wording and perhaps implicit in the attribute itself? Thanks, Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id 2018-10-10 20:38 ` Alex Williamson @ 2018-10-10 23:22 ` Tian, Kevin 2018-10-11 1:14 ` Zhenyu Wang 0 siblings, 1 reply; 6+ messages in thread From: Tian, Kevin @ 2018-10-10 23:22 UTC (permalink / raw) To: Alex Williamson, Kirti Wankhede Cc: cjia@nvidia.com, kvm@vger.kernel.org, Wang, Zhenyu Z, eskultet@redhat.com, libvir-list@redhat.com, intel-gvt-dev@lists.freedesktop.org > From: Alex Williamson > Sent: Thursday, October 11, 2018 4:39 AM > > On Tue, 9 Oct 2018 01:40:17 +0530 > Kirti Wankhede <kwankhede@nvidia.com> wrote: > > > Generally a single instance of mdev device, a share of physical device, is > > assigned to user space application or a VM. There are cases when > multiple > > instances of mdev devices of same or different types are required by user > > space application or VM. For example in case of vGPU, multiple mdev > devices > > of type which represents whole GPU can be assigned to one instance of > > application or VM. > > > > All types of mdev devices may not support assigning multiple mdev > devices > > to a user space application. In that case vendor driver can fail open() > > call of mdev device. But there is no way to know User space application to > > about the configuration supported by vendor driver. > > > > To expose supported configuration, vendor driver should add > > 'single_usage_restriction' attribute to type-id directory. Returning Y for > > this attribute indicates vendor driver has restriction of single mdev > > device of particular <type-id> assigned to one user space application. > > Returning N indicates that multiple mdev devices of particular <type-id> > > can be assigned to one user space application. > > > > User space application should read if 'single_usage_restriction' attibute > > is present in <type-id> directory of all mdev devices which are going to be > > used. If all read N then user space application can proceed with multiple > > mdev devices. > > > > This is optional and readonly attribute. > > > > Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com> > > Reviewed-by: Neo Jia <cjia@nvidia.com> > > --- > > Documentation/ABI/testing/sysfs-bus-vfio-mdev | 16 ++++++++++++++++ > > 1 file changed, 16 insertions(+) > > > > diff --git a/Documentation/ABI/testing/sysfs-bus-vfio-mdev > b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > index 452dbe39270e..3aca352a70e5 100644 > > --- a/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > +++ b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > @@ -85,6 +85,22 @@ Users: > > a particular <type-id> that can help in understanding the > > features provided by that type of mediated device. > > > > +What: /sys/.../mdev_supported_types/<type- > id>/single_usage_restriction > > +Date: October 2018 > > +Contact: Kirti Wankhede <kwankhede@nvidia.com> > > +Description: > > + Reading this attribute will return Y or N. Returning Y > indicates > > + vendor driver has restriction of single mdev device of this > > + particular <type-id> assigned to one user space application. > > + Returning N indicates that multiple mdev devices of > particular > > + <type-id> can be assigned to one user space application. > > + This is optional and readonly attribute. > > +Users: > > + User space application should read if > 'single_usage_restriction' > > + attibute is present in <type-id> directory of all mdev devices > > + which are going to be used. If all read N then user space > > + application can proceed with multiple mdev devices. > > But we don't say what userspace should do when this optional attribute > is not present. Do we know of any cases other than the NVIDIA GRID > vGPU drivers that have this restriction? Intel folks, are multiple > GVT-g mdevs currently allowed in a VM? I don't think the libvirt > algorithm is going to be as simple as suggested here and we should > probably understand what it really needs to be. technically I don't see a restriction in GVT-g side, i.e. multiple GVT-g mdevs can be assigned to same VM. But the fact is that Intel GPU is integrated thus just one per platform. Then guest i915 driver may have problem to operate multiple vGPUs if with some assumption on integrated part. I don't think we verified such configuration. Zhenyu? > > There's also a scope issue that's unclear here, the verbiage above > suggests that I can't combine a 'single_usage_restriction=Y' mdev with > any other mdev, but clearly there's no dependency between adding both > an NVIDIA and Intel vGPU to the same VM, right? Or NVIDIA and any of > the mdev sample drivers. The restriction is across mdev types and > parent devices in this case, but I think it stops at the vendor driver, > right? How do we make that more clear, both in wording and perhaps > implicit in the attribute itself? Thanks, > > Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id 2018-10-10 23:22 ` Tian, Kevin @ 2018-10-11 1:14 ` Zhenyu Wang 2018-10-11 2:08 ` Alex Williamson 0 siblings, 1 reply; 6+ messages in thread From: Zhenyu Wang @ 2018-10-11 1:14 UTC (permalink / raw) To: Tian, Kevin Cc: cjia@nvidia.com, kvm@vger.kernel.org, eskultet@redhat.com, libvir-list@redhat.com, Kirti Wankhede, intel-gvt-dev@lists.freedesktop.org [-- Attachment #1.1: Type: text/plain, Size: 4443 bytes --] On 2018.10.10 23:22:20 +0000, Tian, Kevin wrote: > > From: Alex Williamson > > Sent: Thursday, October 11, 2018 4:39 AM > > > > On Tue, 9 Oct 2018 01:40:17 +0530 > > Kirti Wankhede <kwankhede@nvidia.com> wrote: > > > > > Generally a single instance of mdev device, a share of physical device, is > > > assigned to user space application or a VM. There are cases when > > multiple > > > instances of mdev devices of same or different types are required by user > > > space application or VM. For example in case of vGPU, multiple mdev > > devices > > > of type which represents whole GPU can be assigned to one instance of > > > application or VM. > > > > > > All types of mdev devices may not support assigning multiple mdev > > devices > > > to a user space application. In that case vendor driver can fail open() > > > call of mdev device. But there is no way to know User space application to > > > about the configuration supported by vendor driver. > > > > > > To expose supported configuration, vendor driver should add > > > 'single_usage_restriction' attribute to type-id directory. Returning Y for > > > this attribute indicates vendor driver has restriction of single mdev > > > device of particular <type-id> assigned to one user space application. > > > Returning N indicates that multiple mdev devices of particular <type-id> > > > can be assigned to one user space application. > > > > > > User space application should read if 'single_usage_restriction' attibute > > > is present in <type-id> directory of all mdev devices which are going to be > > > used. If all read N then user space application can proceed with multiple > > > mdev devices. > > > > > > This is optional and readonly attribute. > > > > > > Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com> > > > Reviewed-by: Neo Jia <cjia@nvidia.com> > > > --- > > > Documentation/ABI/testing/sysfs-bus-vfio-mdev | 16 ++++++++++++++++ > > > 1 file changed, 16 insertions(+) > > > > > > diff --git a/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > > index 452dbe39270e..3aca352a70e5 100644 > > > --- a/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > > +++ b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > > @@ -85,6 +85,22 @@ Users: > > > a particular <type-id> that can help in understanding the > > > features provided by that type of mediated device. > > > > > > +What: /sys/.../mdev_supported_types/<type- > > id>/single_usage_restriction > > > +Date: October 2018 > > > +Contact: Kirti Wankhede <kwankhede@nvidia.com> > > > +Description: > > > + Reading this attribute will return Y or N. Returning Y > > indicates > > > + vendor driver has restriction of single mdev device of this > > > + particular <type-id> assigned to one user space application. > > > + Returning N indicates that multiple mdev devices of > > particular > > > + <type-id> can be assigned to one user space application. > > > + This is optional and readonly attribute. > > > +Users: > > > + User space application should read if > > 'single_usage_restriction' > > > + attibute is present in <type-id> directory of all mdev devices > > > + which are going to be used. If all read N then user space > > > + application can proceed with multiple mdev devices. > > > > But we don't say what userspace should do when this optional attribute > > is not present. Do we know of any cases other than the NVIDIA GRID > > vGPU drivers that have this restriction? Intel folks, are multiple > > GVT-g mdevs currently allowed in a VM? I don't think the libvirt > > algorithm is going to be as simple as suggested here and we should > > probably understand what it really needs to be. > > technically I don't see a restriction in GVT-g side, i.e. multiple GVT-g > mdevs can be assigned to same VM. But the fact is that Intel GPU is > integrated thus just one per platform. Then guest i915 driver may have > problem to operate multiple vGPUs if with some assumption on integrated > part. I don't think we verified such configuration. Zhenyu? > yeah, as current intel gpu is single device, even guest i915 driver could operate but user space gfx driver is not ready for multiple devices, so won't be useful now. -- Open Source Technology Center, Intel ltd. $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827 [-- Attachment #1.2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id 2018-10-11 1:14 ` Zhenyu Wang @ 2018-10-11 2:08 ` Alex Williamson 0 siblings, 0 replies; 6+ messages in thread From: Alex Williamson @ 2018-10-11 2:08 UTC (permalink / raw) To: Zhenyu Wang Cc: Tian, Kevin, cjia@nvidia.com, kvm@vger.kernel.org, eskultet@redhat.com, libvir-list@redhat.com, Kirti Wankhede, intel-gvt-dev@lists.freedesktop.org On Thu, 11 Oct 2018 09:14:22 +0800 Zhenyu Wang <zhenyuw@linux.intel.com> wrote: > On 2018.10.10 23:22:20 +0000, Tian, Kevin wrote: > > > From: Alex Williamson > > > Sent: Thursday, October 11, 2018 4:39 AM > > > > > > On Tue, 9 Oct 2018 01:40:17 +0530 > > > Kirti Wankhede <kwankhede@nvidia.com> wrote: > > > > > > > Generally a single instance of mdev device, a share of physical device, is > > > > assigned to user space application or a VM. There are cases when > > > multiple > > > > instances of mdev devices of same or different types are required by user > > > > space application or VM. For example in case of vGPU, multiple mdev > > > devices > > > > of type which represents whole GPU can be assigned to one instance of > > > > application or VM. > > > > > > > > All types of mdev devices may not support assigning multiple mdev > > > devices > > > > to a user space application. In that case vendor driver can fail open() > > > > call of mdev device. But there is no way to know User space application to > > > > about the configuration supported by vendor driver. > > > > > > > > To expose supported configuration, vendor driver should add > > > > 'single_usage_restriction' attribute to type-id directory. Returning Y for > > > > this attribute indicates vendor driver has restriction of single mdev > > > > device of particular <type-id> assigned to one user space application. > > > > Returning N indicates that multiple mdev devices of particular <type-id> > > > > can be assigned to one user space application. > > > > > > > > User space application should read if 'single_usage_restriction' attibute > > > > is present in <type-id> directory of all mdev devices which are going to be > > > > used. If all read N then user space application can proceed with multiple > > > > mdev devices. > > > > > > > > This is optional and readonly attribute. > > > > > > > > Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com> > > > > Reviewed-by: Neo Jia <cjia@nvidia.com> > > > > --- > > > > Documentation/ABI/testing/sysfs-bus-vfio-mdev | 16 ++++++++++++++++ > > > > 1 file changed, 16 insertions(+) > > > > > > > > diff --git a/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > > b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > > > index 452dbe39270e..3aca352a70e5 100644 > > > > --- a/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > > > +++ b/Documentation/ABI/testing/sysfs-bus-vfio-mdev > > > > @@ -85,6 +85,22 @@ Users: > > > > a particular <type-id> that can help in understanding the > > > > features provided by that type of mediated device. > > > > > > > > +What: /sys/.../mdev_supported_types/<type- > > > id>/single_usage_restriction > > > > +Date: October 2018 > > > > +Contact: Kirti Wankhede <kwankhede@nvidia.com> > > > > +Description: > > > > + Reading this attribute will return Y or N. Returning Y > > > indicates > > > > + vendor driver has restriction of single mdev device of this > > > > + particular <type-id> assigned to one user space application. > > > > + Returning N indicates that multiple mdev devices of > > > particular > > > > + <type-id> can be assigned to one user space application. > > > > + This is optional and readonly attribute. > > > > +Users: > > > > + User space application should read if > > > 'single_usage_restriction' > > > > + attibute is present in <type-id> directory of all mdev devices > > > > + which are going to be used. If all read N then user space > > > > + application can proceed with multiple mdev devices. > > > > > > But we don't say what userspace should do when this optional attribute > > > is not present. Do we know of any cases other than the NVIDIA GRID > > > vGPU drivers that have this restriction? Intel folks, are multiple > > > GVT-g mdevs currently allowed in a VM? I don't think the libvirt > > > algorithm is going to be as simple as suggested here and we should > > > probably understand what it really needs to be. > > > > technically I don't see a restriction in GVT-g side, i.e. multiple GVT-g > > mdevs can be assigned to same VM. But the fact is that Intel GPU is > > integrated thus just one per platform. Then guest i915 driver may have > > problem to operate multiple vGPUs if with some assumption on integrated > > part. I don't think we verified such configuration. Zhenyu? > > > > yeah, as current intel gpu is single device, even guest i915 driver could > operate but user space gfx driver is not ready for multiple devices, so won't > be useful now. The restriction sounds different here though, in the case of NVIDIA it's the mdev vendor driver with the restriction, with Intel I think I'm hearing that it's the guest graphics driver that's the restriction. I don't think we'd want to place an artificial restriction at the mdev layer for a guest driver deficiency that could theoretically be fixed independently from the host mdev vendor driver. Thanks, Alex ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id 2018-10-08 20:10 [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id Kirti Wankhede 2018-10-10 20:38 ` Alex Williamson @ 2018-10-17 9:05 ` Christoph Hellwig 1 sibling, 0 replies; 6+ messages in thread From: Christoph Hellwig @ 2018-10-17 9:05 UTC (permalink / raw) To: Kirti Wankhede; +Cc: libvir-list, cjia, kvm, eskultet I don't see any patch actually adding (nevermind using) the attibute, so something seems wrong here. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-10-17 9:05 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-10-08 20:10 [libvirt] [PATCH v2 1/1] Add attribute single_usage_restriction for mdev type-id Kirti Wankhede 2018-10-10 20:38 ` Alex Williamson 2018-10-10 23:22 ` Tian, Kevin 2018-10-11 1:14 ` Zhenyu Wang 2018-10-11 2:08 ` Alex Williamson 2018-10-17 9:05 ` Christoph Hellwig
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox