* [PATCH 00/10] Embed struct vfio_device in all sub-structures
@ 2021-03-09 21:38 Jason Gunthorpe
2021-03-09 21:38 ` [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops Jason Gunthorpe
` (3 more replies)
0 siblings, 4 replies; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-09 21:38 UTC (permalink / raw)
To: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, kvm, Kirti Wankhede, linux-doc
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
Prologue
========
This series is part of a larger work that arose from the minor remark that
the mdev_parent_ops indirection shim is useless and complicates
things.
The entire project is about 70 patches broken into 5 subseries, each on a
theme:
#1 - (this series) Add type safety to the core VFIO
#2 - Add type safety to MDEV
The mdev transformation is involved, compiler assistance through actual
static type checking makes the transformation much more reliable, thus
the first two steps add most of the missing types.
#3 - Make all mdev drivers register directly with the core code,
delete vfio_mdev.c
#4 - Various minor tidies that arise from the above three series
#5 - Complete type annotations and remove unused code
A preview of the future series's is here:
https://github.com/jgunthorpe/linux/pull/3/commits
It turns out a bunch of stuff exists in the way it does because the
'struct vfio_device' was not obviously available in places that naturally
wanted it. Across the project the following APIs are deleted as reorg
removes all the users:
mdev_uuid()
mdev_dev()
mdev_get_drvdata()
mdev_set_drvdata()
struct mdev_parent_ops
vfio_iommu_group_get()
vfio_iommu_group_put(),
vfio_group_get_external_user_from_dev()
vfio_group_pin_pages()
vfio_group_unpin_pages()
vfio_group_get()
vfio_device_data()
The remaining vfio_device related APIs in mdev.h and vfio.h have correct,
specific, types instead of 'void *' or 'struct device *'.
This work is related to, but seperate from, Max's series to split
vfio_pci. When layered on this vfio_pci_core will use a similiar
container_of scheme and layer the ultimate end-driver with container_of
all the way back to a vfio_device. Types are explicit and natural to
understand through all the layers.
Further mdev and pci get a similiar design with a set of core code
supporting normal 'struct device_driver's that directly create
vfio_device's.
In essence vfio becomes close to a normal driver subsystem pattern with a
bunch of device drivers creating vfio_devices'
========
This series:
The main focus of this series is to make VFIO follow the normal kernel
convention of structure embedding for structure inheritance instead of
linking using a 'void *opaque'. Here we focus on moving the vfio_device to
be a member of every struct vfio_XX_device that is linked by a
vfio_add_group_dev().
In turn this allows 'struct vfio_device *' to be used everwhere, and the
public API out of vfio.c can be cleaned to remove places using 'struct
device *' and 'void *' as surrogates to refer to the device.
While this has the minor trade off of moving 'struct vfio_device' the
clarity of the design is worth it. I can speak directly to this idea, as
I've invested a fair amount of time carefully working backwards what all
the type-erased APIs are supposed to be and it is certainly not trivial or
intuitive.
When we get into mdev land things become even more inscrutable, and while
I now have a pretty clear picture, it was hard to obtain. I think this
agrees with the kernel style ideal of being explicit in typing and not
sacrificing clarity to create opaque structs.
After this series the general rules are:
- Any vfio_XX_device * can be obtained at no cost from a vfio_device *
using container_of(), and the reverse is possible by &XXdev->vdev
This is similar to how 'struct pci_device' and 'struct device' are
interrelated.
This allows 'device_data' to be completely removed from the vfio.c API.
- The drvdata for a struct device points at the vfio_XX_device that
belongs to the driver that was probed. drvdata is removed from the core
code, and only used as part of the implementation of the struct
device_driver.
- The lifetime of vfio_XX_device and vfio_device are identical, they are
the same memory.
This follows the existing model where vfio_del_group_dev() blocks until
all vfio_device_put()'s are completed. This in turn means the struct
device_driver remove() blocks, and thus under the driver_lock() a bound
driver must have a valid drvdata pointing at both vfio device
structs. A following series exploits this further.
Most vfio_XX_device structs have data that duplicates the 'struct
device *dev' member of vfio_device, a following series removes that
duplication too.
Jason
Jason Gunthorpe (10):
vfio: Simplify the lifetime logic for vfio_device
vfio: Split creation of a vfio_device into init and register ops
vfio/platform: Use vfio_init/register/unregister_group_dev
vfio/fsl-mc: Use vfio_init/register/unregister_group_dev
vfio/pci: Use vfio_init/register/unregister_group_dev
vfio/mdev: Use vfio_init/register/unregister_group_dev
vfio/mdev: Make to_mdev_device() into a static inline
vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of
'void *'
vfio/pci: Replace uses of vfio_device_data() with container_of
vfio: Remove device_data from the vfio bus driver API
Documentation/driver-api/vfio.rst | 48 ++--
drivers/vfio/fsl-mc/vfio_fsl_mc.c | 69 +++---
drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 1 +
drivers/vfio/mdev/mdev_private.h | 5 +-
drivers/vfio/mdev/vfio_mdev.c | 57 +++--
drivers/vfio/pci/vfio_pci.c | 109 +++++----
drivers/vfio/pci/vfio_pci_private.h | 1 +
drivers/vfio/platform/vfio_amba.c | 8 +-
drivers/vfio/platform/vfio_platform.c | 21 +-
drivers/vfio/platform/vfio_platform_common.c | 56 ++---
drivers/vfio/platform/vfio_platform_private.h | 5 +-
drivers/vfio/vfio.c | 210 ++++++------------
include/linux/vfio.h | 37 +--
13 files changed, 299 insertions(+), 328 deletions(-)
--
2.30.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops
2021-03-09 21:38 [PATCH 00/10] Embed struct vfio_device in all sub-structures Jason Gunthorpe
@ 2021-03-09 21:38 ` Jason Gunthorpe
2021-03-10 7:26 ` Christoph Hellwig
2021-03-12 13:04 ` Liu, Yi L
2021-03-09 21:38 ` [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *' Jason Gunthorpe
` (2 subsequent siblings)
3 siblings, 2 replies; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-09 21:38 UTC (permalink / raw)
To: Alex Williamson, Cornelia Huck, Jonathan Corbet, kvm, linux-doc
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This makes the struct vfio_pci_device part of the public interface so it
can be used with container_of and so forth, as is typical for a Linux
subystem.
This is the first step to bring some type-safety to the vfio interface by
allowing the replacement of 'void *' and 'struct device *' inputs with a
simple and clear 'struct vfio_pci_device *'
For now the self-allocating vfio_add_group_dev() interface is kept so each
user can be updated as a separate patch.
The expected usage pattern is
driver core probe() function:
my_device = kzalloc(sizeof(*mydevice));
vfio_init_group_dev(&my_device->vdev, dev, ops, mydevice);
/* other driver specific prep */
vfio_register_group_dev(&my_device->vdev);
dev_set_drvdata(my_device);
driver core remove() function:
my_device = dev_get_drvdata(dev);
vfio_unregister_group_dev(&my_device->vdev);
/* other driver specific tear down */
kfree(my_device);
Allowing the driver to be able to use the drvdata and vifo_device to go
to/from its own data.
The pattern also makes it clear that vfio_register_group_dev() must be
last in the sequence, as once it is called the core code can immediately
start calling ops. The init/register gap is provided to allow for the
driver to do setup before ops can be called and thus avoid races.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
Documentation/driver-api/vfio.rst | 31 ++++---
drivers/vfio/vfio.c | 132 ++++++++++++++----------------
include/linux/vfio.h | 16 ++++
3 files changed, 96 insertions(+), 83 deletions(-)
diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst
index f1a4d3c3ba0bb1..d3a02300913a7f 100644
--- a/Documentation/driver-api/vfio.rst
+++ b/Documentation/driver-api/vfio.rst
@@ -249,18 +249,23 @@ VFIO bus driver API
VFIO bus drivers, such as vfio-pci make use of only a few interfaces
into VFIO core. When devices are bound and unbound to the driver,
-the driver should call vfio_add_group_dev() and vfio_del_group_dev()
-respectively::
-
- extern int vfio_add_group_dev(struct device *dev,
- const struct vfio_device_ops *ops,
- void *device_data);
-
- extern void *vfio_del_group_dev(struct device *dev);
-
-vfio_add_group_dev() indicates to the core to begin tracking the
-iommu_group of the specified dev and register the dev as owned by
-a VFIO bus driver. The driver provides an ops structure for callbacks
+the driver should call vfio_register_group_dev() and
+vfio_unregister_group_dev() respectively::
+
+ void vfio_init_group_dev(struct vfio_device *device,
+ struct device *dev,
+ const struct vfio_device_ops *ops,
+ void *device_data);
+ int vfio_register_group_dev(struct vfio_device *device);
+ void vfio_unregister_group_dev(struct vfio_device *device);
+
+The driver should embed the vfio_device in its own structure and call
+vfio_init_group_dev() to pre-configure it before going to registration.
+vfio_register_group_dev() indicates to the core to begin tracking the
+iommu_group of the specified dev and register the dev as owned by a VFIO bus
+driver. Once vfio_register_group_dev() returns it is possible for userspace to
+start accessing the driver, thus the driver should ensure it is completely
+ready before calling it. The driver provides an ops structure for callbacks
similar to a file operations structure::
struct vfio_device_ops {
@@ -276,7 +281,7 @@ similar to a file operations structure::
};
Each function is passed the device_data that was originally registered
-in the vfio_add_group_dev() call above. This allows the bus driver
+in the vfio_register_group_dev() call above. This allows the bus driver
an easy place to store its opaque, private data. The open/release
callbacks are issued when a new file descriptor is created for a
device (via VFIO_GROUP_GET_DEVICE_FD). The ioctl interface provides
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 04e24248e77f50..cfa06ae3b9018b 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -89,16 +89,6 @@ struct vfio_group {
struct blocking_notifier_head notifier;
};
-struct vfio_device {
- refcount_t refcount;
- struct completion comp;
- struct device *dev;
- const struct vfio_device_ops *ops;
- struct vfio_group *group;
- struct list_head group_next;
- void *device_data;
-};
-
#ifdef CONFIG_VFIO_NOIOMMU
static bool noiommu __read_mostly;
module_param_named(enable_unsafe_noiommu_mode,
@@ -532,40 +522,6 @@ static struct vfio_group *vfio_group_get_from_dev(struct device *dev)
/**
* Device objects - create, release, get, put, search
*/
-static
-struct vfio_device *vfio_group_create_device(struct vfio_group *group,
- struct device *dev,
- const struct vfio_device_ops *ops,
- void *device_data)
-{
- struct vfio_device *device;
-
- device = kzalloc(sizeof(*device), GFP_KERNEL);
- if (!device)
- return ERR_PTR(-ENOMEM);
-
- refcount_set(&device->refcount, 1);
- init_completion(&device->comp);
- device->dev = dev;
- device->group = group;
- device->ops = ops;
- device->device_data = device_data;
- dev_set_drvdata(dev, device);
-
- /*
- * No need to get group_lock, caller has group reference, matching put
- * is in vfio_del_group_dev()
- */
- vfio_group_get(group);
-
- mutex_lock(&group->device_lock);
- list_add(&device->group_next, &group->device_list);
- group->dev_counter++;
- mutex_unlock(&group->device_lock);
-
- return device;
-}
-
/* Device reference always implies a group reference */
void vfio_device_put(struct vfio_device *device)
{
@@ -784,14 +740,23 @@ static int vfio_iommu_group_notifier(struct notifier_block *nb,
/**
* VFIO driver API
*/
-int vfio_add_group_dev(struct device *dev,
- const struct vfio_device_ops *ops, void *device_data)
+void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
+ const struct vfio_device_ops *ops, void *device_data)
{
+ init_completion(&device->comp);
+ device->dev = dev;
+ device->ops = ops;
+ device->device_data = device_data;
+}
+EXPORT_SYMBOL_GPL(vfio_init_group_dev);
+
+int vfio_register_group_dev(struct vfio_device *device)
+{
+ struct vfio_device *existing_device;
struct iommu_group *iommu_group;
struct vfio_group *group;
- struct vfio_device *device;
- iommu_group = iommu_group_get(dev);
+ iommu_group = iommu_group_get(device->dev);
if (!iommu_group)
return -EINVAL;
@@ -810,30 +775,51 @@ int vfio_add_group_dev(struct device *dev,
iommu_group_put(iommu_group);
}
- device = vfio_group_get_device(group, dev);
- if (device) {
- dev_WARN(dev, "Device already exists on group %d\n",
+ existing_device = vfio_group_get_device(group, device->dev);
+ if (existing_device) {
+ dev_WARN(device->dev, "Device already exists on group %d\n",
iommu_group_id(iommu_group));
- vfio_device_put(device);
+ vfio_device_put(existing_device);
vfio_group_put(group);
return -EBUSY;
}
- device = vfio_group_create_device(group, dev, ops, device_data);
- if (IS_ERR(device)) {
- vfio_group_put(group);
- return PTR_ERR(device);
- }
+ /* Our reference on group is moved to the device */
+ device->group = group;
- /*
- * Drop all but the vfio_device reference. The vfio_device holds
- * a reference to the vfio_group, which holds a reference to the
- * iommu_group.
- */
- vfio_group_put(group);
+ /* Refcounting can't start until the driver calls register */
+ refcount_set(&device->refcount, 1);
+
+ mutex_lock(&group->device_lock);
+ list_add(&device->group_next, &group->device_list);
+ group->dev_counter++;
+ mutex_unlock(&group->device_lock);
return 0;
}
+EXPORT_SYMBOL_GPL(vfio_register_group_dev);
+
+int vfio_add_group_dev(struct device *dev, const struct vfio_device_ops *ops,
+ void *device_data)
+{
+ struct vfio_device *device;
+ int ret;
+
+ device = kzalloc(sizeof(*device), GFP_KERNEL);
+ if (!device)
+ return -ENOMEM;
+
+ vfio_init_group_dev(device, dev, ops, device_data);
+ ret = vfio_register_group_dev(device);
+ if (ret)
+ goto err_kfree;
+ dev_set_drvdata(dev, device);
+ return 0;
+
+err_kfree:
+ kfree(device);
+ return ret;
+}
EXPORT_SYMBOL_GPL(vfio_add_group_dev);
/**
@@ -900,11 +886,9 @@ EXPORT_SYMBOL_GPL(vfio_device_data);
/*
* Decrement the device reference count and wait for the device to be
* removed. Open file descriptors for the device... */
-void *vfio_del_group_dev(struct device *dev)
+void vfio_unregister_group_dev(struct vfio_device *device)
{
- struct vfio_device *device = dev_get_drvdata(dev);
struct vfio_group *group = device->group;
- void *device_data = device->device_data;
struct vfio_unbound_dev *unbound;
unsigned int i = 0;
bool interrupted = false;
@@ -921,7 +905,7 @@ void *vfio_del_group_dev(struct device *dev)
*/
unbound = kzalloc(sizeof(*unbound), GFP_KERNEL);
if (unbound) {
- unbound->dev = dev;
+ unbound->dev = device->dev;
mutex_lock(&group->unbound_lock);
list_add(&unbound->unbound_next, &group->unbound_list);
mutex_unlock(&group->unbound_lock);
@@ -932,7 +916,7 @@ void *vfio_del_group_dev(struct device *dev)
rc = try_wait_for_completion(&device->comp);
while (rc <= 0) {
if (device->ops->request)
- device->ops->request(device_data, i++);
+ device->ops->request(device->device_data, i++);
if (interrupted) {
rc = wait_for_completion_timeout(&device->comp,
@@ -942,7 +926,7 @@ void *vfio_del_group_dev(struct device *dev)
&device->comp, HZ * 10);
if (rc < 0) {
interrupted = true;
- dev_warn(dev,
+ dev_warn(device->dev,
"Device is currently in use, task"
" \"%s\" (%d) "
"blocked until device is released",
@@ -975,9 +959,17 @@ void *vfio_del_group_dev(struct device *dev)
/* Matches the get in vfio_group_create_device() */
vfio_group_put(group);
+}
+EXPORT_SYMBOL_GPL(vfio_unregister_group_dev);
+
+void *vfio_del_group_dev(struct device *dev)
+{
+ struct vfio_device *device = dev_get_drvdata(dev);
+ void *device_data = device->device_data;
+
+ vfio_unregister_group_dev(device);
dev_set_drvdata(dev, NULL);
kfree(device);
-
return device_data;
}
EXPORT_SYMBOL_GPL(vfio_del_group_dev);
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index b7e18bde5aa8b3..ad8b579d67d34a 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -15,6 +15,18 @@
#include <linux/poll.h>
#include <uapi/linux/vfio.h>
+struct vfio_device {
+ struct device *dev;
+ const struct vfio_device_ops *ops;
+ struct vfio_group *group;
+
+ /* Members below here are private, not for driver use */
+ refcount_t refcount;
+ struct completion comp;
+ struct list_head group_next;
+ void *device_data;
+};
+
/**
* struct vfio_device_ops - VFIO bus driver device callbacks
*
@@ -48,11 +60,15 @@ struct vfio_device_ops {
extern struct iommu_group *vfio_iommu_group_get(struct device *dev);
extern void vfio_iommu_group_put(struct iommu_group *group, struct device *dev);
+void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
+ const struct vfio_device_ops *ops, void *device_data);
+int vfio_register_group_dev(struct vfio_device *device);
extern int vfio_add_group_dev(struct device *dev,
const struct vfio_device_ops *ops,
void *device_data);
extern void *vfio_del_group_dev(struct device *dev);
+void vfio_unregister_group_dev(struct vfio_device *device);
extern struct vfio_device *vfio_device_get_from_dev(struct device *dev);
extern void vfio_device_put(struct vfio_device *device);
extern void *vfio_device_data(struct vfio_device *device);
--
2.30.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *'
2021-03-09 21:38 [PATCH 00/10] Embed struct vfio_device in all sub-structures Jason Gunthorpe
2021-03-09 21:38 ` [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops Jason Gunthorpe
@ 2021-03-09 21:38 ` Jason Gunthorpe
2021-03-10 5:52 ` Dan Williams
2021-03-12 13:42 ` Liu, Yi L
2021-03-09 21:38 ` [PATCH 10/10] vfio: Remove device_data from the vfio bus driver API Jason Gunthorpe
2021-03-10 23:52 ` [PATCH 00/10] Embed struct vfio_device in all sub-structures Alex Williamson
3 siblings, 2 replies; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-09 21:38 UTC (permalink / raw)
To: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, kvm, Kirti Wankhede, linux-doc
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
This is the standard kernel pattern, the ops associated with a struct get
the struct pointer in for typesafety. The expected design is to use
container_of to cleanly go from the subsystem level type to the driver
level type without having any type erasure in a void *.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
Documentation/driver-api/vfio.rst | 18 ++++----
drivers/vfio/fsl-mc/vfio_fsl_mc.c | 36 +++++++++------
drivers/vfio/mdev/vfio_mdev.c | 33 +++++++-------
drivers/vfio/pci/vfio_pci.c | 47 ++++++++++++--------
drivers/vfio/platform/vfio_platform_common.c | 33 ++++++++------
drivers/vfio/vfio.c | 20 ++++-----
include/linux/vfio.h | 16 +++----
7 files changed, 117 insertions(+), 86 deletions(-)
diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst
index d3a02300913a7f..3337f337293a32 100644
--- a/Documentation/driver-api/vfio.rst
+++ b/Documentation/driver-api/vfio.rst
@@ -269,20 +269,22 @@ ready before calling it. The driver provides an ops structure for callbacks
similar to a file operations structure::
struct vfio_device_ops {
- int (*open)(void *device_data);
- void (*release)(void *device_data);
- ssize_t (*read)(void *device_data, char __user *buf,
+ int (*open)(struct vfio_device *vdev);
+ void (*release)(struct vfio_device *vdev);
+ ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
size_t count, loff_t *ppos);
- ssize_t (*write)(void *device_data, const char __user *buf,
+ ssize_t (*write)(struct vfio_device *vdev,
+ const char __user *buf,
size_t size, loff_t *ppos);
- long (*ioctl)(void *device_data, unsigned int cmd,
+ long (*ioctl)(struct vfio_device *vdev, unsigned int cmd,
unsigned long arg);
- int (*mmap)(void *device_data, struct vm_area_struct *vma);
+ int (*mmap)(struct vfio_device *vdev,
+ struct vm_area_struct *vma);
};
-Each function is passed the device_data that was originally registered
+Each function is passed the vdev that was originally registered
in the vfio_register_group_dev() call above. This allows the bus driver
-an easy place to store its opaque, private data. The open/release
+to obtain its private data using container_of(). The open/release
callbacks are issued when a new file descriptor is created for a
device (via VFIO_GROUP_GET_DEVICE_FD). The ioctl interface provides
a direct pass through for VFIO_DEVICE_* ioctls. The read/write/mmap
diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
index ddee6ed20c4523..74a5de1b791934 100644
--- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c
+++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
@@ -135,9 +135,10 @@ static void vfio_fsl_mc_regions_cleanup(struct vfio_fsl_mc_device *vdev)
kfree(vdev->regions);
}
-static int vfio_fsl_mc_open(void *device_data)
+static int vfio_fsl_mc_open(struct vfio_device *core_vdev)
{
- struct vfio_fsl_mc_device *vdev = device_data;
+ struct vfio_fsl_mc_device *vdev =
+ container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
int ret;
if (!try_module_get(THIS_MODULE))
@@ -161,9 +162,10 @@ static int vfio_fsl_mc_open(void *device_data)
return ret;
}
-static void vfio_fsl_mc_release(void *device_data)
+static void vfio_fsl_mc_release(struct vfio_device *core_vdev)
{
- struct vfio_fsl_mc_device *vdev = device_data;
+ struct vfio_fsl_mc_device *vdev =
+ container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
int ret;
mutex_lock(&vdev->reflck->lock);
@@ -197,11 +199,12 @@ static void vfio_fsl_mc_release(void *device_data)
module_put(THIS_MODULE);
}
-static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd,
- unsigned long arg)
+static long vfio_fsl_mc_ioctl(struct vfio_device *core_vdev,
+ unsigned int cmd, unsigned long arg)
{
unsigned long minsz;
- struct vfio_fsl_mc_device *vdev = device_data;
+ struct vfio_fsl_mc_device *vdev =
+ container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
struct fsl_mc_device *mc_dev = vdev->mc_dev;
switch (cmd) {
@@ -327,10 +330,11 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd,
}
}
-static ssize_t vfio_fsl_mc_read(void *device_data, char __user *buf,
+static ssize_t vfio_fsl_mc_read(struct vfio_device *core_vdev, char __user *buf,
size_t count, loff_t *ppos)
{
- struct vfio_fsl_mc_device *vdev = device_data;
+ struct vfio_fsl_mc_device *vdev =
+ container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos);
loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK;
struct fsl_mc_device *mc_dev = vdev->mc_dev;
@@ -404,10 +408,12 @@ static int vfio_fsl_mc_send_command(void __iomem *ioaddr, uint64_t *cmd_data)
return 0;
}
-static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf,
- size_t count, loff_t *ppos)
+static ssize_t vfio_fsl_mc_write(struct vfio_device *core_vdev,
+ const char __user *buf, size_t count,
+ loff_t *ppos)
{
- struct vfio_fsl_mc_device *vdev = device_data;
+ struct vfio_fsl_mc_device *vdev =
+ container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos);
loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK;
struct fsl_mc_device *mc_dev = vdev->mc_dev;
@@ -468,9 +474,11 @@ static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region,
size, vma->vm_page_prot);
}
-static int vfio_fsl_mc_mmap(void *device_data, struct vm_area_struct *vma)
+static int vfio_fsl_mc_mmap(struct vfio_device *core_vdev,
+ struct vm_area_struct *vma)
{
- struct vfio_fsl_mc_device *vdev = device_data;
+ struct vfio_fsl_mc_device *vdev =
+ container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
struct fsl_mc_device *mc_dev = vdev->mc_dev;
unsigned int index;
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
index 4469aaf31b56cb..e7309caa99c71b 100644
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ b/drivers/vfio/mdev/vfio_mdev.c
@@ -25,10 +25,11 @@ struct mdev_vfio_device {
struct vfio_device vdev;
};
-static int vfio_mdev_open(void *device_data)
+static int vfio_mdev_open(struct vfio_device *core_vdev)
{
- struct mdev_device *mdev = device_data;
+ struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
struct mdev_parent *parent = mdev->parent;
+
int ret;
if (unlikely(!parent->ops->open))
@@ -44,9 +45,9 @@ static int vfio_mdev_open(void *device_data)
return ret;
}
-static void vfio_mdev_release(void *device_data)
+static void vfio_mdev_release(struct vfio_device *core_vdev)
{
- struct mdev_device *mdev = device_data;
+ struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
struct mdev_parent *parent = mdev->parent;
if (likely(parent->ops->release))
@@ -55,10 +56,10 @@ static void vfio_mdev_release(void *device_data)
module_put(THIS_MODULE);
}
-static long vfio_mdev_unlocked_ioctl(void *device_data,
+static long vfio_mdev_unlocked_ioctl(struct vfio_device *core_vdev,
unsigned int cmd, unsigned long arg)
{
- struct mdev_device *mdev = device_data;
+ struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
struct mdev_parent *parent = mdev->parent;
if (unlikely(!parent->ops->ioctl))
@@ -67,10 +68,10 @@ static long vfio_mdev_unlocked_ioctl(void *device_data,
return parent->ops->ioctl(mdev, cmd, arg);
}
-static ssize_t vfio_mdev_read(void *device_data, char __user *buf,
+static ssize_t vfio_mdev_read(struct vfio_device *core_vdev, char __user *buf,
size_t count, loff_t *ppos)
{
- struct mdev_device *mdev = device_data;
+ struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
struct mdev_parent *parent = mdev->parent;
if (unlikely(!parent->ops->read))
@@ -79,10 +80,11 @@ static ssize_t vfio_mdev_read(void *device_data, char __user *buf,
return parent->ops->read(mdev, buf, count, ppos);
}
-static ssize_t vfio_mdev_write(void *device_data, const char __user *buf,
- size_t count, loff_t *ppos)
+static ssize_t vfio_mdev_write(struct vfio_device *core_vdev,
+ const char __user *buf, size_t count,
+ loff_t *ppos)
{
- struct mdev_device *mdev = device_data;
+ struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
struct mdev_parent *parent = mdev->parent;
if (unlikely(!parent->ops->write))
@@ -91,9 +93,10 @@ static ssize_t vfio_mdev_write(void *device_data, const char __user *buf,
return parent->ops->write(mdev, buf, count, ppos);
}
-static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma)
+static int vfio_mdev_mmap(struct vfio_device *core_vdev,
+ struct vm_area_struct *vma)
{
- struct mdev_device *mdev = device_data;
+ struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
struct mdev_parent *parent = mdev->parent;
if (unlikely(!parent->ops->mmap))
@@ -102,9 +105,9 @@ static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma)
return parent->ops->mmap(mdev, vma);
}
-static void vfio_mdev_request(void *device_data, unsigned int count)
+static void vfio_mdev_request(struct vfio_device *core_vdev, unsigned int count)
{
- struct mdev_device *mdev = device_data;
+ struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
struct mdev_parent *parent = mdev->parent;
if (parent->ops->request)
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index fae573c6f86bdf..af5696a96a76e0 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -553,9 +553,10 @@ static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val)
vfio_device_put(pf_dev);
}
-static void vfio_pci_release(void *device_data)
+static void vfio_pci_release(struct vfio_device *core_vdev)
{
- struct vfio_pci_device *vdev = device_data;
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
mutex_lock(&vdev->reflck->lock);
@@ -581,9 +582,10 @@ static void vfio_pci_release(void *device_data)
module_put(THIS_MODULE);
}
-static int vfio_pci_open(void *device_data)
+static int vfio_pci_open(struct vfio_device *core_vdev)
{
- struct vfio_pci_device *vdev = device_data;
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
int ret = 0;
if (!try_module_get(THIS_MODULE))
@@ -797,10 +799,11 @@ struct vfio_devices {
int max_index;
};
-static long vfio_pci_ioctl(void *device_data,
+static long vfio_pci_ioctl(struct vfio_device *core_vdev,
unsigned int cmd, unsigned long arg)
{
- struct vfio_pci_device *vdev = device_data;
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
unsigned long minsz;
if (cmd == VFIO_DEVICE_GET_INFO) {
@@ -1402,11 +1405,10 @@ static long vfio_pci_ioctl(void *device_data,
return -ENOTTY;
}
-static ssize_t vfio_pci_rw(void *device_data, char __user *buf,
+static ssize_t vfio_pci_rw(struct vfio_pci_device *vdev, char __user *buf,
size_t count, loff_t *ppos, bool iswrite)
{
unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
- struct vfio_pci_device *vdev = device_data;
if (index >= VFIO_PCI_NUM_REGIONS + vdev->num_regions)
return -EINVAL;
@@ -1434,22 +1436,28 @@ static ssize_t vfio_pci_rw(void *device_data, char __user *buf,
return -EINVAL;
}
-static ssize_t vfio_pci_read(void *device_data, char __user *buf,
+static ssize_t vfio_pci_read(struct vfio_device *core_vdev, char __user *buf,
size_t count, loff_t *ppos)
{
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
+
if (!count)
return 0;
- return vfio_pci_rw(device_data, buf, count, ppos, false);
+ return vfio_pci_rw(vdev, buf, count, ppos, false);
}
-static ssize_t vfio_pci_write(void *device_data, const char __user *buf,
+static ssize_t vfio_pci_write(struct vfio_device *core_vdev, const char __user *buf,
size_t count, loff_t *ppos)
{
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
+
if (!count)
return 0;
- return vfio_pci_rw(device_data, (char __user *)buf, count, ppos, true);
+ return vfio_pci_rw(vdev, (char __user *)buf, count, ppos, true);
}
/* Return 1 on zap and vma_lock acquired, 0 on contention (only with @try) */
@@ -1646,9 +1654,10 @@ static const struct vm_operations_struct vfio_pci_mmap_ops = {
.fault = vfio_pci_mmap_fault,
};
-static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
+static int vfio_pci_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma)
{
- struct vfio_pci_device *vdev = device_data;
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
struct pci_dev *pdev = vdev->pdev;
unsigned int index;
u64 phys_len, req_len, pgoff, req_start;
@@ -1714,9 +1723,10 @@ static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
return 0;
}
-static void vfio_pci_request(void *device_data, unsigned int count)
+static void vfio_pci_request(struct vfio_device *core_vdev, unsigned int count)
{
- struct vfio_pci_device *vdev = device_data;
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
struct pci_dev *pdev = vdev->pdev;
mutex_lock(&vdev->igate);
@@ -1830,9 +1840,10 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev,
#define VF_TOKEN_ARG "vf_token="
-static int vfio_pci_match(void *device_data, char *buf)
+static int vfio_pci_match(struct vfio_device *core_vdev, char *buf)
{
- struct vfio_pci_device *vdev = device_data;
+ struct vfio_pci_device *vdev =
+ container_of(core_vdev, struct vfio_pci_device, vdev);
bool vf_token = false;
uuid_t uuid;
int ret;
diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c
index 6eb749250ee41c..f5f6b537084a67 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -218,9 +218,10 @@ static int vfio_platform_call_reset(struct vfio_platform_device *vdev,
return -EINVAL;
}
-static void vfio_platform_release(void *device_data)
+static void vfio_platform_release(struct vfio_device *core_vdev)
{
- struct vfio_platform_device *vdev = device_data;
+ struct vfio_platform_device *vdev =
+ container_of(core_vdev, struct vfio_platform_device, vdev);
mutex_lock(&driver_lock);
@@ -244,9 +245,10 @@ static void vfio_platform_release(void *device_data)
module_put(vdev->parent_module);
}
-static int vfio_platform_open(void *device_data)
+static int vfio_platform_open(struct vfio_device *core_vdev)
{
- struct vfio_platform_device *vdev = device_data;
+ struct vfio_platform_device *vdev =
+ container_of(core_vdev, struct vfio_platform_device, vdev);
int ret;
if (!try_module_get(vdev->parent_module))
@@ -293,10 +295,12 @@ static int vfio_platform_open(void *device_data)
return ret;
}
-static long vfio_platform_ioctl(void *device_data,
+static long vfio_platform_ioctl(struct vfio_device *core_vdev,
unsigned int cmd, unsigned long arg)
{
- struct vfio_platform_device *vdev = device_data;
+ struct vfio_platform_device *vdev =
+ container_of(core_vdev, struct vfio_platform_device, vdev);
+
unsigned long minsz;
if (cmd == VFIO_DEVICE_GET_INFO) {
@@ -455,10 +459,11 @@ static ssize_t vfio_platform_read_mmio(struct vfio_platform_region *reg,
return -EFAULT;
}
-static ssize_t vfio_platform_read(void *device_data, char __user *buf,
- size_t count, loff_t *ppos)
+static ssize_t vfio_platform_read(struct vfio_device *core_vdev,
+ char __user *buf, size_t count, loff_t *ppos)
{
- struct vfio_platform_device *vdev = device_data;
+ struct vfio_platform_device *vdev =
+ container_of(core_vdev, struct vfio_platform_device, vdev);
unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos);
loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK;
@@ -531,10 +536,11 @@ static ssize_t vfio_platform_write_mmio(struct vfio_platform_region *reg,
return -EFAULT;
}
-static ssize_t vfio_platform_write(void *device_data, const char __user *buf,
+static ssize_t vfio_platform_write(struct vfio_device *core_vdev, const char __user *buf,
size_t count, loff_t *ppos)
{
- struct vfio_platform_device *vdev = device_data;
+ struct vfio_platform_device *vdev =
+ container_of(core_vdev, struct vfio_platform_device, vdev);
unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos);
loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK;
@@ -573,9 +579,10 @@ static int vfio_platform_mmap_mmio(struct vfio_platform_region region,
req_len, vma->vm_page_prot);
}
-static int vfio_platform_mmap(void *device_data, struct vm_area_struct *vma)
+static int vfio_platform_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma)
{
- struct vfio_platform_device *vdev = device_data;
+ struct vfio_platform_device *vdev =
+ container_of(core_vdev, struct vfio_platform_device, vdev);
unsigned int index;
index = vma->vm_pgoff >> (VFIO_PLATFORM_OFFSET_SHIFT - PAGE_SHIFT);
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 2d6d7cc1d1ebf9..01de47d1810b6b 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -832,7 +832,7 @@ static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group,
int ret;
if (it->ops->match) {
- ret = it->ops->match(it->device_data, buf);
+ ret = it->ops->match(it, buf);
if (ret < 0) {
device = ERR_PTR(ret);
break;
@@ -893,7 +893,7 @@ void vfio_unregister_group_dev(struct vfio_device *device)
rc = try_wait_for_completion(&device->comp);
while (rc <= 0) {
if (device->ops->request)
- device->ops->request(device->device_data, i++);
+ device->ops->request(device, i++);
if (interrupted) {
rc = wait_for_completion_timeout(&device->comp,
@@ -1379,7 +1379,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
if (IS_ERR(device))
return PTR_ERR(device);
- ret = device->ops->open(device->device_data);
+ ret = device->ops->open(device);
if (ret) {
vfio_device_put(device);
return ret;
@@ -1391,7 +1391,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
*/
ret = get_unused_fd_flags(O_CLOEXEC);
if (ret < 0) {
- device->ops->release(device->device_data);
+ device->ops->release(device);
vfio_device_put(device);
return ret;
}
@@ -1401,7 +1401,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
if (IS_ERR(filep)) {
put_unused_fd(ret);
ret = PTR_ERR(filep);
- device->ops->release(device->device_data);
+ device->ops->release(device);
vfio_device_put(device);
return ret;
}
@@ -1558,7 +1558,7 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
{
struct vfio_device *device = filep->private_data;
- device->ops->release(device->device_data);
+ device->ops->release(device);
vfio_group_try_dissolve_container(device->group);
@@ -1575,7 +1575,7 @@ static long vfio_device_fops_unl_ioctl(struct file *filep,
if (unlikely(!device->ops->ioctl))
return -EINVAL;
- return device->ops->ioctl(device->device_data, cmd, arg);
+ return device->ops->ioctl(device, cmd, arg);
}
static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
@@ -1586,7 +1586,7 @@ static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
if (unlikely(!device->ops->read))
return -EINVAL;
- return device->ops->read(device->device_data, buf, count, ppos);
+ return device->ops->read(device, buf, count, ppos);
}
static ssize_t vfio_device_fops_write(struct file *filep,
@@ -1598,7 +1598,7 @@ static ssize_t vfio_device_fops_write(struct file *filep,
if (unlikely(!device->ops->write))
return -EINVAL;
- return device->ops->write(device->device_data, buf, count, ppos);
+ return device->ops->write(device, buf, count, ppos);
}
static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
@@ -1608,7 +1608,7 @@ static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
if (unlikely(!device->ops->mmap))
return -EINVAL;
- return device->ops->mmap(device->device_data, vma);
+ return device->ops->mmap(device, vma);
}
static const struct file_operations vfio_device_fops = {
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 4995faf51efeae..784c34c0a28763 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -44,17 +44,17 @@ struct vfio_device {
*/
struct vfio_device_ops {
char *name;
- int (*open)(void *device_data);
- void (*release)(void *device_data);
- ssize_t (*read)(void *device_data, char __user *buf,
+ int (*open)(struct vfio_device *vdev);
+ void (*release)(struct vfio_device *vdev);
+ ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
size_t count, loff_t *ppos);
- ssize_t (*write)(void *device_data, const char __user *buf,
+ ssize_t (*write)(struct vfio_device *vdev, const char __user *buf,
size_t count, loff_t *size);
- long (*ioctl)(void *device_data, unsigned int cmd,
+ long (*ioctl)(struct vfio_device *vdev, unsigned int cmd,
unsigned long arg);
- int (*mmap)(void *device_data, struct vm_area_struct *vma);
- void (*request)(void *device_data, unsigned int count);
- int (*match)(void *device_data, char *buf);
+ int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
+ void (*request)(struct vfio_device *vdev, unsigned int count);
+ int (*match)(struct vfio_device *vdev, char *buf);
};
extern struct iommu_group *vfio_iommu_group_get(struct device *dev);
--
2.30.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH 10/10] vfio: Remove device_data from the vfio bus driver API
2021-03-09 21:38 [PATCH 00/10] Embed struct vfio_device in all sub-structures Jason Gunthorpe
2021-03-09 21:38 ` [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops Jason Gunthorpe
2021-03-09 21:38 ` [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *' Jason Gunthorpe
@ 2021-03-09 21:38 ` Jason Gunthorpe
2021-03-10 7:37 ` Christoph Hellwig
2021-03-10 23:52 ` [PATCH 00/10] Embed struct vfio_device in all sub-structures Alex Williamson
3 siblings, 1 reply; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-09 21:38 UTC (permalink / raw)
To: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, kvm, Kirti Wankhede, linux-doc
Cc: Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
There are no longer any users, so it can go away. Everything is using
container_of now.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
Documentation/driver-api/vfio.rst | 3 +--
drivers/vfio/fsl-mc/vfio_fsl_mc.c | 5 +++--
drivers/vfio/mdev/vfio_mdev.c | 2 +-
drivers/vfio/pci/vfio_pci.c | 2 +-
drivers/vfio/platform/vfio_platform_common.c | 2 +-
drivers/vfio/vfio.c | 12 +-----------
include/linux/vfio.h | 4 +---
7 files changed, 9 insertions(+), 21 deletions(-)
diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst
index 3337f337293a32..decc68cb8114ac 100644
--- a/Documentation/driver-api/vfio.rst
+++ b/Documentation/driver-api/vfio.rst
@@ -254,8 +254,7 @@ vfio_unregister_group_dev() respectively::
void vfio_init_group_dev(struct vfio_device *device,
struct device *dev,
- const struct vfio_device_ops *ops,
- void *device_data);
+ const struct vfio_device_ops *ops);
int vfio_register_group_dev(struct vfio_device *device);
void vfio_unregister_group_dev(struct vfio_device *device);
diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
index 74a5de1b791934..07f636b9f6b472 100644
--- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c
+++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
@@ -75,7 +75,8 @@ static int vfio_fsl_mc_reflck_attach(struct vfio_fsl_mc_device *vdev)
goto unlock;
}
- cont_vdev = vfio_device_data(device);
+ cont_vdev =
+ container_of(device, struct vfio_fsl_mc_device, vdev);
if (!cont_vdev || !cont_vdev->reflck) {
vfio_device_put(device);
ret = -ENODEV;
@@ -614,7 +615,7 @@ static int vfio_fsl_mc_probe(struct fsl_mc_device *mc_dev)
goto out_group_put;
}
- vfio_init_group_dev(&vdev->vdev, dev, &vfio_fsl_mc_ops, vdev);
+ vfio_init_group_dev(&vdev->vdev, dev, &vfio_fsl_mc_ops);
mutex_init(&vdev->igate);
vdev->mc_dev = mc_dev;
diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
index e7309caa99c71b..71bd28f976e5af 100644
--- a/drivers/vfio/mdev/vfio_mdev.c
+++ b/drivers/vfio/mdev/vfio_mdev.c
@@ -138,7 +138,7 @@ static int vfio_mdev_probe(struct device *dev)
if (!mvdev)
return -ENOMEM;
- vfio_init_group_dev(&mvdev->vdev, &mdev->dev, &vfio_mdev_dev_ops, mdev);
+ vfio_init_group_dev(&mvdev->vdev, &mdev->dev, &vfio_mdev_dev_ops);
ret = vfio_register_group_dev(&mvdev->vdev);
if (ret) {
kfree(mvdev);
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 4b0d60f7602e40..3c2497ba79460c 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -1962,7 +1962,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
goto out_group_put;
}
- vfio_init_group_dev(&vdev->vdev, &pdev->dev, &vfio_pci_ops, vdev);
+ vfio_init_group_dev(&vdev->vdev, &pdev->dev, &vfio_pci_ops);
vdev->pdev = pdev;
vdev->irq_type = VFIO_PCI_NUM_IRQS;
mutex_init(&vdev->igate);
diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c
index f5f6b537084a67..361e5b57e36932 100644
--- a/drivers/vfio/platform/vfio_platform_common.c
+++ b/drivers/vfio/platform/vfio_platform_common.c
@@ -666,7 +666,7 @@ int vfio_platform_probe_common(struct vfio_platform_device *vdev,
struct iommu_group *group;
int ret;
- vfio_init_group_dev(&vdev->vdev, dev, &vfio_platform_ops, vdev);
+ vfio_init_group_dev(&vdev->vdev, dev, &vfio_platform_ops);
ret = vfio_platform_acpi_probe(vdev, dev);
if (ret)
diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 01de47d1810b6b..39ea77557ba0c4 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -741,12 +741,11 @@ static int vfio_iommu_group_notifier(struct notifier_block *nb,
* VFIO driver API
*/
void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
- const struct vfio_device_ops *ops, void *device_data)
+ const struct vfio_device_ops *ops)
{
init_completion(&device->comp);
device->dev = dev;
device->ops = ops;
- device->device_data = device_data;
}
EXPORT_SYMBOL_GPL(vfio_init_group_dev);
@@ -851,15 +850,6 @@ static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group,
return device;
}
-/*
- * Caller must hold a reference to the vfio_device
- */
-void *vfio_device_data(struct vfio_device *device)
-{
- return device->device_data;
-}
-EXPORT_SYMBOL_GPL(vfio_device_data);
-
/*
* Decrement the device reference count and wait for the device to be
* removed. Open file descriptors for the device... */
diff --git a/include/linux/vfio.h b/include/linux/vfio.h
index 784c34c0a28763..a2c5b30e1763ba 100644
--- a/include/linux/vfio.h
+++ b/include/linux/vfio.h
@@ -24,7 +24,6 @@ struct vfio_device {
refcount_t refcount;
struct completion comp;
struct list_head group_next;
- void *device_data;
};
/**
@@ -61,12 +60,11 @@ extern struct iommu_group *vfio_iommu_group_get(struct device *dev);
extern void vfio_iommu_group_put(struct iommu_group *group, struct device *dev);
void vfio_init_group_dev(struct vfio_device *device, struct device *dev,
- const struct vfio_device_ops *ops, void *device_data);
+ const struct vfio_device_ops *ops);
int vfio_register_group_dev(struct vfio_device *device);
void vfio_unregister_group_dev(struct vfio_device *device);
extern struct vfio_device *vfio_device_get_from_dev(struct device *dev);
extern void vfio_device_put(struct vfio_device *device);
-extern void *vfio_device_data(struct vfio_device *device);
/* events for the backend driver notify callback */
enum vfio_iommu_notify_type {
--
2.30.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *'
2021-03-09 21:38 ` [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *' Jason Gunthorpe
@ 2021-03-10 5:52 ` Dan Williams
2021-03-10 6:24 ` Leon Romanovsky
2021-03-10 12:58 ` Jason Gunthorpe
2021-03-12 13:42 ` Liu, Yi L
1 sibling, 2 replies; 17+ messages in thread
From: Dan Williams @ 2021-03-10 5:52 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, KVM list, Kirti Wankhede, Linux Doc Mailing List,
Raj, Ashok, Daniel Vetter, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On Tue, Mar 9, 2021 at 1:39 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> This is the standard kernel pattern, the ops associated with a struct get
> the struct pointer in for typesafety. The expected design is to use
> container_of to cleanly go from the subsystem level type to the driver
> level type without having any type erasure in a void *.
This patch alone is worth the price of admission.
Seems like it would be worth adding
to_vfio_{pci,platform,fsl_mc}_device() helpers in this patch as well.
I've sometimes added runtime type safety to to_* helpers for early
warning of mistakes that happen when refactoring...
static inline struct vfio_pci_device *
to_vfio_pci_device(struct vfio_device *core_dev)
{
if (dev_WARN_ONCE(core_dev->dev, core_dev->ops != &vfio_pci_ops,
"not a vfio_pci_device!\n"))
return NULL;
return container_of(core_vdev, struct vfio_pci_device, vdev);
}
...but typed ops is already a significant idiomatic improvement.
>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> Documentation/driver-api/vfio.rst | 18 ++++----
> drivers/vfio/fsl-mc/vfio_fsl_mc.c | 36 +++++++++------
> drivers/vfio/mdev/vfio_mdev.c | 33 +++++++-------
> drivers/vfio/pci/vfio_pci.c | 47 ++++++++++++--------
> drivers/vfio/platform/vfio_platform_common.c | 33 ++++++++------
> drivers/vfio/vfio.c | 20 ++++-----
> include/linux/vfio.h | 16 +++----
> 7 files changed, 117 insertions(+), 86 deletions(-)
>
> diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst
> index d3a02300913a7f..3337f337293a32 100644
> --- a/Documentation/driver-api/vfio.rst
> +++ b/Documentation/driver-api/vfio.rst
> @@ -269,20 +269,22 @@ ready before calling it. The driver provides an ops structure for callbacks
> similar to a file operations structure::
>
> struct vfio_device_ops {
> - int (*open)(void *device_data);
> - void (*release)(void *device_data);
> - ssize_t (*read)(void *device_data, char __user *buf,
> + int (*open)(struct vfio_device *vdev);
> + void (*release)(struct vfio_device *vdev);
> + ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
> size_t count, loff_t *ppos);
> - ssize_t (*write)(void *device_data, const char __user *buf,
> + ssize_t (*write)(struct vfio_device *vdev,
> + const char __user *buf,
> size_t size, loff_t *ppos);
> - long (*ioctl)(void *device_data, unsigned int cmd,
> + long (*ioctl)(struct vfio_device *vdev, unsigned int cmd,
> unsigned long arg);
> - int (*mmap)(void *device_data, struct vm_area_struct *vma);
> + int (*mmap)(struct vfio_device *vdev,
> + struct vm_area_struct *vma);
> };
>
> -Each function is passed the device_data that was originally registered
> +Each function is passed the vdev that was originally registered
> in the vfio_register_group_dev() call above. This allows the bus driver
> -an easy place to store its opaque, private data. The open/release
> +to obtain its private data using container_of(). The open/release
> callbacks are issued when a new file descriptor is created for a
> device (via VFIO_GROUP_GET_DEVICE_FD). The ioctl interface provides
> a direct pass through for VFIO_DEVICE_* ioctls. The read/write/mmap
> diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
> index ddee6ed20c4523..74a5de1b791934 100644
> --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c
> +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
> @@ -135,9 +135,10 @@ static void vfio_fsl_mc_regions_cleanup(struct vfio_fsl_mc_device *vdev)
> kfree(vdev->regions);
> }
>
> -static int vfio_fsl_mc_open(void *device_data)
> +static int vfio_fsl_mc_open(struct vfio_device *core_vdev)
> {
> - struct vfio_fsl_mc_device *vdev = device_data;
> + struct vfio_fsl_mc_device *vdev =
> + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> int ret;
>
> if (!try_module_get(THIS_MODULE))
> @@ -161,9 +162,10 @@ static int vfio_fsl_mc_open(void *device_data)
> return ret;
> }
>
> -static void vfio_fsl_mc_release(void *device_data)
> +static void vfio_fsl_mc_release(struct vfio_device *core_vdev)
> {
> - struct vfio_fsl_mc_device *vdev = device_data;
> + struct vfio_fsl_mc_device *vdev =
> + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> int ret;
>
> mutex_lock(&vdev->reflck->lock);
> @@ -197,11 +199,12 @@ static void vfio_fsl_mc_release(void *device_data)
> module_put(THIS_MODULE);
> }
>
> -static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd,
> - unsigned long arg)
> +static long vfio_fsl_mc_ioctl(struct vfio_device *core_vdev,
> + unsigned int cmd, unsigned long arg)
> {
> unsigned long minsz;
> - struct vfio_fsl_mc_device *vdev = device_data;
> + struct vfio_fsl_mc_device *vdev =
> + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> struct fsl_mc_device *mc_dev = vdev->mc_dev;
>
> switch (cmd) {
> @@ -327,10 +330,11 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd,
> }
> }
>
> -static ssize_t vfio_fsl_mc_read(void *device_data, char __user *buf,
> +static ssize_t vfio_fsl_mc_read(struct vfio_device *core_vdev, char __user *buf,
> size_t count, loff_t *ppos)
> {
> - struct vfio_fsl_mc_device *vdev = device_data;
> + struct vfio_fsl_mc_device *vdev =
> + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos);
> loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK;
> struct fsl_mc_device *mc_dev = vdev->mc_dev;
> @@ -404,10 +408,12 @@ static int vfio_fsl_mc_send_command(void __iomem *ioaddr, uint64_t *cmd_data)
> return 0;
> }
>
> -static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf,
> - size_t count, loff_t *ppos)
> +static ssize_t vfio_fsl_mc_write(struct vfio_device *core_vdev,
> + const char __user *buf, size_t count,
> + loff_t *ppos)
> {
> - struct vfio_fsl_mc_device *vdev = device_data;
> + struct vfio_fsl_mc_device *vdev =
> + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos);
> loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK;
> struct fsl_mc_device *mc_dev = vdev->mc_dev;
> @@ -468,9 +474,11 @@ static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region,
> size, vma->vm_page_prot);
> }
>
> -static int vfio_fsl_mc_mmap(void *device_data, struct vm_area_struct *vma)
> +static int vfio_fsl_mc_mmap(struct vfio_device *core_vdev,
> + struct vm_area_struct *vma)
> {
> - struct vfio_fsl_mc_device *vdev = device_data;
> + struct vfio_fsl_mc_device *vdev =
> + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> struct fsl_mc_device *mc_dev = vdev->mc_dev;
> unsigned int index;
>
> diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
> index 4469aaf31b56cb..e7309caa99c71b 100644
> --- a/drivers/vfio/mdev/vfio_mdev.c
> +++ b/drivers/vfio/mdev/vfio_mdev.c
> @@ -25,10 +25,11 @@ struct mdev_vfio_device {
> struct vfio_device vdev;
> };
>
> -static int vfio_mdev_open(void *device_data)
> +static int vfio_mdev_open(struct vfio_device *core_vdev)
> {
> - struct mdev_device *mdev = device_data;
> + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> struct mdev_parent *parent = mdev->parent;
> +
> int ret;
>
> if (unlikely(!parent->ops->open))
> @@ -44,9 +45,9 @@ static int vfio_mdev_open(void *device_data)
> return ret;
> }
>
> -static void vfio_mdev_release(void *device_data)
> +static void vfio_mdev_release(struct vfio_device *core_vdev)
> {
> - struct mdev_device *mdev = device_data;
> + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> struct mdev_parent *parent = mdev->parent;
>
> if (likely(parent->ops->release))
> @@ -55,10 +56,10 @@ static void vfio_mdev_release(void *device_data)
> module_put(THIS_MODULE);
> }
>
> -static long vfio_mdev_unlocked_ioctl(void *device_data,
> +static long vfio_mdev_unlocked_ioctl(struct vfio_device *core_vdev,
> unsigned int cmd, unsigned long arg)
> {
> - struct mdev_device *mdev = device_data;
> + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> struct mdev_parent *parent = mdev->parent;
>
> if (unlikely(!parent->ops->ioctl))
> @@ -67,10 +68,10 @@ static long vfio_mdev_unlocked_ioctl(void *device_data,
> return parent->ops->ioctl(mdev, cmd, arg);
> }
>
> -static ssize_t vfio_mdev_read(void *device_data, char __user *buf,
> +static ssize_t vfio_mdev_read(struct vfio_device *core_vdev, char __user *buf,
> size_t count, loff_t *ppos)
> {
> - struct mdev_device *mdev = device_data;
> + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> struct mdev_parent *parent = mdev->parent;
>
> if (unlikely(!parent->ops->read))
> @@ -79,10 +80,11 @@ static ssize_t vfio_mdev_read(void *device_data, char __user *buf,
> return parent->ops->read(mdev, buf, count, ppos);
> }
>
> -static ssize_t vfio_mdev_write(void *device_data, const char __user *buf,
> - size_t count, loff_t *ppos)
> +static ssize_t vfio_mdev_write(struct vfio_device *core_vdev,
> + const char __user *buf, size_t count,
> + loff_t *ppos)
> {
> - struct mdev_device *mdev = device_data;
> + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> struct mdev_parent *parent = mdev->parent;
>
> if (unlikely(!parent->ops->write))
> @@ -91,9 +93,10 @@ static ssize_t vfio_mdev_write(void *device_data, const char __user *buf,
> return parent->ops->write(mdev, buf, count, ppos);
> }
>
> -static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma)
> +static int vfio_mdev_mmap(struct vfio_device *core_vdev,
> + struct vm_area_struct *vma)
> {
> - struct mdev_device *mdev = device_data;
> + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> struct mdev_parent *parent = mdev->parent;
>
> if (unlikely(!parent->ops->mmap))
> @@ -102,9 +105,9 @@ static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma)
> return parent->ops->mmap(mdev, vma);
> }
>
> -static void vfio_mdev_request(void *device_data, unsigned int count)
> +static void vfio_mdev_request(struct vfio_device *core_vdev, unsigned int count)
> {
> - struct mdev_device *mdev = device_data;
> + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> struct mdev_parent *parent = mdev->parent;
>
> if (parent->ops->request)
> diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
> index fae573c6f86bdf..af5696a96a76e0 100644
> --- a/drivers/vfio/pci/vfio_pci.c
> +++ b/drivers/vfio/pci/vfio_pci.c
> @@ -553,9 +553,10 @@ static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val)
> vfio_device_put(pf_dev);
> }
>
> -static void vfio_pci_release(void *device_data)
> +static void vfio_pci_release(struct vfio_device *core_vdev)
> {
> - struct vfio_pci_device *vdev = device_data;
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
>
> mutex_lock(&vdev->reflck->lock);
>
> @@ -581,9 +582,10 @@ static void vfio_pci_release(void *device_data)
> module_put(THIS_MODULE);
> }
>
> -static int vfio_pci_open(void *device_data)
> +static int vfio_pci_open(struct vfio_device *core_vdev)
> {
> - struct vfio_pci_device *vdev = device_data;
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
> int ret = 0;
>
> if (!try_module_get(THIS_MODULE))
> @@ -797,10 +799,11 @@ struct vfio_devices {
> int max_index;
> };
>
> -static long vfio_pci_ioctl(void *device_data,
> +static long vfio_pci_ioctl(struct vfio_device *core_vdev,
> unsigned int cmd, unsigned long arg)
> {
> - struct vfio_pci_device *vdev = device_data;
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
> unsigned long minsz;
>
> if (cmd == VFIO_DEVICE_GET_INFO) {
> @@ -1402,11 +1405,10 @@ static long vfio_pci_ioctl(void *device_data,
> return -ENOTTY;
> }
>
> -static ssize_t vfio_pci_rw(void *device_data, char __user *buf,
> +static ssize_t vfio_pci_rw(struct vfio_pci_device *vdev, char __user *buf,
> size_t count, loff_t *ppos, bool iswrite)
> {
> unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
> - struct vfio_pci_device *vdev = device_data;
>
> if (index >= VFIO_PCI_NUM_REGIONS + vdev->num_regions)
> return -EINVAL;
> @@ -1434,22 +1436,28 @@ static ssize_t vfio_pci_rw(void *device_data, char __user *buf,
> return -EINVAL;
> }
>
> -static ssize_t vfio_pci_read(void *device_data, char __user *buf,
> +static ssize_t vfio_pci_read(struct vfio_device *core_vdev, char __user *buf,
> size_t count, loff_t *ppos)
> {
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
> +
> if (!count)
> return 0;
>
> - return vfio_pci_rw(device_data, buf, count, ppos, false);
> + return vfio_pci_rw(vdev, buf, count, ppos, false);
> }
>
> -static ssize_t vfio_pci_write(void *device_data, const char __user *buf,
> +static ssize_t vfio_pci_write(struct vfio_device *core_vdev, const char __user *buf,
> size_t count, loff_t *ppos)
> {
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
> +
> if (!count)
> return 0;
>
> - return vfio_pci_rw(device_data, (char __user *)buf, count, ppos, true);
> + return vfio_pci_rw(vdev, (char __user *)buf, count, ppos, true);
> }
>
> /* Return 1 on zap and vma_lock acquired, 0 on contention (only with @try) */
> @@ -1646,9 +1654,10 @@ static const struct vm_operations_struct vfio_pci_mmap_ops = {
> .fault = vfio_pci_mmap_fault,
> };
>
> -static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
> +static int vfio_pci_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma)
> {
> - struct vfio_pci_device *vdev = device_data;
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
> struct pci_dev *pdev = vdev->pdev;
> unsigned int index;
> u64 phys_len, req_len, pgoff, req_start;
> @@ -1714,9 +1723,10 @@ static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
> return 0;
> }
>
> -static void vfio_pci_request(void *device_data, unsigned int count)
> +static void vfio_pci_request(struct vfio_device *core_vdev, unsigned int count)
> {
> - struct vfio_pci_device *vdev = device_data;
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
> struct pci_dev *pdev = vdev->pdev;
>
> mutex_lock(&vdev->igate);
> @@ -1830,9 +1840,10 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev,
>
> #define VF_TOKEN_ARG "vf_token="
>
> -static int vfio_pci_match(void *device_data, char *buf)
> +static int vfio_pci_match(struct vfio_device *core_vdev, char *buf)
> {
> - struct vfio_pci_device *vdev = device_data;
> + struct vfio_pci_device *vdev =
> + container_of(core_vdev, struct vfio_pci_device, vdev);
> bool vf_token = false;
> uuid_t uuid;
> int ret;
> diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c
> index 6eb749250ee41c..f5f6b537084a67 100644
> --- a/drivers/vfio/platform/vfio_platform_common.c
> +++ b/drivers/vfio/platform/vfio_platform_common.c
> @@ -218,9 +218,10 @@ static int vfio_platform_call_reset(struct vfio_platform_device *vdev,
> return -EINVAL;
> }
>
> -static void vfio_platform_release(void *device_data)
> +static void vfio_platform_release(struct vfio_device *core_vdev)
> {
> - struct vfio_platform_device *vdev = device_data;
> + struct vfio_platform_device *vdev =
> + container_of(core_vdev, struct vfio_platform_device, vdev);
>
> mutex_lock(&driver_lock);
>
> @@ -244,9 +245,10 @@ static void vfio_platform_release(void *device_data)
> module_put(vdev->parent_module);
> }
>
> -static int vfio_platform_open(void *device_data)
> +static int vfio_platform_open(struct vfio_device *core_vdev)
> {
> - struct vfio_platform_device *vdev = device_data;
> + struct vfio_platform_device *vdev =
> + container_of(core_vdev, struct vfio_platform_device, vdev);
> int ret;
>
> if (!try_module_get(vdev->parent_module))
> @@ -293,10 +295,12 @@ static int vfio_platform_open(void *device_data)
> return ret;
> }
>
> -static long vfio_platform_ioctl(void *device_data,
> +static long vfio_platform_ioctl(struct vfio_device *core_vdev,
> unsigned int cmd, unsigned long arg)
> {
> - struct vfio_platform_device *vdev = device_data;
> + struct vfio_platform_device *vdev =
> + container_of(core_vdev, struct vfio_platform_device, vdev);
> +
> unsigned long minsz;
>
> if (cmd == VFIO_DEVICE_GET_INFO) {
> @@ -455,10 +459,11 @@ static ssize_t vfio_platform_read_mmio(struct vfio_platform_region *reg,
> return -EFAULT;
> }
>
> -static ssize_t vfio_platform_read(void *device_data, char __user *buf,
> - size_t count, loff_t *ppos)
> +static ssize_t vfio_platform_read(struct vfio_device *core_vdev,
> + char __user *buf, size_t count, loff_t *ppos)
> {
> - struct vfio_platform_device *vdev = device_data;
> + struct vfio_platform_device *vdev =
> + container_of(core_vdev, struct vfio_platform_device, vdev);
> unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos);
> loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK;
>
> @@ -531,10 +536,11 @@ static ssize_t vfio_platform_write_mmio(struct vfio_platform_region *reg,
> return -EFAULT;
> }
>
> -static ssize_t vfio_platform_write(void *device_data, const char __user *buf,
> +static ssize_t vfio_platform_write(struct vfio_device *core_vdev, const char __user *buf,
> size_t count, loff_t *ppos)
> {
> - struct vfio_platform_device *vdev = device_data;
> + struct vfio_platform_device *vdev =
> + container_of(core_vdev, struct vfio_platform_device, vdev);
> unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos);
> loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK;
>
> @@ -573,9 +579,10 @@ static int vfio_platform_mmap_mmio(struct vfio_platform_region region,
> req_len, vma->vm_page_prot);
> }
>
> -static int vfio_platform_mmap(void *device_data, struct vm_area_struct *vma)
> +static int vfio_platform_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma)
> {
> - struct vfio_platform_device *vdev = device_data;
> + struct vfio_platform_device *vdev =
> + container_of(core_vdev, struct vfio_platform_device, vdev);
> unsigned int index;
>
> index = vma->vm_pgoff >> (VFIO_PLATFORM_OFFSET_SHIFT - PAGE_SHIFT);
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 2d6d7cc1d1ebf9..01de47d1810b6b 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -832,7 +832,7 @@ static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group,
> int ret;
>
> if (it->ops->match) {
> - ret = it->ops->match(it->device_data, buf);
> + ret = it->ops->match(it, buf);
> if (ret < 0) {
> device = ERR_PTR(ret);
> break;
> @@ -893,7 +893,7 @@ void vfio_unregister_group_dev(struct vfio_device *device)
> rc = try_wait_for_completion(&device->comp);
> while (rc <= 0) {
> if (device->ops->request)
> - device->ops->request(device->device_data, i++);
> + device->ops->request(device, i++);
>
> if (interrupted) {
> rc = wait_for_completion_timeout(&device->comp,
> @@ -1379,7 +1379,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
> if (IS_ERR(device))
> return PTR_ERR(device);
>
> - ret = device->ops->open(device->device_data);
> + ret = device->ops->open(device);
> if (ret) {
> vfio_device_put(device);
> return ret;
> @@ -1391,7 +1391,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
> */
> ret = get_unused_fd_flags(O_CLOEXEC);
> if (ret < 0) {
> - device->ops->release(device->device_data);
> + device->ops->release(device);
> vfio_device_put(device);
> return ret;
> }
> @@ -1401,7 +1401,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
> if (IS_ERR(filep)) {
> put_unused_fd(ret);
> ret = PTR_ERR(filep);
> - device->ops->release(device->device_data);
> + device->ops->release(device);
> vfio_device_put(device);
> return ret;
> }
> @@ -1558,7 +1558,7 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> {
> struct vfio_device *device = filep->private_data;
>
> - device->ops->release(device->device_data);
> + device->ops->release(device);
>
> vfio_group_try_dissolve_container(device->group);
>
> @@ -1575,7 +1575,7 @@ static long vfio_device_fops_unl_ioctl(struct file *filep,
> if (unlikely(!device->ops->ioctl))
> return -EINVAL;
>
> - return device->ops->ioctl(device->device_data, cmd, arg);
> + return device->ops->ioctl(device, cmd, arg);
> }
>
> static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
> @@ -1586,7 +1586,7 @@ static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
> if (unlikely(!device->ops->read))
> return -EINVAL;
>
> - return device->ops->read(device->device_data, buf, count, ppos);
> + return device->ops->read(device, buf, count, ppos);
> }
>
> static ssize_t vfio_device_fops_write(struct file *filep,
> @@ -1598,7 +1598,7 @@ static ssize_t vfio_device_fops_write(struct file *filep,
> if (unlikely(!device->ops->write))
> return -EINVAL;
>
> - return device->ops->write(device->device_data, buf, count, ppos);
> + return device->ops->write(device, buf, count, ppos);
> }
>
> static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
> @@ -1608,7 +1608,7 @@ static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
> if (unlikely(!device->ops->mmap))
> return -EINVAL;
>
> - return device->ops->mmap(device->device_data, vma);
> + return device->ops->mmap(device, vma);
> }
>
> static const struct file_operations vfio_device_fops = {
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index 4995faf51efeae..784c34c0a28763 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -44,17 +44,17 @@ struct vfio_device {
> */
> struct vfio_device_ops {
> char *name;
> - int (*open)(void *device_data);
> - void (*release)(void *device_data);
> - ssize_t (*read)(void *device_data, char __user *buf,
> + int (*open)(struct vfio_device *vdev);
> + void (*release)(struct vfio_device *vdev);
> + ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
> size_t count, loff_t *ppos);
> - ssize_t (*write)(void *device_data, const char __user *buf,
> + ssize_t (*write)(struct vfio_device *vdev, const char __user *buf,
> size_t count, loff_t *size);
> - long (*ioctl)(void *device_data, unsigned int cmd,
> + long (*ioctl)(struct vfio_device *vdev, unsigned int cmd,
> unsigned long arg);
> - int (*mmap)(void *device_data, struct vm_area_struct *vma);
> - void (*request)(void *device_data, unsigned int count);
> - int (*match)(void *device_data, char *buf);
> + int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> + void (*request)(struct vfio_device *vdev, unsigned int count);
> + int (*match)(struct vfio_device *vdev, char *buf);
> };
>
> extern struct iommu_group *vfio_iommu_group_get(struct device *dev);
> --
> 2.30.1
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *'
2021-03-10 5:52 ` Dan Williams
@ 2021-03-10 6:24 ` Leon Romanovsky
2021-03-10 12:58 ` Jason Gunthorpe
1 sibling, 0 replies; 17+ messages in thread
From: Leon Romanovsky @ 2021-03-10 6:24 UTC (permalink / raw)
To: Dan Williams
Cc: Jason Gunthorpe, Alex Williamson, Cornelia Huck, Jonathan Corbet,
Diana Craciun, Eric Auger, KVM list, Kirti Wankhede,
Linux Doc Mailing List, Raj, Ashok, Daniel Vetter,
Christoph Hellwig, Max Gurtovoy, Tarun Gupta
On Tue, Mar 09, 2021 at 09:52:30PM -0800, Dan Williams wrote:
> On Tue, Mar 9, 2021 at 1:39 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > This is the standard kernel pattern, the ops associated with a struct get
> > the struct pointer in for typesafety. The expected design is to use
> > container_of to cleanly go from the subsystem level type to the driver
> > level type without having any type erasure in a void *.
>
> This patch alone is worth the price of admission.
>
> Seems like it would be worth adding
> to_vfio_{pci,platform,fsl_mc}_device() helpers in this patch as well.
>
> I've sometimes added runtime type safety to to_* helpers for early
> warning of mistakes that happen when refactoring...
>
> static inline struct vfio_pci_device *
> to_vfio_pci_device(struct vfio_device *core_dev)
> {
> if (dev_WARN_ONCE(core_dev->dev, core_dev->ops != &vfio_pci_ops,
> "not a vfio_pci_device!\n"))
> return NULL;
I personally didn't see any bug related to the problem presented by you.
For this discussion let's assume that the extra check can be sometimes useful,
however the "return NULL" construction is extremely harmful. This requires callers
to check if it is not NULL, which is wrong for container_of(..) callers. Such checks
are sort of wrong assumptions later.
Just as an example of this bad behaviour, I have series in progress that fixes it:
https://elixir.bootlin.com/linux/v5.12-rc2/source/drivers/infiniband/hw/bnxt_re/ib_verbs.c#L1100
srq = container_of(init_attr->srq, struct bnxt_re_srq, ib_srq);
if (!srq) {
ibdev_err(&rdev->ibdev, "SRQ not found");
return -EINVAL;
}
> return container_of(core_vdev, struct vfio_pci_device, vdev);
> }
>
> ...but typed ops is already a significant idiomatic improvement.
Everything here is an improvement, in-kernel vfio implementation is too far
from canonical kernel code and I'm glad that Jason proposes way to improve
this situation.
Thanks
>
>
> >
> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> > ---
> > Documentation/driver-api/vfio.rst | 18 ++++----
> > drivers/vfio/fsl-mc/vfio_fsl_mc.c | 36 +++++++++------
> > drivers/vfio/mdev/vfio_mdev.c | 33 +++++++-------
> > drivers/vfio/pci/vfio_pci.c | 47 ++++++++++++--------
> > drivers/vfio/platform/vfio_platform_common.c | 33 ++++++++------
> > drivers/vfio/vfio.c | 20 ++++-----
> > include/linux/vfio.h | 16 +++----
> > 7 files changed, 117 insertions(+), 86 deletions(-)
> >
> > diff --git a/Documentation/driver-api/vfio.rst b/Documentation/driver-api/vfio.rst
> > index d3a02300913a7f..3337f337293a32 100644
> > --- a/Documentation/driver-api/vfio.rst
> > +++ b/Documentation/driver-api/vfio.rst
> > @@ -269,20 +269,22 @@ ready before calling it. The driver provides an ops structure for callbacks
> > similar to a file operations structure::
> >
> > struct vfio_device_ops {
> > - int (*open)(void *device_data);
> > - void (*release)(void *device_data);
> > - ssize_t (*read)(void *device_data, char __user *buf,
> > + int (*open)(struct vfio_device *vdev);
> > + void (*release)(struct vfio_device *vdev);
> > + ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
> > size_t count, loff_t *ppos);
> > - ssize_t (*write)(void *device_data, const char __user *buf,
> > + ssize_t (*write)(struct vfio_device *vdev,
> > + const char __user *buf,
> > size_t size, loff_t *ppos);
> > - long (*ioctl)(void *device_data, unsigned int cmd,
> > + long (*ioctl)(struct vfio_device *vdev, unsigned int cmd,
> > unsigned long arg);
> > - int (*mmap)(void *device_data, struct vm_area_struct *vma);
> > + int (*mmap)(struct vfio_device *vdev,
> > + struct vm_area_struct *vma);
> > };
> >
> > -Each function is passed the device_data that was originally registered
> > +Each function is passed the vdev that was originally registered
> > in the vfio_register_group_dev() call above. This allows the bus driver
> > -an easy place to store its opaque, private data. The open/release
> > +to obtain its private data using container_of(). The open/release
> > callbacks are issued when a new file descriptor is created for a
> > device (via VFIO_GROUP_GET_DEVICE_FD). The ioctl interface provides
> > a direct pass through for VFIO_DEVICE_* ioctls. The read/write/mmap
> > diff --git a/drivers/vfio/fsl-mc/vfio_fsl_mc.c b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
> > index ddee6ed20c4523..74a5de1b791934 100644
> > --- a/drivers/vfio/fsl-mc/vfio_fsl_mc.c
> > +++ b/drivers/vfio/fsl-mc/vfio_fsl_mc.c
> > @@ -135,9 +135,10 @@ static void vfio_fsl_mc_regions_cleanup(struct vfio_fsl_mc_device *vdev)
> > kfree(vdev->regions);
> > }
> >
> > -static int vfio_fsl_mc_open(void *device_data)
> > +static int vfio_fsl_mc_open(struct vfio_device *core_vdev)
> > {
> > - struct vfio_fsl_mc_device *vdev = device_data;
> > + struct vfio_fsl_mc_device *vdev =
> > + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> > int ret;
> >
> > if (!try_module_get(THIS_MODULE))
> > @@ -161,9 +162,10 @@ static int vfio_fsl_mc_open(void *device_data)
> > return ret;
> > }
> >
> > -static void vfio_fsl_mc_release(void *device_data)
> > +static void vfio_fsl_mc_release(struct vfio_device *core_vdev)
> > {
> > - struct vfio_fsl_mc_device *vdev = device_data;
> > + struct vfio_fsl_mc_device *vdev =
> > + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> > int ret;
> >
> > mutex_lock(&vdev->reflck->lock);
> > @@ -197,11 +199,12 @@ static void vfio_fsl_mc_release(void *device_data)
> > module_put(THIS_MODULE);
> > }
> >
> > -static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd,
> > - unsigned long arg)
> > +static long vfio_fsl_mc_ioctl(struct vfio_device *core_vdev,
> > + unsigned int cmd, unsigned long arg)
> > {
> > unsigned long minsz;
> > - struct vfio_fsl_mc_device *vdev = device_data;
> > + struct vfio_fsl_mc_device *vdev =
> > + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> > struct fsl_mc_device *mc_dev = vdev->mc_dev;
> >
> > switch (cmd) {
> > @@ -327,10 +330,11 @@ static long vfio_fsl_mc_ioctl(void *device_data, unsigned int cmd,
> > }
> > }
> >
> > -static ssize_t vfio_fsl_mc_read(void *device_data, char __user *buf,
> > +static ssize_t vfio_fsl_mc_read(struct vfio_device *core_vdev, char __user *buf,
> > size_t count, loff_t *ppos)
> > {
> > - struct vfio_fsl_mc_device *vdev = device_data;
> > + struct vfio_fsl_mc_device *vdev =
> > + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> > unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos);
> > loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK;
> > struct fsl_mc_device *mc_dev = vdev->mc_dev;
> > @@ -404,10 +408,12 @@ static int vfio_fsl_mc_send_command(void __iomem *ioaddr, uint64_t *cmd_data)
> > return 0;
> > }
> >
> > -static ssize_t vfio_fsl_mc_write(void *device_data, const char __user *buf,
> > - size_t count, loff_t *ppos)
> > +static ssize_t vfio_fsl_mc_write(struct vfio_device *core_vdev,
> > + const char __user *buf, size_t count,
> > + loff_t *ppos)
> > {
> > - struct vfio_fsl_mc_device *vdev = device_data;
> > + struct vfio_fsl_mc_device *vdev =
> > + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> > unsigned int index = VFIO_FSL_MC_OFFSET_TO_INDEX(*ppos);
> > loff_t off = *ppos & VFIO_FSL_MC_OFFSET_MASK;
> > struct fsl_mc_device *mc_dev = vdev->mc_dev;
> > @@ -468,9 +474,11 @@ static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region,
> > size, vma->vm_page_prot);
> > }
> >
> > -static int vfio_fsl_mc_mmap(void *device_data, struct vm_area_struct *vma)
> > +static int vfio_fsl_mc_mmap(struct vfio_device *core_vdev,
> > + struct vm_area_struct *vma)
> > {
> > - struct vfio_fsl_mc_device *vdev = device_data;
> > + struct vfio_fsl_mc_device *vdev =
> > + container_of(core_vdev, struct vfio_fsl_mc_device, vdev);
> > struct fsl_mc_device *mc_dev = vdev->mc_dev;
> > unsigned int index;
> >
> > diff --git a/drivers/vfio/mdev/vfio_mdev.c b/drivers/vfio/mdev/vfio_mdev.c
> > index 4469aaf31b56cb..e7309caa99c71b 100644
> > --- a/drivers/vfio/mdev/vfio_mdev.c
> > +++ b/drivers/vfio/mdev/vfio_mdev.c
> > @@ -25,10 +25,11 @@ struct mdev_vfio_device {
> > struct vfio_device vdev;
> > };
> >
> > -static int vfio_mdev_open(void *device_data)
> > +static int vfio_mdev_open(struct vfio_device *core_vdev)
> > {
> > - struct mdev_device *mdev = device_data;
> > + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> > struct mdev_parent *parent = mdev->parent;
> > +
> > int ret;
> >
> > if (unlikely(!parent->ops->open))
> > @@ -44,9 +45,9 @@ static int vfio_mdev_open(void *device_data)
> > return ret;
> > }
> >
> > -static void vfio_mdev_release(void *device_data)
> > +static void vfio_mdev_release(struct vfio_device *core_vdev)
> > {
> > - struct mdev_device *mdev = device_data;
> > + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> > struct mdev_parent *parent = mdev->parent;
> >
> > if (likely(parent->ops->release))
> > @@ -55,10 +56,10 @@ static void vfio_mdev_release(void *device_data)
> > module_put(THIS_MODULE);
> > }
> >
> > -static long vfio_mdev_unlocked_ioctl(void *device_data,
> > +static long vfio_mdev_unlocked_ioctl(struct vfio_device *core_vdev,
> > unsigned int cmd, unsigned long arg)
> > {
> > - struct mdev_device *mdev = device_data;
> > + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> > struct mdev_parent *parent = mdev->parent;
> >
> > if (unlikely(!parent->ops->ioctl))
> > @@ -67,10 +68,10 @@ static long vfio_mdev_unlocked_ioctl(void *device_data,
> > return parent->ops->ioctl(mdev, cmd, arg);
> > }
> >
> > -static ssize_t vfio_mdev_read(void *device_data, char __user *buf,
> > +static ssize_t vfio_mdev_read(struct vfio_device *core_vdev, char __user *buf,
> > size_t count, loff_t *ppos)
> > {
> > - struct mdev_device *mdev = device_data;
> > + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> > struct mdev_parent *parent = mdev->parent;
> >
> > if (unlikely(!parent->ops->read))
> > @@ -79,10 +80,11 @@ static ssize_t vfio_mdev_read(void *device_data, char __user *buf,
> > return parent->ops->read(mdev, buf, count, ppos);
> > }
> >
> > -static ssize_t vfio_mdev_write(void *device_data, const char __user *buf,
> > - size_t count, loff_t *ppos)
> > +static ssize_t vfio_mdev_write(struct vfio_device *core_vdev,
> > + const char __user *buf, size_t count,
> > + loff_t *ppos)
> > {
> > - struct mdev_device *mdev = device_data;
> > + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> > struct mdev_parent *parent = mdev->parent;
> >
> > if (unlikely(!parent->ops->write))
> > @@ -91,9 +93,10 @@ static ssize_t vfio_mdev_write(void *device_data, const char __user *buf,
> > return parent->ops->write(mdev, buf, count, ppos);
> > }
> >
> > -static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma)
> > +static int vfio_mdev_mmap(struct vfio_device *core_vdev,
> > + struct vm_area_struct *vma)
> > {
> > - struct mdev_device *mdev = device_data;
> > + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> > struct mdev_parent *parent = mdev->parent;
> >
> > if (unlikely(!parent->ops->mmap))
> > @@ -102,9 +105,9 @@ static int vfio_mdev_mmap(void *device_data, struct vm_area_struct *vma)
> > return parent->ops->mmap(mdev, vma);
> > }
> >
> > -static void vfio_mdev_request(void *device_data, unsigned int count)
> > +static void vfio_mdev_request(struct vfio_device *core_vdev, unsigned int count)
> > {
> > - struct mdev_device *mdev = device_data;
> > + struct mdev_device *mdev = to_mdev_device(core_vdev->dev);
> > struct mdev_parent *parent = mdev->parent;
> >
> > if (parent->ops->request)
> > diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
> > index fae573c6f86bdf..af5696a96a76e0 100644
> > --- a/drivers/vfio/pci/vfio_pci.c
> > +++ b/drivers/vfio/pci/vfio_pci.c
> > @@ -553,9 +553,10 @@ static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val)
> > vfio_device_put(pf_dev);
> > }
> >
> > -static void vfio_pci_release(void *device_data)
> > +static void vfio_pci_release(struct vfio_device *core_vdev)
> > {
> > - struct vfio_pci_device *vdev = device_data;
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> >
> > mutex_lock(&vdev->reflck->lock);
> >
> > @@ -581,9 +582,10 @@ static void vfio_pci_release(void *device_data)
> > module_put(THIS_MODULE);
> > }
> >
> > -static int vfio_pci_open(void *device_data)
> > +static int vfio_pci_open(struct vfio_device *core_vdev)
> > {
> > - struct vfio_pci_device *vdev = device_data;
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> > int ret = 0;
> >
> > if (!try_module_get(THIS_MODULE))
> > @@ -797,10 +799,11 @@ struct vfio_devices {
> > int max_index;
> > };
> >
> > -static long vfio_pci_ioctl(void *device_data,
> > +static long vfio_pci_ioctl(struct vfio_device *core_vdev,
> > unsigned int cmd, unsigned long arg)
> > {
> > - struct vfio_pci_device *vdev = device_data;
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> > unsigned long minsz;
> >
> > if (cmd == VFIO_DEVICE_GET_INFO) {
> > @@ -1402,11 +1405,10 @@ static long vfio_pci_ioctl(void *device_data,
> > return -ENOTTY;
> > }
> >
> > -static ssize_t vfio_pci_rw(void *device_data, char __user *buf,
> > +static ssize_t vfio_pci_rw(struct vfio_pci_device *vdev, char __user *buf,
> > size_t count, loff_t *ppos, bool iswrite)
> > {
> > unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos);
> > - struct vfio_pci_device *vdev = device_data;
> >
> > if (index >= VFIO_PCI_NUM_REGIONS + vdev->num_regions)
> > return -EINVAL;
> > @@ -1434,22 +1436,28 @@ static ssize_t vfio_pci_rw(void *device_data, char __user *buf,
> > return -EINVAL;
> > }
> >
> > -static ssize_t vfio_pci_read(void *device_data, char __user *buf,
> > +static ssize_t vfio_pci_read(struct vfio_device *core_vdev, char __user *buf,
> > size_t count, loff_t *ppos)
> > {
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> > +
> > if (!count)
> > return 0;
> >
> > - return vfio_pci_rw(device_data, buf, count, ppos, false);
> > + return vfio_pci_rw(vdev, buf, count, ppos, false);
> > }
> >
> > -static ssize_t vfio_pci_write(void *device_data, const char __user *buf,
> > +static ssize_t vfio_pci_write(struct vfio_device *core_vdev, const char __user *buf,
> > size_t count, loff_t *ppos)
> > {
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> > +
> > if (!count)
> > return 0;
> >
> > - return vfio_pci_rw(device_data, (char __user *)buf, count, ppos, true);
> > + return vfio_pci_rw(vdev, (char __user *)buf, count, ppos, true);
> > }
> >
> > /* Return 1 on zap and vma_lock acquired, 0 on contention (only with @try) */
> > @@ -1646,9 +1654,10 @@ static const struct vm_operations_struct vfio_pci_mmap_ops = {
> > .fault = vfio_pci_mmap_fault,
> > };
> >
> > -static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
> > +static int vfio_pci_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma)
> > {
> > - struct vfio_pci_device *vdev = device_data;
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> > struct pci_dev *pdev = vdev->pdev;
> > unsigned int index;
> > u64 phys_len, req_len, pgoff, req_start;
> > @@ -1714,9 +1723,10 @@ static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
> > return 0;
> > }
> >
> > -static void vfio_pci_request(void *device_data, unsigned int count)
> > +static void vfio_pci_request(struct vfio_device *core_vdev, unsigned int count)
> > {
> > - struct vfio_pci_device *vdev = device_data;
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> > struct pci_dev *pdev = vdev->pdev;
> >
> > mutex_lock(&vdev->igate);
> > @@ -1830,9 +1840,10 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev,
> >
> > #define VF_TOKEN_ARG "vf_token="
> >
> > -static int vfio_pci_match(void *device_data, char *buf)
> > +static int vfio_pci_match(struct vfio_device *core_vdev, char *buf)
> > {
> > - struct vfio_pci_device *vdev = device_data;
> > + struct vfio_pci_device *vdev =
> > + container_of(core_vdev, struct vfio_pci_device, vdev);
> > bool vf_token = false;
> > uuid_t uuid;
> > int ret;
> > diff --git a/drivers/vfio/platform/vfio_platform_common.c b/drivers/vfio/platform/vfio_platform_common.c
> > index 6eb749250ee41c..f5f6b537084a67 100644
> > --- a/drivers/vfio/platform/vfio_platform_common.c
> > +++ b/drivers/vfio/platform/vfio_platform_common.c
> > @@ -218,9 +218,10 @@ static int vfio_platform_call_reset(struct vfio_platform_device *vdev,
> > return -EINVAL;
> > }
> >
> > -static void vfio_platform_release(void *device_data)
> > +static void vfio_platform_release(struct vfio_device *core_vdev)
> > {
> > - struct vfio_platform_device *vdev = device_data;
> > + struct vfio_platform_device *vdev =
> > + container_of(core_vdev, struct vfio_platform_device, vdev);
> >
> > mutex_lock(&driver_lock);
> >
> > @@ -244,9 +245,10 @@ static void vfio_platform_release(void *device_data)
> > module_put(vdev->parent_module);
> > }
> >
> > -static int vfio_platform_open(void *device_data)
> > +static int vfio_platform_open(struct vfio_device *core_vdev)
> > {
> > - struct vfio_platform_device *vdev = device_data;
> > + struct vfio_platform_device *vdev =
> > + container_of(core_vdev, struct vfio_platform_device, vdev);
> > int ret;
> >
> > if (!try_module_get(vdev->parent_module))
> > @@ -293,10 +295,12 @@ static int vfio_platform_open(void *device_data)
> > return ret;
> > }
> >
> > -static long vfio_platform_ioctl(void *device_data,
> > +static long vfio_platform_ioctl(struct vfio_device *core_vdev,
> > unsigned int cmd, unsigned long arg)
> > {
> > - struct vfio_platform_device *vdev = device_data;
> > + struct vfio_platform_device *vdev =
> > + container_of(core_vdev, struct vfio_platform_device, vdev);
> > +
> > unsigned long minsz;
> >
> > if (cmd == VFIO_DEVICE_GET_INFO) {
> > @@ -455,10 +459,11 @@ static ssize_t vfio_platform_read_mmio(struct vfio_platform_region *reg,
> > return -EFAULT;
> > }
> >
> > -static ssize_t vfio_platform_read(void *device_data, char __user *buf,
> > - size_t count, loff_t *ppos)
> > +static ssize_t vfio_platform_read(struct vfio_device *core_vdev,
> > + char __user *buf, size_t count, loff_t *ppos)
> > {
> > - struct vfio_platform_device *vdev = device_data;
> > + struct vfio_platform_device *vdev =
> > + container_of(core_vdev, struct vfio_platform_device, vdev);
> > unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos);
> > loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK;
> >
> > @@ -531,10 +536,11 @@ static ssize_t vfio_platform_write_mmio(struct vfio_platform_region *reg,
> > return -EFAULT;
> > }
> >
> > -static ssize_t vfio_platform_write(void *device_data, const char __user *buf,
> > +static ssize_t vfio_platform_write(struct vfio_device *core_vdev, const char __user *buf,
> > size_t count, loff_t *ppos)
> > {
> > - struct vfio_platform_device *vdev = device_data;
> > + struct vfio_platform_device *vdev =
> > + container_of(core_vdev, struct vfio_platform_device, vdev);
> > unsigned int index = VFIO_PLATFORM_OFFSET_TO_INDEX(*ppos);
> > loff_t off = *ppos & VFIO_PLATFORM_OFFSET_MASK;
> >
> > @@ -573,9 +579,10 @@ static int vfio_platform_mmap_mmio(struct vfio_platform_region region,
> > req_len, vma->vm_page_prot);
> > }
> >
> > -static int vfio_platform_mmap(void *device_data, struct vm_area_struct *vma)
> > +static int vfio_platform_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma)
> > {
> > - struct vfio_platform_device *vdev = device_data;
> > + struct vfio_platform_device *vdev =
> > + container_of(core_vdev, struct vfio_platform_device, vdev);
> > unsigned int index;
> >
> > index = vma->vm_pgoff >> (VFIO_PLATFORM_OFFSET_SHIFT - PAGE_SHIFT);
> > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> > index 2d6d7cc1d1ebf9..01de47d1810b6b 100644
> > --- a/drivers/vfio/vfio.c
> > +++ b/drivers/vfio/vfio.c
> > @@ -832,7 +832,7 @@ static struct vfio_device *vfio_device_get_from_name(struct vfio_group *group,
> > int ret;
> >
> > if (it->ops->match) {
> > - ret = it->ops->match(it->device_data, buf);
> > + ret = it->ops->match(it, buf);
> > if (ret < 0) {
> > device = ERR_PTR(ret);
> > break;
> > @@ -893,7 +893,7 @@ void vfio_unregister_group_dev(struct vfio_device *device)
> > rc = try_wait_for_completion(&device->comp);
> > while (rc <= 0) {
> > if (device->ops->request)
> > - device->ops->request(device->device_data, i++);
> > + device->ops->request(device, i++);
> >
> > if (interrupted) {
> > rc = wait_for_completion_timeout(&device->comp,
> > @@ -1379,7 +1379,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
> > if (IS_ERR(device))
> > return PTR_ERR(device);
> >
> > - ret = device->ops->open(device->device_data);
> > + ret = device->ops->open(device);
> > if (ret) {
> > vfio_device_put(device);
> > return ret;
> > @@ -1391,7 +1391,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
> > */
> > ret = get_unused_fd_flags(O_CLOEXEC);
> > if (ret < 0) {
> > - device->ops->release(device->device_data);
> > + device->ops->release(device);
> > vfio_device_put(device);
> > return ret;
> > }
> > @@ -1401,7 +1401,7 @@ static int vfio_group_get_device_fd(struct vfio_group *group, char *buf)
> > if (IS_ERR(filep)) {
> > put_unused_fd(ret);
> > ret = PTR_ERR(filep);
> > - device->ops->release(device->device_data);
> > + device->ops->release(device);
> > vfio_device_put(device);
> > return ret;
> > }
> > @@ -1558,7 +1558,7 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep)
> > {
> > struct vfio_device *device = filep->private_data;
> >
> > - device->ops->release(device->device_data);
> > + device->ops->release(device);
> >
> > vfio_group_try_dissolve_container(device->group);
> >
> > @@ -1575,7 +1575,7 @@ static long vfio_device_fops_unl_ioctl(struct file *filep,
> > if (unlikely(!device->ops->ioctl))
> > return -EINVAL;
> >
> > - return device->ops->ioctl(device->device_data, cmd, arg);
> > + return device->ops->ioctl(device, cmd, arg);
> > }
> >
> > static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
> > @@ -1586,7 +1586,7 @@ static ssize_t vfio_device_fops_read(struct file *filep, char __user *buf,
> > if (unlikely(!device->ops->read))
> > return -EINVAL;
> >
> > - return device->ops->read(device->device_data, buf, count, ppos);
> > + return device->ops->read(device, buf, count, ppos);
> > }
> >
> > static ssize_t vfio_device_fops_write(struct file *filep,
> > @@ -1598,7 +1598,7 @@ static ssize_t vfio_device_fops_write(struct file *filep,
> > if (unlikely(!device->ops->write))
> > return -EINVAL;
> >
> > - return device->ops->write(device->device_data, buf, count, ppos);
> > + return device->ops->write(device, buf, count, ppos);
> > }
> >
> > static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
> > @@ -1608,7 +1608,7 @@ static int vfio_device_fops_mmap(struct file *filep, struct vm_area_struct *vma)
> > if (unlikely(!device->ops->mmap))
> > return -EINVAL;
> >
> > - return device->ops->mmap(device->device_data, vma);
> > + return device->ops->mmap(device, vma);
> > }
> >
> > static const struct file_operations vfio_device_fops = {
> > diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> > index 4995faf51efeae..784c34c0a28763 100644
> > --- a/include/linux/vfio.h
> > +++ b/include/linux/vfio.h
> > @@ -44,17 +44,17 @@ struct vfio_device {
> > */
> > struct vfio_device_ops {
> > char *name;
> > - int (*open)(void *device_data);
> > - void (*release)(void *device_data);
> > - ssize_t (*read)(void *device_data, char __user *buf,
> > + int (*open)(struct vfio_device *vdev);
> > + void (*release)(struct vfio_device *vdev);
> > + ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
> > size_t count, loff_t *ppos);
> > - ssize_t (*write)(void *device_data, const char __user *buf,
> > + ssize_t (*write)(struct vfio_device *vdev, const char __user *buf,
> > size_t count, loff_t *size);
> > - long (*ioctl)(void *device_data, unsigned int cmd,
> > + long (*ioctl)(struct vfio_device *vdev, unsigned int cmd,
> > unsigned long arg);
> > - int (*mmap)(void *device_data, struct vm_area_struct *vma);
> > - void (*request)(void *device_data, unsigned int count);
> > - int (*match)(void *device_data, char *buf);
> > + int (*mmap)(struct vfio_device *vdev, struct vm_area_struct *vma);
> > + void (*request)(struct vfio_device *vdev, unsigned int count);
> > + int (*match)(struct vfio_device *vdev, char *buf);
> > };
> >
> > extern struct iommu_group *vfio_iommu_group_get(struct device *dev);
> > --
> > 2.30.1
> >
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops
2021-03-09 21:38 ` [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops Jason Gunthorpe
@ 2021-03-10 7:26 ` Christoph Hellwig
2021-03-12 13:04 ` Liu, Yi L
1 sibling, 0 replies; 17+ messages in thread
From: Christoph Hellwig @ 2021-03-10 7:26 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alex Williamson, Cornelia Huck, Jonathan Corbet, kvm, linux-doc,
Raj, Ashok, Dan Williams, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Tue, Mar 09, 2021 at 05:38:44PM -0400, Jason Gunthorpe wrote:
> The pattern also makes it clear that vfio_register_group_dev() must be
> last in the sequence, as once it is called the core code can immediately
> start calling ops. The init/register gap is provided to allow for the
> driver to do setup before ops can be called and thus avoid races.
Yes, APIs that init and register together are generatelly a rather
bad idea.
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 10/10] vfio: Remove device_data from the vfio bus driver API
2021-03-09 21:38 ` [PATCH 10/10] vfio: Remove device_data from the vfio bus driver API Jason Gunthorpe
@ 2021-03-10 7:37 ` Christoph Hellwig
0 siblings, 0 replies; 17+ messages in thread
From: Christoph Hellwig @ 2021-03-10 7:37 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, kvm, Kirti Wankhede, linux-doc, Raj, Ashok,
Dan Williams, Daniel Vetter, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *'
2021-03-10 5:52 ` Dan Williams
2021-03-10 6:24 ` Leon Romanovsky
@ 2021-03-10 12:58 ` Jason Gunthorpe
2021-03-10 20:01 ` Dan Williams
1 sibling, 1 reply; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-10 12:58 UTC (permalink / raw)
To: Dan Williams
Cc: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, KVM list, Kirti Wankhede, Linux Doc Mailing List,
Raj, Ashok, Daniel Vetter, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On Tue, Mar 09, 2021 at 09:52:30PM -0800, Dan Williams wrote:
> On Tue, Mar 9, 2021 at 1:39 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > This is the standard kernel pattern, the ops associated with a struct get
> > the struct pointer in for typesafety. The expected design is to use
> > container_of to cleanly go from the subsystem level type to the driver
> > level type without having any type erasure in a void *.
>
> This patch alone is worth the price of admission.
Right, this is how I got here as I was going to do the mdev stuff and
we end up doing transformations like this:
-static long mtty_ioctl(struct mdev_device *mdev, unsigned int cmd,
+static long mtty_ioctl(struct vfio_device *vdev, unsigned int cmd,
unsigned long arg)
And if the 'struct vfio_device *vdev' was left a 'void *' then the
compiler doesn't get to help any more :(
> Seems like it would be worth adding
> to_vfio_{pci,platform,fsl_mc}_device() helpers in this patch as well.
I have mixed feelings on these one-liners. If people feel they are
worthwhile I'll add them
> I've sometimes added runtime type safety to to_* helpers for early
> warning of mistakes that happen when refactoring...
>
> static inline struct vfio_pci_device *
> to_vfio_pci_device(struct vfio_device *core_dev)
> {
> if (dev_WARN_ONCE(core_dev->dev, core_dev->ops != &vfio_pci_ops,
> "not a vfio_pci_device!\n"))
> return NULL;
> return container_of(core_vdev, struct vfio_pci_device, vdev);
In this case I don't think we need to worry as everything is tidy in a
single module such that the compilation units can't see other
container_of options anyhow and vfio core isn't going to accidently
call an ops with the wrong type.
Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *'
2021-03-10 12:58 ` Jason Gunthorpe
@ 2021-03-10 20:01 ` Dan Williams
0 siblings, 0 replies; 17+ messages in thread
From: Dan Williams @ 2021-03-10 20:01 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, KVM list, Kirti Wankhede, Linux Doc Mailing List,
Raj, Ashok, Daniel Vetter, Christoph Hellwig, Leon Romanovsky,
Max Gurtovoy, Tarun Gupta
On Wed, Mar 10, 2021 at 4:58 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Tue, Mar 09, 2021 at 09:52:30PM -0800, Dan Williams wrote:
> > On Tue, Mar 9, 2021 at 1:39 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
> > >
> > > This is the standard kernel pattern, the ops associated with a struct get
> > > the struct pointer in for typesafety. The expected design is to use
> > > container_of to cleanly go from the subsystem level type to the driver
> > > level type without having any type erasure in a void *.
> >
> > This patch alone is worth the price of admission.
>
> Right, this is how I got here as I was going to do the mdev stuff and
> we end up doing transformations like this:
>
> -static long mtty_ioctl(struct mdev_device *mdev, unsigned int cmd,
> +static long mtty_ioctl(struct vfio_device *vdev, unsigned int cmd,
> unsigned long arg)
>
> And if the 'struct vfio_device *vdev' was left a 'void *' then the
> compiler doesn't get to help any more :(
>
> > Seems like it would be worth adding
> > to_vfio_{pci,platform,fsl_mc}_device() helpers in this patch as well.
>
> I have mixed feelings on these one-liners. If people feel they are
> worthwhile I'll add them
It's only worthwhile in my opinion if it makes the diffstat more
favorable for lines removed.
>
> > I've sometimes added runtime type safety to to_* helpers for early
> > warning of mistakes that happen when refactoring...
> >
> > static inline struct vfio_pci_device *
> > to_vfio_pci_device(struct vfio_device *core_dev)
> > {
> > if (dev_WARN_ONCE(core_dev->dev, core_dev->ops != &vfio_pci_ops,
> > "not a vfio_pci_device!\n"))
> > return NULL;
> > return container_of(core_vdev, struct vfio_pci_device, vdev);
>
> In this case I don't think we need to worry as everything is tidy in a
> single module such that the compilation units can't see other
> container_of options anyhow and vfio core isn't going to accidently
> call an ops with the wrong type.
Sounds good.
You can add:
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 00/10] Embed struct vfio_device in all sub-structures
2021-03-09 21:38 [PATCH 00/10] Embed struct vfio_device in all sub-structures Jason Gunthorpe
` (2 preceding siblings ...)
2021-03-09 21:38 ` [PATCH 10/10] vfio: Remove device_data from the vfio bus driver API Jason Gunthorpe
@ 2021-03-10 23:52 ` Alex Williamson
2021-03-10 23:57 ` Jason Gunthorpe
3 siblings, 1 reply; 17+ messages in thread
From: Alex Williamson @ 2021-03-10 23:52 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Cornelia Huck, Jonathan Corbet, Diana Craciun, Eric Auger, kvm,
Kirti Wankhede, linux-doc, Raj, Ashok, Dan Williams,
Daniel Vetter, Christoph Hellwig, Leon Romanovsky, Max Gurtovoy,
Tarun Gupta
On Tue, 9 Mar 2021 17:38:42 -0400
Jason Gunthorpe <jgg@nvidia.com> wrote:
> This series:
>
> The main focus of this series is to make VFIO follow the normal kernel
> convention of structure embedding for structure inheritance instead of
> linking using a 'void *opaque'. Here we focus on moving the vfio_device to
> be a member of every struct vfio_XX_device that is linked by a
> vfio_add_group_dev().
>
> In turn this allows 'struct vfio_device *' to be used everwhere, and the
> public API out of vfio.c can be cleaned to remove places using 'struct
> device *' and 'void *' as surrogates to refer to the device.
>
> While this has the minor trade off of moving 'struct vfio_device' the
> clarity of the design is worth it. I can speak directly to this idea, as
> I've invested a fair amount of time carefully working backwards what all
> the type-erased APIs are supposed to be and it is certainly not trivial or
> intuitive.
>
> When we get into mdev land things become even more inscrutable, and while
> I now have a pretty clear picture, it was hard to obtain. I think this
> agrees with the kernel style ideal of being explicit in typing and not
> sacrificing clarity to create opaque structs.
>
> After this series the general rules are:
> - Any vfio_XX_device * can be obtained at no cost from a vfio_device *
> using container_of(), and the reverse is possible by &XXdev->vdev
>
> This is similar to how 'struct pci_device' and 'struct device' are
> interrelated.
>
> This allows 'device_data' to be completely removed from the vfio.c API.
>
> - The drvdata for a struct device points at the vfio_XX_device that
> belongs to the driver that was probed. drvdata is removed from the core
> code, and only used as part of the implementation of the struct
> device_driver.
>
> - The lifetime of vfio_XX_device and vfio_device are identical, they are
> the same memory.
>
> This follows the existing model where vfio_del_group_dev() blocks until
> all vfio_device_put()'s are completed. This in turn means the struct
> device_driver remove() blocks, and thus under the driver_lock() a bound
> driver must have a valid drvdata pointing at both vfio device
> structs. A following series exploits this further.
>
> Most vfio_XX_device structs have data that duplicates the 'struct
> device *dev' member of vfio_device, a following series removes that
> duplication too.
>
> Jason
>
> Jason Gunthorpe (10):
> vfio: Simplify the lifetime logic for vfio_device
> vfio: Split creation of a vfio_device into init and register ops
> vfio/platform: Use vfio_init/register/unregister_group_dev
> vfio/fsl-mc: Use vfio_init/register/unregister_group_dev
> vfio/pci: Use vfio_init/register/unregister_group_dev
> vfio/mdev: Use vfio_init/register/unregister_group_dev
> vfio/mdev: Make to_mdev_device() into a static inline
> vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of
> 'void *'
> vfio/pci: Replace uses of vfio_device_data() with container_of
> vfio: Remove device_data from the vfio bus driver API
>
> Documentation/driver-api/vfio.rst | 48 ++--
> drivers/vfio/fsl-mc/vfio_fsl_mc.c | 69 +++---
> drivers/vfio/fsl-mc/vfio_fsl_mc_private.h | 1 +
> drivers/vfio/mdev/mdev_private.h | 5 +-
> drivers/vfio/mdev/vfio_mdev.c | 57 +++--
> drivers/vfio/pci/vfio_pci.c | 109 +++++----
> drivers/vfio/pci/vfio_pci_private.h | 1 +
> drivers/vfio/platform/vfio_amba.c | 8 +-
> drivers/vfio/platform/vfio_platform.c | 21 +-
> drivers/vfio/platform/vfio_platform_common.c | 56 ++---
> drivers/vfio/platform/vfio_platform_private.h | 5 +-
> drivers/vfio/vfio.c | 210 ++++++------------
> include/linux/vfio.h | 37 +--
> 13 files changed, 299 insertions(+), 328 deletions(-)
>
This looks great. As Christoph noted, addressing those init vs
register races in the bus drivers don't seem too difficult or out of
scope for this series. Thanks,
Alex
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 00/10] Embed struct vfio_device in all sub-structures
2021-03-10 23:52 ` [PATCH 00/10] Embed struct vfio_device in all sub-structures Alex Williamson
@ 2021-03-10 23:57 ` Jason Gunthorpe
0 siblings, 0 replies; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-10 23:57 UTC (permalink / raw)
To: Alex Williamson
Cc: Cornelia Huck, Jonathan Corbet, Diana Craciun, Eric Auger, kvm,
Kirti Wankhede, linux-doc, Raj, Ashok, Dan Williams,
Daniel Vetter, Christoph Hellwig, Leon Romanovsky, Max Gurtovoy,
Tarun Gupta
On Wed, Mar 10, 2021 at 04:52:47PM -0700, Alex Williamson wrote:
> This looks great. As Christoph noted, addressing those init vs
> register races in the bus drivers don't seem too difficult or out of
> scope for this series. Thanks,
Sure, I'm happy to add it. I need to check vfio-pci closely that there
is no hidden dependency, but fsl looked fine
I'll look at splitting patch 1 as well and send a v2.
Thanks
Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops
2021-03-09 21:38 ` [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops Jason Gunthorpe
2021-03-10 7:26 ` Christoph Hellwig
@ 2021-03-12 13:04 ` Liu, Yi L
2021-03-12 14:23 ` Jason Gunthorpe
1 sibling, 1 reply; 17+ messages in thread
From: Liu, Yi L @ 2021-03-12 13:04 UTC (permalink / raw)
To: Jason Gunthorpe, Alex Williamson, Cornelia Huck, Jonathan Corbet,
kvm@vger.kernel.org, linux-doc@vger.kernel.org
Cc: Raj, Ashok, Williams, Dan J, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
Hi Jason,
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, March 10, 2021 5:39 AM
>
[...]
> diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> index b7e18bde5aa8b3..ad8b579d67d34a 100644
> --- a/include/linux/vfio.h
> +++ b/include/linux/vfio.h
> @@ -15,6 +15,18 @@
> #include <linux/poll.h>
> #include <uapi/linux/vfio.h>
>
> +struct vfio_device {
> + struct device *dev;
> + const struct vfio_device_ops *ops;
> + struct vfio_group *group;
> +
> + /* Members below here are private, not for driver use */
> + refcount_t refcount;
> + struct completion comp;
> + struct list_head group_next;
> + void *device_data;
A dumb question. If these fields are not supposed to be used by
"external modules" like vfio_pci driver, how about defining a private
struct vfio_dev_prive within vfio.c and embed here?
Other parts look good to me.
Reviewed-by: Liu Yi L <yi.l.liu@intel.com>
Regards,
Yi Liu
^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *'
2021-03-09 21:38 ` [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *' Jason Gunthorpe
2021-03-10 5:52 ` Dan Williams
@ 2021-03-12 13:42 ` Liu, Yi L
2021-03-12 14:06 ` Jason Gunthorpe
1 sibling, 1 reply; 17+ messages in thread
From: Liu, Yi L @ 2021-03-12 13:42 UTC (permalink / raw)
To: Jason Gunthorpe, Alex Williamson, Cornelia Huck, Jonathan Corbet,
Diana Craciun, Eric Auger, kvm@vger.kernel.org, Kirti Wankhede,
linux-doc@vger.kernel.org
Cc: Raj, Ashok, Williams, Dan J, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
Hi Jason,
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, March 10, 2021 5:39 AM
[...]
>
> -Each function is passed the device_data that was originally registered
> +Each function is passed the vdev that was originally registered
> in the vfio_register_group_dev() call above. This allows the bus driver
> -an easy place to store its opaque, private data. The open/release
> +to obtain its private data using container_of(). The open/release
I think the is to let VFIO device drivers (e.g. vfio_pci driver) to obtain
its private data by using container_of instead of let the bus driver.
right?
Regards,
Yi Liu
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *'
2021-03-12 13:42 ` Liu, Yi L
@ 2021-03-12 14:06 ` Jason Gunthorpe
0 siblings, 0 replies; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-12 14:06 UTC (permalink / raw)
To: Liu, Yi L
Cc: Alex Williamson, Cornelia Huck, Jonathan Corbet, Diana Craciun,
Eric Auger, kvm@vger.kernel.org, Kirti Wankhede,
linux-doc@vger.kernel.org, Raj, Ashok, Williams, Dan J,
Daniel Vetter, Christoph Hellwig, Leon Romanovsky, Max Gurtovoy,
Tarun Gupta
On Fri, Mar 12, 2021 at 01:42:38PM +0000, Liu, Yi L wrote:
> > -Each function is passed the device_data that was originally registered
> > +Each function is passed the vdev that was originally registered
> > in the vfio_register_group_dev() call above. This allows the bus driver
> > -an easy place to store its opaque, private data. The open/release
> > +to obtain its private data using container_of(). The open/release
>
> I think the is to let VFIO device drivers (e.g. vfio_pci driver) to obtain
> its private data by using container_of instead of let the bus driver.
> right?
vfio_pci is the "bus driver" in VFIO speak
It replaces the void * vfio_device_data() with container_of
Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops
2021-03-12 13:04 ` Liu, Yi L
@ 2021-03-12 14:23 ` Jason Gunthorpe
2021-03-12 16:31 ` Christoph Hellwig
0 siblings, 1 reply; 17+ messages in thread
From: Jason Gunthorpe @ 2021-03-12 14:23 UTC (permalink / raw)
To: Liu, Yi L
Cc: Alex Williamson, Cornelia Huck, Jonathan Corbet,
kvm@vger.kernel.org, linux-doc@vger.kernel.org, Raj, Ashok,
Williams, Dan J, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, Mar 12, 2021 at 01:04:29PM +0000, Liu, Yi L wrote:
> Hi Jason,
>
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Wednesday, March 10, 2021 5:39 AM
> >
> [...]
> > diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> > index b7e18bde5aa8b3..ad8b579d67d34a 100644
> > +++ b/include/linux/vfio.h
> > @@ -15,6 +15,18 @@
> > #include <linux/poll.h>
> > #include <uapi/linux/vfio.h>
> >
> > +struct vfio_device {
> > + struct device *dev;
> > + const struct vfio_device_ops *ops;
> > + struct vfio_group *group;
> > +
> > + /* Members below here are private, not for driver use */
> > + refcount_t refcount;
> > + struct completion comp;
> > + struct list_head group_next;
> > + void *device_data;
>
> A dumb question. If these fields are not supposed to be used by
> "external modules" like vfio_pci driver, how about defining a private
> struct vfio_dev_prive within vfio.c and embed here?
This is rarely done, there should be a good reason to do it, as making
a private structure in a container_of system requires another memory
allocation.
'struct device' has this for instance, look at the 'p' member.
In this case I can't see much value
Jason
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops
2021-03-12 14:23 ` Jason Gunthorpe
@ 2021-03-12 16:31 ` Christoph Hellwig
0 siblings, 0 replies; 17+ messages in thread
From: Christoph Hellwig @ 2021-03-12 16:31 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Liu, Yi L, Alex Williamson, Cornelia Huck, Jonathan Corbet,
kvm@vger.kernel.org, linux-doc@vger.kernel.org, Raj, Ashok,
Williams, Dan J, Daniel Vetter, Christoph Hellwig,
Leon Romanovsky, Max Gurtovoy, Tarun Gupta
On Fri, Mar 12, 2021 at 10:23:26AM -0400, Jason Gunthorpe wrote:
> This is rarely done, there should be a good reason to do it, as making
> a private structure in a container_of system requires another memory
> allocation.
>
> 'struct device' has this for instance, look at the 'p' member.
>
> In this case I can't see much value
Agreed. Moving this code to the normal kernel pattern really not just
helps to find bugs and reduce complexity, but also makes it much easier
for random kernel developers to actually be able to understand the code.
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2021-03-12 16:32 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-03-09 21:38 [PATCH 00/10] Embed struct vfio_device in all sub-structures Jason Gunthorpe
2021-03-09 21:38 ` [PATCH 02/10] vfio: Split creation of a vfio_device into init and register ops Jason Gunthorpe
2021-03-10 7:26 ` Christoph Hellwig
2021-03-12 13:04 ` Liu, Yi L
2021-03-12 14:23 ` Jason Gunthorpe
2021-03-12 16:31 ` Christoph Hellwig
2021-03-09 21:38 ` [PATCH 08/10] vfio: Make vfio_device_ops pass a 'struct vfio_device *' instead of 'void *' Jason Gunthorpe
2021-03-10 5:52 ` Dan Williams
2021-03-10 6:24 ` Leon Romanovsky
2021-03-10 12:58 ` Jason Gunthorpe
2021-03-10 20:01 ` Dan Williams
2021-03-12 13:42 ` Liu, Yi L
2021-03-12 14:06 ` Jason Gunthorpe
2021-03-09 21:38 ` [PATCH 10/10] vfio: Remove device_data from the vfio bus driver API Jason Gunthorpe
2021-03-10 7:37 ` Christoph Hellwig
2021-03-10 23:52 ` [PATCH 00/10] Embed struct vfio_device in all sub-structures Alex Williamson
2021-03-10 23:57 ` Jason Gunthorpe
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).