[PATCH v2 0/3] vfio: use __aligned_u64 for ioctl structs

[PATCH v2 0/3] vfio: use __aligned_u64 for ioctl structs

[Stefan Hajnoczi]

v2:
- Rebased onto https://github.com/awilliam/linux-vfio.git next to get the
  vfio_iommu_type1_info pad field [Kevin]
- Fixed min(minsz, sizeof(dmabuf)) -> min(dmabuf.argsz, sizeof(dmabuf)) [Jason, Kevin]
- Squashed Patch 3 (vfio_iommu_type1_info) into Patch 1 since it is trivial now
  that the padding field is already there.

Jason Gunthorpe <jgg@nvidia.com> pointed out that u64 VFIO ioctl struct fields
have architecture-dependent alignment. iommufd already uses __aligned_u64 to
avoid this problem.

See the __aligned_u64 typedef in <uapi/linux/types.h> for details on why it is
a good idea for kernel<->user interfaces.

This series modifies the VFIO ioctl structs to use __aligned_u64. Some of the
changes preserve the existing memory layout on all architectures, so I put them
together into the first patch. The remaining patches are for structs where
explanation is necessary about why changing the memory layout does not break
the uapi.

Stefan Hajnoczi (3):
  vfio: trivially use __aligned_u64 for ioctl structs
  vfio: use __aligned_u64 in struct vfio_device_gfx_plane_info
  vfio: use __aligned_u64 in struct vfio_device_ioeventfd

 include/uapi/linux/vfio.h        | 26 ++++++++++++++------------
 drivers/gpu/drm/i915/gvt/kvmgt.c |  4 +++-
 samples/vfio-mdev/mbochs.c       |  6 ++++--
 samples/vfio-mdev/mdpy.c         |  4 +++-
 4 files changed, 24 insertions(+), 16 deletions(-)

-- 
2.41.0

[PATCH v2 1/3] vfio: trivially use __aligned_u64 for ioctl structs

[Stefan Hajnoczi]

u64 alignment behaves differently depending on the architecture and so
<uapi/linux/types.h> offers __aligned_u64 to achieve consistent behavior
in kernel<->userspace ABIs.

There are structs in <uapi/linux/vfio.h> that can trivially be updated
to __aligned_u64 because the struct sizes are multiples of 8 bytes.
There is no change in memory layout on any CPU architecture and
therefore this change is safe.

The commits that follow this one handle the trickier cases where
explanation about ABI breakage is necessary.

Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/uapi/linux/vfio.h | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index f9c6f3e2cf6e..94007ca348ed 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -277,8 +277,8 @@ struct vfio_region_info {
 #define VFIO_REGION_INFO_FLAG_CAPS	(1 << 3) /* Info supports caps */
 	__u32	index;		/* Region index */
 	__u32	cap_offset;	/* Offset within info struct of first cap */
-	__u64	size;		/* Region size (bytes) */
-	__u64	offset;		/* Region offset from start of device fd */
+	__aligned_u64	size;	/* Region size (bytes) */
+	__aligned_u64	offset;	/* Region offset from start of device fd */
 };
 #define VFIO_DEVICE_GET_REGION_INFO	_IO(VFIO_TYPE, VFIO_BASE + 8)
 
@@ -294,8 +294,8 @@ struct vfio_region_info {
 #define VFIO_REGION_INFO_CAP_SPARSE_MMAP	1
 
 struct vfio_region_sparse_mmap_area {
-	__u64	offset;	/* Offset of mmap'able area within region */
-	__u64	size;	/* Size of mmap'able area */
+	__aligned_u64	offset;	/* Offset of mmap'able area within region */
+	__aligned_u64	size;	/* Size of mmap'able area */
 };
 
 struct vfio_region_info_cap_sparse_mmap {
@@ -450,9 +450,9 @@ struct vfio_device_migration_info {
 					     VFIO_DEVICE_STATE_V1_RESUMING)
 
 	__u32 reserved;
-	__u64 pending_bytes;
-	__u64 data_offset;
-	__u64 data_size;
+	__aligned_u64 pending_bytes;
+	__aligned_u64 data_offset;
+	__aligned_u64 data_size;
 };
 
 /*
@@ -476,7 +476,7 @@ struct vfio_device_migration_info {
 
 struct vfio_region_info_cap_nvlink2_ssatgt {
 	struct vfio_info_cap_header header;
-	__u64 tgt;
+	__aligned_u64 tgt;
 };
 
 /*
@@ -1443,7 +1443,7 @@ struct vfio_iommu_type1_info {
 	__u32	flags;
 #define VFIO_IOMMU_INFO_PGSIZES (1 << 0)	/* supported page sizes info */
 #define VFIO_IOMMU_INFO_CAPS	(1 << 1)	/* Info supports caps */
-	__u64	iova_pgsizes;	/* Bitmap of supported page sizes */
+	__aligned_u64	iova_pgsizes;		/* Bitmap of supported page sizes */
 	__u32   cap_offset;	/* Offset within info struct of first cap */
 	__u32   pad;
 };
-- 
2.41.0

Re: [PATCH v2 1/3] vfio: trivially use __aligned_u64 for ioctl structs

[Philippe Mathieu-Daudé]

On 29/8/23 20:27, Stefan Hajnoczi wrote:
> u64 alignment behaves differently depending on the architecture and so
> <uapi/linux/types.h> offers __aligned_u64 to achieve consistent behavior
> in kernel<->userspace ABIs.
> 
> There are structs in <uapi/linux/vfio.h> that can trivially be updated
> to __aligned_u64 because the struct sizes are multiples of 8 bytes.
> There is no change in memory layout on any CPU architecture and
> therefore this change is safe.
> 
> The commits that follow this one handle the trickier cases where
> explanation about ABI breakage is necessary.
> 
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>   include/uapi/linux/vfio.h | 18 +++++++++---------
>   1 file changed, 9 insertions(+), 9 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

RE: [PATCH v2 1/3] vfio: trivially use __aligned_u64 for ioctl structs

[Tian, Kevin]

> From: Stefan Hajnoczi <stefanha@redhat.com>
> Sent: Wednesday, August 30, 2023 2:27 AM
> 
> u64 alignment behaves differently depending on the architecture and so
> <uapi/linux/types.h> offers __aligned_u64 to achieve consistent behavior
> in kernel<->userspace ABIs.
> 
> There are structs in <uapi/linux/vfio.h> that can trivially be updated
> to __aligned_u64 because the struct sizes are multiples of 8 bytes.
> There is no change in memory layout on any CPU architecture and
> therefore this change is safe.
> 
> The commits that follow this one handle the trickier cases where
> explanation about ABI breakage is necessary.
> 
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

[PATCH v2 2/3] vfio: use __aligned_u64 in struct vfio_device_gfx_plane_info

[Stefan Hajnoczi]

The memory layout of struct vfio_device_gfx_plane_info is
architecture-dependent due to a u64 field and a struct size that is not
a multiple of 8 bytes:
- On x86_64 the struct size is padded to a multiple of 8 bytes.
- On x32 the struct size is only a multiple of 4 bytes, not 8.
- Other architectures may vary.

Use __aligned_u64 to make memory layout consistent. This reduces the
chance of 32-bit userspace on a 64-bit kernel breakage.

This patch increases the struct size on x32 but this is safe because of
the struct's argsz field. The kernel may grow the struct as long as it
still supports smaller argsz values from userspace (e.g. applications
compiled against older kernel headers).

Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/uapi/linux/vfio.h        | 3 ++-
 drivers/gpu/drm/i915/gvt/kvmgt.c | 4 +++-
 samples/vfio-mdev/mbochs.c       | 6 ++++--
 samples/vfio-mdev/mdpy.c         | 4 +++-
 4 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 94007ca348ed..777374dd7725 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -816,7 +816,7 @@ struct vfio_device_gfx_plane_info {
 	__u32 drm_plane_type;	/* type of plane: DRM_PLANE_TYPE_* */
 	/* out */
 	__u32 drm_format;	/* drm format of plane */
-	__u64 drm_format_mod;   /* tiled mode */
+	__aligned_u64 drm_format_mod;   /* tiled mode */
 	__u32 width;	/* width of plane */
 	__u32 height;	/* height of plane */
 	__u32 stride;	/* stride of plane */
@@ -829,6 +829,7 @@ struct vfio_device_gfx_plane_info {
 		__u32 region_index;	/* region index */
 		__u32 dmabuf_id;	/* dma-buf id */
 	};
+	__u32 reserved;
 };
 
 #define VFIO_DEVICE_QUERY_GFX_PLANE _IO(VFIO_TYPE, VFIO_BASE + 14)
diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
index 9cd9e9da60dd..813cfef23453 100644
--- a/drivers/gpu/drm/i915/gvt/kvmgt.c
+++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
@@ -1382,7 +1382,7 @@ static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
 		intel_gvt_reset_vgpu(vgpu);
 		return 0;
 	} else if (cmd == VFIO_DEVICE_QUERY_GFX_PLANE) {
-		struct vfio_device_gfx_plane_info dmabuf;
+		struct vfio_device_gfx_plane_info dmabuf = {};
 		int ret = 0;
 
 		minsz = offsetofend(struct vfio_device_gfx_plane_info,
@@ -1392,6 +1392,8 @@ static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
 		if (dmabuf.argsz < minsz)
 			return -EINVAL;
 
+		minsz = min(dmabuf.argsz, sizeof(dmabuf));
+
 		ret = intel_vgpu_query_plane(vgpu, &dmabuf);
 		if (ret != 0)
 			return ret;
diff --git a/samples/vfio-mdev/mbochs.c b/samples/vfio-mdev/mbochs.c
index 3764d1911b51..78aa977ae597 100644
--- a/samples/vfio-mdev/mbochs.c
+++ b/samples/vfio-mdev/mbochs.c
@@ -1262,7 +1262,7 @@ static long mbochs_ioctl(struct vfio_device *vdev, unsigned int cmd,
 
 	case VFIO_DEVICE_QUERY_GFX_PLANE:
 	{
-		struct vfio_device_gfx_plane_info plane;
+		struct vfio_device_gfx_plane_info plane = {};
 
 		minsz = offsetofend(struct vfio_device_gfx_plane_info,
 				    region_index);
@@ -1273,11 +1273,13 @@ static long mbochs_ioctl(struct vfio_device *vdev, unsigned int cmd,
 		if (plane.argsz < minsz)
 			return -EINVAL;
 
+		outsz = min_t(unsigned long, plane.argsz, sizeof(plane));
+
 		ret = mbochs_query_gfx_plane(mdev_state, &plane);
 		if (ret)
 			return ret;
 
-		if (copy_to_user((void __user *)arg, &plane, minsz))
+		if (copy_to_user((void __user *)arg, &plane, outsz))
 			return -EFAULT;
 
 		return 0;
diff --git a/samples/vfio-mdev/mdpy.c b/samples/vfio-mdev/mdpy.c
index 064e1c0a7aa8..f5c2effc1cec 100644
--- a/samples/vfio-mdev/mdpy.c
+++ b/samples/vfio-mdev/mdpy.c
@@ -591,7 +591,7 @@ static long mdpy_ioctl(struct vfio_device *vdev, unsigned int cmd,
 
 	case VFIO_DEVICE_QUERY_GFX_PLANE:
 	{
-		struct vfio_device_gfx_plane_info plane;
+		struct vfio_device_gfx_plane_info plane = {};
 
 		minsz = offsetofend(struct vfio_device_gfx_plane_info,
 				    region_index);
@@ -602,6 +602,8 @@ static long mdpy_ioctl(struct vfio_device *vdev, unsigned int cmd,
 		if (plane.argsz < minsz)
 			return -EINVAL;
 
+		minsz = min_t(unsigned long, plane.argsz, sizeof(plane));
+
 		ret = mdpy_query_gfx_plane(mdev_state, &plane);
 		if (ret)
 			return ret;
-- 
2.41.0

Re: [PATCH v2 2/3] vfio: use __aligned_u64 in struct vfio_device_gfx_plane_info

[Jason Gunthorpe]

On Tue, Aug 29, 2023 at 02:27:19PM -0400, Stefan Hajnoczi wrote:
> The memory layout of struct vfio_device_gfx_plane_info is
> architecture-dependent due to a u64 field and a struct size that is not
> a multiple of 8 bytes:
> - On x86_64 the struct size is padded to a multiple of 8 bytes.
> - On x32 the struct size is only a multiple of 4 bytes, not 8.
> - Other architectures may vary.
> 
> Use __aligned_u64 to make memory layout consistent. This reduces the
> chance of 32-bit userspace on a 64-bit kernel breakage.
> 
> This patch increases the struct size on x32 but this is safe because of
> the struct's argsz field. The kernel may grow the struct as long as it
> still supports smaller argsz values from userspace (e.g. applications
> compiled against older kernel headers).
> 
> Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  include/uapi/linux/vfio.h        | 3 ++-
>  drivers/gpu/drm/i915/gvt/kvmgt.c | 4 +++-
>  samples/vfio-mdev/mbochs.c       | 6 ++++--
>  samples/vfio-mdev/mdpy.c         | 4 +++-
>  4 files changed, 12 insertions(+), 5 deletions(-)

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

Jason

RE: [PATCH v2 2/3] vfio: use __aligned_u64 in struct vfio_device_gfx_plane_info

[Tian, Kevin]

> From: Stefan Hajnoczi <stefanha@redhat.com>
> Sent: Wednesday, August 30, 2023 2:27 AM
> 
> The memory layout of struct vfio_device_gfx_plane_info is
> architecture-dependent due to a u64 field and a struct size that is not
> a multiple of 8 bytes:
> - On x86_64 the struct size is padded to a multiple of 8 bytes.
> - On x32 the struct size is only a multiple of 4 bytes, not 8.
> - Other architectures may vary.
> 
> Use __aligned_u64 to make memory layout consistent. This reduces the
> chance of 32-bit userspace on a 64-bit kernel breakage.
> 
> This patch increases the struct size on x32 but this is safe because of
> the struct's argsz field. The kernel may grow the struct as long as it
> still supports smaller argsz values from userspace (e.g. applications
> compiled against older kernel headers).
> 
> Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH v2 2/3] vfio: use __aligned_u64 in struct vfio_device_gfx_plane_info

[Alex Williamson]

On Tue, 29 Aug 2023 14:27:19 -0400
Stefan Hajnoczi <stefanha@redhat.com> wrote:

> The memory layout of struct vfio_device_gfx_plane_info is
> architecture-dependent due to a u64 field and a struct size that is not
> a multiple of 8 bytes:
> - On x86_64 the struct size is padded to a multiple of 8 bytes.
> - On x32 the struct size is only a multiple of 4 bytes, not 8.
> - Other architectures may vary.
> 
> Use __aligned_u64 to make memory layout consistent. This reduces the
> chance of 32-bit userspace on a 64-bit kernel breakage.
> 
> This patch increases the struct size on x32 but this is safe because of
> the struct's argsz field. The kernel may grow the struct as long as it
> still supports smaller argsz values from userspace (e.g. applications
> compiled against older kernel headers).
> 
> Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  include/uapi/linux/vfio.h        | 3 ++-
>  drivers/gpu/drm/i915/gvt/kvmgt.c | 4 +++-
>  samples/vfio-mdev/mbochs.c       | 6 ++++--
>  samples/vfio-mdev/mdpy.c         | 4 +++-
>  4 files changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index 94007ca348ed..777374dd7725 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -816,7 +816,7 @@ struct vfio_device_gfx_plane_info {
>  	__u32 drm_plane_type;	/* type of plane: DRM_PLANE_TYPE_* */
>  	/* out */
>  	__u32 drm_format;	/* drm format of plane */
> -	__u64 drm_format_mod;   /* tiled mode */
> +	__aligned_u64 drm_format_mod;   /* tiled mode */
>  	__u32 width;	/* width of plane */
>  	__u32 height;	/* height of plane */
>  	__u32 stride;	/* stride of plane */
> @@ -829,6 +829,7 @@ struct vfio_device_gfx_plane_info {
>  		__u32 region_index;	/* region index */
>  		__u32 dmabuf_id;	/* dma-buf id */
>  	};
> +	__u32 reserved;
>  };
>  
>  #define VFIO_DEVICE_QUERY_GFX_PLANE _IO(VFIO_TYPE, VFIO_BASE + 14)
> diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
> index 9cd9e9da60dd..813cfef23453 100644
> --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> @@ -1382,7 +1382,7 @@ static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
>  		intel_gvt_reset_vgpu(vgpu);
>  		return 0;
>  	} else if (cmd == VFIO_DEVICE_QUERY_GFX_PLANE) {
> -		struct vfio_device_gfx_plane_info dmabuf;
> +		struct vfio_device_gfx_plane_info dmabuf = {};
>  		int ret = 0;
>  
>  		minsz = offsetofend(struct vfio_device_gfx_plane_info,
> @@ -1392,6 +1392,8 @@ static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
>  		if (dmabuf.argsz < minsz)
>  			return -EINVAL;
>  
> +		minsz = min(dmabuf.argsz, sizeof(dmabuf));
> +
>  		ret = intel_vgpu_query_plane(vgpu, &dmabuf);
>  		if (ret != 0)
>  			return ret;
> diff --git a/samples/vfio-mdev/mbochs.c b/samples/vfio-mdev/mbochs.c
> index 3764d1911b51..78aa977ae597 100644
> --- a/samples/vfio-mdev/mbochs.c
> +++ b/samples/vfio-mdev/mbochs.c
> @@ -1262,7 +1262,7 @@ static long mbochs_ioctl(struct vfio_device *vdev, unsigned int cmd,
>  
>  	case VFIO_DEVICE_QUERY_GFX_PLANE:
>  	{
> -		struct vfio_device_gfx_plane_info plane;
> +		struct vfio_device_gfx_plane_info plane = {};
>  
>  		minsz = offsetofend(struct vfio_device_gfx_plane_info,
>  				    region_index);
> @@ -1273,11 +1273,13 @@ static long mbochs_ioctl(struct vfio_device *vdev, unsigned int cmd,
>  		if (plane.argsz < minsz)
>  			return -EINVAL;
>  
> +		outsz = min_t(unsigned long, plane.argsz, sizeof(plane));

Sorry, I'm struggling with why these two sample drivers use min_t()
when passed the exact same args as kvmgt above which just uses min().

But more importantly I'm also confused why we need this at all.  The
buffer we're copying to is provided by the user, so what's wrong with
leaving the user provided reserved data?  Are we just trying to return
a zero'd reserved field if argsz allows for it?

Any use of the reserved field other than as undefined data would need
to be associated with a flags bit, so I don't think it's buying us
anything to return it zero'd.  What am I missing?  Thanks,

Alex

> +
>  		ret = mbochs_query_gfx_plane(mdev_state, &plane);
>  		if (ret)
>  			return ret;
>  
> -		if (copy_to_user((void __user *)arg, &plane, minsz))
> +		if (copy_to_user((void __user *)arg, &plane, outsz))
>  			return -EFAULT;
>  
>  		return 0;
> diff --git a/samples/vfio-mdev/mdpy.c b/samples/vfio-mdev/mdpy.c
> index 064e1c0a7aa8..f5c2effc1cec 100644
> --- a/samples/vfio-mdev/mdpy.c
> +++ b/samples/vfio-mdev/mdpy.c
> @@ -591,7 +591,7 @@ static long mdpy_ioctl(struct vfio_device *vdev, unsigned int cmd,
>  
>  	case VFIO_DEVICE_QUERY_GFX_PLANE:
>  	{
> -		struct vfio_device_gfx_plane_info plane;
> +		struct vfio_device_gfx_plane_info plane = {};
>  
>  		minsz = offsetofend(struct vfio_device_gfx_plane_info,
>  				    region_index);
> @@ -602,6 +602,8 @@ static long mdpy_ioctl(struct vfio_device *vdev, unsigned int cmd,
>  		if (plane.argsz < minsz)
>  			return -EINVAL;
>  
> +		minsz = min_t(unsigned long, plane.argsz, sizeof(plane));
> +
>  		ret = mdpy_query_gfx_plane(mdev_state, &plane);
>  		if (ret)
>  			return ret;

Re: [PATCH v2 2/3] vfio: use __aligned_u64 in struct vfio_device_gfx_plane_info

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 5872 bytes --]

On Fri, Sep 15, 2023 at 02:04:58PM -0600, Alex Williamson wrote:
> On Tue, 29 Aug 2023 14:27:19 -0400
> Stefan Hajnoczi <stefanha@redhat.com> wrote:
> 
> > The memory layout of struct vfio_device_gfx_plane_info is
> > architecture-dependent due to a u64 field and a struct size that is not
> > a multiple of 8 bytes:
> > - On x86_64 the struct size is padded to a multiple of 8 bytes.
> > - On x32 the struct size is only a multiple of 4 bytes, not 8.
> > - Other architectures may vary.
> > 
> > Use __aligned_u64 to make memory layout consistent. This reduces the
> > chance of 32-bit userspace on a 64-bit kernel breakage.
> > 
> > This patch increases the struct size on x32 but this is safe because of
> > the struct's argsz field. The kernel may grow the struct as long as it
> > still supports smaller argsz values from userspace (e.g. applications
> > compiled against older kernel headers).
> > 
> > Suggested-by: Jason Gunthorpe <jgg@ziepe.ca>
> > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> > ---
> >  include/uapi/linux/vfio.h        | 3 ++-
> >  drivers/gpu/drm/i915/gvt/kvmgt.c | 4 +++-
> >  samples/vfio-mdev/mbochs.c       | 6 ++++--
> >  samples/vfio-mdev/mdpy.c         | 4 +++-
> >  4 files changed, 12 insertions(+), 5 deletions(-)
> > 
> > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > index 94007ca348ed..777374dd7725 100644
> > --- a/include/uapi/linux/vfio.h
> > +++ b/include/uapi/linux/vfio.h
> > @@ -816,7 +816,7 @@ struct vfio_device_gfx_plane_info {
> >  	__u32 drm_plane_type;	/* type of plane: DRM_PLANE_TYPE_* */
> >  	/* out */
> >  	__u32 drm_format;	/* drm format of plane */
> > -	__u64 drm_format_mod;   /* tiled mode */
> > +	__aligned_u64 drm_format_mod;   /* tiled mode */
> >  	__u32 width;	/* width of plane */
> >  	__u32 height;	/* height of plane */
> >  	__u32 stride;	/* stride of plane */
> > @@ -829,6 +829,7 @@ struct vfio_device_gfx_plane_info {
> >  		__u32 region_index;	/* region index */
> >  		__u32 dmabuf_id;	/* dma-buf id */
> >  	};
> > +	__u32 reserved;
> >  };
> >  
> >  #define VFIO_DEVICE_QUERY_GFX_PLANE _IO(VFIO_TYPE, VFIO_BASE + 14)
> > diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c
> > index 9cd9e9da60dd..813cfef23453 100644
> > --- a/drivers/gpu/drm/i915/gvt/kvmgt.c
> > +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c
> > @@ -1382,7 +1382,7 @@ static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
> >  		intel_gvt_reset_vgpu(vgpu);
> >  		return 0;
> >  	} else if (cmd == VFIO_DEVICE_QUERY_GFX_PLANE) {
> > -		struct vfio_device_gfx_plane_info dmabuf;
> > +		struct vfio_device_gfx_plane_info dmabuf = {};
> >  		int ret = 0;
> >  
> >  		minsz = offsetofend(struct vfio_device_gfx_plane_info,
> > @@ -1392,6 +1392,8 @@ static long intel_vgpu_ioctl(struct vfio_device *vfio_dev, unsigned int cmd,
> >  		if (dmabuf.argsz < minsz)
> >  			return -EINVAL;
> >  
> > +		minsz = min(dmabuf.argsz, sizeof(dmabuf));
> > +
> >  		ret = intel_vgpu_query_plane(vgpu, &dmabuf);
> >  		if (ret != 0)
> >  			return ret;
> > diff --git a/samples/vfio-mdev/mbochs.c b/samples/vfio-mdev/mbochs.c
> > index 3764d1911b51..78aa977ae597 100644
> > --- a/samples/vfio-mdev/mbochs.c
> > +++ b/samples/vfio-mdev/mbochs.c
> > @@ -1262,7 +1262,7 @@ static long mbochs_ioctl(struct vfio_device *vdev, unsigned int cmd,
> >  
> >  	case VFIO_DEVICE_QUERY_GFX_PLANE:
> >  	{
> > -		struct vfio_device_gfx_plane_info plane;
> > +		struct vfio_device_gfx_plane_info plane = {};
> >  
> >  		minsz = offsetofend(struct vfio_device_gfx_plane_info,
> >  				    region_index);
> > @@ -1273,11 +1273,13 @@ static long mbochs_ioctl(struct vfio_device *vdev, unsigned int cmd,
> >  		if (plane.argsz < minsz)
> >  			return -EINVAL;
> >  
> > +		outsz = min_t(unsigned long, plane.argsz, sizeof(plane));
> 
> Sorry, I'm struggling with why these two sample drivers use min_t()
> when passed the exact same args as kvmgt above which just uses min().

min() would work fine here, too.

> But more importantly I'm also confused why we need this at all.  The
> buffer we're copying to is provided by the user, so what's wrong with
> leaving the user provided reserved data?  Are we just trying to return
> a zero'd reserved field if argsz allows for it?
> 
> Any use of the reserved field other than as undefined data would need
> to be associated with a flags bit, so I don't think it's buying us
> anything to return it zero'd.  What am I missing?  Thanks,

I don't remember anymore and what you've described makes sense to me.
I'll remove this in the next revision.

Stefan

> 
> Alex
> 
> > +
> >  		ret = mbochs_query_gfx_plane(mdev_state, &plane);
> >  		if (ret)
> >  			return ret;
> >  
> > -		if (copy_to_user((void __user *)arg, &plane, minsz))
> > +		if (copy_to_user((void __user *)arg, &plane, outsz))
> >  			return -EFAULT;
> >  
> >  		return 0;
> > diff --git a/samples/vfio-mdev/mdpy.c b/samples/vfio-mdev/mdpy.c
> > index 064e1c0a7aa8..f5c2effc1cec 100644
> > --- a/samples/vfio-mdev/mdpy.c
> > +++ b/samples/vfio-mdev/mdpy.c
> > @@ -591,7 +591,7 @@ static long mdpy_ioctl(struct vfio_device *vdev, unsigned int cmd,
> >  
> >  	case VFIO_DEVICE_QUERY_GFX_PLANE:
> >  	{
> > -		struct vfio_device_gfx_plane_info plane;
> > +		struct vfio_device_gfx_plane_info plane = {};
> >  
> >  		minsz = offsetofend(struct vfio_device_gfx_plane_info,
> >  				    region_index);
> > @@ -602,6 +602,8 @@ static long mdpy_ioctl(struct vfio_device *vdev, unsigned int cmd,
> >  		if (plane.argsz < minsz)
> >  			return -EINVAL;
> >  
> > +		minsz = min_t(unsigned long, plane.argsz, sizeof(plane));
> > +
> >  		ret = mdpy_query_gfx_plane(mdev_state, &plane);
> >  		if (ret)
> >  			return ret;
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[PATCH v2 3/3] vfio: use __aligned_u64 in struct vfio_device_ioeventfd

[Stefan Hajnoczi]

The memory layout of struct vfio_device_ioeventfd is
architecture-dependent due to a u64 field and a struct size that is not
a multiple of 8 bytes:
- On x86_64 the struct size is padded to a multiple of 8 bytes.
- On x32 the struct size is only a multiple of 4 bytes, not 8.
- Other architectures may vary.

Use __aligned_u64 to make memory layout consistent. This reduces the
chance that 32-bit userspace on a 64-bit kernel breakage.

This patch increases the struct size on x32 but this is safe because of
the struct's argsz field. The kernel may grow the struct as long as it
still supports smaller argsz values from userspace (e.g. applications
compiled against older kernel headers).

The code that uses struct vfio_device_ioeventfd already works correctly
when the struct size grows, so only the struct definition needs to be
changed.

Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 include/uapi/linux/vfio.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 777374dd7725..032e41b56506 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -864,9 +864,10 @@ struct vfio_device_ioeventfd {
 #define VFIO_DEVICE_IOEVENTFD_32	(1 << 2) /* 4-byte write */
 #define VFIO_DEVICE_IOEVENTFD_64	(1 << 3) /* 8-byte write */
 #define VFIO_DEVICE_IOEVENTFD_SIZE_MASK	(0xf)
-	__u64	offset;			/* device fd offset of write */
-	__u64	data;			/* data to be written */
+	__aligned_u64	offset;		/* device fd offset of write */
+	__aligned_u64	data;		/* data to be written */
 	__s32	fd;			/* -1 for de-assignment */
+	__u32	reserved;
 };
 
 #define VFIO_DEVICE_IOEVENTFD		_IO(VFIO_TYPE, VFIO_BASE + 16)
-- 
2.41.0

RE: [PATCH v2 3/3] vfio: use __aligned_u64 in struct vfio_device_ioeventfd

[Tian, Kevin]

> From: Stefan Hajnoczi <stefanha@redhat.com>
> Sent: Wednesday, August 30, 2023 2:27 AM
> 
> The memory layout of struct vfio_device_ioeventfd is
> architecture-dependent due to a u64 field and a struct size that is not
> a multiple of 8 bytes:
> - On x86_64 the struct size is padded to a multiple of 8 bytes.
> - On x32 the struct size is only a multiple of 4 bytes, not 8.
> - Other architectures may vary.
> 
> Use __aligned_u64 to make memory layout consistent. This reduces the
> chance that 32-bit userspace on a 64-bit kernel breakage.
> 
> This patch increases the struct size on x32 but this is safe because of
> the struct's argsz field. The kernel may grow the struct as long as it
> still supports smaller argsz values from userspace (e.g. applications
> compiled against older kernel headers).
> 
> The code that uses struct vfio_device_ioeventfd already works correctly
> when the struct size grows, so only the struct definition needs to be
> changed.
> 
> Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

RE: [PATCH v2 0/3] vfio: use __aligned_u64 for ioctl structs

[David Laight]

From: Stefan Hajnoczi
> Sent: 29 August 2023 19:27
> 
> v2:
> - Rebased onto https://github.com/awilliam/linux-vfio.git next to get the
>   vfio_iommu_type1_info pad field [Kevin]
> - Fixed min(minsz, sizeof(dmabuf)) -> min(dmabuf.argsz, sizeof(dmabuf)) [Jason, Kevin]

You managed to use min_t() instead of fixing the types to match.

> - Squashed Patch 3 (vfio_iommu_type1_info) into Patch 1 since it is trivial now
>   that the padding field is already there.
> 
> Jason Gunthorpe <jgg@nvidia.com> pointed out that u64 VFIO ioctl struct fields
> have architecture-dependent alignment. iommufd already uses __aligned_u64 to
> avoid this problem.
> 
> See the __aligned_u64 typedef in <uapi/linux/types.h> for details on why it is
> a good idea for kernel<->user interfaces.
> 
> This series modifies the VFIO ioctl structs to use __aligned_u64. Some of the
> changes preserve the existing memory layout on all architectures, so I put them
> together into the first patch. The remaining patches are for structs where
> explanation is necessary about why changing the memory layout does not break
> the uapi.

But you are extending a field in the middle of the uapi structure.
This completely breaks any applications.

You could add code to detect the length of the user-provided
structure and use the correct kernel structure that matches
the length of the user-provided one.
That needs the opposite of __aligned_u64 - a 64bit integer with
32bit alignment on x64-64.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

RE: [PATCH v2 0/3] vfio: use __aligned_u64 for ioctl structs

[David Laight]

> -----Original Message-----
> From: David Laight <David.Laight@ACULAB.COM>
> Sent: 29 August 2023 22:10
> To: 'Stefan Hajnoczi' <stefanha@redhat.com>; kvm@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org; Tian, Kevin <kevin.tian@intel.com>; Alex Williamson
> <alex.williamson@redhat.com>; Jason Gunthorpe <jgg@ziepe.ca>
> Subject: RE: [PATCH v2 0/3] vfio: use __aligned_u64 for ioctl structs
> 
> From: Stefan Hajnoczi
> > Sent: 29 August 2023 19:27
> >
> > v2:
> > - Rebased onto https://github.com/awilliam/linux-vfio.git next to get the
> >   vfio_iommu_type1_info pad field [Kevin]
> > - Fixed min(minsz, sizeof(dmabuf)) -> min(dmabuf.argsz, sizeof(dmabuf)) [Jason, Kevin]
> 
> You managed to use min_t() instead of fixing the types to match.
> 
> > - Squashed Patch 3 (vfio_iommu_type1_info) into Patch 1 since it is trivial now
> >   that the padding field is already there.
> >
> > Jason Gunthorpe <jgg@nvidia.com> pointed out that u64 VFIO ioctl struct fields
> > have architecture-dependent alignment. iommufd already uses __aligned_u64 to
> > avoid this problem.
> >
> > See the __aligned_u64 typedef in <uapi/linux/types.h> for details on why it is
> > a good idea for kernel<->user interfaces.
> >
> > This series modifies the VFIO ioctl structs to use __aligned_u64. Some of the
> > changes preserve the existing memory layout on all architectures, so I put them
> > together into the first patch. The remaining patches are for structs where
> > explanation is necessary about why changing the memory layout does not break
> > the uapi.
> 
> But you are extending a field in the middle of the uapi structure.
> This completely breaks any applications.

I've had a closer look this morning.
Your explanations aren't very good.
The structures all have the 64bit fields on their natural boundary
so the memory layout isn't really changed - just extra padding
at the end.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: [PATCH v2 0/3] vfio: use __aligned_u64 for ioctl structs

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 554 bytes --]

On Tue, Aug 29, 2023 at 09:10:06PM +0000, David Laight wrote:
> From: Stefan Hajnoczi
> > Sent: 29 August 2023 19:27
> > 
> > v2:
> > - Rebased onto https://github.com/awilliam/linux-vfio.git next to get the
> >   vfio_iommu_type1_info pad field [Kevin]
> > - Fixed min(minsz, sizeof(dmabuf)) -> min(dmabuf.argsz, sizeof(dmabuf)) [Jason, Kevin]
> 
> You managed to use min_t() instead of fixing the types to match.

I'm not sure what you're referring to here. Can you explain which line
of code and what you'd like to see changed?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]