stable.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Michael S. Tsirkin" <mst@redhat.com>
To: Vladimir Petrzhikovskii <vladimir@dfb.sh>
Cc: "Dragos Tatulea" <dtatulea@nvidia.com>,
	"Jason Wang" <jasowangio@gmail.com>,
	"Eugenio Pérez" <eperezma@redhat.com>,
	"Xuan Zhuo" <xuanzhuo@linux.alibaba.com>,
	virtualization@lists.linux.dev, linux-kernel@vger.kernel.org,
	stable@vger.kernel.org, "Nikolai Kalinin" <nikolai@kalinidze.me>
Subject: Re: [PATCH] vdpa/mlx5: Bound the KLM count of the user MR
Date: Thu, 19 Nov 2026 10:15:32 -0500	[thread overview]
Message-ID: <20261119101523-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <20261109-b4-mlx5-klm-v1-1-787b45145719@dfb.sh>

On Mon, Nov 09, 2026 at 08:21:28PM +0100, Vladimir Petrzhikovskii wrote:
> create_user_mr() builds one indirect MKey that covers the whole vhost
> IOTLB. It fills each gap between two mappings with null KLMs, one per
> GiB, and never checks the total number of KLMs against the device
> limit, MLX5_CAP_GEN(log_max_klm_list_size).
> 
> With a virtio-IOMMU the guest chooses the IOVAs: the VMM (QEMU)
> forwards each map the guest makes to the vhost-vDPA device. If the
> guest maps one page at IOVA 0 and one at IOVA 2^58, the driver needs
> more than 2^28 KLMs. create_indirect_key() computes the buffer size in
> an int, which overflows: the driver allocates 336 bytes, and
> fill_indir() then writes 16 bytes per KLM, 4 GiB in total, far past
> the end of the buffer.
> 
> nnuls is an int, so a gap of 2^61 bytes or more is miscounted on every
> architecture, and on 32-bit the unsigned long num_klms wraps. The gap
> is rounded up to whole GiB as (size + 2^30 - 1) >> 30. For a gap
> larger than 2^64 - 2^30, for example below a page mapped in the top
> GiB, that addition wraps and the driver counts no null KLMs for the
> gap, while fill_indir() still writes one for every GiB of it.
> 
> Count KLMs in u64 and round the gap up without the overflowing
> addition. Before creating any direct key, return -E2BIG with a
> rate-limited warning if the MR needs more KLMs than the device limit,
> or more than fit in a CREATE_MKEY command whose length is an int.
> Allocate the command with kvzalloc(), because a list within the device
> limit can be too large for kzalloc(). A normal guest is unaffected:
> memory at 0-3 GiB and 4-68 GiB needs 68 KLMs.
> 
> Found by an LLM code review agent auditing vhost-vDPA backends. We have
> no mlx5 hardware, so the fix is untested on it; it builds with W=1 on
> x86_64 and i386.
> 
> Fixes: 94abbccdf291 ("vdpa/mlx5: Add shared memory registration code")
> Cc: stable@vger.kernel.org
> Co-developed-by: Nikolai Kalinin <nikolai@kalinidze.me>
> Signed-off-by: Nikolai Kalinin <nikolai@kalinidze.me>
> Signed-off-by: Vladimir Petrzhikovskii <vladimir@dfb.sh>
> ---
> Dragos, one question on the limit: the patch takes
> MLX5_CAP_GEN(log_max_klm_list_size) as the largest KLM list an indirect
> CREATE_MKEY accepts, as mlx5_ib does for fast-reg lists. If firmware
> applies a different limit to CREATE_MKEY, please say so and I will
> change the bound.
> 
> The guest kernel change used for the reachability check is available
> on request.


Dragos, care to answer?

> ---
>  drivers/vdpa/mlx5/core/mlx5_vdpa.h |  2 +-
>  drivers/vdpa/mlx5/core/mr.c        | 30 +++++++++++++++++++++++++-----
>  2 files changed, 26 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h
> index 42f2f44b383c..1d84824ca429 100644
> --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h
> +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h
> @@ -32,7 +32,7 @@ struct mlx5_vdpa_mr {
>  	/* list of direct MRs descendants of this indirect mr */
>  	struct list_head head;
>  	unsigned long num_directs;
> -	unsigned long num_klms;
> +	u64 num_klms;
>  
>  	struct vhost_iotlb *iotlb;
>  
> diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c
> index b0c5ff23d022..03db8345c1bb 100644
> --- a/drivers/vdpa/mlx5/core/mr.c
> +++ b/drivers/vdpa/mlx5/core/mr.c
> @@ -206,6 +206,16 @@ static int klm_byte_size(int nklms)
>  	return 16 * ALIGN(nklms, 4);
>  }
>  
> +/* Largest KLM count whose CREATE_MKEY input length still fits in an int. */
> +#define MLX5_VDPA_MAX_KLMS_INLEN \
> +	ALIGN_DOWN((INT_MAX - MLX5_ST_SZ_BYTES(create_mkey_in)) / 16, 4)
> +
> +static u64 max_klms(struct mlx5_vdpa_dev *mvdev)
> +{
> +	return min_t(u64, BIT_ULL(MLX5_CAP_GEN(mvdev->mdev, log_max_klm_list_size)),
> +		     MLX5_VDPA_MAX_KLMS_INLEN);
> +}
> +
>  #define MLX5_VDPA_MTT_ALIGN 16
>  
>  static int create_direct_keys(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr)
> @@ -338,7 +348,7 @@ static int create_indirect_key(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr
>  		return -EINVAL;
>  
>  	inlen = MLX5_ST_SZ_BYTES(create_mkey_in) + klm_byte_size(mr->num_klms);
> -	in = kzalloc(inlen, GFP_KERNEL);
> +	in = kvzalloc(inlen, GFP_KERNEL);
>  	if (!in)
>  		return -ENOMEM;
>  
> @@ -355,7 +365,7 @@ static int create_indirect_key(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr
>  	MLX5_SET(create_mkey_in, in, translations_octword_actual_size, mr->num_klms);
>  	fill_indir(mvdev, mr, in);
>  	err = mlx5_vdpa_create_mkey(mvdev, &mr->mkey, in, inlen);
> -	kfree(in);
> +	kvfree(in);
>  	return err;
>  }
>  
> @@ -501,6 +511,7 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev,
>  {
>  	struct mlx5_vdpa_direct_mr *dmr;
>  	struct mlx5_vdpa_direct_mr *n;
> +	u64 klm_limit = max_klms(mvdev);
>  	struct vhost_iotlb_map *map;
>  	u32 pperm = U16_MAX;
>  	u64 last = U64_MAX;
> @@ -508,7 +519,7 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev,
>  	u64 pe = U64_MAX;
>  	u64 start = 0;
>  	int err = 0;
> -	int nnuls;
> +	u64 nnuls;
>  
>  	INIT_LIST_HEAD(&mr->head);
>  	for (map = vhost_iotlb_itree_first(iotlb, start, last); map;
> @@ -521,9 +532,10 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev,
>  				if (pe < map->start) {
>  					/* We have a hole in the map. Check how
>  					 * many null keys are required to fill it.
> +					 * The hole can be nearly 2^64 bytes, so
> +					 * round up without adding to its size.
>  					 */
> -					nnuls = MLX5_DIV_ROUND_UP_POW2(map->start - pe,
> -								       LOG_MAX_KLM_SIZE);
> +					nnuls = ((map->start - pe - 1) >> LOG_MAX_KLM_SIZE) + 1;
>  					mr->num_klms += nnuls;
>  				}
>  				err = add_direct_chain(mvdev, mr, ps, pe - ps, pperm, iotlb);
> @@ -539,6 +551,14 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev,
>  	if (err)
>  		goto err_chain;
>  
> +	if (mr->num_klms > klm_limit) {
> +		dev_warn_ratelimited(mvdev->mdev->device,
> +				     "map needs %llu KLMs, limit is %llu\n",
> +				     mr->num_klms, klm_limit);
> +		err = -E2BIG;
> +		goto err_chain;
> +	}
> +
>  	err = create_direct_keys(mvdev, mr);
>  	if (err)
>  		goto err_chain;
> 
> ---
> base-commit: d48723711bae9b8cdc56719c448ba42fef28cbee
> change-id: 20261109-b4-mlx5-klm-798a398a159e
> 
> Best regards,
> --  
> Vladimir Petrzhikovskii <vladimir@dfb.sh>
> 


      reply	other threads:[~2026-11-19 15:15 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-09 19:21 [PATCH] vdpa/mlx5: Bound the KLM count of the user MR Vladimir Petrzhikovskii
2026-11-19 15:15 ` Michael S. Tsirkin [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119101523-mutt-send-email-mst@kernel.org \
    --to=mst@redhat.com \
    --cc=dtatulea@nvidia.com \
    --cc=eperezma@redhat.com \
    --cc=jasowangio@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nikolai@kalinidze.me \
    --cc=stable@vger.kernel.org \
    --cc=virtualization@lists.linux.dev \
    --cc=vladimir@dfb.sh \
    --cc=xuanzhuo@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).