From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1795101342; b=kZLj+LwFhoYKL4dKRYtJdYsUM5FqRTiGxYY54mmQjNkIfNxG/70Ixhyf0Ox7Rkqu9Ycv ZDH1NHC+3TF481TCQBDS4NAs4PilsQMCdds3iCzZ2UOc3ESuQRwFl2m+AU6XyFyineyon QENHl0YShA58o147wO4EwOVT5Y1tdm9RS2Ntqg37UO/6s7ZmMC2sh6oeAHQNwkPpC3S6O XU+k2g2XtW/HXaVt70q+oaTV1AcK2dQkniS/q//cm9T1Apc8YlguLKarocODNGO0s7ljf OOv7rPtq3VtTds+C5nfpTTOn46E75Y7NEA9d6TvLBCTzWf9WTT7KbSDl1mrdbrDE5cg== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1795101342; h=DMARC-Filter:DKIM-Signature:Received:X-MC-Unique: X-Mimecast-MFC-AGG-ID:Received:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Gm-Gg:X-Received: X-Received:Received:Date:From:To:Cc:Subject:Message-ID:References: MIME-Version:In-Reply-To:X-Mimecast-Spam-Score:X-Mimecast-MFC-PROC-ID: X-Mimecast-Originator:Content-Type:Content-Disposition; bh=bpdlf5Cz0o8Ft5faD611TCWq5HE674yS0Nmt0tA94GI=; b=BINV0h9a1JtjjWRNoCPZ6WdA2T6GanbeRhD/dk85vxBAWqtRkOWQnAmMFlxWEbbBwJuR 5RbNykfCbqD+XB279fzNoZhAiAm/JthZHQew07cSbluvNYujlP8kbeLOZUSgF9GGWxw1i 9INmjANdREuXErFjxdStTprgrsZkGHQZH0XaqKNrBTuAllrpTsGCia82GMSExDtQQXd1j 3rjsg6BNBBBtFfIyJj/AzvA1J1sbYj9MEN6A0GgkECkFEPM2791jFpaK89P08pjvqdvlm liESGC9/ab9Smoylu09zXqNDvJREX+uZ+jYmX5VoqLsGTqhvnQs7N3EweCq6M8EhDgw== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VG6ON5y0; arc=none smtp.remote-ip=170.10.133.124 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AB05606A325 for ; Thu, 19 Nov 2026 15:15:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 2AB05606A325 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VG6ON5y0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1795101341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bpdlf5Cz0o8Ft5faD611TCWq5HE674yS0Nmt0tA94GI=; b=VG6ON5y0TbfYPw+tC0X+p14S/R+an3NCRQ7cfPVcFh9rdbfYUaQAH72OkW83pGvy+RJ3Iv jWEdl7z1Tqa3vIWMe0QL9Qdv6Bt/1p7nvXjXdhlAIrRsg9sAs961z29MfAKFgl9TgWPmN5 6c2ohI/B/ZL7JTWVWQr3RdoDFNFtKYE= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-222-5Dxcp0NHPSyV4BJeqpcUfw-1; Thu, 19 Nov 2026 10:15:38 -0500 X-MC-Unique: 5Dxcp0NHPSyV4BJeqpcUfw-1 X-Mimecast-MFC-AGG-ID: 5Dxcp0NHPSyV4BJeqpcUfw_1795101337 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4bb6bac1e84so16640795e9.0 for ; Thu, 19 Nov 2026 07:15:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795101337; x=1795706137; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=bpdlf5Cz0o8Ft5faD611TCWq5HE674yS0Nmt0tA94GI=; b=oHs5qrMf2QLcA9xmWErF87KscK0bz1lGv264O5aT04fKGupoop8Qwh7ah1BkjcFd9+ W4+yvJviHr5+aubi4GYzpnQbpa5O7iTxKXNrRIGk26piDUNWWSyYHVUwif+KNvASJJKE 90ZL1Kg35yD3BOEXSCUBjf4lrJhGGkyy3RLnNUV18j+NYaQpaRang+ZjgwwejmbIjRGp 4qETEgR56kGsFSOTrl16nVA3vPjQrXypDNKanwbKbGf3JoU1mx7Ecaggct1xZtp1V1Rt /AO7KWtYy3Y1gqIEsXFzRMxX/PRTOmpzQID9DrGamxtl0vDYtYNq9KTh2o3B/1pc8tPF Dkow== X-Forwarded-Encrypted: i=1; AL+hErFJZHgYIid9OGNdguRKzHeuZU9ld2qVT+4O45FG/05F+gyqmYa+QVqyc37pBIJPqzRsxAqQENSLuDAkgO+DpA==@lists.linux.dev X-Gm-Message-State: APuU4Z8W7mMLJJ/n9ogRR5R7qk6GrozieTZnW0J2f3MMKHYqK9oNMx7p yE81IdorI/uC0YnvsDeJ2T9CgKDMpmp7AoBDcwCH65svgD/4dlRRD7oe0W36RA/9pQF3ixI0Ccf W/+W15y6K2VqaVfMrS8PewXvTAasxQXNzlnWYMvRbf3nXw7VLyZ2qNViZHPR8HP+zTydI X-Gm-Gg: AYBFou14EIkO2aqUu5VMp81/DitU8NQHLiPwbc1C7xqe/dzs412MOiPSO3ReYeO9GMu vCqDz7zeBWs4IxiLK+f1zzc80jESvPyTADRDmth9clXpl+wu2HUcSk2Xkx2R0yDgSPGDSDovdZo /IoRZNE1/hqq1U+JxHBi5/pJpCqhyqWXvv2t1dI2Y6kuICuNXDTTKqR6fluIjKI/vNnhJ1BQ0fu uV8QE7zr47jCtKaeXBzgvkiJEAd1oyy23rCJtd6dE9S1mST1buzIto9K+umALWLto47MGMgVcUt XHU0UjqlJlBZb26XiwDXrMOSxAGQdrID76H7bq8zPXeoTwyrShGx+cLp9pHtlN65cPUEPKTrlUq ctSwnnXV9vnHKbIj+lTrjU/XNRa0FuUQtcjZe8rG7cQ== X-Received: by 2002:a05:600c:1c11:b0:4a1:e6a9:4086 with SMTP id 5b1f17b1804b1-4bb7d57a536mr140198015e9.0.1795101337274; Thu, 19 Nov 2026 07:15:37 -0800 (PST) X-Received: by 2002:a05:600c:1c11:b0:4a1:e6a9:4086 with SMTP id 5b1f17b1804b1-4bb7d57a536mr140197095e9.0.1795101336658; Thu, 19 Nov 2026 07:15:36 -0800 (PST) Received: from redhat.com ([2a0d:6fc0:1522:6500:ac93:c62:f076:48c6]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4bb884911e3sm4180165e9.6.2026.11.19.07.15.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2026 07:15:35 -0800 (PST) Date: Thu, 19 Nov 2026 10:15:32 -0500 From: "Michael S. Tsirkin" To: Vladimir Petrzhikovskii Cc: Dragos Tatulea , Jason Wang , Eugenio =?iso-8859-1?Q?P=E9rez?= , Xuan Zhuo , virtualization@lists.linux.dev, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Nikolai Kalinin Subject: Re: [PATCH] vdpa/mlx5: Bound the KLM count of the user MR Message-ID: <20261119101523-mutt-send-email-mst@kernel.org> References: <20261109-b4-mlx5-klm-v1-1-787b45145719@dfb.sh> X-Mailing-List: virtualization@lists.linux.dev List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 In-Reply-To: <20261109-b4-mlx5-klm-v1-1-787b45145719@dfb.sh> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 4VMWFi4KDIo5o_xOVdqD3JIgMbQdjok16XUTAb2xekw_1795101337 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Nov 09, 2026 at 08:21:28PM +0100, Vladimir Petrzhikovskii wrote: > create_user_mr() builds one indirect MKey that covers the whole vhost > IOTLB. It fills each gap between two mappings with null KLMs, one per > GiB, and never checks the total number of KLMs against the device > limit, MLX5_CAP_GEN(log_max_klm_list_size). > > With a virtio-IOMMU the guest chooses the IOVAs: the VMM (QEMU) > forwards each map the guest makes to the vhost-vDPA device. If the > guest maps one page at IOVA 0 and one at IOVA 2^58, the driver needs > more than 2^28 KLMs. create_indirect_key() computes the buffer size in > an int, which overflows: the driver allocates 336 bytes, and > fill_indir() then writes 16 bytes per KLM, 4 GiB in total, far past > the end of the buffer. > > nnuls is an int, so a gap of 2^61 bytes or more is miscounted on every > architecture, and on 32-bit the unsigned long num_klms wraps. The gap > is rounded up to whole GiB as (size + 2^30 - 1) >> 30. For a gap > larger than 2^64 - 2^30, for example below a page mapped in the top > GiB, that addition wraps and the driver counts no null KLMs for the > gap, while fill_indir() still writes one for every GiB of it. > > Count KLMs in u64 and round the gap up without the overflowing > addition. Before creating any direct key, return -E2BIG with a > rate-limited warning if the MR needs more KLMs than the device limit, > or more than fit in a CREATE_MKEY command whose length is an int. > Allocate the command with kvzalloc(), because a list within the device > limit can be too large for kzalloc(). A normal guest is unaffected: > memory at 0-3 GiB and 4-68 GiB needs 68 KLMs. > > Found by an LLM code review agent auditing vhost-vDPA backends. We have > no mlx5 hardware, so the fix is untested on it; it builds with W=1 on > x86_64 and i386. > > Fixes: 94abbccdf291 ("vdpa/mlx5: Add shared memory registration code") > Cc: stable@vger.kernel.org > Co-developed-by: Nikolai Kalinin > Signed-off-by: Nikolai Kalinin > Signed-off-by: Vladimir Petrzhikovskii > --- > Dragos, one question on the limit: the patch takes > MLX5_CAP_GEN(log_max_klm_list_size) as the largest KLM list an indirect > CREATE_MKEY accepts, as mlx5_ib does for fast-reg lists. If firmware > applies a different limit to CREATE_MKEY, please say so and I will > change the bound. > > The guest kernel change used for the reachability check is available > on request. Dragos, care to answer? > --- > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 2 +- > drivers/vdpa/mlx5/core/mr.c | 30 +++++++++++++++++++++++++----- > 2 files changed, 26 insertions(+), 6 deletions(-) > > diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h > index 42f2f44b383c..1d84824ca429 100644 > --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h > +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h > @@ -32,7 +32,7 @@ struct mlx5_vdpa_mr { > /* list of direct MRs descendants of this indirect mr */ > struct list_head head; > unsigned long num_directs; > - unsigned long num_klms; > + u64 num_klms; > > struct vhost_iotlb *iotlb; > > diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c > index b0c5ff23d022..03db8345c1bb 100644 > --- a/drivers/vdpa/mlx5/core/mr.c > +++ b/drivers/vdpa/mlx5/core/mr.c > @@ -206,6 +206,16 @@ static int klm_byte_size(int nklms) > return 16 * ALIGN(nklms, 4); > } > > +/* Largest KLM count whose CREATE_MKEY input length still fits in an int. */ > +#define MLX5_VDPA_MAX_KLMS_INLEN \ > + ALIGN_DOWN((INT_MAX - MLX5_ST_SZ_BYTES(create_mkey_in)) / 16, 4) > + > +static u64 max_klms(struct mlx5_vdpa_dev *mvdev) > +{ > + return min_t(u64, BIT_ULL(MLX5_CAP_GEN(mvdev->mdev, log_max_klm_list_size)), > + MLX5_VDPA_MAX_KLMS_INLEN); > +} > + > #define MLX5_VDPA_MTT_ALIGN 16 > > static int create_direct_keys(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr *mr) > @@ -338,7 +348,7 @@ static int create_indirect_key(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr > return -EINVAL; > > inlen = MLX5_ST_SZ_BYTES(create_mkey_in) + klm_byte_size(mr->num_klms); > - in = kzalloc(inlen, GFP_KERNEL); > + in = kvzalloc(inlen, GFP_KERNEL); > if (!in) > return -ENOMEM; > > @@ -355,7 +365,7 @@ static int create_indirect_key(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr > MLX5_SET(create_mkey_in, in, translations_octword_actual_size, mr->num_klms); > fill_indir(mvdev, mr, in); > err = mlx5_vdpa_create_mkey(mvdev, &mr->mkey, in, inlen); > - kfree(in); > + kvfree(in); > return err; > } > > @@ -501,6 +511,7 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev, > { > struct mlx5_vdpa_direct_mr *dmr; > struct mlx5_vdpa_direct_mr *n; > + u64 klm_limit = max_klms(mvdev); > struct vhost_iotlb_map *map; > u32 pperm = U16_MAX; > u64 last = U64_MAX; > @@ -508,7 +519,7 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev, > u64 pe = U64_MAX; > u64 start = 0; > int err = 0; > - int nnuls; > + u64 nnuls; > > INIT_LIST_HEAD(&mr->head); > for (map = vhost_iotlb_itree_first(iotlb, start, last); map; > @@ -521,9 +532,10 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev, > if (pe < map->start) { > /* We have a hole in the map. Check how > * many null keys are required to fill it. > + * The hole can be nearly 2^64 bytes, so > + * round up without adding to its size. > */ > - nnuls = MLX5_DIV_ROUND_UP_POW2(map->start - pe, > - LOG_MAX_KLM_SIZE); > + nnuls = ((map->start - pe - 1) >> LOG_MAX_KLM_SIZE) + 1; > mr->num_klms += nnuls; > } > err = add_direct_chain(mvdev, mr, ps, pe - ps, pperm, iotlb); > @@ -539,6 +551,14 @@ static int create_user_mr(struct mlx5_vdpa_dev *mvdev, > if (err) > goto err_chain; > > + if (mr->num_klms > klm_limit) { > + dev_warn_ratelimited(mvdev->mdev->device, > + "map needs %llu KLMs, limit is %llu\n", > + mr->num_klms, klm_limit); > + err = -E2BIG; > + goto err_chain; > + } > + > err = create_direct_keys(mvdev, mr); > if (err) > goto err_chain; > > --- > base-commit: d48723711bae9b8cdc56719c448ba42fef28cbee > change-id: 20261109-b4-mlx5-klm-798a398a159e > > Best regards, > -- > Vladimir Petrzhikovskii >