From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 202783F167C for ; Wed, 6 May 2026 13:53:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778075618; cv=none; b=SRrbQ1gWVHHT47pkWDeaPD9riHhW0N7Gqy/yAEwQuYJzFtGk5BdxzeEKTUpFNjzNXSsZ4QVfC6wN+AMUjGKP3PNbPGYb3TkkEFSvKfXxNPRbyLaOitxmcu8fyfRtBB051IMS3jWqm6vreYIB4c2sweI8zABuz4T1yiuaePcLjGQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778075618; c=relaxed/simple; bh=Yf3p9DePpxrz1wRV7pVbqVJsbvljiS6iIc3Z+7d2LCk=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=kON/vKG8alSF+TEj6C2sRB11mhAgS7W388r0BS69Ss/5fG7fIsMGBusSTu/otj+RLD3mujuAsb1t/gAk0kK0Y9fwX8DzyxFFhDlNbSPxDQtdOr2SppD/DJHa74qxoz9wmC0OavKKCV1MyKKuRUs5bMm1XIHbVfG3+Yu3tqazUvM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=gW3xmhO/; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="gW3xmhO/" Received: from pps.filterd (m0528005.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 6462B7uG1335254 for ; Wed, 6 May 2026 06:53:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2025-q2; bh=Wr10VTJN3duBxSrKADAl6R/6nUDyzwk22GvJwJKGKOw=; b=gW3xmhO/sRQz XJqsiB30MPqABfSYdPyLP0Gn/OezmfyWpq/Qzqjhz2ze7SQRMEXSQHa99g+b7Iir gFzVLc2FOh5mAKnK8IvSBVns4QA0YXbcRl87u0f1pieyGY90bP6vERpbS+L7g+P9 DHCWf4PcInhjkxO+Gm7vkvPJ6GVW5B0TpH8Jp8xFidA4r6Hlogd/JJy3L8saGMRk 15ap4MCtUXuicSVe2/hcNksNiN8B1EbtQCmsjgszWsFwGMWCoArp5Upp87gHiWGn 78NOQtEVTH4koE76G93+00BqHzW9i4PIfjFdjf47b5lVOIeDazmu1SVOr2EArvRI msxzNEBCAw== Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 4dx2uhvkxk-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Wed, 06 May 2026 06:53:35 -0700 (PDT) Received: by mail-ed1-f72.google.com with SMTP id 4fb4d7f45d1cf-6718d711cecso5351957a12.1 for ; Wed, 06 May 2026 06:53:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778075615; x=1778680415; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Wr10VTJN3duBxSrKADAl6R/6nUDyzwk22GvJwJKGKOw=; b=ZNT18TD896EyK0cvdcP+G3UAWUJszPxMMv3Um2HWxX6R+UOi7TiHBo6Zb4NGPw/kG8 /rpZoBVN36a6vwENdA392qlM4QneQ1SyDMpK3umm6QDOH3aeOC9Y5MRENdjk+sRuWLEa iUc5Z74pjmd6s97ul34Aq6dBByvOxoF/86G7JNND6W0bYAw0AdzcS3Eh/ZdV8NoVkZrU 9tFwGVXwPVmNR1NNuguMXHxJ518ttu1u1JM78TLcwqoJlRdGS7UY9n0fsAnC0Vik5A1s MKNfODv+pUQ1Qmn1Sx7nz+22VQIcarnXHo2E7G6Do4clfIBuzq4HDlVBvwW1NOhUi3Oq r46Q== X-Forwarded-Encrypted: i=1; AFNElJ8lHeyKPJUOK4tQiCHpNwz8b54jqfxdkM7KnPj/pKWizVNyZ5iGnBX31+3KXqMTrTiDnGp9iSHM5w+VQHE=@vger.kernel.org X-Gm-Message-State: AOJu0YyMOql0X6YAqetRMYa2G2GvhLlVm/hqubX6qmxoa/fwiFeoELQW 7dWsY7S9Z5TeQ1J36fXxL8K+9gP/DbsV2bc5QwfDumxzKV7528E89Pl3XhKNhr2EJMrCBXGqsuM 4SBadO/QaU1Eg3c13WLewl8h9xMTNia+RQyopC/TWvxm1iOCTaUjFr1qjAQq9LNW7 X-Gm-Gg: AeBDiesugNNl21PGuH/k1IKbC9wrPiiAf1WUPFv/Wv/nrv5X6OvXE4Ktct9Nk/w9bha SX6Ae1+ubz5k31pmVp6eWipgxXJhvdNcnfYyDLrQUE3FEIeUD1Z8HpNyt2ITgJgg7RaK/mF6Eaz csDCWwl8TQZftV1IC+naugcoUZ9db577BCqC9SfkpnL1T/yKRNefVcWP0z9Ke/n7w2Ex0tX8cEt r37B2vz0rmHMf9q7XTZty8PMI1Uwm9Ohv2u3A+pauSO5UpA7PA/ef0+rGcM6UpeoF6y0nwnWo3X ZY2sM6XaWiEqbgoQvk9mb3vt9GQ/Qq0WTK8FMmrINdVxfqaTXtRYysgAzAXU7/coI8X5vCjf5S0 sThiyiuBafUsjx0Py1yVZzlTULXKwKTERx0M9U26107cINnxXj6JoW5BWOFV2UWBq/pwiXv6oLZ ZOLoK9xOoZmY+lXPDm9VaUV9spt0sWb4COa0SOrs++R3ELc05Mpo9TqVSGGmawZsX3AT0qIhjS4 QLgSf10VwN4qiB4bqrqQQB7xa5EJugOCQ== X-Received: by 2002:a05:6402:3788:b0:674:b1b1:d039 with SMTP id 4fb4d7f45d1cf-67d63db380emr1617980a12.11.1778075614662; Wed, 06 May 2026 06:53:34 -0700 (PDT) X-Received: by 2002:a05:6402:3788:b0:674:b1b1:d039 with SMTP id 4fb4d7f45d1cf-67d63db380emr1617931a12.11.1778075614040; Wed, 06 May 2026 06:53:34 -0700 (PDT) Received: from ?IPV6:2001:8b0:8b6:13d4:102e:f2af:e074:5cde? (e.d.c.5.4.7.0.e.f.a.2.f.e.2.0.1.4.d.3.1.6.b.8.0.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:8b6:13d4:102e:f2af:e074:5cde]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-67cd904fe68sm1337174a12.0.2026.05.06.06.53.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 May 2026 06:53:33 -0700 (PDT) Message-ID: <9304aada-ee84-4cf2-a1d7-82313eda07aa@meta.com> Date: Wed, 6 May 2026 14:53:31 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/9] vfio/pci: Fix vfio_pci_dma_buf_cleanup() double-put Content-Language: en-GB To: Alex Williamson Cc: Leon Romanovsky , Jason Gunthorpe , Alex Mastro , =?UTF-8?Q?Christian_K=C3=B6nig?= , Mahmoud Adam , David Matlack , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , Sumit Semwal , Kevin Tian , Ankit Agrawal , Pranjal Shrivastava , Alistair Popple , Vivek Kasireddy , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, kvm@vger.kernel.org, =?UTF-8?Q?Carlos_L=C3=B3pez?= References: <20260416131815.2729131-1-mattev@meta.com> <20260416131815.2729131-2-mattev@meta.com> <20260501131236.278ac431@shazbot.org> From: Matt Evans In-Reply-To: <20260501131236.278ac431@shazbot.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Proofpoint-ORIG-GUID: KYhGtvgYGIYpAaGp-BhcPldVQJ43KS-9 X-Proofpoint-GUID: KYhGtvgYGIYpAaGp-BhcPldVQJ43KS-9 X-Authority-Analysis: v=2.4 cv=DtFmPm/+ c=1 sm=1 tr=0 ts=69fb47df cx=c_pps a=DTy5UCxgudPLKrSn28m7Kw==:117 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=7x6HtfJdh03M6CCDgxCd:22 a=jCddH8ec0KUNCymVuxII:22 a=VwQbUJbxAAAA:8 a=UqCG9HQmAAAA:8 a=Ikd4Dj_1AAAA:8 a=VabnemYjAAAA:8 a=CQcdbiei6IkXp1Z0tZsA:9 a=QEXdDO2ut3YA:10 a=VzYV69SsQLkmnS9OQLw-:22 a=gKebqoRLp9LExxC7YDUY:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTA2MDEzNiBTYWx0ZWRfX1n3HHO452SJU pxptyBHs9nCvEz3g6cTLfs6B0RXMaUyED9i8Xguipumj/twr9A9JSwZN2ZIXlq1PtYfIj3HjLFh q0sYLQTJSQSsiev7cSu40jILNfg6XWywfkdUECozdSv+O1Xf7GC2vnuHivoi9AWoMWN+vmPyVHy y/wkkHmb+TyUqeZ60dEm35qIb+Os1Ndg8UP6HhwbQAl7h6SQ1XKxjjFd8Cvkic1xV77NgWZzy7J vGUXN4uk53EZkDfZg65QYVWw0EOXNGyqGl1vRYNOrccrkW2i+F3c5GNdwx74QZWaIXKfsHoaOw0 /+R5CtoXh6+kcjS3BCBIOBQVonzWmvpRvzvPzmypbIZ8rT2MqIAYRXBeDGOiLlN86tyOmpuTwJQ GFXrs1sQIWHM7PZUSVlsTrV226zUOCvpYu6i8hdekp45acDFTq7+kx7QqHxe6eywDyEkq19nqRI fm+Jd6wjYj3xvgLNNqA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-05_03,2026-05-06_01,2025-10-01_01 Hi Alex, On 01/05/2026 20:12, Alex Williamson wrote: > > On Thu, 16 Apr 2026 06:17:44 -0700 > Matt Evans wrote: > >> vfio_pci_dma_buf_cleanup() assumed all VFIO device DMABUFs need to be >> revoked. However, if vfio_pci_dma_buf_move() revokes DMABUFs before >> the fd/device closes, then vfio_pci_dma_buf_cleanup() would do a >> second/underflowing kref_put() then wait_for_completion() on a >> completion that never fires. Fixed by predicating on revocation >> status. >> >> This could happen if PCI_COMMAND_MEMORY is cleared before closing the >> device fd (but the scenario is more likely to hit when future commits >> add more methods to revoke DMABUFs). >> >> Fixes: 1a8a5227f2299 ("vfio: Wait for dma-buf invalidation to complete") >> Signed-off-by: Matt Evans >> --- >> >> (Just a fix, but later "vfio/pci: Convert BAR mmap() to use a DMABUF" >> and "vfio/pci: Permanently revoke a DMABUF on request" depend on this >> context, so including in this series.) > > We really need a fix for this split out from this series, It's already > been shown[1] that this is trivially reachable. Carlos proposed[2] a > similar solution to the one below. I was concurrently working on the > issued and suggested an alternative[3]. Let's pick a solution for > 7.1-rc. Thanks, It looks like [3] is progressing, so I'll drop this one when I can rebase onto it. I noticed [3] removes the dma_resv_lock(priv->dmabuf->resv) around the priv->vdev = NULL, and this series' vfio_pci_mmap_huge_fault() relies on vdev only changing whilst resv is held to resolve a race between a fault and cleanup (see patch 7 of this series). The handler takes resv so that it can stably test vdev in order to take memory_lock. Must your fix change vdev outside of holding resv? I'm still sketching alternatives; at first glance perhaps the fault handler could rely on vdev being valid if !revoked, which can be tested holding [only] resv. Thanks, Matt > > Alex > > [1]https://lore.kernel.org/all/GVXPR02MB12019AA6014F27EF5D773E89BFB372@GVXPR02MB12019.eurprd02.prod.outlook.com/ > [2]https://lore.kernel.org/all/20260429182736.409323-2-clopez@suse.de/ > [3]https://lore.kernel.org/all/20260429142242.70f746b4@nvidia.com/ > > >> drivers/vfio/pci/vfio_pci_dmabuf.c | 9 +++++++-- >> 1 file changed, 7 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c >> index 281ba7d69567..04478b7415a0 100644 >> --- a/drivers/vfio/pci/vfio_pci_dmabuf.c >> +++ b/drivers/vfio/pci/vfio_pci_dmabuf.c >> @@ -395,20 +395,25 @@ void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev) >> >> down_write(&vdev->memory_lock); >> list_for_each_entry_safe(priv, tmp, &vdev->dmabufs, dmabufs_elm) { >> + bool was_revoked; >> + >> if (!get_file_active(&priv->dmabuf->file)) >> continue; >> >> dma_resv_lock(priv->dmabuf->resv, NULL); >> list_del_init(&priv->dmabufs_elm); >> priv->vdev = NULL; >> + was_revoked = priv->revoked; >> priv->revoked = true; >> dma_buf_invalidate_mappings(priv->dmabuf); >> dma_resv_wait_timeout(priv->dmabuf->resv, >> DMA_RESV_USAGE_BOOKKEEP, false, >> MAX_SCHEDULE_TIMEOUT); >> dma_resv_unlock(priv->dmabuf->resv); >> - kref_put(&priv->kref, vfio_pci_dma_buf_done); >> - wait_for_completion(&priv->comp); >> + if (!was_revoked) { >> + kref_put(&priv->kref, vfio_pci_dma_buf_done); >> + wait_for_completion(&priv->comp); >> + } >> vfio_device_put_registration(&vdev->vdev); >> fput(priv->dmabuf->file); >> } >