From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26A7342E8C0 for ; Mon, 10 Aug 2026 17:59:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786384788; cv=none; b=mqJ0+WU+dKY/kXGEf1OIAgrH5IGPZ9nIY+Wy51kFqEXUIHRgANjnX34l81AwlKXaSwA8e+sCddTuw9zHaphhyIZoImh8rtD3wRo6cFn5SM/SHSL7IQpYaJh2CANEBrLRVgtnC4mYgIHfAMlto2S1nVH/JHCWuk2r7Hpf3fqdDxc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786384788; c=relaxed/simple; bh=jshTh5JB+/7VGuIDYbUMdVTS0P25FSA6MRRrjP4R9Uw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=o3E6CsFxEsSKnCBfb8CUMP3TeddZUoWO7t1ddtxihct5SIO0HKepS5tCW/fH2kLt8VLejzUAyRwUoTkUkTV+EBhBQTD0IlLFoDKMjDcQpWNUbo3MSa6nK781Gl1zWx/6RQ+EIjTU/JKXs+sJn0n3ShCpKtcgLj/wahNVNA0D3ak= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UQlIx58g; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UQlIx58g" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2ccdf36f63dso12075ad.0 for ; Mon, 10 Aug 2026 10:59:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786384786; x=1786989586; darn=vger.kernel.org; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to:content-type; bh=lpTAXT6Y1ZP0dqnAfIHtJoG8pQyeKSS7NFanb0wNrm0=; b=UQlIx58gaSYAZ6hz6UprFzP9YjWMCGjh7/dWUqcvIJfoLijPtmOv2/S/vpeZOGomvl X17VUiwy3KP1G/kKPUCTl5PUGuT4W14hWbkAd+pgBn7U9qdOMa9io1a3ieM3TY4wpfNG +mv9JlmoKs+CQ+93YI+FZPPoCoWeBjjfoLOCzgwANyCwxKmeXOknefGnT+fjoVmcozxw D6GrOPF+BQAYZ+aaKXH5J9xDM/PumeEKh6TeSVEX20MFWgH0+YM+HwdXahxZqOJSmX3K fV8HHkixICnLP+T+1OstSyK3hO+7Oc/TZXa57sv0+O7Qh4BaYvnf+fSTfqvGlD2/qnz2 +/8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786384786; x=1786989586; h=in-reply-to:content-disposition:content-type:mime-version :references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=lpTAXT6Y1ZP0dqnAfIHtJoG8pQyeKSS7NFanb0wNrm0=; b=Ms6tD78VVp3phQGkfbGxVsO2HcYi9wAwkmZg0bS2ouCIoG2Fb7Gil83nDG/qf1UKAJ poinuhkLNpYXIPXaEpFlo4ow2Qsf7fUWvrVDv59l4swwNmL89AWCQEzPJkuJ27jIzfh2 PdNYGT/Z1re1OQhPAiglqXEVkRhU61X3I5dSJ37tCxYMwpbvROmUlEmTSNkbECvXqYbI 9fSK8Ue4XfuI0Uh4DU9TrjKOdK3QYRhZx9HQweRa5sfR3t7Zj0cKpC/htXzXAGM9Xdeo YgHXQaGClHAzJD3OOQHNugCMSQgCTu1JyOQEDoFCiUO8FhANnj9dBtQrpNGxvnJaoCnj Lnyw== X-Forwarded-Encrypted: i=1; AHgh+RpyUZpfHJGITmGcecm7I4OR04shJlDW7IW3Yt89PaOHJPiyIJaU5ShihUCuNA1asodGv4ms0r0B7pAIyuI=@vger.kernel.org X-Gm-Message-State: AOJu0YxmfFCiXvWEpHuPz6rRP79aeQ1DKdMoH/zvLu5uezuUC8I0vAao Nv+yChnacOFqubOVrRXmifJ6z/jUahnZtYU7zXL5Ne8zezDEksiubTC258+uSKgyNw== X-Gm-Gg: AR+sD13OyxymswOFkhx4Of/BLnFQid1hkXo2ylBQH//LmPUYDQAmTlnCNtvK6NLpH/u 2p9qZ/zdrf4XlkEmvk1dkKtaV2d6xwpW9Gin5KnJkGmCGUXBBfFnQDYMtkC5rRSyiT1xH0rVBph bmCp3TJVbyhyDj7CDKo0Dggx0nVIndw2q1g0GZ9Q8v81CmFVdylVDJxVeuABuPgGHGHDExUyoCt wd0YeJvSOB0tu+UmDUxTF1xqfPhdFP66l4LAErbyu2ahLRYeXSkHAPyp6a3DHfe0eLniOCsaZ8f 3Q/qAbhPPTW60Z0kMR0ljelXTR7d06YQEzo3VeSTii4n58Ggi8rhOsDRXwgnz7RoMbQg/M9I7fZ cHDAvbC8k6Xvme0KCrrnjNBN3nKPzQtHQRsUhE85jm8DHduT4NVKwjp6p55tusqEyRlUocOMFv+ XJ4mzADRQXd8PO6o5u5VeNqEaQbIGYatS+D8lo0Rl0pkJNiVFjVfA7n+ye/KusEtg7AYBJcvCsi ypv0HU1TGHCtRTMuWvR3Wy8JQ== X-Received: by 2002:a17:903:1aa7:b0:2c9:b404:b55 with SMTP id d9443c01a7336-2d3105d4b0dmr814415ad.6.1786384785937; Mon, 10 Aug 2026 10:59:45 -0700 (PDT) Received: from google.com (199.255.142.34.bc.googleusercontent.com. [34.142.255.199]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84f5a51e99asm4247058b3a.33.2026.08.10.10.59.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Aug 2026 10:59:44 -0700 (PDT) Date: Mon, 10 Aug 2026 17:59:37 +0000 From: Pranjal Shrivastava To: Alex Williamson Cc: Kevin Tian , kvm@vger.kernel.org, Jason Gunthorpe , Ankit Agrawal , Matt Evans , Leon Romanovsky , Vivek Kasireddy , Jacob Moroni , David Hu , Samiullah Khawaja , linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v1 0/1] vfio/pci: Revoke BARs and DMABUFs during sysfs-triggered PCI reset Message-ID: References: <20260807201405.3717430-1-praan@google.com> <20260810100352.1e2004c8@shazbot.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260810100352.1e2004c8@shazbot.org> On Mon, Aug 10, 2026 at 10:03:52AM -0600, Alex Williamson wrote: Hi Alex, > On Fri, 7 Aug 2026 20:14:04 +0000 > Pranjal Shrivastava wrote: > > > Introduce PCI .reset_prepare and .reset_done handlers to safely revoke > > active userspace mappings and exported DMABUFs during sysfs-triggered > > device resets. > > > > We are seeing a situation where system health and monitoring daemons > > (at times erroneously) issue device resets via sysfs for devices bound > > to vfio-pci: > > > > echo 1 > /sys/bus/pci/devices/0000:01:00.0/reset > > > > However, because vfio-pci does not implement the .reset_prepare and > > .reset_done error handlers, this hardware reset occurs completely unnoticed > > by the VFIO driver. > > > > Consequently, active traditional userspace BAR mappings and exported DMABUFs > > are never zapped or revoked. Importers of the DMABUFs (e.g., RDMA drivers) > > continue to issue DMAs (such as PCIe Memory Writes) toward the Endpoint. > > These transactions are silently dropped by the root port or trigger CTOs > > while higher-level actions (e.g., RDMA reg_mr) continue to succeed. > > > > We'd like to fix this by implementing the PCI reset ops for vfio-pci > > that revoke the DMABUFs and zap the BARs while holding the memory lock > > allowing concurrent user accesses to sleep and fault back in once the reset > > completes. > > That sounds like a nice, serene solution, but that's not actually what > happens. Due to the write vs read memory_lock semaphore, CPU faults > are stalled. On the other hand, DMA mappings via IOMMUFD/dmabuf are > lost. They require the userspace driver to be involved to perform the > unmap/remap. > > Potentially this is all better than letting the device generate a > machine check as it's still trying to run across the reset, but let's > not pretend this is just a hiccup for the device that will continue > running after the rogue reset. Thanks, > I tend to agree. My intention is definitely not to pretend this is a seamless hiccup or allow the device/user to carry on as if nothing happened. In fact, the very problem today with exported DMABUFs is that the user/importer *does* silently continue to register DMABUFs with the RDMA subsystem and attempts issuing DMAs to a reset device because nothing told them the state was gone. I don't mind permanently tearing down the CPU mappings as well along with revoking the DMABUFs. That way, we fail loudly and force userspace to unmap and re-initialize (with a dev_warn() explaining that an out-of-band reset occurred). What do you think about that approach? Thanks, Praan