From: Samiullah Khawaja <skhawaja@google.com>
To: Baolu Lu <baolu.lu@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>,
Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
Jason Gunthorpe <jgg@ziepe.ca>,
Robin Murphy <robin.murphy@arm.com>,
Kevin Tian <kevin.tian@intel.com>,
Alex Williamson <alex@shazbot.org>,
Shuah Khan <shuah@kernel.org>,
iommu@lists.linux.dev, linux-kernel@vger.kernel.org,
kvm@vger.kernel.org, Pratyush Yadav <pratyush@kernel.org>,
Pasha Tatashin <pasha.tatashin@soleen.com>,
David Matlack <dmatlack@google.com>,
Andrew Morton <akpm@linux-foundation.org>,
Pranjal Shrivastava <praan@google.com>,
Vipin Sharma <vipinsh@google.com>
Subject: Re: [PATCH v3 08/18] iommu/vt-d: clear unpreserved context entries during shutdown
Date: Mon, 22 Jun 2026 22:56:46 +0000 [thread overview]
Message-ID: <ajmYQFemGb2JD0BF@google.com> (raw)
In-Reply-To: <51e03b42-796c-489c-a0fa-525ed1a492ca@linux.intel.com>
On Mon, Jun 22, 2026 at 10:47:05AM +0800, Baolu Lu wrote:
>On 6/15/26 07:37, Samiullah Khawaja wrote:
>>During normal shutdown the iommu translation is disabled. Since the root
>>table is preserved during live update, it needs to be cleaned up and the
>>context entries of the unpreserved devices and root entries for the
>>unpreserved context tables need to be cleared.
>>
>>Signed-off-by: Samiullah Khawaja <skhawaja@google.com>
>>---
>> drivers/iommu/intel/iommu.c | 9 ++--
>> drivers/iommu/intel/iommu.h | 6 +++
>> drivers/iommu/intel/liveupdate.c | 74 ++++++++++++++++++++++++++++++++
>> 3 files changed, 86 insertions(+), 3 deletions(-)
>
>Please tweak the commit title to: "iommu/vt-d: Clear unpreserved..."
>(capitalize "Clear") to match the driver's commit history style.
Agreed. Will do.
>
>A high-level question: have you looked at how the suspend/resume path
>behaves with the iommu and device preservation? DMA translation is
>disabled and re-enabled there. I don't see any immediate changes are
>needed there, but it would be good to call it out explicitly if it was
>overlooked.
I looked into this during early implementation. The preserved state data
structures are not affected by the suspension of the IOMMU, as during
resume it reuses those exact same data structures in RAM. But I have not
tested this since it is unlikely that in the environments where
liveupdate is used, the suspend/resume functionality is also used.
But I plan to simply return -EBUSY from iommu_suspend() if the IOMMU is
currently preserved. This explicitly prevents the two states from
overlapping and avoids any complex edge cases. I'll add that check in
v4.
>
>>
>>diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>>index 715b538e7efe..26258861e3bf 100644
>>--- a/drivers/iommu/intel/iommu.c
>>+++ b/drivers/iommu/intel/iommu.c
>>@@ -2374,8 +2374,11 @@ void intel_iommu_shutdown(void)
>> /* Disable PMRs explicitly here. */
>> iommu_disable_protect_mem_regions(iommu);
>>- /* Make sure the IOMMUs are switched off */
>>- iommu_disable_translation(iommu);
>>+ /* Make sure the IOMMUs are switched off if not preserved. */
>>+ if (iommu_preserved_state(&iommu->iommu))
>>+ clear_unpreserved_context_entries(iommu);
>
>How are PCI devices handled during a live update kexec? Do they go
>through the standard iommu_release_device() path?
>
>I assume they do not, because if they did, the context entries for
>preserved devices could be updated after preservation. If they do bypass
>the release_device path, why not just explicitly invoke
>iommu_release_device() for all devices that are not preserved?
>
>Using iommu_release_device() for the unpreserved devices would naturally
>erase their context entries and securely park those devices in a DMA
>blocking state.
I considered reusing iommu_release_device(), but it leaves the hardware
in a state that is unsafe to carry across a kexec boundary when global
translation remains enabled.
- In legacy mode, iommu_release_device() does not clear the context
entries. It assumes device_block_translation() was called during a
prior domain detach to clear context entries.
- In scalable mode, while it does clear the context entries, it skips
the PASID cache invalidations (per VT-d spec 6.5.3.3 Table 25, second
row). It assumes intel_pasid_tear_down_entry() was already called
during domain detach to handle the invalidations.
During a normal shutdown, this is safe because the IOMMU driver disables
translation globally. However, since live update keeps global
translation enabled across the kexec, relying on iommu_release_device()
would lead to DMAR faults from unpreserved devices.
Manually clearing the entries and issuing global invalidations is the
safest way to guarantee the unpreserved devices are securely parked.
>
>>+ else
>>+ iommu_disable_translation(iommu);
>> }
>> }
>>
>
>[...]
>
>Thanks,
>baolu
Thanks,
Sami
next prev parent reply other threads:[~2026-06-22 22:56 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-14 23:37 [PATCH v3 00/18] iommu: Add live update state preservation Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 01/18] memfd: export memfd_get_seals() Samiullah Khawaja
2026-06-15 5:14 ` Ankit Soni
2026-06-15 11:45 ` Pratyush Yadav
2026-06-14 23:37 ` [PATCH v3 02/18] iommu: Implement IOMMU Live update FLB callbacks Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 03/18] iommu/pages: Add APIs to preserve/unpreserve/restore iommu pages Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 04/18] iommupt: Implement preserve/unpreserve/restore callbacks Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 05/18] iommu: Implement IOMMU domain preservation Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 06/18] iommu: Implement device and IOMMU HW preservation Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 07/18] iommu/vt-d: Implement device and iommu preserve/unpreserve ops Samiullah Khawaja
2026-06-22 1:50 ` Baolu Lu
2026-06-22 19:19 ` Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 08/18] iommu/vt-d: clear unpreserved context entries during shutdown Samiullah Khawaja
2026-06-22 2:47 ` Baolu Lu
2026-06-22 22:56 ` Samiullah Khawaja [this message]
2026-06-14 23:37 ` [PATCH v3 09/18] iommu: Add APIs to get iommu and device preserved state Samiullah Khawaja
2026-06-22 3:10 ` Baolu Lu
2026-06-22 23:27 ` Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 10/18] iommu/vt-d: Restore IOMMU state and reclaimed domain ids Samiullah Khawaja
2026-06-22 5:14 ` Baolu Lu
2026-06-22 23:30 ` Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 11/18] iommu: Restore and reattach preserved domains to devices Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 12/18] iommu/vt-d: Handle reattach of the restored domain Samiullah Khawaja
2026-06-22 5:44 ` Baolu Lu
2026-06-23 0:26 ` Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 13/18] iommu/vt-d: preserve PASID table of preserved device Samiullah Khawaja
2026-06-22 6:01 ` Baolu Lu
2026-06-23 0:36 ` Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 14/18] iommufd: Implement ioctl to mark HWPT for preservation Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 15/18] iommufd: Persist iommu hardware pagetables for live update Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 16/18] iommufd: Add APIs to preserve/unpreserve a vfio cdev Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 17/18] vfio/pci: Preserve the iommufd state of the " Samiullah Khawaja
2026-06-14 23:37 ` [PATCH v3 18/18] iommufd/selftest: Add test to verify iommufd preservation Samiullah Khawaja
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ajmYQFemGb2JD0BF@google.com \
--to=skhawaja@google.com \
--cc=akpm@linux-foundation.org \
--cc=alex@shazbot.org \
--cc=baolu.lu@linux.intel.com \
--cc=dmatlack@google.com \
--cc=dwmw2@infradead.org \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pasha.tatashin@soleen.com \
--cc=praan@google.com \
--cc=pratyush@kernel.org \
--cc=robin.murphy@arm.com \
--cc=shuah@kernel.org \
--cc=vipinsh@google.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.