Kernel KVM virtualization development
 help / color / mirror / Atom feed
From: Baolu Lu <baolu.lu@linux.intel.com>
To: Samiullah Khawaja <skhawaja@google.com>
Cc: baolu.lu@linux.intel.com, David Woodhouse <dwmw2@infradead.org>,
	Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
	Jason Gunthorpe <jgg@ziepe.ca>,
	Robin Murphy <robin.murphy@arm.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Alex Williamson <alex@shazbot.org>, Shuah Khan <shuah@kernel.org>,
	iommu@lists.linux.dev, linux-kernel@vger.kernel.org,
	kvm@vger.kernel.org, Saeed Mahameed <saeedm@nvidia.com>,
	Adithya Jayachandran <ajayachandra@nvidia.com>,
	Parav Pandit <parav@nvidia.com>,
	Leon Romanovsky <leonro@nvidia.com>, William Tu <witu@nvidia.com>,
	Pratyush Yadav <pratyush@kernel.org>,
	Pasha Tatashin <pasha.tatashin@soleen.com>,
	David Matlack <dmatlack@google.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Chris Li <chrisl@kernel.org>,
	Pranjal Shrivastava <praan@google.com>,
	Vipin Sharma <vipinsh@google.com>,
	YiFei Zhu <zhuyifei@google.com>
Subject: Re: [PATCH v2 11/16] iommu/vt-d: preserve PASID table of preserved device
Date: Tue, 12 May 2026 19:32:08 +0800	[thread overview]
Message-ID: <416088c2-b1b9-4011-bd72-4368647cb2a1@linux.intel.com> (raw)
In-Reply-To: <agIGEyL12Djj2clV@google.com>

On 5/12/2026 2:45 AM, Samiullah Khawaja wrote:
> On Fri, May 08, 2026 at 02:05:58PM +0800, Baolu Lu wrote:
>> On 4/28/26 01:56, Samiullah Khawaja wrote:
>>> In scalable mode the PASID table is used to fetch the io page tables.
>>> Preserve and restore the PASID table of the preserved devices.
>>>
>>> Signed-off-by: Samiullah Khawaja<skhawaja@google.com>
>>> ---
>>>  drivers/iommu/intel/iommu.c      |   5 +-
>>>  drivers/iommu/intel/iommu.h      |  12 +++
>>>  drivers/iommu/intel/liveupdate.c | 141 +++++++++++++++++++++++++++++++
>>>  drivers/iommu/intel/pasid.c      |   7 +-
>>>  drivers/iommu/intel/pasid.h      |   9 ++
>>>  include/linux/kho/abi/iommu.h    |  13 +++
>>>  6 files changed, 184 insertions(+), 3 deletions(-)
>>>
> 
> [snip]
>>> +
>>> +void pasid_cleanup_preserved_table(struct device *dev)
>>> +{
>>> +    struct pasid_table *pasid_table;
>>> +    struct pasid_dir_entry *dir;
>>> +    struct pasid_entry *table;
>>> +    size_t dir_size;
>>> +
>>> +    pasid_table = intel_pasid_get_table(dev);
>>> +    if (!pasid_table)
>>> +        return;
>>> +
>>> +    dir = pasid_table->table;
>>> +    table = get_pasid_table_from_pde(&dir[0]);
>>> +    if (!table)
>>> +        return;
>>> +
>>> +    /* Clear everything except the first entry in table. */
>>> +    memset(&table[1], 0, SZ_4K - sizeof(*table));
>>> +
>>> +    /* Use the folio order to calculate the size of Pasid Directory */
>>> +    dir_size = (1 << (folio_order(virt_to_folio(dir)) + PAGE_SHIFT));
>>> +
>>> +    /* Clear everything except the first entry in directory */
>>> +    memset(&dir[1], 0, dir_size - sizeof(struct pasid_dir_entry));
>>> +
>>> +    clflush_cache_range(&table[0], SZ_4K);
>>> +    clflush_cache_range(&dir[0], dir_size);
>>> +}
>>
>> The PASID table is currently active and in use by the hardware. Clearing
>> the entries without the necessary hardware cache invalidation is buggy.
>>
>> It seems this manual clearing is a workaround because PASID domain
>> preservation isn't supported yet. If so, rather than clearing the table
>> blindly, the code should verify if any PASIDs (other than
>> IOMMU_NO_PASID) are actually in use. If there are, the preserve callback
>> should return an error.
> 
> Thanks for looking into this, I agree and will remove the clearing logic
> here.
> 
> Yes, we do this check in iommufd, as it is the dma owner of the device,
> and only DMA owned devices are allowed to be preserved.
> 
> During preservation iommufd returns an error if the device has PASID
> (non NO_PASID) attachments. And once the device is preserved, any PASID
> attachments are not allowed until the device is unpreserved.
> 
> I think I will make this check robust by moving it into core and use
> pasid_array. It will require some plumbing as pasid_array exists in
> iommu.c file.

Yeah, that would be better.

Thanks,
baolu

  reply	other threads:[~2026-05-12 11:32 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-04-27 17:56 [PATCH v2 00/16] iommu: Add live update state preservation Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 01/16] liveupdate: luo_file: Add internal APIs for file preservation Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 02/16] iommu: Implement IOMMU Live update FLB callbacks Samiullah Khawaja
2026-05-01 21:45   ` David Matlack
2026-04-27 17:56 ` [PATCH v2 03/16] iommu: Implement IOMMU domain preservation Samiullah Khawaja
2026-05-01 22:08   ` David Matlack
2026-05-04 18:33     ` Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 04/16] iommu: Implement device and IOMMU HW preservation Samiullah Khawaja
2026-05-01 22:42   ` David Matlack
2026-05-04 19:06     ` Samiullah Khawaja
2026-05-07  2:07   ` Baolu Lu
2026-05-07 18:47     ` Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 05/16] iommu/pages: Add APIs to preserve/unpreserve/restore iommu pages Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 06/16] iommupt: Implement preserve/unpreserve/restore callbacks Samiullah Khawaja
2026-05-07  2:55   ` Baolu Lu
2026-05-07 18:40     ` Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 07/16] iommu/vt-d: Implement device and iommu preserve/unpreserve ops Samiullah Khawaja
2026-05-07  6:25   ` Baolu Lu
2026-05-08  2:36     ` Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 08/16] iommu: Add APIs to get iommu and device preserved state Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 09/16] iommu/vt-d: Restore IOMMU state and reclaimed domain ids Samiullah Khawaja
2026-05-07  9:05   ` Baolu Lu
2026-05-07 17:35     ` Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 10/16] iommu: Restore and reattach preserved domains to devices Samiullah Khawaja
2026-05-07 13:54   ` Baolu Lu
2026-05-07 16:52     ` Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 11/16] iommu/vt-d: preserve PASID table of preserved device Samiullah Khawaja
2026-05-08  6:05   ` Baolu Lu
2026-05-11 18:45     ` Samiullah Khawaja
2026-05-12 11:32       ` Baolu Lu [this message]
2026-04-27 17:56 ` [PATCH v2 12/16] iommufd: Implement ioctl to mark HWPT for preservation Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 13/16] iommufd: Persist iommu hardware pagetables for live update Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 14/16] iommufd: Add APIs to preserve/unpreserve a vfio cdev Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 15/16] vfio/pci: Preserve the iommufd state of the " Samiullah Khawaja
2026-04-27 17:56 ` [PATCH v2 16/16] iommufd/selftest: Add test to verify iommufd preservation Samiullah Khawaja

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=416088c2-b1b9-4011-bd72-4368647cb2a1@linux.intel.com \
    --to=baolu.lu@linux.intel.com \
    --cc=ajayachandra@nvidia.com \
    --cc=akpm@linux-foundation.org \
    --cc=alex@shazbot.org \
    --cc=chrisl@kernel.org \
    --cc=dmatlack@google.com \
    --cc=dwmw2@infradead.org \
    --cc=iommu@lists.linux.dev \
    --cc=jgg@ziepe.ca \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=leonro@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=parav@nvidia.com \
    --cc=pasha.tatashin@soleen.com \
    --cc=praan@google.com \
    --cc=pratyush@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=saeedm@nvidia.com \
    --cc=shuah@kernel.org \
    --cc=skhawaja@google.com \
    --cc=vipinsh@google.com \
    --cc=will@kernel.org \
    --cc=witu@nvidia.com \
    --cc=zhuyifei@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox