From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E06338A716 for ; Mon, 18 May 2026 18:55:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779130556; cv=none; b=FrFafsSkaLqx9jhCFkAmHJxNn1/h9QTNmwc6x1E6WfdOb1zGZHnBY/EPWfdE7D6413qPX52e2mBGsvmHz7tz5OGBPyFKz4VRM0tMbEuF0iOoZPVbdb4qiShaiL+ZJ1HKwZd70iFnHT6jPCZ5Jn6cr1vojmrEcf65Y5m0Yqs/aM4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779130556; c=relaxed/simple; bh=haAsUB2P2dMZXBWY/VxFwqzdDlbH/EE+xsFbEoz18JY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=P4COhwav4ieTGSVItgdX7iGqhAFFmgH963a2Ynj8MlyL5UDu6Vhy6afTW02TaGUHs0Oq9vuBePn9sby7WLT4emaCg+OzZ4aQSlEyEg69a5PM2w//ItDc+wCjbM7jQIK5smqxSo207vTI5V7dyh7JDDjwq0E5JmclU1pL7NrODok= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qLxLNpw0; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qLxLNpw0" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2ba3b9bcf69so105ad.0 for ; Mon, 18 May 2026 11:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1779130554; x=1779735354; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=N8E7VNGYLDNKxW/Ot3tdafHEntRbQFW1KJ60umQhURs=; b=qLxLNpw0rzfB24LC3wkoeSwtOCEYy24DFL9ydIBwyMHgqESR8DmoDUQEseVlmQKQ5d ZcAkKH+bZeJni5tlZT2bCmtgrJqRbbz2h51NaYtBw05gJE2URt1gZiBHI4i97jg+6auD Vo/dO/p9iwaGsl+yXswFAILi92tYf6rDZoM43WakuDElGq1L2Qi9fF0yi6SzzFy903mT z+4X5DTLGMgDkHEMhj9Kth4E+XYS9oEir3uMGICcl8iIqhJwPmTNCjzy/te3Ge9Yy7gE EEUFmNTF5zX8OQ5sqy2tmhs4OzNO9tKyi7DyR6O/gXcLdcR3w87l5BQZ88UkjbJJc9Yq lRzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779130554; x=1779735354; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=N8E7VNGYLDNKxW/Ot3tdafHEntRbQFW1KJ60umQhURs=; b=kuvoVq4xw0ebNH9ynPgw50JRPkeDPk2xKjFw7+/6PR88Y/zLEpEJnwMGdv5mpjNMOB 9yglfb2RUB0sNuCBYAJYP8CPF2tKHemxSyxlt75ceU9sLVW7u3MleLbbxyHyRPneByiV CRSPjC5bWQdrWZmk2UMOqga2H5bDnysamktfN+h7inyt9bBIkB8rXgX+kyCeJEjDgKaz j9hQkodJWkXheUBoykaNZwaAuEhcLDaJntCLVY/PNMiExIN6/FQANtrKHlvOX2b2vN4h VtFdY1jt/q7/AbRwzPh440w4+TSf2GhPtFzK2xAJD87EhzAXTOPFDyxEay4w/LiST309 mofQ== X-Forwarded-Encrypted: i=1; AFNElJ9RrGTzgHRF+IKEXh8iKht0M10dhAsU5jnigmpq0PkY9o+j4tP9DlIr/vmCOkuUELF4Rta+syxSIrpSVac=@vger.kernel.org X-Gm-Message-State: AOJu0Yzudzquxq/lQqW19viZL+bBKWC0J0LxNRETrNx2Ps8Jb9sBje7R j0IKvEQbB+5QZW0EtozzxRrt4RtRgrPtZumDSSg/S42IZ10AA1XGI5ebaHW0dy51aehJeo/fzOn e30ZdRBUJ X-Gm-Gg: Acq92OF2Otuu/dnOVB+FVWVVE7M6Zf4cFIiPS11/DZyTdlkQtMxWpxFPLLhcmpI+3fe 9Fup1amWnskGy4dUAaD4AFnhPv3Jlire41XyQGJ+PaLk7wtSjbeFmwC3QSuIiGDuqb6JXz1P4IG /qylFSDKwYXNtbbu7Jk9CwHo+cCrBYreL5hxtg3XZNwdHMRsqqhcoXceSpcxuw/CTkpBd0AQwXm rE7Pi6arhlAvYdzIr0ociV9K4eI++HppEB0CBfAw5RtL4jFUDLUy/9IaPhd9U7twBVRbXaIgaXv calhs24xeTH+qiarYpNWq31Z/v/H2ydwO1HnZ9e0/imSzO7wI5PSP2rHM/b7O1lXY3coKJjKGOk OlExzDeZvEsqyZTMvfv0enCllwNGUKZuMzpkwLGcGm/oDuErIKPZxGipbf2M8EbmYBvJ1Id/y7z BLhrgk5buCl70Rmq0feOeUSSjdpV2fa10yLejnK4KvPWoLdDDXOYTVOMp/32VmjhmEsQmBSg== X-Received: by 2002:a17:902:d512:b0:2bc:dc0b:ab29 with SMTP id d9443c01a7336-2bdb0109472mr3404845ad.0.1779130553814; Mon, 18 May 2026 11:55:53 -0700 (PDT) Received: from google.com (153.46.83.34.bc.googleusercontent.com. [34.83.46.153]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2bd5d11c7f3sm154208605ad.70.2026.05.18.11.55.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 May 2026 11:55:53 -0700 (PDT) Date: Mon, 18 May 2026 18:55:50 +0000 From: Samiullah Khawaja To: Pranjal Shrivastava Cc: David Woodhouse , Lu Baolu , Joerg Roedel , Will Deacon , Jason Gunthorpe , Robin Murphy , Kevin Tian , Alex Williamson , Shuah Khan , iommu@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Saeed Mahameed , Adithya Jayachandran , Parav Pandit , Leon Romanovsky , William Tu , Pratyush Yadav , Pasha Tatashin , David Matlack , Andrew Morton , Chris Li , Vipin Sharma , YiFei Zhu Subject: Re: [PATCH v2 03/16] iommu: Implement IOMMU domain preservation Message-ID: References: <20260427175633.1978233-1-skhawaja@google.com> <20260427175633.1978233-4-skhawaja@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: On Mon, May 18, 2026 at 01:13:49PM +0000, Pranjal Shrivastava wrote: >On Mon, Apr 27, 2026 at 05:56:20PM +0000, Samiullah Khawaja wrote: >> Add IOMMU domain ops that can be implemented by the IOMMU drivers if >> they support IOMMU domain preservation across liveupdate. The new IOMMU >> domain preserve, unpreserve and restore APIs call these ops to perform >> respective live update operations. >> >> Signed-off-by: Samiullah Khawaja >> --- >> drivers/iommu/liveupdate.c | 97 ++++++++++++++++++++++++++++++++ >> include/linux/iommu-liveupdate.h | 14 +++++ >> include/linux/iommu.h | 13 +++++ >> 3 files changed, 124 insertions(+) >> >> diff --git a/drivers/iommu/liveupdate.c b/drivers/iommu/liveupdate.c >> index a26099b145c3..f71f14518248 100644 >> --- a/drivers/iommu/liveupdate.c >> +++ b/drivers/iommu/liveupdate.c >> @@ -13,6 +13,9 @@ >> #include >> #include >> >> +#define iommu_max_objs_per_page(_array) \ >> + ((PAGE_SIZE - sizeof(struct iommu_array_hdr_ser)) / sizeof((_array)->objects[0])) >> + >> static void *iommu_liveupdate_restore_array(u64 array_phys) >> { >> struct iommu_array_hdr_ser *array_hdr; >> @@ -196,3 +199,97 @@ void iommu_liveupdate_unregister_flb(struct liveupdate_file_handler *handler) >> liveupdate_unregister_flb(handler, &iommu_flb); >> } >> EXPORT_SYMBOL(iommu_liveupdate_unregister_flb); >> + >> +static int alloc_object_ser(struct iommu_array_hdr_ser **curr_array_ptr, u64 max_objs) >> +{ >> + struct iommu_array_hdr_ser *curr_array = *curr_array_ptr; >> + struct iommu_array_hdr_ser *next_array; >> + >> + if (curr_array->nr_objects >= max_objs) { >> + next_array = kho_alloc_preserve(PAGE_SIZE); >> + if (IS_ERR(next_array)) >> + return PTR_ERR(next_array); >> + >> + curr_array->next_array_phys = virt_to_phys(next_array); >> + *curr_array_ptr = next_array; >> + curr_array = next_array; >> + } >> + >> + return curr_array->nr_objects++; >> +} >> + >> +static struct iommu_domain_ser *alloc_iommu_domain_ser(struct iommu_flb_obj *flb) >> +{ >> + int idx; >> + >> + idx = alloc_object_ser((struct iommu_array_hdr_ser **)&flb->curr_domain_array, > >Nit: Such type-casts could be brittle and risk strict-aliasing warnings >based on the compiler. > >Since alloc_object_ser only updates the top-level tracking pointer, we >can make this much cleaner by changing alloc_object_ser to take a >void curr_array_ptr. That way, we can pass (void )&flb->curr_domain_array >cleanly without the double-pointer type-punning. Something like: > >static int alloc_object_ser(void **curr_array_ptr, u64 max_objs) >{ > struct iommu_array_hdr_ser *curr_array = *curr_array_ptr; > struct iommu_array_hdr_ser *next_array; > > ... >} > >and we can pass it as: > >idx = alloc_object_ser((void **)&flb->curr_domain_array, > iommu_max_objs_per_page(flb->curr_domain_array)); > Agreed. I will update this (including other places). >> + iommu_max_objs_per_page(flb->curr_domain_array)); >> + if (idx < 0) >> + return ERR_PTR(idx); >> + >> + flb->curr_domain_array->objects[idx].hdr.ref_count = 1; >> + return &flb->curr_domain_array->objects[idx]; >> +} >> + >> +int iommu_domain_preserve(struct iommu_domain *domain, struct iommu_domain_ser **ser) >> +{ >> + struct iommu_domain_ser *domain_ser; >> + struct iommu_flb_obj *flb_obj; >> + int ret; >> + >> + if (!domain->ops->preserve) >> + return -EOPNOTSUPP; >> + >> + ret = liveupdate_flb_get_outgoing(&iommu_flb, (void **)&flb_obj); >> + if (ret) >> + return ret; >> + >> + guard(mutex)(&flb_obj->lock); >> + domain_ser = alloc_iommu_domain_ser(flb_obj); >> + if (IS_ERR(domain_ser)) >> + return PTR_ERR(domain_ser); >> + >> + ret = domain->ops->preserve(domain, domain_ser); >> + if (ret) { >> + domain_ser->hdr.deleted = true; > >Nit: This will become domain_ser->hdr.flags |= IOMMU_SER_FLAG_DELETED; >once we convert the bitfields in PATCH 2 to flags. Agreed. I think it is being discussed that bitfields are fine in the other thread, but I will switch it to flags for readibility anyway. > >> + return ret; >> + } >> + >> + domain->preserved_state = domain_ser; >> + *ser = domain_ser; >> + return 0; >> +} >> +EXPORT_SYMBOL_GPL(iommu_domain_preserve); >> + > >[...] > >> static inline bool iommu_is_dma_domain(struct iommu_domain *domain) >> @@ -752,6 +757,11 @@ struct iommu_ops { >> * specific mechanisms. >> * @set_pgtable_quirks: Set io page table quirks (IO_PGTABLE_QUIRK_*) >> * @free: Release the domain after use. >> + * @preserve: Preserve the iommu domain for liveupdate. >> + * Returns 0 on success, a negative errno on failure. >> + * @unpreserve: Unpreserve the iommu domain that was preserved earlier. >> + * @restore: Restore the iommu domain after liveupdate. >> + * Returns 0 on success, a negative errno on failure. >> */ >> struct iommu_domain_ops { >> int (*attach_dev)(struct iommu_domain *domain, struct device *dev, >> @@ -782,6 +792,9 @@ struct iommu_domain_ops { >> unsigned long quirks); >> >> void (*free)(struct iommu_domain *domain); >> + int (*preserve)(struct iommu_domain *domain, struct iommu_domain_ser *ser); >> + void (*unpreserve)(struct iommu_domain *domain, struct iommu_domain_ser *ser); >> + int (*restore)(struct iommu_domain *domain, struct iommu_domain_ser *ser); > >Nit: Should we add restore in the patch we implement it? The ops are implemented in iommupt (separate scope) and it is done after the iommu pages patch as that is also required for it. > >> }; > >With the above nits, > >Reviewed-by: Pranjal Shrivastava Thanks Pranjal. > >Thanks, >Praan Sami