From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6C20DC982ED for ; Mon, 21 Sep 2026 14:51:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7EC426B0088; Mon, 21 Sep 2026 10:51:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C3C46B00EC; Mon, 21 Sep 2026 10:51:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D9B36B00ED; Mon, 21 Sep 2026 10:51:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3B4EC6B00EC for ; Mon, 21 Sep 2026 10:51:07 -0400 (EDT) Received: from smtpin10.hostedemail.com (lb01a-stub [10.200.18.249]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B7130A4AA0 for ; Mon, 21 Sep 2026 14:51:06 +0000 (UTC) X-FDA: 85238056932.10.B00FFC1 Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf24.hostedemail.com (Postfix) with ESMTP id 2CC3E180002 for ; Mon, 21 Sep 2026 14:51:05 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20260515 header.b=U2t85MqE; spf=pass (imf24.hostedemail.com: domain of aneesh.kumar@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=aneesh.kumar@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1790002265; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RIpxqITDfA7RmolIrlPIbBKnqZiSPFBAJdQmg5tAnIk=; b=6MgB8qOKsRMRmOEfY0dx2zj+2LdxU+FuAqMI4duAvcxw5MsMczc3lDDcSYXF5BJEhNX+qt 51BEpFbePzk7WPuejti5fMnqTmqiFsKFyVMewbpZqeNfJ9oZOQu2BXMW9B5IAe1nSbgern pIr5HAj7VDTl7eaV76+0NHdc8l53yIg= ARC-Seal: i=1; a=rsa-sha256; d=hostedemail.com; s=arc-20220608; cv=none; t=1790002265; b=jFbInOcwj52C+Cj74ae/z8YMhCPygeW0WMJrhsbR0luU/+eAqFUMlPV/cjGgbH/L8vbd5V ri7G80eVHPXdhZvflLtrd4xKc5BvY4aF0N7KirzPmPB1JF9XGB6vpKQ/9Gav9L12Fn4xrx ShfUyd1Nh0Osgu5nZNoulQ43bfmDdgI= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20260515 header.b=U2t85MqE; spf=pass (imf24.hostedemail.com: domain of aneesh.kumar@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=aneesh.kumar@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) by tor.source.kernel.org (Postfix) with ESMTP id A1CA9600D1; Mon, 21 Sep 2026 14:51:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 77EE21F00893; Mon, 21 Sep 2026 14:50:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002264; bh=RIpxqITDfA7RmolIrlPIbBKnqZiSPFBAJdQmg5tAnIk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=U2t85MqEar9/QE6fbQs7GNiO7mB2uSgtDrQex0/73Ec54N3fnUaqTEvimBm4eBh5X Mx7reovji1y0h9moEFn5hOC1cwh4mQ1xNkSDZv1aKe5jZbyNPpY6qd8s0qXiiaepDg vpI3sL1MsD2yNWaA2h7v2fj/Z0cfBMWQRwFa487uO6sTn+Z1INBuR7Ifd5ndf98yls 1incq5rJC0DTmnsm2F8SmMB0/9YCzOL3YnishM1Zuy9vjpkRuAbnWCnqfuUK0wHZGb L725hd/X0O6zpYej/UqSZ4PFCbDhcjLR9kbVZ/qZxUXm6yYazpLKIenCh2gvu2BCwV IlJ65pPpNT/eg== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 13/13] swiotlb: Make rounded shared pool capacity allocatable Date: Mon, 21 Sep 2026 20:18:47 +0530 Message-ID: <20260921144847.501151-14-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam04 X-Rspam-User: X-Stat-Signature: b7ady9316bur3nutypse1cwddyay6zc5 X-Rspamd-Queue-Id: 2CC3E180002 X-HE-Tag: 1790002265-972903 X-HE-Meta: U2FsdGVkX1/qQI5+HEc+wlSsDO7Kysfnpha635sNsgXBH7Hur+BPxB3rtQCehNC1104cqafT9rpHYV7UYIBcLv7Fe7M/7bppRfrDJ5VZ0/cZ2mJJwZJKwV7zOUX1bXVZFlS6bYXZaRgZoe+ODqLW3IX5UP6Y2UuVQXvpIM7+ntK/OLvD/KigJXv+wfwCHsGOfdnYbTP+KwnFDcqJrdt46l0UQX1b0ZxSglJgrf8dM3q7Aip/Np/wzB1xKurYRvuJ930GwGowupEpvrUG39bMimFBP99ri6tSepqoD5JUO81QY7Ekxo1NrHAV8FFbWLmwLUVj+/ecjmf78n4XK9+7hdgsvYkPr+kX11R4tlkmHBa/5+LfATKqCogrk9YGDFIg9qolhr5C7qi/hOeGLJmpH1yVvmF0ryTxiSPUaRjboHUbg8ymKMSp5TUjIUAI+rDfJTLOKlhGmM2oLJ8E3iwLBDicGEVMlFRppUu+UevnlIcYsFc9qe/RjnxmcwB9acYENchxFaXAS9rCzXe5TWpQeS4j0LLyr1VKmH8J9BTeNrxqev+jf9DzlYBhEdgb4HQ+hO/tlgAHFGLOY9B7QK6uBauqd/8Khyam7vcexG42ODLEHRGCU9g/1rko7yb0drhy6Pc6Q4qCVP1ATwcHBgBSxitNCBVB4kKT5kXQHCMGl/PKrzig+QYTLc2sIicAXDKLhiDDGdYXN1YwxE8u7nuukGKEPvFvZ3RV6CRKx8gO2s3qhDWeiuIU+ZjHI7TqR3jWUTeqX21ufTn6z3ROi5CkOcQfy2YPMwQ6UpGronTE8tGuqC1cleK0XPQlJI6wblX4MtXVkeeCybnBTa4pIoFckTBrityZWD56+LAvgieXxT0zUMAz4W+JoNa4tlEQbMLAwu/PLUPY/nU0cgQVDfUZ7zC9r8zSZlL6v9P44JYw+jiEJUQErqtZcImMfRFTSb4uQvbxSnmVzG7nt2Y0I7k XxVe7NOL NETex+oy4Um+g+lXFH5I5rqnv6z9e7LhcHk2XQOi6XqUe1ab2GOJM4hRTn7v1QF7UpUV1kMyv/7UL04x7u1DRvY4tlp+rxQKTXLLlcWmV9SiOumtkq9y3JRqmnsZQWFgY55+zDIQxJ4bnAJyBQoB25A6lUanFH1AWc3d0M2uoQgVeIYka+qvm3AbEccHzVsqXqzN0IsOcHWk64RxlWUKCd4qGwnescZZgjuLLF+QwaZCx3VE= Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: CoCo shared memory may need to be allocated and transitioned in units larger than the requested object. Before this change, users handled the resulting capacity as follows: User Rounded capacity reused dma-buf system heap no DMA-direct no regular GIC tables no small GIC ITTs yes, through a gen_pool early SWIOTLB pool no late SWIOTLB pool yes persistent dynamic SWIOTLB no transient dynamic SWIOTLB no, one mapping only atomic DMA pools yes, through a gen_pool restricted SWIOTLB pool no additional padding Improve the early and persistent dynamic SWIOTLB pools. They already own and transition backing rounded to the shared granule size, and SWIOTLB is itself a suballocator. Advertise the rounded extent as slots, size the slot metadata to match. This makes the extra capacity available without reserving more backing memory. Keep transient dynamic pools unchanged. A transient pool belongs to one DMA mapping and is destroyed when that mapping is unmapped, so its spare backing cannot satisfy a later request without changing the lifetime model. Do not attempt the same optimization for dma-buf, DMA-direct or regular GIC objects. Those allocations have independent caller-visible sizes and lifetimes. Reusing their padding requires a shared-granule suballocator with reference counting, per-object mappings and accounting. Note: For the current 64 KiB CCA shared granule size, SWIOTLB pool sizes are already multiples of the 256 KiB IO_TLB segment size. Consequently, the rounding does not change any runtime values on current CCA systems. It instead makes the code express the intended invariant that pool metadata describes the complete shared-granule-aligned backing allocation. Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index cb67105b8812..9577a8807b07 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -330,6 +330,14 @@ static inline unsigned long nr_slots(u64 val) return DIV_ROUND_UP(val, IO_TLB_SIZE); } +static unsigned long swiotlb_align_nslabs(unsigned long nslabs) +{ + unsigned long granule_nslabs; + + granule_nslabs = cc_shared_granule_size() >> IO_TLB_SHIFT; + return ALIGN(nslabs, granule_nslabs); +} + static void swiotlb_mark_pool_used(struct io_tlb_pool *pool) { unsigned long i; @@ -435,11 +443,12 @@ static void add_mem_pool(struct io_tlb_mem *mem, struct io_tlb_pool *pool) } static void __init *swiotlb_memblock_alloc(unsigned long nslabs, - unsigned int flags, + unsigned long *alloc_nslabs, unsigned int flags, int (*remap)(void *tlb, unsigned long nslabs)) { + unsigned long aligned_nslabs = swiotlb_align_nslabs(nslabs); + size_t bytes = aligned_nslabs << IO_TLB_SHIFT; void *tlb; - size_t bytes = ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); /* * By default allocate the bounce buffer memory from low memory, but @@ -457,12 +466,13 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs, return NULL; } - if (remap && remap(tlb, nslabs) < 0) { + if (remap && remap(tlb, aligned_nslabs) < 0) { memblock_free(tlb, bytes); pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes); return NULL; } + *alloc_nslabs = aligned_nslabs; return tlb; } @@ -475,6 +485,7 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags, { struct io_tlb_pool *mem = &io_tlb_default_mem.defpool; unsigned long nslabs; + unsigned long alloc_nslabs; unsigned int nareas; size_t alloc_size; void *tlb; @@ -499,13 +510,14 @@ void __init swiotlb_init_remap(bool addressing_limit, unsigned int flags, swiotlb_adjust_nareas(num_possible_cpus()); nslabs = default_nslabs; - nareas = limit_nareas(default_nareas, nslabs); - while ((tlb = swiotlb_memblock_alloc(nslabs, flags, remap)) == NULL) { + while ((tlb = swiotlb_memblock_alloc(nslabs, &alloc_nslabs, flags, + remap)) == NULL) { if (nslabs <= IO_TLB_MIN_SLABS) return; nslabs = ALIGN(nslabs >> 1, IO_TLB_SEGSIZE); - nareas = limit_nareas(nareas, nslabs); } + nslabs = alloc_nslabs; + nareas = limit_nareas(default_nareas, nslabs); if (default_nslabs != nslabs) { pr_info("SWIOTLB bounce buffer size adjusted %lu -> %lu slabs", @@ -871,6 +883,12 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct device *dev, tlb_size = nslabs << IO_TLB_SHIFT; } + /* Transient pools are tied to one mapping and cannot reuse padding. */ + if (mem->cc_shared && !dev) { + nslabs = swiotlb_align_nslabs(nslabs); + tlb_size = nslabs << IO_TLB_SHIFT; + } + slot_order = get_order(array_size(sizeof(*pool->slots), nslabs)); pool->slots = (struct io_tlb_slot *) __get_free_pages(gfp, slot_order); -- 2.43.0