From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BBBBEB64DC for ; Thu, 6 Jul 2023 08:07:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=P0hGGPPfajjD0m785f9n7m5chY0xUsOiSgycB2cXuUQ=; b=bQD49zjW9IWAPn UITQ2tRubbf29XvTRUtcRsmHeuPWMQbVr76nSFR26DarN5qs/OGkhymarh21dqZNBC8N6IMFJgis9 553nZfAaMzgDgSrQJcQAAdn5wwWJW/XSZBsq684Ddxn6MmCqns3PL1+3tYhxY2KQMaQCUUmuKq8xP ZfeeWFzbki9yC5GkdSSAxZsc174ksRe1FJA8lm36+a5BNRqP6tcsiryRWJpmuMn6oSoLau440hcsH ynlJlSOYQCVEcTCNfEKvXg6tylqRmeCmU7/hRLr9gFDvVuyfpHJzicTtFGb8DhLZ5MnVYl423Eq1j YfH4YBMEbda8hc7X640Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qHK1H-000tpI-0P; Thu, 06 Jul 2023 08:07:19 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qHK1E-000toQ-32 for linux-arm-kernel@lists.infradead.org; Thu, 06 Jul 2023 08:07:18 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9D2FB61701; Thu, 6 Jul 2023 08:07:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DD56C433C8; Thu, 6 Jul 2023 08:07:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1688630835; bh=iNI/V0FUmVFi1ifs3C8XNjXKbihtOUYD+TDijCk37HE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=OB+27qMDSzSjagtWO3H4twriHs6EnYntX7f3TliqB+th2ekfBwodAiGjyoiF52+0o SDWo0DcO9dd1onHNcAuM7KPg9vg1oxEShO52NsmMLe5nM7cg41BHSJXYx+G0uAVF8v PtEyANTWI23BraWaPOeLX69jQfpIO3MWHj2Wyu5E= Date: Thu, 6 Jul 2023 09:07:12 +0100 From: Greg Kroah-Hartman To: "Michael Kelley (LINUX)" Cc: Petr Tesarik , Stefano Stabellini , Thomas Bogendoerfer , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , "H. Peter Anvin" , "Rafael J. Wysocki" , Juergen Gross , Oleksandr Tyshchenko , Christoph Hellwig , Marek Szyprowski , Robin Murphy , Andy Shevchenko , Hans de Goede , Jason Gunthorpe , Kees Cook , Saravana Kannan , "moderated list:XEN HYPERVISOR ARM" , "moderated list:ARM PORT" , open list , "open list:MIPS" , "open list:XEN SWIOTLB SUBSYSTEM" , Roberto Sassu , Kefeng Wang , "petr@tesarici.cz" Subject: Re: [PATCH v3 4/7] swiotlb: if swiotlb is full, fall back to a transient memory pool Message-ID: <2023070626-boxcar-bubbly-471d@gregkh> References: <34c2a1ba721a7bc496128aac5e20724e4077f1ab.1687859323.git.petr.tesarik.ext@huawei.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230706_010717_100234_6D92A007 X-CRM114-Status: GOOD ( 29.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Jul 06, 2023 at 03:50:55AM +0000, Michael Kelley (LINUX) wrote: > From: Petr Tesarik Sent: Tuesday, June 27, 2023 2:54 AM > > > > Try to allocate a transient memory pool if no suitable slots can be found, > > except when allocating from a restricted pool. The transient pool is just > > enough big for this one bounce buffer. It is inserted into a per-device > > list of transient memory pools, and it is freed again when the bounce > > buffer is unmapped. > > > > Transient memory pools are kept in an RCU list. A memory barrier is > > required after adding a new entry, because any address within a transient > > buffer must be immediately recognized as belonging to the SWIOTLB, even if > > it is passed to another CPU. > > > > Deletion does not require any synchronization beyond RCU ordering > > guarantees. After a buffer is unmapped, its physical addresses may no > > longer be passed to the DMA API, so the memory range of the corresponding > > stale entry in the RCU list never matches. If the memory range gets > > allocated again, then it happens only after a RCU quiescent state. > > > > Since bounce buffers can now be allocated from different pools, add a > > parameter to swiotlb_alloc_pool() to let the caller know which memory pool > > is used. Add swiotlb_find_pool() to find the memory pool corresponding to > > an address. This function is now also used by is_swiotlb_buffer(), because > > a simple boundary check is no longer sufficient. > > > > The logic in swiotlb_alloc_tlb() is taken from __dma_direct_alloc_pages(), > > simplified and enhanced to use coherent memory pools if needed. > > > > Note that this is not the most efficient way to provide a bounce buffer, > > but when a DMA buffer can't be mapped, something may (and will) actually > > break. At that point it is better to make an allocation, even if it may be > > an expensive operation. > > I continue to think about swiotlb memory management from the standpoint > of CoCo VMs that may be quite large with high network and storage loads. > These VMs are often running mission-critical workloads that can't tolerate > a bounce buffer allocation failure. To prevent such failures, the swiotlb > memory size must be overly large, which wastes memory. If "mission critical workloads" are in a vm that allowes overcommit and no control over other vms in that same system, then you have worse problems, sorry. Just don't do that. thanks, greg k-h _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel