From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF6DCC433DB for ; Fri, 19 Mar 2021 00:49:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AAF3664E83 for ; Fri, 19 Mar 2021 00:49:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231349AbhCSAso (ORCPT ); Thu, 18 Mar 2021 20:48:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232221AbhCSAsU (ORCPT ); Thu, 18 Mar 2021 20:48:20 -0400 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC889C06174A; Thu, 18 Mar 2021 17:48:18 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id f17so2144236plr.0; Thu, 18 Mar 2021 17:48:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=eG1QzwIi3xEaakEPyWiwcIUMucRPcW0kIWxB+VdX0Ok=; b=RAJG3mFP99nhvmv/KZWxUCdFpGEmKA3rijSW/KwwIvPUb9UZQ6Jg0EyQveb7G4h9fU pkKdfJGMdcIm4HI8uAVUa5Nzv5Zg4BsnDy2d+JmuFQzST8gNpib1b6rpiGNtvsMPhPOC LuJikCi/pTxL5zf9qwXIeLEFzpgbaTtIx5lksikP75lA4DjNhVOHDtGblLuffgfQNSmG iftKfGmNegPkTPItGtRXgUJVFUVTSrMS/FWSdPOvaAyUNDzyCrYvqQgRk1G/AUC4p7UC x53i7x8hpHXAlhD+e1Wa/QaFLzV3EUGQP9EqVnhWX4iPJvyM6NevwDVaL7Kw5/iotfQp ujjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=eG1QzwIi3xEaakEPyWiwcIUMucRPcW0kIWxB+VdX0Ok=; b=BhJ732lROpyQxDoBghK/D82NDhtqDRrXRLXxQSGR4PwDMdvEEIJCspVpYA4yiRKvwG RLWst+8RxItwIybhs5f9X3/sfm+ujcrQcCpZdties2QhWbnWlVYHSndA+beCaBd+FY+L andrKEjOerN33OxiLfGPsGZT8MFHbRd8Sxo3ok6nuYJoKutelQuc4bb0jaejlVxeyQYp z8zYEsv4+8F9VMfyqoZBELocXr4MON6xB4qPFaGq1uOK2KACIqu8Qeix1cQ3qCSDN0OE ziceKYCrJ2xgq06MUnrOyh58Hgu580/D7fSSbRRDCYlA9QUgD3fHQjXTGvua6aoI79M4 7YGA== X-Gm-Message-State: AOAM530ooVV2YCKoSJWJjoMwK6nEXqbSn78euKr0PzLpd965NEMYMOrQ /mHsBXmK/Xv7wulY8uSC16w= X-Google-Smtp-Source: ABdhPJytY1xkTdXpw1JPFJu5Rw5Dhn60wo+Djui6kIuuXt0aDtxR6iyqkWGn1AA/gx0cUCg8L8D8GA== X-Received: by 2002:a17:902:bf92:b029:e6:bc0:25ac with SMTP id v18-20020a170902bf92b02900e60bc025acmr12452727pls.49.1616114898109; Thu, 18 Mar 2021 17:48:18 -0700 (PDT) Received: from [10.230.29.202] ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id g2sm3388274pfi.28.2021.03.18.17.48.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 18 Mar 2021 17:48:17 -0700 (PDT) Subject: Re: [PATCH] swiotlb: Add swiotlb=off to disable SWIOTLB To: Robin Murphy , linux-kernel@vger.kernel.org, Christoph Hellwig Cc: Jonathan Corbet , opendmb@gmail.com, "Paul E. McKenney" , Konrad Rzeszutek Wilk , Mauro Carvalho Chehab , Viresh Kumar , Randy Dunlap , "open list:DOCUMENTATION" , Peter Zijlstra , "open list:SWIOTLB SUBSYSTEM" , Andrew Morton , Mike Kravetz , Thomas Gleixner References: <20210318191816.4185226-1-f.fainelli@gmail.com> <16d1c66f-5451-2515-af73-a6b44d996e92@arm.com> <3dd81519-4a73-efb8-abf0-0b766f993a8b@arm.com> From: Florian Fainelli Message-ID: Date: Thu, 18 Mar 2021 17:48:12 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <3dd81519-4a73-efb8-abf0-0b766f993a8b@arm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On 3/18/2021 4:35 PM, Robin Murphy wrote: > On 2021-03-18 21:31, Florian Fainelli wrote: >> >> >> On 3/18/2021 12:53 PM, Robin Murphy wrote: >>> On 2021-03-18 19:43, Florian Fainelli wrote: >>>> >>>> >>>> On 3/18/2021 12:34 PM, Robin Murphy wrote: >>>>> On 2021-03-18 19:22, Florian Fainelli wrote: >>>>>> >>>>>> >>>>>> On 3/18/2021 12:18 PM, Florian Fainelli wrote: >>>>>>> It may be useful to disable the SWIOTLB completely for testing or >>>>>>> when a >>>>>>> platform is known not to have any DRAM addressing limitations >>>>>>> what so >>>>>>> ever. >>>>> >>>>> Isn't that what "swiotlb=noforce" is for? If you're confident that >>>>> we've >>>>> really ironed out *all* the awkward corners that used to blow up if >>>>> various internal bits were left uninitialised, then it would make >>>>> sense >>>>> to just tweak the implementation of what we already have. >>>> >>>> swiotlb=noforce does prevent dma_direct_map_page() from resorting to >>>> the >>>> swiotlb, however what I am also after is reclaiming these 64MB of >>>> default SWIOTLB bounce buffering memory because my systems run with >>>> large amounts of reserved memory into ZONE_MOVABLE and everything in >>>> ZONE_NORMAL is precious at that point. >>> >>> It also forces io_tlb_nslabs to the minimum, so it should be claiming >>> considerably less than 64MB. IIRC the original proposal *did* skip >>> initialisation completely, but that turned up the aforementioned issues. >> >> AFAICT in that case we will have iotlb_n_slabs will set to 1, which will >> still make us allocate io_tlb_n_slabs << IO_TLB_SHIFT bytes in >> swiotlb_init(), which still gives us 64MB. > > Eh? When did 2KB become 64MB? IO_TLB_SHIFT is 11, so that's at most one > page in anyone's money... Yes, sorry incorrect shift applied here. Still, and I believe this is what you mean below, architecture code setting swiotlb_force = SWIOTLB_NO_FORCE does not result in not allocating the SWIOTLB, because io_tlb_nslabs is still left set to 0 so swiotlb_init() will proceed with allocating the default size. > >>>>> I wouldn't necessarily disagree with adding "off" as an additional >>>>> alias >>>>> for "noforce", though, since it does come across as a bit wacky for >>>>> general use. >>>>> >>>>>>> Signed-off-by: Florian Fainelli >>>>>> >>>>>> Christoph, in addition to this change, how would you feel if we >>>>>> qualified the swiotlb_init() in arch/arm/mm/init.c with a: >>>>>> >>>>>> >>>>>> if (memblock_end_of_DRAM() >= SZ_4G) >>>>>>       swiotlb_init(1) >>>>> >>>>> Modulo "swiotlb=force", of course ;) >>>> >>>> Indeed, we would need to handle that case as well. Does it sound >>>> reasonable to do that to you as well? >>> >>> I wouldn't like it done to me personally, but for arm64, observe what >>> mem_init() in arch/arm64/mm/init.c already does. > > In fact I should have looked more closely at that myself - checking > debugfs on my 4GB arm64 board actually shows io_tlb_nslabs = 0, and > indeed we are bypassing initialisation completely and (ab)using > SWIOTLB_NO_FORCE to cover it up, so I guess it probably *is* safe now > for the noforce option to do the same for itself and save even that one > page. OK, I can submit a patch that does that. 5.12-rc3 works correctly for me here as well and only allocates SWIOTLB when needed which in our case is either: - we have DRAM at PA >= 4GB - we have limited peripherals (Raspberry Pi 4 derivative) that can only address the lower 1GB Now let's see if we can get ARM 32-bit to match :) -- Florian