From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19F06C7EE29 for ; Mon, 29 May 2023 06:11:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229815AbjE2GL3 (ORCPT ); Mon, 29 May 2023 02:11:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229604AbjE2GL1 (ORCPT ); Mon, 29 May 2023 02:11:27 -0400 Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com [IPv6:2607:f8b0:4864:20::b31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11AFBB1 for ; Sun, 28 May 2023 23:11:25 -0700 (PDT) Received: by mail-yb1-xb31.google.com with SMTP id 3f1490d57ef6-bacfb7acdb7so4576678276.0 for ; Sun, 28 May 2023 23:11:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685340684; x=1687932684; h=mime-version:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to; bh=zliJFPKkBRA/n2GaYWcI8flBsJ+sn1iKXV6efEOvVuc=; b=7B1W5MLwR/lNmx3BIWIEBSexKnAAtAeI5OObdohvQtJ0ya6QvwHlgYFcOox2PoWx79 AqTZ5kdBzWLNorOsK4QCVE297PN18hQKAQbCdVGnbvFaS7kyYm3WQLVA/nvW5TUXBwEK caAcy7rQi84u8SNtdxTLyllY4LVN73jaN501s4k727QQrqblWsE0z78z1nbHIMjpBe5J U7bsOrnIMfDvYrfJ38H4/OhgD5cA+Y2quHyXE8xkEJQ1/qi+xCB9+P6Rf/V3USkqhcRr ADhdVtWOZGV+AvVYyI9ulfCMHPe9IScFN/bL7d58cmN+M8/9WGgQpLLoAa3PpgIRJ6V+ lp1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685340684; x=1687932684; h=mime-version:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zliJFPKkBRA/n2GaYWcI8flBsJ+sn1iKXV6efEOvVuc=; b=BfteKBvN3G4zV+xrxKrV5u3eGUGzyn+t3S/a3EEHOi/b/jvV+PBIio/mJH/kgxC3Ek /3MbotwTMAvSLwkQ19hV7dEh1vkU/qQWUYkrwn+6YRl5xdbWEvmxm5MQTMZ4l6j0phnd btH+/NCToeJb1NYc0mVEFBbb2TkwPKFEdTPSXKmS45D/iZvvt12DHmYPDrIWCdT1Noet PXfZqbsT+ljL/+dikOo4pLoZAMgqGPW0NamcqOl8MgZdoI+BtGoi8LUpc+ExHvMW1iEp yZDPB7kC5KWJ95q6yHpkTL6chD4DJr4pbz1VQdMGzJmNYQm7BSW6k43bhQ6Hjyl0yQ+x Wufw== X-Gm-Message-State: AC+VfDwE97IiKS50CK+qExPeQhm0KztFv0zsN/TQSv1Im6Iu1NfqHZwa BhpbUHX8xIRY6ogrMYx4O5shYg== X-Google-Smtp-Source: ACHHUZ6vifKmHFoYdv3qrFkDElnMJbREaNq58bFiSjvre/dDskro6b5jD1pQPpN+OFNUHzxvXn3Qhw== X-Received: by 2002:a25:d4b:0:b0:ba8:6530:d561 with SMTP id 72-20020a250d4b000000b00ba86530d561mr10306665ybn.30.1685340684077; Sun, 28 May 2023 23:11:24 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id r63-20020a252b42000000b00b7b0aba5cccsm2703954ybr.22.2023.05.28.23.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 May 2023 23:11:23 -0700 (PDT) Date: Sun, 28 May 2023 23:11:07 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David S. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Claudio Imbrenda , Alexander Gordeev , Jann Horn , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 00/12] mm: free retracted page table by RCU Message-ID: <35e983f5-7ed3-b310-d949-9ae8b130cdab@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Precedence: bulk List-ID: X-Mailing-List: linux-s390@vger.kernel.org Here is the third series of patches to mm (and a few architectures), based on v6.4-rc3 with the preceding two series applied: in which khugepaged takes advantage of pte_offset_map[_lock]() allowing for pmd transitions. This follows on from the "arch: allow pte_offset_map[_lock]() to fail" https://lore.kernel.org/linux-mm/77a5d8c-406b-7068-4f17-23b7ac53bc83@google.com/ series of 23 posted on 2023-05-09, and the "mm: allow pte_offset_map[_lock]() to fail" https://lore.kernel.org/linux-mm/68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com/ series of 31 posted on 2023-05-21. Those two series were "independent": neither depending for build or correctness on the other, but both series needed before this third one can safely make the effective changes. I'll send v2 of those two series in a couple of days, incorporating Acks and Revieweds and the minor fixes. What is it all about? Some mmap_lock avoidance i.e. latency reduction. Initially just for the case of collapsing shmem or file pages to THPs: the usefulness of MADV_COLLAPSE on shmem is being limited by that mmap_write_lock it currently requires. Likely to be relied upon later in other contexts e.g. freeing of empty page tables (but that's not work I'm doing). mmap_write_lock avoidance when collapsing to anon THPs? Perhaps, but again that's not work I've done: a quick attempt was not as easy as the shmem/file case. These changes (though of course not these exact patches) have been in Google's data centre kernel for three years now: we do rely upon them. Based on the preceding two series over v6.4-rc3, but good over v6.4-rc[1-4], current mm-everything or current linux-next. 01/12 mm/pgtable: add rcu_read_lock() and rcu_read_unlock()s 02/12 mm/pgtable: add PAE safety to __pte_offset_map() 03/12 arm: adjust_pte() use pte_offset_map_nolock() 04/12 powerpc: assert_pte_locked() use pte_offset_map_nolock() 05/12 powerpc: add pte_free_defer() for pgtables sharing page 06/12 sparc: add pte_free_defer() for pgtables sharing page 07/12 s390: add pte_free_defer(), with use of mmdrop_async() 08/12 mm/pgtable: add pte_free_defer() for pgtable as page 09/12 mm/khugepaged: retract_page_tables() without mmap or vma lock 10/12 mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock() 11/12 mm/khugepaged: delete khugepaged_collapse_pte_mapped_thps() 12/12 mm: delete mmap_write_trylock() and vma_try_start_write() arch/arm/mm/fault-armv.c | 3 +- arch/powerpc/include/asm/pgalloc.h | 4 + arch/powerpc/mm/pgtable-frag.c | 18 ++ arch/powerpc/mm/pgtable.c | 16 +- arch/s390/include/asm/pgalloc.h | 4 + arch/s390/mm/pgalloc.c | 34 +++ arch/sparc/include/asm/pgalloc_64.h | 4 + arch/sparc/mm/init_64.c | 16 ++ include/linux/mm.h | 17 -- include/linux/mm_types.h | 2 +- include/linux/mmap_lock.h | 10 - include/linux/pgtable.h | 6 +- include/linux/sched/mm.h | 1 + kernel/fork.c | 2 +- mm/khugepaged.c | 425 ++++++++---------------------- mm/pgtable-generic.c | 44 +++- 16 files changed, 253 insertions(+), 353 deletions(-) Hugh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AED5DC7EE23 for ; Mon, 29 May 2023 06:12:21 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4QV4tq67nJz3fFH for ; Mon, 29 May 2023 16:12:19 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20221208 header.b=7B1W5MLw; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=google.com (client-ip=2607:f8b0:4864:20::b34; helo=mail-yb1-xb34.google.com; envelope-from=hughd@google.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20221208 header.b=7B1W5MLw; dkim-atps=neutral Received: from mail-yb1-xb34.google.com (mail-yb1-xb34.google.com [IPv6:2607:f8b0:4864:20::b34]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4QV4sr6NMJz3f3k for ; Mon, 29 May 2023 16:11:27 +1000 (AEST) Received: by mail-yb1-xb34.google.com with SMTP id 3f1490d57ef6-bacf5b89da7so4531576276.2 for ; Sun, 28 May 2023 23:11:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685340684; x=1687932684; h=mime-version:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to; bh=zliJFPKkBRA/n2GaYWcI8flBsJ+sn1iKXV6efEOvVuc=; b=7B1W5MLwR/lNmx3BIWIEBSexKnAAtAeI5OObdohvQtJ0ya6QvwHlgYFcOox2PoWx79 AqTZ5kdBzWLNorOsK4QCVE297PN18hQKAQbCdVGnbvFaS7kyYm3WQLVA/nvW5TUXBwEK caAcy7rQi84u8SNtdxTLyllY4LVN73jaN501s4k727QQrqblWsE0z78z1nbHIMjpBe5J U7bsOrnIMfDvYrfJ38H4/OhgD5cA+Y2quHyXE8xkEJQ1/qi+xCB9+P6Rf/V3USkqhcRr ADhdVtWOZGV+AvVYyI9ulfCMHPe9IScFN/bL7d58cmN+M8/9WGgQpLLoAa3PpgIRJ6V+ lp1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685340684; x=1687932684; h=mime-version:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zliJFPKkBRA/n2GaYWcI8flBsJ+sn1iKXV6efEOvVuc=; b=VDDO/1smORDCgd7GvHnOQxRQPqysOtokG4xq1w5JGNxphapgqGO823Uv1WP+sQHhgm vn00gLLSae8cbUG6gjWQNrrcS36UHYFBesrzl8x8LGXSGC1P8HnoFhggwTZfU+jFHY2e 7+BFQe5oOW4OdxvL4lUvc/cYdZeK8xZ/x1gGfJCe2krEwi7GnbTkYlx19sB6FeUVu+An 9rPVMMeA2zGi7UxFFFkXTjF9l1fghKDm6EWGlwIqC2Q2lH9Vpd943ncQr7RRMb/kM74D FjN0E/OJf3ALLkQ6ckmXhXJlNX9oBkcs6Y6Kz0YwNggSzS4KlVIuTxLBhPjJZx9hHqWB sWcg== X-Gm-Message-State: AC+VfDyehp7ieLhnxELBHmmcfSnq5F7yR6XJeZaVThXXQIz1+lUCBGd/ d+/l5KpoQoHXRNf9aHioN5kbXA== X-Google-Smtp-Source: ACHHUZ6vifKmHFoYdv3qrFkDElnMJbREaNq58bFiSjvre/dDskro6b5jD1pQPpN+OFNUHzxvXn3Qhw== X-Received: by 2002:a25:d4b:0:b0:ba8:6530:d561 with SMTP id 72-20020a250d4b000000b00ba86530d561mr10306665ybn.30.1685340684077; Sun, 28 May 2023 23:11:24 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id r63-20020a252b42000000b00b7b0aba5cccsm2703954ybr.22.2023.05.28.23.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 May 2023 23:11:23 -0700 (PDT) Date: Sun, 28 May 2023 23:11:07 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton Subject: [PATCH 00/12] mm: free retracted page table by RCU Message-ID: <35e983f5-7ed3-b310-d949-9ae8b130cdab@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Miaohe Lin , David Hildenbrand , Peter Zijlstra , Yang Shi , Peter Xu , Song Liu , sparclinux@vger.kernel.org, Alexander Gordeev , Claudio Imbrenda , Will Deacon , linux-s390@vger.kernel.org, Yu Zhao , Ira Weiny , Alistair Popple , Russell King , Matthew Wilcox , Steven Price , Christoph Hellwig , Jason Gunthorpe , "Aneesh Kumar K.V" , Axel Rasmussen , Christian Borntraeger , Thomas Hellstrom , Ralph Campbell , Pasha Tatashin , Anshuman Khandual , Heiko Ca rstens , Qi Zheng , Suren Baghdasaryan , linux-arm-kernel@lists.infradead.org, SeongJae Park , Jann Horn , linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, "Kirill A. Shutemov" , Naoya Horiguchi , linux-kernel@vger.kernel.org, Minchan Kim , Mike Rapoport , Mel Gorman , "David S. Miller" , Zack Rusin , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Here is the third series of patches to mm (and a few architectures), based on v6.4-rc3 with the preceding two series applied: in which khugepaged takes advantage of pte_offset_map[_lock]() allowing for pmd transitions. This follows on from the "arch: allow pte_offset_map[_lock]() to fail" https://lore.kernel.org/linux-mm/77a5d8c-406b-7068-4f17-23b7ac53bc83@google.com/ series of 23 posted on 2023-05-09, and the "mm: allow pte_offset_map[_lock]() to fail" https://lore.kernel.org/linux-mm/68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com/ series of 31 posted on 2023-05-21. Those two series were "independent": neither depending for build or correctness on the other, but both series needed before this third one can safely make the effective changes. I'll send v2 of those two series in a couple of days, incorporating Acks and Revieweds and the minor fixes. What is it all about? Some mmap_lock avoidance i.e. latency reduction. Initially just for the case of collapsing shmem or file pages to THPs: the usefulness of MADV_COLLAPSE on shmem is being limited by that mmap_write_lock it currently requires. Likely to be relied upon later in other contexts e.g. freeing of empty page tables (but that's not work I'm doing). mmap_write_lock avoidance when collapsing to anon THPs? Perhaps, but again that's not work I've done: a quick attempt was not as easy as the shmem/file case. These changes (though of course not these exact patches) have been in Google's data centre kernel for three years now: we do rely upon them. Based on the preceding two series over v6.4-rc3, but good over v6.4-rc[1-4], current mm-everything or current linux-next. 01/12 mm/pgtable: add rcu_read_lock() and rcu_read_unlock()s 02/12 mm/pgtable: add PAE safety to __pte_offset_map() 03/12 arm: adjust_pte() use pte_offset_map_nolock() 04/12 powerpc: assert_pte_locked() use pte_offset_map_nolock() 05/12 powerpc: add pte_free_defer() for pgtables sharing page 06/12 sparc: add pte_free_defer() for pgtables sharing page 07/12 s390: add pte_free_defer(), with use of mmdrop_async() 08/12 mm/pgtable: add pte_free_defer() for pgtable as page 09/12 mm/khugepaged: retract_page_tables() without mmap or vma lock 10/12 mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock() 11/12 mm/khugepaged: delete khugepaged_collapse_pte_mapped_thps() 12/12 mm: delete mmap_write_trylock() and vma_try_start_write() arch/arm/mm/fault-armv.c | 3 +- arch/powerpc/include/asm/pgalloc.h | 4 + arch/powerpc/mm/pgtable-frag.c | 18 ++ arch/powerpc/mm/pgtable.c | 16 +- arch/s390/include/asm/pgalloc.h | 4 + arch/s390/mm/pgalloc.c | 34 +++ arch/sparc/include/asm/pgalloc_64.h | 4 + arch/sparc/mm/init_64.c | 16 ++ include/linux/mm.h | 17 -- include/linux/mm_types.h | 2 +- include/linux/mmap_lock.h | 10 - include/linux/pgtable.h | 6 +- include/linux/sched/mm.h | 1 + kernel/fork.c | 2 +- mm/khugepaged.c | 425 ++++++++---------------------- mm/pgtable-generic.c | 44 +++- 16 files changed, 253 insertions(+), 353 deletions(-) Hugh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 565FFC77B7A for ; Mon, 29 May 2023 06:12:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Subject:cc:To: From:Date:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=2hxrDyOoqlC6EzXMQwLsgdV37ewqU77WeHTNw3nN7wo=; b=3SfXs1SGzYSO/P CUGKcHPNDMAU3hs6MTByTrXXuSANkpoJPBEtAtqklEWC1aAJiHNq/QHiovuyXmlo6JOS4Q/ofABZi If1SD8xwrWE/jMFkeERLAOfqn+l/IAaBUjsT2QQPkRuY3uGizVWHQqEiHoaiFMmwjLtOVo1LXqQqD GCVHnKbILCFVzsHh/YK77FCG4D8DRW81j/6mhvhYaO7MPCHU0wT0QCpKHXcdftQLXSDTt738N97UO s12wQLqpozYeytGXp3OA0lbXv6/yq8/NUOcTon78NXUQ+vAM7nXtKTKi8zkc/VdGjN/6+SPNoGD6d Qpn3V1QefneaC1bteDeQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q3W6V-009K7V-1T; Mon, 29 May 2023 06:11:39 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q3W6S-009K73-1s for linux-arm-kernel@bombadil.infradead.org; Mon, 29 May 2023 06:11:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:Message-ID: Subject:cc:To:From:Date:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:In-Reply-To:References; bh=zliJFPKkBRA/n2GaYWcI8flBsJ+sn1iKXV6efEOvVuc=; b=Hc9CwilOQdGWKH/6WpDFjxMocw SCGFBMPguZEmTxnzV4cEbbHH7ns9xemfVlTkqqY0RrD2wa+NSMZkuXOfMEnC3ysV2rydx0n3DgXvT hcaeaQuZC4fxJPwOtH5XuY/PStzdO8+vlwPRQhmeh9AVX2eFaNTPk6euF6lYCe1nxdTwdGSazi9yy HcyDlK8dSOpymrLbyBsm0eNxltVwRcxBR5X6SCkgQbaWo5Be02kqCswHBCVP4qHhEgx+2GWpmdlB3 AGjrWYQEOtLQcZpbChXWFOdkfTwg/YV3Dw1mYo7RZR7YxTxxt3JdtHMy0xV+JmRxb1+/bl+Yqq9Cg Hb7W6Z7Q==; Received: from mail-yb1-xb2e.google.com ([2607:f8b0:4864:20::b2e]) by desiato.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q3W6O-00BnKY-1B for linux-arm-kernel@lists.infradead.org; Mon, 29 May 2023 06:11:35 +0000 Received: by mail-yb1-xb2e.google.com with SMTP id 3f1490d57ef6-ba81ded8d3eso4527892276.3 for ; Sun, 28 May 2023 23:11:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685340684; x=1687932684; h=mime-version:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to; bh=zliJFPKkBRA/n2GaYWcI8flBsJ+sn1iKXV6efEOvVuc=; b=7B1W5MLwR/lNmx3BIWIEBSexKnAAtAeI5OObdohvQtJ0ya6QvwHlgYFcOox2PoWx79 AqTZ5kdBzWLNorOsK4QCVE297PN18hQKAQbCdVGnbvFaS7kyYm3WQLVA/nvW5TUXBwEK caAcy7rQi84u8SNtdxTLyllY4LVN73jaN501s4k727QQrqblWsE0z78z1nbHIMjpBe5J U7bsOrnIMfDvYrfJ38H4/OhgD5cA+Y2quHyXE8xkEJQ1/qi+xCB9+P6Rf/V3USkqhcRr ADhdVtWOZGV+AvVYyI9ulfCMHPe9IScFN/bL7d58cmN+M8/9WGgQpLLoAa3PpgIRJ6V+ lp1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685340684; x=1687932684; h=mime-version:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=zliJFPKkBRA/n2GaYWcI8flBsJ+sn1iKXV6efEOvVuc=; b=eAoVFK/3RetP9nH9xLpe+QRxVrkWg75Ky+nKtlHAhQk81rLYtoJ45yhlUmEngkg2Md ObTK/YpvgTqsg19vDuXzvFyCrTPZbi71eFBONZilx75mbNXyApZl4qSmumvuIgECrSTu UfURs8G10Q6m7RdV/glB1Om4r+wM5PuQAWVlM60mOhRWlvfkiDYKb8yXjGizi4u0VtnM dxuccLuD6JFL20SH6RtksW16JdkEnMixSy2jn6GIQlHWkX3UgKvdlUPkKhY1DQN8b7m6 bUXj89NQ3lAwO27H69e8VkOwNl1b+uWUS7ZPj363lvyqwBgqTwOdJaZ7isw2r8c4HzwT 8onw== X-Gm-Message-State: AC+VfDw1S0ZhmZ+QRmijejrRerpVx8N/9GqsCdidQ1wABw/GzVzWxEi9 3LgnP3YNr6DKN9fZ6XnZMz1etA== X-Google-Smtp-Source: ACHHUZ6vifKmHFoYdv3qrFkDElnMJbREaNq58bFiSjvre/dDskro6b5jD1pQPpN+OFNUHzxvXn3Qhw== X-Received: by 2002:a25:d4b:0:b0:ba8:6530:d561 with SMTP id 72-20020a250d4b000000b00ba86530d561mr10306665ybn.30.1685340684077; Sun, 28 May 2023 23:11:24 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id r63-20020a252b42000000b00b7b0aba5cccsm2703954ybr.22.2023.05.28.23.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 May 2023 23:11:23 -0700 (PDT) Date: Sun, 28 May 2023 23:11:07 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David S. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Claudio Imbrenda , Alexander Gordeev , Jann Horn , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 00/12] mm: free retracted page table by RCU Message-ID: <35e983f5-7ed3-b310-d949-9ae8b130cdab@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230529_071132_852051_3930752B X-CRM114-Status: GOOD ( 17.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Here is the third series of patches to mm (and a few architectures), based on v6.4-rc3 with the preceding two series applied: in which khugepaged takes advantage of pte_offset_map[_lock]() allowing for pmd transitions. This follows on from the "arch: allow pte_offset_map[_lock]() to fail" https://lore.kernel.org/linux-mm/77a5d8c-406b-7068-4f17-23b7ac53bc83@google.com/ series of 23 posted on 2023-05-09, and the "mm: allow pte_offset_map[_lock]() to fail" https://lore.kernel.org/linux-mm/68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com/ series of 31 posted on 2023-05-21. Those two series were "independent": neither depending for build or correctness on the other, but both series needed before this third one can safely make the effective changes. I'll send v2 of those two series in a couple of days, incorporating Acks and Revieweds and the minor fixes. What is it all about? Some mmap_lock avoidance i.e. latency reduction. Initially just for the case of collapsing shmem or file pages to THPs: the usefulness of MADV_COLLAPSE on shmem is being limited by that mmap_write_lock it currently requires. Likely to be relied upon later in other contexts e.g. freeing of empty page tables (but that's not work I'm doing). mmap_write_lock avoidance when collapsing to anon THPs? Perhaps, but again that's not work I've done: a quick attempt was not as easy as the shmem/file case. These changes (though of course not these exact patches) have been in Google's data centre kernel for three years now: we do rely upon them. Based on the preceding two series over v6.4-rc3, but good over v6.4-rc[1-4], current mm-everything or current linux-next. 01/12 mm/pgtable: add rcu_read_lock() and rcu_read_unlock()s 02/12 mm/pgtable: add PAE safety to __pte_offset_map() 03/12 arm: adjust_pte() use pte_offset_map_nolock() 04/12 powerpc: assert_pte_locked() use pte_offset_map_nolock() 05/12 powerpc: add pte_free_defer() for pgtables sharing page 06/12 sparc: add pte_free_defer() for pgtables sharing page 07/12 s390: add pte_free_defer(), with use of mmdrop_async() 08/12 mm/pgtable: add pte_free_defer() for pgtable as page 09/12 mm/khugepaged: retract_page_tables() without mmap or vma lock 10/12 mm/khugepaged: collapse_pte_mapped_thp() with mmap_read_lock() 11/12 mm/khugepaged: delete khugepaged_collapse_pte_mapped_thps() 12/12 mm: delete mmap_write_trylock() and vma_try_start_write() arch/arm/mm/fault-armv.c | 3 +- arch/powerpc/include/asm/pgalloc.h | 4 + arch/powerpc/mm/pgtable-frag.c | 18 ++ arch/powerpc/mm/pgtable.c | 16 +- arch/s390/include/asm/pgalloc.h | 4 + arch/s390/mm/pgalloc.c | 34 +++ arch/sparc/include/asm/pgalloc_64.h | 4 + arch/sparc/mm/init_64.c | 16 ++ include/linux/mm.h | 17 -- include/linux/mm_types.h | 2 +- include/linux/mmap_lock.h | 10 - include/linux/pgtable.h | 6 +- include/linux/sched/mm.h | 1 + kernel/fork.c | 2 +- mm/khugepaged.c | 425 ++++++++---------------------- mm/pgtable-generic.c | 44 +++- 16 files changed, 253 insertions(+), 353 deletions(-) Hugh _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel