From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F03EC433F5 for ; Sat, 21 May 2022 16:07:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C91EA6B0072; Sat, 21 May 2022 12:07:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C1B306B0073; Sat, 21 May 2022 12:07:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A475D6B0074; Sat, 21 May 2022 12:07:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 941126B0072 for ; Sat, 21 May 2022 12:07:37 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6EC4F7C2 for ; Sat, 21 May 2022 16:07:37 +0000 (UTC) X-FDA: 79490230554.11.A35A3DA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf14.hostedemail.com (Postfix) with ESMTP id A14FB10001C for ; Sat, 21 May 2022 16:07:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653149256; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zEmZQdHTHnyukv03PYK4ijYW3MmF2DErRa+vYpL0aV4=; b=eh9xdWXpmqeIjvDD8bePaWflXsLrbzlXVa6zB+IcOOIYkm3YDQ8knE1ox6cxE2ZS6KCA+s MOCkIuwrxOXi9SVERfYAXP9iu0GMcWMNZ+0XxHKz0eMgJOhnYbi3DwFiiv8Q+PYJ5BKrpe alca+NaESpw/0ES1zz0eov4R2GeBY2I= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-282-ja_s0RN5Pb2aacOXIbcGfA-1; Sat, 21 May 2022 12:07:33 -0400 X-MC-Unique: ja_s0RN5Pb2aacOXIbcGfA-1 Received: by mail-ed1-f69.google.com with SMTP id l18-20020aa7d952000000b0042ab7be9adaso7572796eds.21 for ; Sat, 21 May 2022 09:07:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=zEmZQdHTHnyukv03PYK4ijYW3MmF2DErRa+vYpL0aV4=; b=L4GTyBqbYcga+7uMVJC7G00AuhZzkcELP4g/1vCIsLMeryuCTnXSLy7lcJtp8I2oLC yE9K9GasKCGLlufNajRtIG7jYouNkRYV89ZaWQAYES+NbyCgKBqzd/IxvTC0I9S4iMUd ykwbyTMefL6zRwZcj54k3J6N9WdPyAFwiyHhDOvbp+s3x65ZV3yJ/NuX3Agca5alcri3 57B8GhpL1YKxcOUYJpoDgwA4NxmkxhiUnMahZKSEK/LYRiWwV879In6wir8lVV2i0Qeq ciQL0WVKKjD8YglRtdSrSJSlIfZN6JDH9CIGhQGAEzWApLrW01PpeUS4PBvB81MHlVg+ h7Sg== X-Gm-Message-State: AOAM530eSZlwG2RBbuly1rhQe6bQgFr1FSGiWFOo6K1XuEDnH3jf0auN CThsr0jfR+4+XaUWFKfgqwX50sxxMUj70a6A1CLsq9/EitapexUKvFyrZr3ZHS5iVu5c8G766+p ZDe+kbC+ziyw= X-Received: by 2002:a05:6402:524d:b0:42a:f9e9:8013 with SMTP id t13-20020a056402524d00b0042af9e98013mr15080669edd.65.1653149252576; Sat, 21 May 2022 09:07:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxdpjMVJmJlPFOlsiFKQDHC2Jzo57iB/6ZxKyq3bvzfZFtw58NeDUiz3GtqrSeSSEi/eB0F+g== X-Received: by 2002:a05:6402:524d:b0:42a:f9e9:8013 with SMTP id t13-20020a056402524d00b0042af9e98013mr15080623edd.65.1653149252328; Sat, 21 May 2022 09:07:32 -0700 (PDT) Received: from [172.29.4.249] ([45.90.93.190]) by smtp.gmail.com with ESMTPSA id r22-20020a170906351600b006f3ef214ddcsm407359eja.66.2022.05.21.09.07.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 21 May 2022 09:07:31 -0700 (PDT) Message-ID: Date: Sat, 21 May 2022 18:07:27 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 To: Chih-En Lin , Andrew Morton , linux-mm@kvack.org Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Christian Brauner , "Matthew Wilcox (Oracle)" , Vlastimil Babka , William Kucharski , John Hubbard , Yunsheng Lin , Arnd Bergmann , Suren Baghdasaryan , Colin Cross , Feng Tang , "Eric W. Biederman" , Mike Rapoport , Geert Uytterhoeven , Anshuman Khandual , "Aneesh Kumar K.V" , Daniel Axtens , Jonathan Marek , Christophe Leroy , Pasha Tatashin , Peter Xu , Andrea Arcangeli , Thomas Gleixner , Andy Lutomirski , Sebastian Andrzej Siewior , Fenghua Yu , linux-kernel@vger.kernel.org, Kaiyang Zhao , Huichun Feng , Jim Huang References: <20220519183127.3909598-1-shiyn.lin@gmail.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [RFC PATCH 0/6] Introduce Copy-On-Write to Page Table In-Reply-To: <20220519183127.3909598-1-shiyn.lin@gmail.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: A14FB10001C X-Stat-Signature: ppcpfpagowuaycmnr6qhshqugde747od Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eh9xdWXp; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf14.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1653149253-48070 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 19.05.22 20:31, Chih-En Lin wrote: > When creating the user process, it usually uses the Copy-On-Write (COW) > mechanism to save the memory usage and the cost of time for copying. > COW defers the work of copying private memory and shares it across the > processes as read-only. If either process wants to write in these > memories, it will page fault and copy the shared memory, so the process > will now get its private memory right here, which is called break COW. Yes. Lately we've been dealing with advanced COW+GUP pinnings (which resulted in PageAnonExclusive, which should hit upstream soon), and hearing about COW of page tables (and wondering how it will interact with the mapcount, refcount, PageAnonExclusive of anonymous pages) makes me feel a bit uneasy :) > > Presently this kind of technology is only used as the mapping memory. > It still needs to copy the entire page table from the parent. > It might cost a lot of time and memory to copy each page table when the > parent already has a lot of page tables allocated. For example, here is > the state table for mapping the 1 GB memory of forking. > > mmap before fork mmap after fork > MemTotal: 32746776 kB 32746776 kB > MemFree: 31468152 kB 31463244 kB > AnonPages: 1073836 kB 1073628 kB > Mapped: 39520 kB 39992 kB > PageTables: 3356 kB 5432 kB I'm missing the most important point: why do we care and why should we care to make our COW/fork implementation even more complicated? Yes, we might save some page tables and we might reduce the fork() time, however, which specific workload really benefits from this and why do we really care about that workload? Without even hearing about an example user in this cover letter (unless I missed it), I naturally wonder about relevance in practice. I assume it really only matters if we fork() realtively large processes, like databases for snapshotting. However, fork() is already a pretty sever performance hit due to COW, and there are alternatives getting developed as a replacement for such use cases (e.g., uffd-wp). I'm also missing a performance evaluation: I'd expect some simple workloads that use fork() might be even slower after fork() with this change. (I don't have time to read the paper, I'd expect an independent summary in the cover letter) I have tons of questions regarding rmap, accounting, GUP, page table walkers, OOM situations in page walkers, but at this point I am not (yet) convinced that the added complexity is really worth it. So I'd appreciate some additional information. [...] > TODO list: > - Handle the swap Scary if that's not easy to handle :/ > - Rewrite the TLB flush for zapping the COW PTE table. > - Experiment COW to the entire page table. (Now just for PTE level) > - Bug in some case from copy_pte_range()::vm_normal_page()::print_bad_pte(). > - Bug of Bad RSS counter in multiple times COW PTE table forking. -- Thanks, David / dhildenb