From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C487FC433F5 for ; Tue, 12 Oct 2021 01:20:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4D2A61040 for ; Tue, 12 Oct 2021 01:20:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235215AbhJLBWi (ORCPT ); Mon, 11 Oct 2021 21:22:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50624 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234595AbhJLBWh (ORCPT ); Mon, 11 Oct 2021 21:22:37 -0400 Received: from mail-yb1-xb34.google.com (mail-yb1-xb34.google.com [IPv6:2607:f8b0:4864:20::b34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CB91C061570 for ; Mon, 11 Oct 2021 18:20:37 -0700 (PDT) Received: by mail-yb1-xb34.google.com with SMTP id s4so42993083ybs.8 for ; Mon, 11 Oct 2021 18:20:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=T0rvH8tccG1GCW1xVscvfSQVrt0Y/dmYu+9N0LqR3IU=; b=ozWNgkyPQ6g2Pb6GhDQaCx4ZxPKDrBt5s2ggV9Ns3aDuMRwgN/pb8qCCc75VBqb19f OghPvT9NZ3DJ8gXsGqPKYt6+SQFN/ApQF6XXqcBQDxiC5rgqd0w20vWV7y/3n/HBX73A mYHYuTI/yuzdz6aA8LJWPIynztB+pp8z1VmlcJSARvWWwH3sjX/rD1nXx/GWDMfJbRiY onfCMl0VsmdBL4rZnW+ek3I+1+Sk62K28tZE0iSXgGlhh5683azjY2imwUSYEYfhUaqE 8T9k1TImgSZOiUTnlqCMBWhndlrEIqEIpm5qjv40lClFReap2zuQBi62mOwgtVEqrVGk z2Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=T0rvH8tccG1GCW1xVscvfSQVrt0Y/dmYu+9N0LqR3IU=; b=d7uwcsIBUttcU89xZD8dKAoVBHoqEPgp8/WrMLqhBHO+EaejQ7+s4bqTVVL12R1CRq WowY/n6Rvb5fZSD5O4wzIqt8odyL/Bgd6tSUYc7MzS24uXNBbiY3wuWGDg9sD/UdlLGd 08O1fRTKLJIay9xaO0oNAbJvmGUztE7vsEbCh6b6j6+FqJP51PEZ5K6gTpsG/cWt6WAn kEQDDInVvN1T2gzpVMkOOaPoZaKMA1MqR+HMISNrUmcLDFKEXeY84wlvvkD2lBi6qWS2 9y+sXcftGIelDhwU0QYeoVs00vjMPeSPTTdnQa7hGk7N+iYUNob2Uohz8brnCIwwK22z 1rmQ== X-Gm-Message-State: AOAM533PS1LFOggyJeMU2SHpcfIze/yPfPbIUwAuZDVdwhA5YY738n6e qTKkjOAMt0mlmeF/BSAugc3TpHXXGaBvADC9VpvntA== X-Google-Smtp-Source: ABdhPJyHvt4aySXDLkg2z+6x0kGcvsx1rVoezM+PTBjUy0+IBRYK/uij8JJWNi9p7n7FF7CAOZG+U+VrGupgEfmgznE= X-Received: by 2002:a25:8411:: with SMTP id u17mr25321543ybk.376.1634001636162; Mon, 11 Oct 2021 18:20:36 -0700 (PDT) MIME-Version: 1.0 References: <92cbfe3b-f3d1-a8e1-7eb9-bab735e782f6@rasmusvillemoes.dk> <20211007101527.GA26288@duo.ucw.cz> <202110071111.DF87B4EE3@keescook> <202110081344.FE6A7A82@keescook> In-Reply-To: From: Suren Baghdasaryan Date: Mon, 11 Oct 2021 18:20:25 -0700 Message-ID: Subject: Re: [PATCH v10 3/3] mm: add anonymous vma name refcounting To: Michal Hocko Cc: Kees Cook , Pavel Machek , Rasmus Villemoes , David Hildenbrand , John Hubbard , Andrew Morton , Colin Cross , Sumit Semwal , Dave Hansen , Matthew Wilcox , "Kirill A . Shutemov" , Vlastimil Babka , Johannes Weiner , Jonathan Corbet , Al Viro , Randy Dunlap , Kalesh Singh , Peter Xu , rppt@kernel.org, Peter Zijlstra , Catalin Marinas , vincenzo.frascino@arm.com, =?UTF-8?B?Q2hpbndlbiBDaGFuZyAo5by16Yym5paHKQ==?= , Axel Rasmussen , Andrea Arcangeli , Jann Horn , apopple@nvidia.com, Yu Zhao , Will Deacon , fenghua.yu@intel.com, thunder.leizhen@huawei.com, Hugh Dickins , feng.tang@intel.com, Jason Gunthorpe , Roman Gushchin , Thomas Gleixner , krisman@collabora.com, Chris Hyser , Peter Collingbourne , "Eric W. Biederman" , Jens Axboe , legion@kernel.org, Rolf Eike Beer , Cyrill Gorcunov , Muchun Song , Viresh Kumar , Thomas Cedeno , sashal@kernel.org, cxfcosmos@gmail.com, LKML , linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm , kernel-team , Tim Murray Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Mon, Oct 11, 2021 at 6:18 PM Suren Baghdasaryan wrote: > > On Mon, Oct 11, 2021 at 1:36 AM Michal Hocko wrote: > > > > On Fri 08-10-21 13:58:01, Kees Cook wrote: > > > - Strings for "anon" specifically have no required format (this is good) > > > it's informational like the task_struct::comm and can (roughly) > > > anything. There's no naming convention for memfds, AF_UNIX, etc. Why > > > is one needed here? That seems like a completely unreasonable > > > requirement. > > > > I might be misreading the justification for the feature. Patch 2 is > > talking about tools that need to understand memeory usage to make > > further actions. Also Suren was suggesting "numbering convetion" as an > > argument against. > > > > So can we get a clear example how is this being used actually? If this > > is just to be used to debug by humans than I can see an argument for > > human readable form. If this is, however, meant to be used by tools to > > make some actions then the argument for strings is much weaker. > > The simplest usecase is when we notice that a process consumes more > memory than usual and we do "cat /proc/$(pidof my_process)/maps" to > check which area is contributing to this growth. The names we assign > to anonymous areas are descriptive enough for a developer to get an > idea where the increased consumption is coming from and how to proceed > with their investigation. > There are of course cases when tools are involved, but the end-user is > always a human and the final report should contain easily > understandable data. > > IIUC, the main argument here is whether the userspace can provide > tools to perform the translations between ids and names, with the > kernel accepting and reporting ids instead of strings. Technically > it's possible, but to be practical that conversion should be fast > because we will need to make name->id conversion potentially for each > mmap. On the consumer side the performance is not as critical, but the > fact that instead of dumping /proc/$pid/maps we will have to parse the > file, do id->name conversion and replace all [anon:id] with > [anon:name] would be an issue when we do that in bulk, for example > when collecting system-wide data for a bugreport. > > I went ahead and implemented the proposed userspace solution involving > tmpfs as a repository for name->id mapping (more precisely > filename->inode mapping). Profiling shows that open()+fstat()+close() > takes: > - roughly 15 times longer than mmap() with 1000 unique names each > being reused 50 times. > - roughly 3 times longer than mmap() with 100 unique names each being > reused 500 times. This is due to lstat() optimization suggested by > Rasmus which avoids open() and close(). > For comparison, proposed prctl() takes roughly the same amount of time > as mmap() and does not depend on the number of unique names. > > I'm still evaluating the proposal to use memfds but I'm not sure if > the issue that David Hildenbrand mentioned about additional memory > consumed in pagecache (which has to be addressed) is the only one we > will encounter with this approach. If anyone knows of any potential > issues with using memfds as named anonymous memory, I would really > appreciate your feedback before I go too far in that direction. > Thanks, > Suren. Just noticed that timmurray@ was dropped from the last reply. Adding him back. > > > -- > > Michal Hocko > > SUSE Labs