From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D5C6C32751 for ; Wed, 31 Jul 2019 06:44:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E2C8A2089E for ; Wed, 31 Jul 2019 06:44:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E2C8A2089E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.ibm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9A7338E0003; Wed, 31 Jul 2019 02:44:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 958068E0001; Wed, 31 Jul 2019 02:44:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81FCB8E0003; Wed, 31 Jul 2019 02:44:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f69.google.com (mail-yw1-f69.google.com [209.85.161.69]) by kanga.kvack.org (Postfix) with ESMTP id 610318E0001 for ; Wed, 31 Jul 2019 02:44:18 -0400 (EDT) Received: by mail-yw1-f69.google.com with SMTP id b75so49580220ywh.8 for ; Tue, 30 Jul 2019 23:44:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:date:from:to :cc:subject:references:mime-version:content-disposition:in-reply-to :user-agent:message-id; bh=eJpTqhjq32xxR611Iu+PnjDy/EPpQxCVR5srDH+Bu+g=; b=MzzRYbMOurDu2x3ebGR/etwrXkUtl2PcEq9DwG5G8Uj2gn5uhdsP5C7gpoXuDUl/6i 7Zfl8Kh2TSNtHyzPb3b1nzF/rkGGheIY1i3nw3RXosivkPc+jSo6KkrOrdZ2vTtsqXz6 0JXNJDb8QdLfYArZHqqcBRC9YEg+03fXJbLzVTCtszGDncs9ppoekVcOILGkYZYFI50X IIkSmxqhC/FX/IQdKEBh5WjnYcg9LF3lqsTVGLArKg7Dl9bTuesaYmFUDw/kEig/Pb6d 3Orlp0C3uk9XoTJF7B+HANYCnnaJtZdwo0K1YMilRavpgan1e6/jqfDsYbDC4ux8KFfE oWzg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rppt@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=rppt@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com X-Gm-Message-State: APjAAAWbdveRgtqQf4u5YyQ6WQPTPlg8IdvvAl0uJ5lSCxkQsrToYpes CAJ+gYbNCH/yhB9Vz9kRh7Z07KlcGvV/uL+ws3JlPJpXPt0o2I3QwkBWHxTH9rSa5I1gqOxLHav 8XsZSeoLvJF9ZFcCejMOZAAmzU3cr7n92sUIh3wWvxOb2VuBXFEacY9G/MZn51n9nrQ== X-Received: by 2002:a25:ba07:: with SMTP id t7mr14725707ybg.261.1564555458091; Tue, 30 Jul 2019 23:44:18 -0700 (PDT) X-Google-Smtp-Source: APXvYqz5WlBLDqGGa/ywe0JPjQTeNr3UD5oGh2fttm4yNpRFxvzd4OAZQjTZUuVpgBebX33gb3OM X-Received: by 2002:a25:ba07:: with SMTP id t7mr14725699ybg.261.1564555457512; Tue, 30 Jul 2019 23:44:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564555457; cv=none; d=google.com; s=arc-20160816; b=j6NM0qX0hY6py7ya5LmUafGHL8Bf9Jcwig5P6QJ7b9220AbLWmMui7m8W3o/0xI/yB ElwkPnXt5tkV2WG5HYNawa2HK/6eeng1Z7pziAwyxa6SG8H/sXNUO3sByYbNKkBZw3xb 0pPHc+TN05YvDRj4jJ0waGVKr29nbK61GYdHwkUW2T5qFTOu2c0uGprF9pjY7c7aKi8Q /68F6C/KpewPam4GbBX7ymEJJGiSyAsnnfpRVnmuIdvtxg38Vhh+bgMFj2yNcdt0vH2p guFE+xaRQ4oXVLqBCMEUWGz64OzVwERDl5TVlqJx8nODiEMpkX3k3MHRBIjARDylm918 VPJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:user-agent:in-reply-to:content-disposition:mime-version :references:subject:cc:to:from:date; bh=eJpTqhjq32xxR611Iu+PnjDy/EPpQxCVR5srDH+Bu+g=; b=NLIqYyr42T59ASr+nUBwbY4zDC7kyMSrbZO+9fmdd98MaRPwBg5xrcxlpVPOJ9h/bV Bx875ve5u+kldacLn5I2BF6CmEU2UbVKQounP3a2qn+5wssUqA8YVk1facw5qAf5lNGe ivFesUnWo3DBvRgOunz7nC54XFVkqDLv69zowx0dzQ2u5vVCDGTkD7HQMYkxc6+WTWrw d4dzzQEyI+nHwe7srd5bCIypnWJqm9UkceLJOi9ImAdKi3r23k8C5Sk0Cw9CzawNQPA5 CInL8QClkL/+eoFtDT8aQ7nDfW1V8zOhhHTWlRANYMbcAhWBN9nERx5oK1iR+Js+SDJo 1qzg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rppt@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=rppt@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com. [148.163.158.5]) by mx.google.com with ESMTPS id 63si1378923ywd.389.2019.07.30.23.44.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Jul 2019 23:44:17 -0700 (PDT) Received-SPF: pass (google.com: domain of rppt@linux.ibm.com designates 148.163.158.5 as permitted sender) client-ip=148.163.158.5; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rppt@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=rppt@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x6V6gWF5142464 for ; Wed, 31 Jul 2019 02:44:17 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0b-001b2d01.pphosted.com with ESMTP id 2u34xhaf34-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 31 Jul 2019 02:44:16 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 31 Jul 2019 07:44:14 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 31 Jul 2019 07:44:07 +0100 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x6V6i6kP42270910 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 Jul 2019 06:44:06 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 389A64C04E; Wed, 31 Jul 2019 06:44:06 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5E4614C04A; Wed, 31 Jul 2019 06:44:04 +0000 (GMT) Received: from rapoport-lnx (unknown [9.148.8.168]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Wed, 31 Jul 2019 06:44:04 +0000 (GMT) Date: Wed, 31 Jul 2019 09:44:02 +0300 From: Mike Rapoport To: "Joel Fernandes (Google)" Cc: linux-kernel@vger.kernel.org, Alexey Dobriyan , Andrew Morton , Brendan Gregg , Christian Hansen , dancol@google.com, fmayer@google.com, joaodias@google.com, joelaf@google.com, Jonathan Corbet , Kees Cook , kernel-team@android.com, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Michal Hocko , minchan@kernel.org, namhyung@google.com, Roman Gushchin , Stephen Rothwell , surenb@google.com, tkjos@google.com, Vladimir Davydov , Vlastimil Babka , wvw@google.com Subject: Re: [PATCH v3 2/2] doc: Update documentation for page_idle virtual address indexing References: <20190726152319.134152-1-joel@joelfernandes.org> <20190726152319.134152-2-joel@joelfernandes.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190726152319.134152-2-joel@joelfernandes.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-TM-AS-GCONF: 00 x-cbid: 19073106-0016-0000-0000-00000297D60A X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19073106-0017-0000-0000-000032F5E768 Message-Id: <20190731064400.GD21422@rapoport-lnx> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-07-31_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907310070 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jul 26, 2019 at 11:23:19AM -0400, Joel Fernandes (Google) wrote: > This patch updates the documentation with the new page_idle tracking > feature which uses virtual address indexing. > > Signed-off-by: Joel Fernandes (Google) One nit below, otherwise Reviewed-by: Mike Rapoport > --- > .../admin-guide/mm/idle_page_tracking.rst | 43 ++++++++++++++++--- > 1 file changed, 36 insertions(+), 7 deletions(-) > > diff --git a/Documentation/admin-guide/mm/idle_page_tracking.rst b/Documentation/admin-guide/mm/idle_page_tracking.rst > index df9394fb39c2..1eeac78c94a7 100644 > --- a/Documentation/admin-guide/mm/idle_page_tracking.rst > +++ b/Documentation/admin-guide/mm/idle_page_tracking.rst > @@ -19,10 +19,14 @@ It is enabled by CONFIG_IDLE_PAGE_TRACKING=y. > > User API > ======== > +There are 2 ways to access the idle page tracking API. One uses physical > +address indexing, another uses a simpler virtual address indexing scheme. > > -The idle page tracking API is located at ``/sys/kernel/mm/page_idle``. > -Currently, it consists of the only read-write file, > -``/sys/kernel/mm/page_idle/bitmap``. > +Physical address indexing > +------------------------- > +The idle page tracking API for physical address indexing using page frame > +numbers (PFN) is located at ``/sys/kernel/mm/page_idle``. Currently, it > +consists of the only read-write file, ``/sys/kernel/mm/page_idle/bitmap``. > > The file implements a bitmap where each bit corresponds to a memory page. The > bitmap is represented by an array of 8-byte integers, and the page at PFN #i is > @@ -74,6 +78,31 @@ See :ref:`Documentation/admin-guide/mm/pagemap.rst ` for more > information about ``/proc/pid/pagemap``, ``/proc/kpageflags``, and > ``/proc/kpagecgroup``. > > +Virtual address indexing > +------------------------ > +The idle page tracking API for virtual address indexing using virtual page > +frame numbers (VFN) is located at ``/proc//page_idle``. It is a bitmap > +that follows the same semantics as ``/sys/kernel/mm/page_idle/bitmap`` > +except that it uses virtual instead of physical frame numbers. Can you please make it more explicit that VFNs are in the 's address space? > + > +This idle page tracking API does not need deal with PFN so it does not require > +prior lookups of ``pagemap`` in order to find if page is idle or not. This is > +an advantage on some systems where looking up PFN is considered a security > +issue. Also in some cases, this interface could be slightly more reliable to > +use than physical address indexing, since in physical address indexing, address > +space changes can occur between reading the ``pagemap`` and reading the > +``bitmap``, while in virtual address indexing, the process's ``mmap_sem`` is > +held for the duration of the access. > + > +To estimate the amount of pages that are not used by a workload one should: > + > + 1. Mark all the workload's pages as idle by setting corresponding bits in > + ``/proc//page_idle``. > + > + 2. Wait until the workload accesses its working set. > + > + 3. Read ``/proc//page_idle`` and count the number of bits set. > + > .. _impl_details: > > Implementation Details > @@ -99,10 +128,10 @@ When a dirty page is written to swap or disk as a result of memory reclaim or > exceeding the dirty memory limit, it is not marked referenced. > > The idle memory tracking feature adds a new page flag, the Idle flag. This flag > -is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` (see the > -:ref:`User API ` > -section), and cleared automatically whenever a page is referenced as defined > -above. > +is set manually, by writing to ``/sys/kernel/mm/page_idle/bitmap`` for physical > +addressing or by writing to ``/proc//page_idle`` for virtual > +addressing (see the :ref:`User API ` section), and cleared > +automatically whenever a page is referenced as defined above. > > When a page is marked idle, the Accessed bit must be cleared in all PTEs it is > mapped to, otherwise we will not be able to detect accesses to the page coming > -- > 2.22.0.709.g102302147b-goog > -- Sincerely yours, Mike.