From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E11AC4167B for ; Sun, 3 Dec 2023 05:43:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D58426B0295; Sun, 3 Dec 2023 00:43:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D085C6B0367; Sun, 3 Dec 2023 00:43:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BCFEE6B036C; Sun, 3 Dec 2023 00:43:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A962D6B0295 for ; Sun, 3 Dec 2023 00:43:21 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6D3EF1201CB for ; Sun, 3 Dec 2023 05:43:21 +0000 (UTC) X-FDA: 81524414202.03.FA5BBB6 Received: from mail-oa1-f53.google.com (mail-oa1-f53.google.com [209.85.160.53]) by imf03.hostedemail.com (Postfix) with ESMTP id 8A62D20014 for ; Sun, 3 Dec 2023 05:43:19 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WFdlhzgC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of cuiyangpei@gmail.com designates 209.85.160.53 as permitted sender) smtp.mailfrom=cuiyangpei@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701582199; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ulpex2/K0nFLvwaganrs1elk8GQQ5l+5GEHyn5YwooQ=; b=ZdOk21xWHvwiHXgr++LOe1sOwj3cyHTeXXRTOeuzyBOb2tDvVsQ9Sl6auEQaxbBPC2XAL0 ILYQb41MfKYvlnZf/A5dn/OgvvN24OOCoZyko/JDzuMi7hmTGKqXHGxsyTv8YlyFWW06FG RhO9sd1S3uommZWdVp5GW4thxKV9R+8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WFdlhzgC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of cuiyangpei@gmail.com designates 209.85.160.53 as permitted sender) smtp.mailfrom=cuiyangpei@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701582199; a=rsa-sha256; cv=none; b=s/QQG6jJNPkL9kdcRStpMMs3qyHCQr7xJwmfqHqxVrqMBMUwve69LB3tKGi4BbULlgW4p7 u252PrbhgrMKRX5RF9b1g9/Be6HxjT6KWx66JaSwWIP4PivV6PF5cZ6ByFpyQE2401wxLc GuOLujFmuwEPWuRsh+9ycQUTa25eR1k= Received: by mail-oa1-f53.google.com with SMTP id 586e51a60fabf-1fb04b2251bso1067494fac.0 for ; Sat, 02 Dec 2023 21:43:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701582198; x=1702186998; darn=kvack.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=Ulpex2/K0nFLvwaganrs1elk8GQQ5l+5GEHyn5YwooQ=; b=WFdlhzgCPsNFlwxGFj7RNxM+2GnKmyalHxYHWvrXSSwESJYAB9e1qoZ7P+ZfosR7tP fuChQuGPY5WDJWIJQiNT+aLHIe4bxnHhGlZSOtUGPP3+551rEJW7VInYsmH+AxtZxz1f dleQopsj09AjNtxDII1YfSoRiEzqhVErqpeXDrLhLYPyjfHPf1PyChh3GG+Tv4veESYx GutnUC63K6VBV+kbm206MZRL3gV2suCswvyYbVH/OkMGBwQDPeDUa5EBTFAPPMHKDpk7 vPAxbMrjhECgeZrsGrIDEa3rihKBOghfVTinQK9lGicxg0fvELPGcxfx31srqPymKHis Z41A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701582198; x=1702186998; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ulpex2/K0nFLvwaganrs1elk8GQQ5l+5GEHyn5YwooQ=; b=BH50KzUZb0bBsNriiEvzdIyUBPBEwfy/wnZK9UPxYTEzI8KRkJDpW5wrXJywW1hG2r SPTc6peInZIIafgK0ejWAxb3UD9AGfP8G6ZQe4BLKViB7HWKCARePdM8CdhBAf6ZUNn0 38uDo/fzF8v+02+qjurHRz6+o69Ga03A0WylBFShG6JHRyGOW0885irz7GpDxkmrfTb0 vEG+cLyBo+zojXA0tH/tzpanTJyozMsWLsouvYc73tdrZtKkS+Yy86wI3k3RMlrub2Zp +KET0SoEjaOOYlUIv5NHvMKbhe/Z4bz4eQIZABmHNiE9IJ5jAdiyPyaleKoAdZKm106M /T4Q== X-Gm-Message-State: AOJu0YxvMN0iVobgR5q+92xXNeOd7FC0q2w7ee9niCAxP5uCwDy1s8Zs csJbqayW8Rz98LL+JkRyXAE= X-Google-Smtp-Source: AGHT+IHw+PS2DOyoNngsHLoHfwVNHqBw9yoPg6sTyYXO+vualSGM3APgi04B0s49HgFnLD/jC6UXmg== X-Received: by 2002:a05:6870:d24a:b0:1fb:22f4:c49d with SMTP id h10-20020a056870d24a00b001fb22f4c49dmr1342317oac.23.1701582198569; Sat, 02 Dec 2023 21:43:18 -0800 (PST) Received: from cuiyangpei ([43.224.245.227]) by smtp.gmail.com with ESMTPSA id a1-20020a656041000000b005891f3af36asm4872025pgp.87.2023.12.02.21.43.16 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 02 Dec 2023 21:43:18 -0800 (PST) Date: Sun, 3 Dec 2023 13:43:13 +0800 From: cuiyangpei To: SeongJae Park , akpm@linux-foundation.org, damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: xiongping1@xiaomi.com Subject: Re: [PATCH 1/2] mm/damon/sysfs: Implement recording feature Message-ID: <20231203054313.GA14316@cuiyangpei> References: <20231201122507.GA15806@cuiyangpei> <20231201173112.64300-1-sj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231201173112.64300-1-sj@kernel.org> User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: 8A62D20014 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: jxf1w1osryz38epfdebhzztptgfrowdf X-HE-Tag: 1701582199-80330 X-HE-Meta: U2FsdGVkX1+VidY41XC5o7QM+EPC+4muphQiwB9uOS9C90RW7QEGYQFHbJ8J2ONaIn2xopdFpoWYEbIo+xvRFyPM1yOVnHnpVzZaoj7nwvKAj1moQbEF1lYrWgzpsBDHeF8oVCJaM1PzriC1ZJyQO83RGjJbjSK54U2O+gTfNjsQyZ7RMtqy5T/o/Rstsgb2b4rEV5teUMlP9yIR6mJIJAT6qxo85VdVrkcDC9oqYLPPu2jBGrWckYx2k1Kr8vE3n5Jdc5zltoLJf2HoWmo8hrT3JAbIlZdr5As9GqfhXcXhTWRifKe+1r0+WLt74j4Ch+Nx3YmXoRaWZv+7o97COU3luygXp3hOphqhnIudNqSpM2Frd+0msBjzOu9h4NEtGPQaX9/KVb+zOZM0i0ub7/+6wR8LNLo/W4NrQ0kAc+3lGzrS4cvfTXBKV4MbaJu0ZYrlkiiBWrM3wM49SkxZenEx/thzYsuOLzY4ErGwoXENPerAxa4cEHCkLneqopxC+30VEY2dqP4khUiTaMz1fS4GPOv1YTlw+XwUn826oJ2OQ9L8KZPryiNLmtaMrnAkC5edCI4eEmr+EucwfHzyCxQCffQ/w076wHa1WaRkU5Ou4KjxX422l6mhx6vtLiBcNBAsGbZfxIIhD6Qm3SlbFf93Il5J7/n9GwgsetiRqQ5W7uqWfjvJL87fla+d2Hpg3R4dbWGeqeHAUDkd15qXycbE6Bts+31DtsUHrrfu1yquCXyzjVpfry9T7owxpjQcOKiH5CIZf9shYgZUt/k77GkCpiD89/Osqw3U9lzC+kOCQH5VJGGEM8Lxf5aFtQ/B7xSswArVfDSOvG8616udgLAk3revkyuoJJqnYJgDVlOxhzlYQQXKMRIYHldf5+P/Biep1A9wcaX6xeL6N8DSrd9hWFaA3VVoCtEjXlHyOOnFbdlowKJ3ZAzknEb/nLU/1AmxiE6HPBLUS2NJexm Sa4Vk47Z PZ/2IuDHG6b+uiStED0FLFlVHeYhOQWc5TruYJo5ghrhWagay2kBMKwnuJ7FodZowJ/ae6UEqEuNdb+2v1ejtTcPZs8LrhU0OMXYK5+pGVh3NTJcuqIlbifgpYJ8XZu99Au63z1FYRCRxQGn/pi0HVk+abHNLKDeW7SJcp6RufQ/TYz+Z8mxaA9zxO8HATuvbpGeCCwxDJz4a2XbyrA5uYuff66ibFYaPcwiGoeCbpV4Gwk5Ydf8AcHaXv4dTbMlXuREOrN6N7FHZj7XcTLz3/mthaW5SbPFNfs1Pl800NY9tTQr2pcA1b6iviLk1wZBDuOuDo+nDK6Ey1/6u3LQ6EFFXOVPN85WRThqkhiFmR0N7R4thruguFWe1ZgEQQestf2r5/5tbf5qITtfjZPB3EyLWl/bWha4cnfTfrr6mFau2G0vb33jJliPpA5UaLMafG0d/CgChvulic/f5Mrq72VXWFYK3doBRXNpIACxKaya5/5nQnaU4nfFh6feo3j3N9Ac68FsFrtqptT0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.025157, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Dec 01, 2023 at 05:31:12PM +0000, SeongJae Park wrote: > Hi Cuiyangpei, > > On Fri, 1 Dec 2023 20:25:07 +0800 cuiyangpei wrote: > > > On Thu, Nov 30, 2023 at 07:44:20PM +0000, SeongJae Park wrote: > > > Hi Cuiyangpei, > > > > > > On Thu, 30 Nov 2023 17:14:26 +0800 cuiyangpei wrote: > > > > > > > Hi SeongJae, > > > > > > > > We also investigated the operation schemes you mentioned, but we don't > > > > think it can fit our needs. > > > > > > > > On android, user will open many apps and switch between these apps as > > > > needs. We hope to monitor apps' memory access only when they are on > > > > foreground and record the memory access pattern when they are switched > > > > to the background. > > > > > > > > When avaliable memory reaches a threshold, we will use these access > > > > patterns with some strategies to recognize those memory that will have > > > > little impact on user experience and to reclaim them proactively. > > > > > > > > I'm not sure I have clarified it clearly, if you still have questions > > > > on this, please let us know. > > > > > > So, to my understanding, you expect applications may keep similar access > > > pattern when they are in foreground, but have a different, less aggressive > > > access pattern in background, and therefore reclaim memory based on the > > > foreground-access pattern, right? > > > > > > > Different apps may have different access pattern. On android, the apps will > > join in freeze cgroup and be frozen after switch to the background. So we > > monitor apps' memory access only when they are in foreground. > > Thank you for this enlightening me :) > > > > Very interesting idea, thank you for sharing! > > > > > > Then, yes, I agree current DAMOS might not that helpful for the situation, and > > > this record feature could be useful for your case. > > > > > > That said, do you really need full recording of the monitoring results? If > > > not, DAMOS provides DAMOS tried regions feature[1], which allows users get the > > > monitoring results snapshot that include both frequency and recency of all > > > regions in an efficient way. If single snapshot is not having enough > > > information for you, you could collect multiple snapshots. > > > > > > You mentioned absence of Python on Android as a blocker of DAMOS use on the > > > previous reply[2], but DAMOS tried regions feature is not depend on tracepoints > > > or Python. > > > > > > Of course, I think you might already surveyed it but found some problems. > > > Could you please share that in detail if so? > > > > > DAMOS tried regions feature you mentioned is not fully applicable. It needs to > > apply schemes on regions. There is no available scheme we can use for our use > > case. What we need is to return regions with access frequency and recency to > > userspace for later use. > > > Thank you for the answer, I understand your concern. One of the available > DAMOS action is 'stat'[1], which does nothing but just count the statistic. > Using DAMOS scheme for any access pattern with 'stat' action, you can extract > the access pattern via DAMOS tried regions feature of DAMON sysfs interface, > without making any unnecessary impact to the workload. > > Quote from [2]: > > The expected usage of this directory is investigations of schemes' behaviors, > and query-like efficient data access monitoring results retrievals. For the > latter use case, in particular, users can set the action as stat and set the > access pattern as their interested pattern that they want to query. > > For example, you could > > # cd /sys/kernel/mm/damon/admin > # > # # populate directories > # echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts; > # echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes > # cd kdamonds/0/contexts/0/schemes/0 > # > # # set the access pattern for any case (max as 2**64 - 1), and action as stat > # echo 0 > access_pattern/sz/min > # echo 18446744073709551615 > access_pattern/sz/max > # echo 0 > access_pattern/nr_accesses/min > # echo 18446744073709551615 > access_pattern/nr_accesses/max > # echo 0 > access_pattern/age/min > # echo 18446744073709551615 > access_pattern/age/max > # echo stat > action > > And this is how DAMON user-space tool is getting the snapshot with 'damo show' > command[3]. > > Could this be used for your case? Please ask any question if you have :) > > [1] https://docs.kernel.org/admin-guide/mm/damon/usage.html#schemes-n > [2] https://docs.kernel.org/admin-guide/mm/damon/usage.html#schemes-n-tried-regions, > [3] https://github.com/awslabs/damo/blob/next/USAGE.md#damo-show Thank you for your detailed response, it is very helpful to us. We will look into it and contact you if we have any questions. > > > Thanks, > SJ > > > > [1] https://docs.kernel.org/admin-guide/mm/damon/usage.html#schemes-n-tried-regions > > > [2] https://lore.kernel.org/damon/20231129131315.GB12957@cuiyangpei/ > > > > > > > > > Thanks, > > > SJ > > > > > > > > > > > Thanks.