From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3411AEB64DD for ; Thu, 20 Jul 2023 22:31:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79B4128016A; Thu, 20 Jul 2023 18:31:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 74AF028004C; Thu, 20 Jul 2023 18:31:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6138B28016A; Thu, 20 Jul 2023 18:31:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 51DF928004C for ; Thu, 20 Jul 2023 18:31:08 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 16D0414027B for ; Thu, 20 Jul 2023 22:31:08 +0000 (UTC) X-FDA: 81033437016.04.A6A501B Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf06.hostedemail.com (Postfix) with ESMTP id B5BE618001F for ; Thu, 20 Jul 2023 22:31:05 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=ghf2BigS; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=pass (imf06.hostedemail.com: domain of htejun@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=htejun@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689892265; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NZpl8s75HuZGEhNzUvLR3L4qywqf/Lpjo/SjREf5tA4=; b=DmKklf5123GEVxl9yCsQFD+lxHsbvktPa40Uzni+5dVWcpDyZLDAjszxJSbHIJY6+iHlYW xiz8A9N3A2h4pm0kDGYq+q2dCw5OSLmu9MLwygHys6poXiLgMBGruEWNFS6onNNq7ExYzk XP70gXM89j9p6PyUbM93yA/1xyfDk2o= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=ghf2BigS; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=pass (imf06.hostedemail.com: domain of htejun@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=htejun@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689892265; a=rsa-sha256; cv=none; b=jCx05MPa7M5oSQIxZPpTv6LKqXYZHm0b9rTEDEG8jk3UsbwKAwU0w2ivPMELTw5b40GwFH BEokXR9tD5WN8pf3QkUcYGVIC/Lrg4oBAygcOxUsTBnFXS4pmdYn0fjX8P4BF/yFCzEGGg rfcQAfRkuvyTIGg4ufzAaA/u8+Lgp1A= Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-6726d5d92afso1686156b3a.1 for ; Thu, 20 Jul 2023 15:31:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689892264; x=1690497064; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=NZpl8s75HuZGEhNzUvLR3L4qywqf/Lpjo/SjREf5tA4=; b=ghf2BigSLopuzIAO9mo5bilaWUW4pw/MFFRLacsY7gWLDlVj9giiAM8MRIJR4yUUN1 0eZXp+66XIpghEylgvGqAISiRJtWYiAeKFk7WqWG9YoONhzF7W8uockvwN6JHX9P2ZkX 1WQhxBW7HjKnkQF7XhjgGOYKP2fXw2EG0F8J+r2UjwCm20cngLNEKgvq2w+1/xMghmP4 0F1mD2djp/qp+HRvQPIcmx4EbINiffoW4s2adiKXZS/QnQC/F3zsDi5AZw11mkZQfaLw JFzkRMQNS3a5BSHUM/Gm5XZF6bkZeN/kuIDV0MNaB/JLMC1CDfRMDUaljWa1nfZfBNCU Dtbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689892264; x=1690497064; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NZpl8s75HuZGEhNzUvLR3L4qywqf/Lpjo/SjREf5tA4=; b=V+KNytOLJQ1aYKu2+Mi9xZJS89UXimGpdkDEbFF48uhaWS026CxWu1OwclzQAJ8YJE 6yR0cmxO1OiYI8e6dodsXpp97vLt6zBiGU2rVu/Q27R/EwIyRrYsaW1KxRIisBrb5ypH y4wuBZg+oB570nOeVC3d9W9H/p1F92UNWES16oSqky0xFIigmkI+aKOReediSPDpj20+ vbbcUYl1UsU6QOsDXEm6lqcJYx2srhv9ZiqO0rTXkQt5LL24q68FddUuLbabvYmSiLnU R6ruorib/bjvJ6Q0wTRjeDKnkZyqm3epVF3znSRXwbzub+Vhf2H8Bdy7P1akhRVsTpF7 CwiQ== X-Gm-Message-State: ABy/qLYNntY4aSE/+S4n/RpVtVu1smpzoLq7JH7Jqb6OOb/IXEFAHYgI PmeUodgO9KVc08/NGDDWPKQ= X-Google-Smtp-Source: APBJJlEzg0Yil3Q92R9mP6SC1Y2AeH6CTu+qTw9rhX9Zts56WnHzBb54bFsTGCD2r51MF7MGt0yucQ== X-Received: by 2002:a05:6a20:8401:b0:132:7d91:aadb with SMTP id c1-20020a056a20840100b001327d91aadbmr381237pzd.6.1689892264054; Thu, 20 Jul 2023 15:31:04 -0700 (PDT) Received: from localhost ([2620:10d:c090:400::5:fbd8]) by smtp.gmail.com with ESMTPSA id d20-20020aa78154000000b00682a8e600f0sm1684817pfn.35.2023.07.20.15.31.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jul 2023 15:31:03 -0700 (PDT) Date: Thu, 20 Jul 2023 12:31:02 -1000 From: Tejun Heo To: Yosry Ahmed Cc: Johannes Weiner , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , "Matthew Wilcox (Oracle)" , Zefan Li , Yu Zhao , Luis Chamberlain , Kees Cook , Iurii Zaikin , "T.J. Mercier" , Greg Thelen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org Subject: Re: [RFC PATCH 0/8] memory recharging for offline memcgs Message-ID: References: <20230720070825.992023-1-yosryahmed@google.com> <20230720153515.GA1003248@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: B5BE618001F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 5p1pzywfafakewgr6j6nc4e7hazf7j5u X-HE-Tag: 1689892265-840086 X-HE-Meta: U2FsdGVkX18QiidTODk/inmT4DMZfEhPww7aImu/B3QiJgYIY26y0+C5othekLSzgDyNSbmRo1KSQKZshUPGnpDkdY8KTJys6TQbLnG7ZbNfaHNpOvr45RDVUla4uF3bwsGDoW7eEt6gU22f4iLc3g2zvIU76HG4l1rBDFKc/m0Jg6R8FEiaEDH5oF5ELU2GPj3uiM3xlKDj9KbXIyRGCaGRR3jQmQq02qXgTtq0T30S3aMplPXS87B4JkblgAUb47PwH91QXWrvD5loEsafJ3oIeRQNthvPxio5jzD7vt+yi0AhetFZ/UbduExTBJxTtgMT5WujpqBqdhozxCTz6io4/vXLfdRg6Paso4everZJFSmCQ6MBHUqYuS1qr7UsK5eO2xDkpNsdSku9K87qrbOpyDkrQNv3629HFtjSfM94FXs3C0o45ubXkE8sEBfIxCl9Z/EdlUgZByliy7yrmHLKWkb7wqSl2LIPyQ+Iu5/xsGzl2poR3E3ivcUOYhf8HfwDzLzFk+Fa7d80QGj1xjxT8pRyceIBU0HVMCoAv1YksSDb9XvhNLOYsjZuMxGZpn7YBdLSZKpMfZ5v3h0OxfttfxdLCjFdd9X4eZFc+d8slLzB4jndrJ9mgs43A35lCcn44g71aeaKt5LeA8wmMKmcISRkxt0ndeOWdz1QL4XDxGFF8qKfB1K5Tzy/cXneo8xWPMHWmTXhkJH4kFN26fHFtf21PJw3AKFE8Pd3zCxxHPUOGr+UITGvB/LwxfO0HySB53zWR/ZYQCkwneyQvdgpyOub/05txh7CH+BmF4T9uNiK0uo1oZ85dKEfIEE8nTsT/Zf82Jc0n0l1ctaqatNi965dP2SeVhR/xRN1CatZos38vnRStjanhU9bBtu8wVCC57MGLa24hNXzdlajKCqqtOV17N4S8hS7c3OzMhxNX5O+TckTcyISqGrWisay5ys0/fFn4/b4lugRP/r BQ7P3/Wg ql+MrjlktblC57NxUr170VYoGSqcF0coNj0sGBCNkkCcytaa9f0nFPsDRP8KvvlAjqJq4Y6pWemeIPW9IeD51AaWe5nIjIoGOrdxalwP4CR0Ne8m97bzeAvDNCUi9K3YEB4GnV5OF8H7r3x9CCuHRtjzVbk6Jn0nNDCSJJsfmu849cYnUBmVhNgZhbw9x90xwdpCVYOTMkCiJxR0BQJH2Fxe8+4MuSlR+sQUw0dauwZplhRJf1YwVvyeqRfbqOgdihjZZgsKTXxd2gxR6sKW7CddoD3nXJE8G3fJjrMcikNMtwBHgra2/c5e92WUfQSPmO2nFepuIwvaSTaXplfOsm1huLmhqgE47hjEvpj2LMXBRwigUM3vnTAOIc5TUSeOdZPgYx0yp/JffuqMWloYWi5XGOr+As07olxSZe5dAi8Efd0133X1rpjKxJ3LHJ6E1larIrDUWr4QTCs1fzFVnlaEH51bRRBJiaGfCUoAIcAM8PnUYWsV+EPuS+cYNCHeN3We9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello, On Thu, Jul 20, 2023 at 03:23:59PM -0700, Yosry Ahmed wrote: > > On its own, AFAICS, I'm not sure the scope of problems it can actually solve > > is justifiably greater than what can be achieved with simple nesting. > > In our use case nesting is not a viable option. As I said, in a large > fleet where a lot of different workloads are dynamically being > scheduled on different machines, and where there is no way of knowing > what resources are being shared among what workloads, and even if we > do, it wouldn't be constant, it's very difficult to construct the > hierarchy with nesting to keep the resources confined. Hmm... so, usually, the problems we see are resources that are persistent across different instances of the same application as they may want to share large chunks of memory like on-memory cache. I get that machines get different dynamic jobs but unrelated jobs usually don't share huge amount of memory at least in our case. The sharing across them comes down to things like some common library pages which don't really account for much these days. > Keep in mind that the environment is dynamic, workloads are constantly > coming and going. Even if find the perfect nesting to appropriately > scope resources, some rescheduling may render the hierarchy obsolete > and require us to start over. Can you please go into more details on how much memory is shared for what across unrelated dynamic workloads? That sounds different from other use cases. Thanks. -- tejun