From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9163FC54EAA for ; Fri, 27 Jan 2023 19:29:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDE876B0074; Fri, 27 Jan 2023 14:29:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D677D6B0075; Fri, 27 Jan 2023 14:29:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE1256B0078; Fri, 27 Jan 2023 14:29:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AB0CA6B0074 for ; Fri, 27 Jan 2023 14:29:51 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 72501AB743 for ; Fri, 27 Jan 2023 19:29:51 +0000 (UTC) X-FDA: 80401568982.05.8311398 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 650DD8001D for ; Fri, 27 Jan 2023 19:29:44 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EJRjtkSs; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of leobras@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=leobras@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674847784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SUv5DFOx1orc2Szz1mjtgk5jcU9W4yiwziY2UuvBPys=; b=fEeMCJfJTa8x86Cf3/NH0hzq8wA1vVpyjvEs+9F30+ficAeRQo1aLTB2K8h4Op15O2gS+o Mgmywj3caCROgD2RppqEzfubyPrHE9PFb7z8aNl6ICFJZPdYpQYK6UGgRawev0rkTpwUnL JK3RiiY2isarg198vcwpm8Spf3/br7w= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EJRjtkSs; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of leobras@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=leobras@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674847784; a=rsa-sha256; cv=none; b=lzaAt8uXSWuqmdeoB/otoexeDktYae1VOLKyIl3nl6OIaMdiGwwXBvAe651MZ9e1/vFw5c vh6S90Je5qCSrGJ1Ja95I+DGMv+9i2lXp79j3V/4L+Rsq8g+oRUGwSvXiTw/ann/rs4+pS XRRH4qciMpRLnztzoTWk9eNcEORaEfE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674847783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SUv5DFOx1orc2Szz1mjtgk5jcU9W4yiwziY2UuvBPys=; b=EJRjtkSs4wO8UEiFb4+XLL3tCn01j3aXo7v/Bnge5HtUNGXBCtdXR59hvdx85raCIpijbE LZq0oS+XKbNL9kHShnazwL14/kz0sLCmL8g4YcpQ5TBMqqD9//QruMJlMvC+SiUhguIJ5U zZ40iGB8Sck6WgIe4cJO2EkxQfJE1UA= Received: from mail-oi1-f199.google.com (mail-oi1-f199.google.com [209.85.167.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-511-fZ0Xp7x1N4-PAOK596K-rg-1; Fri, 27 Jan 2023 14:29:42 -0500 X-MC-Unique: fZ0Xp7x1N4-PAOK596K-rg-1 Received: by mail-oi1-f199.google.com with SMTP id z8-20020a056808064800b0036aae651f2aso2227660oih.17 for ; Fri, 27 Jan 2023 11:29:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:cc:to:from:subject:message-id:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=SUv5DFOx1orc2Szz1mjtgk5jcU9W4yiwziY2UuvBPys=; b=i5OWr5MaDifZrRafZTnMrOuJEn65RxiqxKjyZyKAxIICsJquxltV/YsNjZ7mwdSz2I YfKgTwx/iQ2n9/Atis6ZucpebmaZepmXCMCziUY/xsUFsfM4teBzF7lW+F+Jc+TVTCud mAU7yKKIRmNeQEtnHBiiHoKdkXgF8eYCVvKmcFeMP/gyH9/H51G2VNi2Hhwm+DOscZE8 23mRonAI50rG93m0aFWL/F+Og/HEM2yxP+vJ98k74j1iOJvF9qDKP/8tm9VY1vZI0HNk rzDS8fffC3eWO375X0InaEgUZ6v0nxrhAYvYp3PknbmCw2q6gGvj4DbWg0A5ElrxrcX3 2HhQ== X-Gm-Message-State: AO0yUKUXIaiwxASD0npgOq7Frsc3c9Ebzs++dwCZA9509vmWx7ounDEA mbammogrx7g4/QgZ9shZloldKneArdxUt9c1e+yum/1DaGKuF4iv7XvciS3nbik6Z2X+KZ7y5qJ ed96lX+7vHR8= X-Received: by 2002:a9d:3f1:0:b0:68b:ba7e:1800 with SMTP id f104-20020a9d03f1000000b0068bba7e1800mr38636otf.34.1674847781948; Fri, 27 Jan 2023 11:29:41 -0800 (PST) X-Google-Smtp-Source: AK7set+Ph9lhLfqTNVR5otrvd7xQnQfYQPxx7b1GPoc50tFYq9YnPESypV2bWjtmZa5dzVRAfQJQCQ== X-Received: by 2002:a9d:3f1:0:b0:68b:ba7e:1800 with SMTP id f104-20020a9d03f1000000b0068bba7e1800mr38629otf.34.1674847781695; Fri, 27 Jan 2023 11:29:41 -0800 (PST) Received: from ?IPv6:2804:1b3:a800:6912:c477:c73a:cf7c:3a27? ([2804:1b3:a800:6912:c477:c73a:cf7c:3a27]) by smtp.gmail.com with ESMTPSA id cp11-20020a056830660b00b0066ca61230casm2160524otb.8.2023.01.27.11.29.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Jan 2023 11:29:41 -0800 (PST) Message-ID: <029147be35b5173d5eb10c182e124ac9d2f1f0ba.camel@redhat.com> Subject: Re: [PATCH v2 0/5] Introduce memcg_stock_pcp remote draining From: Leonardo =?ISO-8859-1?Q?Br=E1s?= To: Michal Hocko Cc: Roman Gushchin , Marcelo Tosatti , Johannes Weiner , Shakeel Butt , Muchun Song , Andrew Morton , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Fri, 27 Jan 2023 16:29:37 -0300 In-Reply-To: References: <20230125073502.743446-1-leobras@redhat.com> <9e61ab53e1419a144f774b95230b789244895424.camel@redhat.com> <55ac6e3cbb97c7d13c49c3125c1455d8a2c785c3.camel@redhat.com> <15c605f27f87d732e80e294f13fd9513697b65e3.camel@redhat.com> User-Agent: Evolution 3.46.2 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 650DD8001D X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 59uhxzgj8fh1subqi9yejaf9zg8kypsw X-HE-Tag: 1674847784-826379 X-HE-Meta: U2FsdGVkX184VOO5fdn3UlUw/qmLq6VXA9R77TKQI4rri8/gpRYucpMEAMeKS/3M4xAykfV7Fr/vfwifqbg1R0pwKFEOjQboZ2oDvl03ZNyH2Q26ySdKqS/2cQPKT5/T8FltTdaUy6hpcrSvvfkx1wqlpfCjNriCR8B3jpIw02h41HIMGZRPEU9NvPpT3UwM1wx/HQs+EWeuxSyQil6WhyJO7WHEWqdxHu0Jb5Z3dEwsR8ttCwDNDD+9N7xV2yFmxs1XUUVooTxnAFNnU7fgViOv8bnAZ7fggkd+HEu75V7B2jRm2RO4OzXlThNbe9aJA/lZ4rBWRhskj3NmMOcUxeRmhmI7tIT0eWyITwxr3jarbpGyhBf98cH4KVV27k1aRPNVG+/v8uolPwBsATuyx/apuswdI9CNy83T2vnADhLTNxpzKQqimpRptUVwRPVUNGZ4AZbgt2uN71hMhv7TUAeC3ztuZiiy53eBouepfklYluZPHWQQv1fbAwLV4G2139OJ9wa3QegLv1dRqTTJlNrUXNlFFg1zMGAbRV3OZK/HoQV25Rf0ipVMdWeOo0MbEY0B/g8eqqsWLA/UsTbvKVyMg0ESPHt0vuOSky6kIMq7y6x8a4Z5o9ChM4+3m/JzlgMnH+Z/dVcGEuPs/J4b5IsWU4nwTUqxIUOZcmNfQqGNOCjyx8MGP9VdGQkHSqkjbNSM/4u7rR9gGkJueJ4OCWaLYtBmiWFLohoL/r91oNzoWUVjNv2IVV/a6iJmgKhuffV6BQr2FWmgulMECWGK7S34hhXl0s3G+IgVkGMV8AiVQeUWLKVpkd9rMpJQR00Lt5n6ute0dPWBFSYxog4FbfhS8mKpBnydqeClZYC0EeP3a/OsoA+/1UZTyu3sjGxmdcdHCaPlacp2uWEv5KELEzaxg5QcWeR21rb/8q7ErZUdfNgIcCvHOq9n45TllT0CTxiJEHGVMbbQ1lhzs66 nzRYuRjF Y6aM7JbUQgR9xcCA2kzapL8iNxCk7syjMl1yNSI0rqNMjvqwn324GAYpx3/uP2v6XTIRQ1dgz6raQ6X37i+d9dNEnUoRUA4Xn/2Z1p6BRY5vv4vVUKsMpD1G4dudoOx4MbAhANGnGW87z4lvmD5CRhk2IQ4HI/HbFu4auAdd84EvroA3K9NTDvdNp0jHLmG2nTTJ6fHasI+a7VirFMz7dEuXzTf8FRetJGGaO9m68fL5slmHVkLsMex6qpiS6HPMORbKMYuIWkXR6vDtumchG830qkpyToxMzvzSymKbHikgdf6ReROk/dtWdLhsObiAkVswpolCox1OrDpKqQNEuNugbS2X8Oh1xLUm+RHJr3w4PxUq3cv4zDNUXq359nUjCajI+n7deR/mV3eDQJ6HGWurSxWzZEbO9ncbr3dYL+LQkJy0NYctJTJ2Xux//jHjuPHm6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 2023-01-27 at 10:29 +0100, Michal Hocko wrote: > On Fri 27-01-23 04:35:22, Leonardo Br=C3=A1s wrote: > > On Fri, 2023-01-27 at 08:20 +0100, Michal Hocko wrote: > > > On Fri 27-01-23 04:14:19, Leonardo Br=C3=A1s wrote: > > > > On Thu, 2023-01-26 at 15:12 -0800, Roman Gushchin wrote: > > > [...] > > > > > I'd rather opt out of stock draining for isolated cpus: it might = slightly reduce > > > > > the accuracy of memory limits and slightly increase the memory fo= otprint (all > > > > > those dying memcgs...), but the impact will be limited. Actually = it is limited > > > > > by the number of cpus. > > > >=20 > > > > I was discussing this same idea with Marcelo yesterday morning. > > > >=20 > > > > The questions had in the topic were: > > > > a - About how many pages the pcp cache will hold before draining th= em itself?=C2=A0 > > >=20 > > > MEMCG_CHARGE_BATCH (64 currently). And one more clarification. The ca= che > > > doesn't really hold any pages. It is a mere counter of how many charg= es > > > have been accounted for the memcg page counter. So it is not really > > > consuming proportional amount of resources. It just pins the > > > corresponding memcg. Have a look at consume_stock and refill_stock > >=20 > > I see. Thanks for pointing that out! > >=20 > > So in worst case scenario the memcg would have reserved 64 pages * (num= cpus - 1) >=20 > s@numcpus@num_isolated_cpus@ I was thinking worst case scenario being (ncpus - 1) being isolated. >=20 > > that are not getting used, and may cause an 'earlier' OOM if this amoun= t is > > needed but can't be freed. >=20 > s@OOM@memcg OOM@ =20 > > In the wave of worst case, supposing a big powerpc machine, 256 CPUs, e= ach > > holding 64k * 64 pages =3D> 1GB memory - 4MB (one cpu using resources). > > It's starting to get too big, but still ok for a machine this size. >=20 > It is more about the memcg limit rather than the size of the machine. > Again, let's focus on actual usacase. What is the usual memcg setup with > those isolcpus I understand it's about the limit, not actually allocated memory. When I po= int the machine size, I mean what is expected to be acceptable from a user in t= hat machine. >=20 > > The thing is that it can present an odd behavior:=20 > > You have a cgroup created before, now empty, and try to run given appli= cation, > > and hits OOM. >=20 > The application would either consume those cached charges or flush them > if it is running in a different memcg. Or what do you have in mind? 1 - Create a memcg with a VM inside, multiple vcpus pinned to isolated cpus= .=20 2 - Run multi-cpu task inside the VM, it allocates memory for every CPU and= keep the pcp cache 3 - Try to run a single-cpu task (pinned?) inside the VM, which uses almost= all the available memory. 4 - memcg OOM. Does it make sense? >=20 > > You then restart the cgroup, run the same application without an issue. > >=20 > > Even though it looks a good possibility, this can be perceived by user = as > > instability. > >=20 > > >=20 > > > > b - Would it cache any kind of bigger page, or huge page in this sa= me aspect? > > >=20 > > > The above should answer this as well as those following up I hope. If > > > not let me know. > >=20 > > IIUC we are talking normal pages, is that it? >=20 > We are talking about memcg charges and those have page granularity. >=20 Thanks for the info! Also, thanks for the feedback! Leo