From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75FD3C352A1 for ; Wed, 7 Dec 2022 12:53:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E46A68E0003; Wed, 7 Dec 2022 07:53:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DF6B18E0001; Wed, 7 Dec 2022 07:53:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C96C28E0003; Wed, 7 Dec 2022 07:53:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BA19B8E0001 for ; Wed, 7 Dec 2022 07:53:06 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 796ED40D4F for ; Wed, 7 Dec 2022 12:53:06 +0000 (UTC) X-FDA: 80215500372.24.F9291B5 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by imf19.hostedemail.com (Postfix) with ESMTP id 00E721A000D for ; Wed, 7 Dec 2022 12:53:04 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=Zn0bxdNA; spf=pass (imf19.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.128.41 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670417585; a=rsa-sha256; cv=none; b=cl2iJAQlo5I88whdgiQjoCZyF48vFMKgpkZ56NIvdGZyYYzS9QQFtpmSkDSFAvCG3Vl92B CHl8lSXsoou7GPJu6Vv06vDRi8iDthaD9mHhgN17A1TU03HERnyYu9yZSupp7+gUnTs49Q 0GuiAqgVAb1T5stveAQ+QIyqLMyZJVM= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=Zn0bxdNA; spf=pass (imf19.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.128.41 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670417585; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SaZKvzrELtZk2moYXHTejAAx0pVU4ccKfYCaOf5kf6s=; b=rcnGcZY04vF75N2VHXIyCe83maLwmaQon68s/gP4adp8TvMMLVpJS7egFi0DvsKLxSNx1g IXq+6MBpyjSV9R5txJR/+nM8OtNbcN5qVHDZMzUtgJZ8T0eSLYMwhU0VTy54dIXXbvpKL/ woR4CDXBJHnaEpFA6wtgyvVMOGzLGzA= Received: by mail-wm1-f41.google.com with SMTP id l26so7285668wms.4 for ; Wed, 07 Dec 2022 04:53:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=SaZKvzrELtZk2moYXHTejAAx0pVU4ccKfYCaOf5kf6s=; b=Zn0bxdNAFoGshkFY4WWwXydY9YieUgDTNZ7UMeR67e3fjlwuS3bimPdqo+MlcClmlA fLNNhn7lOtPh3kHboREx0u0pavIRPoZ8MPQtUqmVWQhHoWMnG69jDb8GbMwd6C7GFJsQ NuSKOtY9XaouhxW8KB/Vri7ug8VeMKV2+LGU19xE5ea0m30ANCKbs/VGpcQEbVv40n9x I985ZKWJvAV1wo2PsPaLr88Rk9C81QmEfEhpUq5bQA8od3/kVF2kWGBu0xyDFcEgAESi qlcoV56MVSQKnM4xnFKdxcjsZ6xpA9yE+2XNVbWboluVsLgl9Hf4w/PNkj/u1euW8ww8 DS2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=SaZKvzrELtZk2moYXHTejAAx0pVU4ccKfYCaOf5kf6s=; b=3+c14+NQW/tWNYSMDplmKXikKWhAnXzuDMLVYc28G8lWiRSxm5el8QZLeLg5gTI8Ho HgOEP2k69veJPURfcqCgRzCXBdneuGN2iD2ImmuEgE46717Cbp9hSNPP7FLLGKQSmPPC ATTNGrbrf5x3l3b6UFjJWvfKTo9JF3IV4Z1f8i2Kt/PtYrbCMdK5dPMBx+WUT1+od7fm vye9YTzTL9GuBjvfRM4OOsCbeFzu2oxqqbQlHzd/mE3xrlcmIM4IZLvilZ+y0CXBCst0 /dlHcXzm3TgdgU77TB5O1+7rYKTcH08YDa3S7mibLoNts7RKv8kxKY1pQZoQEbsgPo4S hBHw== X-Gm-Message-State: ANoB5pnRhQNKoHdd8s7K3iLiBqnWBuNIfCBT9UWLoIacSdJ6nuZqznTw 8FGRUIRMKGMNXKhr2ZsEn1dorQ== X-Google-Smtp-Source: AA0mqf6jVbDAqxIxt3K1pXrMoZOl/WacnQGFWIw2cU8q357bgB5/zhXEO0AhmwEdMCP8UHHxuv0l0g== X-Received: by 2002:a1c:6a02:0:b0:3cf:71e4:75b with SMTP id f2-20020a1c6a02000000b003cf71e4075bmr56055226wmc.114.1670417583450; Wed, 07 Dec 2022 04:53:03 -0800 (PST) Received: from localhost (ip-046-005-139-011.um12.pools.vodafone-ip.de. [46.5.139.11]) by smtp.gmail.com with ESMTPSA id d5-20020a5d6445000000b002368f6b56desm23188922wrw.18.2022.12.07.04.53.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 04:53:03 -0800 (PST) Date: Wed, 7 Dec 2022 13:53:00 +0100 From: Johannes Weiner To: Shakeel Butt Cc: Eric Dumazet , Ivan Babrou , Linux MM , Linux Kernel Network Developers , linux-kernel , Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton , "David S. Miller" , Hideaki YOSHIFUJI , David Ahern , Jakub Kicinski , Paolo Abeni , cgroups@vger.kernel.org, kernel-team Subject: Re: Low TCP throughput due to vmpressure with swap enabled Message-ID: References: <20221206231049.g35ltbxbk54izrie@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221206231049.g35ltbxbk54izrie@google.com> X-Rspam-User: X-Spamd-Result: default: False [2.13 / 9.00]; SORBS_IRL_BL(3.00)[209.85.128.41:from]; BAYES_HAM(-0.97)[79.58%]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; BAD_REP_POLICIES(0.10)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[17]; DMARC_POLICY_ALLOW(0.00)[cmpxchg.org,none]; DKIM_TRACE(0.00)[cmpxchg-org.20210112.gappssmtp.com:+]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:209.85.128.0/17]; RCVD_COUNT_THREE(0.00)[3]; R_DKIM_ALLOW(0.00)[cmpxchg-org.20210112.gappssmtp.com:s=20210112]; ARC_SIGNED(0.00)[hostedemail.com:s=arc-20220608:i=1]; PREVIOUSLY_DELIVERED(0.00)[linux-mm@kvack.org]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 00E721A000D X-Rspamd-Server: rspam01 X-Stat-Signature: h4n8j3ajw47hmcq87nqzi5196s8u9y1d X-HE-Tag: 1670417584-756753 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Dec 06, 2022 at 11:10:49PM +0000, Shakeel Butt wrote: > On Tue, Dec 06, 2022 at 09:51:01PM +0100, Johannes Weiner wrote: > > On Tue, Dec 06, 2022 at 08:13:50PM +0100, Eric Dumazet wrote: > > > On Tue, Dec 6, 2022 at 8:00 PM Johannes Weiner wrote: > > > > @@ -1701,10 +1701,10 @@ void mem_cgroup_sk_alloc(struct sock *sk); > > > > void mem_cgroup_sk_free(struct sock *sk); > > > > static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg) > > > > { > > > > - if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && memcg->tcpmem_pressure) > > > > + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && memcg->socket_pressure) > > > > > > && READ_ONCE(memcg->socket_pressure)) > > > > > > > return true; > > > > do { > > > > - if (time_before(jiffies, READ_ONCE(memcg->socket_pressure))) > > > > + if (memcg->socket_pressure) > > > > > > if (READ_ONCE(...)) > > > > Good point, I'll add those. > > > > > > @@ -7195,10 +7194,10 @@ bool mem_cgroup_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages, > > > > struct page_counter *fail; > > > > > > > > if (page_counter_try_charge(&memcg->tcpmem, nr_pages, &fail)) { > > > > - memcg->tcpmem_pressure = 0; > > > > > > Orthogonal to your patch, but: > > > > > > Maybe avoid touching this cache line too often and use READ/WRITE_ONCE() ? > > > > > > if (READ_ONCE(memcg->socket_pressure)) > > > WRITE_ONCE(memcg->socket_pressure, false); > > > > Ah, that's a good idea. > > > > I think it'll be fine in the failure case, since that's associated > > with OOM and total performance breakdown anyway. > > > > But certainly, in the common case of the charge succeeding, we should > > not keep hammering false into that variable over and over. > > > > How about the delta below? I also flipped the branches around to keep > > the common path at the first indentation level, hopefully making that > > a bit clearer too. > > > > Thanks for taking a look, Eric! > > > > I still think we should not put a persistent state of socket pressure on > unsuccessful charge which will only get reset on successful charge. I > think the better approach would be to limit the pressure state by time > window same as today but set it on charge path. Something like below: I don't mind doing that if necessary, but looking at the code I don't see why it would be. The socket code sets protocol memory pressure on allocations that run into limits, and clears pressure on allocations that succeed and frees. Why shouldn't we do the same thing for memcg? @@ -7237,6 +7235,9 @@ void mem_cgroup_uncharge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages) mod_memcg_state(memcg, MEMCG_SOCK, -nr_pages); refill_stock(memcg, nr_pages); + + if (unlikely(READ_ONCE(memcg->socket_pressure))) + WRITE_ONCE(memcg->socket_pressure, false); }