From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751385AbdH3Kzz (ORCPT ); Wed, 30 Aug 2017 06:55:55 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:39113 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751263AbdH3Kzw (ORCPT ); Wed, 30 Aug 2017 06:55:52 -0400 Date: Wed, 30 Aug 2017 11:55:24 +0100 From: Roman Gushchin To: Johannes Weiner CC: , Michal Hocko , Vladimir Davydov , , , Subject: Re: [PATCH] mm: memcontrol: use per-cpu stocks for socket memory uncharging Message-ID: <20170830105524.GA2852@castle.dhcp.TheFacebook.com> References: <20170829100150.4580-1-guro@fb.com> <20170829192621.GA5447@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20170829192621.GA5447@cmpxchg.org> User-Agent: Mutt/1.8.3 (2017-05-23) X-Originating-IP: [2620:10d:c092:200::1:4542] X-ClientProxiedBy: DB6PR0902CA0018.eurprd09.prod.outlook.com (2603:10a6:6:2::31) To CO1PR15MB1079.namprd15.prod.outlook.com (2a01:111:e400:7b66::9) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2d8f56a1-7915-4fa9-4de3-08d4ef95a6e4 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:CO1PR15MB1079; X-Microsoft-Exchange-Diagnostics: 1;CO1PR15MB1079;3:14AxAmN2QF1X6DSZBxZQGCktstI5rTeT4l/3SFrmKy5iD4JtojtQ4z3hfU83iZR/txx6uFH+B31swqn7SUDvk+AzcjopPZvoJGsWNXBKNuHp6GULppYRj/FApw8EppUvVWnd5RuRmxaaSKORf7+4T0dEqmJBDBXZbj6DecEfobdfRjuonlmu3GxtQlSOmp+XrnRdnwo6gKdxz5b99xPdWgrE5/uTD49efn4c3m8x+1OcNKfzXzP8x0PWTVkDDtlP;25:ZWcQ7DE0ds8gvhCfsCkfHTMQEveY+cImXtRv+nKlSvHvNT5O+RImzQnDFANKKsn74iLz+DXw9Jhw5VO2XGBNPEKRsG9gEmenO1bkbyw9+qCCNeZhbeSGbCXaiZtTNHzQji5BhdqoNRLapLgm76U0aSE4ZB38wGz36WsgbXoV/yOlA7hz9uVnQnP95LvCGQ9gQA2t5qFGI+ILmWzr6VUPEgx/CoYZd9TcmLvXY2so/pOqeFp4nlZUZ/GiDxQmB12K/ecDmjJzZGLIjcO8XtCj57bApB49kiqOR+QYgNY2a3S1UJfCUtVwjQ+61enT+9n175+2YuOEzBl8goXpMxB5lQ==;31:oeZgep72Pq6RfNcU21og6R4dEjD7n6yVTlIqSakHMAYjlgPEB8QaKTxU+f0y9jjAhPVcK7XVavH/mct/GqXnuwYaL02sOKB5HUWN0bixGqYu7MYvGQY5Z6AC5XIRzW6I5nqkTNEVKLV3zVizbniZ56uuYMhCfBlX0FjtKHOpNr0/2ogHI0gVFdaeEGeYwavqp+1FdYs8kNXSzrxjtjNEm201sEHSRTddawTvuklgbno= X-MS-TrafficTypeDiagnostic: CO1PR15MB1079: X-Microsoft-Exchange-Diagnostics: 1;CO1PR15MB1079;20:pEK6eTxl6V8OS/HL8Z6W9a8M3KChvZvv5nkhHSiW2f6eicD5BGoVKaTthJ7Pnhlds1n6ZtGp0933pi26f/u2aH3GSo23qDvPhvc571a6BO7nPnZsYESfejG4+c5rCZMiNAAQmJ118kW42uJZtL2rctyfGN5A9zUlLK5+OHtevkiyvxlpRgVbzN0Nx2TkGqWX1uYu2a9jx9Sg6IehLUmlGeES0mxZcCbsP2zNgw+sv3z/mkWep+EoZiobZ1Q6tUI4hNpEVzWU5jBINjGmuS9EJJpihc77Cg0Ou1KyKGtE277YNLmHRYAwURjr/g434IJqHyP5M7hjlrFyzbaqSpaeBGFxuzummKT2yciRF4kQ6UcSNzHLVTvvUcv9r+7zpFWPOMz2dcm3HbRM0r1xBuzAv85lD+7VeEazqMmqLzZ9tNAbxCWFfbN7ZxNn81oN7psSilVepSH/RfQlV/uIFYzyj2ZgK09utbLP3hkf5zB9etGrrITfYryYdtvAgJZf9MUD;4:FWRU1DfO5pyTgM/f3NsPAYpQGw+cGFJh7IdCWI1IPrQ1WtBpQptpACLBTV37y6sDjLtwDMrrlACehQkYk4BWKxCUxlhTpMZKDwbqK0wJwWaCnzEMQZ6m6ykRS/nfVJKhYQMY7fbDMJ8Gcmf5E7QruNpPI7GMGgtwBJ6r21pCaKpqkmWS18LJ95jKuXGqP0JfzYjUjmVHdAbQPDnN93GIjlnJgeqhj5iWcOOzuL5JypJzaF+bLQKd5GEz28yKOsVzIgfN/ZX2UDPTszwiGbP9sIEZnoTWf4T/tdjGMCJIZ9Ji2253+p6xdI0zL24eu3lUwbuQgQ3/YzJh/Npzei9/sQ== X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(67672495146484); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(10201501046)(100000703101)(100105400095)(6041248)(20161123564025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123562025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:CO1PR15MB1079;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CO1PR15MB1079; X-Forefront-PRVS: 041517DFAB X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(6009001)(189002)(199003)(24454002)(33656002)(6506006)(53936002)(6666003)(81166006)(8676002)(81156014)(229853002)(9686003)(54906002)(55016002)(305945005)(8936002)(7736002)(105586002)(86362001)(42186005)(106356001)(6916009)(5660300001)(2950100002)(189998001)(50466002)(68736007)(2906002)(23726003)(6116002)(25786009)(478600001)(47776003)(4326008)(54356999)(76176999)(50986999)(83506001)(1076002)(101416001)(97736004)(6246003)(39060400002)(4001350100001)(110136004)(18370500001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:CO1PR15MB1079;H:castle.dhcp.TheFacebook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;CO1PR15MB1079;23:1HtFrZmfKBij1TTxeSCJ/QAZt8ixvDhhj5tkJClY4?= =?us-ascii?Q?7FTI5yn3w3kKh2UFqj37Wetz7OKvy2x6ecYnrq2fMIit+X1+gOko+HCHFou3?= =?us-ascii?Q?XyQQ1a0yyuqoQRZ+mffegmpr0EOiZZqzhlW7g8QuFb4EeNonG8R6v5PMjwNV?= =?us-ascii?Q?oKHWwiuJxgFv42VZyDgBLZTpKNkKflMlbrTQ/1nQH2wI7yyPj1jmE3D21INJ?= =?us-ascii?Q?Uoiv62QWuTy46W01AvCu3mnTAIZ+9FGWgWXozuIeLdlCgT55pmosiF+HJ1Ci?= =?us-ascii?Q?ufvVgVXDVQiZsbOqHPPh7a2Yo8I9RvQxL1uVAEZ4xfcQZZR/89KkRISCWz2W?= =?us-ascii?Q?xNrkUBcAqomvG3NYTXHAUi/b/m2pDNFd+QbHKIXMoOg5V8J/v4hGuml4Hw94?= =?us-ascii?Q?u+Arh5j39Xxbk2jWry9LMKR0H1tFbam6qTDL8rgqU2CxMMpNG5UgTvU5xfmw?= =?us-ascii?Q?ro1g+FT85KhOjNIvHfK3pzUYQ8qY596nY7XuELb8nh3isU/D0Az7bDXeVg2y?= =?us-ascii?Q?2RsIIUnjIJzJYxf/kQE43QFNDpYZhXLssHUr1mbHS96MO3HC+pUV8BlQtdF7?= =?us-ascii?Q?6gS743uEo9JrYf9BLCiuco5S0lUwH7iRF7kHPQprvJpZOWNoxHTkpmoGFgTn?= =?us-ascii?Q?42U/7wlUdJB1Cvzmvb1pVVawq4YaECHZ2QSEqk2Wo90hanXjzDvpRIGQlnUn?= =?us-ascii?Q?QPTdm3VRl3+0ZN3sAVOxqIhl2RRvOel/ma1CVyS9Q3L46rMDCah6xWbQCwnU?= =?us-ascii?Q?eXm67DrDRJFnFaz45EacL2N0dz/EuBYp/je9Rbhx3uYVKYhVBnRP9PMAOR12?= =?us-ascii?Q?ymplkyhAmCct7AyC9Q040TwpJmOO+nWO36r2o4CKKr8aQqqZtZaECbRgYYrm?= =?us-ascii?Q?ZNSklbkOIk8Rhqgp1YEzxD8UrZQ9XdJAiMSfzN3hbF5y4cGEsOBVqwf0D70o?= =?us-ascii?Q?wDjSAf7a2UHEnY8/VzpveYFsM4v4aPkqwWkxdjYMcmSwzAcbkL8TLe96xjDR?= =?us-ascii?Q?4qOlRBE7KWeaSxEYaLekPM/foaO4gnULhKmdzHfFlp6DZk2H8PDceZVdo+3k?= =?us-ascii?Q?jnNpPCaK+FxfyWTUIH52cKndTxQLeutPXtp2beDHmtfgC2+g921XoKELRXjx?= =?us-ascii?Q?n8ePbagF4zsKCZK6NTuQMOiydN17HFxpkYVFFYyO/qp4uipp/tUFA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;CO1PR15MB1079;6:O6PeHN79gTFM18oXbE4hc5JlkGjEy+zurGFMgRdueHU4jbWrbjRfsOqH//51ypzJlvJR9Lo+4yBKcD1kXrJKOtLOdyRazC9V7X5O6eR/km/h4UfI9+Y2N3LGIzv3+vYJ6KIOPZJ92JXOaE7/al1f9IEZ8YoN4w/k9jYcarbRlfbxKBh1uCax6DPyaO8tnEJU8VUxlzXIJ0gu2jwmRcPxLVL65dcH2hF/6O5T4pYDgtCjoxLzdO28hf21LkS/da8feRQcjVds2atTDW8+zeiwIlc7xlY3s+1Twgid8qrOQ7kDL3qTWklfBC/s/FAewVKkMEG1ZJ0PxgHaCZ3pcFN6dQ==;5:33tWnxvoY+Og7jsEjl1AUIbsRjY8Dc0sRWseo6hflbA3vDubMrgz5a5NPFmqj/TnOuHTGRrTPpFhVThzXeXVTeuvcZ7w+CSu8UIlZMqwQZVTLt3TL6U/CB2tyiuTxWxR/G0D+FLCS6sYazF7yqhqRA==;24:5mgWRDA0so618t44l3JumMIbyEs9ASfyIp3WY0OSXEiy5cXtBDBLuQ94HoCD2zXQEg8ZRbyK1QvK9IwDKHj7nltNDiVdRluhEN+r7v8ObQo=;7:OpEJbimNMiLkk1d64EFG/xpa/Lmcb0QosvWyMtjoVZrl1aqb4yy+900vh1qwBB02L0Nfw9DjYB+1R7iNnKd79VgHPyYm+vDvpmNX3X37KJSkNcHBwqARYVof4meg8xkWlJq2NN29ge0ak9mGpqw41Gr0evV67HZlTEV/3sqR+NIIS++M/BxlnN5oJh4TcERnlDS5CjknlRrf2BYgygNKbC4KGfK33aD2v4AaUpr2jrM= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CO1PR15MB1079;20:Oas/AYVG2Hoez0Bsp9FNZVnu1Y0mNjm5SafJyq19YwISFkkM5Tzmt/XbvpAlTWNtMWB6iQyes2htBQzXYZ4fFF3h2b4wjZc2/Y6sF7o9jgB2+G5d49C94qsRr1G5hEF+dvv9N6AeiJwIgysfDpim6OCkbudAhm22ciikOZ8w0dY= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2017 10:55:38.0540 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR15MB1079 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-08-30_05:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Aug 29, 2017 at 03:26:21PM -0400, Johannes Weiner wrote: > On Tue, Aug 29, 2017 at 11:01:50AM +0100, Roman Gushchin wrote: > > We've noticed a quite sensible performance overhead on some hosts > > with significant network traffic when socket memory accounting > > is enabled. > > > > Perf top shows that socket memory uncharging path is hot: > > 2.13% [kernel] [k] page_counter_cancel > > 1.14% [kernel] [k] __sk_mem_reduce_allocated > > 1.14% [kernel] [k] _raw_spin_lock > > 0.87% [kernel] [k] _raw_spin_lock_irqsave > > 0.84% [kernel] [k] tcp_ack > > 0.84% [kernel] [k] ixgbe_poll > > 0.83% < workload > > > 0.82% [kernel] [k] enqueue_entity > > 0.68% [kernel] [k] __fget > > 0.68% [kernel] [k] tcp_delack_timer_handler > > 0.67% [kernel] [k] __schedule > > 0.60% < workload > > > 0.59% [kernel] [k] __inet6_lookup_established > > 0.55% [kernel] [k] __switch_to > > 0.55% [kernel] [k] menu_select > > 0.54% libc-2.20.so [.] __memcpy_avx_unaligned > > > > To address this issue, the existing per-cpu stock infrastructure > > can be used. > > > > refill_stock() can be called from mem_cgroup_uncharge_skmem() > > to move charge to a per-cpu stock instead of calling atomic > > page_counter_uncharge(). > > > > To prevent the uncontrolled growth of per-cpu stocks, > > refill_stock() will explicitly drain the cached charge, > > if the cached value exceeds CHARGE_BATCH. > > > > This allows significantly optimize the load: > > 1.21% [kernel] [k] _raw_spin_lock > > 1.01% [kernel] [k] ixgbe_poll > > 0.92% [kernel] [k] _raw_spin_lock_irqsave > > 0.90% [kernel] [k] enqueue_entity > > 0.86% [kernel] [k] tcp_ack > > 0.85% < workload > > > 0.74% perf-11120.map [.] 0x000000000061bf24 > > 0.73% [kernel] [k] __schedule > > 0.67% [kernel] [k] __fget > > 0.63% [kernel] [k] __inet6_lookup_established > > 0.62% [kernel] [k] menu_select > > 0.59% < workload > > > 0.59% [kernel] [k] __switch_to > > 0.57% libc-2.20.so [.] __memcpy_avx_unaligned > > > > Signed-off-by: Roman Gushchin > > Cc: Johannes Weiner > > Cc: Michal Hocko > > Cc: Vladimir Davydov > > Cc: cgroups@vger.kernel.org > > Cc: kernel-team@fb.com > > Cc: linux-mm@kvack.org > > Cc: linux-kernel@vger.kernel.org > > Acked-by: Johannes Weiner > > Neat! > > As far as other types of pages go: page cache and anon are already > batched pretty well, but I think kmem might benefit from this > too. Have you considered using the stock in memcg_kmem_uncharge()? Good idea! I'll try to find an appropriate testcase and check if it really brings any benefits. If so, I'll master a patch. Thanks!