From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933576AbcHBFLJ (ORCPT ); Tue, 2 Aug 2016 01:11:09 -0400 Received: from mail-ve1eur01on0114.outbound.protection.outlook.com ([104.47.1.114]:37384 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750869AbcHBFLA (ORCPT ); Tue, 2 Aug 2016 01:11:00 -0400 X-Greylist: delayed 25263 seconds by postgrey-1.27 at vger.kernel.org; Tue, 02 Aug 2016 01:10:59 EDT Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=VDavydov@virtuozzo.com; Date: Mon, 1 Aug 2016 17:26:15 +0300 From: Vladimir Davydov To: Michal Hocko CC: Andrew Morton , Johannes Weiner , , LKML Subject: Re: [PATCH] memcg: put soft limit reclaim out of way if the excess tree is empty Message-ID: <20160801142615.GC19395@esperanza> References: <1470045621-14335-1-git-send-email-mhocko@kernel.org> <20160801135757.GB19395@esperanza> <20160801141227.GI13544@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <20160801141227.GI13544@dhcp22.suse.cz> X-Originating-IP: [195.214.232.10] X-ClientProxiedBy: AM4PR0501CA0031.eurprd05.prod.outlook.com (10.167.83.169) To VI1PR0801MB1872.eurprd08.prod.outlook.com (10.173.73.22) X-MS-Office365-Filtering-Correlation-Id: 43f65d5d-bc7e-422d-f113-08d3ba17cedf X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1872;2:R3SGb3mO/bgPoOCWdfycYoVbYzKK8y02hmJsy78grAAd9+jhPxosj8y32V18KJpthgwdo7x6Tc8TuIzmbGeKXTwPAjja7OA2Ok7Wv4TQVJq4d3vU/DxyiSfqE69Gok6IKOCLFFamgb3l49uGH3iPb2/LYc/6F9k44IBpmSRUUKfTvyTtcyx2a5BlpuJ0qsT9;3:P7ZGhHn2dQZOmrpYAMCZtMJ1BMa8Z6V0PdWPynlOroNsE4NEr7W9om7LYCdvN9I3Rr5Ci/aD/6FKMoF4rafqCPHuG5UzzdDZRLYH1Uwb3SBp/vpH39899sek/AXM/5wK;25:5H5rmJ+eQ+qfwnw9mhLilJvGbw8lNr1WSYjldIw0Z/qC/HI5St+LhVOSauQhuxw7o9bo2Y1Ng0GCb+pH/02L/FIDf2Hdj9ALrKebvta2y3VBvrkC4yYrpO1ITkCaV7Fps/PzW9b3YIpaTXcumsHUXY1XALQGFCOUEffnN5fUNzMlBMlMuW3I/O0KO7jod7nB89zKF6Hd5TCIc59ug7BgtG4lFAmbfyK7ikBGmZ5aI5jHoIFcsh0E9VOGcdGv8tHa0nCKhT2svr3cOAVpWrHhFN5pQfpmYSDXkgFsYfN42LPlhL/43WAZAu81fnr/hr98lHraJ990dpwy226JJRnGWgEV5mmip8FvfDqXq0rb5Ox1WAb4BNP1pM50Ad8j9zAVvbulxk5sSLgKM2nHPkFkJg5YYYjdG+2IWNJ57M2hSyM= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0801MB1872; X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1872;31:lt8gyY9pqqjMe0Zz2Rjn2brK48DqS2r3EIHpD6oTjAJImx3LrcHtLXb6jQ6/tsvmslyg3ayRolxhDk1EQxnyGzGskPnTnPpoxCqK6J7h83wmdL3D3Ma1hQn9R7lZLqkf8dxibNVuZz/BcVae0w71TOD26Y0/6dEyO0mGALc0cUxyd4/DVzrVlsR/5gu1Okt3libDDYkEe8wKFAFonFqOdQ==;4:7s4bG2ZnRdd+hNb4U2qCoj0ycNhfxB+6HaMCcm1W2l+qM+DuaMAAcDNwWtppXRDr/WVnPjoee/PE4+0zPzks6QvX8bw6N7szDYdpyNO6qLI0TfURxp70ovnGiDO1bsFeavwZS7JlMuU+50MNOJUtsYlCADJC1QMe/WHaLKMdzJPeAGt8BMSgp1z9WWlRpHcI2oMVEXScM99iytV8V6SosCYuxmmKJ3f5sqhUsplEFPzatvKUCm21nDgdZxwmq++LF0oq4BTP/LE2+09G7sqOgXUhkez54vT8nYVEZViH4uxpqfUWcN7iT9ixekB2RTvqj04HHyWd3qjT0FPK1+e+p+0aeFeHcCZQAFAUhz/JPCefXq0sR624KLLOerbcH/k82WiepjTqirybShZFYKzwSYak4QhF9tX9Y7KzswMetYw= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040130)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6041072)(6043046);SRVR:VI1PR0801MB1872;BCL:0;PCL:0;RULEID:;SRVR:VI1PR0801MB1872; X-Forefront-PRVS: 0021920B5A X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(7916002)(24454002)(199003)(189002)(97736004)(80792005)(92566002)(76176999)(47776003)(8676002)(101416001)(81166006)(42186005)(81156014)(33656002)(54356999)(189998001)(3846002)(6116002)(4326007)(50466002)(110136002)(1076002)(33716001)(23726003)(305945005)(66066001)(586003)(2906002)(77096005)(9686002)(86362001)(106356001)(2950100001)(68736007)(50986999)(19580405001)(19580395003)(7846002)(46406003)(7736002)(105586002)(97756001);DIR:OUT;SFP:1102;SCL:1;SRVR:VI1PR0801MB1872;H:esperanza;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;VI1PR0801MB1872;23:QfDTq22CYsDmOvR1Ukii78El1NqDx9iUdd1saZw?= =?us-ascii?Q?yBDTURRACg3yxpVJb+m+j7ZL/4m5QQWmEeSWcS87gOLvsfiOC1dOKeE2Q90w?= =?us-ascii?Q?OnPkROZrl7fkWw2Jg/WJ2UkC4QEt5peKj8pFwHvhqzoSQ9plysLCEYbxjQeD?= =?us-ascii?Q?TsCgOQnxzX+wsiCQXD5dvGCJZJQ2nqsqbVWRcGKGnSy/hHlOMyaESAj/+I0T?= =?us-ascii?Q?UdLMNYtEFHPqAQbqXl5kAvskDiPKiAB1AQYehhyc/mxb006qE/u99FIacaXt?= =?us-ascii?Q?b/pVvKEY5mibrIx0MQailxMhLPWN3Iz57suspyE7p0VY52PiUo7Fjav6K1ts?= =?us-ascii?Q?u6boY6F97nvQcEZxAIZVqJo8owMcIS/FpwuEgjHJPsQGJprSP44U17fSZlki?= =?us-ascii?Q?xeuepP+NGLYgA94GnKc1S4zAdOfA5V3HpFKfqAjsA8zI5jfxDqoSmN+4fc4Z?= =?us-ascii?Q?r10K5gm7J3DShYctqHK/tnfFBR7XN6ou9H+zvlLmsmMfev9/H+D813yo57Dj?= =?us-ascii?Q?kP48Ix6WlSf22dnt8hgZM/3Gd6vDuDG/t5n7DC0vVWJDkV4pD5nAWw2UV5Z6?= =?us-ascii?Q?h7e8aopvFc40jhB32DgBK+RDjkxC9oRh4XqjMhu2LzZrLU4VwyyoetRlazB9?= =?us-ascii?Q?Nsw47MEAGlmPiDwv4BG3WMIhkjV/RmYLI/yfdg6ZFre2wYpgAT26m55/kJ81?= =?us-ascii?Q?/I3YIs8sysz4aBZ47psqdPpbT+I7fGADwsGKRZ7VxQjW6njBYpOYMiJvTzlQ?= =?us-ascii?Q?nXGy2HkqHuYyvl6fQVMhdP2pI4f+bzpv+0/vRuu1QNYhnqh/pLa4g0FE1v7E?= =?us-ascii?Q?xwXFju/2H/dI4KhakgdDYX/BUbI+wmxTgb+1kwXJ/zEbQg+fIX/RvZ8y7olH?= =?us-ascii?Q?o46QhNpT83w8IUSENvbiBrhL2kN2+ZeMoivcyKIiQpIUS8dbcHkOlfhHxu/z?= =?us-ascii?Q?kndP6wOURohf84jhAIEKQHjasLd6nbn56R3NFaDOoc4TJCHMrEl7GMXp9b7Q?= =?us-ascii?Q?CXg1ppBKxxKFpDkSfjxtQ0/SHr+D3KD4VLKmOwLwR74luyOvwBRZaMlRSJYp?= =?us-ascii?Q?u2k5svOB+DVwcV3OFM2YxVxD06HD2PWkYZdNap71mlvk+MSzDdlRIOoa11tD?= =?us-ascii?Q?Bj8SCjQdVQZw=3D?= X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1872;6:IBJcSQToOKTRPJYarA+xsNwGmlG6rOxD7DThekg0K7gBc3MXTwHRWhe2NFTACs2d2gLq2rFMDGvhEVZAWyiZzxUf/VqEMM0vxCOVyraUyqiRCVUYaV1VP5Tfx34gkU3kKC7HjtAjkwTyMIyrkk7TXcBc1tUkrbsDDtuXCGuvVeFZWgCEFlM6DVpdxWI+TdB7uP/af4Yu6/8n2V6iUu4fQCY1DdKdPNLHIW33r3Ga1h/NMjirgCBmOdmXdg85giYcNYkcYiTRsAL7gX87he6ALKmIo2uP9aZi48mFbYdy7ku3FfIxoLBvak05cIt6/vsa;5:g3xEDDnkNOq5YceqFaKqjLP/oll86o0u4SUsnmTYULfmrphXbR8BVuuQWdXt4uYnR+fO5DZ44uSfIOsV9eoFuJkfIOPKGSRzmlVT4kS9IInKi/OUZ9sH6l66r1Sb/Q2S4mZMGnK23euLdWgl7Nw9Tg==;24:o9TEoOFMd3t0smk8BEDFBE048K+QqpdvWcRMw1qBYHSuF+/g6LPlVKFVH+l8BOCyR/Pgzxe7+H6DKmzqHUElcggd2L9+PIHr4sDjNb2rgdk=;7:0cpRYgw0UYm0F76a+eRRuY8jZpLm7VyDilRDJLDg+HKdbQgf/0s4LpGtpW3gPpSsFN1uT5EpuJFhyHOzJcymATJ3yZuYWuDyUvDwNsNd2l1lB10JDlsv2iaDVPcHVEme5Ck3pSDQUSq8Ilzqj6MJlnjz6uzhGH3+IOEigvp4W+8V6tgYFnZPGIwhneWTbUb8O8tqORCc9Msz2yElrwEtM+SqANXRxSQlfNxWWvdoImycpiZATk3kPrqoFGo2wMWE SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;VI1PR0801MB1872;20:Hz4nJ05OOlzzWrXTUMwPn/zecffG1xLH9lGg2TB8/fua3lUofS/EIdOE0yA0AyyrbU/PpFHsDvrRvRJ5I/ZrCcCw/gvyY/pZpjt5Ey/zbzqcYQnpnaM8na00JHiMRvl07A425oiHhxvAEImszpfh75f0OeTLLKkJi9PIzYCLEDg= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2016 14:26:19.9912 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1872 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 01, 2016 at 04:12:28PM +0200, Michal Hocko wrote: ... > From: Michal Hocko > Date: Mon, 1 Aug 2016 10:42:06 +0200 > Subject: [PATCH] memcg: put soft limit reclaim out of way if the excess tree > is empty > > We've had a report about soft lockups caused by lock bouncing in the > soft reclaim path: > > [331404.849734] BUG: soft lockup - CPU#0 stuck for 22s! [kav4proxy-kavic:3128] > [331404.849920] RIP: 0010:[] [] _raw_spin_lock+0x18/0x20 > [331404.849997] Call Trace: > [331404.850010] [] mem_cgroup_soft_limit_reclaim+0x25a/0x280 > [331404.850020] [] shrink_zones+0xed/0x200 > [331404.850027] [] do_try_to_free_pages+0x74/0x320 > [331404.850034] [] try_to_free_pages+0x112/0x180 > [331404.850042] [] __alloc_pages_slowpath+0x3ff/0x820 > [331404.850049] [] __alloc_pages_nodemask+0x1e9/0x200 > [331404.850056] [] alloc_pages_vma+0xe1/0x290 > [331404.850064] [] do_wp_page+0x19f/0x840 > [331404.850071] [] handle_pte_fault+0x1cd/0x230 > [331404.850079] [] do_page_fault+0x1fd/0x4c0 > [331404.850087] [] page_fault+0x25/0x30 > > There are no memcgs created so there cannot be any in the soft limit > excess obviously: > [...] > memory 0 1 1 > > so all this just seems to be mem_cgroup_largest_soft_limit_node > trying to get spin_lock_irq(&mctz->lock) just to find out that the soft > limit excess tree is empty. This is just pointless waisting of cycles > and cache line bouncing during heavy parallel reclaim on large machines. > The particular machine wasn't very healthy and most probably suffering > from a memory leak which just caused the memory reclaim to trash > heavily. But bouncing on the lock certainly didn't help... > > Introduce soft_limit_tree_empty which does the optimistic lockless check > and bail out early if the tree is empty. This is theoretically racy but > that shouldn't matter all that much. First of all soft limit is a best > effort feature and it is slowly getting deprecated and its usage should > be really scarce. Bouncing on a lock without a good reason is surely > much bigger problem, especially on large CPU machines. > > Signed-off-by: Michal Hocko Acked-by: Vladimir Davydov