From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vladimir Davydov Subject: Re: [PATCH 5/9] slab: link memcg kmem_caches on their associated memory cgroup Date: Sat, 14 Jan 2017 16:33:56 +0300 Message-ID: <20170114133356.GD2668@esperanza> References: <20170114055449.11044-1-tj@kernel.org> <20170114055449.11044-6-tj@kernel.org> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tarantool.org; s=mailru; h=In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=dzEyH9UBxXuaF1C88Dll+4ESqh/Qki3hdaqIsJZOsHA=; b=PVST++LxLfO6apkUBr2nbyDdloAEA5XLpUMnwkXIxClmolpfCMoffCAD5fmAmFBjov6X6j03T31m8620kTYfuWYvUO+K4XH7kKBsa64QRRhIZOcEtmHXGfkpUU6v5Gm1mvSmMdoI8gl6mECXliB+gKVMnIapQexII9mK1H0quFQ=; Content-Disposition: inline In-Reply-To: <20170114055449.11044-6-tj@kernel.org> Sender: owner-linux-mm@kvack.org List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Tejun Heo Cc: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, jsvana@fb.com, hannes@cmpxchg.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, kernel-team@fb.com On Sat, Jan 14, 2017 at 12:54:45AM -0500, Tejun Heo wrote: > With kmem cgroup support enabled, kmem_caches can be created and > destroyed frequently and a great number of near empty kmem_caches can > accumulate if there are a lot of transient cgroups and the system is > not under memory pressure. When memory reclaim starts under such > conditions, it can lead to consecutive deactivation and destruction of > many kmem_caches, easily hundreds of thousands on moderately large > systems, exposing scalability issues in the current slab management > code. This is one of the patches to address the issue. > > While a memcg kmem_cache is listed on its root cache's ->children > list, there is no direct way to iterate all kmem_caches which are > assocaited with a memory cgroup. The only way to iterate them is > walking all caches while filtering out caches which don't match, which > would be most of them. > > This makes memcg destruction operations O(N^2) where N is the total > number of slab caches which can be huge. This combined with the > synchronous RCU operations can tie up a CPU and affect the whole > machine for many hours when memory reclaim triggers offlining and > destruction of the stale memcgs. > > This patch adds mem_cgroup->kmem_caches list which goes through > memcg_cache_params->kmem_caches_node of all kmem_caches which are > associated with the memcg. All memcg specific iterations, including > stat file access, are updated to use the new list instead. > > Signed-off-by: Tejun Heo > Reported-by: Jay Vana > Cc: Vladimir Davydov > Cc: Christoph Lameter > Cc: Pekka Enberg > Cc: David Rientjes > Cc: Joonsoo Kim > Cc: Andrew Morton Acked-by: Vladimir Davydov -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750952AbdANNeK (ORCPT ); Sat, 14 Jan 2017 08:34:10 -0500 Received: from smtp24.mail.ru ([94.100.181.179]:53758 "EHLO smtp24.mail.ru" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750782AbdANNeJ (ORCPT ); Sat, 14 Jan 2017 08:34:09 -0500 Date: Sat, 14 Jan 2017 16:33:56 +0300 From: Vladimir Davydov To: Tejun Heo Cc: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, jsvana@fb.com, hannes@cmpxchg.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, kernel-team@fb.com Subject: Re: [PATCH 5/9] slab: link memcg kmem_caches on their associated memory cgroup Message-ID: <20170114133356.GD2668@esperanza> References: <20170114055449.11044-1-tj@kernel.org> <20170114055449.11044-6-tj@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170114055449.11044-6-tj@kernel.org> Authentication-Results: smtp24.mail.ru; auth=pass smtp.auth=vdavydov@tarantool.org smtp.mailfrom=vdavydov@tarantool.org X-E1FCDC63: 32EF25A17406312B0D74BD136D86FE4FF253FDB7F52A2E4C X-E1FCDC64: 6980DBB58B4C2A881CD671EB85FFC86A31020030F48383E6BBFB72C787D86275 X-Mailru-Sender: AA5F055C295B4E991E00E7277EE5FAA73F497D02B157F9F3E617C474B75BC720669590CB21AA14C7 X-Mras: OK Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, Jan 14, 2017 at 12:54:45AM -0500, Tejun Heo wrote: > With kmem cgroup support enabled, kmem_caches can be created and > destroyed frequently and a great number of near empty kmem_caches can > accumulate if there are a lot of transient cgroups and the system is > not under memory pressure. When memory reclaim starts under such > conditions, it can lead to consecutive deactivation and destruction of > many kmem_caches, easily hundreds of thousands on moderately large > systems, exposing scalability issues in the current slab management > code. This is one of the patches to address the issue. > > While a memcg kmem_cache is listed on its root cache's ->children > list, there is no direct way to iterate all kmem_caches which are > assocaited with a memory cgroup. The only way to iterate them is > walking all caches while filtering out caches which don't match, which > would be most of them. > > This makes memcg destruction operations O(N^2) where N is the total > number of slab caches which can be huge. This combined with the > synchronous RCU operations can tie up a CPU and affect the whole > machine for many hours when memory reclaim triggers offlining and > destruction of the stale memcgs. > > This patch adds mem_cgroup->kmem_caches list which goes through > memcg_cache_params->kmem_caches_node of all kmem_caches which are > associated with the memcg. All memcg specific iterations, including > stat file access, are updated to use the new list instead. > > Signed-off-by: Tejun Heo > Reported-by: Jay Vana > Cc: Vladimir Davydov > Cc: Christoph Lameter > Cc: Pekka Enberg > Cc: David Rientjes > Cc: Joonsoo Kim > Cc: Andrew Morton Acked-by: Vladimir Davydov