From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A487DC433EF for ; Tue, 19 Jul 2022 11:30:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233767AbiGSLay (ORCPT ); Tue, 19 Jul 2022 07:30:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231491AbiGSLay (ORCPT ); Tue, 19 Jul 2022 07:30:54 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DE5D60CA for ; Tue, 19 Jul 2022 04:30:53 -0700 (PDT) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id D0D3A3510A; Tue, 19 Jul 2022 11:30:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1658230251; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gEeCCLSta3HcRHaPk6iGTZCTxTMRWXyTHqEYQ05Ux9w=; b=CVeCxGYYUGLBzabGk0Qenp6GKA/EIm671QkK9E8AKYbh4yLJY9CWEkKLyii4XtALul/0JP Z1qQmhICaixrcdJzX727B+/FUBmszeOirlXsIWPYhbiiltq6C2XA2FB+LqaqjaoTC/YfrD iy8wPkjA0tjzeXUDx0MateWjxmj73vI= Received: from suse.cz (unknown [10.100.201.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 81FEA2C141; Tue, 19 Jul 2022 11:30:50 +0000 (UTC) Date: Tue, 19 Jul 2022 13:30:49 +0200 From: Michal Hocko To: Yosry Ahmed Cc: Roman Gushchin , Yafang Shao , Alexei Starovoitov , Shakeel Butt , Matthew Wilcox , Christoph Hellwig , "David S. Miller" , Daniel Borkmann , Andrii Nakryiko , Tejun Heo , Martin KaFai Lau , bpf , Kernel Team , linux-mm , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka Subject: cgroup specific sticky resources (was: Re: [PATCH bpf-next 0/5] bpf: BPF specific memory allocator.) Message-ID: References: <20220708215536.pqclxdqvtrfll2y4@google.com> <20220710073213.bkkdweiqrlnr35sv@google.com> <20220712043914.pxmbm7vockuvpmmh@macbook-pro-3.dhcp.thefacebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Mon 18-07-22 10:55:59, Yosry Ahmed wrote: > On Tue, Jul 12, 2022 at 7:39 PM Roman Gushchin wrote: > > > > On Tue, Jul 12, 2022 at 11:52:11AM +0200, Michal Hocko wrote: > > > On Tue 12-07-22 16:39:48, Yafang Shao wrote: > > > > On Tue, Jul 12, 2022 at 3:40 PM Michal Hocko wrote: > > > [...] > > > > > > Roman already sent reparenting fix: > > > > > > https://patchwork.kernel.org/project/netdevbpf/patch/20220711162827.184743-1-roman.gushchin@linux.dev/ > > > > > > > > > > Reparenting is nice but not a silver bullet. Consider a shallow > > > > > hierarchy where the charging happens in the first level under the root > > > > > memcg. Reparenting to the root is just pushing everything under the > > > > > system resources category. > > > > > > > > > > > > > Agreed. That's why I don't like reparenting. > > > > Reparenting just reparent the charged pages and then redirect the new > > > > charge, but can't reparents the 'limit' of the original memcg. > > > > So it is a risk if the original memcg is still being charged. We have > > > > to forbid the destruction of the original memcg. > > > > I agree, I also don't like reparenting for !kmem case. For kmem (and *maybe* > > bpf maps is an exception), I don't think there is a better choice. > > > > > yes, I was toying with an idea like that. I guess we really want a > > > measure to keep cgroups around if they are bound to a resource which is > > > sticky itself. I am not sure how many other resources like BPF (aka > > > module like) we already do charge for memcg but considering the > > > potential memory consumption just reparenting will not help in general > > > case I am afraid. > > > > Well, then we have to make these objects a first-class citizens in cgroup API, > > like processes. E.g. introduce cgroup.bpf.maps, cgroup.mounts.tmpfs etc. > > I easily can see some value here, but it's a big API change. > > > > With the current approach when a bpf map pins a memory cgroup of the creator > > process (which I think is completely transparent for most bpf users), I don't > > think preventing the deletion of a such cgroup is possible. It will break too > > many things. > > > > But honestly I don't see why userspace can't handle it. If there is a cgroup which > > contains shared bpf maps, why would it delete it? It's a weird use case, I don't > > think we have to optimize for it. Also, we do a ton of optimizations for live > > cgroups (e.g. css refcounting being percpu) which are not working for a deleted > > cgroup. So noone really should expect any properties from dying cgroups. > > > > Just a random thought here, and I can easily be wrong (and this can > easily be the wrong thread for this), but if we introduce a more > generic concept to generally tie a resource explicitly to a cgroup > (tmpfs, bpf maps, etc) using cgroupfs interfaces, and then prevent the > cgroup from being deleted unless the resource is freed or moved to a > different cgroup? My understanding is that Tejun would prefer a user space defined policy by a proper layering. And I would tend to agree that this is less prone to corner cases. Anyway, how would you envision such an interface? > This would be optional, so the current status quo is maintainable, but > also gives flexibility to admins to assign resources to cgroups to > make sure nothing is ( unaccounted / accounted to a zombie memcg / > reparented to an unrelated parent ). This might be too fine-grained to > be practical but I just thought it might be useful. We will also need > to define an OOM behavior for such resources. Things like bpf maps > will be unreclaimable, but tmpfs memory can be swapped out. Keep in mind that the swap is a shared resource in itself. So tmpfs is essentially a sticky resource as well. A tmpfs file is not bound to any proces life time the same way BPF program is. You might need less priviledges to remove a file but in principle they are consuming resources without any explicit owner. -- Michal Hocko SUSE Labs