From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50762137F for ; Fri, 1 Apr 2022 18:21:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F190C3410F; Fri, 1 Apr 2022 18:21:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1648837276; bh=j3G5qLSFoKcDWuso+x7UqSY0ZG6NlWyASPqJ1eYQrJE=; h=Date:To:From:In-Reply-To:Subject:From; b=xwVG9jsTYWxFeX0EIHI7Rm+8T6JoFAbTCBUwVjF6YU1EBYuJAq/blaFdWGYHb+mzG FGEDE75OS48xwkw1EUnHQajzjome11aexWWqGjgGMd1NRrAvN8oJ/eZIQMsFfEuCmM DGxIsPc7TwyS6sJ+vnQd+W+U3lGBoBCI7FFx5Kgk= Date: Fri, 01 Apr 2022 11:21:15 -0700 To: roman.gushchin@linux.dev,glider@google.com,elver@google.com,dvyukov@google.com,duanxiongchun@bytedance.com,songmuchun@bytedance.com,akpm@linux-foundation.org,patches@lists.linux.dev,linux-mm@kvack.org,mm-commits@vger.kernel.org,torvalds@linux-foundation.org,akpm@linux-foundation.org From: Andrew Morton In-Reply-To: Subject: [patch 09/16] mm: kfence: fix objcgs vector allocation Message-Id: <20220401182116.0F190C3410F@smtp.kernel.org> Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: From: Muchun Song Subject: mm: kfence: fix objcgs vector allocation If the kfence object is allocated to be used for objects vector, then this slot of the pool eventually being occupied permanently since the vector is never freed. The solutions could be 1) freeing vector when the kfence object is freed or 2) allocating all vectors statically. Since the memory consumption of object vectors is low, it is better to chose 2) to fix the issue and it is also can reduce overhead of vectors allocating in the future. Link: https://lkml.kernel.org/r/20220328132843.16624-1-songmuchun@bytedance.com Fixes: d3fb45f370d9 ("mm, kfence: insert KFENCE hooks for SLAB") Signed-off-by: Muchun Song Reviewed-by: Marco Elver Reviewed-by: Roman Gushchin Cc: Alexander Potapenko Cc: Dmitry Vyukov Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- mm/kfence/core.c | 11 ++++++++++- mm/kfence/kfence.h | 3 +++ 2 files changed, 13 insertions(+), 1 deletion(-) --- a/mm/kfence/core.c~mm-kfence-fix-objcgs-vector-allocation +++ a/mm/kfence/core.c @@ -566,6 +566,8 @@ static unsigned long kfence_init_pool(vo * enters __slab_free() slow-path. */ for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) { + struct slab *slab = page_slab(&pages[i]); + if (!i || (i % 2)) continue; @@ -573,7 +575,11 @@ static unsigned long kfence_init_pool(vo if (WARN_ON(compound_head(&pages[i]) != &pages[i])) return addr; - __SetPageSlab(&pages[i]); + __folio_set_slab(slab_folio(slab)); +#ifdef CONFIG_MEMCG + slab->memcg_data = (unsigned long)&kfence_metadata[i / 2 - 1].objcg | + MEMCG_DATA_OBJCGS; +#endif } /* @@ -1033,6 +1039,9 @@ void __kfence_free(void *addr) { struct kfence_metadata *meta = addr_to_metadata((unsigned long)addr); +#ifdef CONFIG_MEMCG + KFENCE_WARN_ON(meta->objcg); +#endif /* * If the objects of the cache are SLAB_TYPESAFE_BY_RCU, defer freeing * the object, as the object page may be recycled for other-typed --- a/mm/kfence/kfence.h~mm-kfence-fix-objcgs-vector-allocation +++ a/mm/kfence/kfence.h @@ -89,6 +89,9 @@ struct kfence_metadata { struct kfence_track free_track; /* For updating alloc_covered on frees. */ u32 alloc_stack_hash; +#ifdef CONFIG_MEMCG + struct obj_cgroup *objcg; +#endif }; extern struct kfence_metadata kfence_metadata[CONFIG_KFENCE_NUM_OBJECTS]; _ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5572C4332F for ; Fri, 1 Apr 2022 18:21:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349792AbiDASXJ (ORCPT ); Fri, 1 Apr 2022 14:23:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349432AbiDASXI (ORCPT ); Fri, 1 Apr 2022 14:23:08 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9570A38198 for ; Fri, 1 Apr 2022 11:21:18 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4E52FB82594 for ; Fri, 1 Apr 2022 18:21:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F190C3410F; Fri, 1 Apr 2022 18:21:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1648837276; bh=j3G5qLSFoKcDWuso+x7UqSY0ZG6NlWyASPqJ1eYQrJE=; h=Date:To:From:In-Reply-To:Subject:From; b=xwVG9jsTYWxFeX0EIHI7Rm+8T6JoFAbTCBUwVjF6YU1EBYuJAq/blaFdWGYHb+mzG FGEDE75OS48xwkw1EUnHQajzjome11aexWWqGjgGMd1NRrAvN8oJ/eZIQMsFfEuCmM DGxIsPc7TwyS6sJ+vnQd+W+U3lGBoBCI7FFx5Kgk= Date: Fri, 01 Apr 2022 11:21:15 -0700 To: roman.gushchin@linux.dev, glider@google.com, elver@google.com, dvyukov@google.com, duanxiongchun@bytedance.com, songmuchun@bytedance.com, akpm@linux-foundation.org, patches@lists.linux.dev, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, akpm@linux-foundation.org From: Andrew Morton In-Reply-To: Subject: [patch 09/16] mm: kfence: fix objcgs vector allocation Message-Id: <20220401182116.0F190C3410F@smtp.kernel.org> Precedence: bulk Reply-To: linux-kernel@vger.kernel.org List-ID: X-Mailing-List: mm-commits@vger.kernel.org From: Muchun Song Subject: mm: kfence: fix objcgs vector allocation If the kfence object is allocated to be used for objects vector, then this slot of the pool eventually being occupied permanently since the vector is never freed. The solutions could be 1) freeing vector when the kfence object is freed or 2) allocating all vectors statically. Since the memory consumption of object vectors is low, it is better to chose 2) to fix the issue and it is also can reduce overhead of vectors allocating in the future. Link: https://lkml.kernel.org/r/20220328132843.16624-1-songmuchun@bytedance.com Fixes: d3fb45f370d9 ("mm, kfence: insert KFENCE hooks for SLAB") Signed-off-by: Muchun Song Reviewed-by: Marco Elver Reviewed-by: Roman Gushchin Cc: Alexander Potapenko Cc: Dmitry Vyukov Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- mm/kfence/core.c | 11 ++++++++++- mm/kfence/kfence.h | 3 +++ 2 files changed, 13 insertions(+), 1 deletion(-) --- a/mm/kfence/core.c~mm-kfence-fix-objcgs-vector-allocation +++ a/mm/kfence/core.c @@ -566,6 +566,8 @@ static unsigned long kfence_init_pool(vo * enters __slab_free() slow-path. */ for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) { + struct slab *slab = page_slab(&pages[i]); + if (!i || (i % 2)) continue; @@ -573,7 +575,11 @@ static unsigned long kfence_init_pool(vo if (WARN_ON(compound_head(&pages[i]) != &pages[i])) return addr; - __SetPageSlab(&pages[i]); + __folio_set_slab(slab_folio(slab)); +#ifdef CONFIG_MEMCG + slab->memcg_data = (unsigned long)&kfence_metadata[i / 2 - 1].objcg | + MEMCG_DATA_OBJCGS; +#endif } /* @@ -1033,6 +1039,9 @@ void __kfence_free(void *addr) { struct kfence_metadata *meta = addr_to_metadata((unsigned long)addr); +#ifdef CONFIG_MEMCG + KFENCE_WARN_ON(meta->objcg); +#endif /* * If the objects of the cache are SLAB_TYPESAFE_BY_RCU, defer freeing * the object, as the object page may be recycled for other-typed --- a/mm/kfence/kfence.h~mm-kfence-fix-objcgs-vector-allocation +++ a/mm/kfence/kfence.h @@ -89,6 +89,9 @@ struct kfence_metadata { struct kfence_track free_track; /* For updating alloc_covered on frees. */ u32 alloc_stack_hash; +#ifdef CONFIG_MEMCG + struct obj_cgroup *objcg; +#endif }; extern struct kfence_metadata kfence_metadata[CONFIG_KFENCE_NUM_OBJECTS]; _