From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 804F5C433DB for ; Mon, 8 Mar 2021 11:47:50 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C184A64DDA for ; Mon, 8 Mar 2021 11:47:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C184A64DDA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=r2i2H57m8cAQdM4+sGMIogdzNoDN41NrYj9skyeMIgo=; b=nwPvC7wWdCm9MGyorzQAHARS6 5acj/DUoxnmf62wBMQa6UUwfwHuPblOrpbxJbOLPWXE7DRX1a3JE+l12UqJSd5RRCyYTR1efvMMS8 mKQkj/KNn39EP0InrkV3TiDKUE9mLQNQckoNMibSCj8BFxlKOkmbFMX2LfgW/pcNEzaNp/YBWEydZ Oo5MR0Nh94GWQCZVsAxa8AuYlFctF/HsndLNrkhfXtk/byryeFhID7LOgv2UtVeTscvba4YfCkBOl TFyQs+j2Mr4niKE5Z7LlyJHpuT3kgPrJ8JX+ySqMd7TlWoujQwpbIAa5vfxZt0cI5se0vO9mx3lPF VJLnPzc2Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lJEKn-00Gqk5-11; Mon, 08 Mar 2021 11:46:01 +0000 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lJEKf-00Gqha-OI for linux-arm-kernel@lists.infradead.org; Mon, 08 Mar 2021 11:45:57 +0000 Received: by mail-wr1-x430.google.com with SMTP id f12so11093433wrx.8 for ; Mon, 08 Mar 2021 03:45:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=cCYaodgn23RpIhtLxdUS0nQYEGZQi2gHsnzCEWxqT00=; b=LnjULmcfZdIhPoi+O0rkNKPVbzSSqaT/HB91JBDOs4Z4GCjdr7uiw926tDU+oHaWDZ JCsqzhLPLY4CNjUWgldVxKCPT+Lp+96EM6UWZFzm4wC28AT71vchFa2/1fZOJG9MLNQq vurZGIPsqbAPCk8xczDRm16/JQrWeS+Jinlk3smweNIqWR66/Xrl62izNPtT2KN6xJJO jN0/e7CXjwkYM5JsPEgqpzqctGCvXxSHoyNHYZxjvP13hQVu8gOql4uaGA3xFFcm5Ye6 m1FGQpX2opcLsyFc5K/lGh06/w9E+n6PliOAcYVQ1ZYx9xUC/p7E8/utoByBKvejFRcI hmwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=cCYaodgn23RpIhtLxdUS0nQYEGZQi2gHsnzCEWxqT00=; b=CbTMA6pE+h0k2X+ge2d9qhN9+KaSECj/MMJ3qvMFh87bgEvebGxLQJU24VmoMQIhCg ewIUx11KCGnaoAck/2Qzq/gSQBvrf37h1jcT4eYK1Hw5geZYTDIvNIwZmcZyGWA9A06Y F4Fi2fFuqfYbQm6ZMGbKhLVs66nmX42FbA9kcV1v6yxrL7y3IE05dGwVah2dTRrLRSbc k+AA5aiY+kWAruZysRTpSzTUG9lTjCooGmNTPvZ78PMY3XBRstxHvYN2WS0agW9ka/Ha MYexkOWHHxDWzHsaon6pKzUx1fpqB6s6TOygSC8M1HDU8USQTRmtXlJLg5LodOewnzxd NQ/Q== X-Gm-Message-State: AOAM530TYNIXhUbLzjhpiGA2dWxtcTLmxlfgJiKUnvl2bB4PV+HLak/A NzTrwXqvxaf/whIA4Zw6caJv/Q== X-Google-Smtp-Source: ABdhPJwRSPlTzcZm0ECLRCu1KHbIaQ2MJNcKKoSiOlHJpmNR8Av1XU9V0Ii3vTqTzePs6P4gMqjaqw== X-Received: by 2002:adf:f148:: with SMTP id y8mr21886984wro.107.1615203952103; Mon, 08 Mar 2021 03:45:52 -0800 (PST) Received: from elver.google.com ([2a00:79e0:15:13:9d1d:b6a0:d116:531b]) by smtp.gmail.com with ESMTPSA id p16sm21167989wrt.54.2021.03.08.03.45.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Mar 2021 03:45:51 -0800 (PST) Date: Mon, 8 Mar 2021 12:45:45 +0100 From: Marco Elver To: Andrey Konovalov Cc: Alexander Potapenko , Andrew Morton , Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Peter Collingbourne , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 5/5] kasan, mm: integrate slab init_on_free with HW_TAGS Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/2.0.5 (2021-01-21) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210308_114554_108552_C050C20E X-CRM114-Status: GOOD ( 36.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sat, Mar 06, 2021 at 01:15AM +0100, Andrey Konovalov wrote: > This change uses the previously added memory initialization feature > of HW_TAGS KASAN routines for slab memory when init_on_free is enabled. > > With this change, memory initialization memset() is no longer called > when both HW_TAGS KASAN and init_on_free are enabled. Instead, memory > is initialized in KASAN runtime. > > For SLUB, the memory initialization memset() is moved into > slab_free_hook() that currently directly follows the initialization loop. > A new argument is added to slab_free_hook() that indicates whether to > initialize the memory or not. > > To avoid discrepancies with which memory gets initialized that can be > caused by future changes, both KASAN hook and initialization memset() > are put together and a warning comment is added. > > Combining setting allocation tags with memory initialization improves > HW_TAGS KASAN performance when init_on_free is enabled. > > Signed-off-by: Andrey Konovalov > --- > include/linux/kasan.h | 10 ++++++---- > mm/kasan/common.c | 13 +++++++------ > mm/slab.c | 15 +++++++++++---- > mm/slub.c | 43 ++++++++++++++++++++++++------------------- > 4 files changed, 48 insertions(+), 33 deletions(-) > > diff --git a/include/linux/kasan.h b/include/linux/kasan.h > index bb756f6c73b5..1df0f7f0b493 100644 > --- a/include/linux/kasan.h > +++ b/include/linux/kasan.h > @@ -193,11 +193,13 @@ static __always_inline void * __must_check kasan_init_slab_obj( > return (void *)object; > } > > -bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); > -static __always_inline bool kasan_slab_free(struct kmem_cache *s, void *object) > +bool __kasan_slab_free(struct kmem_cache *s, void *object, > + unsigned long ip, bool init); > +static __always_inline bool kasan_slab_free(struct kmem_cache *s, > + void *object, bool init) > { > if (kasan_enabled()) > - return __kasan_slab_free(s, object, _RET_IP_); > + return __kasan_slab_free(s, object, _RET_IP_, init); > return false; > } > > @@ -299,7 +301,7 @@ static inline void *kasan_init_slab_obj(struct kmem_cache *cache, > { > return (void *)object; > } > -static inline bool kasan_slab_free(struct kmem_cache *s, void *object) > +static inline bool kasan_slab_free(struct kmem_cache *s, void *object, bool init) > { > return false; > } > diff --git a/mm/kasan/common.c b/mm/kasan/common.c > index 7ea747b18c26..623cf94288a2 100644 > --- a/mm/kasan/common.c > +++ b/mm/kasan/common.c > @@ -322,8 +322,8 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, > return (void *)object; > } > > -static inline bool ____kasan_slab_free(struct kmem_cache *cache, > - void *object, unsigned long ip, bool quarantine) > +static inline bool ____kasan_slab_free(struct kmem_cache *cache, void *object, > + unsigned long ip, bool quarantine, bool init) > { > u8 tag; > void *tagged_object; > @@ -351,7 +351,7 @@ static inline bool ____kasan_slab_free(struct kmem_cache *cache, > } > > kasan_poison(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), > - KASAN_KMALLOC_FREE, false); > + KASAN_KMALLOC_FREE, init); > > if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine)) > return false; > @@ -362,9 +362,10 @@ static inline bool ____kasan_slab_free(struct kmem_cache *cache, > return kasan_quarantine_put(cache, object); > } > > -bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) > +bool __kasan_slab_free(struct kmem_cache *cache, void *object, > + unsigned long ip, bool init) > { > - return ____kasan_slab_free(cache, object, ip, true); > + return ____kasan_slab_free(cache, object, ip, true, init); > } > > static inline bool ____kasan_kfree_large(void *ptr, unsigned long ip) > @@ -409,7 +410,7 @@ void __kasan_slab_free_mempool(void *ptr, unsigned long ip) > return; > kasan_poison(ptr, page_size(page), KASAN_FREE_PAGE, false); > } else { > - ____kasan_slab_free(page->slab_cache, ptr, ip, false); > + ____kasan_slab_free(page->slab_cache, ptr, ip, false, false); > } > } > > diff --git a/mm/slab.c b/mm/slab.c > index 936dd686dec9..d12ce9e5c3ed 100644 > --- a/mm/slab.c > +++ b/mm/slab.c > @@ -3425,17 +3425,24 @@ static void cache_flusharray(struct kmem_cache *cachep, struct array_cache *ac) > static __always_inline void __cache_free(struct kmem_cache *cachep, void *objp, > unsigned long caller) > { > + bool init; > + > if (is_kfence_address(objp)) { > kmemleak_free_recursive(objp, cachep->flags); > __kfence_free(objp); > return; > } > > - if (unlikely(slab_want_init_on_free(cachep))) > + /* > + * As memory initialization is integrated with hardware tag-based This may no longer be true if the HW-tags architecture doesn't support init (although currently it is certainly true). Perhaps: "As memory initialization may be accelerated by some KASAN implementations (such as some HW_TAGS architectures) ..." or whatever else is appropriate. > + * KASAN, kasan_slab_free and initialization memset must be > + * kept together to avoid discrepancies in behavior. > + */ > + init = slab_want_init_on_free(cachep); > + if (init && !IS_ENABLED(CONFIG_KASAN_HW_TAGS)) > memset(objp, 0, cachep->object_size); Same as the other patch, it seems awkward to have !IS_ENABLED(CONFIG_KASAN_HW_TAGS) rather than a kasan_*() helper that returns this information. > - > - /* Put the object into the quarantine, don't touch it for now. */ > - if (kasan_slab_free(cachep, objp)) > + /* KASAN might put objp into memory quarantine, delaying its reuse. */ > + if (kasan_slab_free(cachep, objp, init)) > return; > > /* Use KCSAN to help debug racy use-after-free. */ > diff --git a/mm/slub.c b/mm/slub.c > index f53df23760e3..c2755670d6bd 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -1532,7 +1532,8 @@ static __always_inline void kfree_hook(void *x) > kasan_kfree_large(x); > } > > -static __always_inline bool slab_free_hook(struct kmem_cache *s, void *x) > +static __always_inline bool slab_free_hook(struct kmem_cache *s, > + void *x, bool init) > { > kmemleak_free_recursive(x, s->flags); > > @@ -1558,8 +1559,25 @@ static __always_inline bool slab_free_hook(struct kmem_cache *s, void *x) > __kcsan_check_access(x, s->object_size, > KCSAN_ACCESS_WRITE | KCSAN_ACCESS_ASSERT); > > - /* KASAN might put x into memory quarantine, delaying its reuse */ > - return kasan_slab_free(s, x); > + /* > + * As memory initialization is integrated with hardware tag-based > + * KASAN, kasan_slab_free and initialization memset's must be > + * kept together to avoid discrepancies in behavior. > + * > + * The initialization memset's clear the object and the metadata, > + * but don't touch the SLAB redzone. > + */ > + if (init) { > + int rsize; > + > + if (!IS_ENABLED(CONFIG_KASAN_HW_TAGS)) > + memset(kasan_reset_tag(x), 0, s->object_size); > + rsize = (s->flags & SLAB_RED_ZONE) ? s->red_left_pad : 0; > + memset((char *)kasan_reset_tag(x) + s->inuse, 0, > + s->size - s->inuse - rsize); > + } > + /* KASAN might put x into memory quarantine, delaying its reuse. */ > + return kasan_slab_free(s, x, init); > } > > static inline bool slab_free_freelist_hook(struct kmem_cache *s, > @@ -1569,10 +1587,9 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, > void *object; > void *next = *head; > void *old_tail = *tail ? *tail : *head; > - int rsize; > > if (is_kfence_address(next)) { > - slab_free_hook(s, next); > + slab_free_hook(s, next, false); > return true; > } > > @@ -1584,20 +1601,8 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, > object = next; > next = get_freepointer(s, object); > > - if (slab_want_init_on_free(s)) { > - /* > - * Clear the object and the metadata, but don't touch > - * the redzone. > - */ > - memset(kasan_reset_tag(object), 0, s->object_size); > - rsize = (s->flags & SLAB_RED_ZONE) ? s->red_left_pad > - : 0; > - memset((char *)kasan_reset_tag(object) + s->inuse, 0, > - s->size - s->inuse - rsize); > - > - } > /* If object's reuse doesn't have to be delayed */ > - if (!slab_free_hook(s, object)) { > + if (!slab_free_hook(s, object, slab_want_init_on_free(s))) { > /* Move object to the new freelist */ > set_freepointer(s, object, *head); > *head = object; > @@ -3235,7 +3240,7 @@ int build_detached_freelist(struct kmem_cache *s, size_t size, > } > > if (is_kfence_address(object)) { > - slab_free_hook(df->s, object); > + slab_free_hook(df->s, object, false); > __kfence_free(object); > p[size] = NULL; /* mark object processed */ > return size; > -- > 2.30.1.766.gb4fecdf3b7-goog > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel