From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05B70CD37AC for ; Fri, 15 Sep 2023 21:06:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89AEC8D002F; Fri, 15 Sep 2023 17:06:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 82D738D0005; Fri, 15 Sep 2023 17:06:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C5A38D002F; Fri, 15 Sep 2023 17:06:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 57BB18D0005 for ; Fri, 15 Sep 2023 17:06:29 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2BD251CA96D for ; Fri, 15 Sep 2023 21:06:29 +0000 (UTC) X-FDA: 81240065298.23.EE57597 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf12.hostedemail.com (Postfix) with ESMTP id 5A1AF40018 for ; Fri, 15 Sep 2023 21:06:27 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ZRFWsTy2; spf=pass (imf12.hostedemail.com: domain of keescook@chromium.org designates 209.85.210.178 as permitted sender) smtp.mailfrom=keescook@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694811987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sWYMfZmYzixMw5Wj94BXAnknUexxKUO9Nr6vnuKcqis=; b=XtDbBpk/WXg3dazxbOEs3r9O8XQktc6xBXBUZal7bju61+3/2CZyp7z7wHIIMx9LdXo3Gg SiaKQXNkvWw24AU40CRsm3E6sbnlISP74nEPXMfAEvcqIzL4D701Eq4XNOExt1dcSeUTw2 eaTIRSgoqbUiFFwkczhIRKEagrqzb9k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694811987; a=rsa-sha256; cv=none; b=25EgUv85TW2nfKIIKmC9neA50CgHDzSPg3iDo99IfONo7c8DmZ739uIkWxQNGPSLe3Ht2x 57zyr7BMKB+imUP6su/17/OFEAM9hWZ1ZaQpKZ8/5JGcKiWLW2t6LkLCfxpceVAUqXUash PRnz0dAfWyaRSqZ/dhf6zLz1ny3Zc74= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ZRFWsTy2; spf=pass (imf12.hostedemail.com: domain of keescook@chromium.org designates 209.85.210.178 as permitted sender) smtp.mailfrom=keescook@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-68fac346f6aso2358619b3a.3 for ; Fri, 15 Sep 2023 14:06:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1694811986; x=1695416786; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=sWYMfZmYzixMw5Wj94BXAnknUexxKUO9Nr6vnuKcqis=; b=ZRFWsTy2BeUhRinOc5SZmC51VHWQgNLg3zGjAIV+gAuj9hRKDafGiQD9B7NX6m6cCx rxRmDfO9m6g46QLcuQE///hNsLQAMRU1rcMzN3r7peqoxP/htipt/uG5hcWj40I+v2vb PuCOmujAZ47lXY7f/l/Zh46tbxbR1xUN5DrJs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694811986; x=1695416786; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=sWYMfZmYzixMw5Wj94BXAnknUexxKUO9Nr6vnuKcqis=; b=Kgq6s1uHc3s3gRDbawlzFKw1iGEPdfb693yXBAggkwDG7dwe8oQevnja3l3OFbOAhB xa5wEfbuysVMhkFLfLNvtn4pZZGrctrl09QnevpLzZ8ACTdzzeOxxIwVTcVv7+AiPnKA 9SR8WnCIMN9A8+6DIa/nIweMqnW1nguPx/6269w+KpY/Hmor+wIDtbey+rbMDZ7nZVdf c+hg8FcW47cLaNuFEtMsd7OAT2O6CIXjnvlJk7XzeILWXzLBnlWa6YEQ8/6M1fGD/7DU cIvd2wtUWb7+20BGQmRqCZzbuWO+KYwMY9SFHSEda8dylWT0OtqNWhAwtkPR9yp3EMmz 7AgQ== X-Gm-Message-State: AOJu0YyShXZSxmzJjGkxCxkw0AUXZ/BQLGto6CpI1FkfLiCpGPbERqha Kha7Zf3jxLBLFvweEDL1IRRmCQ== X-Google-Smtp-Source: AGHT+IEqb9glFqR0LSiulXgUW5CKMVUOPBjgUHY3VfZDvGmOAuYzOtvUasxUFHUypis7+bgW99YIsQ== X-Received: by 2002:a05:6a21:6da2:b0:153:b245:50ff with SMTP id wl34-20020a056a216da200b00153b24550ffmr3685579pzb.40.1694811986264; Fri, 15 Sep 2023 14:06:26 -0700 (PDT) Received: from www.outflux.net (198-0-35-241-static.hfc.comcastbusiness.net. [198.0.35.241]) by smtp.gmail.com with ESMTPSA id l2-20020a17090aaa8200b00262ca945cecsm5149986pjq.54.2023.09.15.14.06.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 14:06:25 -0700 (PDT) Date: Fri, 15 Sep 2023 14:06:25 -0700 From: Kees Cook To: Matteo Rizzo Cc: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, corbet@lwn.net, luto@kernel.org, peterz@infradead.org, jannh@google.com, evn@google.com, poprdi@google.com, jordyzomer@google.com Subject: Re: [RFC PATCH 07/14] mm/slub: pass slab pointer to the freeptr decode helper Message-ID: <202309151404.B24C17C@keescook> References: <20230915105933.495735-1-matteorizzo@google.com> <20230915105933.495735-8-matteorizzo@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230915105933.495735-8-matteorizzo@google.com> X-Rspamd-Queue-Id: 5A1AF40018 X-Rspam-User: X-Stat-Signature: k4aaocafq3agjo7ey1ho3mq83byqomob X-Rspamd-Server: rspam03 X-HE-Tag: 1694811987-49796 X-HE-Meta: U2FsdGVkX19nqVdt6SroiygB+QXZa5d9GN/EwIgVqTCWq5ft3r97fqbNX2Zzb7uYXzZJvJMX883wT86eFDrqTV02yA4axuTtjJ7gicH8pWU+S2IHyfcwkNZCiZ0nUvSy+ogPCDzfqpOlEbT2+PVSlB1MJxKwzWt2sVzAc+ZBWtlN+dt6U0Nqy2kdcSg++nks0EDl6+T3dOSsPzNDsAJN7Qn9Ei7RnKWQpUXSKbXq8qWFc/C/rdVK/K+9ImjVcCWz8oGgxD9SJLY1xyjclJJ8z8Bv7SmjWzATyYsMYvG1Lp4WkRN3WQNmrGkwOSVd2vlsLWi0+wkocJ4WuLekzvDaBlmSQ/qlnuVj8qLiF/hiY0vDR/tELjjpCZhnBablIsmBrm0+KWtnwHJb91mFVHrYcHodx0d3vR1thXYZy2w8N5zAtj2lPJrIU3Td6sInIRMyzdSdZgLoVMbQmieCMWXrGwfyponxBqrCOM8ojY2rG4VuJ+sVrt/2d/3ZbhkqaN5E+SssHG2Eq1UlveiJ7Ue4BZgoo7eHBRhP65uOhwb/uQIEMl80eleBNt3SrCURat6Vb9jC8wzdCLoLLN/QSdc2s0o9esuJPsbSfdfwDaWdvseJ6iu9Is03rz6mzL6yOs0BWt7xPEy4YJVSQH0w/na7LzQk/DoWPZzRVRgAsqFDhfwDqJrij1SPYPe0UsIRGs2lSb5W6YIkbYm61Mtx5TMMKQV1dSGN2XNbLdz819UJEC5K47xD2ly78NbxIdKI31VAe1P9pt7V+Vp1tyR9JNroDtJeeHpeNOa4qNYLRJZ+go76HwENCvFu5Rxi3mZfSbX3kFiGKSOReplQ2mwCbsbkop6yvh+/Jed7LNlGb9uxh7iOabE6RnYFDNxPGbTS9lIoLz1Pcw9jUGh0G2b/0fPRHuKTJCpNa/nTCv2FsooFB1zVCUpHvOZ3jneJZ1nGboK+OPUS86rbQDqKwhmwHbu E3GHWPuk 3YNusEw7BJYBhlY46JFrzgJGUB9HYVmE/HZCZTomigUAlNmoSxL6FvBUkKpoPPCQhEQbK04f43Se6gOhtkmjpXnJTzo/2lFn7gKgg3pWjAJn2N51iVKUEaAXOMx18o16E/dT0bVeVIKkkmul4GwQS1HGUZGqKPQ0v8Y4ixy1+wgifwYKsvGD4CPdBKmUCwzgAtjVlUk+RCFgxgyP+Pmkehemt/zJkICnz6MiUbU4pdnug2LxG6+QfSvuIh9ZZHY3ZG7UdJwSmlBRnV4UhM4hjpQTHaZjoEymntg/o3Z4k6tVuP5jlQGT7Dlbm0TSdLWcxxRvUMfVkZ7UJ0uMVeTtlbcP+Lgin4YWsrGIPGlPdAO58ruRWLSl+MpP6aiWDexReDSlL/tM7cyk8j6PBj8GP2owmqG2PjocHwVOsS4aTUVNFiy+XvET9t4j6kwPTu7EcZBlU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 15, 2023 at 10:59:26AM +0000, Matteo Rizzo wrote: > From: Jann Horn > > This is refactoring in preparation for checking freeptrs for corruption > inside freelist_ptr_decode(). > > Signed-off-by: Jann Horn > Co-developed-by: Matteo Rizzo > Signed-off-by: Matteo Rizzo > --- > mm/slub.c | 43 +++++++++++++++++++++++-------------------- > 1 file changed, 23 insertions(+), 20 deletions(-) > > diff --git a/mm/slub.c b/mm/slub.c > index eaa1256aff89..42e7cc0b4452 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -383,7 +383,8 @@ static inline freeptr_t freelist_ptr_encode(const struct kmem_cache *s, > } > > static inline void *freelist_ptr_decode(const struct kmem_cache *s, > - freeptr_t ptr, unsigned long ptr_addr) > + freeptr_t ptr, unsigned long ptr_addr, > + struct slab *slab) Can't "s" be dropped in these cases, since "s" should be "slab->kmem_cache"? > { > void *decoded; > > @@ -395,7 +396,8 @@ static inline void *freelist_ptr_decode(const struct kmem_cache *s, > return decoded; > } > > -static inline void *get_freepointer(struct kmem_cache *s, void *object) > +static inline void *get_freepointer(struct kmem_cache *s, void *object, > + struct slab *slab) > { > unsigned long ptr_addr; > freeptr_t p; > @@ -403,7 +405,7 @@ static inline void *get_freepointer(struct kmem_cache *s, void *object) > object = kasan_reset_tag(object); > ptr_addr = (unsigned long)object + s->offset; > p = *(freeptr_t *)(ptr_addr); > - return freelist_ptr_decode(s, p, ptr_addr); > + return freelist_ptr_decode(s, p, ptr_addr, slab); > } > > #ifndef CONFIG_SLUB_TINY > @@ -424,18 +426,19 @@ static void prefetch_freepointer(const struct kmem_cache *s, void *object) > * get_freepointer_safe() returns initialized memory. > */ > __no_kmsan_checks > -static inline void *get_freepointer_safe(struct kmem_cache *s, void *object) > +static inline void *get_freepointer_safe(struct kmem_cache *s, void *object, > + struct slab *slab) > { > unsigned long freepointer_addr; > freeptr_t p; > > if (!debug_pagealloc_enabled_static()) > - return get_freepointer(s, object); > + return get_freepointer(s, object, slab); > > object = kasan_reset_tag(object); > freepointer_addr = (unsigned long)object + s->offset; > copy_from_kernel_nofault(&p, (freeptr_t *)freepointer_addr, sizeof(p)); > - return freelist_ptr_decode(s, p, freepointer_addr); > + return freelist_ptr_decode(s, p, freepointer_addr, slab); > } > > static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) > @@ -627,7 +630,7 @@ static void __fill_map(unsigned long *obj_map, struct kmem_cache *s, > > bitmap_zero(obj_map, slab->objects); > > - for (p = slab->freelist; p; p = get_freepointer(s, p)) > + for (p = slab->freelist; p; p = get_freepointer(s, p, slab)) > set_bit(__obj_to_index(s, addr, p), obj_map); > } > > @@ -937,7 +940,7 @@ static void print_trailer(struct kmem_cache *s, struct slab *slab, u8 *p) > print_slab_info(slab); > > pr_err("Object 0x%p @offset=%tu fp=0x%p\n\n", > - p, p - addr, get_freepointer(s, p)); > + p, p - addr, get_freepointer(s, p, slab)); > > if (s->flags & SLAB_RED_ZONE) > print_section(KERN_ERR, "Redzone ", p - s->red_left_pad, > @@ -1230,7 +1233,7 @@ static int check_object(struct kmem_cache *s, struct slab *slab, > return 1; > > /* Check free pointer validity */ > - if (!check_valid_pointer(s, slab, get_freepointer(s, p))) { > + if (!check_valid_pointer(s, slab, get_freepointer(s, p, slab))) { > object_err(s, slab, p, "Freepointer corrupt"); > /* > * No choice but to zap it and thus lose the remainder > @@ -1298,7 +1301,7 @@ static int on_freelist(struct kmem_cache *s, struct slab *slab, void *search) > break; > } > object = fp; > - fp = get_freepointer(s, object); > + fp = get_freepointer(s, object, slab); > nr++; > } > > @@ -1810,7 +1813,7 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, > object = next; > /* Single objects don't actually contain a freepointer */ > if (object != old_tail) > - next = get_freepointer(s, object); > + next = get_freepointer(s, object, virt_to_slab(object)); > > /* If object's reuse doesn't have to be delayed */ > if (!slab_free_hook(s, object, slab_want_init_on_free(s))) { > @@ -2161,7 +2164,7 @@ static void *alloc_single_from_partial(struct kmem_cache *s, > lockdep_assert_held(&n->list_lock); > > object = slab->freelist; > - slab->freelist = get_freepointer(s, object); > + slab->freelist = get_freepointer(s, object, slab); > slab->inuse++; > > if (!alloc_debug_processing(s, slab, object, orig_size)) { > @@ -2192,7 +2195,7 @@ static void *alloc_single_from_new_slab(struct kmem_cache *s, > > > object = slab->freelist; > - slab->freelist = get_freepointer(s, object); > + slab->freelist = get_freepointer(s, object, slab); > slab->inuse = 1; > > if (!alloc_debug_processing(s, slab, object, orig_size)) > @@ -2517,7 +2520,7 @@ static void deactivate_slab(struct kmem_cache *s, struct slab *slab, > freelist_tail = NULL; > freelist_iter = freelist; > while (freelist_iter) { > - nextfree = get_freepointer(s, freelist_iter); > + nextfree = get_freepointer(s, freelist_iter, slab); > > /* > * If 'nextfree' is invalid, it is possible that the object at > @@ -2944,7 +2947,7 @@ static inline bool free_debug_processing(struct kmem_cache *s, > > /* Reached end of constructed freelist yet? */ > if (object != tail) { > - object = get_freepointer(s, object); > + object = get_freepointer(s, object, slab); > goto next_object; > } > checks_ok = true; > @@ -3173,7 +3176,7 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, > * That slab must be frozen for per cpu allocations to work. > */ > VM_BUG_ON(!c->slab->frozen); > - c->freelist = get_freepointer(s, freelist); > + c->freelist = get_freepointer(s, freelist, c->slab); > c->tid = next_tid(c->tid); > local_unlock_irqrestore(&s->cpu_slab->lock, flags); > return freelist; > @@ -3275,7 +3278,7 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, > * For !pfmemalloc_match() case we don't load freelist so that > * we don't make further mismatched allocations easier. > */ > - deactivate_slab(s, slab, get_freepointer(s, freelist)); > + deactivate_slab(s, slab, get_freepointer(s, freelist, slab)); > return freelist; > } > > @@ -3377,7 +3380,7 @@ static __always_inline void *__slab_alloc_node(struct kmem_cache *s, > unlikely(!object || !slab || !node_match(slab, node))) { > object = __slab_alloc(s, gfpflags, node, addr, c, orig_size); > } else { > - void *next_object = get_freepointer_safe(s, object); > + void *next_object = get_freepointer_safe(s, object, slab); > > /* > * The cmpxchg will only match if there was no additional > @@ -3984,7 +3987,7 @@ static inline int __kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, > > continue; /* goto for-loop */ > } > - c->freelist = get_freepointer(s, object); > + c->freelist = get_freepointer(s, object, c->slab); > p[i] = object; > maybe_wipe_obj_freeptr(s, p[i]); > } > @@ -4275,7 +4278,7 @@ static void early_kmem_cache_node_alloc(int node) > init_tracking(kmem_cache_node, n); > #endif > n = kasan_slab_alloc(kmem_cache_node, n, GFP_KERNEL, false); > - slab->freelist = get_freepointer(kmem_cache_node, n); > + slab->freelist = get_freepointer(kmem_cache_node, n, slab); > slab->inuse = 1; > kmem_cache_node->node[node] = n; > init_kmem_cache_node(n); > -- > 2.42.0.459.ge4e396fd5e-goog > -- Kees Cook