From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64142C4345F for ; Tue, 30 Apr 2024 11:35:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E990D6B007B; Tue, 30 Apr 2024 07:35:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E24326B0083; Tue, 30 Apr 2024 07:35:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC3C66B0085; Tue, 30 Apr 2024 07:35:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id ACAAA6B007B for ; Tue, 30 Apr 2024 07:35:06 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 669ED160522 for ; Tue, 30 Apr 2024 11:35:06 +0000 (UTC) X-FDA: 82065991812.11.2D17DA0 Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by imf23.hostedemail.com (Postfix) with ESMTP id 8F75514000E for ; Tue, 30 Apr 2024 11:35:04 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=clip-os.org header.s=gm1 header.b=nM0URJ6x; dmarc=pass (policy=none) header.from=clip-os.org; spf=pass (imf23.hostedemail.com: domain of nicolas.bouchinet@clip-os.org designates 217.70.183.195 as permitted sender) smtp.mailfrom=nicolas.bouchinet@clip-os.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714476905; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=M0gCyZUgKj7iV3h/gwYac96jecnR+4EjQmwFBZXj584=; b=nOrVJ0l6AV+FVvSFXXz2I787fO5nZIS/cjyQgoPZO3qcOxi6ewy4QXMOS3AQFUldEOkgTK EoMk9mGsli+qpBy3acysVMecb2yOCmEFz4uJo+WyTi6LuFik8EbWwOLq412Njuqvui1w2i w6RQfZqMoPbkk/EJ7CDAgWPsoKLGl24= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=clip-os.org header.s=gm1 header.b=nM0URJ6x; dmarc=pass (policy=none) header.from=clip-os.org; spf=pass (imf23.hostedemail.com: domain of nicolas.bouchinet@clip-os.org designates 217.70.183.195 as permitted sender) smtp.mailfrom=nicolas.bouchinet@clip-os.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714476905; a=rsa-sha256; cv=none; b=XxzIYXkZbVWJtgP4nKiH6sMFJigz6SVPqro5H+BElRHAZnbQ2WZw/Sxi6J4YWqnFn4P5o/ Fk4Ems1gS8awA6FZ1DuAUVpQWmEiuqiNAEHdf0mBBRlscF5k7+zM7Qo5ioq1BVXD/OU1KC kr/7SuHJjLVpJdswEekzT1oo3YnV1ws= Received: by mail.gandi.net (Postfix) with ESMTPSA id 560CF60007; Tue, 30 Apr 2024 11:35:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=clip-os.org; s=gm1; t=1714476902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=M0gCyZUgKj7iV3h/gwYac96jecnR+4EjQmwFBZXj584=; b=nM0URJ6xiUOjlSg9J/l+2jP270ghHlvF+mpUuOm4n3R6s7J4d88Qx6Bg1Tx6+nyYwYexaP 5g7NiwAQ1S/X4G5VFVYSZHIr9Ky1615t2aUuYAGOulK9Qq3Uc8gp+fSzsYHyqOVZmHeMuZ 69eiP8stHv8cRpBpzTwjyU7Q5k6zQ1KwtnpnHuTjzzGe7TvF49shxJVREcZRoSjTbT6/N5 KBR459cNEASDd2k+pLdFaEod9GWNS+Y0HjS9qPpA/labyVLo2e8TAn9SQEFtJQcDkorLX5 m87rGXS34s+kA/NLRGbMlBwJcOAJzvTX8FsU4K4yfUGZ8mgTLEzJBMRYOjv+tA== Date: Tue, 30 Apr 2024 13:34:59 +0200 From: Nicolas Bouchinet To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Chengming Zhou , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH v3] slub: Fixes freepointer encoding for single free Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-GND-Sasl: nicolas.bouchinet@clip-os.org X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8F75514000E X-Stat-Signature: 6k7axkoowg339bgrja1o784q8y3z9ftq X-Rspam-User: X-HE-Tag: 1714476904-370340 X-HE-Meta: U2FsdGVkX18zLlAcGkTfZ/SRDTyHZ6xZ3OWholu8vopXDIBkMO5sFoHCaWd4G/YnJutXskp7V8EQvOYqnZIRDIwFqIauVAYUz7bqJjxBQagOSshKQ4/hPlNPwMO2g3pd2AkA95YsPixD4R0axwYyzFJz0Q+3szNAJUWc1OZ5thBGdJI+JDX7zbIrDIs3o2lp0oNxIiJIA6W4cO0nhPeHeJcqblBoLAS4/c4xmS5FY+Kmdw5xNvBfb789HXvIPbcpt71KUX65HOm36dGaLlOKY7TGc7uMpWebKEVKnE/cqYDWuDRGOO0/utcUtB1LRVEVhPSq71HRzVEvnRDdp7FH7uOuu6kXrFTIhXiDW/S8uLV/3E/Qp8QVo2WOOr3/yC+w1bI7CyUIKO5yIhAYb9GDuVCHuv4pWHdWpDtIxMGJ9r6vnKYsf9awb1T5iHP6nl6Dij/iUY8BdR0Neyt/8zUfiAaNBBxhxbQ4UEJ5iqoZjgTSXscJ3+8hIJOW68z0nEIczJTciz05d8d9Z6DIfVVgwKx7muBCgQlRSsdb1fNrORoIBLBi0KM/0uu35oeoQWYiSZqK0lPSGV1oqFK9gn97oMRVgb6CVB9bDN0Dicvz4rxFsTg9TwJxvwUFwK20WRh4xiHsXRaRoIL9jV/dxMaLoBqftFPp7jMk7nIXjX4tvx/sT62/s+lmWZlK+ulE3eMf2pspwgPoNdLk2rluK4cdh5hcNeq/2DhukGO7++XtGBb3mBRfI8hCGsOwu++Bno1bnvMthVikqvHJUmS3LaSSOp1MmhqWpMO3n2QIdF6k7R4HnMIFuPYr+iNXBI/oTzkj5t02/iKuDOI7tvcBDLdSXco6Atq+Jcn3YI8ZaR5ef/z9s/bH4NlMtW13dkcKbavyamOufhIXbd1otviv+LJOqpeiC++PYyS0zKoUeYbwd6W3P58nUnuBcgqIaeR8iK9L1BDhAjLe+Yjb8s+Rm51 onvfQaj+ Yru4zZDi+bzELxxyk7FvytIV2CHreYr3iiSPTntH/EyOQ8H4XF6HJBjgOfkUTeT7z8QwY5zsiOmzCtbtc5S8PculDWhJhtdk0MxOvDsdjBStG7JMWe9h3+IlVFpkG8TyHTqxWA+FZ73yOGlO6BVHobXlUcM/kEU4+AItIjq02MpLJY3ipYDRUuKaa4WwtyI+SmgHSJnaxRivKjNF1QD95VHuhrQEOV7C5hjgdEGfMYMthZjq6+pQ0ZsyOfwblWRXyCkR3AuTNx3/EJT9CiWJ2Y62x8xK+ct16HGeHyfUgwc2rn75wQSQWoSrs+g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Nicolas Bouchinet Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing separately") splits single and bulk object freeing in two functions slab_free() and slab_free_bulk() which leads slab_free() to call slab_free_hook() directly instead of slab_free_freelist_hook(). If `init_on_free` is set, slab_free_hook() zeroes the object. Afterward, if `slub_debug=F` and `CONFIG_SLAB_FREELIST_HARDENED` are set, the do_slab_free() slowpath executes freelist consistency checks and try to decode a zeroed freepointer which leads to a "Freepointer corrupt" detection in check_object(). During bulk free, slab_free_freelist_hook() isn't affected as it always sets it objects freepointer using set_freepointer() to maintain its reconstructed freelist after `init_on_free`. For single free, object's freepointer thus needs to be avoided when stored outside the object if `init_on_free` is set. The freepointer left as is, check_object() may later detect an invalid pointer value due to objects overflow. To reproduce, set `slub_debug=FU init_on_free=1 log_level=7` on the command line of a kernel build with `CONFIG_SLAB_FREELIST_HARDENED=y`. dmesg sample log: [ 10.708715] ============================================================================= [ 10.710323] BUG kmalloc-rnd-05-32 (Tainted: G B T ): Freepointer corrupt [ 10.712695] ----------------------------------------------------------------------------- [ 10.712695] [ 10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2) [ 10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c [ 10.716698] [ 10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.720703] Object ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 10.724696] Padding ffff9d9a8035667c: 00 00 00 00 .... [ 10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed Co-developed-by: Chengming Zhou Signed-off-by: Nicolas Bouchinet --- Changes since v2: https://lore.kernel.org/all/ZjCxZfD1d36zfq-R@archlinux/ * Reword commit message in order to clarify the patch approach as suggested by Vlastimil Babka Changes since v1: https://lore.kernel.org/all/Zij_fGjRS_rK-65r@archlinux/ * Jump above out of object freepointer if init_on_free is set instead of initializing it with set_freepointer() as suggested by Vlastimil Babka. * Adapt maybe_wipe_obj_freeptr() to avoid wiping out of object on alloc freepointer as suggested by Chengming Zhou. * Reword commit message. --- mm/slub.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 3aa12b9b323d..173c340ec1d3 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2102,15 +2102,20 @@ bool slab_free_hook(struct kmem_cache *s, void *x, bool init) * * The initialization memset's clear the object and the metadata, * but don't touch the SLAB redzone. + * + * The object's freepointer is also avoided if stored outside the + * object. */ if (unlikely(init)) { int rsize; + unsigned int inuse; + inuse = get_info_end(s); if (!kasan_has_integrated_init()) memset(kasan_reset_tag(x), 0, s->object_size); rsize = (s->flags & SLAB_RED_ZONE) ? s->red_left_pad : 0; - memset((char *)kasan_reset_tag(x) + s->inuse, 0, - s->size - s->inuse - rsize); + memset((char *)kasan_reset_tag(x) + inuse, 0, + s->size - inuse - rsize); } /* KASAN might put x into memory quarantine, delaying its reuse. */ return !kasan_slab_free(s, x, init); @@ -3789,7 +3794,7 @@ static void *__slab_alloc_node(struct kmem_cache *s, static __always_inline void maybe_wipe_obj_freeptr(struct kmem_cache *s, void *obj) { - if (unlikely(slab_want_init_on_free(s)) && obj) + if (unlikely(slab_want_init_on_free(s)) && obj && !freeptr_outside_object(s)) memset((void *)((char *)kasan_reset_tag(obj) + s->offset), 0, sizeof(void *)); } -- 2.44.0