From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37BB8C07545 for ; Tue, 24 Oct 2023 09:32:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 999A26B01F7; Tue, 24 Oct 2023 05:32:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 949BD6B01F8; Tue, 24 Oct 2023 05:32:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 811AA6B01F9; Tue, 24 Oct 2023 05:32:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 723356B01F7 for ; Tue, 24 Oct 2023 05:32:49 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3F62A140EC6 for ; Tue, 24 Oct 2023 09:32:49 +0000 (UTC) X-FDA: 81379840458.25.97C5316 Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by imf06.hostedemail.com (Postfix) with ESMTP id 40638180021 for ; Tue, 24 Oct 2023 09:32:46 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=O0xsY9qN; spf=pass (imf06.hostedemail.com: domain of anders.roxell@linaro.org designates 209.85.167.49 as permitted sender) smtp.mailfrom=anders.roxell@linaro.org; dmarc=pass (policy=none) header.from=linaro.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698139967; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1Rw/jYGs06Fd+viAjMbCyeWD4YePZEfjBTa+fHZnvXM=; b=Tupl+DEW83blLj6t/lCo+fe2U4Wwf4ljKxbKoBSAt1ReXsNOB+VBsZad5Yv6hvdo58dZlp z/HPzzGF9cHBiKR2limlKOgO6P2cY/5pXSLewiDeJhMUJBf6IAB/daTQU7pOvhThE30tTy 5rkW//fQwNvUFlKykv3ha0yrQ2cYH+c= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=O0xsY9qN; spf=pass (imf06.hostedemail.com: domain of anders.roxell@linaro.org designates 209.85.167.49 as permitted sender) smtp.mailfrom=anders.roxell@linaro.org; dmarc=pass (policy=none) header.from=linaro.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698139967; a=rsa-sha256; cv=none; b=zpNm82UQqXgEpSHK/jEmdPjw846fWHkPv0w52/caxsDxiJplazoRMhicjCg1HoYWFVKNmh IJKN5hKej9tnH4TnteL09FsLtb+RuiQhiRSyIday3MbLG5wA+3PVkmycCfK2W4t/KdE1Ae veItagnLurbKtd7R26mHi+qHjA92ngI= Received: by mail-lf1-f49.google.com with SMTP id 2adb3069b0e04-507d1cc0538so6182378e87.2 for ; Tue, 24 Oct 2023 02:32:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1698139965; x=1698744765; darn=kvack.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=1Rw/jYGs06Fd+viAjMbCyeWD4YePZEfjBTa+fHZnvXM=; b=O0xsY9qN4UVNEn/ibQXn/0HR2+J91zds6Ja+DroxzGne1M4V0WhoO+lX57H4j2vvK9 WLbNqCctyrDElvdjN6g5w1DTtCEZ4W99cZ6oxlmyDxqzNa2MBpxR78504HcI9Qii2sTZ nrwd9f3ynhEZllRFpKtMQIfobWndu5mgJFRPXisanU+6jYw0iuCeyhpcHKCY+PRp1mIm o1YOy3PbgI7YMmvTVphiIl4ajAsR2W9R95ntDalEG2DXp9yfHGernphxDldcSkFGCVuu 1+rPL51Bx0i3LUIFfojt7X0z4SfJSH9LeyyiLEu4zwRYD9TUCO0CDovQH3/yyqTW7FZx BJCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698139965; x=1698744765; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1Rw/jYGs06Fd+viAjMbCyeWD4YePZEfjBTa+fHZnvXM=; b=a2H1FBmAaHIn9DIfpN5E7DNuYDZnEvTuvlPosrgt6MTL+LtBdoLBV4USGt8Aju0Nir PyM2es3kpomZwiaoKpxfpWOJFx30sscRqqurECpC0eEps70YtvhTgc6ZRUBVU6GRBuRT PzMc7E0UOi4ylEfHz7yy9HUgFHbCHC1tkWoaNr3tkzFSNYICGh23Nrj8fvOwi9nGCyiU kuUQz4ne1ep07KHzObEz2j9lJ05D++M2iti6PZioEG9AVYnEUeHPq9oB4euQwxeDlnlk 0EnRmrHCcDJyE1ZrgjUatOcjFGcXNIHRP1Cn4lnemBvTxDIQh1LL2DqsWU7uv/Sn1/JI 5ZwA== X-Gm-Message-State: AOJu0YxtWGJr6ihG8wmz4cgwLsi2ZX97zUGbY8qUDfxJiqwzAlhkKmgo Y7vrm9REUnZHV92StrXJyz7qFw== X-Google-Smtp-Source: AGHT+IFxlTzrN8/W7frKk9B975akFVD/HyCjPVnWB73dVpbGyDcneRj0f2SEoX2NIvY9qHyCsm3pgg== X-Received: by 2002:ac2:4202:0:b0:500:9214:b308 with SMTP id y2-20020ac24202000000b005009214b308mr8046002lfh.65.1698139965282; Tue, 24 Oct 2023 02:32:45 -0700 (PDT) Received: from mutt (c-9b0ee555.07-21-73746f28.bbcust.telenor.se. [85.229.14.155]) by smtp.gmail.com with ESMTPSA id c12-20020ac25f6c000000b004fbc82dd1a5sm2063075lfc.13.2023.10.24.02.32.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Oct 2023 02:32:44 -0700 (PDT) Date: Tue, 24 Oct 2023 11:32:43 +0200 From: Anders Roxell To: andrey.konovalov@linux.dev Cc: Marco Elver , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Oscar Salvador , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: Re: [PATCH v3 00/19] stackdepot: allow evicting stack traces Message-ID: <20231024093243.GA3298341@mutt> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 40638180021 X-Rspam-User: X-Stat-Signature: wjb5twsjo3qyhi45yookdkfs71jrjhyk X-Rspamd-Server: rspam01 X-HE-Tag: 1698139966-163334 X-HE-Meta: U2FsdGVkX19uJXTY5xGfEEtwrZs0o4OI19Ycx+G21Mp4ot5OHKd+ABt42/dbx4NvBt2TKRmad8Cwo1G8DvC4ZvJXxGNHmuX6ygX72bN8ojYoqo24LJUFwNPVPEwfyfveX6gSDmoiCEvz0SpiMtocwyvGHKh5dJUosowV7akI3/Rhge0RSZbsIK46x84hrWzLdQQVWkYoAd502zTJL4AcBzgry7Ld3SmdogzWwNoFvHPQluIKDjah9wXkZd+NbT9aTS26UDO0iqaiHXbSYnryxDSYClLThHcEfljtot34zbEmu2Wdz5BCrnhapIpBxywjRv2UJIaONcm1nzLKbeDB27oe2jGli47UOneiI4ITIZzakjxtygWfgQUvDLp7Nd0rBt6UE32p8VTxatoV1js7CSkdbGZ/keRNhZA/UsaKD1bq8WS1pZIpXyvv43GFUtcI3w42g7yPimDFX7RktXtaOZPKtnJDKoLo+JcX5v+ukimD1VqqTzuhA07AjzcSgib0+lYF4hNEa3TNTUXIQLj4PKraFAJLP3Qytd8t30vnQU2EgejFc1fael67+XzycPfcDAMU4wnvv+Ltunv56ZMbbXQ56jB4iGnP6c5JDmn3kUeRURZjz39HiTXmBj8etTmo88gK24RsnsTMEe0GJP49y4WTyXWkBEm54MG+tgQhb3JHHC3OvC7EyBj7mn+PWagB20U5K0ecQj+BvSDzRBQu2JFYRk96xU2rS/zYB8YtlXp7Q5RXOgkmm+z2ZuqTcKzFFtoRiw0xHgu15wlppwEDjSWx8bXlgQ0C0t413hnIAX8MlELM96GQUuU3IVvzYc8sl/4veqZtDTJXKdQgGkW7JnlJ/jmn0NWasvu4WaXBUMuvoz3pKW7hBp+GHTLVRYDBX6WOwYLDlFa9p/z6yn6dicL84Z0w3P2h9Tts0SKEtdsCrXJV0kmYbD5/X6jxRnl1T14Spd6F3KXfGMUlKpO 3MHtDmo0 DREpDiBlVq6AigPyeMO87ZJ3K+aN/Lqk4r/C6oNOQ9f/NR8abohMjJizb0G4cUgvqcT5f1NZuaok+2Yb7VPnbbxv1l6oCCVUn8pXM+9+yNv9u77F8UACptD3TYaF6nSw/q5RKt5FkeqpneQAND8uyG9zzyTw7kC7m5Mz42WV6pXryoEyZEbDIj6+OEmq8KAh+6e+BW39ypGsmsHKicNagfV5HjgrQTv/5E8wglAV3ArGTU5uAimSFzBy4QWEA+aKM5oBHuhywDlDq83MRG3cMTW7+wj2WNhPoMdB6Fwnk3bGwmwGuB6OQ/5BbvtrUpjhbHr1qtbWea17CB5kV7dvKCMXKW6eEFZ/wgSOqMWmVwyCCrOFhnOpx286AoSv9oXqV6tfV+/bRxVvNrZf1oQdhaad4O1oM5ZzxHwcjOCT+MYbfJWI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2023-10-23 18:22, andrey.konovalov@linux.dev wrote: > From: Andrey Konovalov > > Currently, the stack depot grows indefinitely until it reaches its > capacity. Once that happens, the stack depot stops saving new stack > traces. > > This creates a problem for using the stack depot for in-field testing > and in production. > > For such uses, an ideal stack trace storage should: > > 1. Allow saving fresh stack traces on systems with a large uptime while > limiting the amount of memory used to store the traces; > 2. Have a low performance impact. > > Implementing #1 in the stack depot is impossible with the current > keep-forever approach. This series targets to address that. Issue #2 is > left to be addressed in a future series. > > This series changes the stack depot implementation to allow evicting > unneeded stack traces from the stack depot. The users of the stack depot > can do that via new stack_depot_save_flags(STACK_DEPOT_FLAG_GET) and > stack_depot_put APIs. > > Internal changes to the stack depot code include: > > 1. Storing stack traces in fixed-frame-sized slots; the slot size is > controlled via CONFIG_STACKDEPOT_MAX_FRAMES (vs precisely-sized > slots in the current implementation); > 2. Keeping available slots in a freelist (vs keeping an offset to the next > free slot); > 3. Using a read/write lock for synchronization (vs a lock-free approach > combined with a spinlock). > > This series also integrates the eviction functionality in the tag-based > KASAN modes. > > Despite wasting some space on rounding up the size of each stack record, > with CONFIG_STACKDEPOT_MAX_FRAMES=32, the tag-based KASAN modes end up > consuming ~5% less memory in stack depot during boot (with the default > stack ring size of 32k entries). The reason for this is the eviction of > irrelevant stack traces from the stack depot, which frees up space for > other stack traces. > > For other tools that heavily rely on the stack depot, like Generic KASAN > and KMSAN, this change leads to the stack depot capacity being reached > sooner than before. However, as these tools are mainly used in fuzzing > scenarios where the kernel is frequently rebooted, this outcome should > be acceptable. > > There is no measurable boot time performance impact of these changes for > KASAN on x86-64. I haven't done any tests for arm64 modes (the stack > depot without performance optimizations is not suitable for intended use > of those anyway), but I expect a similar result. Obtaining and copying > stack trace frames when saving them into stack depot is what takes the > most time. > > This series does not yet provide a way to configure the maximum size of > the stack depot externally (e.g. via a command-line parameter). This will > be added in a separate series, possibly together with the performance > improvement changes. > > --- > > Changes v2->v3: > - Fix null-ptr-deref by using the proper number of entries for > initializing the stack table when alloc_large_system_hash() > auto-calculates the number (see patch #12). > - Keep STACKDEPOT/STACKDEPOT_ALWAYS_INIT Kconfig options not configurable > by users. > - Use lockdep_assert_held_read annotation in depot_fetch_stack. > - WARN_ON invalid flags in stack_depot_save_flags. > - Moved "../slab.h" include in mm/kasan/report_tags.c in the right patch. > - Various comment fixes. > > Changes v1->v2: > - Rework API to stack_depot_save_flags(STACK_DEPOT_FLAG_GET) + > stack_depot_put. > - Add CONFIG_STACKDEPOT_MAX_FRAMES Kconfig option. > - Switch stack depot to using list_head's. > - Assorted minor changes, see the commit message for each path. > > Andrey Konovalov (19): > lib/stackdepot: check disabled flag when fetching > lib/stackdepot: simplify __stack_depot_save > lib/stackdepot: drop valid bit from handles > lib/stackdepot: add depot_fetch_stack helper > lib/stackdepot: use fixed-sized slots for stack records > lib/stackdepot: fix and clean-up atomic annotations > lib/stackdepot: rework helpers for depot_alloc_stack > lib/stackdepot: rename next_pool_required to new_pool_required > lib/stackdepot: store next pool pointer in new_pool > lib/stackdepot: store free stack records in a freelist > lib/stackdepot: use read/write lock > lib/stackdepot: use list_head for stack record links > kmsan: use stack_depot_save instead of __stack_depot_save > lib/stackdepot, kasan: add flags to __stack_depot_save and rename > lib/stackdepot: add refcount for records > lib/stackdepot: allow users to evict stack traces > kasan: remove atomic accesses to stack ring entries > kasan: check object_size in kasan_complete_mode_report_info > kasan: use stack_depot_put for tag-based modes Tested-by: Anders Roxell Applied this patchset to linux-next tag next-20231023 and built an arm64 kernel and that booted fine in QEMU. Cheers, Anders