From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35024C05027 for ; Tue, 14 Feb 2023 18:29:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE1236B007E; Tue, 14 Feb 2023 13:29:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C91866B0080; Tue, 14 Feb 2023 13:29:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B595F6B0081; Tue, 14 Feb 2023 13:29:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A6CBE6B007E for ; Tue, 14 Feb 2023 13:29:42 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6C5131408C4 for ; Tue, 14 Feb 2023 18:29:42 +0000 (UTC) X-FDA: 80466735804.10.CB05940 Received: from mail-yb1-f177.google.com (mail-yb1-f177.google.com [209.85.219.177]) by imf09.hostedemail.com (Postfix) with ESMTP id 68DD4140020 for ; Tue, 14 Feb 2023 18:29:40 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ByI7f+xm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of surenb@google.com designates 209.85.219.177 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676399380; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/SlM87rL56jccJmLjLD6pTt8gewwZHtk9Lfqq8eDJzo=; b=iVSw1+ZCqNOlOQ0M1DLVKqP90Le1ws2+JRYOA8CtyQZWXDFawQz2P0EYVTC+1BEGPfuoY6 3MYWqNok6n1XoqBmwmQos6EtEQX4DS6XdVPabIQDUx1WWisLNwiLfx7/USdR1RbM4UYw+r njffcQh6A8GESQ4G7yExVnIEpXEpfQA= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ByI7f+xm; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of surenb@google.com designates 209.85.219.177 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676399380; a=rsa-sha256; cv=none; b=SacFKWW9bq3h9aILs8fw/MngQ2R5tnJltobxx5Rhzfx2D6pdoWZXnBBsVy6NjKNZap0+bU u2ioag3xkLkxVyEd8LRbMV45xzPZbwrJ9zYvyGqkhe4BZ3Gq+dN5kykMAXQileEpBPqk/c EHqnxrK9iIKASyh4pjfpCDc8fD0tmQI= Received: by mail-yb1-f177.google.com with SMTP id b132so18097807ybc.12 for ; Tue, 14 Feb 2023 10:29:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=/SlM87rL56jccJmLjLD6pTt8gewwZHtk9Lfqq8eDJzo=; b=ByI7f+xmR9BKi76Wa/CKqqPsRIOf+GohZ7Iw+mMNInS9m2L51FLkqlKXF6QPwu4Erm OwGHtnbcHAaOpmdEpqdC7wBiX09LgERIvazIDD5poNKNskyiK/M0v1e7BfWX4eMYMf7U I2kkH1iNg4dxsMkJE9k7OHHlNW51rjVuP0eUT/+qrB89Y/s2lj1VHhobvkbO67cz9OSR 3IRfKLKW/NsnZPjIMiF7FcLWZrphA6iEymXHsyu+ZoDFH3aVkrKe22WTJS5NGeNnFnCR E8ffWxqSxZmhu/MQ6F0PZIrrF/Jm8AcOTLyW7fuXr7DB8+uB7BMp4vRUhGO1A7JgTBzG 4vpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/SlM87rL56jccJmLjLD6pTt8gewwZHtk9Lfqq8eDJzo=; b=yxQLXlMjH3C+VLxi+LfJzgXLfhYKnBeHTbyHs9O+WL8M7F8pyHqKgry6QS59ZyPt7b HZmw4YR2SQmLaWs/p36cYUcXmKclSGTtxU/asy8RlUvhPhkd4iDU7w6naJBHSajYVjSN uKXTkyNVn7Q5Jg04HejtcIIn1RWNJ5uJMCPnGzTdGJUvQJpjdKbRM5BXQGqZ/EfAqSr7 hs9QLEcJgmMvC7Hho70DKtrxFiovS2FHpAgNcuunj8HOgp8bmUYyqHW8+DxUu9KM1cdN YaRK25PAGy/fCv4tt3RgcBaqfU5wo05neCiqKJc+x2+vqa1YDHXiRzz+7Wim4We701tp bppw== X-Gm-Message-State: AO0yUKUQcfz/hZ+SXXBUf8jllVj6jFCiRFQfu5pAi6y6mhgGWyuMP6Fu 0wMqwlpAFvczt2iIEAJz7E3tE4Uw8fH0b/YAd+NjFw== X-Google-Smtp-Source: AK7set/RQifId3+U+Ef+RNudtbMOC6rMvxl7BSMB5lhMb7vx0iw6fZCsAUo6JgnOSCLyXjhfZMxKnfrP5WeO/seS6s0= X-Received: by 2002:a25:9709:0:b0:921:6df4:c136 with SMTP id d9-20020a259709000000b009216df4c136mr357279ybo.201.1676399379141; Tue, 14 Feb 2023 10:29:39 -0800 (PST) MIME-Version: 1.0 References: <20230214181335.3946674-1-kamatam@amazon.com> In-Reply-To: From: Suren Baghdasaryan Date: Tue, 14 Feb 2023 10:29:27 -0800 Message-ID: Subject: Re: [PATCH v3] sched/psi: fix use-after-free in ep_remove_wait_queue() To: Munehisa Kamata , Peter Zijlstra Cc: ebiggers@kernel.org, hannes@cmpxchg.org, hdanton@sina.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mengcc@amazon.com, stable@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 68DD4140020 X-Stat-Signature: ki9hnmnt3esszfx6a99c35mufzsy9cmh X-HE-Tag: 1676399380-208108 X-HE-Meta: U2FsdGVkX1/w7j0g5Pm9JtwPGlo+i8APn60RJJDJKhhNcFoCP2nIJERFO4PxQYaPNg4Xb26nfAHHvb0GG6eIivKtwO15VAzkbiIjH6JZHMda0U1QbLw6P4WCoEEQbVBfheLuILF4a/sISBJ+gJjuUMj5CPaeVnE1tfYd+tCZBRtCYhy0rr+S5MFl0IEXg6jUXwJRLkxjor1ZP/ijnlI/bV6MecrsPJIk2n3DDfENh/Gr0FcaBqowwVqQyWyin+0Os/ygATP7Q5xJA9CczG8FIVn744Ia0CgzUH5BtHnXKqLNLT4Mqr5Fn71O/W1x6mrpTfJGY6fIEkxFBL5IgUHixpw/PbX3zTLxU6r3ezZgyo40mSy/gK1Rarq9j7b4TucyFd6EF61Lb6u8MVfNv4NfepN/c1Jnqr2kSyOEBx34RtcsVwCM6HQnhuZH6E+7DZlZMfvALKcoJWa4dLfd91QihVANOHV7eaHTWDotWZPNjB170vQ0tkYLcJyTU7Ba+p/PZE/NtHcN57SvEeGHjjDs6ZeYar2ITPn9BxgU6IVVk/C5yhrzVdYs7tkoU5jDnwkAvpZUEQq/ccurWVfR+WqGMtbyotRogH+67okqVnmQrU+YgrzK4yKFnUPKidhqTqZjpeblVnqKcmnEAiK3IAmxoG4LKGf2Fdcxn3BJLAfDW84zJZUyob8DNCrfAStlhgUoahUPK8QLkD3yhEuRtb1nzI05PWAe2n9JaglTSdbIySUZqcCE8+bEAGfO2ofRtDpw0K+LGaSXUGxFlMGgUDyyEUbuUU7HaWO52vSmaSK7XcKM9Dtz/QhNmOmMEIbKOmrb3/kvjjAc6hZuH1azKHc6DDb8Bes6E1aqH4eqYq0w9N8JYfSSxeqvtUZRF5br+4gqZtBye2PtzpaZBbcK6VNZM3gG60bXKKeW4RyVmQNs1gMInPunJq40yWD5SlnfzUTg1mHAMDynTjbfwrhtYZK eLRkJ4IY hyswW73SvEii08VhcwfdoIiZx2sWG8830f9zBVPzktyU5HW/N5tMDG7ggty52psYg94hw331W5MqDSMVcvXvqMaFcfrwUQmsXvxyloPFUIXAUCyTsDbOEYvfuxAzhiCetUY12EjEu8AXEV7W8OT1uvy6NY7FrQHdGnQjRsxpPlB+VsN/Tt605ZHrxdyF9jWAT/nlQapmtO5DHSk7+yyqF+PtXk0vRp3gr1M+wfj5pRMrJlnuAfz3uwPxyPjkrZKM4IxIhc5Av6ZwvdRP/Gp7xq6r6ldZNu2lS/eOz09creoZKvweYarDNRQpb8ri535ICP3qVB7AyUrx4LIYVikbSBsU1P0kQyGO/WLdzqi8Is4tkZjynWtScCz2hwjcLJbRD+BoVF7FVdLwOjeM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Feb 14, 2023 at 10:28 AM Suren Baghdasaryan wrote: > > On Tue, Feb 14, 2023 at 10:13 AM Munehisa Kamata wrote: > > > > If a non-root cgroup gets removed when there is a thread that registered > > trigger and is polling on a pressure file within the cgroup, the polling > > waitqueue gets freed in the following path. > > > > do_rmdir > > cgroup_rmdir > > kernfs_drain_open_files > > cgroup_file_release > > cgroup_pressure_release > > psi_trigger_destroy > > > > However, the polling thread still has a reference to the pressure file and > > will access the freed waitqueue when the file is closed or upon exit. > > > > fput > > ep_eventpoll_release > > ep_free > > ep_remove_wait_queue > > remove_wait_queue > > > > This results in use-after-free as pasted below. > > > > The fundamental problem here is that cgroup_file_release() (and > > consequently waitqueue's lifetime) is not tied to the file's real lifetime. > > Using wake_up_pollfree() here might be less than ideal, but it also is not > > fully contradicting the comment at commit 42288cb44c4b ("wait: add > > wake_up_pollfree()") since the waitqueue's lifetime is not tied to file's > > one and can be considered as another special case. While this would be > > fixable by somehow making cgroup_file_release() be tied to the fput(), it > > would require sizable refactoring at cgroups or higher layer which might be > > more justifiable if we identify more cases like this. > > > > BUG: KASAN: use-after-free in _raw_spin_lock_irqsave+0x60/0xc0 > > Write of size 4 at addr ffff88810e625328 by task a.out/4404 > > > > CPU: 19 PID: 4404 Comm: a.out Not tainted 6.2.0-rc6 #38 > > Hardware name: Amazon EC2 c5a.8xlarge/, BIOS 1.0 10/16/2017 > > Call Trace: > > > > dump_stack_lvl+0x73/0xa0 > > print_report+0x16c/0x4e0 > > ? _printk+0x59/0x80 > > ? __virt_addr_valid+0xb8/0x130 > > ? _raw_spin_lock_irqsave+0x60/0xc0 > > kasan_report+0xc3/0xf0 > > ? _raw_spin_lock_irqsave+0x60/0xc0 > > kasan_check_range+0x2d2/0x310 > > _raw_spin_lock_irqsave+0x60/0xc0 > > remove_wait_queue+0x1a/0xa0 > > ep_free+0x12c/0x170 > > ep_eventpoll_release+0x26/0x30 > > __fput+0x202/0x400 > > task_work_run+0x11d/0x170 > > do_exit+0x495/0x1130 > > ? update_cfs_rq_load_avg+0x2c2/0x2e0 > > do_group_exit+0x100/0x100 > > get_signal+0xd67/0xde0 > > ? finish_task_switch+0x15f/0x3a0 > > arch_do_signal_or_restart+0x2a/0x2b0 > > exit_to_user_mode_prepare+0x94/0x100 > > syscall_exit_to_user_mode+0x20/0x40 > > do_syscall_64+0x52/0x90 > > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > RIP: 0033:0x7f8e392bfb91 > > Code: Unable to access opcode bytes at 0x7f8e392bfb67. > > RSP: 002b:00007fff261e08d8 EFLAGS: 00000246 ORIG_RAX: 0000000000000022 > > RAX: fffffffffffffdfe RBX: 0000000000000000 RCX: 00007f8e392bfb91 > > RDX: 0000000000000001 RSI: 00007fff261e08e8 RDI: 0000000000000004 > > RBP: 00007fff261e0920 R08: 0000000000400780 R09: 00007f8e3960f240 > > R10: 00000000000003df R11: 0000000000000246 R12: 00000000004005a0 > > R13: 00007fff261e0a00 R14: 0000000000000000 R15: 0000000000000000 > > > > > > Allocated by task 4404: > > kasan_set_track+0x3d/0x60 > > __kasan_kmalloc+0x85/0x90 > > psi_trigger_create+0x113/0x3e0 > > pressure_write+0x146/0x2e0 > > cgroup_file_write+0x11c/0x250 > > kernfs_fop_write_iter+0x186/0x220 > > vfs_write+0x3d8/0x5c0 > > ksys_write+0x90/0x110 > > do_syscall_64+0x43/0x90 > > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > > > Freed by task 4407: > > kasan_set_track+0x3d/0x60 > > kasan_save_free_info+0x27/0x40 > > ____kasan_slab_free+0x11d/0x170 > > slab_free_freelist_hook+0x87/0x150 > > __kmem_cache_free+0xcb/0x180 > > psi_trigger_destroy+0x2e8/0x310 > > cgroup_file_release+0x4f/0xb0 > > kernfs_drain_open_files+0x165/0x1f0 > > kernfs_drain+0x162/0x1a0 > > __kernfs_remove+0x1fb/0x310 > > kernfs_remove_by_name_ns+0x95/0xe0 > > cgroup_addrm_files+0x67f/0x700 > > cgroup_destroy_locked+0x283/0x3c0 > > cgroup_rmdir+0x29/0x100 > > kernfs_iop_rmdir+0xd1/0x140 > > vfs_rmdir+0xfe/0x240 > > do_rmdir+0x13d/0x280 > > __x64_sys_rmdir+0x2c/0x30 > > do_syscall_64+0x43/0x90 > > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > > > v3: updated commit message and the comment in the code > > v2: updated commit message > > > > Link: https://lore.kernel.org/lkml/20230106224859.4123476-1-kamatam@amazon.com/ > > Fixes: 0e94682b73bf ("psi: introduce psi monitor") > > Cc: stable@vger.kernel.org > > Signed-off-by: Munehisa Kamata > > Signed-off-by: Mengchi Cheng > Acked-by: Suren Baghdasaryan CC'ing PeterZ directly for inclusion into his tree. > > > --- > > kernel/sched/psi.c | 7 ++++--- > > 1 file changed, 4 insertions(+), 3 deletions(-) > > > > diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c > > index 8ac8b81bfee6..02e011cabe91 100644 > > --- a/kernel/sched/psi.c > > +++ b/kernel/sched/psi.c > > @@ -1343,10 +1343,11 @@ void psi_trigger_destroy(struct psi_trigger *t) > > > > group = t->group; > > /* > > - * Wakeup waiters to stop polling. Can happen if cgroup is deleted > > - * from under a polling process. > > + * Wakeup waiters to stop polling and clear the queue to prevent it from > > + * being accessed later. Can happen if cgroup is deleted from under a > > + * polling process. > > */ > > - wake_up_interruptible(&t->event_wait); > > + wake_up_pollfree(&t->event_wait); > > > > mutex_lock(&group->trigger_lock); > > > > -- > > 2.38.1 > >