From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B526CECAAD3 for ; Thu, 15 Sep 2022 07:28:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EA2898D0001; Thu, 15 Sep 2022 03:28:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E51B66B0073; Thu, 15 Sep 2022 03:28:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D409A8D0001; Thu, 15 Sep 2022 03:28:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C39766B0071 for ; Thu, 15 Sep 2022 03:28:33 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 98E02AAC22 for ; Thu, 15 Sep 2022 07:28:33 +0000 (UTC) X-FDA: 79913492106.01.61920A8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id 69A4040097 for ; Thu, 15 Sep 2022 07:28:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=jZjXacIHtf71qVEc6F7nmis+mD+u/JEdyW+ir9p4QvI=; b=Tm1mpm+PSptPgt+Dy3cmUPgS4x i+lGxOmra5kHPfQdRFZIZqfmRGLc5RvWdhltMtOpgbU04M+XF2jiFII3ziIRrDPufwmFeIk9RGCyx V0WbPfCdylvwuQYj2TFMh87T329sKOh3U/sp2DsebspDEj3y89jjiCFBj7SyZMiuX4ntEE+cpp01h 4k9HSaYH3kjmFiUIQtE/1LcdyHOd98v97UB9shIhRXxZaP2U6YFUEwGGK+AJuR8Gdkc1X7L9r+UaE cViwxmw4IX9usMVcvkDLbNqK3HlVjhfAjxOJTLmH5CKaDOs2jLBQN0KRq1lyX/qRTG4KPyZ3nfxwE F4Fk60DA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1oYjIQ-000tE2-Ad; Thu, 15 Sep 2022 07:28:26 +0000 Date: Thu, 15 Sep 2022 08:28:26 +0100 From: Matthew Wilcox To: Hongchen Zhang Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm/vmscan: don't scan adjust too much if current is not kswapd Message-ID: References: <20220914023318.549118-1-zhanghongchen@loongson.cn> <20220914155142.bf388515a39fb45bae987231@linux-foundation.org> <6bcb4883-03d0-88eb-4c42-84fff0a9a141@loongson.cn> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6bcb4883-03d0-88eb-4c42-84fff0a9a141@loongson.cn> ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Tm1mpm+P; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663226913; a=rsa-sha256; cv=none; b=gY6EN1Q0cNbg3/8XQyCBd4WpkTs8Er3YWc6rXziyzkAfE4dappDwLRWF+OwUnNdZSGvNTW Da9bniWX+mlkJxgxTQvcB/BLNHpuGV1F6fvNdcdNBEOTOCRsDzlaReuChTAlNp3oTnxqO6 kNb6P8iIUm2rLpUTItbltVLJWrAj42I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663226913; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jZjXacIHtf71qVEc6F7nmis+mD+u/JEdyW+ir9p4QvI=; b=cq9zJNyOsbfekeUfksaHz+aP1Y8jEwqUQTAVO9VUyv5K/kAy1Yq/5yZNwax0jCzmB2l+l4 7N4r3djuFOfyeSLYpZIhAfzc/n73edQocJ5JBvl46etlYW55/0gxC+KpC+6EuUjYOq/Cx1 R8hB74zhBwJrWlJ5wbgVcWBithYW8l8= X-Stat-Signature: ijhbmrt4ka66f16q6mgsqe8jh8k6hate X-Rspamd-Queue-Id: 69A4040097 X-Rspam-User: Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Tm1mpm+P; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspamd-Server: rspam12 X-HE-Tag: 1663226912-542253 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 15, 2022 at 09:19:48AM +0800, Hongchen Zhang wrote: > [ 3748.453561] INFO: task float_bessel:77920 blocked for more than 120 > seconds. > [ 3748.460839] Not tainted 5.15.0-46-generic #49-Ubuntu > [ 3748.466490] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables > this message. > [ 3748.474618] task:float_bessel state:D stack: 0 pid:77920 ppid: > 77327 flags:0x00004002 > [ 3748.483358] Call Trace: > [ 3748.485964] > [ 3748.488150] __schedule+0x23d/0x590 > [ 3748.491804] schedule+0x4e/0xc0 > [ 3748.495038] rwsem_down_read_slowpath+0x336/0x390 > [ 3748.499886] ? copy_user_enhanced_fast_string+0xe/0x40 > [ 3748.505181] down_read+0x43/0xa0 > [ 3748.508518] do_user_addr_fault+0x41c/0x670 > [ 3748.512799] exc_page_fault+0x77/0x170 > [ 3748.516673] asm_exc_page_fault+0x26/0x30 > [ 3748.520824] RIP: 0010:copy_user_enhanced_fast_string+0xe/0x40 > [ 3748.526764] Code: 89 d1 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 31 c0 0f > 01 ca c3 cc cc cc cc 0f 1f 00 0f 01 cb 83 fa 40 0f 82 70 ff ff ff 89 d1 > a4 31 c0 0f 01 ca c3 cc cc cc cc 66 08 > [ 3748.546120] RSP: 0018:ffffaa9248fffb90 EFLAGS: 00050206 > [ 3748.551495] RAX: 00007f99faa1a010 RBX: ffffaa9248fffd88 RCX: > 0000000000000010 > [ 3748.558828] RDX: 0000000000001000 RSI: ffff9db397ab8ff0 RDI: > 00007f99faa1a000 > [ 3748.566160] RBP: ffffaa9248fffbf0 R08: ffffcc2fc2965d80 R09: > 0000000000000014 > [ 3748.573492] R10: 0000000000000000 R11: 0000000000000014 R12: > 0000000000001000 > [ 3748.580858] R13: 0000000000001000 R14: 0000000000000000 R15: > ffffaa9248fffd98 > [ 3748.588196] ? copy_page_to_iter+0x10e/0x400 > [ 3748.592614] filemap_read+0x174/0x3e0 Interesting; it wasn't the process itself which triggered the page fault; the process called read() and the kernel took the page fault to satisfy the read() call. > [ 3748.596354] ? ima_file_check+0x6a/0xa0 > [ 3748.600301] generic_file_read_iter+0xe5/0x150 > [ 3748.604884] ext4_file_read_iter+0x5b/0x190 > [ 3748.609164] ? aa_file_perm+0x102/0x250 > [ 3748.613125] new_sync_read+0x10d/0x1a0 > [ 3748.617009] vfs_read+0x103/0x1a0 > [ 3748.620423] ksys_read+0x67/0xf0 > [ 3748.623743] __x64_sys_read+0x19/0x20 > [ 3748.627511] do_syscall_64+0x59/0xc0 > [ 3748.631203] ? syscall_exit_to_user_mode+0x27/0x50 > [ 3748.636144] ? do_syscall_64+0x69/0xc0 > [ 3748.639992] ? exit_to_user_mode_prepare+0x96/0xb0 > [ 3748.644931] ? irqentry_exit_to_user_mode+0x9/0x20 > [ 3748.649872] ? irqentry_exit+0x1d/0x30 > [ 3748.653737] ? exc_page_fault+0x89/0x170 > [ 3748.657795] entry_SYSCALL_64_after_hwframe+0x61/0xcb > [ 3748.663030] RIP: 0033:0x7f9a852989cc > [ 3748.666713] RSP: 002b:00007f9a8497dc90 EFLAGS: 00000246 ORIG_RAX: > 0000000000000000 > [ 3748.674487] RAX: ffffffffffffffda RBX: 00007f9a8497f5c0 RCX: > 00007f9a852989cc > [ 3748.681817] RDX: 0000000000027100 RSI: 00007f99faa18010 RDI: > 0000000000000061 > [ 3748.689150] RBP: 00007f9a8497dd60 R08: 0000000000000000 R09: > 00007f99faa18010 > [ 3748.696493] R10: 0000000000000000 R11: 0000000000000246 R12: > 00007f99faa18010 > [ 3748.703841] R13: 00005605e11c406f R14: 0000000000000001 R15: > 0000000000027100 ORIG_RAX is 0, which matches sys_read. RDI is file descriptor 0x61 RSI is plausibly a userspace pointer, 0x7f99faa18010 RDX is the length, 0x27100 or 160kB. That all seems reasonable. What I really want to know is who is _holding_ the lock. We stash a pointer to the task_struct in 'owner', so we could clearly find this out in the 'blocked for too long' report, and print their stack trace. You must have done something like this already in order to deduce that it was the direct reclaim path that was the problem?