From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9337CC433EF for ; Tue, 10 May 2022 01:32:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26D686B0072; Mon, 9 May 2022 21:32:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F55C6B0073; Mon, 9 May 2022 21:32:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06EF36B0074; Mon, 9 May 2022 21:32:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E93C26B0072 for ; Mon, 9 May 2022 21:32:50 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id B7CCB81342 for ; Tue, 10 May 2022 01:32:50 +0000 (UTC) X-FDA: 79448109300.12.AC61D98 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by imf20.hostedemail.com (Postfix) with ESMTP id 8D0A01C0070 for ; Tue, 10 May 2022 01:32:41 +0000 (UTC) Received: from cwcc.thunk.org (pool-108-7-220-252.bstnma.fios.verizon.net [108.7.220.252]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id 24A1WF09029249 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 9 May 2022 21:32:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mit.edu; s=outgoing; t=1652146345; bh=kNxC32quScbCpNyL/wTfsRZfYUcVDoGEmAnwF+CO/RI=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=km6W4YaByvgKGbZv4iigLhBLQKaouM7qOJbks5QBOy26HI3QALcwGS+dbcTKQS5sm Y/RfTPjh7Z/nGp7uw6lZldW4W7fDFvGwL50TgxrjuvPL3iRd3c4r5VsKVtA18CxFta K6h0nLQnpaLVIuO5cinKaeUKJnjTcwOHBqP8XAHNvf5t67+lRQdtNCcZd/4CjPjwJt TlD8gzwU7/lyuUjHaeuowq/VPSNP4EPKVvgcV9OldxEhxxvenlEiwduS3jOCyFWYYn fsw/0mi83B+RJaEAF7dXVO08S3PaX0rrxg/1+1D2s0+ctpfGHSZ240yR1Of3O+QVFm xIZ7hXIiuS2Nw== Received: by cwcc.thunk.org (Postfix, from userid 15806) id 5B4DC15C3F0A; Mon, 9 May 2022 21:32:15 -0400 (EDT) Date: Mon, 9 May 2022 21:32:15 -0400 From: "Theodore Ts'o" To: Byungchul Park Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, linux-kernel@vger.kernel.org, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, chris@chris-wilson.co.uk, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, bfields@fieldses.org, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jack@suse.com, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, airlied@linux.ie, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com Subject: Re: [PATCH RFC v6 00/21] DEPT(Dependency Tracker) Message-ID: References: <1651652269-15342-1-git-send-email-byungchul.park@lge.com> <20220510003213.GD6047@X58A-UD3R> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220510003213.GD6047@X58A-UD3R> X-Stat-Signature: hw176twwz1k3o8e6a7cdnasg8ycmyf9m X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 8D0A01C0070 Authentication-Results: imf20.hostedemail.com; dkim=fail ("headers rsa verify failed") header.d=mit.edu header.s=outgoing header.b=km6W4YaB; spf=none (imf20.hostedemail.com: domain of tytso@mit.edu has no SPF policy when checking 18.9.28.11) smtp.mailfrom=tytso@mit.edu; dmarc=fail reason="No valid SPF" header.from=mit.edu (policy=none) X-Rspam-User: X-HE-Tag: 1652146361-388716 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 10, 2022 at 09:32:13AM +0900, Byungchul Park wrote: > Yes, right. DEPT has never been optimized. It rather turns on > CONFIG_LOCKDEP and even CONFIG_PROVE_LOCKING when CONFIG_DEPT gets on > because of porting issue. I have no choice but to rely on those to > develop DEPT out of tree. Of course, that's what I don't like. Sure, but blaming the overhead on unnecessary CONFIG_PROVE_LOCKING overhead can explain only a tiny fraction of the slowdown. Consider: if time to first test (time to boot the kernel, setup the test environment, figure out which tests to run, etc.) is 12 seconds w/o LOCKDEP, 49 seconds with LOCKDEP/PROVE_LOCKING and 602 seconds with DEPT, you can really only blame 37 seconds out of the 602 seconds of DEPT on unnecessary PROVE_LOCKING overhead. So let's assume we can get rid of all of the PROVE_LOCKING overhead. We're still talking about 12 seconds for time-to-first test without any lock debugging, versus ** 565 ** seconds for time-to-first test with DEPT. That's a factor of 47x for DEPT sans LOCKDEP overhead, compared to a 4x overhead for PROVE_LOCKING. > Plus, for now, I'm focusing on removing false positives. Once it's > considered settled down, I will work on performance optimizaition. But > it should still keep relying on Lockdep CONFIGs and adding additional > overhead on it until DEPT can be developed in the tree. Well, please take a look at the false positive which I reported. I suspect that in order to fix that particular false positive, we'll either need to have a way to disable DEPT on waiting on all page/folio dirty bits, or it will need to treat pages from different inodes and/or address spaces as being entirely separate classes, instead of collapsing all inode dirty bits, and all of various inode's mutexes (such as ext4's i_data_sem) as being part of a single object class. > DEPT is tracking way more objects than Lockdep so it's inevitable to be > slower, but let me try to make it have the similar performance to > Lockdep. In order to eliminate some of these false positives, I suspect it's going to increase the number of object classes that DEPT will need to track even *more*. At which point, the cost/benefit of DEPT may get called into question, especially if all of the false positives can't be suppressed. - Ted