From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 805E6C433FE for ; Tue, 10 May 2022 00:33:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233756AbiEJAht (ORCPT ); Mon, 9 May 2022 20:37:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229696AbiEJAhs (ORCPT ); Mon, 9 May 2022 20:37:48 -0400 Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 8FF892AEDB8 for ; Mon, 9 May 2022 17:33:51 -0700 (PDT) Received: from unknown (HELO lgemrelse6q.lge.com) (156.147.1.121) by 156.147.23.51 with ESMTP; 10 May 2022 09:33:49 +0900 X-Original-SENDERIP: 156.147.1.121 X-Original-MAILFROM: byungchul.park@lge.com Received: from unknown (HELO X58A-UD3R) (10.177.244.38) by 156.147.1.121 with ESMTP; 10 May 2022 09:33:49 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: byungchul.park@lge.com Date: Tue, 10 May 2022 09:32:13 +0900 From: Byungchul Park To: Theodore Ts'o Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, linux-kernel@vger.kernel.org, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, chris@chris-wilson.co.uk, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, bfields@fieldses.org, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jack@suse.com, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, airlied@linux.ie, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com Subject: Re: [PATCH RFC v6 00/21] DEPT(Dependency Tracker) Message-ID: <20220510003213.GD6047@X58A-UD3R> References: <1651652269-15342-1-git-send-email-byungchul.park@lge.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Mon, May 09, 2022 at 06:28:17PM -0400, Theodore Ts'o wrote: > Oh, one other problem with DEPT --- it's SLOW --- the overhead is > enormous. Using kvm-xfstests[1] running "kvm-xfstests smoke", here > are some sample times: Yes, right. DEPT has never been optimized. It rather turns on CONFIG_LOCKDEP and even CONFIG_PROVE_LOCKING when CONFIG_DEPT gets on because of porting issue. I have no choice but to rely on those to develop DEPT out of tree. Of course, that's what I don't like. Plus, for now, I'm focusing on removing false positives. Once it's considered settled down, I will work on performance optimizaition. But it should still keep relying on Lockdep CONFIGs and adding additional overhead on it until DEPT can be developed in the tree. > LOCKDEP DEPT > Time to first test 49 seconds 602 seconds > ext4/001 2 s 22 s > ext4/003 2 s 8 s > ext4/005 0 s 7 s > ext4/020 1 s 8 s > ext4/021 11 s 17 s > ext4/023 0 s 83 s > generic/001 4 s 76 s > generic/002 0 s 11 s > generic/003 10 s 19 s > > There are some large variations; in some cases, some xfstests take 10x > as much time or more to run. In fact, when I first started the > kvm-xfstests run with DEPT, I thought something had hung and that > tests would never start. (In fact, with gce-xfstests the default > watchdog "something has gone terribly wrong with the kexec" had fired, > and I didn't get any test results using gce-xfstests at all. If DEPT > goes in without any optimizations, I'm going to have to adjust the > watchdogs timers for gce-xfstests.) Thank you for informing it. I will go for the optimization as well. > The bottom line is that at the moment, between the false positives, > and the significant overhead imposed by DEPT, I would suggest that if > DEPT ever does go in, that it should be possible to disable DEPT and > only use the existing CONFIG_PROVE_LOCKING version of LOCKDEP, just > because DEPT is S - L - O - W. > > [1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md > > - Ted > > P.S. Darrick and I both have disabled using LOCKDEP by default > because it slows down ext4 -g auto testing by a factor 2, and xfs -g > auto testing by a factor of 3. So the fact that DEPT is a factor of > 2x to 10x or more slower than LOCKDEP when running various xfstests > tests should be a real concern. DEPT is tracking way more objects than Lockdep so it's inevitable to be slower, but let me try to make it have the similar performance to Lockdep. Byungchul From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8318C433EF for ; Tue, 10 May 2022 00:33:54 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EAAEA10F2AC; Tue, 10 May 2022 00:33:53 +0000 (UTC) Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by gabe.freedesktop.org (Postfix) with ESMTP id BE82E88BE3 for ; Tue, 10 May 2022 00:33:51 +0000 (UTC) Received: from unknown (HELO lgemrelse6q.lge.com) (156.147.1.121) by 156.147.23.51 with ESMTP; 10 May 2022 09:33:49 +0900 X-Original-SENDERIP: 156.147.1.121 X-Original-MAILFROM: byungchul.park@lge.com Received: from unknown (HELO X58A-UD3R) (10.177.244.38) by 156.147.1.121 with ESMTP; 10 May 2022 09:33:49 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: byungchul.park@lge.com Date: Tue, 10 May 2022 09:32:13 +0900 From: Byungchul Park To: Theodore Ts'o Subject: Re: [PATCH RFC v6 00/21] DEPT(Dependency Tracker) Message-ID: <20220510003213.GD6047@X58A-UD3R> References: <1651652269-15342-1-git-send-email-byungchul.park@lge.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: hamohammed.sa@gmail.com, jack@suse.cz, peterz@infradead.org, daniel.vetter@ffwll.ch, amir73il@gmail.com, david@fromorbit.com, dri-devel@lists.freedesktop.org, chris@chris-wilson.co.uk, bfields@fieldses.org, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, joel@joelfernandes.org, 42.hyeyoo@gmail.com, cl@linux.com, will@kernel.org, duyuyang@gmail.com, sashal@kernel.org, paolo.valente@linaro.org, damien.lemoal@opensource.wdc.com, willy@infradead.org, hch@infradead.org, airlied@linux.ie, mingo@redhat.com, djwong@kernel.org, vdavydov.dev@gmail.com, rientjes@google.com, dennis@kernel.org, linux-ext4@vger.kernel.org, linux-mm@kvack.org, ngupta@vflare.org, johannes.berg@intel.com, jack@suse.com, dan.j.williams@intel.com, josef@toxicpanda.com, rostedt@goodmis.org, linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, jglisse@redhat.com, viro@zeniv.linux.org.uk, tglx@linutronix.de, mhocko@kernel.org, vbabka@suse.cz, melissa.srw@gmail.com, sj@kernel.org, rodrigosiqueiramelo@gmail.com, kernel-team@lge.com, gregkh@linuxfoundation.org, jlayton@kernel.org, linux-kernel@vger.kernel.org, penberg@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, tj@kernel.org, akpm@linux-foundation.org, torvalds@linux-foundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Mon, May 09, 2022 at 06:28:17PM -0400, Theodore Ts'o wrote: > Oh, one other problem with DEPT --- it's SLOW --- the overhead is > enormous. Using kvm-xfstests[1] running "kvm-xfstests smoke", here > are some sample times: Yes, right. DEPT has never been optimized. It rather turns on CONFIG_LOCKDEP and even CONFIG_PROVE_LOCKING when CONFIG_DEPT gets on because of porting issue. I have no choice but to rely on those to develop DEPT out of tree. Of course, that's what I don't like. Plus, for now, I'm focusing on removing false positives. Once it's considered settled down, I will work on performance optimizaition. But it should still keep relying on Lockdep CONFIGs and adding additional overhead on it until DEPT can be developed in the tree. > LOCKDEP DEPT > Time to first test 49 seconds 602 seconds > ext4/001 2 s 22 s > ext4/003 2 s 8 s > ext4/005 0 s 7 s > ext4/020 1 s 8 s > ext4/021 11 s 17 s > ext4/023 0 s 83 s > generic/001 4 s 76 s > generic/002 0 s 11 s > generic/003 10 s 19 s > > There are some large variations; in some cases, some xfstests take 10x > as much time or more to run. In fact, when I first started the > kvm-xfstests run with DEPT, I thought something had hung and that > tests would never start. (In fact, with gce-xfstests the default > watchdog "something has gone terribly wrong with the kexec" had fired, > and I didn't get any test results using gce-xfstests at all. If DEPT > goes in without any optimizations, I'm going to have to adjust the > watchdogs timers for gce-xfstests.) Thank you for informing it. I will go for the optimization as well. > The bottom line is that at the moment, between the false positives, > and the significant overhead imposed by DEPT, I would suggest that if > DEPT ever does go in, that it should be possible to disable DEPT and > only use the existing CONFIG_PROVE_LOCKING version of LOCKDEP, just > because DEPT is S - L - O - W. > > [1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md > > - Ted > > P.S. Darrick and I both have disabled using LOCKDEP by default > because it slows down ext4 -g auto testing by a factor 2, and xfs -g > auto testing by a factor of 3. So the fact that DEPT is a factor of > 2x to 10x or more slower than LOCKDEP when running various xfstests > tests should be a real concern. DEPT is tracking way more objects than Lockdep so it's inevitable to be slower, but let me try to make it have the similar performance to Lockdep. Byungchul