From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BE9CC10F0E for ; Thu, 18 Apr 2019 18:22:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 017902064A for ; Thu, 18 Apr 2019 18:22:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403913AbfDRSWD (ORCPT ); Thu, 18 Apr 2019 14:22:03 -0400 Received: from prvmx02.microfocus.com ([130.57.1.217]:12106 "EHLO prvmx02.microfocus.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390790AbfDRSWD (ORCPT ); Thu, 18 Apr 2019 14:22:03 -0400 Received: from prvxcaht01.microfocus.com (Not Verified[137.65.249.208]) by prvmx02.microfocus.com with Trustwave SEG (v7,5,8,10121) (using TLS: TLSv1.2, AES256-SHA256) id ; Thu, 18 Apr 2019 12:22:01 -0600 Received: from dbueso.openvpn2.suse.de (10.163.9.8) by mail.microfocus.com (137.65.249.208) with Microsoft SMTP Server (TLS) id 14.3.339.0; Thu, 18 Apr 2019 12:22:01 -0600 Message-ID: <1555611694.18313.12.camel@suse.com> Subject: Re: [POC][PATCH] xfs: reduce ilock contention on buffered randrw workload From: Davidlohr Bueso To: Dave Chinner CC: Jan Kara , Amir Goldstein , "Darrick J . Wong" , Christoph Hellwig , "Matthew Wilcox" , , Date: Thu, 18 Apr 2019 11:21:34 -0700 In-Reply-To: <20190418031013.GX29573@dread.disaster.area> References: <20190404165737.30889-1-amir73il@gmail.com> <20190404211730.GD26298@dastard> <20190408103303.GA18239@quack2.suse.cz> <1554741429.3326.43.camel@suse.com> <20190411011117.GC29573@dread.disaster.area> <20190416122240.GN29573@dread.disaster.area> <20190418031013.GX29573@dread.disaster.area> Organization: SUSE Labs Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.26.6 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.163.9.8] Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Thu, 2019-04-18 at 13:10 +1000, Dave Chinner wrote: > Now the stuff I've been working on has the same interface as > Davidlohr's patch, so I can swap and change them without thinking > about it. It's still completely unoptimised, but: > > IOPS read/write (direct IO) > processes rwsem DB rangelock XFS > rangelock > 1 78k / 78k 75k / 75k 72k / 72k > 2 131k / 131k 123k / 123k 133k / 133k > 4 267k / 267k 183k / 183k 237k / 237k > 8 372k / 372k 177k / 177k 265k / 265k > 16 315k / 315k 135k / 135k 228k / 228k > > It's still substantially faster than the interval tree code. In general another big difference between both rangelock vs rwsems (when comparing them with full ranges) is that the latter will do writer optimistic spinning, so saving a context switch under the right scenarios provides mayor wins for rwsems -- I'm not sure if this applies to your fio tests, though. And pretty soon readers will also do this, hence rwsem will become a try-hard-not-to-sleep lock. One of the reasons why I was hesitant with Btrees was the fact that insertion requires memory allocation, something I wanted to avoid... per your numbers, sacrificing tree depth was the wrong choice. Thanks for sharing these numbers. > > BTW, if I take away the rwsem serialisation altogether, this > test tops out at just under 500k/500k at 8 threads, and at 16 > threads has started dropping off (~440k/440k). So the rwsem is > a scalability limitation at just 8 threads.... > > /me goes off and thinks more about adding optimistic lock coupling > to the XFS iext btree to get rid of the need for tree-wide > locking altogether I was not aware of this code. Thanks, Davidlohr