From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAACCC433E6 for ; Tue, 16 Feb 2021 11:05:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A9C8464E00 for ; Tue, 16 Feb 2021 11:05:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230313AbhBPLFJ (ORCPT ); Tue, 16 Feb 2021 06:05:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:30463 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230231AbhBPLDH (ORCPT ); Tue, 16 Feb 2021 06:03:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613473282; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UjPevUc/f1h27YC/iWCBse0mv72F3ucIKOdIwEX5LG8=; b=AvkgOPp+DBPc0FrKWJwDubOtuTW7N7QZR8zo7kgmXnXHIMctpATd5nCqJxHvmK79+qP+s2 P2dvCOwOuspzHa8D9x+3Z2BShEWiivzSmv4nj2i+hOXHyXL4DuFXTaDgfZ1qTrAhE669IH yYZhIxt6TEZ6qc+Gby+0O1WX0NUh1rg= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-501-Qsndgl3FOU2F2Ys04VJ13Q-1; Tue, 16 Feb 2021 06:01:21 -0500 X-MC-Unique: Qsndgl3FOU2F2Ys04VJ13Q-1 Received: by mail-qt1-f198.google.com with SMTP id n4so7420206qte.11 for ; Tue, 16 Feb 2021 03:01:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=UjPevUc/f1h27YC/iWCBse0mv72F3ucIKOdIwEX5LG8=; b=GPK7SbZPfBherENvvc36NYD05C1wyVRbG7FX9MlXZEDaDqssXH2f2DcJ11m2hYxNNU 156f5a7zSKvlaB6sM/T842CtIHXGgfe7XDgfVlAxynTo6hPpJOW4VY1l3+ek6y0pTetY rHvCHYCNQsqZw9P4QTWBkAmztgP992AzcYkp9skV4qM9ey5GL2NQpHskmsCJT3tlWxHa QRgZgKNtixecVJDRONrHwNTZmzPwBcbXAR3+Ysl0KpE0NbHksqFJvj3NCr7Ujv2dGmRL /39vr1S/yFZS7b0w/18GXxo+BrHw42yTwXjWWD6ZGhyD2uFSbLY6V44qco1XpY6l0GED wdIg== X-Gm-Message-State: AOAM533aKSk/4ndRXwdo0vm4KrkRpHQ9n3Lsiy2Rmc1YeZmNxkdg2m6O 0UAkiJ/gdTcU+j0GjaV0+gPKMid2V5FdxSiKST2zEgWHZKtf01eEvNaBpxL/+mo7sho3M90oeDZ deidDSDHiGSJphRi2MSCC X-Received: by 2002:a05:6214:292:: with SMTP id l18mr19206671qvv.5.1613473277869; Tue, 16 Feb 2021 03:01:17 -0800 (PST) X-Google-Smtp-Source: ABdhPJwfxkMdLP8GICFrYv5qcHZzcwXchEHaGAgSDG1h4XsMOzzlGSXOw3kqQRk/cyROnYM/B9O9Vw== X-Received: by 2002:a05:6214:292:: with SMTP id l18mr19206653qvv.5.1613473277670; Tue, 16 Feb 2021 03:01:17 -0800 (PST) Received: from tleilax.poochiereds.net (68-20-15-154.lightspeed.rlghnc.sbcglobal.net. [68.20.15.154]) by smtp.gmail.com with ESMTPSA id n5sm13100571qtd.5.2021.02.16.03.01.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Feb 2021 03:01:17 -0800 (PST) Message-ID: <752a8c91b7a418fa52cb8a8f28cb30155a574904.camel@redhat.com> Subject: Re: [PATCH 00/33] Network fs helper library & fscache kiocb API [ver #3] From: Jeff Layton To: Steve French Cc: David Howells , Trond Myklebust , Anna Schumaker , Steve French , Dominique Martinet , CIFS , ceph-devel@vger.kernel.org, Matthew Wilcox , linux-cachefs@redhat.com, Alexander Viro , linux-mm , linux-afs@lists.infradead.org, v9fs-developer@lists.sourceforge.net, Christoph Hellwig , linux-fsdevel , linux-nfs , Linus Torvalds , David Wysochanski , LKML Date: Tue, 16 Feb 2021 06:01:16 -0500 In-Reply-To: References: <161340385320.1303470.2392622971006879777.stgit@warthog.procyon.org.uk> <9e49f96cd80eaf9c8ed267a7fbbcb4c6467ee790.camel@redhat.com> Content-Type: text/plain; charset="ISO-8859-15" User-Agent: Evolution 3.38.4 (3.38.4-1.fc33) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org On Mon, 2021-02-15 at 18:40 -0600, Steve French wrote: > Jeff, > What are the performance differences you are seeing (positive or > negative) with ceph and netfs, especially with simple examples like > file copy or grep of large files? > > It could be good if netfs simplifies the problem experienced by > network filesystems on Linux with readahead on large sequential reads > - where we don't get as much parallelism due to only having one > readahead request at a time (thus in many cases there is 'dead time' > on either the network or the file server while waiting for the next > readpages request to be issued). This can be a significant > performance problem for current readpages when network latency is long > (or e.g. in cases when network encryption is enabled, and hardware > offload not available so time consuming on the server or client to > encrypt the packet). > > Do you see netfs much faster than currentreadpages for ceph? > > Have you been able to get much benefit from throttling readahead with > ceph from the current netfs approach for clamping i/o? > I haven't seen big performance differences at all with this set. It's pretty much a wash, and it doesn't seem to change how the I/Os are ultimately driven on the wire. For instance, the clamp_length op basically just mirrors what ceph does today -- it ensures that the length of the I/O can't go past the end of the current object. The main benefits are that we get a large swath of readpage, readpages amd write_begin code out of ceph altogether. All of the netfs's need to gather and vet pages for I/O, etc. Most of that doesn't have anything to do with the filesystem itself. By offloading that into the netfs lib, most of that is taken care of for us and we don't need to bother with doing that ourselves. -- Jeff Layton